MIPS64® I6400 Multiprocessing System Programmer’s Guide

Revision 1.00 March 29, 2017 Public

Copyright © 2017 Imagination Technologies LTD. and/or its Affiliated Group Companies. All rights reserved. Public. This publication contains proprietary information which is subject to change without notice and is supplied ‘as is’, without any warranty of any kind.

Document Number: MD01196

MIPS64® I6400 Multiprocessing System Programmer’s Guide, Revision 1.00

Copyright © 2017 Imagination Technologies Ltd. and/or its Affiliated Group Companies. All rights reserved.

0 40 .. 19 ...... 23 ...... 40 ...... 9 ...... 12 ...... 17 ...... 25 ...... 24 ...... 12 ...... 21 ...... 26 ...... 23 ...... 14 ...... 16 ...... 24 ...... 16 ...... 28 ...... 12 ...... 17 ...... 33 ...... 22 ...... 12 ...... 34 ...... 33 ...... 13 ...... 35 ...... 14 ...... 34 ...... 20 ...... 15 ...... 16 ...... 23 ...... 19 ...... 29 ...... 15 ...... 16 ...... 34 ...... 16 ...... 38 ...... 10 ...... 31 ...... 39 ...... 12 ...... 40 ...... 19 gister Select 28, 0)...... 41 register 28, Select 1) ...... 41 1) ...... 4 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ...... egister 16, Selectegister 16, 0) ...... (CP0 register 27, Select 0)...... 41 s ...... T Support...... ation...... 3.4.1.2: Config1 Register (CP0 register 16, Select 3.4.1.3: CacheErr Register 3.4.1.4: L1 Instruction Cache TagLo Register (CP0 re 3.4.1.5: L1 Instruction Cache DataLo Register (CP0 3.4.1.1: Config Register (CP0 r 2.1.1: TLB Types...... 2.1.2: TLB Instructions ...... 2.1.3: Shared FTLB Translations ...... 2.1.4: Global TLB Invalidate...... 2.2.6: a Programming TLB Entry...... 2.2.7: Hardwiring VTLB Entries. 2.2.8: FTLB Hashing Scheme and the TLBWI Instruction ...... 29 2.2.1: Assembly Language Conventions...... 2.2.2: Determining the VTLB Size...... 2.2.3:Configuration...... FTLB Page Size 2.2.4: VTLB and FTLB Initialization...... 2.2.5: Indexing the VTLB and FTLB...... 1.2.2: SIMD MIPS® Architecture ...... 1.2.3: MIPS® Virtualization ...... 1.2.4: System-level Features ...... 1.2.5: Core-level Features...... 1.2.1: MIPS64® Release 6 Architecture ...... 3.1.1: L1 Instruction Cache ...... 3.1.2: L1 Data Cache ...... 3.1.3: L2 Cache...... 3.1.4: Cache Instructions ...... 1.5.1: MIPS ...... 1.5.2: MIPS Android...... 1.5.3: Codescape MIPS SDK...... 1.5.4: Codescape Debugger ...... 1.5.5: Compilers...... 1.5.6: Boot Loader...... 1.5.7:MIPS RTOS and Io 1.5.8: Developer Resources...... 3.4.1: L1 Instruction Cache Control Registers ...... Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 2.4: Additional Inform 2.3:TLB Exception Handler...... 2.2: MMU Programming ...... 3.1: Cache Subsystem Overview ...... 2.1: Overview...... 1.2: I6400 Features ...... 1.1: Product Overview ...... 1.3: I6400 Core Block Diagram ...... 1.4: CP0 Register to Assembler Mapping ...... 1.5: Imagination Software Tool 3.2: Cache Coherency...... 3.3: Self-modified Code ...... 3.4: Register Interface ...... MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 1 Chapter 3: Chapter Caches ...... Chapter 2: Memory Management Unit Memory Management 2: Chapter ...... Chapter 1: Architecture Overview...... 1: Architecture Chapter

2 2 42 .55 .. 44 ... 43 ...... 43 ...... 45 ...... 49 ...... 62 ...... 59 ...... 60 ...... 46 ...... 41 ...... 47 ...... 59 ...... 48 ...... 46 ...... 52 ...... 57 ...... 50 ...... 57 ...... 58 ...... 46 ...... 58 ...... 70 ...... 64 ...... 46 ...... 48 ...... 58 ...... 53 ...... 69 ...... 45 ...... 48 ...... em Programmer’s Guide, Revision 1.00 ...... 45 ess...... Address...... 44 )...... 44 register 29, Select 1)...... 41 er 29, Selecter 29, 3)...... 43 er 28, Select 2)...... er 28, Select 42 ster 28, Selectster 28, 3) ...... 42 ...... e GINVI Instruction...... 48 ytes of the Physical Address ...... 61 1) ...... 4 es of the PhysicalAddress ...... 61 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ...... Reset Exception Vectors...... Reset Cache Coherency Routine ...... egister 16, Select egister 16, 0) ...... ess per Exception Type...... ess per Exception 6 NF Register (Offset 0x0008)...... (CP0 register (CP0 register 27, Select 0)...... 42 ion Cache ...... the Lower 4 GBytes of the Physical Addr the Lower 512 MBytesof the Physical ines ...... rdware Cache Initialization . 3.4.3.3: L2_RAM_Config Register (Offset 0x0240) ...... 44 3.4.3.4: Register L2_PFT_Control (Offset 0x0300) ...... 44 3.4.3.5: L2_PFT_Control_B Register (Offset 0x0308 3.4.3.6: L2_TAG_ADDR Register (Offset 0x0600)...... 44 3.4.3.7: L2_TAG_STATE Register (Offset 0x0608) ...... 44 3.4.3.8: L2_DATA Register (Offset 0x0610)...... 3.4.3.9: L2_DATA_ECC Register (Offset0x0618) .... 3.4.3.10: L2SM_COP Register (Offset 0x0620) ...... 44 3.4.3.11: L2SM_TAG_ADDR_COPRegister (Offset 0x0628)...... 45 3.4.3.12: CPC_CL_STAT_CO 3.4.2.1: Config Register r (CP0 3.4.2.2: Config1 Register (CP0 register 16, Select 3.4.2.3: CacheErr Register 3.4.2.6: L1 Data Cache DataHi Register (CP0 regist 3.4.3.1: GCR_ERR_CONTROL (Offset 0x0038)...... 43 3.4.3.2: L2_Config Register (Offset 0x0130) ...... 3.7.1.1: L1 Instruction Cache Invalidation Using th 3.7.1.2: L1 Cache Initialization 3.4.1.6: L1 Instruction DataHi Register Cache (CP0 3.4.2.4: L1 Data Cache TagLo Register(CP0 regist 3.4.2.5: L1 Data Cache DataLo Register (CP0 regi 4.2.6: Exception Vector Base Addr 4.2.2: Mapping the BEV to 4.2.3: Mapping the Reset Vector to the Lower 512 MB 4.2.4: Mapping the Reset Vector to the Lower 4 GByt 4.2.5: Selecting Between the and BEV 4.2.1: Mapping the BEV to 3.5.2: Manual Hardware Cache Initialization...... 3.5.3: Software Cache Initialization...... 3.7.2: Initializing Datathe Cache ...... 3.7.3: Initializing Level 2 the Cache ...... 3.4.2:Data L1 Cache Control Registers ...... 3.4.3: L2 Cache CM GCR Control Registers ...... 3.6.1: L2 Cache Flush...... 3.6.2: L2 Cache Burst Operations...... 3.6.3: Abort Operations...... 4.1.1: Exception Types...... 4.1.2: Detecting an Exception ...... 4.1.3: Exception Conditions ...... 3.5.1: Automatic Ha 3.7.1: Initializing Instructthe Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 4.4: Hypervisor Exception Priorities...... 4.3: Core-Level Exception Priorities ...... 4.5:General Exception Processing ...... 3.6: L2 Cache Flush, Burst, and Abort ...... 4.2: Defining the Exception Vector Locations... 4.1: Overview of Exception Processing ...... 3.5: L2 Cache Initialization Options ...... 3.8: Flushing the L1 Data Cache ...... 3.7: Cache Initialization Rout 3.9: Setting the KSEG0 Memory Space 2 MIPS64® I6400 Multiprocessing Syst Chapter 4: Chapter Exceptions......

79 .92 105 .. 91 .... 84 ...... 97 ...... 108 ...... 110 ...... 85 ..... 103 ...... 107 ...... 79 ...... 112 ...... 107 ...... 71 ...... 101 ...... 77 ...... 108 ...... 97 ...... 82 ...... 104 ...... 104 ...... 101 ...... 74 ...... 97 ...... 76 ...... 84 ...... 87 ...... 109 ...... 106 ...... 74 ...... 98 ...... 83 ...... 88 ...... 105 ...... 83 ...... 103 ...... 101 ...... 104 ...... 97 ...... 96 ...... 96 ...... 99 ...... 100 ...... 79 ...... 103 Interrupt Controller...... 95 ramming Sequence...... 110 Core ...... 89 the Same Core ...... 88 ID’s...... its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ...... System Components ...... 93 ...... esponding to Anotheresponding to ample — Register Prog ample — Register Core-Other Registers in the Global gister Address Map ...... Register Usage...... rol Blocks ...... de ...... lay...... sions ...... Addresses in Memory ...... 6.2.8.1:Clock Domain Change Ex 6.2.8.2: Clock Change De 6.1.1: Power Domains...... 6.1.2: Clock Domains...... 6.1.3: Core and IOCU Selection...... 6.1.4: Overview of Power States...... 6.2.1: Cluster Power Controller Re 6.2.5: Requestor to CPC Registers ...... 6.2.6: Enabling Coherent Mode ...... 6.2.7: Master Clock Prescaler...... 6.2.8: Individual Device Clock Ratio Modification 4.7.1: Interrupt Compatibility Mo 5.7.1: Prefetch Enable...... 5.7.2: Select Ports for L2 Prefetching ...... 5.7.3: Enabling Code Prefetch ...... 5.12.1: CM GPR Register Interface ...... 5.12.2: MMIO Region Control ...... 4.7.2: Interrupt Vectored Mode ...... 4.7.3: External Interrupt Controller Mode...... 6.2.2: CPC Base Address ...... 6.2.3: Global Control Block Register Map...... 6.2.4: Local and Core-Other Cont 5.1.1: CM Interface — Register and Device Ring Bus 5.1.2: CM GCR Register Map ...... 5.1.3: Core-Local GCRs...... 5.1.4: Core-Other GCRs ...... 5.1.5: Core-Local and Core-Other 5.5.1: Another Programming Virtual Processor (VP) in 5.5.2: Programming Local GCR’s Corr 5.5.3: Accessing the CPC Local Registers via the CM ...... 5.5.4: Powering Up the Debug Unit (DBU) via the CM ...... 5.5.5: Setting the RatiosClock Between the I6400 5.5.6: Accessing the Core-Local and Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 5.8: CM Uncached Semaphore Management ...... 5.9: Custom GCR Implementation Processing...... 5.10: Error 5.11: IOCUInterface...... 5.12:Address Regions MMIO ...... 6.2: CPC Register Programming ...... 6.1: Overview...... 5.2: Verifying Overall System Configuration..... 5.3: the Programming Base 5.5: Examples...... CM Programming 5.1: CM Overview ...... 4.6: Exception Handling and FlowchartsServicing ...... 4.7: Interrupt Mode Code Examples...... 4.7: Mode Code Interrupt 5.6: Coherency Enable ...... 5.7: L2 Cache Prefetch...... 5.4: CM Register Access Permis MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 3 Chapter 6: Power Management6: Power Chapter ...... Chapter 5: Chapter Coherence Manager......

6 128 145 . 115 . 115 .... 143 .... 114 .... 115 ..... 125 ..... 126 ..... 126 ...... 133 ..... 119 ...... 131 ...... 132 ...... 132 ...... 133 ...... 139 ...... 142 ...... 121 ...... 142 ...... 142 ...... 112 ...... 136 ...... 119 ...... 124 ...... 143 ...... 117 ...... 136 ...... 119 ...... 120 ...... 120 ...... 116 ...... 134 ...... 113 ...... 146 ...... 120 ...... 124 ...... 112 ...... 127 ...... 119 ...... 122 ...... 143 ...... 119 em Programmer’s Guide, Revision 1.00 ...... Delay ...... 114 ...... 135 ...... 14 r Domain...... 115 ...... in the System...... 123 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ...... Delays...... d Root Interrupts d Root or Disabled Interrupt StateInterrupt or Disabled ...... 132 y Count ...... Timer Interrupts...... Group...... rrupts ...... est Software Access to GIC registers...... 144 urces ...... se Address and Enabling the GIC...... 123 ...... Powerup ...... g ...... sor Interrupts...... 7.2.4.1: Trigger Type Register 7.2.4.2: Edge Type Register Group...... 7.2.4.3: Polarity Type Register Group ...... 7.2.7.1: WEDGE Register Programming Example .... 7.2.5.1: Mapping an Interrupt Source to a VP ...... 7.2.5.2: an Mapping Interrupt Source to a Specific Processor Pin ...... 129 7.2.6.1: External Enabling Inte 7.2.6.2: Disabling External Interrupts ...... 7.2.6.3: Determining the Enabled 7.2.6.5: Programming Example ...... 7.2.9.1: Local Interrupt Routing ...... 7.2.9.2: Local Interrupt Masking ...... 7.2.6.4: Polling for an Active Interrupt ...... 7.2.8.1: GIC Interval Timer ...... 7.2.8.2: GIC Watchdog Timer...... 7.1.2.1: Non-EIC Mode...... 7.1.2.2: EIC Mode...... 6.2.14.1:Sequence Global Dela 6.2.14.2: Rail Delay ...... 6.2.14.3: ResetDelay ...... 6.2.12.2: RAM Shut Down Mode ...... 6.2.12.1: RAM Deep Sleep Mode...... 7.2.7: Inter-proces 7.3.3: Qualification of Root or Gu 7.3.4:Guest Mode Count-Compare 7.2.8: Local Timer Configuration...... 7.2.1:Ba Setting the GIC 7.2.4: ConfiguringInterrupt So 7.2.5: Interrupt Routin 7.3.1: Enabling Virtualization Mode...... 7.3.2: Routing of Guest External Source Interrupts ...... 7.2.2: the Number Determining of External Interrupts 7.2.6: Enabling, PollingDisabling, and Interrupts 7.2.3: EIC Setting...... Mode 7.2.9: Local Interrupt Routing and Masking ...... 7.1.1: GIC Virtualization ...... 7.1.2: GIC Operating Modes ...... 7.3.5: Watchdog (WD) Timer Guest an 7.1.3: GIC Register Types...... 7.1.4: GIC Register Distribution ...... 7.1.5: GIC Address Space Configuration...... 6.2.13: Accessing the CPC Registers Anotherin Powe 6.2.14: Fine Tuning Internal and External Signal 6.2.9: Standalone CM 6.2.10: ResetDetection...... 6.2.11:Run/Suspend...... VP 6.2.12: Local RAM Deep / Sleep Wakeup Shutdown and Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 7.3: Virtualization Support ... 7.2: GIC Programming...... 7.4: GIC User-Mode Visible Section...... 7.1: Overview...... 4 MIPS64® I6400 Multiprocessing Syst Chapter 7: Global Interrupt Controller...... Interrupt 7: Global Chapter

163 .... 170 .... 158 ...... 167 ..... 147 ...... 150 ...... 169 ...... 159 ...... 162 ...... 165 ...... 168 ...... 157 ...... 156 ...... 153 ...... 173 ...... 167 ...... 173 ...... 159 ...... 169 ...... 151 ...... 171 ...... 172 ...... 151 ...... 153 ...... 147 ...... 147 ...... 161 ...... 158 ...... 172 ...... 158 ...... 148 ...... 174 ...... 148 ...... 149 ...... 155 ...... 152 ...... 173 ...... 155 ...... 165 ...... 170 ...... 161 ...... 150 ...... 159 ...... 167 ...... 147 ...... sters ...... 154 ...... its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ...... A and non-MSA Functions...... 163 nt Register Usage for -mmsa and -msimd-abi=msa ...... 163 ...... Point Registers toPoint Registers MSAVector Regi g Modes ...... ction...... int FCSRRegister...... s...... itecture ...... ns...... ation...... 9.4.1.1: ABI Requirements ...... 9.4.1.2: Command Line Options and Function Attributes...... 162 9.4.1.3:Vector and Floating-Poi 9.4.1.4: Between Inter-calling MS 9.4.1.5: GNU MSA Options Directives...... and 10.3.3: Debug Mode...... 10.3.1: Root Mode Operation..... 10.3.2: Guest Mode Oper 8.1.1: IEEE Standard 754 ...... 8.1.2: RegisterFloating Point 9.3.1: MSA Exception Types...... 9.3.2: MSA Non-Trapping Exceptions...... 9.3.3: MSACSR Cause Register Field Update Pseudocode...... 160 Guest Operatin 10.1.1: Root and 10.1.2: Introduction to the Hypervisor ...... Guest Mode Translations...... 10.1.3: Enabling 10.1.4: MMU Consideratio 10.1.5: GuestID...... 10.1.6: CP0 Structure in Root and Guest Mode...... 10.1.7: New Registers...... CP0 10.1.8: New CP0 Instructions...... 9.4.1: MSA ABI...... 9.2.1: Enabling MSA ...... 9.2.2:Setting a MSAException ...... 9.2.3:Mode...... Rounding Setting the 9.2.4: Operation of the FS Bit...... 9.2.5: Operation of the NX Bit ...... 9.2.6: Programming the MSA CSR Register...... 9.1.1: MSA Instruction Formats...... 9.1.2: SIMD Instructions...... 9.1.3: MSA Vector Registers...... 9.1.4: Layout of MSA Registers ...... 9.1.5: Mapping of Scalar Floating- 9.4.2: MSA Vector Element Sele 9.4.3: Examples ...... Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 8.3: Setting a Floating Point Exception...... 8.3:Setting 8.2: Enabling the Floating-Point Unit ...... 8.4: Setting the ModeRounding ...... 8.5: Operation of the FS Bit ...... 8.6: the Programming Floating Po 10.2: Software Detection of Virtualization...... Operation10.3: Modes Of ...... 9.4: MSA GNU Compiler Support...... 9.3: MSA Exceptions ...... 10.1: Overview...... 9.2: MSA Programming ...... 9.1: Overview of the SIMD Arch 8.1: Overview...... MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 5 Chapter 10: Chapter Virtualization ...... Chapter 9: MIPS® SIMD Architecture (MSA)Architecture 9: MIPS® SIMD Chapter ...... 151 Chapter 8: Floating-Point Unit (FPU)8: Floating-Point Chapter ......

0 . 178 .. 178 ... 177 .... 185 ..... 188 ...... 210 ...... 200 ...... 200 ...... 192 ...... 180 ...... 176 ...... 178 ...... 178 ...... 189 ...... 194 ...... 210 ...... 179 ...... 213 ...... 196 ...... 185 ...... 206 ...... 174 ...... 199 ...... 205 ...... 201 ...... 194 ...... 188 ...... 211 ...... 181 ...... 202 ...... 209 ...... 193 ...... 211 ...... 203 ...... 211 ...... 209 ...... 211 ...... 205 ...... 202 ...... 213 ...... 214 ...... 178 ...... 199 ...... 186 er 9, Select 6) Select 7) ...... s ...... 208 Cells...... 207 em Programmer’s Guide, Revision 1.00 ...... 177 ...... its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ...... No Index Shift and No Invalid Cells...... 206 2-Bit Index Shift and No Invalid 4-Bit Index Shift and Invalid Cell Register Index — SAARI (CP0 Regist — SAARI Index Register Register — SAAR (CP0 9, Register s Hypervisor Exceptionss Hypervisor ...... 185 quence...... Exception...... de ...... ures ...... Access Shared TLB to the ...... r Exceptions from Guest Mode...... 18 ...... er Allocation...... er Access...... er Initialization Control...... and Locations...... upts...... Register Management...... Interface...... porting ...... 10.7.1.2: EIC Interrupt Handling ...... 10.7.1.1: Non-EIC Interrupt Handling ...... 11.1.1.1: Special Access Address 11.1.1.2: Special Access Address 11.2.5.1: Examplewith32 Cells 1: 11.2.5.2: Examplewith32 Cells 2: 11.2.5.3: Example 2: 20 Cells with20 Cells11.2.5.3: Example 2: 10.5.1.1: Root10.5.1.1: and Guest Wired10.5.1.2: 10.5.1.3: CP0 Regist 10.5.1.4: CP0 Regist 10.5.1.5: CP0 Regist 10.7.2: Derivation of Guest.CauseIP/RIPL...... 10.7.2: Derivation of 10.7.1: External Interr 10.7.1: External Interrupts...... 10.7.3: Timer 10.7.4: Performance Counter Interrupts...... 11.4.2: Programming Constraints11.4.2: Programming ...... 11.5.1: AXI Bus Error11.5.1: AXI ...... 11.5.2: Parity Error...... Error11.5.3: Execution ...... 10.6.3: Guest Initiated Root TLB 10.6.4: Exception Priority10.6.4: Exception .... 10.6.5: Exception Vector10.6.5: Exception 10.6.6: Synchronous and Synchronou 10.6.7: Guest Exception Code in Root Context .. 11.2.1: ITU Cell Types ...... 11.2.2: Cell Views ...... 11.2.3: Cell State...... 11.2.4: ITU Cell Addressing ...... 11.1.2: ITU Control Register ...... 11.2.5: Cell Indexing Examples...... 11.4.1: Register Programming Se 11.1.1: New CP0 Registers...... 10.6.2: Faulting Address fo 10.6.2: Faulting 10.5.1: Root and Guest Shared TLB Operation...... Shared Guest and 10.5.1: Root 10.6.1: ExceptionsMo in Guest Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 10.8: Watchpoint Debug Support ...... 10.9: Guest Mode and Debug Feat 10.7: Interrupts ...... 11.5: ITU Error Re 11.5: ITU 11.2: ITU Cell Structure ...... Software 11.3: ITU 11.4: Accessing the ITU Module...... 12.2: Data Flow ...... 12.1: Instruction Flow12.1: Instruction ...... 11.1: Overview...... 10.5: HandlingException Root in Guest and Mode...... 10.4: AddressTranslation Pseudocode...... 10.6: Exceptions ...... 6 MIPS64® I6400 Multiprocessing Syst Chapter 12: Chapter Multithreading ...... Chapter 11: Inter-Thread Communication Unit...... Communication Inter-Thread 11: Chapter 199

0 7 220 .. 220 .... 220 .... 220 .... 215 .... 215 ...... 219 ...... 220 ...... 219 ...... 219 ...... 219 ...... 219 ...... 219 ...... 219 ...... 220 ...... 217 ...... 215 ...... 217 ...... 219 ...... 219 ...... 219 ...... 214 ...... 219 ...... 220 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ...... n ...... r) (v3) ...... Configuration Registers) ...... 220 ontroller) ...... (DVP) Instruction ...... iguration Registers) ...... 22 ...... urces...... l Interrupt Controller) ...... sters ...... 13.1.4.1: BreakpointController. 13.1.5.6: IOCU (I/O Coherence Unit) ...... 13.1.4.2: Dseg ...... 13.1.4.3: Dmseg ...... 13.1.4.4: Drseg ...... 13.1.4.5: CP0 Regi 13.1.5.1: GIC(Globa C (Cluster Power 13.1.5.2: CPC 13.1.5.3: GCR (Global Conf 13.1.5.4: CGCR - (Custom Global Manage 13.1.5.5: CM- (Coherence 13.1.1.1: APB Slave Port...... 13.1.1.2: JTAG TAP ...... 13.1.1.3: Debug Monitor ...... 13.1.1.4: RAM...... 13.1.4: Per Core/VP Reso 13.1.5: Coherence Devices...... 12.3.1: Disable Virtual Processor Virtual Processor 12.3.1: Disable 12.3.2: VirtualEnable Processor (EVP) Instructio 13.1.1: Debug Unit (DBU)13.1.1: Debug Unit ...... Bus...... 13.1.2: Register 13.1.3: Number of Breakpoints . Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 13.2: More Information13.2: More ...... 12.4: Independent Exception Model ...... 13.1: OCI System Debug Overview...... 12.3: Thread Management12.3: Thread ...... MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 7 Chapter 13: MIPS On-Chip Instrumentation...... On-Chip 13: MIPS Chapter 21 em Programmer’s Guide, Revision 1.00 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 8 MIPS64® I6400 Multiprocessing Syst

TLB architecture, ster ster interface to each onents. For the exact revi- onents. ster address space is also is space ster address exception handler. a programmable parameter overview of the ssing registers in ssing registers another power ming examples of a clock domain for each block, as well as more in well as as block, for each scription of its functionality, and a and of its functionality, scription elements of the Translation Lookaside the Translation of elements set can be used to perform the same theset can be used to perform lists each device ID on the bus. The pro- The ID on the bus. lists each device ing up cache coherency, handling cache coherency, cache ing up ption of the CP0 regi example of an . The majority. The of blocks in the diagram have at ndalone mode(no cores enabled), reset detection, VP wer Controller (CPC), and/or Debug Unit (DBU) reg- wer Controller on howon to set the CPC memorybase address in is pro- l exception vectors in memorylistA is also covered. overview of the CM regi in the same core, accessing a VP in another core, VP in another a core, accessing the same in ram the CM totheram perform various CM functions,including set- ce of how to set or changehowset or toce of the various I6400comp system and clockdomains the programmer can use to manage the hardware and code. using registers assembly The regis- ects of the I6400 64-bit MIPS ects of MultiprocessingSystem Figure 1.1 delays to help the programmer easily integrate the device easily the programmer help to delays its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its utdown and wake-up procedure, acce utdown and wake-up procedure, ssing System. The first section gives an gives first section The ssing System. s, system power-up policy, program policy, power-up s, system ring bus and associated table that ger, refer to the Release Notes. Release the refer to ger, alongassembly withlanguage an lization code for all three caches, sett three caches, all for lization code overview of the cache architecture, a de a cache architecture, overview of the amples show how the MIPS instruction : This chapter provides an overview of how power is managed in the I6400 Mul- ackgroundinformation required bythe programmer in order tounderstand the as enabling as and initialization are provided : This chapter describes the programmable describes chapter : This that go into programing the caches. A descri A the caches. that go into programing (CM): The I6400 MPS contains a third generation Coherence Manager. This chapter pro- containsMPS a thirdThe I6400 generation(CM): Coherence Manager. : This chapter describes: This chapter an overviewof exceptiona definitionand processingof the interrupt modes. : This chapter provides an an chapter provides This : sion number of the Coherence Mana changeclockand delay change, poweringin sta up the CPC run/suspend mechanism, local RAMsh vided. Other programming principles include setting the device to coherent or non-coherent mode, requestor register of CPC access (core or IOCU) provided. In addition, the chapter describes how to prog (GIC), Cluster Po Controller Interrupt the Global accessing theisters viasetting CM, and clock the ratios between tiprocessing System and identifies the various power power consumptiondevice.In addition,the in a procedure domain, and fine tuninginternal and external signal system environment. into a Cluster Power Controller (CPC) Power Controller Cluster ting the base addresses in memory, accessing another VP accessing in memory, ting the base addresses Caches Exceptions Buffer or TLB of the I6400Buffer Multiproce descriptiona of its functionality and a descriptionelementsthe of that go into programmingThe sec- TLB. the thattions follow specific information cover programmingon (TLB). Lookaside Buffer for the Translation elements description of the initia as well as provided, cache is exceptions, and testing the cache RAM. Information on how to program the boot, reset, and genera exceptionof prioritiesprovided, is Memory Management (MMU) grammer uses this information to access these devices. An Coherence Manager Coherence CM register an overview of the vides • • • using registers. The assembly code ex registers. The assembly using function. provides the relevant b chapter Each such examples Common examples. depthrelative examplesblock. to that An overview of the material provided in this document is as follows: • least one dedicated chapter that describes how to control how describes chapter that least one dedicated examplesprogrammingter-programming describe a sequen This documentdescribes the software-programmable asp consistsThe devicelogic of the (MPS). blocksin shown • Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 9 Architecture Overview Architecture Chapter 1 Chapter

and machine states to states and machine on, and system implementa-on, and system rface and external debugging and external rface hip Instrumentation(OCI) maintains Level 2 (L2) cache maintains Level 2 (L2) ples that describe how various pro- functionality is already built to in provides in a best class power effi- such, it is not necessary for the pro- for necessary such, it is not em Programmer’s Guide, Revision 1.00 e setting the operating mode, setting up e to Linked-Load/Store-Conditional structions, registers, registers, structions, by programmer’s writingby programmer’s their own code to t Architecture with full hardware multi- (MD01041), This companion document provides fication, system integrati oherent interfaces, and L2 cache size. ief overview of the inte ief overview of the . t library, RTOS, or their own tool chain. However, most or theirtool ownchain. However, RTOS, t library, of virtualized systems. The Virtualization Module of virtualized is systems. The Virtualization zing gating storage. The chapter describes the purpose for describes the purpose The chapter zing gating storage. the core can be configured with a SIMDengine supporting ems and allows for the execution of Guest Operating Sys- Guest Operating the execution of ems and allows for incorporate the MIPS On-C its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its d operationdthe Flush-to-Zero of function. (FS) The I6400 Coherence Manager contained in the document suite. This documentcontains sup- : The ITU provides an alternativ ITU : The how to enable how tofloating how MSA, point toregistersscalar to map MSA ters, there are assembly language exam ples. ples. Some includ of these elements examples when using Codescape as this Codescape when using examples multi-core system that system multi-core microprocessor of the As MIPS Codescape tool chain. : This chapter describes how to program the various elements of theGIC using : This chapterMIPSthe describes Single-Instruction-Multiple-Data (SIMD) : This chapter provides a br chapter provides This : riable number of cores, I/O c I6400 Programmers Guide I6400 Programmers : This chapter provides information on how to enable the FPU, how to handle floating d in software. These examples can be used examples d in software. These the Release 6 of the MIPS64 Instruction Se 6 of Release the n-chip (SoC) applications. n-chip (SoC) d MSAexception handling. : The Virtualization Module defines a set of new in : The Virtualization : This chapter providesmulti-threadingof an overview the hardware mechanism I6400 in the architecture. It provides information on Floating Point Unit (FPU) Floating (MSA) MIPS Architecture SIMD the address map,GICthe address register layoutdistribution,and settingaddress, the determining GIC base the number of external interrupts, and configuring individualinterrupt sources. point exceptions, how to set the rounding mode, an an vector registers, Global (GIC) Interrupt Controller exam examples and code both register implementationcorethe I6400 manage to the efficient designedvirtualization to enablefull of operating syst Inter-Thread Communication Unit (ITU) Inter-Thread tems in a fully virtualized environment. Virtualization (VZ) Virtualization Multi-threading On-Chip Instrumentation(OCI) synchronization for fine grained multithreading by utili thethe ITUand configuration and programmingaspects. MPS. environment required to debug MIPS processors that debug system for multi-core designs. plementalinformation to the 64-bit MIPS I6400 Technical Reference Manual Reference 64-bit MIPS I6400 Technical hardware details abouthardware the device, including functional veri tion. 64-bit MIPS I6400 Multiprocessing System Integrator’s Guide System Integrator’s 64-bit MIPS I6400 Multiprocessing ciency for use in system-o ciency The I6400 series is a high performance series is The I6400 • • and system level coherency between all cores, main memory, and I/O devices.and The I6400 MultiprocessingSystem leveland system coherency between all cores, main memory, va configured with a (MPS) can be threading and hardware virtualization support. In addition, single and double precision, and floatingfixed point andoperations. integer, • Each I6400 core implements Each • • • • Throughout all of the aforementioned chap gramming elements are handle program a particular block, or for writing a low-level suppor of the code are part examples described the Codescape software. the Codescape This document meantis to be used withcompanion other two documents: • grammer to manually execute these code code these grammer to manually execute • Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 10 MIPS64® I6400 Multiprocessing Syst 1.1 Overview Product

chap- Device Device

JTAG document chapter of I/O I/O

Coherence Coherence Coherent Coherent I6400 Registers Unit

chapter of this manual. chapter of Power Controller Power Controller

Debug Global Interrupt Controller Optional IOCU0 IOCU1 Cluster this block are used throughout the Coherence Manager Coherence

chapter of this manual. block, refer to the the to refer block, block, refer to the Control Custom Registers block, refer to the companion Caches its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its , refer to the to refer ,

Global Control Registers , refer to the Cache

rious CM register programmingexamples. L2

4 ‐ Global n package. Selected registers from Interrupt Controller Integrated

Controller ith integrated L2-cache

AXI Cache with

5

Global Interrupt Controller (GIC) Global Interrupt Controller Coherence Manager (CM) Coherence Memory L2 Cache Cluster Power Controller (CPC) Controller Power Cluster

Global Configuration Registers L2 Power

Core Manager

Cluster 1 Figure 1.1 System-level Block Diagram of I6400 Multiprocessing System shows a blocka shows diagram I6400of Multiprocessing a System. I6400 The MPS contains the followinglogic

Coherence chapter of thischapter manual of to provide va Core 0 this manual. this For more informationthe on For more information onthe blocks: • Up to six cores • Coherence Manager (CM) w • Up to two I/O Coherence Units (IOCU) • Cluster Power Controller(CPC) • Global InterruptController (GIC) • Global ConfigurationRegisters (GCR) • Multiprocessor debug via in-systemUnit Debug(DBU) Figure 1.1 ter of this manual.of this ter For more informationthe on For more informationthe on For more information on the For that is includedthat in the documentatio Manager

Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies Core MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 11

Figure rallel processing rallel processing , etc. refer to the to refer , etc. MSA , rtualized). The core element FPU ates a software-programmable a software-programmable ates features as described in the follow- as described in the features , lowing efficient pa lowing efficient te with the Hypervisor and with each with te with the Hypervisor and em Programmer’s Guide, Revision 1.00 ration by keeping frequently accessed frequently accessed by keeping ration or managesor memory all I/O privileges. MMU e many compute-intensive applications by set-associativity , such as and reliability wideof range concernsdevices. for a or" type so thattype your so use of C/C++ code can makeor" echnology incorpor Core hardware assistance (fully vi r Portal about recommendations on which Hypervisors are ecution of high-level languages. Arithmetic and logic oper- ed hardware. This programmable solutionallowsfor its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its the resources by defining access policies for each execution policies access defining by resources the itecture, system, and core-level main itecture, system, and ting 8-way and 16-way ting 8-way ilers to further optimize code gene liability of the system by allowing the rest of the guests to operate reliably instructions improve performance by al sed on a fixed-length,encodedregularly instruction a load/set, and it uses ated from each other, but can communica ated from each other, MD Architecture (MSA) t herwise becomes corrupted. The hypervis corrupted. The herwise becomes In addition, the MSA is designedthe In addition, to accelerat •of the L1 data caches Inclusive . Contact Imagination Customer Support through our Partne environment or “guest.” Guests are isol environment or available for use. • cores R6 MIPS64 coherent six to Up • L2 cache Integrated, controller suppor 1.2 subsections. ing privacy The hardware virtualization support addresses security, with or withonly, can be achieved software Virtualization a smallof virtualization body of trustedis the Hypervisor, and privileged code that sits above the hardware, managing It manages resources. SoC all of the and orchestrating other via secure APIs.ensures This the re ot fails or guests one of the even if Single Instruction Single Multiple Data (SIMD) The MIPS64 6 architecture Release is ba The I6400 MPS contains the following arch MPS contains the following The I6400 store data model. It is streamlined to support optimized ex ations use a three-operand format, allowing compilers of to optimize complex expressions formulation. Availability comp 32 general-purpose registers enables registers. data in For more informationprogrammablethe on blocks within the of vector operations. The MIPS® SI increased system flexibility. increased system flexibility. solutiontothose handle functions not covered by dedicat compiler support. generic enabling MIPS gcc compilertunedbeenThe to"vect understand the has SIMD vector features. Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 1.2.4 System-level Features 1.2.3 MIPS® Virtualization 1.2.2 MIPS® SIMD Architecture 1.2.1 MIPS64® Release 6 Architecture 12 MIPS64® I6400 Multiprocessing Syst 1.2 Features I6400

memcopy address and 256-bit data paths ormance of workloads such as workloads ormance of acked by fast programmableacked by fast second-level on-core vari- its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its t Architecture nked and Store Conditionaloperations (LL/SC) Store nked and on to an external memory array. on to an external memory roller significantly improves perf significantly roller e AXI-4 bus protocol with 48-bit e AXI-4 bus 6 Instruction Se ectionCode (ECC) protection • 256 KB to 8 sizes MB cache • Error correction and detection able page size TLB (VTLB)able page and fixed page size TLB (FTLB) • Highdata bandwidthcore 128-bit pathsCoherence and the between each Manager • power management level controlled core Software • Debug port supporting multi-coredebug (JTAG/APB) • L2 cache prefetch cont Hardware • and IOCU ports Independent clock ratios on core, memory, • system SoC interface supports th • High-speedL2 cache initialization • to CPC shut down idle cores for power efficiency • Up totwo IOCUs • modulesupport Virtualization • Cache-to-cache data transfers • Out-of-order data return •connecti for AXI-4 interface Provides • MIPS64 Release 64-bit Full • 48-bit virtual and physical addresses • design efficient Power • Dual issue instruction fetch, decode, issue, and graduate •(VP) support Processor multithreading Hardware through seamless Virtual • support Virtualization • with Error Corr L1 caches • Load and store bondingsupport •support Unalignedload in hardware / store • Accelerated Uncached support • Optional Communication Inter-thread Unit(ITU) • Support for uncached and paired Load-Li • MemoryManagementUnit with first-level ITLB/DTLB b Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 1.2.5 Core-level Features MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 13

, refer tothe chapter of this *UDG *58 ,QVWUXFWLRQ Figure 1.2 Caches erical versions.  :5) :ULWH 5HVXOW &ROOHFW Idiom Name Register 'HEXJ,QWHUIDFH 5HVHW,QWHUUXSW 3RZHU0DQDJHPHQW em Programmer’s Guide, Revision 1.00 order for the assembler to for the order understand assembler Assembler blocks shown in 'DWD &DFKH blocks,the refer to their respective num 0'8 'HEXJ&RQILJXUDWLRQ,QWHUIDFH $/8/68 $/8&78 VTLB/FTLB )3806$3LSH$ )3806$3LSH% ion that the assembler will understand. The numerical ver- chapter of this manual. chapter of this manual. $GGUHVV *HQHUDWLRQ its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its , and L1 Data Cache FPU MSA and   Data TLB 7/% 'DWD :5) ,VVXH 5HDG , %\SDVV 2SHUDQG ,QVWUXFWLRQ %XV,QWHUIDFH8QLW %,8 0&3%XV referredto as C0_. In Idiom Register Name maps the root register names to Assembler block, to refer the block, to refer the 933LSH 933LSH 933LSH 933LSH Instruction TLB L1 Instruction Cache MSA FPU Figure 1.2Figure Diagram Block Core-level I6400 'HFRGH0DS 'HS&KHFN Table 1.1 Table 1.1Table of CP0 Registers Mapping Assembler &RQWURO5HJLVWHUV chapterthis manual. of 3ULYLOLJH$UFKLWHFWXUH &DFKH ,QVWUXFWLRQ shows a block diagram of a single I6400 core. %+7 -5& 536 7/% 97/% )7/% $GGUHVV *HQHUDWLRQ ,QVWUXFWLRQ Throughoutdocument, this registers are these names, they must be mapped to their numerical vers they these names, $x, y reference. uses a sion For more informationthe on Figure 1.2 Memory Management For more informationthe on For more informationthe on manual. For more informationthe on $0, $0, 0 $0, 4 $2, 0 $3, 0 C0_INDEX C0_VPCONTROL C0_ENTRYLO0 C0_ENTRYLO1 7 $12, 6 $12, 0 $13, 0 $14, C0_GTOFFSET C0_GUESTCTL0 C0_CAUSE 0 $24, 0 $23, C0_EPC 0 $25, C0_DEBUG C0_DEPC 1 $25, C0_PERFCTL0 C0_PERFCNT0 Idiom Name Register Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies Assembler 14 MIPS64® I6400 Multiprocessing Syst 1.4 Register to Assembler Mapping CP0 1.3 Diagram Core Block I6400

. Click on: itecture. Linux kernel and ildroot, Yocto, and GEN- ildroot, Yocto, Idiom Name Register If the device is in mode, Guest only a www.imgtec.com Assembler (continued) ude Debian, OpenWRT, Bu ude Debian, OpenWRT, r, compilers, MIPS boot loader, and MIPS RTOS and MIPS RTOS and compilers,boot loader, MIPS r, nux kernel for the MIPS® arch 0_LLADDR 3 $28, C0_DDATALO its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its CMGCRBASE 5 $25, C0_PERFCNT2 0_CONFIG2 0 $26, C0_ERRCTL 0_WATCHHI1 5 $31, C0_KSCRATCH4 address all stages of productstagesaddress all of development, including MIPS e CP0 registers available in Root mode. e CP0 registers Idiom Register Name the MIPS architecture incl Assembler MIPS Linux

Table 1.1Table CP0 Registers of Mapping Assembler MIPS Insider  TOO. For more information on the MIPS Linux, refer to the Imagination website at Community distributions that currently support Imagination actively supports, develops and improves the Li Imagination offers a complete a Imagination portfolioofthat tools offers Note that the above table indicates thos Note that the above table subset of these registers are available. Linux, MIPS Android, CodescapeSDK, Codescape debugge IoT support.provided Someof the toolsdescribed are in the following subsections. $9,6$9,7 C0_SAARI C0_SAAR 2 $17, 0 $18, C0_MAARI C0_WATCHLO0 0 $30, 3 $29, C0_ERROREPC C0_DDATAHI $4,5 C0_MMID 0 $16, C0_CONFIG 6 $25, C0_PERFCTL3 $6, $6, 0 $7, 0 $8, 0 $8, 1 C0_WIRED C0_HWRENA 2$8, C0_BADVADDR$9, 0 C0_BADINSTR 3 $16, C0_BADINSTRP 4 $16, 5 $16, C0_COUNT 7 $16, 0 $17, C0_CONFIG3 C0_CONFIG4 C0_CONFIG5 $17, 1 C0_CONFIG7 C 0 $27, 0 $28, 1 $28, 2 $28, C0_MAAR C0_CACHERR C0_ITAGLO C0_IDATALO C0_DTAGLO 1 $29, C0_IDATAHI $3, $3, 1 $4, 0 $4, 2 C0_GLOBALNUM 4$4, C0_CONTEXT C0_USERLOCAL 0 $15, $5, 0 C0_DBGCONTEXTID 1$5, 1 $15, 3 $15, 2 $15, C0_PAGEMASK C0_PRID C0_PAGEGRAIN C0_ C0_CDMMBASE C0_EBASE 1 $16, 2 $16, 2 $25, 4 $25, C0_CONFIG1 3 $25, C C0_PERFCTL1 C0_PERFCTL2 C0_PERFCNT1 7 $25, C0_PERFCNT3 $11, 0$11, 4$11, C0_COMPARE C0_GUESTCTL0EXT 1 $19, 0 $19, C C0_WATCHHI0 4 $31, C0_KSCRATCH3 $12, 0$12, 1$12, 2$12, C0_STATUS C0_INTCTL C0_SRSCTL 2 $19, 3 $19, 0 $20, C0_WATCHHI2 C0_WATCHHI3 C0_XCONTEXT 6 $31, 7 $31, C0_KSCRATCH5 C0_KSCRATCH6 $10, 0$10, 4$10, 5$10, C0_ENTRYHI C0_GUESTCTL1 C0_GUESTCTL2 1 $18, 2 $18, 3 $18, C0_WATCHLO1 C0_WATCHLO2 C0_WATCHLO3 0 $31, 2 $31, 3 $31, C0_KSCRATCH1 C0_DESAVE C0_KSCRATCH2 Idiom Name Register Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 1.5.1 MIPS Linux Assembler MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 15 1.5 Tools Software Imagination

. Click . Clickon: . Click on: . Click on: . Click on: www.imgtec.com es. Therefore, it is recom- PS processors. Codescape PS processors. www.imgtec.com . For every Android www.imgtec.com ble for MIPS processors. www.imgtec.com running on MIPS cores. MIPS on running www.imgtec.com . em Programmer’s Guide, Revision 1.00 optimizedlibraries, and profilingtools. . . CC) and provides prebuilt tool chains in the Codes- . a complete a compile, suitedebug, of and profile tools velopment.Fully supporting MIPS architectural all fea- eased by eased Imagination Technologi its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its Source Project releases by releases Source Project ware development platforms, linked by a commondevelop- Codescape MIPS SDK MIPS Codescape g availa compilers are also r Imagination Technologies’ MI r Technologies’ Imagination izing MIPS cores and facilitatingandizing MIPS cores debugging.These include make the most of software most themake of  the mosttheofversion stable Android sources for MIPS. Compilers Boot Loaders Codescape Debugger    ader, refer to the Imagination website at ader, gether a wealth of expertise in the form of Windows and Linux software gethera wealthWindows of expertise form of the in ound in the process are fixed. These bug-fixes,fixed. These ound alongare in theoptimiza- with process MIPS any Developer Tools Developer Tools

(bare-metal and Linux), iaSIM simulator, simulator, iaSIM and Linux), (bare-metal 

Developer Tools Developer Tools Developer Tools Developer Tools Developer Tools MIPS Android MIPS PS software, Codescape refer website to the Imagination at

   

MIPS MIPS Insider  MIPS Insider MIPS Insider MIPS Insider MIPS Insider     Community MIPS SDK include cross compiler cross SDK include MIPS on: ment environment. ment For more informationMIthe on With Codescape, Imagination brings to Codescape, With developmenthardware tools, supporthardand packages Imagination offers a wideof range a Imagination solutions for initial offers tures, the Codescape Debugger enables developers to developers Debugger enables Codescape tures, the to the Imagination refer websiteFor at more informationMIPSthe on Debugger, Imagination ports and maintains the GNU Compiler Collection (G wide rangecape MIPS SDK. A of other industry leadin For more informationMIPStothe on the Imagination Compilers, refer website at Community open-source and proprietary solutionsany to suit requirement. For more informationMIPSthe on Boot Lo Community Imagination’s debug environment debug for heterogeneous SoC de Imagination’s The Codescape MIPS software development toolkit provides releaseQAperformed a is f and bugs tions,on go top of the Android release branch and get rel developingand libraries for and debugging software fo The MIPS emulatorThe Androidbe built can the fromOpen Community Community Instructions are on the web site. are on Instructions For more informationMIPSthe onto Android, the Imagination refer at website mended to downloadMIPS releases to get the latest Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 1.5.6 BootLoader 1.5.5 Compilers 1.5.4 Codescape Debugger 1.5.3 Codescape SDK MIPS 1.5.2 MIPS Android 16 MIPS64® I6400 Multiprocessing Syst

. . Click www.imgtec.com www.imgtec.com . stems. In addition,Imagi- e video training courses. the kernel or user software develop-thesoftware kernel or user MIPS RTOS and IoT Support RTOS MIPS n of IoT specific Operating Sy Operating n of IoT specific  . its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its s, documentation,and on-lin sources, refer to the Imagination website at to the Imagination website refer sources, s and associated resources to aid in to aid resources s and associated Operating System (MEOS) with Virtualization extensions that target extensionstarget that System Operating with (MEOS) Virtualization Developer Tools Developer Tools

ce and commercial partnersprovide to support MIPS many for of the popular 

Developer Resources

MIPS Insider  and IoT Support and IoT MIPS MIPS Insider  Community Community MIPS offers a variety of MIPS offers development board mentResources process. include development platform For more informationthe on MIPS Developer Re Imagination collaborates with open-sour Real Time Operating Systems (RTOS) and the new generatio and the new Operating Systems (RTOS) Real Time nation has developedthe MIPS Embedded space. and the IoT deeply embedded applications and IoT support, Imagination refer to the website at For more informationMIPSthe on RTOS Click on: on: Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 1.5.8 Developer Resources 1.5.7 MIPS RTOS MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 17 em Programmer’s Guide, Revision 1.00 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 18 MIPS64® I6400 Multiprocessing Syst

ntrol for differ- blocks are not soft- blocks are th the information used to System. The first section System. The first or TLBWR)move to the data

r operating systems r operating systems that manage must s and provides access co access s and provides respectively. These These respectively. the number of VPs. The ITLB maps4of only VPs. The ITLB the number e number of VPs. The DTLB maps onlyThe DTLB 4 KB, numberVPs. e of e instruction (TLBWI TLB is used as a backup structure for the ITLB.a the If for backup structure used as a is TLB nsistsof four address-translation lookaside buffers e same virtual address space. The MMU also enforces its functionality and a description of the elements that go B in theB in I6400 Multiprocessing 0 (CP0) registers 0 (CP0) registers wi coprocessor its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its core, to physical addresses used to access caches, memory caches, used to access addresses core, to physical tributes. The I6400 MMU implements a Translation Looka- MMU I6400The Translation tributes. implements a ess translation is especially ess translation is especially useful fo en executes a TLB writ en executes , the translation, the information iscopied into the ITLB/DTLB foruse. future the ITLB orthe DTLB, the VTLB attempts ITLB followingthe to translate it in clock addresses to 48-bit physical addresse to addresses ations and data caches for the instruction . Number of ITLB entries. Numberon of ITLB based varies 1 . Numbervaries of DTLB entries on th based 1 cycle or when available. If successful when available. cycle or fetch address cannot be translated by be fetch address cannot 16 KB, or 64 KB16 KB, or 64pages.is managed The DTLB hardware by and is transparent to software. KB, 16 KB, or16 64 KB pages. KB, transparentandITLB is managedKB, Thesoftware.to is by hardware –= 8 entries VP 1 – VPs = 14 entries 2 – VPs = 20 entries 4 –= 6 entries VP 1 – VPs = 12 entries 2 – VPs = 18 entries 4 initialize and modify entries in the TLB, th from the registers to the TLB. the registers from ware visible and are shown only shown for completeness. ware visible and are • V The larger VP. per (VTLB) TLB Variable 16 dual-entry ent page segments of memory. The core writes to internal The core writes to internal ent page segments of memory. The MemoryThe ManagementUnitthe (MMU) in co I6400 core and other devices. Virtual-to-physical addr devices. Virtual-to-physical and other 48-bit virtual TLB translates The I6400 (TLB): • TLB (ITLB) Instruction The MMU translates virtual addresses generated by the generated by MMUvirtual addresses The translates physical memory to accommodate multipleth active in tasks the protection of memory areas and defines the cache at (TLB). side Buffer of the TL elements programmable the This chapter covers gives an overview of the TLB architecture, a description of into programmingthatThe sections follow TLB. the cover specific informationprogramming on for the TLB. •(DTLB) TLB Data Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 1. The ITLB and DTLB perform address transl 2.1.1 TLB Types MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 19 2.1 Overview Memory Management Unit Management Memory Chapter 2 Chapter

included in the docu- ation on the Guest ation on the is not found, the VTLB/ the size of the VTLB an the the size of tically cleared whenever the whenever tically cleared associ- Data em Programmer’s Guide, Revision 1.00 Hit/Miss Hit/Miss Instruction the TLB. For inform the DTLB, the VTLB/FTLB is accessed. If is VTLB/FTLB the DTLB, the rst. If the translation If the translation rst.

and Index match.Indexand TLB entries which have their G RAM Cache RAM

all VPs. The FTLB extends Cache I6400 Technical Reference Manual Reference I6400 Technical exception is taken. Similarly, when a data reference is to data reference is when a exception is taken. Similarly, Tag Tag Instruction Data Comparator Comparator its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its inal translation requested. n. The OS should process the exception by overwriting a TLB as the VTLB when miss occurs a in the ITLB/DTLB. Entry Entry the VTLB when required, and automa when required, and the VTLB ITLB FTLB DTLB VTLB/ t. If the address is not present in not present is If the address t. of instructions used when accessing used when of instructions IVA DVA architecture, refer architecture, refer to the the I6400 MMU architecture. the IVA DVA

TLB or FTLB with the orig with FTLB or TLB

address is to be translated, the ITLB is accessed fi the ITLB is translated, to be address is Data Address Address Figure Figure 2.1 Core I6400 in the MMU Architecture of Overview Calculator Calculator Instruction — Invalidates a set of TLB entries based on Index match.set of Invalidates — a — Invalidates— of a set TLB entries based on ASID shows an overview of an overview shows Entries are automatically refilled from automatically refilled Entries are ated VTLB is updated. is ated VTLB extra 512 entries and is accessed at the same time same the at entries and is accessed 512 extra TLBINV bit set to 1 are not modified. TLBINVF •dual-entry 512 Fixed isshared between (FTLB) that TLB When an instruction be translated, the DTLB is accessed firs accessed is be translated, the DTLB Figure 2.1 an the VTLB/FTLB, in a miss If there is FTLB is accessed. there is a miss in the VTLB/FTLB, an exception is take entry from the appropriate V For more information on the MMU mentation package. TLB instructions used in Virtualization, refer to the Virtualization chapter of thismanual. chapter the refer to Virtualization TLB instructionsVirtualization, usedin • • This section defines the various types defines This section Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 2.1.2 TLB Instructions 20 MIPS64® I6400 Multiprocessing Syst

register is register is Index registers to be register. create a global name global name create a across cores and VPs. EntryHi ed by hardware tobewritten PageMask bit MMID (versus the 10-bit MMID (versus bit , and fy shared translations. However, to fy shared translations. However, and reduces contention. Even under virtual address. The virtual address. ications, there be multiple threads can Select 5. There is one MemoryMapID will be disabled. However, the VPID willHowever, be disabled. res/VPs with the same GID + MMIDres/VPs with the same GID + the I6400 uses a feature called Memor- a feature called the I6400 uses mmon name space enables sharing of mmon name space enables same translations on different VPs. same translations on different Identifier (ASID) to Identifier (ASID) registers. section of this chapter for more information. this section of a core. On previous generation cores, the FTLB a core. On previous registers. PageMask est ID (GID) that is common ID (GID) est both ASID and MMID. EitherASID oneboth and MMID. can be selected using , and the TLBa specific for its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its acement for the ASID. The 16- PageMask EntryHi, EntryLo0, EntryLo1 EntryLo0, EntryHi, n causes a random TLB entry select a random causes n , and Global TLB Invalidate d will enable the hardware to identi whose contents match the contents of the the contents of contents match whose register located at CP0 register located Register 4, , translations are common for all co are translations , anslations increases the FTLB capacity anslations increases anslations across all anslations VPs in the translations were not. In many appl across all VPs in a core and all cores, a core and in all VPs across ed with the same process and use the use and ed with the same process B flush when recycling ASIDs, the co B ASIDs, flush when recycling means that the VPID check in the FTLB VPID check in the the means that in thein legacy mode (Config5.MI = 0). or runningdata.In this situation, the same application sometranslations different on globalized TLB globalized invalidates. used to replace the traditional Address Space traditional Address the replace used to EntryHi, EntryLo0, EntryLo1 EntryLo0, EntryHi, EntryLo0/1 EntryHi, — The TLB Write Random instructio — The TLBWrite — The TLB Write Index instructionthe TLB entry causes pointed—to TLB The theWrite by Index to be written register — The Global — The InvalidateTLB instructiongloballya way to provides invalidate all TLBentriesmul- in — The TLB Read instruction causes the instruction causes Read TLB The — — The TLB Probe instruction is used to probe loaded with the address of the TLB entry the of the address loaded with TLBP TLBWR loaded with the contents of the TLB entry pointed to by the Index register. TLBWI the with the contents of the with the contents of TLBR GINVT tiple ways or the entire TLB. Refer to theRefer tipleTLB. entire ways or the check is still needed when operating needed when check is still Constraints Software MMIDtheWhen functionto is enabled, kernel software must the following adhere programming constraints: • All VPs for a given Guest ID (GID) must have the same setting in their Config5.MI field. •mustConfig5.MI.a changeto The be flushed FTLB before •givenVPs All GID must for a have thesetting. same FTLB page size • Rootand Guest need not have the same setting in their Config5.MI fields. CP0 MMID Register stored in the MemoryMapIDMMID is The VP. register per Aside from reducing the need for a TL for need Aside from reducing the VPs and translations between 1) (Config5.MI = enabled MMID is When root).(GID This= 0 as includes GID ASID) is sufficient to create a global name space for each Gu ASID) is sufficient space that is common across cores and VPs an VPs and cores common across space that is maintain backward compatibility, the I6400supports core maintainbackward compatibility, register. the CP0 Config5 repl is a (MMID) I6400 core, the MemoryMapID the In yMapID. This feature can be To enable sharing of FTLB translations sharing enable To The I6400 core supports shared FTLB tr core supports The I6400 entries were shared across the VPs, entries were shared across but • are common across VPs and sharing the tr and sharing are common across VPs that are working cooperatively that are working cooperatively • • • Linux, multiple threads can be associat • Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 2.1.3 Shared FTLB Translations MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 21

0 R2’b11 R/W 0 l entries in the VTLB/FTLB VTLB/FTLB the in l entries shows only those bits that are new bits that only those shows em Programmer’s Guide, Revision 1.00 Data TLB (DTLB) entries thatData TLB(DTLB) match entries the in Figure 2.3 instruction is executed, al its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its the ASID or MMID mapping is enabled, as well as a 2-bit well as mapping is enabled, as or MMID the ASID er Field Descriptions fer to the CP0 Config5 register for register for fer to the CP0 Config5 is used for FTLB translations. FTLB for is used 16 15 0, ASID(X) writes 0, and reads will Refer to the CP0 Config5 register for register for Refer to the CP0 Config5 1 to indicate that both global instruc- gister. MMID writes are dropped and are dropped MMID writes gister. structions are supported. In the I6400 the are supported. In structions instructions are supported. instructions Figure 2.3Figure Format Config5 Register Figure 2.2 Figure Format Register MemoryMapID In addition,In all InstructionTLB (ITLB) and Table 2.2Table Regist Config5 Table 2.1Table Descriptions Register MemoryMapID Reserved MMID more information. core, this field is hardwired to 2’b1 core, this field is hardwired 0: ASIDis enabled. is enabled.1: MMID are ASID(X) and writes allowed, are and reads writes = 1, MMID MI If MI = 0. When as and read dropped access the lower 8+2 bits of the re return 0. reads tion cache and TLB invalidate more information. 17 Indicates whether the ASID or MMID 14:0 Re version. from previous changes No 15:0 translation. ID used for the map value memory the Stores R/W 0 16:15 if global invalidate in Indicates 31:18 previous version. from No changes 31:16R Reserved. 0 Refer to the CP0 Config 5 register CP0 Config to the Refer MI GIregister Config 5 the CP0 to Refer field that indicates that the core supports globalinvalidate instructions. I6400in the core. All other Config5the bits remain same as before. CP0 Config5 Register CP0Config5The register has new fieldsindicate thatif VTLB are also invalidated. GINVTinstructionThe provides the optionto invalidate the TLB entries in the following ways: are invalidated in all cores. are invalidated in all The I6400 core provides kernel core providesThe I6400 thesoftware with ability globally to invalidatethe using the VTLB/FTLB structure new GINVT (GlobalInvalidate TLB) instruction. this When GI MI Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MMID Name Bit(s) Description Read/ Write Reset State Name Bit(s) Description Read/ Write Reset State 2.1.4 Global TLB Invalidate Reserved 31 18 17 16 15 14 0 31 22 MIPS64® I6400 Multiprocessing Syst

as $16,4. If the the standardlibraries tools the TLB entries across all entries across all the TLB information, as well as an e is non-zero, such to by their register name. For exam- TLB entries across all cores are invali- indicates the size of the VTLB. This the size of the indicates 2’b00, all TLB entries in all cores are 2’b00,all in all cores TLB entries the ‘type’ field is 2’b11, the ‘type’ In the I6400 core the VTLB sizecoreI6400 In the is fixed the VTLB at 16 entries, chapter are also provided chapter in register), the select numbernot shown is in the #define ster is located at CP0 register 16,CP0 registerat select 4 ($16,4). The ster is located ing optionsMMU.I6400 for theprovidesEach section 2’b01, the TLB entries across all cores are invalidated all cores are invalidated TLB entries across the 2’b01, ed to determine the required its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its program the various functions required to manage the MMU. Wired instructionsdetermine used to VTLB the size. , with C0_, with indicating the registerof is part the CP0 register set. . If the ‘type’ field is 2’b10, the 2’b10, the ‘type’ field is . If ual address of memoryual address map IDmateh. iting their own support library, RTOS, or tool chain. Notethat most of the RTOS, iting their own support library, number) are used when the select valu the select number) are used when ument, the CP0 registers are referred registers CP0 ument, the e’ field in the instruction is set to set to is instruction the in e’ field C0_Config4 register (CP0 Register 16, Select 1) Register 16, Select register (CP0 Config1 0 (for example, 6,0 for the MIPS Codescape SDK. SDK. MIPS Codescape d is 0x0Fis d (15 decimal). field in the MMUSize invalidated, without regard for any virt onlydatedmemory for those mapsthat match the MemoryMapIDvalue. onlymatch for those addresses thatvirtual the address. #define C0_CONFIG1mfc0ext $16,1 addiu C0_CONFIG1 t0, 6 t0, 25, t1, 1 t1, t0 into and place register Config1 //read t1 into and place 30:25 in bits value //extract in t1 value 1 to the //add cores are invalidated only for those memory maps thatthe match MemoryMapID value virtualas the well as address. •If the ‘type’ field is Invalidateaddress by virtual only. Determining the VTLB Size Code Code VTLB Size Example the Determining followingThe example showsassembly the language value is loaded by hardware based on the systemthe onbased is loadedvalue by hardware configuration. fiel of this so the size compiler interprets only the numerical value. Both variables (register number and select Register Interface Register 6-bit The ple, the Config4 register is referred to as referred to is register ple, the Config4 A separate #definestatementindicates that the Config4 regi select value of a register is Throughout the code examples in this doc CP0 registerCP0 information listingregisterthe and field(s) us The followingThe subsectionssome describeprogramm of the assembly code assembly example. This section is intended to provide examples of how to incorporated into the • TLB. the entire the ‘typ Invalidate If • Invalidate by Memoryonly Map IDvalue • Invalidateaddress by virtual and Memory If ID value. Map a programmer wr reference for a goodis It functionalityprogrammingthe of examples provided in this statementwill and be interpreted compiler. theas zero by Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 2.2.2 Determining the VTLB Size 2.2.1 Assembly Language Conventions MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 23 2.2 MMU Programming

size. This R/W field can be in the In FTLB. the I6400 core, em Programmer’s Guide, Revision 1.00 the 16 KB recommendation is when ze and organization of the MMU.ze and organization The num- Size FTLB Ways FTLB Sets ) determines the FTLB page re. Only the page FTLB size is configurable using the CP0 ecommendsimprovea 16 KB pagesize to using perfor- its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its instructions used to select a 16 KByte page size. select used to instructions ) indicates the number of ways the indicates ) ) indicates the number of FTLB sets per way. In the I6400 core,I6400the In indicatessets per way. ) of the number FTLB FTLB Page Size FTLB Sets FTLB Ways FTLB gisters are used to initialize the VTLB. the initialize are used to gisters field. er of TLB One exception misses. to register fields to determine register fields the si to determine Config4 Config4 Config4 INDEX Index iguration are described in the following subsections. addressingVTLBcan be used thebe mustinitialized. initialized The FTLB is automati- register ( bit definitions FTLB Page Figure 2.4Figure Fields FTLB Register Config4 CP0 register ( register ( Config4 Config4 Config4 See CP0 Register set for mance by greatly reducing the numb programmed to select pages sizes of 4 KB, 16ofKB.programmed KB or 64 4 KB, to select pages sizes The traditionalmost page butsize has been 4 KB implementationsOS supportMIPS r 16 KB. 4 KB or theusing Android OS which only supportsKB4page a size. field This encodedas follows: is #define C0_CONFIG4mfc0li $16,4 C0_CONFIG4 a0, insmtc0 2 a3, a3, 8, 5 a0, C0_CONFIG4 a0, into a0 and place register Config4 the // read register Config4 the of a0 into contents the // write a0 12:8 of in bits Size field Page FTLB // insert a3 into for a 16k Page size value // set thisread-onlyalwaysvalue fixedper is at 64 sets way. thisread-onlyalwaysvalue fixedways. is at 4 – 0x1: 4 KB – 0x2: 16 KB – 0x3: 64 KB Setting the FTLB Page Size Code Size Code Example FTLB Page the Setting followingThe example showsassembly the language • Bits 7:4 of the • Bits 12:8the of cally incally hardware and does notinvolvement. requirekernel software any Interface Register CP0 re following steps and associated The This section describes the procedure for VTLB/FTLB initialization. When the core is first powered up the TLB is not virtual Before for use. ready 1.to the CP0 zeros all Write The FTLB page The FTLB and conf size Interface Register CP0 core uses the following The I6400 ber of FTLB sets and FTLB ways are fixedin the I6400 co • the of 3:0 Bits Config4 register as follows. Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 2.2.4 VTLB and FTLB Initialization 2.2.3 FTLB Configuration Page Size 31 13 12 8 7 4 3 0 24 MIPS64® I6400 Multiprocessing Syst

reg- Index instruction TLBWI sfully translated by the core). translated sfully entry VTLB, whichadded to is instruction. This register is also used for the TLBWI software intervention is requiredto initializeFTLB. the 528 dual entries of the528 is FTLB. This value VTLB andentries of dual its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its field. Each VP contains a 16- field. Each VP invalidate is enabled. When this bit is set, the ether a particular ether address was succes INDEX ted on each VP before the TLB can be used. before ted on each VP // and invalidates all VTLB entries all invalidates // and Index TLBWI instruction. TLBWR register (CP0 register 0). 0, Select bit to indicate that Index EHINV instruction whichto failsmatch find a for the specified virtualbitof 31 address sets the instruction (used to determine wh instruction (used to determine EntryHi TLBP TLBP register determines which TLB entry is accessed by a TLB entry is which determines register the 512 entry shared FTLB to determine the total number of TLBentries. #define C0_INDEXli $0,0 mtc0 0x0000010F t0, C0_INDEX t0, entries) (# of VTLB 255 + 16 field = Index //set register into Index value //write #define C0_INDEXli $0,0 mtc0 0x0000000A t0, C0_INDEX t0, 10 field = Index //set register into Index value //write Index #define C0_INDEX mtc0 $0,0 tlbinvf C0_INDEX zero, register the Index into a zero // move caches TLB internal flushes Flush Invalidate // TLB acts as a TLB invalidateoperation, valid setting the hardware bit associated withentry theto TLB the invalid state. This bit is ignored on a Indexing the VTLB Example the Code Indexing followingThe example showsassembly the language instructionsindex used to VTLB entry 10. ister, indicating a probe failure. ister, Interface Register 1. Set the Indexing the FTLB Code Example the Indexing followingThe example showsassembly the language instructionsindex used to FTLB entry 255. stored in bits9:0 of the Initializing the FTLB Code Example I6400theIn the FTLB is initialized No kernel in hardware. indexA 10-bitusedto valuemaximum up index is to a of The instruction sequence above must be execu instruction sequence above The 2. instruction Execute a TLBINVF to initialize all entriesVTLB. in the Initializing Example the VTLB Code entriesAll theinitialized in VTLB are sameat thetime using the TLBINVF instruction. The Note that a that Note result of a 2. theappropriate TLB index to the Write Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 2.2.5 Indexing FTLB the VTLB and MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 25

. In this fig- exception. registers. The registers. register is used to register is Figure 2.5 EntryLo1 Index register. The ‘VPN2’ register. itable. If this bit is a one, the VTLB. If the value in TLB Modified and exception. Software can use use exception. Software can one of three cache coherency . e ASID. For more informa- register depends register on the page e appropriate data, then exe- appropriate data, e EntryHi r-up, this register is set to 0, and is register this r-up, n Linux, for example, each applica- Linux, for n EntryHi EntryLo0 e page size of the TLB entry. For the For TLB entry. e page size of the by the operating system. The ASID by the operating system. The em Programmer’s Guide, Revision 1.00 be set to the page size of the FTLB when the the FTLB when the size of page be set to the command is executed and the value in the in and the value command is executed e is stored translation in are set in the MMU s been written, and/or is wr er (which along with the offset becomeser (which the along with physical offset the . translation will most likely be stored in the FTLB. nd corresponding fields to be programmed when -size virtual-size regionwhich maps tophysical a pair of TLBWR mbernormal duringTLB data in the operation. Each data entry can have Each e first write that causes an causes e first write that ed as a replacement for th for a replacement ed as e supported. For the FTLB, page sizes of 4 KB, 16 KB, or s the registers above with th with above s the registers rtual to physical mapping. The TLB are filledentries using Cached Coherent Read-Share Cached Coherent of TLB flushing on a context switch. The ASID field its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its , or size of the VPN field in the of the VPN size cations to co-exist in the TLB (i bit is a cleared, stores to the page cause a page cause the to cleared, stores bit is a anslationwillVTLB. be stored to the a tag portion dual-data and shown portion in as e loaded with the appropriate information, the ace identifier (ASID) register. This register determines th register. command. When a When command. clature. The registers a clature. The registers 8-bit memory space assigned identifier command is executed. During reset or powe During reset or is executed. command Shared FTLB Translations TLBWI PageMask TLBWI used to manage the TLB entries. Uncached Accelerated , multiple different appli multiple different instruction. Note that the Uncached — Indicates the TLB page size. This register must register This the TLB page size. Indicates — TLBWR — Stores odd numbered frame physicalthein number TLB data during operation.Stores — normal — Stores even numbered physical frame nu — Stores — Stores the virtualin page numberduring TLB tag the — Stores normaloperation. or — Used when a stores to thestores to page are permitted. this If bitin shouldThe D the exceptiononhandler be set th thisbit that have to track pages been written clearingby to this bitpage when theismapped.first attributes: this value is written to all VTLB/FTLB entries. EntryHi EntryLo0 EntryLo1 PageMask FTLB is accessed using a registerPageMaskthequal does not page size, the FTLB address). the PageMask register FTLB page size, equals the the tr a slight chance the is there However, TLBWI • The D bit is the dirty flag and indicates that the page ha tion, to refer the section entitled • • Index designation indicates that this address is for a double-page helps to reduce the frequency field The ASID pages. extends the virtual address with an allows translations allows translations for code and data lyingthein sameregion). different addresstion virtual has (MMID) can be us I6400 core, the MemoryMapID the In VTLB, page sizes of 4 KB to 1 GB in powers of four ar 64 KB are supported. • • •to cache data for this page. The how C field indicates ‘0’ indicates the even numbered TLB entry, and the ‘1’ nomenclature and theindicates the odd numbered TLBentry. ‘0’ indicates the even numbered TLB entry, • Thecorresponding PFN stores the physicalnumb frame • accessing the TLB are listed as follows. listed as are the TLB accessing • set is PageMask in the CP0 • PFN0/1, C0/1,V0/1, D0/1, G0/1,andin RI0/1,set the MMU XI0/1 are bits size as noted by the ‘x’ and ‘x-1’ nomen ‘x’ and the as noted by size • sp address (VPN2) and address Virtual This section describesThis a vi how the TLB to create to setup the registers listed below. After the registers ar After the registers listed below. the registers determine which entry will be written. Interface Register of consists the VTLB/FTLB Each TLB entry in ure, the following registers are ure, the following registers To fill an entry in the VTLB/FTLB, kernel software update fillVTLB/FTLB, an entry in the kernel software To cutes a Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 2.2.6 Programming TLB Entry a 26 MIPS64® I6400 Multiprocessing Syst

TLB Invalid TLB ASID One TLB Entry bit is always writable RI concatenated with the PFN EntryHi TLB Tag TLB ddresses must be enabled by setting Entry 1 Entry Entry 0 TLB Data TLB VPN2 47 x x-1 10 9 0 (Valid) bit is set. The (Valid) en if the V (Valid) bit is set. The XI bit is writ-The XI bit is set. (Valid) Ven if the V Logical AND Logical V0 48 y, the G bits of both Entry 0 and Entry 1 reflect the y, registers is writable and registers is writable entry G bit is a one, then the ASID comparisons are the ASID then a one, is G bit entry 61 gical AND ofgical AND the G bits inand both the Entry 0 Entry 1 62 register is always set. D1 V1 D0 R is set, which is always the case in the I6400 core. in the case always which is is set, 63 0 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its this bit is a zero, accesses to the page cause a cause to the page accesses a zero, is this bit EntryLo1 C1 1 C0 Flags = 1), even if the 2 and PageGrain IEC 3 CDVG CDVG XI0 XI1 register. Setting this bit allows for 48-bit register. physical addresses. When this VPN2 ASID G ts that are stored in bits 63:62 of the virtual address and divide the virtual are used, support for large physical a are used, support for large 65 EntryLo0 bit of the RI0 RI1 PageGrain RIE 13 12 PageGrain R PageMask EntryLo0 PFNX/PFN PFNX/PFN exception ( EntryLo1 PageMask 41 PFN0 PFN1 TLBRI 42 Figure 2.5Figure TLB and Entries Registers CP0 Between Relationship PageMask 31 30 0 registers become the G bit in the TLB entry. If the TLB If the G bit in the TLB entry. registers become the causes a a bit is set, accesses to the page are permitted. If to the page set, accesses is bit exception. On a matches. ignored during TLB read from a TLB entr TLB G bit. the state of in the I6400 core since the in the the ELPA bit of the CP0 the ELPA virtual page causes a TLB Invalid or a TLBXI exception, ev a TLB Invalid or a TLBXI page causes virtual able only if the XIEthe the PageGrainof able only bit if register field to form the full page frame number. memory map into one of four regions. four of one into map memory bit isset, the PFNXfieldbit of the 61 •bit. write, the lo a TLB The Onthe “Global” G bit is • page virtual the on data read to attempt any entry, a TLB in set is bit this If flag. read-inhibit the is RI The • and thus the The V bitvirtual is mapping, page the valid is valid.flag If this indicates and that the TLB entry, • any The attempt XI is the toexecute-inhibit fetch an flag. If instructionthis bit is set fromin the a TLB entry, • The R field in bits are the region bi •than 32 bits larger address sizes If Xl Xl 62 61 42 41 65 3 2 1 0 62 Rl Rl 63 63 63 Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 27

Wired instruction. register is reset to aceable entries that TLBWI register specifies the register specifies field of this register field of Wired Wired fieldavoid to guest TLB Wired e wired. If the value in the wired. If the value e ire a TLB entry. In this example the In this TLB entry. ire a em Programmer’s Guide, Revision 1.00 field, which leaves at least one entry least at field, which leaves s are fixed, non-repl s are fixed, register is dropped. The Guest.Config1.MMUSize Wired Config1.MMUSize its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its 2, 1, and 0 of the VTLB ar VTLB the of 0 and 1, 2, configure the configure the register (CP0 register 6, Select 0). The //by 6 bits //by load TLB to instruction TLBWI //execute language instructions used to hard-w used language instructions Wired field is set to 0 by default, indicating that all but entries one of the VTLB , thenthe the write to , Root should instruction. However, wired entries instruction. However, can be overwritten by a random entries in the VTLB. Wired entrie random entries in the VTLB. Wired Wired.Limit TLBWR field can be set to the value in the the value in to set field can be Config1.MMUSize accomplished using the Limit #define C0_WIREDli mtc0 0x000F0005 t0, $6,0 C0_WIRED t0, = 5 field 15 and Wired field = Limit //set register into Wired value //write #define#define C0_ENTRYHI#define C0_ENTRYLO0#define $10,0 C0_ENTRYLO1 $2,0 #define C0_PAGEMASK $3,0 C0_INDEX $5,0 li mtc0 0x12340000 t0, li $0,0 $10 t0, li>> 6 0x23450000 t1, dinsmtc0 0x002F t2, t2, 0, 6 t1, 0 ASID = VA = 0x12340000, right //set value the PFN and shift PA = 0x23450000 //set mtc0 $2 t1, li $3 zero, mtc0 0x00007FFF t2, li $5 ASID VA and t2, with EntryHi //write mtc0 5:0 bits into EntryLo field CDVG //insert tlbwi 4 t3, $0 t3, = 5,1,1,1 CDVG //load KB to 16 page size //set page even EntryLo page, invalidate //write odd EntryLo //write register PageMask //write register Index //write 4 entry VTLB //select boundary between the wired and the wired boundary between cannot be overwrittencannot be a by The wired entries in the VTLB must be contiguous and start from 0. For example, if the can be wired. The can be wired. open for Guest random open for Guest replacement. zero by a Reset exception. a Reset zero by the core, I6400 the in that Note CP0 Programming Interface CP0 Programming The I6400 core allows up to 15 entries of the VTLB to be hardwired such that they cannot be replaced by a TLBWR instruction. This is Programming a TLB Entry Code Example Code Entry a TLB Programming The following assemble language example shows how to create a single mapping virtual from address 0x12340000 to andthespecific value.toset entry G bits of a physical 0x23450000 address withV, C,the D, randomization for replacingRoot wired entries. Example Code a TLB Entry Hardwiring shows the assembly The following example first the 5 entries of VTLB are wired. register is greater than contains a value of 0x5, this indicates that entries 4, 3, 4, this indicates that entries 0x5, of value contains a Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 2.2.7 Hardwiring VTLB Entries 28 MIPS64® I6400 Multiprocessing Syst

y to index the entry. The indexy toentry. the mode does notmode have permission does tion is taken. This scheme is used register must be consistent register with set the FTLB Index size to sizea particular index main-in order to FTLB set kernel, user, and supervisorand modes. kernel, user, bit is 1, hashingbit is 1, is ignored and indexing the are entries register matches the page size the FTLB currently sup- matches the page size the FTLB register FTLB set and choose FTLB set and choose a random wa its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its EHINV Pagemask exception is taken when supervisor exception is taken when supervisor modehave does not permission EntryHi ere is a TLB hit, but the valid bit for that TLB entry is not set. any mode, there is a TLB miss. a TLB any mode, there is registers. If not, a machine check excep AdES AdEL exceptionwhen is taken user mode havepermission does not for the exceptionis takenmode does when user notthe have permission for eithertheVTLB the or the FTLB, the I6400 followingfor core allows types of and the TLBWIand the Instruction AdES AdEL PageMask and bit is 0. When the 0. When bit is EHINV exceptions(TLBL TLBS) are taken under andfollowing the conditions. EntryHi exceptions (AdEL in are used and AdES) exception (TLBL, TLBS) is taken on any TLB miss regardless of the operating mode and uses the fol- EntryHi bit is ignored for a TLBWR bit is ignoredfor instruction. for the address being accessed. being accessed. for the address being accessed. for the address address being accessed. accessed. being address accessed. being address EHINV TLB Invalidate Address Error Address • TLBLnon-store, th exception: On a •instore supervisor On a an mode, • TLBS exception:storeOnis a TLB hit, but in any mode, a there the validfor that bitset. TLB entry is not • mode, an user in a store On •supervisorin On a load an mode, • a TLB miss. is store in any mode, there TLBSOn a exception: •in TLBL non-store exception: On a • mode, an user in a load On • Addresserror (AdELAdES) or • TLBS) TLB Refill (TLBL, • TLB Invalidate TLBS) (TLBL, • TLBRead Inhibit (TLBRI) • TLB Execute Inhibit (TLBXI) • TLB Modified (TLBM) • Parity FTLB • Machine Check TLB Refill The The lowing TLBL and opcodes. TLBS ports, hardware scheme to uses the hashing calculate the TLB exceptions. The EntryHi occurs in TLB miss a the event that In tain consistency. the the FTLB, is executed targeting a TLBWI When instruction calculated from the The I6400 core uses a hashing scheme based on VPN and page and VPN on based scheme uses hashing a core The I6400 invalidated. the a TLBWRand When instruction is executed only when the Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 2.2.8 FTLB Hashing Scheme MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 29 2.3 Handler Exception TLB

register. register. bit is not EHINV BadInstr capture address infor- EntryHi meof the information ssociated with that entry with ssociated ructure that stores virtual- ster that captures the most the ster that captures register that captures the the captures that register the FTLB page size setting eared, FTLB parity errors are errors eared, FTLB parity ster containing a pointer to an the case in the I6400 core. I6400 in the case the . The FTLB parity exception is . The FTLB parity exception lting instructionis ina branch the case in the I6400 core. use with the XTLB Refill handler, onsistency. The machine check onsistency. the pagethe table entry (PTE) array. register is provided to accelera- allow em Programmer’s Guide, Revision 1.00 register does not it during a load operation, the RI bit of the register duplicates so register duplicates BadInstr register is a read-only BadVAddr a a store and the Dirty bit companion document providedthe in documenta- ) is set. If thiscl bit is register is a read-only regi is register PE . an operating system data st system an operating ster does not correspond to XContext ). register is usedin register conjunction with the register is a read/write regi P ErrCtl BadVAddr its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its bits are set, whichbits is always are set, BadInstr XIE bits are set, bits are set, is always which CP0 Registers anch instruction, when the fau register is only set by exceptionsregister which are synchronous to an cessor detects an internal inc ity error occurs on an FTLB read ity error XContext BadInstr RIE FTLB Page Size Page FTLB register ( ntainspointer the an entry to in register is primarily intended for ap is detected across the FTLB/VTLB. across ap is detected e they are not addressing errors. addressing e they are not BadInstr PageGrain the exception to occur. The occur. the exception to Config4 registersmanage toexceptions. TLB PageGrain at caused the exception. at caused The and XContext IEC Error Control Error and IEC exception (TLBXI) is taken when there is a TLB hit during an instruction fetch, the XI bit of PageGrain register contains the prior br exception (TLBRI) is taken when (TLBRI)exception there is taken is a TLB h P exception occurs when the pro when the occurs exception PageGrain (CP08,The Select 0):register 64-bit (CP0 register 8, Select 2): The (CP0 The 20, Select 0): register (CP0 register 8, Select 1): The 64-bit exception is taken whenever a par exception is taken whenever (CP0 register0): 4, Select Co exception is taken whenever there is a TLB hit on hit TLB is a whenever there is taken exception BadInstr delay slot. delay recent instruction which caused recent instruction which caused entry in the page table entry (PTE) array. This array is This table entry (PTE) array. page entry in the to-physical translations.The The TLB Refill. on a also loaded by hardware is but instruction. most recent virtual address th BadInstr BadInstrP BadVAddr Context XContext mation for cache or bus errors, sinc The tion of instructionemulation. The provided in the BadVAddr register. provided in the BadVAddr Machine Check TLB Read TLB Read Inhibit • • • set. 12:8in bitsof the Config4 register ( • • TLB Execute Inhibit TLB Execute FTLB Parity Parity FTLB TLB Modified exception can be either precise or imprecise depending on the type of error. The followingThe conditionsa cause exceptionor can be either imprecise precise depending on the type error. of machine check exception: • A TLBWI instructionandthe to the index FTLB and VPN2 are not consistentwhen the is not set. An taken only when bit 31 of the CP0 CP0 the of 31 bit when only taken entry is set, and the ignored. The tion package. For more informationon these registers, refer to the A The • TLBWI instruction A to the FTLB and the PageMask regi •a duplicate/overl TLBP instruction and A • lookupTLB Anya duplicate/overlap and the is detected FTLB/VTLB. across Interface Register CP0 core uses the following The I6400 the entry is set, and the set, and the entry is The Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 30 MIPS64® I6400 Multiprocessing Syst

register so register so EntryHi 64-bit addressing modes, and XContext register instead of XContext register this chapterand provides multi-threading considerations,guest ly languagely implementation a TLB of register as the memory address to read the the to read the memory address register as register and their own scheme to access the correct to access scheme own register and their translation flowillustrate to theunder circumstances serves as a supplement to as serves Context example needs to read the example needs to read w to select between 32- and its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its BadVaddr Virtual PageNumber (VPN) that missedto the Virtual , supervisor, user, and debug and operating modes. user, supervisor, , rity errors, address error detection, The assemb example shows the following ly usely value the CP0 the in I6400 Technical Reference Manual Reference I6400 Technical dmfc0 C0_XCONTEXT k1, 20) register (CP0 register XContext // Get .set noreorder .set #define#define C0_ENTRYLO0#define C0_ENTRYLO1 $2,0 #define C0_CONTEXT $3,0 C0_XCONTEXT $4,0 $20,0 dmfc0 C0_CONTEXT k1, ldld 4) register (CP0 register Context // Get dmtc0 0(k1) k0, C0_ENTRYLO0 k0, dmtc0 8(k1) k1, 2) register (CP0 CP0 EntryLo0 k0 to // Move C0_ENTRYLO1 k0, ehb 3) register (CP0 CP0 EntryLo1 k0 to // Move tlbwreret K0 into EntryLo0 // Load k1 into EntryLo1 // Load effect write takes CP0 to insure barrier hazard entry // Clear random TLB to // Write TLB exception from // Return TLBmiss32: additional information about the MMU, includingphysicalto an overviewof virtualtranslationaddress with4 KByte,KByte, 16 and 64 KByteexamples, page size address which TLB exceptions are taken,FTLB pa the Context register: the Context Refer to the Linux OS documentationpage for detailstable entry. on the page table handling. The MMU chapter of the and root operating systems, and an in depth discussion of ho Note that some operating systems like Linux use a 3-level Page Table and do not use the Contextthe and do not use or XContext regis- Notethat some operating like systems Linux use a 3-level PageTable ters for table page lookup.they Instead use the CP0 Foraddressing 64-bit modethe first instruction above in the EntryLo0/1also settings.The processor writes the kernel in the address mapping the associated TLB Exception Handler Code Code Example Handler Exception TLB direct handler can exception The it is ready to write the TLB entry. to write the TLBentry. it is ready 32-bitexceptionfor handler addressing mode. Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 31 2.4 Information Additional em Programmer’s Guide, Revision 1.00 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 32 MIPS64® I6400 Multiprocessing Syst

3 that go into program- go into that V P . The L1 instruction and exceptions. well as cache initialization well as cache and instructions, hence the and instructions, hence 2 V P CPU n 1 V P ndling cache ndling cache much main mem- faster than accessing ion, L1 data, and shared L2. These and data, ion, L1 L1 Data Cache Data L1 L1 Instruction Cache 0 V P scription of the elements scription I6400 Multiprocessing System to each cache is provided, as e L2 cache contains both data e L2 cache contains fastest access times and are accessed first. If the data is not accessed first. If the are and times access fastest its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its g up cache coherency and ha g up cache coherency Shared L2 Cache L2 Shared same core. The L2 cache is shared by all cores. is The L2 cache core. same in the L2 cache, the main memory is accessed. memory is accessed. main in the L2 cache, the of information that can be retrieved be of information that retrieved can Coherence(CM3) Manager ains the following caches: L1 instruct ains the cache architecture and a de the cache architecture and a 3 V P cache can be configured as follows: can be configured cache 2 V P Figure 3.1Figure Caches System Multiprocessing I6400 CPU 0 1 V P L1 Data Cache L1 Instruction CacheL1 Instruction 0 V P shows location of the relative the caches within the code. Other programmable elements include settin code. Other programmable elements include ming the caches. A description of the CP0 register interface interface of the CP0 register description A caches. ming the ory. The dedicated L1 instruction and data caches have the and data caches The dedicated L1 instruction ory. Th the L1 cache, the shared L2 cache is accessed. in present not is If the requested data ‘shared’. name of This chapter provides an overview caches provide on-chip caches provide on-chip temporary storage in the VP’s shared by all are L1 data caches In the I6400 MPS, the of size each • L1 InstructionKB32or Cache: 64 KB • L1KB Data Cache: 32or 64 KB • L2KB, Cache: 256or 8 MB KB, 5121 MB, 2 MB, 4 MB, Figure 3.1 The I6400 Multiprocessing System cont System Multiprocessing The I6400 Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 33 3.1 Subsystem Overview Cache Caches Chapter 3 Chapter

the line are Coherence Coherence lly indexed and physi- lly indexed t associativity of the cache. of the t associativity cache. of the t associativity tion Code (ECC) bits. . Allin bytes 64 ndexed and physically tagged, ndexed and physically the line, for a total of 64 bytes. for a total of 64 the line, Total L2 tion of the words in that line depend- larger than the L1 caches. In the L1 caches. than the larger Cache Size dress space. Refer to the dress space. is used when the cache size is 256 KB. em Programmer’s Guide, Revision 1.00 re. Software can check the set size by re. Software can check the struction cache is is struction cache virtua e error checking and correction process is handled process and correction e error checking . ECC code is generated across a 32-bit word. Sub- ss bits and 7 Error Correc ss bits and 7 Error ze is fixed at 64 bytes. The number of sets and ways is ze is fixed at 64 bytes. The number of sets Data is physically i cache data and associated ECC bits corresponding corresponding to the 4-way se corresponding to the 4-way se into the Coherence Manager (CM3.5). The L2 communi-(CM3.5). The L2 into the Coherence Manager Ways L2 communicates with the cores through the proprietary its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its Number of of the CM3.5 register ad register of the CM3.5 physical address (depending physical state on cache size), two coherent coherent state bits (2) stored with the tag. (2) coherent state bits try contains eight 64-bit doublewords in doublewords eight 64-bit try contains miss in the L1 caches. The L2 cache is L2 cache The caches. the L1 in miss either 8 or 16 ways. The 8-way option bit errors can be corrected -modify-write sequence. Th -modify-write sequence. of physical addre 36-bit e, a store operation can update all or a por Way cannot be changedkernel softwa by the Sets per Sets e L2 cache configurations. Table 3.1Table L2 Cache Configurations contains two arrays: tag and data. The L1 in L1 tag and data. The two arrays: contains 64 bytes64 bytes64 bytes64 bytes64 bytes64 512 bytes64 512 1024 2048 4096 8192 8 16 16 16 16 16 KBytes 256 KBytes 512 1 MByte MBytes 2 MBytes 4 MBytes 8 Line Size shows the list of possibl the list shows chapter for more information. After a valid line cach is resident in the present in the data array together, hence the hence the together, array in the data present store. of type the on ing The data cache uses ECC so that single- by doing a read handled are word stores entirelyby hardware and is transparent to kernel software. cates with external memory via an AXI-4 interface. The MIPS Coherence Protocol (MCP) bus. The associativity of the L2 cache can be I6400 Multiprocessingcache is integrated the L2 System, for all other cache sizes. The line si The 16-way option is used The L2 cache processes that transactions selected during the build process and selected during the build process cally tagged. An instruction cache data en cache data An instruction tagged. cally The L1 data cache tag and contains two arrays: L1 data. The The L1 instruction cache The L1 instruction The tag and data arrays hold 4 ways of information per set, hold 4 ways of information The tag and data arrays a virtual aliasing. eliminating the chance of thus per set, hold 4 ways of information The tag and data arrays An cache instruction consists tag entry the 35 of the upperA tag entry consists bits of 34 or bits, and someECC bits. entry A data containsof 64 bytes reading the GCR_L2_CONFIG register, which is part GCR_L2_CONFIGreading the register, Manager Table 3.1 Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 3.1.3 L2 Cache 3.1.2 Cache L1 Data 3.1.1 Cache Instruction L1 34 MIPS64® I6400 Multiprocessing Syst

Cache Cache dirty, set the state of state the set dirty, ration is completed, column: alization. Note that Index alization. program performance. PREF program performance. Cache all instruction caches in a cluster. all instruction caches in a cluster. MPS does not globalize the not globalize the MPS does on cache line. This instruction should e tag. After that ope e line is valid but not valid e line is g instructions: ied index to invalid. ied column. software to invalidate the entire data cache by by cache data entire the to invalidate software software to invalidate the entire instruction instruction entire the to invalidate software

invalidate all L1 instruction caches in the sys- caches in the L1 instruction invalidate all Code kernel kernel ations ations and data on the L1 instruction and the caches ces, except during cache initi Invalidate instructions; these instructions only affect ormed usingormed the CACHE instruction. In this table, bits from the cache, to improve from the cache, to improve using the followin the using e line at the specif at the e line . em. Note that the I6400 Note em. ing accessed as shown in the in shown as ing accessed its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its 0:18] of the CACHE Instruction Table 3.2 400 and can be used to 400 and can be used a data cache line with an instructi tions, including TLB exceptions. exceptions. including TLB tions, ache. The SYNCI instruction operates on The SYNCI instruction operates ache. instructions are "globalized", which means that they will invalidate the tar- the invalidate will they that means which "globalized", are instructions If the state of the cache line at the specified index is valid and dirty, write the line back back line the write dirty, and is valid index specified the at line cache of the state If the cach by the specified address to the memory th cache line to invalid. If the the state of set invalid. to line the This encoding may be used by cache by stepping through all valid indices. through stepping cache by stepping through all valid indi stepping through should be used to initialize the cache at power-up. Tag Store For the L2 cache, this operation will modify the L1 data caches as needed to maintain inclusivity. This encoding may be This encoding used by kernel y cache — Bits [17:16] = 2’b11 = 2’b11 [17:16] y cache — Bits cache — Bits [17:16] = 2’b00 cache — Bits at executed the instruction. ions in are described on indicate the type of cache be of type on indicate the d on the L1I, L1D, and L2 caches L1D, L1I, the d on . Invalidate Table 3.2Table [2 EncodingBits of CACHE I Hit Invalidate CACHE I Hit — This instruction is used toperform various oper — This instruction synchronizes — This instruction — This instruction — This instruction is new to the I6 and — This instruction causes data to be moved causes — This instruction to data or shows the variousof types operations that can be perf D, S Index Writeback SYNCI be used when writingbe used whenthe to program imagein memory to makenewly the stored instructionopcodes visible to logic via the I-C the instruction fetch CACHE SYNCI GINVI L2 cache. These operat These L2 cache. causedoes not addressing-related excep tem. PREF Bits 17:16 of the instructi 17:16 Bits • D indicates L1 data cache — Bits [17:16] = 2’b01 • L2 or secondar indicates S • indicates L1 instruction I CACHE D Hit Invalidate, D Hit Writeback, or D Hit Writeback CACHE D Hit Invalidate, or D Hit Writeback D Hit Writeback, th the L1 D-Cacheof the core Initialization Routines 20:18 of the instruction encode the type of operation as shown in the For more informationhow onto these instructions the examplesection are used, refer the in entitled Table 3.2 geted cache line from all L1 instruction caches in the syst in the caches L1 instruction from all cache line geted • • • The • Operations are performe Operations Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 3.1.4 CacheInstructions Code Cache Name Operation 3’b000 IInvalidate Index cach the of the state Set MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 35

regis- regis- DTagLo DTagLo ITagLo IDataHi register. Lo register be ini- and the ECC bits tag are ITagLo CP0 DTagLo register and DTagLo CP0 rresponding bits are set in the the bits are set in rresponding y and dword index specified. y and dword index IDataLo (continued) taHi registers usedare instead of the ataHi registers are used instead of the of the instead used are registers ataHi requires that the ITag em Programmer’s Guide, Revision 1.00 index into the CM GCR_L2_DATA register index into the GCR_L2_DATA CM ware to initialize the entire data cache by cache data entire the initialize to ware DDataHi register and DDataHi register IDataHi register, and the tag ECC bits are IDataHi register, index into the DDataLo register. index into the DDataLo register. register. e specified index into the the into index specified e s. Doing so requires that the DTagLo register be ini- s. Doing so requires that the DTagLo register are used if the co are used if the register ted in CM address space. address ted in CM register contents at the wa the at contents register IDataLo e line at the specified index from the CM ne at the specified index into the CM the into index the ne specified at the cache line at the specified index from the the from the cache line at the specified index register at offset 0x0600. register at offset register at offset 0x0600. 0x0600. register at offset its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its GCR_L2_ECC registers. nd data for the cache line at the specified index from the CP0 the CP0 from index the specified cache line at the for nd data GCR_L2_DATA 18] of the CACHE Instruction The hardware automatically generates the ECC bits to write into the cache. For test For test cache. the into write to bits ECC the generates automatically hardware The and DDa purposes, the ECC bits from the DTagLo bit is set. when the ErrCtl.PO values generated automatically kernel soft by may be used operation This all valid indice stepping through register loca L2_CONFIG GCR automatically generated values when the ErrCtl.PO bit is set. when the ErrCtl.PO values generated automatically cache instruction entire the initialize to software by kernel be used may operation This so Doing all valid indices. through by stepping first. tialized and DDataLow first. tialized CGR_L2_TAG_ADDR The hardware automatically generates the ECC bits to write into the cache. For test For test cache. the into write to bits ECC the generates automatically hardware The and ID purposes, the ECC bits from the ITagLo the ECC purposes, For test calculated. is automatically value ECC the tag By default, bitsfrom the CM The ECC bits are by the generated always hardware. ter. The data comes from the the from comes data The ter. read into the CP0 DTagLo register. DTagLo the CP0 read into CGR_L2_TAG_ADDR 0x0610. at offset Read the data to the dword corresponding Read the data corresponding Read the data corresponding to dword index into the the ters. to the CP0 bits are stored The ECC data register. storedCP0 to the ITagLo to the word read data corresponding the CP0 the into read are bits ECC data The Table 3.2Table [20: of Bits Encoding S Tag Index Store cach L2 the tag for the Write S Data Index Store CM the Write S Tag Load Index li cache the for tag the Read D Tag Index Store tag a the Write D Index Load Tag Read the tag for the cache line at th Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies Code Cache Name Operation 3’b011 I, D Reserved Executed as a no-op. 3’b010 I Index Store Tag for (and data) tag the Write 3’b001 I Tag Index Load the into index the specified cache line at the tag for the Read 36 MIPS64® I6400 Multiprocessing Syst

te instruction. nd dirty, write the write nd dirty, the cache line to invalid. cache the t Writeback instruction. t Writeback range of addresses from of addresses from range range of addresses from from of addresses range range of addresses from of addresses from range Hit Writeback Invalidate Invalidate Hit Writeback valid and dirty, write write the valid and dirty, CHE D Hit Invalida (continued) s range by the line size s range the cache. of range by the line size of the cache. size of line the range by range by the line size of the cache. size of line the range by ess and it is valid a is valid it ess and ready in the cache. In that case, the existing the existing In that case, cache. the in ready caches, meaning that when executed, the the executed, when that meaning caches, software to invalidate a a invalidate to software software to invalidate a a invalidate to software a invalidate to software

d address, set the state of state set the address, d not globalize the CACHE D Hi the CACHE D not globalize not globalize the CA the not globalize specified address is it address and specified s not globalize the CACHE D the s not globalize its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its 18] of the CACHE Instruction contents back to memory. After the operation is completed, leave the state of the line line of the state the leave completed, is the operation After to memory. back contents state. the dirty valid, but clear MPS does the I6400 Note that instruction. This instruction only affects the L1 D-Cache of the core that executed the instruction. For the L2 cache, this operation will modify the L1 data caches as needed to maintain inclusivity. the instruction. executed that core of the D-Cache the L1 only affects instruction This For the L2 cache, this operation will modify the L1 data caches as needed to maintain inclusivity. the data cache by stepping through the addres stepping through by the data cache MPS doe the I6400 Note that This instruction is This instruction globalized for the I in the caches instruction all L1 line from cache the targeted invalidate will instruction system. For the L2 cache, this operation will modify the L1 data caches as needed to maintain inclusivity. kernel be used by operation may This the caches by stepping through the address the address through by stepping the caches This operation may be used by kernel be used by operation may This the caches by stepping through the address the address through by stepping the caches Note that the I6400 MPS does the I6400 Note that If the cache line contains the specified addr the specified line contains cache If the copy in the cache is invalidated cache is the in copy This instruction only affects the L1 D-Cache of the core that executed the instruction. the instruction. executed that core of the D-Cache the L1 only affects instruction This The cachelineis refetched even is if it al contents back to memory. After that operation is completed, set the state of the cache cache of the state the set completed, is operation that After to memory. back contents invalid. to line the of state the set dirty, not but is valid line If the invalid. to line kernel be used by operation may This Invalidate Table 3.2Table [20: of Bits Encoding D Hit Invalidate invalid. to line cache of the state the set address, specified the contains line cache If the D, S HitWriteBack Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies Code Cache Name Operation 3’b110 D, S WriteBack Hit the line contains cache If the 3’b100 I, S Hit Invalidate specifie line contains the If the cache 3’b101 I Fill Fill the cache from the specified address. MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 37

L2 field of the GCR the GCR field of Writeback Invalidate, Writeback L1 instruction L1 instruction data and : Use coherent data. Load : Use coherent data. STATE STATE from the cache. Stores to cache. Stores the from (continued) ration to the locked line, or via an Index via an Index line, or the locked to ration em Programmer’s Guide, Revision 1.00 er for more efficient bus utilization. bus efficient more er for ches this as operation executes a no-op. ndex Invalidate, Index ndex Invalidate, Index e specified address, fill it from memory and from memory fill it address, e specified is not in the supported I6400 field. register. Attribute ed line in the cache, the line ed line in the cache, is updated to the exclusive . (CCA). The cache coherency can be set in one of three cache coherency can be set in one of The (CCA). ys at a given cache ys at a given index. its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ndicated as uncached are not read are not uncached ndicated as Config.K0 Table 3.3 18] of the CACHE Instruction in memory, without changing cache contents. without in memory, in other L1 data caches are invalidated. L1 data other in ed state. Stores bring data into the cache in an exclusive state - no other - no other state bring data into the cache in an exclusive Stores ed state. e (willnotthe get exclusive being data is if anotherby shared Multi- CPU). Uncached stores are gathered togeth Uncached caches. For the L1 instruction and data ca caches. For the If the locked. and to valid the state Set replaced. being line from the data the writeback cache alreadycontains specified theaddress, the set state locked.to The way selected used. recently least is the memory from on fill I an by executing cleared is state lock The RAM Cache Op Address Tag wa all lock to is illegal It Hit Invalidate, or Hit Writeback Invalidate ope Hit Invalidate, or Hit Writeback operation with the lock bit reset in theassociated Tag Store llowing cacheability attributes: attributes is shownin : Addresses in a memory area i area in a memory Addresses : 1 Table 3.2Table [20: of Bits Encoding 3’b0003’b001#5). (code Mapped to ‘3b101 #5). (code Mapped to ‘3b101 K[2:0] L2 Fetch and Lock If the L2 not contain cache does th such addresses are written directlyma to such addresses are written caches can that line. If a contain same store hits on a shar line the shared copies of state and any Cacheable, coherent, write-back, write-allocate, read misses request shared. (code #5) shared. misses request write-back, write-allocate, read Cacheable, coherent, misses request data in the shared stat request misses shar contain data in the ple caches can Uncached (code #2) Uncached (code #7): (code Uncached Accelerated Table 3.3Table the K0 Field in the CP0 Config Register for Attributes of Cacheability Mapping ways: •set using the CP0 coherency is KSEG0 space, In • The I6400 core defines a set of Cache Coherency Attributes of Cache Coherency set defines a core The I6400 • Usingthe entry TLB for mapped address regions. •segments. the XKPHYS memory Using the fo core supports The I6400 • • The mapping of cacheability The mapping of cacheability Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies Code Cache Name Operation 3’b111 I, D Fetch and Lock The and Fetch Lock encoding 38 MIPS64® I6400 Multiprocessing Syst 3.2 Coherency Cache

(continued) D-Cache before attempting accommodate the Harvard iteback from the L1 D-Cache. iteback from the from all I-Caches in the system. The system. from all I-Caches in the ate CACHE or GINVI instruction (when ad misses request shared. e block ofe blockit instructions invalidates as entire the I- Attribute have been written to the L1 the I6400 core fetches the latestfetches the the I6400coherent core data from the run-time, software must run-time, software e is to e is no need force a wr instruction could be replaced with JALR.HB, ERET, or instruction could be replaced with JALR.HB, ERET, led after the new instruction stream is written to make those s. When using cacheable memory accesses (CCA = 3 or (CCA = memory accesses cacheable using s. When its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its be accomplished by the SYNC and JALR.HB or ERET by the SYNC and JALR.HB be accomplished for the K0 Field in the CP0 Config Register ite-back, write-allocate, re ite-back, write-allocate, YNC instruction is required between the final SYNCI instruction in the loop n could be replaced with an appropri ts 5:3 of the CP0 EntryLo0 and EntryLo1 registers. EntryLo1 and CP0 EntryLo0 the ts 5:3 of th new instructions at th new instructions instruction stream effective to the stream instruction effective r the instruction stream is written. ich means they invalidate the targeted line the targeted invalidate they ich means instructions can be used for this purpose. The SYNCI and CACHE I Hit Invalidate routine which can be cal lable), and that the JR.HB and that the lable), clears instruction hazards. instruction clears 1 3’b111 Accelerated. Uncached 3’b110#5). (code Mapped to ‘3b101 3’b011 #5). to ‘3b101 (code Mapped Cacheable. 3’b1003’b101#5). (code ‘3b101 Mapped to coherent, Cacheable, wr 3’b010 Uncached. K[2:0] to fetch and execute the new instructions. This can This new instructions. the to fetch and execute instructions. GINVI instruction can be morewhen efficient writing a larg instruction. single Cache with a I Index Invalidate or GINVI wh instructions are globalized, The following example shows a changes effective. The SYNCI instructio changes effective. A S appropriate. instructions, as DERET and the instruction that /* * This routinechangesmakes to the * hardware.should Itafte be called effective. are instructions new * On return, the * * Inputs; newinstruction address of stream Start * a0 = in bytes,Size, of newinstruction* a1 = stream */ access access to Coprocessor 0 is avai L1 D- and L2-Caches (includingcores)L1 D- and L2-Caches other fromther so Note that unlike some other MIPS cores, onthatNote I-Cache an miss unlikeMIPS cores, other some 2. The processor mustwait untilof all the newinstructions When the processor writes memory wi When the processor CCA = 5), the followingsteps must prevent be taken to the execution previous of (stale) contentsmodified of the memory addresses: 1. Any stale instructions must be invalidated from the L1 I-Cache. The SYNCI, CACHE I Hit Invalidate, CACHE architecture and write-back policy of the I6400 L1 cache architecture and write-back policy of 1. also mapped This field is to the C field in bi Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies Table 3.3Table Mapping of Cacheability Attributes MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 39 3.3 Self-modified Code

EG0 memory region. em Programmer’s Guide, Revision 1.00 as number of sets, line size, and cache size, line sets, number of as L1 instruction and data caches, and the L2 r the unmapped KS r the ion cache operations. its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its gisters for instruct gisters to obtain the L1 cache parameters such L1 cache parameters to obtain the es require synchronization, */ synchronization, require es eld contains the cache attributes fo attributes the cache eld contains current with end address */ current with end address tion on the CP0 used registers to manage the Config Register 16, Select 0 ITagLo Register 28, Select 0 IDataHi 1 Select 29, Register Config1 Register 16, Select1 IDataLo Register 28, Select 1 CacheErr Register 27, Select0 Table 3.4Table Interface Register Cache Instruction ache Control Registers CP0Registers Register CP0 Number nop+ 1 */ address end Calculate a0, a1/* daddu a1, Get step for SYNCI size */ v0, rdhwr HW_SYNCI_Step/* cach no 20f /* If beq v0, zero, nop daddu a0, a0, v0/*slot size in delay Add step */ brancharound /* */ v1, a0, a1/* Compare sltu bne v1, zero, 10b/* if more Branch do to */ nopsync around /* branch */ nop /* branch around*/ hazards */ memory /* Clear In the Config register - the K0 fi software kernel allows This register The I6400 core uses the following CP0 re core uses the following The I6400 This section provides informa cache. size==0,zero, 20f/* If beq a1, */ */ Synchronize all caches around address synci 0(a0)/* 10: rajr.hb 20: Return, /* clearing instructionhazards */ associativity. 3.4.1.1 0) Select 16, (CP0 register Register Config 3.4.1.2 Config1Register register (CP0 16, Select 1) Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 3.4.1 Instruction C L1 40 MIPS64® I6400 Multiprocessing Syst 3.4 Interface Register

L1 instruction ation (calculated by gister also stores the also stores gister e. The I6400 L1 instruc- e. The I6400 e the associated instruction pre- instruction e the associated occurred. This occurred. This register provides n cache. The I6400 /written by the CACHE load tag/store by /written cache. The cache. The I6400 L1 cache instruction sup- struction precode inform struction precode configuration and are built into the core. r a 32-KByte cache). The re 32-KByte cache). r a cache error that cache r way in the instruction cach instruction in the way r r bits indicate that the tag entry is valid, or if an ECC its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its register described abovestor to L1D tag/data. FTLB tag/data, or L2 tag/data set associativity for the instructio associativity set is used to configure as a 32 KB cache, or 256 sets per way, which used is orper 256a 32 cache, KB configureis used to way, sets as IDataLo indicated by a value of 3 for this field. a value of 3 for this by indicated te cache, and bits 47:13 fo and te cache, g information of a cache line being read line cache a of g information 0 registers for data cache operations. 0 registers tion opcodes)read/written being CACHEthe bytag load tag/store operations. ndicates the line size for the instruction the instruction size for line the ndicates ) are needed to store both the data and in Table 3.5Table Interface Register Cache Data Config Register 16, Select 0 Config1 Register 16, Select 1 DTagLo Register 28, Select 3 CacheErr Register 27, Select 0 y; their values are determined during IP y; their values are determined CP0 Registers CP0 number IDataLo , IDataHi field (bits field (bits 18:16) indicates the field (bits 24:22) indicates the number of sets pe of sets number the indicates 24:22) (bits field containsregister information regarding the type of field (bits 21:19) i 21:19) field (bits IL IA IS . . . CacheErr Config1 Config1 Config1 This register stores the cache address ta the cache stores register This information such as: •error Correctableuncorrectable or • L1I tag/data, the error occurred; where Array • The way where the error cache was detected This register works in conjunction with the hardware unless overridden). This register stores the 64 bits of the load data. code bits, error information, and ECC status. cache is fixed at 4-way set as associative read-onl are all fields These a 64-KBy47:14 for tag operations (bits The tion cache supports 128 sets per way, which tion cache supports 128way, sets per a 64 KB cache. to configure The The I6400 core uses the following CP core uses the following The I6400 The ports a fixed line size ofas indicated 64 bytesoflinea valuefixed by field. 5 for this size ports a The Two registers( Two This registerthe holds data (instruc error has occurred. ECC bits associated with the tag entry. Separate Valid and Erro Separate Valid ECC bits associated with the tag entry. 3.4.1.4 0) Select 28, register (CP0 Register TagLo Cache Instruction L1 3.4.1.6 L1 Instruction Cache Register DataHi (CP0 register 29,Select 1) 3.4.1.3 CacheErr Register (CP0 register 27, Select 0) 3.4.1.5 L1 Instruction Cache Register DataLo (CP0 register 28, Select 1) Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 3.4.2 Cache Control Registers Data L1 MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 41

or to the cache line. This t indicates that an ECC error that an t indicates EG0 memory region. occurred. This occurred. This register provides yte cache). The stores register also em Programmer’s Guide, Revision 1.00 ng read/written by the CACHE load tag/ the CACHE ng read/written by (continued) ng information about the L1 data cache. ng information about configuration and are built into the core. r the unmapped KS r the cache error that cache way in the data cache. The I6400 L1 data cache sup- cache data I6400 L1 The data cache. the in way ta cache. The I6400 L1 data cache supports a fixed cache supports L1 data The I6400 ta cache. the data cache. The I6400 L1 data cache is fixed at 4- cache.at is fixedcache The I6400the data L1 data g entry. An error flag bi flag An error g entry. h loads or stores data from data or stores h loads its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its L1D tag/data. FTLB tag/data, or L2 tag/data to a 32 KB or 64 KB cache, respectively. KBto a 32 instruction, whic KByte cache, and bits 47:13 for a 32-KB ess tag information of a cache line bei el software to obtain the followi to obtain the el software CACHE eld contains the cache attributes fo attributes the cache eld contains by a value ofa valueby 3 for this field. DDataHi Register 29, Select 3 s s in the line L1 data cache data caused the error DDataLo Register 28, Select 3 y; their values are determined during IP bits of thedata. load bits of CP0 Registers CP0 number Table 3.5Table Interface Register Cache Data field (bits 9:7) indicates for the set associativity field (bits 15:13) indicates the number of sets per field (bits 12:10) indicates the line size for the da indicates field (bits 12:10) register contains information regarding the type of DL DA DS . . . CacheErr Config1 Config1 Config1 register stores the lower 32 the lower register stores This is a staging register for a special This is the coherence state (MESI) and ECC bits associated with the ta with bits associated (MESI) and ECC the coherence state was detected by the load index tag operation. This register stores the data cache addr This register information such as: •error Correctable or uncorrectable • L1I tag/data, the error occurred; where Array • Fatal or non-fatal error • The way where the error cache was detected •word four Which one of store tag operations (bits 47:14 for a 64- (bits tag operations store way set associative as indicated way set associative read-onl are all fields These The ports 128 or 256 sets per way, which corresponds ports 128 or 256 sets per way, The The In the Config register - the K0 fi Config1CP0 register allows kern The line size of 64indicated bytesthissize of line field. as for 5 by a value of The 3.4.2.5 3) Select 28, (CP0 register Register DataLo Cache Data L1 3.4.2.4 Registerregister (CP0 28, Select 2) Cache TagLo Data L1 3.4.2.3 CacheErr Register (CP0 register 27, Select 0) 3.4.2.1 0) Select 16, (CP0 register Register Config 3.4.2.2 Config1Register register (CP0 16, Select 1) Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 42 MIPS64® I6400 Multiprocessing Syst

like the L1 instruction L1 instruction like the or This to the cache line. er contains the following an L2 store cacheop is executed. store cacheop an L2 h do store L2 configuration informa- store L2 do h cache operations. Note that these regis- these that cache operations. Note dicate if the hardware detected an ECC if the hardware dicate not located in CP0 space h loads or stores data from data or stores h loads s that the L2 cache information is stored in a memory- e configuration. This regist that the L2 cache has ECC logic. The L2_ECC_EN The ECC logic. bit L2 cache has that the its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its are set, the contents of the respective TAG_ECC and TAG_ECC respective set, the contents of the are the cache (set size, line size, and associativity). line (set size, the cache instruction, whic instruction, the ECC portion of the L2 RAM when the load data, as well as a bitin toas a wellas the load data, CACHE is unlike most previous MIPS cores whic cores MIPS is unlike most previous chapter for more information on accessing these registers. information on accessing more chapter for ) uses the following GCR for L2 uses registers ) CM es information on the L2 cach the L2 es information on Table 3.6Table Interface GCR L2 Cache Register CR Control Registers CC_SUPPORTED field indicates CC_SUPPORTED address space at the offsets shown. They are shown. They are space at the offsets address CM Coherence Manager Coherence CM L2_CONFIG 0x0130 GCR Global GCR_L2_ECC 0x0618 GCR Global GCR_L2_DATA 0x0610 GCR Global GCR Registers Offset Address Address Space GCR_L2SM_COP 0x0620 GCR Global L2_RAM_CONFIG 0x0240 GCR Global L2_PFT_CONTROL 0x0300 GCR Global CPC_STAT_CONFIG 0x0008 CPC Local GCR_L2_TAG_ADDR 0x0600 GCR Global GCR_L2_TAG_STATE 0x0608 GCR Global GCR_ERR_CONTROL 0x0038 GCR Global L2_PFT_CONTROL_B 0x0308 GCR Global DATA_ECC registers are written into are written registers DATA_ECC GCR_L2SM_TAG_ADDR_COP 0x0628 GCR Global information: • Read-only fields that provide the of organization • L2 bypass mode. •WhenECC write protocol. and data these bits Tag The L2_Config register provid enables ECC. enables In this register, the L2_E this register, In Refer to the and data cache control registers. This This cache control registers. and data tion in theThe CP0 registers. CP0 Config5.L2C indicatefield mapped register instead of CP0. mapped register instead The I6400 Coherency Manager ( register the stores ECC information from operation. error during IndexLoadTag the This is a staging register for a speciala staging register for This is ters are located in are ters 3.4.3.2 0x0130) (Offset Register L2_Config 3.4.3.1 0x0038) (Offset GCR_ERR_CONTROL 3.4.2.6 3) Select 29, register (CP0 Register DataHi Cache Data L1 Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 3.4.3 Cache CM G L2 MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 43

instruction is exe- instruction is bit, the number enable of ations. The state machine ations. ate machine is idle or run- is ate machine ruction is executed. If the ruction is executed. If the operation is complete complete is operation the how the prefetch unit handles how the prefetch em Programmer’s Guide, Revision 1.00 L2 Load Tag CACHE L2 Load Tag L2 Store Data CACHE instruction is exe- CACHE instruction is Data L2 Store ates for the Tag RAM’s, Data RAM’s and Data RAM’s RAM’s, the Tag ates for bit is set then value of the DATA_ECC reg- bit is set then value of the DATA_ECC and global code prefetch enable. and global code prefetch n, and burst oper flush, nimum operatingpage systemsize. it indicates whether the L2 st it indicates y field that indicates when L2 Store Tag CACHE instruction is executed. This reg- This executed. CACHE instruction is Tag L2 Store when the hardware cache initialization is complete. is complete. hardware cache initialization when the This cher. This includes a prefetcher This includes a cher. en a L2 Store Tag CACHE inst Tag en a L2 Store rdware prefetcher, including rdware prefetcher, its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its s portion of the L2 Tag RAM when an L2 Store Tag CACHE Tag RAMwhen an L2 Store s portiontheTag L2 of the L2 Tag and Data RAMs when the L2 Load Tag CACHE Tag Load the L2 Data RAMs and when the L2 Tag L2 dynamic sleep mode, and dynamic L2 sleep wake-up delay. L2 Data RAMs when the L2 Load Data CACHE instruction is CACHE instruction L2 Data RAMs when the L2 Load Data e L2 Tag RAMs and LRU information when the L2 Load Tag Tag Load the L2 RAMs and LRU information when L2 Tag e indicates indicates the mi of this register is written to the tag state information portion of the L2 Tag writtenregister is of thistag to the state informationL2 Tag the portion of bit is set then value of the TAG_ECC register is written to the bit ECC portion is set then value of the TAG_ECC e machine during initializatio e written to the L2 Data RAM when an ds that indicate the number of wait st of number indicate the that ds e Tag CACHE instruction is executed. Tag e address from the L2 Tag RAMs when the address from the L2 Tag L2 prefetching enable per port ID, L2 prefetching enable GCR_L2_CONFIG.COP_DATA_ECC_WE eld, and a LRU state field. ains a 64-bit data field. 64-bit data ains a This register contains three 2-bit fiel three contains This register L2 prefetchers in the system, and a mask field that field mask and a the system, in L2 prefetchers This registerinformation contains on the L2 hardware prefet Way Select RAM’s. Another read-only Another read-only bit is set by hardware Select RAM’s. Way register also contains support for HCI supported/done, instruction is executed. If the instruction is executed. This register is loaded with state information from th This registerwith is loadedECC informationthe from This register controls the L2 cache stat cuted. The value of this registertocuted. is written The value of the addres instruction is executed. The value executed. is CACHE instruction is this register of executed. The value cuted. This register cont writtenis to the ECC portionwh RAMister L2 Data of the GCR_L2_CONFIG.COP_TAG_ECC_WE RAM when a L2 Stor of the L2 Tag This register is loaded with data information from the RAM and the LRU data of the LRU and WS RAMs when an a tag state fi contains ister coherent write invalidate requests, This register contains additional informationL2 ha the on This register is loaded with the tag can be startedand stopped usingL2SM_COP_CMD the L2SM_COP_TYPE fieldThe1:0. in bits field the indicates type of operation tob be performed. The L2SM_COP_MODE ning. The L2SM_COP_RESULT fieldread-onl bits in 8:6 is a The L2SM_COP_RESULT ning. and if errors were encountered. and if 3.4.3.3 0x0240) (Offset Register L2_RAM_Config 3.4.3.4 0x0300) (Offset Register L2_PFT_Control 3.4.3.7 0x0608) (Offset Register L2_TAG_STATE 3.4.3.9 0x0618) (Offset Register L2_DATA_ECC 3.4.3.10 0x0620) (Offset Register L2SM_COP 3.4.3.8 Register (Offset 0x0610) L2_DATA 3.4.3.5 0x0308) (Offset Register L2_PFT_Control_B 3.4.3.6Register (Offset 0x0600) L2_TAG_ADDR Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 44 MIPS64® I6400 Multiprocessing Syst

require that CPC Local Status CPC Local Status llowing conditions llowing conditions where the operation begins. where the ware cache initialization can ware cache the MBIST operation is the com- MBIST operation is register as described in the previ- described as register il the MBIST has completed. il the rdware when the fo rdware ess in the cache using the the cache using ess in the at offset 0x0008CPC CM-local in address at offset L2 Cache Initialization Options the L2_HW_INIT_EN bit in the the L2_HW_INIT_EN ) initializes only the L2 tag array. Manually selected onlyarray. L2 tag the) initializes ber of lines to operate on relative to the starting d indicates the address the d indicates at address iven low, indicating that automatic hardware initializa- low, iven lization does not begin until until not begin lization does y has expired, automatic hard its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its to initialize the L2 cache: L2 the to initialize ialization does not begin unt to set the starting addr starting to set the set beforesethardware initialization can proceed. ther the L2 tag array onlyarray (fast mode), tag the L2 or bothdata andther the tag arrays to be automatically initialized by ha be automatically to ormed is programmedis ormed into the L2SM_COP led, the cache initia alization is enabled by setting alization is stem provides three ways rdware cache initialization on, there are two types: register (CPC_CL_STAT_CONF_REG) located register (CPC_CL_STAT_CONF_REG) begin. the delay has expired, the cache init plete. Even if space. and Configuration tion can proceed. •If it is enab not enabled. MBIST is • has expired. Once The L2 initialization delay this dela •initi hardware cache Automatic The I6400 The MPS I6400 allows for the L2 cache The I6400 Multiprocessing Sy Multiprocessing The I6400 • ha selected Automatically •initialization hardware cache Manually selected • initialization cache Software initializatiFor hardware •only array(fast) L2 Tag • and data arrays (slow) L2 Tag cache initialization (fast modeAutomatically selected hardware hardware cache initialization can initialize ei hardware cache initialization can initialize software(slow mode).For initializationonebeinitializedcan or both by the kernel, arrays depending design on the of the software. of these optionsEach the are described in following subsections. The hardwareThe initializationdescribed operations in the section entitled ous subsection. the L2_HW_INIT_ENbitthis (24) register of is For L2 cache burst operations, this register is used this operations, burst For L2 cache L2SM_COP_START_TAG_ADDR field in bits47:6.fiel This L2SM_COP_START_TAG_ADDR address. The actualaddress. The operation to be perf The L2SM_NUM_LINESThe bitsin field 63:48 indicates the num are met at reset: • The external input(si_cpc_l2_hw_init_inhibit) pin is dr 3.4.3.12 0x0008) (Offset Register CPC_CL_STAT_CONF 3.4.3.11 Register (Offset 0x0628) L2SM_TAG_ADDR_COP Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 3.5.1 Automatic Hardware Cache Initialization MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 45 3.5 Cache L2 Initialization Options

L2 L2 Cache Op register ion to proceed. ion to L2 RAM Configura- L2 initialization process. initialization process. register this register is present. A ‘1’ L2 Cache Op State Machine L2 Cache Op State 0x1 in this field indicates that in 0x1 that this type of initialization is of initialization this type that . r cache initializat em Programmer’s Guide, Revision 1.00 e initialization to be performed using the using performed be e initialization to is starts the L2 cache cache the L2 starts is in GCR address space. can Software poll this bit to ess space to determine the state of the L2 state of the L2 state state the determine space to ess is automatically initialized by hardware. No initializa- initialized automatically is lized by hardware. The user can choose to initialize only iven high,iven indicating automatic thatinitializa- hardware flush the entire L2 cache in one operation, perform the oneperform operation, in cache flush the entire L2 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its L2 Cache Op State Machine Config/Control OpL2 CacheState amminginoffield bits 4:2 the the L2SM_COP_TYPE lected and cannot proceed. lected and cannot proceed. e machine is e machine is idle, in order fo Initializing the Level 2 Cache in GCR address space to determine if complete, hardwarecomplete, sets the HCI_DONE bit in the by Note manually initialized software. register (GCR_L2SM_COP).of A value L2 Cache Op State Machine Config/Control OpL2 CacheState e L2SM_COP_CMDe fieldthebits in 1:0 of re initialization options described above.software The code used to perform flush, burst, and abort operations. and abort flush, burst, register (GCR_L2SM_COP).of A valueindicates 0x0the process is still running. kernel software indicates the type cach of indicates software kernel ious subsection is not ious subsection is not se n in the section entitled n in the ine Config/Control must be 0, indicating the stat the 0, indicating must be register (GCR_L2SM_COP) to a value of 0x1. Th value of 0x1. to a (GCR_L2SM_COP) register State Machine Config/Control State A value of 0x1 indicatesA value the that process completedno with errors. Cache Op State Mach Cache Op State RAMis RAM and Data RAM is initialized.Avalue0x2indicatesfield in this of thatthe both Tag only the Tag initialized. Note that this RAM operation only. is slower than initializing the Tag Config/Control (GCR_L2SM_COP) at offset address 0x0620 in GCR addr 0x0620 in GCR address offset at (GCR_L2SM_COP) machine. This bit tion described in the prev tion described in the (GCR_L2SM_COP) at offset address 0x0620 (GCR_L2SM_COP) at offset in this bitin thisoperationthe flush indicatescache is supported. that register (GCR_L2_RAM_CONFIG) address 0x0240 offset at 4.th setting L2 state machine by the Start 5. fieldthebits in 8:6 of determine the resultinitialization, of the poll the L2SM_COP_RESULT To 3.operationthe Set type of performed to be by progr For manual cache initialization, 2.bit Read the L2SM_COP_MODEthe in This section describes the L2 cache the describes This section onlycanAn L2 flush operation be initiated by software. To following steps: The I6400 MPS allows for the L2 cache to be L2 cache for the MPS allows The I6400 tion determineinitialization the when is complete. following procedure. 1. Read the L2SM_COP_REG_PRESENTbitthe in hardwa the than either of much slower cache initialization is show the Tag RAM, or both the Tag RAM and Data RAM, whenandRAMfollowing the Data RAM, met conditions reset: at are both RAM,Tag the or the Tag • The external input(si_cpc_l2_hw_init_inhibit) pin is dr tion code is required. Once the initialization is The I6400 MPS allows cache for the L2 to be manually initia Once all of these conditions are met, the L2 cache Tag RAM Once all of these conditions are met, the L2 cache Tag Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 3.6.1 Cache Flush L2 3.5.3 Cache Initialization Software 3.5.2 Manual Hardware Cache Initialization 46 MIPS64® I6400 Multiprocessing Syst 3.6Cache L2 Flush, Burst, and Abort

L2 register to a register to a register L2 Cache Op validate. A value register _TAG_ADDR_COP) at _TAG_ADDR_COP) order for flush operation register to determine the register to determine the ddresses in the cache. Burst ddresses this register this register is present. A ‘1’ _TYPE field in bits 4:2 of the _TYPE field in bits eld indicates Hit In eld indicates Hit of cache lines requested must be less less be must of cache lines requested GCR L2 Cache Op State Machine Tag Address Tag Machine Cache Op State GCR L2 register (GCR_L2SM register ndicates Hit Writeback. e state machine is idle, in idle, is e state machine performed on a range of a L2 Cache Op State Machine Config/Control CacheOp State L2 so less than 65,536. L2 Cache Op State Config/Control Machine L2 Cache Op State its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its L2 Cache Op State Machine Config/Control OpL2 CacheState 4, programL2SM_COP_CMD4, the fieldbits in 1:0 of the in space GCR address if to determine register. A value of 0x4 A value in of fi this register. and a valueand of 0x6 a i registerofa value to 0x1. This initiates the CacheOp startingthe from field in bitsthe 63:48 of L2 Cache Op State Machine Config/Control OpL2 CacheState L2 Cache Op State Machine Config/Control OpL2 CacheState ing procedure. Note that the number Note ing procedure. rformed on each line using the L2SM_COP the using rformed on each line t must be 0, indicating th indicating be 0, t must it must be 0, indicatingit mustbe 0, tofor the CacheOp the order inidle, state machine is s in the cache and al cache and s in the e the flush operation begins into the L2SM_COP_START_TAG_ADDR field e the flushbegins operation into the L2SM_COP_START_TAG_ADDR register. A value of A valueindicates 0x0 stillprocessthe running. is value0x1 A indi- of register. e full cache flush operation. flush e full cache mpleted with no errors. the cache flush operation. the ine Config/Control GCR L2 Cache Op State Machine Tag Address Tag Machine Op State Cache GCR L2 of operation to be pe Cache Op State Mach Cache Op State address definedand in step 1 continuing for the number ofdefined lines 2. in step The operation be toperformed defined in step 3. cache lines is the selected in each of state of the L2 state machine. This b machine. This state state of the L2 proceed. state of the L2 state machine. This bi state of the L2 state to proceed. value of 0x0. This selects th of 0x0. This selects value Config/Control Machine State cates that the process co of 0x5 indicates Hit Writeback Invalidate, 0x5 indicates Hit Writeback of (GCR_L2SM_COP) at offset address 0x0620 address at offset (GCR_L2SM_COP) of 0x1. This starts value in bits 47:6 of the in this bitin thisoperationthe flushcache indicates is supported. that registerindicate tothe number be flushed of lines to from the startingdefined address in step 1. Machine Config/Control L2 Cache Op State offset address 0x0628 space. in GCR address offset 5. in step determined idle as machine is the state If 3. Program the L2SM_COP_TYPE field in bits 4:2 of the 4.bit the L2SM_COP_MODE Read the in 5. field in bit 8:6 of the determineofresult theoperation,flush the pollL2SM_COP_RESULT the To 4.L2SM_COP_CMDthe Program fieldthe1:0 in bits of 2.bit Read the L2SM_COP_MODEthe in than or equal to the available line cache 3. Program the type operations can be executed using the follow the using be executed operations can 2. Program the L2SM_COP_NUM_LINES The L2 CacheL2the supports The following burst(CacheOps): operations •Hit_Inv • Hit_WB_Inv • Hit_WB by These operations can be requested only and can be software 1. Program the starting address wher 1. Read the L2SM_COP_REG_PRESENTbitthe in Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 3.6.2 Cache Burst Operations L2 MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 47

L2 Cache Op L2 Cache invalidation across cache, whether local whether cache, field is a non-zero L2 Cache Burst L2 Cache d configuration parame- rs instruction cache. In addi- instruction cache. is case, based on the value er value to the value in the ple, assume that core 0 exe- that core assume ple, Automatic Hardware CacheAutomatic Hardware Manual Hardware Cache Manual Hardware routine, or the GINVI instruction. The em Programmer’s Guide, Revision 1.00 e of 5. Hardware would then read the then read 5. Hardware would e of e cache in a known state. This is accom- is state. This known e cache in a L1 instruction cache L1 instruction or a specified single the case where all remote caches are to be all remote where the case all entries of its own L1 but are not allowed.are software to mangeis up to It but are to be invalidated. In th decoding the cache sizes an sizes cache decoding the ribed in the section entitled in the section ribed rent request is generatedrequest is rentduring the initialization pro- of the instruction. of the field and compares that regist ribedsection). in the previous This section indi-provides This code is designed to be portable to microprocessorsThis code is designed to to be portable dcasts the requestother to all cores, instructing them to This is also true forThis the Flush described operations in the rs nter to one of 32 general purpose registers (GPR) in the its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its esponding core number. For exam number. core esponding validated using the GINVI instruction. If the instruction. If GINVI the using validated nds on the rs field ing either the a software invalidation a software either the ing ctionofways: can be used in one two er-up or reset to place the lines of or reset th er-up des subroutines such as architecture and is used to manage used is architecture and NVI instruction invalidates invalidates NVI instruction field of the field instruction of valu contains a instruction, L1 data, and L2 caches. rs ic L1 instruction cache of any core register. A value of A valueindicates 0x0 stillprocess the running. is value0x1 A indi- of register. , and cache burst operations desc cache burst operations , and struction cache is also fully invalidated in is also fully invalidated struction cache ates all remote primary instruction caches, all ates e of the GPR identified by the mpleted with no errors. e Instruction Cache register, which contains the corr which register, L2 Cache Flush field is 0, then all L1 instruction caches of all cores , no coherent requests , no coherent are if requests Even permitted. a cohe , coherent requestscan be generated during this time rs . State Machine Config/Control Machine State cates that process co the ters ters from the are included. CP0 registers The Instruction cache can be initialized us The cache The cache must be initialized during pow initializing the L1 for vidual routines A sampleboot code is shown in the following subsections. ISA, and provi the MIPS implement that During theDuring automatic hardware initialization process described in the section entitled Initialization theas desc L2, for by hardware plished via the boot code (or, 6. field in bit 8:6 of the determineofresultoperation, theflushthe pollL2SM_COP_RESULT the To core. If If the core. GINVI instruction fully invalid or remote. The local primary in Which cache is invalidated depe invalidated. GINVI instru The the system. in all cores •in all cores all L1 instruction cache entries Invalidate •specif all entries in a Invalidate infield bits 20:16 rs of the GINVIThe instructionisa poi The GINVI instruction is new in the R6 is new in GINVI instruction The cedure, it iscedure, it not allowed to enter the pipeline untilprocedurethe is complete. For the manualinitialization hardware procedure describedsectionthe in entitled Initialization requests duringthesethe flow of initialization the process. section entitled in the rs field, the core executing the GI in the rs field, the core thetion,CM,therequest to core sends a which in turn broa invalidate their own L1 instruction caches. own L1 instruction their invalidate caches can also be in L1 instruction Individual Operations value, the core reads the valu CP0 Global Number cutes the GINVI instruction,andcutes the GINVI the 3.7.1.1 L1 Instruction Cache Invalidation Using the GINVI Instruction Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 3.7.1 Initializing th 3.6.3 Abort Operations 48 MIPS64® I6400 Multiprocessing Syst 3.7 Routines Initialization Cache

register CP0 register is not instruction cache Global Number ws each processing ele- ws each processing Global Number is performed and the and performed is mine the exact L1 at if the rs field in the instruction is 0 as in the instruction field rs the at if ruction caches, so the compare of the CM the ruction caches, so field of the field of register. If there is a match, the L1 instruc- register. VPID , and each core. This allo , and ction to operate at the VP level because the L1 ction to operate at the VP GPR register compare GPR er downlevel. to the VP when So request is theout sent e CM to all other cores in the cluster. Each core compares Each core cluster. in the e CM to all other cores ares are done simultaneouslyares and independently of one its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its register is compared to register is compared deter register toifmatches it determineregister their unique core number.If CP0 Global Number lization routine. on is cache invalidated. Note that there is one -cluster system, there are L1 four inst Global Number Global Number lidated automatically. No lidated automatically. all VP’s in a core. As such, the As core. in a all VP’s required for the GINVI instru required for the GINVI ction cache initia the contents the contents to its own that core and the operation is complete. Note th Note operation is complete. and the that core field of the register assigns a number to each VP in a core in a number to each VP assigns register register is not used. CoreNum // Can be skipped if Config7.HCI is set (Hardware Cache Initialization) Cache set (Hardware is Config7.HCI if Can be skipped // mfc0ext 7 C0_CONFIG, TEMP1, bne HCI, 1 TEMP1, TEMP1, nop done_icache zero, TEMP1, is: the I$ how big Determine // Config7 CP0 Read // mfc0 HCI // extract C0_CONFIG1 CONFIG1_a2, C0_Config1 // read size Set line // addiuILINE_SIZE zero, LINE_SIZE_v1, values fixed are and associativity line size Since the // cache of the size the what determines cache is in the of sets the number // register the C0_CONFIG value in the from is determined set size Here the // ext 64 li 3 CFG1_ISSHIFT, CONFIG1_a2, SET_SIZE_a0, TEMP1, sllv SET_SIZE_a0 TEMP1, SET_SIZE_a0, IS // extract ways) of cache (number Set associativity // addiu IASSOC zero, ASSOC_a1, (LINES_PER_ITER) li TEMP1, way Sets per // I$ dmuldmul ASSOC_a1 SET_SIZE_a0, SET_SIZE_a0, dmul LINE_SIZE_v1 SET_SIZE_a0, TOTAL_BYTES, of bytes number // Total TEMP1 LINE_SIZE_v1, BYTES_PER_LOOP_v0, which (0x80000000) of kgeg0 at the beginning address starting Set the // of sets number // Total per loop bytes // Total cache 0 of the 0 index to way corresponds // dli 0x0000000080000000 CURRENT_ADDR, Global Number LEAF(init_icache) Global Number If there is not a match, the core sendsthere throughrequest is not a match, the core theIf th the value inthe value the request to their own described above, the caches are inva above, the caches are described there is a match, the corresponding L1 instructi tion cache is invalidated for GPR 5 register GPR 5 register and compare per core. Therefore, in a 4-core single per core. Therefore, requestwould oncetimes, four be done per core. All comp another. The This section provides the instru This section ment throughout the entire system to haveunique a ID numb by the CM, the to be invalidated. Note that it is not compare. the used during instruction cache is shared between instruction cache is shared 3.7.1.2 L1 Cache Initialization Routine Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 49

em Programmer’s Guide, Revision 1.00 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ta cache initialization routine. .set.set noreorder noat instr. synthetic for to use r1(at) the assembler allow // Don't instructions. to reorder the assembler allow // Don't dsrldaddu 1 BYTES_PER_LOOP_v0, TEMP1, CURRENT_ADDR TEMP1, CURRENT_ADDR, dadduTOTAL_BYTES CURRENT_ADDR, END_ADDR_a3, dsubu BYTES_PER_LOOP_v0 END_ADDR_a3, END_ADDR_a3, registers Clear TagLo/TagHi // // -1 mtc0 C0_ITAGLO zero, address ending // make at a time lines 8 cache loop does this efficient To be more // the clears tag entry, the invalidates Op Tag Cache Index Store // LRF bit the and clears lock bit, // C0_ITagLo write // daddu address line next starting Get BYTES_PER_LOOP_v0// CURRENT_ADDR, bgeuc Done yet? next_icache_tag// CURRENT_ADDR, END_ADDR_a3, nop is jalr) instruction (following slot R6 forbidden MIPS64 needed for // #include // #defines for GPRs #defines // #include CP0 registers for #defines // #include and HCI DLINE_SIZE for ILINE_SIZE, #defines // #include LINE_SIZE_v1#define BYTES_PER_LOOP_v0 #define v0 SET_SIZE_a0#define ASSOC_a1#define v1 CONFIG1_a2 #define END_ADDR_a3#define a0 TOTAL_BYTES#define CURRENT_ADDR#define a2 TEMP1#define a1 a3 TEMP2 #define t0 t1 loop per instructions cache number of 8 // LINES_PER_ITER #define t2 t3 LEAF(init_dcache) next_icache_tag: (ILINE_SIZE*-2)(CURRENT_ADDR) cache 0x8, (ILINE_SIZE*-1)(CURRENT_ADDR) cache 0x8, (ILINE_SIZE*0)(CURRENT_ADDR) cache 0x8, (ILINE_SIZE*1)(CURRENT_ADDR) cache 0x8, (ILINE_SIZE*-4)(CURRENT_ADDR) cache 0x8, (ILINE_SIZE*-3)(CURRENT_ADDR) cache 0x8, (ILINE_SIZE*2)(CURRENT_ADDR) cache 0x8, (ILINE_SIZE*3)(CURRENT_ADDR) cache 0x8, done_icache: jalr ra zero, nop END(init_icache) This section provides the da provides the This section Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 3.7.2 Cache Initializing Data the 50 MIPS64® I6400 Multiprocessing Syst

its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its // Can be skipped if Config7[HCI] set (Hardware Cache Initialization) Cache (Hardware set Config7[HCI] if be skipped Can // mfc0ext 7 C0_CONFIG, TEMP1, bne 1 HCI, TEMP1, TEMP1, nop done_dcache zero, TEMP1, mfc0C0_CONFIG1 CONFIG1_a2, CP0 Config7 // Read size Set line // HCI // extract addiuDLINE_SIZE zero, LINE_SIZE_v1, in of sets number the fixed values are and associativity line size Since the // C0_Config1 // read is set size Here the the cache. size of the determines is what the cache // register C0_CONFIG in the value from the determined // extli3 CFG1_DSSHIFT, CONFIG1_a2, SET_SIZE_a0, sllv 64 SET_SIZE_a0 TEMP1, TEMP1, SET_SIZE_a0, ways) of cache (number Set associativity // addiu DS // extract DASSOC zero, ASSOC_a1, (LINES_PER_ITER) li TEMP1, dmuldmul ASSOC_a1 SET_SIZE_a0, SET_SIZE_a0, dmul way Sets per D$ // LINE_SIZE_v1 SET_SIZE_a0, TOTAL_BYTES, of bytes number TEMP1 // Total LINE_SIZE_v1, BYTES_PER_LOOP_v0, which (0x80000000) kgeg0 of at the beginning address starting Set the // of sets number // Total per loop bytes // Total cache 0 of the 0 index to way corresponds // luisrl 0x8000 CURRENT_ADDR, addu 1 BYTES_PER_LOOP_v0, TEMP1, CURRENT_ADDR TEMP1, CURRENT_ADDR, addusubu TOTAL_BYTES CURRENT_ADDR, END_ADDR_a3, BYTES_PER_LOOP_v0 END_ADDR_a3, END_ADDR_a3, registers Clear TagLo/TagHi // mtc0 // -1 2 C0_TAGLO, zero, address ending // make not size is the line assumes the code restrictions, field due to offset // 64 bytes than // more C0_DTagLo // write Op Tag Cache Index Store // bit the LRF clears bit, and the lock clears entry, the tag // Invalidates (DLINE_SIZE*-2)(CURRENT_ADDR) 0x9, cache line address starting next Get BYTES_PER_LOOP_v0// CURRENT_ADDR, daddu yet? // Done next_dcache_tag CURRENT_ADDR, END_ADDR_a3, bgeuc nop is jalr) instruction (following slot R6 forbidden MIPS64 for // needed next_dcache_tag: (DLINE_SIZE*-1)(CURRENT_ADDR) cache 0x9, (DLINE_SIZE*0)(CURRENT_ADDR) cache 0x9, (DLINE_SIZE*1)(CURRENT_ADDR) cache 0x9, (DLINE_SIZE*-4)(CURRENT_ADDR) cache 0x9, (DLINE_SIZE*-3)(CURRENT_ADDR) cache 0x9, (DLINE_SIZE*2)(CURRENT_ADDR) cache 0x9, (DLINE_SIZE*3)(CURRENT_ADDR) cache 0x9, done_dcache: Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 51

L2 em Programmer’s Guide, Revision 1.00 used during during used the software initialization its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its alization is invoked described as in the section entitled lization routine. This routine lization routine. This is only , then this, then routine used. is not the Level 2 Cache .set.set noreorder noat instr. synthetic for to use r1(at) the assembler allow // Don't instructions. to reorder the assembler allow // Don't bnez0. core done from Only done_L2_cach_init// r8_core_num, // Read L2 Configuration register ld GCR_L2_CONFIG(r22_gcr_addr) CONFIG_L2_a2, dlidins 0x1 TEMP_s1, sd 20, 1 TEMP_s1, CONFIG_L2_a2, register Configuration Write L2 GCR_L2_CONFIG(r22_gcr_addr)// CONFIG_L2_a2, register Configuration the L2 Read back // ld (bypass) to uncached // set bits // Insert GCR_L2_CONFIG(r22_gcr_addr) CONFIG_L2_a2, // Isolate L2$ Line Size dext 8, 4 CONFIG_L2_a2, LINE_SIZE_v1, L2$ if No Skip ahead // beqnop done_l2 zero, LINE_SIZE_v1, LINE_SIZE // extract dlidsllv 2 TEMP_s1, LINE_SIZE_v1 TEMP_s1, LINE_SIZE_v1, bytes size in line true L2$ decode for // // Isolate L2$ Sets per Way dextdli SET_SIZE_a0 4// extract 12, CONFIG_L2_a2, SET_SIZE_a0, 64 TEMP_s1, jalr ra zero, nop END(init_dcache) for GPRs #defines // #include CP0 registers for #defines // #include and HCI DLINE_SIZE for ILINE_SIZE, #defines // #include LINE_SIZE_v1#define BYTES_PER_LOOP_v0 #define v0 SET_SIZE_a0#define ASSOC_a1#define v1 CONFIG1_a2 #define END_ADDR_a3#define a0 TOTAL_BYTES#define CURRENT_ADDR#define a2 TEMP1#define a1 a3 TEMP2 #define t0 t1 loop per instructions cache number of 8 // LINES_PER_ITER #define t2 t3 LEAF(init_L2) This section provides the L2 cache initia cache provides the L2 This section procedure. If either automaticor manual hardware initi Cache Initialization Options Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 3.7.3 Initializing 52 MIPS64® I6400 Multiprocessing Syst

Cache may contain the only copy of data copy only Cache may contain the to force modifiedforce todata to be writtenback from the L1 D- its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ringorcore downthe when interacting This sec- non-coherent with DMA). write-back policy, whichthat means the D- write-backpolicy, dsllv way per sets for decode SET_SIZE_a0// TEMP_s1, SET_SIZE_a0, L2$ // Isolate Associativity dextdaddiu ASSOC_a1 extract 0, 4// CONFIG_L2_a2, ASSOC_a1, of ways # 1// for decode ASSOC_a1,ASSOC_a1, (LINES_PER_ITER) TEMP_s1, dli dmuldmul ASSOC_a1 SET_SIZE_a0, SET_SIZE_a0, dmul LINE_SIZE_v1 SET_SIZE_a0, TOTAL_BYTES, per loop Total bytes of bytes // number TEMP_s1 // Total LINE_SIZE_v1, BYTES_PER_LOOP_v0, L2 sets in number of total Get // dlidaddu for cacheops address a KSeg0 load 0x80000000// CURRENT_ADDR, TOTAL_BYTES CURRENT_ADDR, END_ADDR_a3, dsubu loop per address -1 bytes // ending // make BYTES_PER_LOOP_v0 END_ADDR_a3, END_ADDR_a3, Tag registers Clear L2 // sd 0x600(r22_gcr_addr) zero, sd 0x608(r22_gcr_addr) zero, sd 0x610(r22_gcr_addr) zero, // GCR_L2_TAG_ADDR entry. the tag Op. Invalidates Cache Store Tag L2$ Index // // GCR_L2_TAG_STATE // GCR_L2_DATA Op Tag Cache Index Store // LRF bit clear the and bit, lock clear the entry, the tag // Invalidate (L2LINE_SIZE*-2)(CURRENT_ADDR) 0xB, cache (L2LINE_SIZE*-1)(CURRENT_ADDR) 0xB, cache (L2LINE_SIZE*0)(CURRENT_ADDR) 0xB, cache (L2LINE_SIZE*1)(CURRENT_ADDR) 0xB, cache (L2LINE_SIZE*-4)(CURRENT_ADDR) 0xB, cache (L2LINE_SIZE*-3)(CURRENT_ADDR) 0xB, cache (L2LINE_SIZE*2)(CURRENT_ADDR) 0xB, cache (L2LINE_SIZE*3)(CURRENT_ADDR) 0xB, cache Done yet? // next_L2_cache_tag END_ADDR_a3, CURRENT_ADDR, bne BYTES_PER_LOOP_v0 CURRENT_ADDR, daddu line address next starting // Get next_L2_cache_tag: done_L2_cach_init: L2) ByPass (enable Clear L2 // ld dins GCR_L2_CONFIG(r22_gcr_addr) a0, 1 zero, 20, a0, sd GCR_L2_CONFIG(r22_gcr_addr) a0, done_l2: register L2 Configuration // Read jalr register L2 Configuration // Write nop ra zero, END(init_L2) bits // Insert The I6400 L1 D-Cache uses a D-Cache L1 The I6400 powe before Cache to the L2-Cache (e.g. stored by the core. In some situations,stored by the core. In somesoftware may need Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 53 3.8 Cache the L1 Data Flushing

em Programmer’s Guide, Revision 1.00 ll data from the L1 D-Cache.ll data from the L1 that Note this routine its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its e data cache has been initialized. e data mfc0C0_CONFIG1 CONFIG1_a2, // Isolate D$ Line Size C0_Config1 // read ext done_flush_dcache 3 CFG1_DLSHIFT, CONFIG1_a2, LINE_SIZE_v1, D$ if No Skip ahead // zero, beq LINE_SIZE_v1, nop DL extract // lisllv in bytes line size true D$ have Now 2 LINE_SIZE_v1// TEMP1, TEMP1, LINE_SIZE_v1, extli3 CFG1_DSSHIFT, CONFIG1_a2, SET_SIZE_a0, sllv 64 TEMP1, SET_SIZE_a0 TEMP1, SET_SIZE_a0, DS // extract - 1 == D$ Assoc Config1DA // extaddiu(LINES_PER_ITER) 3 CFG1_DASHIFT, CONFIG1_a2, ASSOC_a1, 1 ASSOC_a1, li TEMP1, way Sets per D$ // DA // extract dmuldmul ASSOC_a1 SET_SIZE_a0, SET_SIZE_a0, dmul LINE_SIZE_v1 SET_SIZE_a0, TOTAL_BYTES, lui TEMP1 LINE_SIZE_v1, BYTES_PER_LOOP_v0, srl 0x8000 CURRENT_ADDR, addu 1 BYTES_PER_LOOP_v0, TEMP1, CURRENT_ADDR TEMP1, CURRENT_ADDR, of bytes number // Total addu of sets number // Total per loop bytes // Total subu TOTAL_BYTES CURRENT_ADDR, END_ADDR_a3, LINE_SIZE_v1 END_ADDR_a3, END_ADDR_a3, cacheops for address a KSeg0 // Get Op Cache invalidate Index writeback // address ending // make clears entry, tag the invalidates memory, back to data modified // Writes any // -1 bit the LRU clears bit, and the lock // (DLINE_SIZE*-2)(CURRENT_ADDR) 0x1, cache (DLINE_SIZE*-1)(CURRENT_ADDR) 0x1, cache (DLINE_SIZE*0)(CURRENT_ADDR) 0x1, cache (DLINE_SIZE*1)(CURRENT_ADDR) 0x1, cache (DLINE_SIZE*-4)(CURRENT_ADDR) 0x1, cache (DLINE_SIZE*-3)(CURRENT_ADDR) 0x1, cache (DLINE_SIZE*2)(CURRENT_ADDR) 0x1, cache (DLINE_SIZE*3)(CURRENT_ADDR) 0x1, cache BYTES_PER_LOOP_v0 CURRENT_ADDR, daddu yet? // Done fnext_dcache_tag CURRENT_ADDR, END_ADDR_a3, bgeuc nopline address next starting // Get sync is jalr) instruction (following slot R6 forbidden MIPS64 for // needed jalrnop ra zero, LEAF(flush_dcache) fnext_dcache_tag: done_flush_dcache: tion describesroutinethe for writing and invalidating back a should not beexecuted should not after th until Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 54 MIPS64® I6400 Multiprocessing Syst

address resides in address resides at address. If the its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ss (not KSGE0 uncached). For the I6400 the CCA is set to uncached).KSGE0 set For(not the CCA is the I6400ss CCA for KSEG0 cannotCCA for KSEG0be executed KSGE0 address. a from address is set by the TLB entry for th ry Space Cache Coherency Cache ry Space END(flush_dcache) #defineLEAF(change_k0_cca) C0_CONFIG CCA for Set uncached). KSGE0 (not in KSEG1 be executed code must NOTE! This // $16,0 to cacheable kseg0 mfc0 li ins C0_CONFIG t1, mtc0 ra zero, jalr.hb 5 t2, t2, 0, 3 t1, nop C0_Config0 // read C0_CONFIG t1, END(change_k0_cca) K0 // instert C0_Config // write for coherent // CCA The Cache Coherency attribute for a mapped The Cache Coherency coherent because all cached access for the I6400 are coherent. the for access cached coherent because all the KSGE0 memory range,Config.K0CCAis set in the the field.following The code showsthishow done. is the that does the modification of that the code Note uncachedanbedone it must addre in KSEG1 or Rather, Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 55 3.9KSEG0 Memo the Setting em Programmer’s Guide, Revision 1.00 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 56 MIPS64® I6400 Multiprocessing Syst

. Table 4.5 . ovides three types the core, which are the register fields that can s to a handler dedicated to to s Interrupt Mode , or external to , or n of the current instruction stream and instruction stream the current n of dicated hardware interrupt pins. When tion and branch to a dedicated kernel dedicated branch to a tion and watchreserved instruc- match, address MIPS architecture pr function of the Coprocessor 0 Overview of Exception Processing the way interrupts are handled to provide full support e I6400 core is identical to the behavior of an imple- core is identicalthee I6400 of an to behavior in memory is also covered. A list of exception priorities prioritize and vector interrupt zationvectoring and ofThe presence of interrupts. this its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its register. Note that the Global Interrupt Controller register. (GIC) serves as Config3 the system Refer is in EIC mode. the to chapter GIC this in manual for re, which are known as internal events as internal known which are re, ts include arithmetic overflows, traps, at causes the core to halt normal execu to halt normal the core at causes The kernel software then halts executio The kernel software then halts Table 4.1Table Modes Interrupt rrupt. These are generated by asserting de determine and resolve the interrupt. The determine and resolve bit in the ception handler. The exceptionhandlerThe is responsible determining for and then resolving ception handler. VEIC shows the current interrupt mode of the processor as a shows the current interrupt mode of the processor the external interrupt controller when the external more information. mode is denoted by the that interrupt. for interrupt an external controllerthat handles prioriti mentation of Release 1 of the Architecture. the 1 of Release mentation of affect the mode. affect Following reset, the I6400 core defaults to Interrupt Compatibility mode. Table 4.1 • External Interrupt Controller (EIC) mode, which redefines The I6400 core includesThe I6400 three support interruptfor modes: • InterruptCompatibility mode, inwhich thebehavior th of • Interrupt (VI)mode, which adds the abilityto Vectored An exception is defined as any event th any event as defined is An exception software routine called an ex the exception. co the can occur within Exception events known as external events. Internal even external events. known as An external event is known as an inte An external event is known as tions, missestranslationthe in (TLB), etc. A complete of listlookaside exceptions is shown in buffer is provided, along with assembly an language exampleof an exception handler. This chapter providesof an overview exceptiona definition and processing of the interrupts Information modes. on how to program the reset, boot, and general exceptionvectors a pin is asserted, an exception is taken. an exception asserted, a pin is branches to the interrupt handler to interrupt to the branches hardware interruptas described modesof section in the entitled 1x x 0 x x x x x x Compatibility Compatibility Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies StatusBEV CauseIV IntCtlVS Config3VINT Config3VEIC MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 57 4.1 Exception Processing of Overview Exceptions Chapter 4 Chapter

is an example of example is an cannot be non-zero if cannot be non-zero VS register is the address of to can be used identify the IntCtl EPC for debugis suffi- exceptions) EPC Interrupt Mode Interrupt tified. A bus error tified. A bus e e execution can restart after the ch instruction immediately preceding ch instruction immediately DEPC em Programmer’s Guide, Revision 1.00 t location in the has completed and potentially after following neither Vectored Con- nor External Interrupt Interrupt Vectored neither is troller mode implemented. for errors or or errors for n in program order. order. program n in (continued) terrupts (enabled or(enabledterrupts disabled).so saved This context is its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ErrorEPC (or exceptions, the restar ) register with the location wher register), the address of the bran register), the address EPC the following actions: tion execution on handler located at a specific address specific at a located on handler EPC ( for which no return can be iden address ecise exceptions are those for which the ecise exceptions are those

that exceptions are take or, if the instruction if the executing was forbidden the delay in slot or a slotor, of Cause ce of instruc ce of Table 4.1Table Modes Interrupt bit in the ception. For precise the exception has been serviced. the exception has been BD 0000 x 1 x 0 x 0 1 Compatibility 0 Interrupt Vectored Interrupt Controller (EIC) External because occur Cannot     tected, the core takes tected, the core takes are taken after the instruction that caused them taken after the instruction are “x” denotes don’t care “x” denotes don’t Exception Program Counter exception has been serviced that it can be restored when • Enters kernel mode • excepti the software execution of Forces Once invoked,includingexception the handlercontents the of the program should savethe context processor, of the current themode, operating ofstatus and the the in counter, the slot. exceptions are those imprecise Conversely, exception. an imprecise instructions have completed. Imprecise exceptions cient to restart execution. It also ensures When a precise exceptiona precise When condition occurs, the instruction causing the exceptionalland those that followin it the any stall conditions and any later exception conditionspipeline that are cancelled. Accordingly, may have referenced this instruction are inhibited. The value in the When an exception is de • Suspends the normal sequen Exceptions precise or imprecise.be may Pr • Loads the the instruction that caused the exception the instruction branch (as by the indicated instruction that caused the ex 01 01 01 xx Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 4.1.3 Exception Conditions 4.1.2 Detecting Exception an 4.1.1 Exception Types StatusBEV CauseIV IntCtlVS Config3VINT Config3VEIC 58 MIPS64® I6400 Multiprocessing Syst

When this bit is is this bit When manual. in the lower 512 Both of these options are are options Both of these stored in a local register, meaning register, a local stored in the by clearing the lower 512lowerphysical the MBytes of global register global space. ftware can program to set the to ftware can program base set or somewhere with I6400 Technical Reference I6400 Technical a binary value ofa binary value 3’b101, tothe causing BEV 0xBFC0_0. This default setting sets the BEV to a of physical memory its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its 4 GByte physical memory space. 4 and pertains to all VP’s in the core. This means that all VP’s thatmeans in the core. This all VP’s pertainsand to all VP’s r 512 MBytes of the Physical Address r 512 MBytes of the Physical nversely, the reset exception vector is vector the reset exception nversely, default for the field is 00, which directly maps to physical address of 0x0000_0000_1FC0_0000. her the exception vector is mapped to vector is her the exception in CM register space that kernel so kernel space that CM register in set, the first instructiontheis fetched fromexception boot in vector (BEV) BASE field are used to place the vect BASE field are used to place the Register (GCR_CL_RESET_BASE)Register in detailCMthe inchapter of the . reset vector code in reset memory. e GCR_BEV_BASE register located in the CM Figure 4.1 e BEV_BASE field are forced by hardware to field are forced by BEV_BASE e ector is stored in a global register stored in ector is ector is placed in the lower 512 MBytes lower the placed in ector is will access the same BEV during boot-up. Co the same BEV during boot-up. will access VP can have its own each that Control bits in these registers also indicate whethermaps the device the exception lower vector to the 512 MBytes of orGByte4 withinlowerthe the in address range as described following subsections. physical memory, are described of these registers Both MByte space. The hardware configuration addressvirtual of FFFF_FFFF_BFC0_00 shownis This concept in describedfollowing in the subsections. located registers contains two The I6400 address for thebootand resetexception vector locations in memory: • Register (GCR_BEV_BASE) BEV Base • Local Reset VP Exception Base v The boot exception memory, as in the legacy mode, or anywhere within the memory, The boot exception The boot exception v memory. Registers in the whet determine memory. BEV_BASE_MODE bit in th cleared, bits 31:29 of th reside in the KSEG 1 address space (always uncached). 28:12 of the BEV_ bits remaining The When the processor is powered up or re up or is powered processor the When Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 4.2.1 Mapping the BEV to the Lowe MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 59 4.2 Locations Vector Exception the Defining

0000_0000 FFFF_FFFF 1FC0_0000 When this bit is is this bit When be used. In this case the be used. In this BEV global register global space. em Programmer’s Guide, Revision 1.00 4 GByte Virtual Address 4 GByte Virtual e XKPhys uncached space which starts at uncached space which starts XKPhys e physical address space by setting the physical address the Physical Address ubsection, a 4 GByte mappinga 4 GByteubsection, can its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its Lower MBytes 512 FFFF_FFFF_BFC0_0000 virtual address virtual FFFF_FFFF_BFC0_0000 address. physical to 1FC0_0000 maps . it maps the 64-bit virtual address to th 64-bit virtual address the it maps placed anywhere in the 4 GByte the Lower 4 GBytes of KSeg1 e GCR_BEV_BASE register located in the CM Figure 4.2 64-bitAddress Virtual Figure 4.1Figure Address MIPS Default the Using 512 MBytes Lower in the BEV the Mapping KX bit in the CP0 Status registerbe mustenableset toKX bitkernel Status in the CP0 64-bit segments. be vector The boot exception can BEV_BASE_MODE bit in th one, bits 31:12 of the BEV_BASE fieldthe are used to mapexception boot vector anywhereGByte within4the 32- bit physical address space. b Setting the BEV_BASE_MODE address 0x9000_0000_0000_0000.virtual This mapsofa physicalto (0x0000_0000_0000_0000 memory space - 0x0000_0000_FFFF_FFFF). shownis This concept in In the case where it is necessary to be able to map the boot exception vector in physical memory to a location outside of the lower 512 MByte range described in the previous s Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 4.2.2 Mapping the BEV to 0000_0000_0000_0000 FFFF_FFFF_A000_0000 FFFF_FFFF_BFFF_FFFF FFFF_FFFF_FFFF_FFFF 60 MIPS64® I6400 Multiprocessing Syst

FFFF_FFFF 0000_0000 be used. In this case the be used. In this memory to a location outside somewhere within the 512 e Physical Address tion vector is local to each VP in the to each VP in local is vector tion settinga virtualBEV to the sets CM Local address space. A logic ‘0’ dress space by setting the in the KSEG 1 address space (always in the previous subsection, where subsection, in the previous 4 GByte Virtual Address 4 GByte Virtual e RESET_BASE field are forced by hard- by forced field are RESET_BASE e the boot exception vector described above. The main dif- ed to place the reset vector reset ed to place the t exception vector in physical vector in t exception ubsection, a 4 GByte mappinga 4 GByteubsection, can its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its the 4 GByte physical ad the 4 GByte mapsto physical 0x0000_0000_1FC0_0000. address of Lower MBytes of th 512 Lower of the Physical Address 4 GBytes BEV can be mapped anywhere in anywhere mapped BEV can be space. 4 GByte physical address the 512 MBytes of physical memory bymemory clearing the of physical 512 MBytes . ESET_BASE register. When this When 31:12is set to one, bits bit of the ESET_BASE register. default for the field is BFC0_0. This boot exception vector anywhere within the 4 GByte 32-bit physical address en this bit is zero, bits 31:29iszero, en this bit of th Figure 4.1 l VP’s in the core, whereas the reset excep core, the in VP’s l sing the reset exception vector to reside exception vector the reset sing RESET_BASE_MODE is 0 as described described 0 as is RESET_BASE_MODE ve its own reset exception vector. ve its e GCR_CL_RESET_BASE register located in XKPhys = 2) (CCA 64-bitAddress Virtual Figure 4.2Figure Mode 64-bit in Vector Boot Exception the Mapping address of FFFF_FFFF_BFC0_0000,address of which directly This concept is the same as in shown ware to a binary value of 3’b101, cau of 3’b101, value a binary to ware uncached). us of the RESET_BASE field are 28:12 bits remaining The MByte space. The hardware configuration RESET_BASE_MODE bit in the GCR_CL_R bit in the RESET_BASE_MODE RESET_BASE field used are to map the when from space. This is different in this field indicates legacy mode. Wh KX bit in the CP0 Status registerbe mustenableset toKX bitkernel in the CP0 Status 64-bit segments. in reset exception vector can be placed anywhere The of the lower 512 MByte range described in the previous s In the case where it is necessary to be able to map the rese to be able to map necessary it is case where the In ference is that the BEV is global to al that the ference is VP can ha meaning that each core, vectorreset exception the lower is placed inThe The reset exception vector is mapped in the same manner as in the same manner mapped is vector The reset exception RESET_BASE_MODE bit in th RESET_BASE_MODE Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 4.2.4 to the Mapping the Reset Vector 4.2.3 to the Mapping the Reset Vector 0000_0000_0000_0000 9000_0000_0000_0000 9000_FFFF_FFFF_FFFF FFFF_FFFF_FFFF_FFFF MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 61

ster to jump to exception vector. vector. exception on an exception, the boot on an exception, register to jump to the jump to to register BEV_BASE + BEV_BASE 0200 tion. Notethat the IV bit in the than the general vector into the lower 512 MBytes of MBytes the lower 512 vector into em Programmer’s Guide, Revision 1.00 BEV CR_CL_RESET_BASE regi ogramming the SELECT_BEV bit (0) in the [31:7] || 7’b0000000 [31:7] || Status ess to the XKPhys uncached space which starts at starts space which uncached the XKPhys to ess ich exception vector is used used is ich exception vector 0x100+ 0300 BEV_BASE 

0xFFFF_FFFF_BFC0.0000 0xFFFF_FFFF_BFC0.0480 0xFFFF_FFFF_FF20.0200 ss stored in the GCR_BEV_BASE ss stored its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its 28..12 DebugVectorAddr 0x000 RESET_BASE + (SELECT_BEV = 0) + (SELECT_BEV RESET_BASE ring concatenation 

has value of the fixed 01 EBase . fset 0x0020 in CM GCR local address space. address 0x0020 in CM GCR local fset  63..12 se address as a function of the excep of function as a address se 31..30 1 the 64-bit virtual addr the 64-bit  dicated exception vector offset, rather dicated exception vector offset, lue, forcing the location of the reset the location of lue, forcing V andReset Exception Vectors EBase EBase VP uses the address stored in the G uses the address VP ‘||’ denotes bit st ‘||’ 63..30 Figure 4.2 Address per Exception Type 10. EBase Note that 2’b exception vector. This is accomplished by pr accomplished by is This exception vector. Table 4.2Table Addresses Base Vector Exception Pro- = 1 and = 0 and = 0 and shows the offsets from the vector ba vector the from the offsets shows register causes interrupts useto a de Exception DmxSegEn DmxSegEn DmxSegEn DmxSegEn = 0 in the VP_Control1 VP_Control1 0 in the = = 1 in the CP0 VP_Control1 Table 4.3 Cause exception vector or the reset The I6400a way for core providesThe the programmer wh to select GCR_CL_RESET_BASE register located at of located at register GCR_CL_RESET_BASE RESET_BASE[31:29] are set to a fixed va are set RESET_BASE[31:29] If the SELECT_BEV bit is 0, then the the 4 GByte physical address space. the 4 GByte physical RESET_BASE_MODESetting the maps bit address 0x9000_0000_0000_0000.virtual maps This of physicala to (0x0000_0000_0000_0000 memory space - 0x0000_0000_FFFF_FFFF). This concept is the same as in shown the corresponding Reset exception vector. the corresponding Reset addre the VP uses then the bit is 1, the SELECT_BEV If global boot exception vector. = 1 register. in the VP_Control1 Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 4.2.6 Base Exception Vector 4.2.5 Selecting Between the BE Reset Debug with NMI Other DCRDVec DCRDVec register. Debug with betrap Debug with Cache Error RDVec RDVec register. 62 MIPS64® I6400 Multiprocessing Syst

) = 1) = 0 0x100  = 0. 0. = VP_Control1.RDVec VP_Control1.RDVec [28:12] [28:12] VS 1 0x000 0x180 0x180 0x200 0x180 VP_Control1.RDVec VP_Control1.RDVec      = 0) e at offset This 0x0680. register is e at offset IntCtl esses as esses of a function the state that + 0x200 EBase VS 2

 0x000 0x180 0x200 Vector RESET_BASE or BEV_BASE) 0x0020. This register is instantiated per- is instantiated This register 0x0020. [63:12] [63:12] [63:12] [63:12] [63:12] [63:12] [63:12] [63:12] [63:12] [63:12] (IntCtl RESET_BASE 7’b0000000 7’b0000000 (if  0b1  BEV_BASE + 0300 BEV_BASE + 0380 BEV_BASE + 0400 BEV_BASE + 0380 BEV_BASE + BEV_BASE 0x380 BEV_BASE EBase EBase EBase EBase EBase 0xFFFF_FFFF_FF20.0200 0xFFFF_FFFF_BFC0_0000 [31:7] [63:30] its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its EBase ring concatenation 0xFFFF_FFFF_BFC0_0480 (if 0xFFFF_FFFF_BFC0_0480 DebugVectorAddr register located in GCR address spac

that contains possible all vector addr ProbeTrap

‘x’ denotes don’t care, don’t ‘x’ denotes

mplexitythein table, it is assumedthat DmxSegEn denotes bit st ‘||’ Table 4.4Table Vectors Exception register in CM3 space at GCR address register in CM3 offset

or Base Address

Table 4.3Table Offsets Vector Exception

IV Cause

EXL Status have its own reset vector. vector. reset its own have Exception Offset Vector

Boot Exception Vect Exception Boot

BEV Status = 1 IV = 0 combines these three tables into one three tables into one these combines EXL VP LocalVP Reset ExceptionBase Cause Table 4.4 can affect the vector selection. To avoid co To the vector selection. can affect Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies General Exception General Interrupt, TLB Refill, Reset, NMI None (uses either Exception VP, which allows each VP to allows each VP which VP, instantiated per-core. instantiated Debugxxx11 TLB Refill 0 0 x x x Debugxxx0x M xxxxx NMI TLB RefillCache Error 1 0 1 x x x x x x x Cache ErrorInterrupt 1 x 0 x 0 x 0 x x x Reset TLB Refill 0 1 x x x Interrupt 0 0 1 x x TLB Refill 1 0 x x x InterruptInterruptAll others 1 1 0 0 0 0 x 1 x x x x x x x All others 1 x x x x 2. global from the Derived 1. from Derived MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 63

Root Root Root Guest Debug Debug Debug

DINT Debug.DSS field. field. bit. Controller (GIC) It is recommended ror). When several exceptions When several ror). occurs during the TLB bit in the DBU Break DBU Break bit in the signal. When an NMI inter- part of a guest (second step) or root TLB. It is recom- em Programmer’s Guide, Revision 1.00 Status.NMI DINT one can single-step into inter- one can single-step into translation, and root TLB opera- have many When causes. this have many When causes. this a guest address translation (first d above other exceptions, including including exceptions, above other d SI_NMI General Interrupt General Interrupt the assertion of the external of the assertion operation (write, probe). ng a lookup, the exact cause is encoded by encoded exact cause is ng a lookup, the ng a lookup, the exact cause is encoded by encoded exact cause is ng a lookup, the lated. This exception lated. This Root.PageGrain.MCAUSE Guest.PageGrain.MCAUSE eck be synchronous. synchronous. eck be et signal. In this case the device is reset. No is reset. the device case In this et signal. bit. bit. levelexception conditions. The exceptions listed are the in ) to lowest (Load/store bus er bit. Note that there is one DINT bit per is one DINT bit per VP. there that Note bit. its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its register, which is part of the register, specific register is written when a Reset exception occurs. exception a when Reset written is register specific exceptions, so that asynchronous handlers. asynchronous) other (or rupt When a DSS exception occurs, sets hardware the CP0 bit. the appropriate setting by input, or register set. Refer to the GIC chapter of this manual for more more for manual this of chapter GIC the to Refer set. register information. CP0 the sets hardware occurs, exception DINT a When Debug.DINT When a DDBLImpr exception occurs, hardware sets CP0 the Debug.DDBLImpr rupt hardware sets the occurs, CP0 lookup process and can only occur as lookup process and address root translation, address for guest tion (write, probe) whether mended that the Machine-Check be synchronous. check A Machine exception can duri exception occurs hardware in the CP0 that the Machine-Ch that the check A Machine exception can duri exception occurs hardware in the CP0 Guestrelated. TLB of part as only occur This can TLB step), and guest e highest priority is taken.of The number the exception taken is recorded register. register. Table 4.5Table Exceptions Priority of Cause Exception Description Mode field of the CP0 of the field containsa list brief and a description of all core ExcCode occur simultaneously, the exception with th occur simultaneously, in the order of their relative priority, from highest priority (Reset order of their relative priority, Table 4.5 Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 0 0x00 Interrupt occurred. interrupt A root-enabled Root 24 0x18 Machine Check - Lookup Root TLB re Root, or n/a n/a DSSPrioritize Step. Single Debug n/a n/a DINTby Caused Debug Interrupt. n/a n/a Resetof Assertion SI_Res n/a n/an/a n/a DDBLImpr Imprecise. Load. Data Break Debug NMI the assertion of the Indicates Field Encoding Cause.ExcCode Decimal Hex 64 MIPS64® I6400 Multiprocessing Syst 4.3 Priorities Exception Core-Level

Root Root Root Root Root Guest Guest Guest Guest Guest Root or register register register exceptions to allow to allow exceptions exceptions to allow to allow exceptions tected on an instruc- Root.WatchHi Root.WatchHi d because EXL was a EXL was d because register. In addition, In register. rred because EXL was a was EXL rred because This can occur due to a Root or a due to can occur This struction fetch or a data load. struction fetch or tch was detected on an instruc- detected was tch tch or a data load. This can can This load. data a or tch on occurs in Root mode, hard- Root mode, in on occurs on occurs in Guest mode, hard- in on occurs on occurred during (I), a on occurred during a fetch ction fetch exceptions to allow ction fetch ion fetch or load. A Root TLB ion occurred during a fetch, a during a fetch, ion occurred est.Cause register. In addition, addition, In est.Cause register. ot TLB entry mapping the address the address mapping ot TLB entry guest context TLB entry mapping TLB entry guest context Root.Cause on Breakpoint (DIB) condition was condition (DIB) on Breakpoint alignment error. A non-word-aligned A non-word-aligned error. alignment (continued) by an instruction fetch. by an instruction above instruction fetch fetch above instruction above instruction fetch fetch above instruction ruction addresses. ruction ruction addresses. ruction tch exception, deferre tch exception, tch address match was de watch exception, defe exception, watch its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its register. register. Debug logic ‘1’ when the exception was detected, was asserted after after asserted was detected, was exception the when ‘1’ logic EXL went to ‘0’. excepti When a deferred WATCH the bit WP in CP0 sets ware hardware sets the I, or W bits R, in the CP0 inst on illegal watch depending on whether the excepti on whether the depending load (R), or a store (W). after asserted was detected, was exception the when ‘1’ logic EXL went to ‘0’. excepti When a deferred WATCH the bit WP in sets CP0 Gu ware hardware sets the I, R, or W bits in the CP0 whether the in depending except store, or a load. tion fetch. Prioritized watch on illegal inst on illegal watch A guest context watch address ma A guest context tion fetch. Prioritized asserted. Prioritized above instru above Prioritized asserted. addresses. break on illegal instruction DBP bit When a occurs, writes of the the DIB exception hardware CP0 Root TLB/XTLB refill - Instruct Root TLB/XTLB refill miss occurred on an instruction fe on an instruction occurred miss or Guest translation. due to a Root occur Guest TLB/XTLB refill - Instructionfetchor data load.AGuest TLB miss occurred on either an in addresswas loaded intotheinthe PC current mode. The valid bit was zero in the Ro bit was zero The valid fetch. by an instruction referenced Guest translation. The valid bit was zero in the referenced the address Refill - Table 4.5Table of Exceptions Priority Exception Description Mode fetch or load fetch or instruction fetch or load fetch instruction TLB Invalid - instruction - instruction TLB Invalid Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 0 0x00 Interrupt occurred. A guest-enabled interrupt Guest 2 0x02 TLBL/XTLBL 4 0x04 AdEL Instruction fetch address 23 0x17 - Instruction Fetch WATCH root context A wa n/a n/a - Root Deferred Watch deferred A Root wa n/a n/a - Guest Deferred Watch deferred Guest A n/a n/a DIB Debug Instructi EJTAG An Field Encoding Cause.ExcCode Decimal Hex MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 65

Root Root Root Root Root Root Guest Guest Guest Guest Guest Guest Guest Guest Guest Root or . field of the by the setting setting the by DExcCode Root.Config5.MSAEn em Programmer’s Guide, Revision 1.00 ss to a coprocessor was per- was ss to a coprocessor bits, but denied bits, ction for a coprocessor that is that ction for a coprocessor n occur due to a Root or Guest Guest a Root or to due occur n TLB entry mapping the address the address mapping TLB entry the MSA unit was permitted by MSA unit was permitted the to a Root or Guest translation. above, but occurs during the Guest or Root address translation, Guest or valid Root TLB entry which had valid Guest TLB entry which valid Guest had on is executed in guest-mode. a Guest addresstranslation, or n was executed.n was When this occurs, guest context TLB entry mapping TLB entry guest context instruction. Root or bits. Instruction. Root or Root Instruction. struction. Root or Root struction. exception is similar to the higher-priority to the higher-priority exception is similar (continued) , but denied by on - Root. on - Root or Guest.Status.CU1-2 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its register. r occurred on an instruction fetch.r occurred Root lid bit was zero in the Root lid bit was zero Root.Status.CU1-2 Debug Coprocessor unusable - guest. Acce unusable Coprocessor by the mitted of the Root TLB related. This Check Machine exception listed rather than the TLB operation during TLB lookup. a as part of occur This can only root- in executed is TLBP/TLBWI/TLBGP/TLBGWI a when or mode. Guestrelated. TLB of part as only occur This can when a TLBP/TLBWIinstructi Floating Point exception. Floating Point Root or Execution of SYSCALL Execution of in BREAK Execution of a Reserved Execution of a coprocessor instru Execution of coproces- CP1 and the CP0 supports core The I6400 not enabled. sors. MSA Disabled excepti guest.AccessMSA Disabled - to Guest.Config5.MSAEn The valid bit was zero in the a referenced during the address store. An instruction matched a fetch due occur This can set. XI bit the referenced during a store. This ca a store. during referenced translation. hardware programs a value of 0x9 into the An instruction matched a fetch XI set. bit the CP0 A Cache error occurred on an instruction fetch. on an instruction A Cache error occurred Root 2 1 1 1 1 1 Table 4.5Table of Exceptions Priority Exception Description Mode TLB Operation instruction fetch instruction (Validity exception) (Validity (Validity exception) (Validity (Validity exception) (Validity (Validity exception) (Validity (Validity exception) (Validity (Execution exception) Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 9 0x09 Bp 3 0x3 TLB Invalid - storeva The 8 0x08 Sys 6 0x06 fetch- instruction IBE Bus erro A 11 0x0B CpU 21 0x14 MSADis 15 0x0F FPE 24 0x18 Machine Check - 10 0x0A RI 20 0x14 TLBXI Execute Inhibit. TLB 30 0x1E- Error I-cache n/a n/a SDBBP SDDBP instructio An EJTAG Field Encoding Cause.ExcCode Decimal Hex 66 MIPS64® I6400 Multiprocessing Syst

Root Root Root Root Root Root Root Guest Guest Guest Guest Guest Guest Guest Guest Root or Root or

Table bit if Debug.DDBL Debug.DDBS tected on the address field. Refer to legal data addresses. that all of the execution excep- execution of all the that The exact type of exception is The exact type of exception tch was detected on the address on the tch was detected ting of VzGuest exception pri- ting of VzGuest unaligned address, or an or unaligned address, was asserted. Prioritized above was asserted. Prioritized y) or a data break on store the current processor mode was mode processor the current the current processor mode was mode processor the current occur due to a Root or Guest occur due to a Root or Guest st TLB entry was found, but the a data access. Root or is encoding encompasses all types all is encoding encompasses ruction that overflowed. that ruction or Root condition condition is true). Root or ess Break. A precise EJTAG break data ess Break. A precise EJTAG (continued) to allow break on il break to allow ent error. An unaligned address, or an An unaligned ent error. oot or Guest mode.oot or Guest Root or h address match was de GuestCtl0.GExcCode a matching root TLB entry was but the found, a matching root TLB entry the was found, root TLB entry a matching but its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its address alignment error. An address alignment error. for more information and a lis a and more information for On a data load, a matching guest TLB entry was but the found, zero. bit was valid (V) valid (V) bit was zero. This can bit was valid (V) translation. A guest TLB miss on occurred Load TLB miss. A root TLB miss occurred on a data access. This This access. on a data occurred A root TLB miss miss. TLB Load can occur due to a Root or Guest translation. valid (V) bit was zero. This can bit was valid (V) translation. gue On a data store, a matching zero. bit was valid (V) address that was inaccessible in by a store instruction. referenced referenced by a load referenced or store. context A watch address ma guest by a load referenced or store. address that was inaccessible in by a load referenced instruction. the error occurred during a store. during occurred error the When this exception occurs, hardware sets the CP0 the sets hardware occurs, exception this When bit if the error occurred during a load, or the on load/store (address match onl (address on load/store (address + data match) condition data fetch exceptions tions have the same priority. Th same priority. tions have the exceptions. guest of virtualization the CP0 to written 4.6 orities. Execution of an arithmetic inst an arithmetic Execution of trap (when a trap Execution of Note Guest exception. Virtualized 2 2 2 Table 4.5Table of Exceptions Priority data access Exception Description Mode (Execution exception) (Execution exception) (Execution exception) Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 3 0x3 Invalid TLB - data store a data store, On 2 0x2 TLB - Invalid data load On a data load, 3 0x03 TLBS miss in R TLB Store 2 0x02 TLBL/XTLBL refill - 5 0x05 AdES - Data Access Store 4 0x04- Data Access AdEL alignm address Load 23 0x17 data access - WATCH watc context root A 12 0x0C Ov 13 0x0D Tr 27 0x1B VzGuest n/a n/a DDBL / DDBSAddr Data Debug Precise Field Encoding Cause.ExcCode Decimal Hex MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 67

Root Root Root Guest Guest em Programmer’s Guide, Revision 1.00 y) condition was asserted. Prior- was y) condition ng root TLB entry was found, and was found, entry TLB ng root Break. A precise EJTAG break data Break. A precise EJTAG the root TLB entry mapping the address mapping the address root TLB entry the (continued) ss, a matching guest TLB entry was found, was found, entry guest TLB a matching ss, occurred on a or load store data reference. Root its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its store bus error. Imprecise. Imprecise. store bus error. Root on load + data match (address onl in complete must access data of the aspects all because last itized match. value data do a to order The dirty bit was zero in the guest TLB entry mapping the address mapping entry TLB guest was zero in the bit The dirty by a store instruction. referenced referenced by a store instruction. referenced On a data read access, a matchi a On a data read access, Guest transla- a Root or due to occur can was set. This RI bit the tion. On a data read acce and RI bit the was set. Table 4.5Table of Exceptions Priority Exception Description Mode ptions have the same priority level. the same ptions have have the same priority level. Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 7 0x07 DBE - Data bus error or Load 1 0x01 TLB Modified in was zero The dirty bit 30 0x1E Error - data access Dcache A cache error 19 0x13 TLBRI Inhibit. TLB Read n/a n/a DDBL Precise Debug Data Address Field Encoding Cause.ExcCode Decimal Hex 1. exce Validity Instruction the All of 2.exceptions Execution the All of 68 MIPS64® I6400 Multiprocessing Syst

Cause.ExcCode be set in order be set in order register for more for more register ption is raised before field as shown. In addi-shown. field as cal Address available. is Address cal exception condition is est Virtual Address avail- Address est Virtual avail- est Physical Address GuestCtl0 = 1 (as appropriate) any before ERL nel mode, but the instruction was not mode, but the instruction was not nel e Guest Physical Address is not avail- estCtl0 register must estCtl0 register estCtl0 register must estCtl0 register able exception would be taken in . When one of the guest-related excep- guest-related the of . When one CP0 context, regardless of whether the of context, regardless CP0 ion of a Guest Privileged Sensitive a Guest Privileged ion of = 1, this root-mode exce 1, this root-mode = GuestCtl0.GExcCode RI ss to core functions. Root.Status mode and the Guest Physi the Guest mode and Table 4.6 at any time. When an any time. When at d changes are not recognized. field,indicating a virtualization related exception. ion can be taken. ion can coding 0x1B (27 decimal) in CP0 the = 1 or = 1 d Root TLB exception has Gu TLB exception d Root d Root TLB exception has Gu TLB exception d Root GuestCtl0 Sensitive instruction. Sensitive instruction. EXL Instruction Redirect. Redirect. Instruction Field Change event. its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its Cause.ExcCode ate in the guest CP0 context is not affected. is not CP0 context ate in the guest Root.Status e conditiona switch requiresmode, to root switch the is made before any able. CP0 Gu of the (29) MC bit Note that the is bit If this change. initiated hardware a on occur to interrupt an the for initiate cleared, hardware able. related TLB Root in a results translation TLB initiated mode a Guest when Set mode and th Root in occurring exception able. related TLB Root in a results translation TLB initiated mode a Guest when Set in Root occurring exception the guest-mode except the This exception is taken when execut attempted from guest-ker was Instruction mode. Refer to the enabled for guest-kernel CP0 information on enabling acce information Note that the MC bit (29) of the CP0 Gu of the (29) bit MC Note that the is cleared, bit If this change. initiated software on a occur to interrupt an the for recognized. not are changes initiated software A Reserved Instruction A Instruction Unus or MDMX Reserved When guest mode. t mode when the exception was detected. detected. exception was t mode when the all exception states are stored into root are states all exception Table 4.6Table Values GExcCode GuestCtl0 Mnemonic Description tion, controltion, can be returned to root mode exceptions entry, which appears as en which appears entry, exceptions to all of the Guest related exceptions described in VZGuest , the Table 4.5 9 0x09 GHFCevent. Change Field Hardware Guest 8 0x08 GVAinitiate mode Guest 0 0x00 GPSI Guest Privileged 12 0x013 GSFC 0x02 0x03 Guest Software HC GRR Hypercall Reserved Guest 10 0x0A GPAinitiate mode Guest tion, hardwaretion, ofa value 0x1B writes to the CP0 During guest mode execu processor was executingprocessorgues was in root or In tions in the table is taken, the actual exception type is encoded into the and th guest mode execution detected during exception state is saved. As a result, exception st exception a result, As is saved. exception state switchmoderootThe to is achieved by setting other state is saved. This ensures that ensures This other state is saved. field, corresponds field, corresponds 4 - 7 - 4 0x7 - 0x4 RSV Reserved. 11 - 3111 0x0B - 0x1F - Reserved Exception code value Decimal Hexadecimal Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 69 4.4 Priorities Exception Hypervisor

Table not be modified by register unless it e same basic processing basic processing e same register is dependent on Cause bitchanged is not the in at which execution is restarted. BD EPC es appropriate to the exception. The to es appropriate . bit in the the in bit em Programmer’s Guide, Revision 1.00 BD EPC r the exception and need r the exception tions, exceptions have tions, exceptions th its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its register register is loaded with the PC stored in EPC/ErrorEPC/DEPC register is not loaded and the and not loaded is register EPC are modified */ are modified EPC register. The value loaded into the register. BD registers are loaded with the valu registers are loaded with Cause the CP0 PC registers, including the CP0 PC registers, rnel software need not look at the e error, and Debug excep Debug and error, e Cause struction that actually caused the exception. struction register. 0x000 ting at the exception vector. register is set, the register is set,  register is zero, the register is zero, the register represents the restart address fo represents the restart address register Status 0x180  fields of the EPC 0 No Addressinstruction of the Yes(PC-4) instruction or jump branch the of Address Status Status is 1, all exceptions go through the general exception vector */ vector exception general the go through exceptions 1, all is 

PC PC - 4 PC - EXL BD = 1 then   In Branch/Jump ExcCode EXL Table 4.7Table in EPC, ErrorEPC, Stored or DEPC on Exception Value Delay/Forbidden Slot?Delay/Forbidden Value bit in the bit in the BD = 0 Cause BD = 1 EPC EPC EPC vectorOffset vectorOffset bitin is set the and bit is set appropriately in the appropriately in bit is set register. register. EXL EXL CE EXL BD endif else if (DS) endif */ of exception type of the a function as vector offsets Compute /* then = TLBRefill ExceptionType if vectorOffset vectorOffset field is loaded, fieldnot but is defined, for any exceptionthan type a coprocessor other unusable exception. shows the in each of value stored else /* and neither the EPC nor Cause the EPC and neither /* /* If Status CE if (expn) if Status Cause whether the instruction is in a forbidden slot, or the delay slot of a branch, or a jump which has delay slots. 4.7 If the The • The wishes to identifyof the in address the Note that individual exception types may load additional information into other registers. This is noted in the descrip- type. tion of each exception Operation: •If the the flow: •If With exception of the cach Reset, NMI, With • Thebegins processor execu into the value loaded The • The exception handler in the normal case. Ke case. in the normal exception handler Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 70 MIPS64® I6400 Multiprocessing Syst 4.5 Processing Exception General

expected debug exceptions to to expected debug exceptions mode. The DERET instruction ction must used at return be from 0b00000)) 

VS ) 29..0 = 0) then */ = 0) VS (IntCtl  rnel software. Note that un rnel software. = 0) then = 0) VS its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its de and returnto non-debug vectorOffset 

29..0 = 1) or (IntCtl = 1) or RIPL BEV 0x200 + (VecNum 0x200 n caused the exception. The DERET instru the exception. The DERET n caused /* No carry between bits 29 and 30 */ bits 29 and between carry /* No = 1 then   = 0) then */ = Cause VIntPriorityEncoder() IV 0x180   VEIC  = 1) or (IntCtl = 1) or register. (vectorBase = 1 then */ = 1 then  BEV

EXL DEPC = 0) then = 0) VecNum VecNum VecNum VecNum 0xFFFF_FFFF_BFC0.0200 IV 63..30 contain flowcharts for the followingexceptionsand guidelines for their handlers:  if Config3 vectorOffset vectorOffset else endif vectorOffset ExceptionType  = 1 then 1

else if (Status endif /* if (Status /* if endif vectorOffset vectorOffset  FaultingCoprocessorNumber BEV

Figure 4.4  else endif /* if (Cause /* if endif if (Cause

EXL CE ExcCode vectorBase and endif /* elseif (ExceptionType = Interrupt) then */ = Interrupt) (ExceptionType /* elseif endif elseif (ExceptionType = Interrupt) then = Interrupt) (ExceptionType elseif vectorBase vectorBase  Status Cause /* Calculate the vector base address */ address base the vector Calculate /* if Status endif /* if Status /* if endif Cause endif */ vectorOffset and vectorBase sum of PC is the Exception /* PC returns tothe address in the the debug exception vector at 0xFFFF_FFFF_BFC0_0200instructionreservedmay be viewedas a since uncon- trolled execution of an SDBBP instructio in order to leave debug mo the debug exception handler, Figure 4.3 • exceptions General •exceptions TLB miss Exceptions are handled by hardware and then serviced by ke Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 71 4.6 Handling and Servicing Flowcharts Exception

em Programmer’s Guide, Revision 1.00 NMI exceptions possible. 0  Mode) Comments ; EXL 0 EPC its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its

 (except interrupt if masked by IE)  * After EXL=0, all exceptions allowed * Unmapped vectorTLBMod, so TLBInv, or Refill TLB exceptions not possible * EXL=1 so Watch and Interrupt exceptions disabled to avoid* OS/System all other exceptions * Only Reset, Soft Reset, * ERET is not allowed in the branch delay slot of another Jump Instruction * PC * LLbit (Optional - only to enable Interrupts while keeping Kernel 1  Cause , 0, IE bits:  Status value to & Jump ERET STATUS , , Status MTC0 - EXL = 1 MTC0 - MFC0 - 0, EXL EPC Figure 4.3Figure (SW) Guidelines Servicing Exception General Service Code EPC Set  , Cause UM appropriate Code Service Check Context Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 72 MIPS64® I6400 Multiprocessing Syst

Context and into the TLB write 0  EntryLo Comments ; EXL ; EXL 0 EPC

 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its  * Load the mapping of the virtual address in Reg. Move it to * There could be a TLB miss again during the mappingThere be a TLB miss could* of the data or instruction address. The processor will jump to the general exception vector since the EXL is level refillthegeneral1. (Optionto completethefirst in exception handler or ERET to the original instruction and take the exception again) * ERET is not allowed in the of branch delay slot another Jump Instruction * PC * LLbit * Unmapped vector so TLBMod, * TLBInv, or TLB Refill exceptions not possible so Watch, Interrupt EXL=1 exceptions* disabled OS/System to* avoid all other exceptions Reset, Only * Reset, NMI exceptions Soft possible. CONTEXT Figure 4.4Figure (SW) Guidelines Servicing TLB Exception ERET Service Code MFC0 - Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 73

*/ VS = 0) or vector offset 0x200 (if 0) offset or vector = IV e Caus em Programmer’s Guide, Revision 1.00 in each of in each of these modes. , the I6400, theinterrupts supports three modes. entered when a Reset exception occurs. In this mode, inter- its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its s are not implementeds are nothave or been disabled. handler look for might Overviewof Exception Processing ndler for compatibilitymode: = 1 (if it were zero, the interrupt exception would have to would exception interrupt the were zero, 1 (if it = IV = 1, or = 0, or = = 0, which is the case if vectored interrupt case 0, which is the = IV BEV VS laaddujr VectorBase k1, k0, k1 k0, nop k0 */ vectors of 8 interrupt base /* Get */ offset base and from target /* Compute */ routine exception to specific /* Jump mfc0mfc0 C0_CAUSE k0, andi C0_STATUS k1, and k0, M_CauseIM k0, beq */ from Cause IP bits only /* Keep k0, k1 k0, clz bits */ for IP register Cause /* Read bits */ for IM register Status /* and zero, Dismiss k0, xori k0 k0, sll */ interrupt - spurious bits set /* no k0, 0x17 k0, k0, VS k0, bits */ with IM mask /* and 7..0 */ => /* 16..23 */ 16..23 k0 = IP7..IP0; bit set, first /* Find IntCtl software emulate to /* Shift = 1). This mode is in effect when any of thewhen following any of conditionseffect is= 1). This modein are true: IV /* analogous interrupt, a specific processes routine processing Each interrupt * routine processing each Since mode. EIC interrupt in VI or reached to those * know to context the it has line, interrupt particular to a is dedicated * further to look may need routine Each processing was asserted. which line * requests interrupt multiple if the interrupt of actual source the to determine * the performed, task is that line. Once IP on a single together are ORed * ways: of two in one processed may be interrupt * * The interrupt). simple UART a level (e.g., at interrupt - Completely * this type. of an example below is routine SimpleInterrupt * this In interrupts. other re-enabling and state sufficient - By saving * /* Assumptions: * - Cause * Cause Status IntCtl * be isolated from the general exception vector before arriving before vector exception the general from isolated be * here) * are available k0 and k1 - GPRs * SW1..SW0) (HW5..HW0, is IP7..IP0 priority - The software * * base exception 0x200 from Offset Location: * */ IVexception: The following subsections show how an interrupt interrupt show how an subsections following The isdefault the isandthe processor interruptThis for mode rupts are non-vectored and dispatched 0x180though (if exceptionvector offset Cause As described in the section entitled Here is a typical exception ha is a Here • • • Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 4.7.1 Mode Interrupt Compatibility 74 MIPS64® I6400 Multiprocessing Syst 4.7 Examples Mode Code Interrupt

its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its /* this must include at least the IM bit */ the IM at least include this must /* */ include and may interrupt, current for the /* */ others /* k0 */ bits in ERL, EXL KSU, /* Clear */ interrupts re-enable /* bits so that “lower” priority interrupts are interrupts priority “lower” so that bits IM andins k0, k1 k0, (W_StatusKSU+W_StatusERL+W_StatusEXL) zero, S_StatusEXL, k0, mtc0 C0_STATUS k0, /* interrupt. device clearing here, including interrupt Process * in running the core done with may be this environments In some * */ of Status in copy bits /* Clear of scope the beyond is well an environment Such user mode. kernel or * */ mode, to kernel switch mask, /* Modify this example. * */ dilwlwmtc0 StatusSave k0, mtc0 EPCSave k1, C0_STATUS k0, C0_EPC k1, */ EXL set) (including Status saved /* Get */ value original the /* Restore */ and EPC /* */ and EPC /* */ be required - may not interrupts /* Disable eret */ code interrupted to /* Return */ context software setup here, and Save GPRs /* mfc0sw C0_EPC k0, mfc0sw EPCSave k0, C0_STATUS k0, li StatusSave k0, ~IMbitsToClear k1, */ address restart /* Get value */ Status /* Get */ interrupt for this to clear IM bits /* Get */ in memory /* Save */ in memory /* Save * case the software model determines which interrupts are disabled during disabled are interrupts which determines model software the case * single the is either this Typically, interrupt. of this processing the * or some being processed, interrupt the to corresponds that bit StatusIM * Status other of collection * /* restored be must values the saved processing, interrupt To complete * restarted. code interrupted original and the * */ * also disabled. The NestedInterrupt routine below is an example of this type. of is an example below routine NestedInterrupt The disabled. also * */ SimpleInterrupt: /* request the interupt and clear here interrupt the device Process * to be may need registers some do this, to order In at the device. * an ERET that is such 0 state The coprocessor restored. saved and * code. interrupted the to return will simply * */ NestedException: /* registers, and Status the EPC saving require typically Nested exceptions * disabling routine, exception nested by the be modified may GPRs that saving any * loop, putting an interrupt to prevent Status bits in IM the appropriate * code The sample interrupts. re-enabling and kernel mode, in the processor * only intended and is this processing of all nuances cover below cannot * concepts. the to demonstrate * */ Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 75

l the following conditions are l em Programmer’s Guide, Revision 1.00 mode is in effect when al is in mode effect zes pending interrupts and generates a vector which can be its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its bypasses the entireof sequence code following the atchingto directly the interrupt processingroutine. /* Clear KSU, ERL, EXL bits in k0 */ bits in ERL, EXL KSU, /* Clear */ interrupts re-enable /* /* this must include at least the IM bit */ the IM at least include this must /* */ include and may interrupt, current for the /* */ others /* a dedicated handler routine. VI a dedicated handler label shown for the compatibility mode handler code example described in the previous subsection. = 0 = 1 = = 0 0  = 1 = VInt VEIC

IV BEV VS mfc0sw C0_EPC k0, mfc0sw EPCSave k0, C0_STATUS k0, li StatusSave k0, */ address restart /* Get ~IMbitsToClear k1, */ value Status /* Get */ interrupt for this to clear IM bits /* Get */ in memory /* Save */ in memory /* Save /* Restore GPRs and software GPRs /* Restore state and software */ eret */ interrupt the /* Dismiss andins k0, k1 k0, (W_StatusKSU+W_StatusERL+W_StatusEXL) zero, S_StatusEXL, k0, mtc0 C0_Status k0, */ interrupt device clearing here, including interrupt Process /* */ of Status in copy bits /* Clear */ mode, to kernel switch mask, /* Modify dilwlw StatusSave k0, EPCSave k1, */ EXL set) (including Status saved /* Get */ and EPC /* */ be required - may not interrupts /* Disable NestedException: /* registers, and Status EPC the saving require typically exceptions * Nested loop, interrupt an to prevent in Status IM bits the appropriate * disabling sample The interrupts. re-enabling and mode, in kernel processor the * putting only is intended and processing of this nuances cover all cannot below * code concepts. the demonstrate * to */ IntCtl Cause Status Config3 Config3 /* restored be must values the saved processing, interrupt To complete * restarted. code interrupted original and the * */ true: • • In Vectored Interrupt (VI) mode, a priority encoder prioriti Vectored In to used to direct each interrupt • • • A typical software handler for Vectored Interruptmode A typical software handlerfor Vectored IVexception Instead,hardwarethe prioritization, performs the disp A nested interruptis similarshown to that for compatibilitya routineSuch mode.might look as follows: Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 4.7.2 Interrupt Mode Vectored 76 MIPS64® I6400 Multiprocessing Syst

label to prevent to IPL Status to IV exception IV RIPL zes these interrupts with flected in the EIC_MODE EIC_MODE flected in the by kernel softwarekernel by to enable or Cause rrupt logic is configured in order to ) and the timer, performance coun- performance timer, the and ) ms the prioritization, dispatching IP1..IP0 code following the Cause e state of this bit is re ) to the GIC, which) to the GIC, prioriti an example of such a routine: such an example of to powerlegacyin upthen mode, to switch EIC mode. terrupt controllerprioritizingis responsiblefor all inter- nnel, and performancecounter interrupts, and directly sup- its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ) register. This bit can be written This bit can ) register. TI/PCI/FDCI /* Clear KSU, ERL, EXL bits in k0 */ bits in ERL, EXL KSU, /* Clear es the entire sequence of Cause above. Instead, thehardware perfor redefines the way that the inte processor GIC_VL_CTL state of the interrupt requests ( requests the interrupt state of l ofl the following conditionstrue: are interrupting the handler. Here is Here interruptinghandler. the = 1 bit register indicates support EIC mode. for Th = 1 = 0 0  = 1 = VEIC VEIC

IV BEV VS mfc0mfc0 C0_CAUSE k1, srl C0_EPC k0, sw k1, S_CauseRIPL k1, mfc0 field */ RIPL justify /* Right sw */ RIPL value to get Cause /* Read EPCSave k0, C0_STATUS k0, insins */ address restart /* Get StatusSave k0, Status */ copy of RIPL in to IPL 6 /* Set k1, S_StatusIPL, k0, */ value Status /* Get (W_StatusKSU+W_StatusERL+W_StatusEXL) zero, S_StatusEXL, k0, */ in memory /* Save */ in memory /* Save mtc0mtc0 C0_STATUS k0, ehb C0_EPC k1, eret */ value original the /* Restore */ and EPC /* */ hazard /* Clear */ interrupt the /* Dismiss Config3 NestedException: /* registers, and Status the EPC saving require typically exceptions * Nested loop, interrupt an to prevent in Status IM bits the appropriate * disabling interrupts. re-enabling and mode, in kernel processor the * putting and is processing this of all nuances not cover can code below sample * The the concepts. only to demonstrate * intended */ Status Config3 IntCtl Cause directly to the interrupt processing routine. A nested interrupt is similar to that shown for compatibility mode. It must also copy shown for the Compatibility-mode handler other hardware interrupts. A typical exception handler for EIC mode bypass • ter, and fast debug channel interruptter, requests ( External Interrupt Controller (EIC) mode (EIC) Controller External Interrupt rupts, including fast debug hardware, cha software, timer, provide support for an external interrupt controller. The in providesupport for an external interrupt controller. plyingtheto processor the vectorof number the highest interrupt. priority al if EIC interruptmode is in effect • • • lower priority interrupts from The disable EIC mode. disable EIC mode. This for is useful systems that may want EICInmode, the processor sends the read-write bit of the GIC VL ControlGIC( VL read-write bit of the Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 4.7.3 External Controller Interrupt Mode MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 77

em Programmer’s Guide, Revision 1.00 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its /* re-enable interrupts */ interrupts re-enable /* mtc0 C0_STATUS k0, */ interrupt device clearing here, including interrupt Process /* */ mode, to kernel switch IPL, /* Modify /* above. for VI mode shown to that identical code is completion The interrupt * */ Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 78 MIPS64® I6400 Multiprocessing Syst

register address space is programming, including the to twoI/O coherence units in the I6400Multiprocessing System table that lists each device ID on the device ID lists each table that . The devices connected to the CM are . The devices vices. An overview of the CM A directory-based coherence protocol is used to effi- coherence directory-based A have up to 6 cores per cluster. per cores 6 to up have the same core, accessing a VP in another core, accessing core, VP in another a same core, accessing the its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its each I6400 core, with up core, with I6400 each program theCM to perform various functions, including set- bes information necessary for necessary information bes es with all cores and other devices other and es with all cores rent access to the L1 Data and L2 caches. to access rent system devices via a register ring bus system devices via a register e CM register ring bus and associated and e CM register ring bus Ring Bus and Device ID’s ation, ation, and the CM register map. . The I6400 Multiprocessing System can System I6400 Multiprocessing . The Figure 5.1 register ring busdevice and ID inform shown in (IOCUs), providing the subsystem cohe I/O of th This chapter provides an overview also provided.In addition,chapter thedescribes howto with the various The CM communicates This section provides an overview of the CM and descri the CM overview of provides an This section ciently maintain coherence among the L1 data caches of data caches the L1 among ciently maintain coherence accessing another VP in ting the base addresses in memory, the General InterruptController (GIC), Cluster Power Controller (CPC), and/or Debug Unit (DBU) registers via the CM, and setting theratios clock between the various I6400 system components.revision For the exactnumber of the to the Release Notes. refer Manager, Coherence The Coherence Manager (CM) communicat The Coherence as coherent devicesas well external (MPS), to the I6400 MPS, to achievesystem-wide coherence. L2 cache. integrated low-latency shared an includes The CM these de access to information this uses The programmer bus. Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 5.1.1 CM Interface — Register MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 79 5.1 Overview CM Coherence Manager Coherence Chapter 5 Chapter

Unit (DBU) . These val- CPC Debug Figure 5.1 bus, indicated by the indicated by bus, the above figure, note above figure, the S M M S e ID number matches that to maximize to throughput. The S MCP 6-bit ID value stored6-bitdestina- in the em Programmer’s Guide, Revision 1.00 IOCU 1 IOCU a device initiates an access to the regis- to the an access device initiates a From I/O From rnal proprietary bus called the MIPS bus rnal proprietary GCR using a register using ring Custom AXI4 in-flight operations. in-flight operations. In S Figure 5.1 packet. Only the device whos packet. Only the as both Master (M) and Slave (S). All other devices, devices, other (M) and Slave (S). All both Master as IOCU 0 From I/O From MCP its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ee unidirectional used channels ribed in the following subsections. All values not shown are AXI4 AXI4 Memory lists the ID values for each logic block shown in the ID values for lists MCP S dest_id / src_id / dest_id Coherence Manager3.5 Core 5 ach device on the ring bus is assigned a (Hexadecimal value)Accessed Device g ID is attached to the attached to the g ID is fields of the packet being sent. When sent. being packet fields of the IOCU’s connect to the CM via an inte connect to the CM via IOCU’s Table 5.1 MCP the various devices shown in devices the various GIC S Table 5.1Table Bus Device Ring ID Values Register ased protocolased allow to multiple simultaneous Core 0 012345 0x00 0x01 0x02 0x03 0x04 0x05 Core 0 Core 1 Core 2 Core 3 Core 4 Core 5 1617 0x10 0x11 IOCU0 IOCU1 MCP dest_id / src_id (Decimal value) Figure 5.1Figure CM to the Ring Bus Interface and Register Ports Interface GCR S = Slave M = Master Legend: S S dotted line.theabove, As shown CM and DBU can function E are slave devices. including the cores, of another device, the correspondin ters desc blocks as these in used to write to registers ues are reserved. tion ID (dest_id) or source ID (src_id) source ID (src_id) tion ID (dest_id) or the transaction. in the packet accepts that thethat I6400 MPS supportsupto upup to six cores and IOCUs. to two The CM accesses the registers of Certain devices such as the cores and such as Certain devices ProtocolCoherence (MCP) bus. This bus consists of thr bus implements a credit-b bus implements a RegisterBus Ring Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 80 MIPS64® I6400 Multiprocessing Syst

, and drives this request involved in the example Table 5.1 color indicates the data return color indicates blue (continued) , except only those devices to read a register from the GIC. The data path for this to read a register from the GIC. The data . In this examplefollowing the. actionswouldoccur. its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its Figure 5.1 s as indicated. as s re 0 over the dedicated MCP ‘Response’ bus. Figure 5.2 dest_id / src_id / dest_id its dedicated register ring port. bus Slave (Hexadecimal value)Accessed Device Slave on the ring bu color indicates the access request path, and the request path, and the color indicates the access . This figure is similarThis figure is . to red 24252632333435 0x1836 0x1937 0x1A62 0x2063 0x21 0x22GCR’s Defined User 0x23 GIC 0x24 0x25 Memory 0x3E CM 0x3F CPC GCR DBU dmxseg_normal DBU Master dmxseg_debug DBU Error No Destination OK No Destination Table 5.1Table Register Ring Bus Device ID Values Figure 5.2 dest_id / src_id (Decimal value) directly because it is only a onto the register ring bus throughport. its Master access is shown in is access transaction are shown. The The following example shows the path taken in order for core 0 path. The following sequence is enumerated in is path. The following sequence 1.0request to sends Core over the CM a MCP theNote 0 that ‘Request’ bus. GIC registers cannot Core access the 2. CM processes The request, this assigns the appropriate ID number as defined in 3. The GIC decodes the ID on bus and gets a the match. 4. The GIC then fetches thedatadrivesandthe requested datathe onto ring bus. 5. Data is returnedto the CM through 6.data back to Co requested the sends The CM Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 81

Table 5.2 GCR Base S = Slave S = M = Master Legend: M Core-Redirect k of registers. k of aining to the aining global sys- 6400 core). Contains reg- as defined inthe 2 into another core’s Core-Local Core-Local core’s another into ssuing the request. Each ssuing the Each core has its request. em Programmer’s Guide, Revision 1.00 accessbloc this (aliased for each I6400 core). This block of This block (aliased each I6400 core). for GCR_BASE (aliased for each I for more information on how these for more information ID values cessing this space, the the space, this cessing . Contains registers pert Contains global registers useful in debugging the debugging in global registers useful Contains 3 Control Block Control Control Block Control

S

GIC its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its in the Local Control sub-block must be set with the CORE- in the Debug Block.

4 MCP our 8 KB subblocks functions. which perform different ther Register Usage d to a contiguousin the figure. as showna 8 KB space to d isters pertaining to the isters pertaining I6400 core i this block. of registers within own copy Core-Other addresses gives each Core a window Block. Before ac Control Register Core. NUM target of the Global MPS. I6400 Global Control Block Global Control can All cores functionality. tem Core-Local ed when accessing these various devices. when accessing ed 6 . For simplicity, the0x0000_1FBF_8valueMIPS default of is used for the simplicity, For . Coherence Manager3 S Core 0 Figure 5.3 Core-Local and Core-O Core-Local 1 Figure 5.2Figure of Registers IOCU0 0 Access of Core Path Data 3-channel bus MCP Table 5.2Table to GCR_BASE[47:15]) Map (Relative Address Space Control I6400 5 . This 32 KByte register block can be mapped Theanywhere Address in memory on a 32 KByte boundary. Address Range GCR_BASE[47:15] Size (bytes) Description S This concept is described inis This concept block is assigne GCR base address. Each register Range column shows bits 47:15. Bits 14:0 are always zero so as to align on a 32 KB boundary. Range columnshows bits 47:15. to Bits 14:032 alignso as are always KB boundary. zero on a The 32 KB CM GCR register block is divided into f shows the address map of the four, 8 KB GCR sub-blocks relative to sub-blocks 8 KB GCR relative the four, shows thethemap address of Register Refer to the section entitled to the section Refer are assigned and the programming sequence us the programming sequence and are assigned RegisterBus Ring 0x0000_4000 0x0000_4000 - 0x0000_5FFF 8 KB 0x0000_6000 0x0000_6000 - 0x0000_7FFF 8 KB 0x0000_2000 0x0000_2000 - 0x0000_3FFF 8 KB 0x0000_0000 0x0000_1FFF0x0000_0000 - 8 KB Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 5.1.2 CM GCR Register Map 82 MIPS64® I6400 Multiprocessing Syst

Core-Redirect Debug Block Core-Local Block Core-Other Block Core-Other Global Control Block . A core can access its own Core-Local its can access . A core registerown in its Core-Local block with the 0x0000_1FBF_E018 0x0000_1FBF_E010 0x0000_1FBF_E008 0x0000_1FBF_E000 0x0000_1FBF_FFFF 0x0000_1FBF_A018 0x0000_1FBF_A010 0x0000_1FBF_A008 0x0000_1FBF_A000 0x0000_1FBF_8018 0x0000_1FBF_8010 0x0000_1FBF_8008 0x0000_1FBF_8000 0x0000_1FBF_C018 0x0000_1FBF_C010 0x0000_1FBF_C008 0x0000_1FBF_C000 0x0000_1FBF_9FFF 0x0000_1FBF_DFFF 0x0000_1FBF_BFFF atus registers for a given core and/or Virtual Processor atus registers for a given core and/or Virtual e cores e have cores access to, for one core and provides to a way its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its core. Parameterscore. include base address assignments, reset Core-Redirect +0x2000 +0x2000 +0x2000 Control must Blockthe be set with core number the (CORENUM) target of ers are per-core, and some are per-VP are and some are per-core, ers another core. Before a core can access the Core-Other space, the the Core-Other can access another core. Before a core 0x0000_1FBF_E000 0x0000_1FBF_A000 0x0000_1FBF_C000 MIPS Default: 0x_0000_1FBF_8 15 GCR_BASE Register Figure 5.3Figure in GCR_BASE MIPS Default the Using Scheme Addressing CM Register GCR_BASE The Core-Other GCR block is a single block that all of th that all a single block Core-Other GCR block is The of registers Core-Local the access The Core-LocalThe GCR block contains the configuration and st (VP). Some of the Core-Local regist (VP). Some of register in that core’s own Core-Local that core’s register in would program the particular core a this case, In core. block to determine the configurableparameters for that exception base, etc. exception base, 47 Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 5.1.4 Core-Other GCRs 5.1.3 Core-Local GCRs MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 83

Global . Table 5.2 If a core wishes to modify wishes core If a eeps one copy of the regis- copy one eeps s for that core. The Core- The core. for that s are hardare wired into the ects the core number and writes this ects the core number and writes ected duringconfiguration. IP If this On-Chip (NOC) coherent intercon- ster to be accessed into the Core-Other into the Core-Other ster to be accessed em Programmer’s Guide, Revision 1.00 that are instantiated per-core, the CM that are instantiated per-core, et address 0x0018. The actual register in the in register 0x0018. The actual et address e system,e of the numbercoherency I/O units are read-only and allow are kernelsoftware quickly to antiated per-VP, the CM k antiated per-VP, core that is them. accessing e Core-Local block located at the address range shown the address e Core-Local block located at vice is built, these values its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its space contains the GCR register space contains the GCR gisters for another core’s Core-Local GCR block. gisters for another core’s e locatedMIPS default at theof 0x0000_1FBF_8000.e address CR registers of another core, it sel hardware interface to the Network- physicalthis memory option if is sel read Communication Unit is present read Communication gisters, it writes to th to writes it gisters, gisters in these blocks. For registers these blocks. For registers in gisters ould then write the contents of the regi the of contents write the ould then For registers that that For registers are inst register provides register the following information: customer selects the number of cores in th in of cores customer selects the number registerown in its Core-Local block at offs System Configuration , the CM provides two, the CM provides blocksof registers. Core-Redirect register address 0x0000. at offset All of these fields Global Configuration Table 5.2 . If a core wishes to program the G nect. • range 0x2000 Core-Local (offset - 0x3FFF) • range 0x4000 Core-Other (offset - 0x5FFF) • Bits 7:0of — Numbersystem cores in the to (up 6) • 2) to (up IOCU’s of Number — 11:8 Bits • Bits 19:16Number — regionsMMIO address of •— Indicates if an Inter-Th 31 Bit • present is Unit — Indicates if a Debug 40 Bit •the type of Indicates 43:41 — Bits Table 5.2 keeps one copycore. of the registerkeeps one per The Core-Local address given core. in a for each VP ter theofand address regions. numberde When the (IOCU’s), The CM maintains a copy of selected re selected copy of maintains a The CM Other address space allows a core to access the GCR re access to a core space allows Other address can be located anywhere in registers These option is not selected, the location of these registers ar The Core-Local block represents registers corresponding to the in the contents of its own set of CM GCR re own of its the contents At IP configurationthe time, As listed inAs listed otherbe core towrittenin block the Core-Other would shown use the corresponding in offset core number to be accessed. The core w core The number to be accessed. core address space. address value into the Configuration configuration. determine the system Interface CM GCR Register Reading the Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 5.1.5 Core-Local Usage Register and Core-Other 84 MIPS64® I6400 Multiprocessing Syst 5.2 Overall Verifying

. Up to 5 Table 5.3 GCR_IOCU_BASE ers. Note that this this that Note ers. GCR registers. This Register. Note that this this that Note Register. different locations using different region must reside on region a register. Note that this region register. register. Note that this region that this Note register. bit in the in the bit disabled via the GGU_EN bit in GGU_EN the disabled via Address register. Sets the base the base Sets register. Address Base Address register. Sets the Sets base Base Address register. se Address register. Sets the Sets base se Address register. se Address register. Sets the base the base Sets register. se Address e base address of the IOCU. This block GCR Custom Base GCR as summarized GCR as summarized in address of the GIC. This GIC region may be disabled via the GIC_EN bit in the GCR_GIC_BASE boundary. KB a 128 reside on must Note that thisregister. boundary. KB 32 address of the CPC. This CPC region may be disabled via bit in the the CPC_EN GCR_CPC_BASE boundary. a 32KB reside on must containsthe IOMMU and associated registers. The IOCU region may be disabled via the IOCU_REG_EN address of the Customer be region may the KB boundary. 64 a on must reside region address of the GCR regist KB boundary. 32 a on must reside region e aforementioned regions at e aforementioned regions ry using the associated Base Address register. Each usingry the associatedBaseregister. Address its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its registers located in the se Addresses in Memory se Addresses dress of that block in memory. dress of thatin block memory. Offset Address Field Name Bits Description Table 5.3Table Setting the Base Address for the CM PeripheralDevices provides an example provides of memory th mappingfor all of GCR_CUSTOM_BASE 0x0060 BASE CUSTOM_ 47:16 Custom the MIPS default base address. default base the MIPS Figure 5.4 fixed-size regions can be mapped anywhere in physical memo register indicates the starting ad starting register indicates the This section describes how to set the basetheThis sectionsetofto various CM logic the describes how address blocks. Interface CM GCR Register programmable through a The address map is of set Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies GIC GCR_GIC_BASE 0x0080 GIC_BASE_ADDR 47:17 GIC Ba CPC GCR_CPC_BASE 0x0088 CPC_BASE_ADDR 47:15 CPC Ba GCR GCR_BASE 0x0008 GCR_BASE_ADDR 47:15 GCR Base GCR IOCU GCR_IOC_BASE 0x0100 IOC_BASE_ADDR 47:15 Sets th Block Register Name Custom Custom MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 85 5.3 Programming the Ba

above. above. GIC Base Figure 5.4 companion document. field of the field of 0x0000_1BDE_7FFF 0x0000_1BDE_0000 0x0000_1BDD_FFFF 0x0000_1FBF_FFFF 0x0000_1FD2_8000 0x0000_1BDC_0000 0x0000_1FD2_0000 0x0000_1BDE_8000 0x0000_1FD1_FFFF 0x0000_1FC0_0000 0x0000_1FD2_7FFF 0x0000_1BDB_FFFF 0x0000_1FBF_8000 0x0000_1FBF_7FFF GIC_BASE I6400 Registers em Programmer’s Guide, Revision 1.00 IOCU Main Memory Main Memory Main CPC (32 KB) Main Memory Main Memory Main GIC (128 KB) CM GCR (32 KB) C. This sets the base address of the GIC registers.sets the base address of This C. (32 K x Number IOCU’s) of its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its Figure 5.4Figure Example Programming Map Address field has a fixed size of 32 KB. 1. Program the GIC_BASE field 2. Program the CPC_BASE field 3. Program the GCR_BASE field 4. Program the IOCU_BASE field 4. Program the IOCU_BASE This field has a fixed size of 32 KB. This field has a fixed size of 128 KB. register located at offset 0x0080value0x0000_1BD with a of register located offset at of the GIC Base register at offset 0x0080. of the GIC Base register at offset of the 0x0088. CPC Base register at offset of the GCR Base register at offset 0x0008 of the GCR Base register at offset of the IOCU Base register at offset 0x0100. of the IOCU Base register at offset The followingprogrammingThe register sequence configure is used to the memoryinas shown map For refer moreto the information on the corresponding Base Address register, 1.addressGIC of the set the base registersMIPS default, to the program the To with the MIPS default of 0x0000_1FBF_8. This Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 86 MIPS64® I6400 Multiprocessing Syst

GCR chapter ng bit of CPC Base field of the field of register to it 0 mapping to it 0 mapping field of the field of the registers by program- registers by field of the the correspondi stor for both the CM GCR CPC Programming GCR_BASE a core, with b IOCU_BASE_ADDR CPC_BASE Global Access Privilege Global Access the CP0 CMGCRBase register. In this In this CMGCRBasethe CP0 register. provided access to the CM to provided access s is granted to the reque the is granted to s ogrammer need only clear ogrammer need register located at offset 0x0120. register located Bits at offset 5:0 and 17:16 BDE_0. This sets the base address of the CPC regis- BDE_0. Thisof the CPC base the sets address this case, the base value is read and an offset is added value is read and an offset case, the base this questor can be either a core or IOCU. an The CM allows nce, a value0x0000_1FBF_8nce, a of to is used (MIPS default) its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its our 8 KB subblocks that containGlobal, the Core-Local, bits 5:0, each bit corresponds to 5:0, each bits gisters to the default, MIPS program the ontroller (CPC) register block. Refer to the e CM registers by that requestor are ignored.e CM registersthat byare requestor sters to the MIPS default, program the led. This example reprograms the CM the led. This example reprograms rticular requestor, the pr rticular requestor, e CM GCR registers is programmed into described above, write acces Global PrivilegeAccess CSR CM global control registers. In register located at offset 0x0100withof 0x0000_1FD2. a value located at offset register pond to a specific requestor. In a specific requestor. to pond field of the of the field programmer can decide which requestor’s are decide which requestor’s programmer can act register address. register located at offset 0x00080x0000_1FBF_8. withof of a value KB the 32baseaddress registerthe located This sets at offset ACCESS_EN Base block of GCR blockregisters. This isdivided f into blocks. register Debug and Core-Other, #define c0_CMGCRBASE#define mfc0dsll c0_CMGCRBASE t1, li t1, 4 t1, $15,3 or 0xA000_0000 t2, t1 into register CP0 CMGCRBase of contents // move t2, t1 t1, base KSeg1 // assign 4 bits left by in t1 value // shift from CGRBase VA // create IOCU Base Address register located at offset 0x0088value0x0000_1 with a of register located at offset ters. ming the core 0 and bitcore 0 and 5 mapping to 5. For core bits 17:16,16 bit maps to IOCU0, and bit 17 mapsIOCU1. to The MIPS meaninginall(all cores and system the all that requestor’s defaultisfield for this 0x0000_0000_0003_003F, set. to the CM register access have IOCU’s) for a pa access to the registers disable To in this manual for more information. PermissionsRegister Access CodeExample The base address for the location of th example, the base address could be any value. As a refere indicatebase the location of the toto it derive the ex core 0. only IOCU0 and enable By default all IOCU’s and cores are enab cores are and IOCU’s By default all 4.addresstheIOCU registers to of the set the base MIPS default, program the To 3. set of the CM GCR re the base address To A requestor can access to selected CM request registers. A re up to six cores and two IOCU’s. Note thatup toall have sixrequestor’s read cores permission and twoto IOCU’s. all CM GCR registers, but write must be granted. registers these to access Interface CM GCR Register During boot time, the 2. set the CPC regi of the base address To Note that by setting one of these bits one that by setting Note of this field each corres this field each of writezero and allthistofield requests to th register block, as well as the Cluster Power C the Cluster well as as register block, Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 87 5.4 Access Permissions Register CM

VP- register compan- field in bits 25:24 of 25:24 field in bits I6400 Registers CT) of this registerCT) of indi- , indicating that the transac- that , indicating Core-Local Reset Exception VP-Local GCR Redirect s. However, if a Global registers. However, accessed. In this case, a different VP In this case, a different accessed. register located at offset address at offset located register would reflect the appropriate register reflect the appropriate register would address in core 1, facilitate VP 2. To er per VP. As such, there can be up to can be As such, there per VP. er Core-Local Reset Exception Base em Programmer’s Guide, Revision 1.00 the Global Interrupt Controller” re, the I6400 allows different register blocks different the I6400 allows re, . This indicatesthe that. register to be programmed field of this same register same this of field e registers are instantiated per-VP in theine registers I6400 are instantiated core. per-VP registers within the same core, but corresponding to a field (bits 13:8) of the field (bits 13:8) by setting the BLOCK_REDIRECT by setting ation).Bits 13:8 (CORE_REDIRE its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its VP-Local GCR Redirect ss 0x0018, indicating the transaction is intended for core 1. Soft- l Processor (VP)in the Same Core” s CorrespondingCore” to Another Processor (VP) in the Same Core // an offset of 0x120. offset // an al and Core-Other Registers in Registers and Core-Other al dicate the Core-Local block of register block Core-Local dicate the There is one instantiation of this regist CORE_REDIRECT accessed by VP1, the value in this field by VP1, the value accessed Local Registers via the CM” the CM to accomplish the following tasks: the the CM to accomplish 0 (VP_REDIRECT) indicate the 0 (VP_REDIRECT) indicate the VP to be gister programminggister sequence for thisto example. the Refer 1, VP 0 wants to modify the reset exception base P). This is done using the register located at offset address 0x0018.In this example, the atregister located offset register located at offset addre register located at offset wing one VP to access another VP within a given co another VP within a given wing one VP to access register being register modified Core-Local is in the registervalue blocktheso the in of VP2, register located at offset address 0x0020. register located Both at offset of thes lisd 0x0001_0001 t0, 0x120 (t1) t0, only and core0 IOCU0 to enable value // set plus in t1 address to the base in t0 value // write Section 5.5.1 “Programming Another Virtua Section 5.5.2 GCR’ Local “Programming 5.5.3Section CPC the “Accessing Section5.5.4 “PoweringDebug Up the Unit (DBU)CM”the via Section5.5.5 “SettingtheRatios Clock I6400 Between the System Components” 5.5.6Section Core-Loc the “Accessing located 0x0018 at offset (physical address of 0x1FBF_A018) tocorresponds core 1. VP-Local GCR Redirect ion documention for more informationthis on register. 1. Core 1 writes a value of 0x01 to the the within the same VP to be accessed. This is accomplished be VP to same the within Local GCR Redirect different Virtual Processor (V Virtual different Address The I6400 MPS providesThe I6400 ability the togiven for access core a This section describes how to program This describes section • • • • • • tion is intendedtion is 2. Software for VP wouldwrite thenthe modified theto value ware would also program a value of 2 into the VP_REDIRECTofthevalue 2 intoware would also program a Base Address block. Interface CM GCR Register show the re The following steps block or Debug register block was being Debug block or BLOCK_REDIRECT field would be 0 to in is being accessed inside the same core. core. the same inside accessed is being that core For example, assume four of these registers in a given core (in a 4-VPgiven configurin core registers a thesefour of bits 2: and cate the core to be accessed, 0x0018 in the Core-Local register block. In addition to allo this transaction, kernel software wouldkernelthistransaction,software program value of 1 into a the CORE_REDIRECTinfieldthe bits 13:8 of Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 5.5.1 Programming Another Virtual 88 MIPS64® I6400 Multiprocessing Syst 5.5 Examples Programming CM

compan- register located register located e Core-Other block at 0x1FBF_C020 0x1FBF_DFFF 0x1FBF_C000 0x1FBF_BFFF 0x1FBF_A018 0x1FBF_A000 0x1FBF_9FFF 0x1FBF_8000 0x1FBF_FFFF 0x1FBF_E000 I6400 Registers 0x2 to program the GCR registers program the GCR to 8 7 3 2 VP Local RedirectVP Local fault addressing scheme and vector location for core 1, VP 0 by 1, VP 0 for core vector location that core boot the other cores in the re 1 writes the1 writes value into appropriate re 0x01 1413 0 register located in th in located register RESET_BASE . 31 31 12 11 0 first. first. Then core 0 is used ddress of 0x1FBF_A018). indicates This that the register field (bits 2:0)of the its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its example uses uses the example MIPS de Reset Exception Base

Core-Local Core-Other Debug Block Global Control VP_REDIRECT is instantiated on a per-VP basis, Co on is instantiated a per-VP r one coreone to boot r have up first, then Corresponding to Another Core VP-Local wered up through the CPC. If the core has not been powered up, refer toupwered the throughrefer thepoweredbeen the CPC. core has not up, If core 0 would0 core program the boot exception . gister programminggister sequence for thisto example. the Refer 1. Program the Program 1. Figure 5.5 2. Program the Program 2. field (bitsof 31:12) the at offset 0x1FBF_A018. at offset at offset 0x1FBF_C020.at offset Accessing the CPC Local Registers via the CM CPC Local Registers Accessing the Exception Base register in Core-Other address space Core-Other address space Figure 5.5Figure 1, VP GCR BEV_BASE 2 Core the of 1, Core VP 0 Accessing the Core-Local address space RESET_BASE field of theRESET_BASE field Reset EXCBase of the VP Local Redirect register in the to be programmedof core 1. 2 to corresponds VP in its own Core-Local 0x0018 block at offset (physical a offset 0x0020(physical offset address of 0x1FBF_C020). assumes assumes that core 1 has already po been section entitled documention for more informationthis on register. setting its Reset Exception Base register. Notethat this setting its Reset Exceptionregister. Base in core 1. This example examines how in core 3. Since the Reset Exception Base register CM GCR Register Interface CM GCR Register show the re The following steps system. In the followingis bootedassume core 0 up example, In a multiprocessorIn system, it is commonfo 2. writes a Core 1 also value of 0x2 to the This concept is shown is This concept in CORE_REDIRECT and VP_REDIRECT fields Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 5.5.2 Programming Local GCR’s MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 89

reg- register register y of the other y of the register located register located . VP-Local Redirect 0x1FBF_C020 0x1FBF_FFFF 0x1FBF_E000 0x1FBF_DFFF 0x1FBF_C000 0x1FBF_BFFF 0x1FBF_A018 0x1FBF_A000 0x1FBF_9FFF 0x1FBF_8000 Because core 0 is settingcore 0 is Because 0x0 Figure 5.6 regardless of the number of the number of regardless register in that core’s own in that core’s register Reset Exception Base Reset Exception Reset Exception Base Reset Exception wants to access an access to wants 8 7 3 2 VP-Local Redirect 0x01 , the number of the core to be writtento be of , the number the core em Programmer’s Guide, Revision 1.00 1413 0 bit of the bit of field2:0)of (bits the field as described in step 2 above. The actual as described field concept is shown in shown concept is VP_Local Redirect RESET_BASE . This indicatesthe that. register to be programmed address space, address space, or anywhere within the 4 GByte 31 1231 11 0 field31:12) (bitsthe of field (bits 13:8) of the ception Base Register of 1 of Core Register Base ception cal address of 0x1FBF_C020).cal address of core, the write is done to theblock.to Core-Othercore, the write is done address des in the Core-Other block, Core-Other des in the can be set during devicebe set duringcan configurationis normally and not its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its VP_REDIRECT VP_REDIRECT field as described in step 1 above. Similarly, fieldthe number as described of in step 1 above. Similarly, field in the field in RESET_BASE_MODE , this means that when one core one that when means , this is written to. This re-Other block as described in step 3 above. 3 step described in re-Other block as Reset_Base Core-Local Core-Other Debug Block programmedcorresponds 1. core VP to 0 of Global Control Table 5.2 CORE_REDIRECT CORE_REDIRECT e registers associated e registers associated with another core in the lower 512 MBytes of in the CORE_REDIRECT CORE_REDIRECT the programming of the opriatevalue into the 0x1FBF_C020. offset 0x1FBF_A018. offset Figure 5.6Figure Ex the Reset 0 Accessing Core Core-Other address space at offset Core-Other address space Reset Base Exception register in the ister. This indicates that the register to be register that the This indicates ister. located in the Core-Other block at offset 0x0020(physilocated in the Core-Otherblock at offset valueopposedownitsto the Reset base 1, as for core located 0x0018 offset at (physical address of 0x1FBF_A018) tocorresponds core 1. 1. Program the CORE_REDIRECT and 2. Program the RESET_BASE field ofRESET_BASE field Program the 2. the cores in the system. The state of the state The system. the in cores cores in the system, the register to be accessed always resi always to be accessed the register system, the in cores address space, depending on space, address the VP to be writtenis programmed into localcores that Since there is onlySince one Core-Otherblock there in register to be programmed is the via accessed Co 3. Core 0 writes appr the 1. a value Core 0 writes of 0x01 to the The reset vector can either be placed Core-Local space determines which core the data which core Core-Local space determines 2.addition, Invalue0x0 to the Core 0 also writes a of located in the Core-Other block at offset 0x0020.located in the Core-Otherblock This bit at offset changed once it is set. Whenever one core reads or writes to th reads or core Whenever one is programmed into that cores local local cores that into programmed is register in the Core-Local address space at register in the Core-Local address space Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies VP_REDIRECT VP-Local Redirect fields of the 90 MIPS64® I6400 Multiprocessing Syst

register located ). This indicates Core-Local Core-Local register located in the ). A value of 0x3this in Figure 5.4 uses the default addressing addressing uses the default field13:8) (bits the of the destination theas described core Figure 5.4 VP_Local Redirect field its own in power up core is 1. This sequence CPC Local CommandCPC Local CORENUM CORE_REDIRECT up. Noteexample that this field (bits field 13:8) of the its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ProgrammingLocal GCR’s Correspondingto Another Core field (bits the 3:0) of field used to indicate the number of to indicate the field used CMD in the GCRCore-Local space. address Core-Local and Core-Other registers to registers Core-Local and Core-Other CORE_REDIRECT Registers via the CM Registers er up the core indicated in the core indicated er up the companionmore information documentfor on register. this . CORE_REDIRECT bed in the section entitled sequence for this example would be as follows: this example would be as for sequence Figure 5.7 register. register. I6400 Registers register located at offset 0x0028 register located at offset VP_Local Redirect CPC Core-Other block at offset 0x0000CPC Core-Other (physical blockoffset address at of 0x1BDE_2000 in field indicates to the CPC to pow in its own Core-Local block at offset 0x0018in its (physical ownaddress block Core-Local of 0x1FBF_A018 at offset in thethat register to be programmedto corresponds core 1. in #1 above, a core can determinecan a core above,in #1 its ownby core number reading the This concept is shownis This concept in 2. Core 0 then writes a value of 0x3 into the This example shows how Core 0 uses the Core 0 uses how This example shows Notethat in addition to the different that the one descri different Identification above, which assumes that core 1 has already been powered has 1 that core above, which assumes to the scheme. Refer Interface CM GCR Register The register programming 1. 0 Core writes a value of 0x01 to the Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 5.5.3 Accessing the CPC Local MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 91

. The ring . The register located ). This indicates 0x1BDE_4000 0x1FBF_BFFF 0x1FBF_A018 0x1FBF_A000 0x1FBF_C000 0x1FBF_9FFF 0x1FBF_8000 Table 5.1 0 0x0 0x3 43 Figure 5.4 to indicate that Core-Other 8 7 3 2 0x01 e MIPS default addressing e MIPS default VP-Local Redirect 1413 0 em Programmer’s Guide, Revision 1.00 g Unit by accessing the DBU copy of the DBU the Unit by accessing g to Power Up Core 1 Core Up Power to . Refer to the I6400 RegistersI6400. Refer to the companion 63 63 field (bits 13:8) of the e VP-Local Redirect register that this example uses th that this example uses its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its CM Core-Local CM Global Control e CPC Core Local Register Local e CPC Core ug Unit (DBU)used thatto is perform debug and analysisthe on various CORE_REDIRECT CPC Core-Other Command ion describes how to power up the Debu bug Unit (DBU) via the CM ence for this example would be as follows as this example would be for ence 0x1FBF_A018. 0x1BDE_4000. and VP_REDIRECT fields of the VP-Local Redirect register in the Figure5.7 Core 0 Usingth Core-Local address space at address Core-Local address space in its own Core-Local block at offset 0x0018in its (physical ownaddress Core-Local block of 0x1FBF_A018 at offset in thethat register to be programmedto corresponds the Debug (DBU). Unit document for more information on this register. 1. Core 0 writes a value of 0x23 to the CPC Core-Local Command register. The DBU has a ring CPC Core-LocalID Command value of 35, or 0x23register. as described in CORE_REDIRECTvalueIDtheis used in of 35 th field of scheme. Interface CM GCR Register The register programming sequ The I6400 MPS contains a dedicated Deb dedicated MPS contains a The I6400 components in the system. This sect Note register. the DBU copy of the should target accesses 1. Software 1. Software programs the CORE_REDIRECT CPC Local Command register in the Core-Other address space at access Core-Other address space Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 5.5.4 Powering Up the De 2. Software programs the CMD2. Software field of the 92 MIPS64® I6400 Multiprocessing Syst

0x1BDE_4000 0x1FBF_BFFF 0x1FBF_A018 0x1FBF_A000 0x1FBF_C000 0x1FBF_9FFF 0x1FBF_8000 0 register located in the register located 0x0 field (bits 13:8) of the ). A value of 0x3this in 0x3 43 8 7 3 2 evious subsections,evious the I6400 0x23 Figure 5.4 of the CPC register. Note that this of the CPC register. 1413 0 e IOCU 0 has a ring ID value of 16, value of a ring ID IOCU 0 has e CORE_REDIRECT CPC Local CommandCPC Local 63 63 to run at various clock frequencies relative to each other. to run at various clock frequencies relative to each other. e DBUas described in the pr its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its field (bits field (bits the of 3:0) CM Core-Local CPC Core-Local CMD CM Global Control tween the I6400 System Components register in the CPC address space. Th address space. in the CPC register ther accesses should target the IOCU copy the IOCU target should ther accesses . . The ringof ID value 0x10 the is used in CORE_REDIRECT field of the VP-Local llows these different elements different these llows Figure 5.8 Table 5.1 default addressing. default register, whichthe in this DBU (0x23).case is register, at address 0x1FBF_A018. at address 0x1BDE_4000. CPC Core-Other address space CPC LocalCommand register in the CPC Local Clock Change Control Figure 5.8Figure Unit Up the to Power Debug Register Core Other CPC the 0 Using Core GCR Redirect CPC Core-Other block at offset 0x0000CPC Core-Other (physical block addressoffset at of 0x1BDE_4000 in field indicates topower to the CPC the up component indicatedin the and VP_REDIRECT field of the VP-Local Register in the Core-Local address space 2. Software programs2. Software the CMD field of the Multiprocessing System also a In additionIn powering to elements upcores and th such as This sectionhow describesIOCU0 copyclockIOCU 0 by writingThis clockbetweenanda 4:1ratio to set to an the core of the 0x10 described as or in that Core-O to indicate Redirect register 2. Core 0 then writes a value of 0x3 into the This concept is shown is This concept in example uses the MIPS example uses 1. Software 1. Software programs the CORE_REDIRECT Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 5.5.5 Setting the Clock Ratios Be MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 93

). ). compan- Figure 5.4 register located ). This indicates is set, then the clock set, is I6400 Registers Figure 5.4 is set, then the clock change VP-Local Redirect CPC Local Clock Change Control em Programmer’s Guide, Revision 1.00 e field SET_CLK_RATIO (bit 8) e field SET_CLK_RATIO ANGE_EN field (bit 8) of the CPC Local Clock Change field (bits 13:8) of the field3:0)of (bits the its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its e clock domain to change rates when the change clock offset 0x0018 (physicaloffset address of 0x1BDE_4018 in CLK_RATIO zero, then the clock change has completed. CORE_REDIRECT by writing a value of 0x1 into the SET_CLK_RATIO field (bitthe 0x8) of valueby writing0x1 a into of the SET_CLK_RATIO If the field CLK_CHANGE_ACTIVE (bit10) . registerto1step inthat determine the IOCU be programmed device to 0 is with the gister programminggister sequence for thisto example. the Refer rmation registers. on these Figure 5.9 VP-Local Redirect A value of 0x3 in thisof 0x3 inA value field clock indicates a ratio of 4:1the between prescaled clock IOCU and 0. Hardware the reads change sequence is still pending. sequence is in process. If both fields are register located inCPC the Core-Other block at associated clock ratio. Control register0x0018Core-Other located in the CPC (physicalofoffset block address 0x1BDE_4018at in ofin A value 0x15.5). this Figure field th enables offset 0x0028(physical offset address of 0x1BDE_0028). If th sequence is started. GlobalClockCPC Control 0x0028 register (physicalCPCthe inaddress Global of block at offset 0x1BDE_0028). in its own Core-Local block at offset 0x0018in its (physical ownaddress Core-Local block of 0x1FBF_A018 at offset in thethat clock ratio be toprogrammed corresponds to IOCU 0. 1. 0 writes a value Core of 0x10 to the iondocument for more info CM GCR Register Interface Register CM GCR show the re steps The following 3.0x1writes0 into of a value Core the CLK_RATIO_CH shownis This concept in 4.sequence the clock change Initiate 5. Poll forchanges clock complete the by reading CPC Global Clock Controlregister in the CPC Global blockat 2.then0 writes Core a value of 0x3the into Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 94 MIPS64® I6400 Multiprocessing Syst

VP- register is 0x1BDE_4018 0x1FBF_BFFF 0x1FBF_A018 0x1FBF_A000 0x1FBF_C000 0x1FBF_9FFF 0x1FBF_8000 0 0x0 0x3 43 GIC Core-Other Core-Other GIC Global Controller Interrupt 8 7 3 2 een selected, the CLK_RATIO CLK_RATIO een selected, the 0x10 the Global Interrupt Con- 1413 0 devices to a value between 1:1 and 1:8, between 1:1 and value devices to a 63 63 ster set that is instantiated on a per-VP basis. The sterthat set is instantiatedper-VP on a e register ring bus except the ring bus except e register , and other non-core devicesandbus.ring on the register non-core other , This ry clockrysimply ratio, substitute0x1Aof a value inthe Register to Set the IOCU 0 Clock Ratio r any of the programmabler any of domains. clock example, For to its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ID 26. Once the device has b Global Interrupt Controller (GIC) chapter of this manual. CM Core-Local VP copy of the register to access when a CM Global Control d Core-Other Registers in , Core-Local, and Core-Other registers of the have CM been used to modify CPC Clock Change Control een the prescaled clock and all selected clock and all the prescaled een register is used to select register is used the target address 0x1BDE_4018. space at offset 0x1FBF_A018. at offset space Figure Figure 5.9 Local CPC Core Using the 0 Core except the CM,theexcept whichlimited is a clockto ratioor of 1:1 1:2. parameters in other cores, other VP’s within the same coreparameters in other cores, other VP’s In the previous subsections, the VP-Local subsections, the previous In programmingmechanism is applicable for all devicesth on own Core-Local and Core-Otherregi (GIC). The GIC has it’s Local Redirect read or written. For more information, refer to the The above to the clock be used procedure set ratio can fo program the clock ratio for IOCU 1, simplyfor 0x10 substitutein the above a value example of 0x11 since IOCU 1 is to set thelocated mainat ring memo bus ID 17. Similarly, at ring bus main memory is located example since above field can be used to set the ratio betw Redirect register in the VP-Local address and VP_REDIRECT fields of the VP-Local in the CPC Core-Other at address space Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 1. Software programs the CORE_REDIRECT1. Software 5.5.6 Accessing the Core-Local an troller the Local Clock CPC Change Control register 2. Software programs the CLK_RATIO field of programs the CLK_RATIO 2. Software MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 95

register at offset 0x0008 at offset register s in the CM local register in the CM local s be used during boot-up and boot-up be used during nt included in the release. itiated, the caches must be itiated, the caches must either a coherent or non-coherent or non-coherent either a coherent em Programmer’s Guide, Revision 1.00 nstruction fetch or load/store) while nstruction fetch or load/store) pin. The state of this pin is reflected in cher used to enhance L2 performance. The L2 to enhance used cher register. This register reside register. t ID.bit If the is set, the corresponding CM portismon- oherent mode should only oherent Core-Local Coherence Enable Coherence Enable Coherence ange to the power state is in state is to the power ange oherence protocol, MIPS recommends that each domain be each domain recommends that MIPS oherence protocol, its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its 6400 Registers companion docume ws each power domain to be placed in placed be to each power domain ws PS contains an L2 prefet contains PS // an offset of 0x2008 to access the Coherence Enable Coherence the to access of 0x2008 offset // an // register. not execute any cacheable memory accesses (i cacheable memory accesses not execute any ng normal operation. The non-c ng normal operation. e followingcapabilities: L2 itored for prefetching. li t1, CPC_BASE_ADDR t1, CPC_BASE_ADDR li t0, 0x0000_0001 li t1 into value CPCBase // move (t1) t0, 0x2008 sd coherence // Enable plus in t1 base address to the in t0 value // write The coherence manager in the I6400 M prefetcher is managedprefetcher is using twoGCR CM registers. • L2 0x0300Control Prefetch(GCR_L2_PFT_CONTROL) register at offset • L22nd Prefetch Control register 0x0308 (GCR_L2_PFT_CONTROL_B) at offset These registers control th •Prefetch enable • Minimum operatingpagesize (supports systemin 4K - 64K multiples pages of two) •Prefetch • Coherent invalidate requests •enable prefetch Code • L2 prefetching port ID. Eachcorresponds bit to a CM por The I6400 Multiprocessing System allo System Multiprocessing The I6400 in the Core-Local register block. block. Core-Local register in the Interface CM GCR Register external the asserts is enabled when hardware Coherency placed in coherent mode duri mode. Because the I6400 implements a directory-basedBecausemode. c implementsI6400 a the block at offset address 0x0008. There is one of these registers per power domain. addressregisters these 0x0008. There is one of block at offset I refer to the For more informationthis on register, bit 11 (COH_EN) of the Core-Local Status and Configuration the(COH_EN)Status Core-Local of 11 bit power-down. Software should should Software power-down. disabled.coherence is the CM,or disabled using the coherency is either enabled In Note that if a power domain is in coherent mode and a ch toflushed disabling prior coherence mode. Example Coherency Enable Code Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 96 MIPS64® I6400 Multiprocessing Syst 5.7 Cache L2 Prefetch 5.6 Enable Coherency

e L2 Prefetch Control e L2 Prefetch regis- prefetching does not occur. prefetchingoccur. not does of all of these ports using the 8-bit em is determined by the user during IP . Code prefetching is enabled by setting efetch units efetch units implemented. 300 in the GCR Global register space. This read-onlythe GCR Global300 in space. register e NPFT field (bits 7:0) of th of 7:0) (bits field e NPFT written in step #1, then a semaphore has been acquired, else its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its greaterin0 than order tothis bit for have meaning. ching. These ports correspond to the (up to) six (up cores and GCR_L2_PFT_CONTROL register. Notethe that number of GCR_L2_PFT_CONTROL register. port. If the bitport. If the is cleared, L2 _B register. Each bit Each of this corresponds field If a a single to port. _B register. determinenumber the of pr = 0 or if the SEM_LOCK bit is currently 0. is if the SEM_LOCK bit = 0 or allows prefetching of the code stream of the code prefetching allows . L2 prefetching can be selected for some can be selected L2 prefetching . implementedthe in Multiprocessing I6400 Syst Figure 5.1 (GCR_SEM) register located at offset address (GCR_SEM) 0x0640. registeroffset at located go to step #1. To release the semaphore: the release To 1.bitregister with the= 0. 31 Write CM GCR Register Interface CM GCR Register acquire the semaphore: To 1.register this withbitslower= 1 and the bit 31VPID.the with threads Write 2. Read the register. 3. If the value read in step #2 is the same as the value as the CEN bit in the GCR_L2_PFT_CONTROL_B register. inthe CEN bit the GCR_L2_PFT_CONTROL_B register. CM providesThe I6400 a mechanism for managingtheis managed by This mechanismsemaphores. uncached Global CM Semaphore A write to this register with write data bit 31 = 1 is inhibited if the SEM_LOCK bit is already 1. A write to this regis- the write data has bit 31 if proceeds normally ter In addition to data prefetching, the CM The CM allows up to 8 ports to be selected for L2 prefet to) two IOCU’s as shown in as to) two IOCU’s The number The number prefetch of units configuration. This value is programmed by into hardware th ter (GCR_L2_PFT_CONTROL) address 0x0 located at offset to way software a convenient kernel field allows PORT_ID field in the GCR_L2_PFT_CONTROL PORT_ID given bit is set, L2 prefetching is monitored for that The field is organized as cores followed by IOCU’s startingSo in a 4-core from bit and 2-IOCU 0. system, bits 0 - 3 followedIOCU’s by as cores isorganized field The of5 the field - 4 wouldBitsIOCU 0 - 1 respectively. represent wouldfieldthe of represent 0 - 3 respectively. cores Bitswould 6 - 7 not used in this be example. CM GCR Register Interface CM GCR Register Prefetchingisenabled by setting PFTEN bit thethe in unitsprefetch implemented as describedmust above be Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 5.7.3 Enabling Code Prefetch 5.7.2 for L2 Prefetching SelectPorts 5.7.1 PrefetchEnable MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 97 5.8 Uncached Semaphore Management CM

reg- bit. If bit becomes GGU_EX RO GGU_EN Global Custom Base R/W Access Access connected to the CM, then the connected to the CM, then 0 GGU_EN bit GGU_EN bit 1/0 State State Enable/disable custom region. GCR address Bit has no meaning. em Programmer’s Guide, Revision 1.00 CM GCR CM GCR RO RO Access Access this bit depends on the state of the bit depends this register address 0x0068at offset inGlobal GCR address ck register (GCR_SEM) 0x0640 at offset in the I6400 0 1 GGU_EX bit GGU_EX bit ned by the system designer and then instantiated into the and then instantiated designer ned by the system its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its dicating a that custom GCR is State State responsibility to program the base address into this field dur- register located0x0068. at offset If a custom block is imple- tion intionduringis selected system the IP Configuration.this option If is designer to implement a 64 KB block of custom registers that can be used this bit is set, in . ssible by kernel software. by kernel ssible Global Custom Status GCR block is GCR block is connected to the CM. on where theon where registers reside must willbe enabled by setting the GGU_ENbit in Logic 1 Logic 0 ng that no custom GCR is connected to the CM, then the the CM, ng that no custom GCR is connected to Global Custom Status No custom GCR block. Figure 5.10 register. Note that Note field doesCUSTOM_BASE the default not andaddress have a this base register. Note that the accessibility of register. Custom GCR block present. GU_Present GU_Present bit becomes R/W and is acce and is R/W bit becomes is cleared (zero), indicati (Hardwired to 0) Global Custom Base Global Custom Global Custom Base Global Custom Block This concept is described inis This concept field is undefinedreset. Therefore, at it is programmer’s bootinga customtime GCR block if is implemented. regi address the selected addition, In the GGU_EX GGU_EN to control system level functions. These registers are defi are registers functions. These level to control system The CM provides the ability for the system design. a custom GCR implementa existence of The selected, the GGU_EX bit is set in the For more information, the refer to Lo Semaphore CM GCR Registers companion document. Registers mented,in the startingofKBblock the 64 address memoryin field is determined using CUSTOM_BASE the 16-bit the space. This bit indicates that a custom that a indicates space. This bit Interface CM GCR Register The CM provides two global registers to handle implementation the of custom registers: the RO and is not accessible by the kernel. If ister at offset 0x0060, and the 0x0060, and ister at offset Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies Custom GCR Custom Figure Figure 5.10 Reset at and GGU_EX Bits GGU_EN the Signal and the CM_Present Between Relationship 98 MIPS64® I6400 Multiprocessing Syst 5.9 GCR Implementation Custom

Global CM tected, infor- and = 1 = 1

this register this to deter- error in more detail and case, the second error Global CM Error Cause Global CM an error is de an error is GCR_ERROR_CAUSE. . The only exception. is if fields can be cleared by second error, only the error second error, ription of each error type and ription of each error . to the to rated if the correspondingrated if bit for that The error corrected is error The if an interrupt Signal CM_ERROR_MASK[1] Signal an interrupt if an interrupt Signal CM_ERROR_MASK[3] Respond with an error to the original the to error with an Respond requestor. if an interrupt Signal = 1 CM_ERROR_MASK[2] the type of error. For more information, refer refer more information, For the type of error. , kernel software can read can , kernel software Global CM Error Cause RegisterError CauseGlobal CM interrupt is generated in addition to the normal ections describe each type of CM Error Multiple Register CM Error reand software errors. When register. Notefor the that register. then an ERROR response is returnedof the regardless located at offset address 0x0040 address (physical addresslocated at offset its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its error type. For a detailed desc GCR_ERROR_MULT.ERR_TYPE CORRECTABLE_ECC_ERR). In this CORRECTABLE_ECC_ERR). I6400 Technical Reference Manual Reference I6400 Technical . Whenalso. thisupdates field is written, hardware the 58-bit manual. GCR_ERROR_CAUSE.ERR_TYPE eld. When an error occurs is loaded, an interruptmay be gene A correctable ECC error occurred occurred error A correctable ECC during an L2 cache access. An uncorrectable ECC error ECC error An uncorrectable access. an L2 cache during occurred A decoding error was detected in the A was detected in the error decoding request. field and the Table 5.4 Table 5.4Table Types Error CM Global CM Error Cause Error Global CM ERR_INFO field for each ERR_TYPE . code field, refer to the code field, refer to the Global Error Mask CM Register associated error address. error associated hardwarethe updates read-only ERR_TYPE field (bitsof 63:58) the . The encoding of these registers is determined by registers . The encoding of these I6400 Technical Reference I6400 Technical setting.mask The setting an controls whether ECC_ERR register. register. Global CM ErrorGlobal Cause Register MP_UNCORRECTABLE_ lists the errors CM.detected by the The following subs MP_REQUEST_DECODE_ERR MP_CORRECTABLE_ECC_ERR GCR_ERROR_CAUSE mine the typeofthetake error and actions. appropriate detected,63:58error is it is captured in bitsa second If of the error (MP_ an L2 RAM correctable was the first error overwrites the first error storedthe in typeof error is set in the either a reset or by writingreset or byeithercurrent the a value of mationthatbedebugging may useful in capturederror the in the is Mask Error Register The CM detects, reports, and handles several types of hardwa of types several handles and reports, CM detects, The Error Address Register error response. error Table 5.4 provides the encoding of the each error of the encoding type is captured, not the the not captured, is type The 0x1FBF_8040). NotethatCM, in theresponse is independent of the error the mask previous from thesetting, whichdifferent gen- is theerationresponse normal CM2.ERROR, If should be an registerthe with values one of listed in variesfield of this organization The ERROR_INFO fieldprovides that additional information about the error. dependingfi onin the value the ERR_TYPE ERR_TYPE When the CM GCR Register Interface CM GCR Register When an error occurs, to the registers in the 0-Reserved- 1 2 3 Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies TYPE Error Name Description Action ERROR MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 99 5.10 Processing Error

Global = 1 = 1 = 1 = 1 = 1 = 1 = 1 = 1 = 1 = 1 = 1 = 1 Global CM Control that bit 13 of this reg- that bit 13 of herency between the CM and the between herency ed in the design. This field is This field design. the ed in em Programmer’s Guide, Revision 1.00 Signal Interrupt if Signal Interrupt if CM_ERROR_MASK[15] Signal Interrupt if Signal Interrupt if CM_ERROR_MASK[14] CM_ERROR_MASK[13] Signal Interrupt if Signal Interrupt if CM_ERROR_MASK[11] CM_ERROR_MASK[12] Signal Interrupt if Signal Interrupt if CM_ERROR_MASK[10] Signal an interrupt if an interrupt Signal CM_ERROR_MASK[4] if an interrupt Signal CM_ERROR_MASK[5] if an interrupt Signal CM_ERROR_MASK[6] CM_ERROR_MASK[7] if an interrupt Signal CM_ERROR_MASK[8] if an interrupt Signal CM_ERROR_MASK[9] ng limit is reached. Note ng limit register (GCR_CONFIG) located at offset register (GCR_CONFIG) at offset located GCR address space. If this bitIf this is set, IOCU GCR address space. e error.e Signal Interrupt if error. Interrupt if Signal (continued) directly programmable. However, the IOCU can be indi- directly programmable. However, was received by received was its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its (IOCU) for managing cache co (IOCU) for managing Global Config cates the number of IOCUs instantiat An illegal request the REGTC. The main pipeline receivederror an half-pipe. from the Thedetected IOCU a parity The IOCU detected a respons An error occurred an AXI during request. An error occurred on the Register the on occurred An error Ring Bus a register during access. A parity error was detected in the L2 L2 the in detected was error A parity data coming from the either core of the memory. lock (FNL) If an L2 fetch and cacheop is one processed when only or zero ways of the cache are unlocked, including pseudo-locks, fails. FNL the then dur- detected was error A decoding ing a request on the BIU. The BIU detected a parity error. Signal an interrupt if error detected a response The BIU the AXI bus. on was detected 0x0010 in CM GCR address space. GCR address in CM 0x0010 Table 5.4Table Types Error CM following register fields: register (GCR_CONTROL) at offset 0x0010 register (GCR_CONTROL) at offset in CM MP_FNL_ERR DECODE_ERR RBI_BUS_ERR IOC_RESP_ERR HALF_PIPE_ERR MP_PARITY_ERR CMBIU_WID_ERR IOC_PARITY_ERR CMBIU_REQUEST_ IOC_REQUEST_ERR CMBIU_PARITY_ERR RBI_REGTC_REQ_ERR CMBIU_AXI_RESP_ERR register (GCR_CONTROL) at offset CM Control the MMIO outstandi regions are blocked once to MMIO accesses must be 0 forister this bit to have meaningas described above. 0x0000 of CM GCR address space and indi space GCR address of CM 0x0000 filledhardware by IP configuration. during • IOCU requestsexternal to devices are counted towardoutstanding the limitrequest the when bit 12 of • IOCU requests are prevented from being issued to MMIO regions by setting the bit 13 of the The I6400 CM contains up to two I/O Coherency Units Units Coherency I/O up to two CM contains The I6400 •of the TheNUMIOCU read-only field11:8 in bits external devices.is The IOCU a hardware block and is not rectly controlled using the 9 4 5 6 7 8 11 15 14 12 13 10 Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies TYPE Error Name Description Action ERROR 100 MIPS64® I6400 Multiprocessing Syst 5.11 IOCU Interface

ound MMIO region reg- CU are allowed to write CU are allowed by programming bits 17:16 of the bits 17:16 by programming in the following equation: in the O_REQ_LIMIT) at offset 0x6F8. O_REQ_LIMIT)at offset , the number of MMIO address regions is 0 in CM GCR address space. Each bit address space. 0 in CM GCR hed by programming the the hed by programming rnal PCIe devices. The MMIO registers The MMIO registers devices. rnal PCIe e Upper and Lower B set, accesses from that IO from set, accesses e if the requeste fallsRegion. into an MMIO The of uncached requestsorder in to avoid potentialdead- to four MMIO regions. Each region is assigned an upper led orled disabled by programmingthe MMIO_EN bit that its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its gion (GCR_MMIO[0-3]_BOTTOM). If the MMIO region is gion (GCR_MMIO[0-3]_BOTTOM). to a MMIO region as shown region as a MMIO to tanding. This is accomplis This tanding. allowed to access the CM GCR registers access the CM allowed to the corresponding bit is the corresponding Request Limit register (GCR_MMI register Limit Request used with communicating used with exte with CCA are used to determin r each MMIO region, with each register containing a 32-bit address bound value. Verifying Overall System Configuration register (GCR_ACCESS) at offset 0x012 register (GCR_ACCESS) at offset nge of each MMIO region is defined using th Global CSR Access Privilege CSR Access Global corresponds to one of two IOCUs. If to one of corresponds the GCR and Cluster Power Controller (CPC) registers. the registers. Cluster Power GCR and (CPC) Controller allow for countingallow for of number of non-speculative fetches code lock condition by having many too requests outs re MMIO the Lower Bound register for each in resides and then the request address enabled, Each of the four MMIO regions listedof the four aboveEach can be enab determined at IP configuration time. The supports I6400 up bound.and lower address to be intended are MMIOThe regions In addition, the address ra Softwarethecan be in-flight can setof MMIO requests that number at any givenby time programming the MMIO of the MMIO_REQ_LIMIT field As described in the section entitleddescribed in the As • IOCUs are can select which Software MMIO_REQ_LIMIT field. MMIO_REQ_LIMIT field. isters. A of registers pair fo are used These registers are located at: • 0x0700bound Lowerof region MMIO 0 (GCR_MMIO0_BOTTOM) at offset •0x0708 at offset UpperMMIO region bound 0 (GCR_MMIO0_TOP) of • 0x0710bound Lowerof region MMIO 1 (GCR_MMIO1_BOTTOM) at offset •0x0718 at offset UpperMMIO region bound 1 (GCR_MMIO1_TOP) of • 0x0720bound Lowerof region MMIO 2 (GCR_MMIO2_BOTTOM) at offset •0x0728 at offset UpperMMIO region bound 2 (GCR_MMIO2_TOP) of • 0x0730bound Lowerof region MMIO 3 (GCR_MMIO3_BOTTOM) at offset •0x0738 at offset UpperMMIO region bound 3 (GCR_MMIO3_TOP) of MMIO_BOTTOM_ADDR[47:16]phys_address[47:16] <= MMIO_TOP_ADDR[47:16] <= bitsphysicaltheaddress 47:16If between of fall value the in MMIO_BOTTOM_ADDR[47:16] and MMIO_TOP_ADDR[47:16], then the access is to the corresponding MMIO region. decoded address is used to determine if the access is is access to determine if the used is decoded address Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 5.12.2 MMIO Region Control 5.12.1 CM GPR Register Interface MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 101 5.12 MMIO Address Regions

ed to reside in MIT. When this bit is MIT. be set to indicatethat allow for forward progress). standing MMIO request is per- MMIO standing fined by the MMIO_REQ_LIMIT a request is determin em Programmer’s Guide, Revision 1.00 See section 5.13. See section and code fetches until a response to an MMIO a response until fetches and code comparison above is further qualifiedby whether Q_LIMIT) register at offset 0x06F8Q_LIMIT) register atin offset GCR address Boundregisterwhere determines the request will be Lower Bound Register canLower Bound Register feature, allowingfeature, outstanding any amount of to requests by the CM, which can by the CM, which can be useful to avoid deadlock when s will be considereds will eligible be hit to the MMIO region. If this field indicates one out requests. The limit is de CCA = UCA. If MMIO_CCA the requestthen 0x3, is CCA= UCA. If = part of the MMIO limit (to of the MMIO part ither UC or UCAither UC or requests can match region. the MMIO MMIO requests. In this case, IOCU are uncached requests its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its used to determineused to whether is to an MMIO regionrequest the CM_MMIO_IOCU_ENABLE_REQ_LI an Auxiliary interface. 0x01, then the address stops serializing uncached stops register address regions,registerthe then address lowest-numbered enabledhit MMIO region which MMIO region the request matches. Once matches. the request which MMIO region request has been received. For example, a value of 0x01 in received. For example, a been request has mitted. Setting this value to 0x00 disables the MMIO limiting The MMIO_DISABLE_REQ_LIMIT bit in the region's occur. space. Once the limit is reached, the CM the space. Once fieldthe7:0 in bits MMIO Request of Limit(GCR_MMIO_RE torequests the particular MMIOregion should not be limited. considered requests are never By default, IOCU uncached via the GCR_CONTROL. controllable this is However, as outstanding counted are requests IOCU uncached set, been reached. has MMIO limit the request blocked if routed. Options are the main memory port or Options are the main memory routed. incoming coherent that also service bridges PCIe accessing an MMIO region, that region MMIO_PORT field in the Lower thatregion,an MMIO region MMIO_PORT can The user the total number limit requests issued of MMIO If MMIO_CCA is If 0x0,to address request is setjust the as shown above. If MMIO_CCA is set to set If MMIO_CCAis above. shown as MMIO_CCAis set to 0x2, then is the requestqualified by addressan hits in multiple If MMIO for determining takes precedence the request has CCAthe requestUC. has only = In other UCrequest words, words, e In other qualifiedUCA. = UC or CC = by CCA Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 102 MIPS64® I6400 Multiprocessing Syst

to individual elements in s are instantiated for each for are instantiated s ous clock domains that can help the programmer easily programmer help the k frequency totheindividual em power-up policy, program- policy, em power-up ers are instantiated on a per-domain ers are instantiated on a per-domain . . is is true for each power domain and power each is true for is down and wakeup procedure, accessing Clock Domains a low-power state. The vari Power Domains the CPC to control the power power the control to CPC the manage power consumption in the device. In addition, a the CPC to control the cloc the the CPC to control d in the I6400 MultiprocessingI6400d in the Systemidentifies and the handledPowerthe by Cluster Controller (CPC).The wer and clockingwer throughout the device.Using registers, and external and signal delaysto Manager (CM). The various power domains that can that be domains power various (CM). The Manager is provided. Other programming principles include setting nce Manager (CM), and memory. In additionclockto man- Managernce (CM), and memory. Multiprocessing Register System. its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its access of CPC registers, syst CPC registers, of access s in order to reduce overall powers in order to reduce consumption. power and regist clock. In each case, the I6400 MultiprocessingCPCthetheSystem, also provides ability to mains mains in the I6400 dually software. Th controlled by kernel themanagement power and clock schemes implemented in the I6400Multipro- run/suspend mechanism, local run/suspend mechanism, RAM shut a system environment. system a shows the various power do the system such as cores, IOCU’s, and the Coherence the as system such cores, IOCU’s, individually controlledthe defined in are section entitled elements in the system such as cores, IOCU’s, Cohere cores, IOCU’s, elements in the system such as agement for the various devices in devices agement for the various and put the caches into change the clock ratios in memory, be individually controlledsectionthe are defined in entitled Figure 6.1 This section provides an overview of provides an This section each clock domain. • For the power domains,inregisters kernel software uses System. cessing power domainallow tofor individual control. Note that in thiscorefigure,core 1 through n are optional blocks dependingonconfiguration. the system Power managementI6400the in Multiprocessing is System I6400domainsconcept CPC uses the of to managepo both domain these or disable enable the programmer can The CPC implements two types of domains; basis that so the can be indivi domain integrate the device into the device integrate • For domains, the clock uses registersinsoftware kernel This chapter providesof an overview how power is manage ming examples of a clock domain change and clock delay change, powering up the CPC in standalone mode (no VP enabled), reset detection, cores internal tuning fine and domain, power another in registers various power domains and clock programmerto the can use procedure on howto set the CPC base address in memory the device to coherent or non-coherent mode, requestor requestor non-coherent mode, or to coherent the device Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 6.1.1 Power Domains MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 103 6.1 Overview Power Management Power Chapter 6 Chapter

its 3:0) of the companion docu- Registers

ng the CMD field (b the ng I6400 em Programmer’s Guide, Revision 1.00 ocal registers that can be used to indepen- Memory Memory Clock Domain Core 5 Core Core n Power Domain Core 5 Core 5 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its Clock Domain CM CM Clock Domain following four power states by programmi DBU Core 1 Core Core 1 the CPC register interface. Debug Unit Power Domain Power Domain , except the CM, contains its own set of Core-L its contains except the CM, , Core 1 IOCU1 Core 1 IOCU1 Register. For more information refer to the on Register. this register, Clock Domain Clock Clock Domain Figure 6.2Figure System Multiprocessing Clock Domains in the I6400 Figure 6.1Figure System Multiprocessing in I6400 Domains the Power Figure 6.1 shows theshows various clock domainsin the I6400 MultiprocessingEach clockdomainSystem. becan shown shows the maximumshowsthe possibleof numberbeinstantiated cores and IOCUsthat can into Up the I6400 MPS. CM3 CM3 Core 0 Core Core 0 Power Domain Power Domain Core 0 IOCU0 Core 0 Core Each device in CPC LocalCommand Figure 6.2 upandIOCU’s. to two to six cores Figure 6.2 individually controlled using ment included in the release. the ment included in dently place device into one of the each IOCU0 Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies Clock Domain Clock Domain 6.1.4 Overview of Power States 6.1.3 Core and IOCU Selection 6.1.2 Clock Domains 104 MIPS64® I6400 Multiprocessing Syst

Clock- ionsdocument in this . a previous command has previous command a executed mode, in coherent A state. domain in the programmed into the 4-bit . state of the domain. state of the a previous command has com- . Refer to the section entitled programmed into the 4-bit CMD . However, the previous steady state state steady the previous . However, ClockOff ers address space. The CPC location CPC location The address space. ers PwrDown command enables power for the domain, power for enables command settings. If the domain was active before and ClockOff register. All address locat All register. PwrUp Enabling CoherentMode stateof 0x2 is programmed when a value into the 4- command to the CPC before to an operational steady state when a value of 0x1 is programmed into the 4-bit command. ent mode. If a command is a command If mode. ent the has transitioned device coherent from to mode non- state when a value of 0x3 is when a state its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its command to the CPC before command to the CPC state when a value of 0x4 is PwrUp tionsbe performedcanregisters. that via the CPC PwrDown ClockOff register. This command uses setup values in the in values uses setup This command register. PwrDown PwrUp GCR_CPC_BASE Reset register. This command uses setup values in the values in the setup command uses This register. register. If the domainwas powereddown power-on before, the register. ClkOff et to be redirected towards be redirected et to the newly is reached programmed state et be redirected to towards register. This commandallows a domainthe in non-coherent operation to register. CPC_CL_STAT_CONF_REG the global, core-local, and core-oth register. register. execution The command of this dependsdomain on the previous register. in the powered-down state, a state, in the powered-down er Register Address Map for more information for more on enablingdisablingand coherence mode. to operationthe using operation. Sending a Sending operation. CPC_CL_CMD_REG CPC_CL_CMD_REG CPC_CL_CMD_REG command given to a domain in coherent operation will remain inactive until the device has left CPC_CL_CMD_REG commanddomainto given a in coherent operation remains inactive untildeviceleft the the has . A power domain. A power is broughtinto - a power domain is brought into - A power domain is brought into - A power domain is brought into state can be state can be sent in ClockOff PwrDown sequence is applied according to sequence is be reset. It also can be sent to a domain in power-down or clock-off mode. The mode. domain or clock-off then willbe reset. It also can be sent become to a domain power-down in leads which active, and a reset sequence is executed CPC_CL_STAT_CONF_REG CPC_CL_STAT_CONF_REG was in non-coherentin operation, was the powerbrought domain is into the Off completed causes the CPC domain targ causes completed the coherentmode of PwrUp Reset A domain is If the power state. theapplies clocks and reset, bringsdomain and the into an operational state. CMD field of theCMD field of CMD field of theCMD field of ClockOff A operation.coherent mode of Sendinga CPC domain targ the pleted causes before can be observed temporarily Enabling CoherentMode PwrDown CMDbit field of the field of thefield of • • • • The CPC uses memory locations within This section describes some of theThis section programming some of describes func withinCPUthemap address is determined by the the command is queued, but not processed by the CPC until by the CPC not processed queued, but is the command coherent to refer For more information, mode. the entitled section Note that each command can only be executed in non-coher can only be executed each command that Note The states are as The states follows: are relative to this base address. address. this base are relative to Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 6.2.1 Cluster Power Controll MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 105 6.2 Register Programming CPC

for more informa- more for core. Contains regis- core. Contains gister and storegister it ning to the global system global ning to the to the Cluster Power Control- a set of registers that is single Figure 6.3 e request. Each core has its own copy copy core has its own request. Each e ess space. The remaining bits (14:0) em Programmer’s Guide, Revision 1.00 window into another Core. into window . Aliased for each I6400 core. This block of block of This core. I6400 each for . Aliased . for each I6400 Aliased section contains load/stores. In load/stores. addition, shown the block offsets . Contains registers pertai . Contains in the GCR_CPC_BASE re in the GCR_CPC_BASE // GCR_CPC_BASE the system and writes that the system and writes is storedinthe bits 47:15 Cluster Power of Controller Control Block Control Control Block Control

, bit 0 is set, to enable the address region for the CPC. bitfor regionenable 0 is set, to, the address

its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its // copy to register ative to to GCR_CPC_BASE[31:15]) ative register located in the CM3. Refer to Refer to CM3. the located in register Global Control Block Control Global This address functionality. visible to all CPUs. Core-Local ters pertainingcore to the issuing th of registers within this block. Core-Other addresses gives each Core a the CPC registers reside on a 32 KB boundary. located at offset 0x0088 in CM addr 0x0088in CM located at offset CPC Locate // . later use in r30_cpc_addrin later use using KSEG1 the equivalentis nowandaddress, done GCR_CPC_Base Figure 6.3 code example is code example is used to read the value Table 6.1Table (Rel Map Address CPC init_cpc , all registers are accessed using 32-bit aligned usingare accessed uncached 32-bit registers , all li a0, CPC_P_BASE_ADDR li a0, (r22_gcr_addr) GCR_CPC_BASE sd a0, li r30_cpc_addr, CPC_BASE_ADDR li r30_cpc_addr, jr ra jr nop Block Offset (bytes) Size Description Table 6.1 0x0000 - 0x1FFF - 0x0000 8 KB 0x2000 - 0x3FFF - 0x2000 0x5FFF - 0x4000 8 KB 8 KB ler Base Address Register. This is a Also physical address. ler Base Address Register. Then the code stores this address for this address stores Then the code settingCPC. up the This completes the CPS initializationreturns to start.code and the done_init_cpc: END(init_cpc) described inis This concept locally for futureuse. LEAF(init_cpc) The code uses the known value of the location of CPC within tion on how to use this register. tionthis on howtoregister. use As mentionedaddressabove,CPC registers of the the base are relativebits tothe 31:15 of Address registerBase (GCR_CPC_BASE) addresstheto are always zero indicateof that In The following Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 6.2.2 CPC Base Address 106 MIPS64® I6400 Multiprocessing Syst

compan- those locations are CPC Core-Other Block CPC Core-Local Block CPC Global Control Block ace return 0x0, and writes to those ssed from other cores by first writing first by from other cores ssed y be accessed using aligned 64-bit some CPC registers, a set of registers some CPC registers, turn 0x0, and writes to I6400 Technical Reference Manual Reference I6400 Technical companion document. in the Core-Localin the ControltheBlock CM. of 0x0000_1BDE_0018 0x0000_1BDE_0010 0x0000_1BDE_0008 0x0000_1BDE_0000 0x0000_1BDE_4018 0x0000_1BDE_4010 0x0000_1BDE_4008 0x0000_1BDE_4000 0x0000_1BDE_2018 0x0000_1BDE_2010 0x0000_1BDE_2008 0x0000_1BDE_2000 0x0000_1BDE_5FFF 0x0000_1BDE_3FFF 0x0000_1BDE_1FFF its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its de and should only be accessed using aligned 64-bit uncached aligned 64-bit de and should only be accessed using I6400 Registers the CPC chapter in the the are 64 bits wide and should onl are lated registers in the CPC address sp CPC address the in lated registers core in the I6400 MPS. In the case of In the case MPS. the I6400 core in +0x2000 +0x2000 domain. These registers can also be domain. acce be These registers can also 0x0000_1BDE_2 0x0000_1BDE_4 0x0000_1BDE_0 ulated registers in ulated registers the CPC address re space e registers, refer to the to registers, refer e these registers, refer to these 15 CPC Local Control Block CPC Local Control Core-LocalRedirect Register (GCR_CL_REDIRECT) GCR_CPC BASE Register GCR_CPC_BASE 47 For more information on ion document. ion exists per power domain or per clock domain or per power exists per All registers in the All registers in the Global Control Block are 64 bits wi 64 bits Block are Control Global in the registers All unpop load/stores. Reads from silently dropped withoutgenerating exceptions. any For more informationthes on Reads from uncached load/stores. unpopu the GCR locations are silentlydropped without generatingexceptions.any for each of these registers exists A set Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies Figure 6.3Figure 0x0000_1BDE_0 of Address Base an Example Using Scheme Addressing CPC Register 6.2.4 Local and Core-Other Control Blocks 6.2.3 Global Control Map Block Register MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 107

at offset 0x120at offset companion document s in the CM local register in s be used during boot-up and boot-up be used during d to it to derive the exact reg- an IOCU. The requestor may itiated, the caches must be itiated, the caches must register located rresponding to that coreIOCU, or and either a coherent or non-coherent either a coherent em Programmer’s Guide, Revision 1.00 nstruction fetch or load/store) while to the CP0 CMGCRBase register. As a As CP0 CMGCRBase register. to the pin. The state of this pin is reflected in I6400 Register the system have access to the CPC register to the CPC register the system have access access to the CPC register set. To disable access to access disable access to the CPC set. To register companion document included in the release. to indicateglobalthecontrol the base location CM of register. This register reside This register register. Global Access Privilege oherent mode should only mode should oherent estor can be either a core or either a be estor can register and an offset is adde is register and an offset Coherence Enable Coherence boot time, the programmer determines which requestor’s ange to the power state is in to the power state is ange oherence protocol, MIPS recommends that each domain beMIPS recommends that each domainoherence protocol, its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its register is located. is register I6400 Registers field (bits 5:0) of this register selects upto six cores, and bits 17:16 rnel software need only clear the bit co rnel software need only ws each power domain to be placed in be to each power domain ws e CM GCR registers is programmed in field is 0x3F, meaning that all cores in field is 0x3F, IOCU1 through IOCU0 allow ACCESS_EN registers by the programming not execute any cacheable memory accesses (i accesses cacheable memory not execute any to CPC Registers to CPC ng normal operation. The non-c Core-Local Status and Configuration Core-Local Status Core Local Coherence Control Local Coherence Core ACCESS_EN #define c0_CMGCRBASE#define mfc0dsll c0_CMGCRBASE t1, li t1, 4 t1, $15,3 0xA000_0000 t2, t1 into register CP0 CMGCRBase of contents // move base KSeg1 // Assign 4 bits left by in t1 value // shift in the CM register map. The 6-bit enable access for IOCU1 through IOCU0 respectively. through IOCU0 respectively. enable access for IOCU1 The MIPS default for The I6400 Multiprocessing System allo System Multiprocessing The I6400 Register Interface Register requ A in a system. to eight requestor’s The CPC allows up By default, coherence is disabledthein I6400By default, coherence is MPS. ister address where the address where ister Note that if a power domain is in coherent mode and a ch toflushed disabling prior coherence mode. EnableCoherent Mode Code Example The base address for the location of th value of 0x0000_1FBF_8reference, a is used (MIPS default) registers. In this case, the base value is read from the CP0 block at offset address 0x0008. There is one of these registers per power domain. addressregisters these 0x0008. There is one of block at offset refer to theFor more informationthis on register, set. In addition, bits 17:16 are to set placed in coherent mode duri mode. Because the I6400 implements a directory-basedBecausemode. c implementsI6400 the a not havenot unrestricted to the access CPC registers.During are provided access are provided access to the CPC the registers for a particular requestor, ke the registers for a particular requestor, registersall write requests the CPC to ignored.requestor by that will be CPC register refer to the listed For more in the informationthis on register, included in the release. power-down. Software should Software should power-down. disabled.coherence is Interface Register external the asserts is enabled when hardware Coherency bit 11 (COH_EN) of the bit 11 Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 6.2.6 Enabling Mode Coherent 6.2.5 Requestor Access 108 MIPS64® I6400 Multiprocessing Syst

register located at register cluded in the release. cluded is set. This bitThisset must is set. be Section 6.2.8.1, "Clock Domain Domain "Clock 6.2.8.1, Section e CP0 CMGCRBase register. As a refer- e CP0 CMGCRBase register. field in bits 26:23 of this register to determine select the prescaler ratio. the prescaler select companion document in to indicate the virtualthe address base location CPC of CPC Prescale Clock Change Control e prescaler).indicatesof A value 0xFF 1:256 a ratio bit of thisbit register of (bit 8) its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its step 1 of the procedure in . Registers

y of all devices in the system simultaneously. system the in of all devices y CM PRESCALE_CLK_RATIO // an offset of 0x2008 to access the Coherence Enable Coherence the access to 0x2008 of offset an // // register. d the output of the prescaler. d the output of the prescaler. e CPC registers is programmed into th into is programmed CPC registers e be programmed as followsto field (bits 7:0)ratio. to set the clockof A value 0x00 a 1:1 indicates clock ratio field can be changed. be field can ogram these fields, refer to refer fields, ogram these ...... 0x000x010x020x030x04 2 input clock by Divide No prescaling 3 input clock by Divide 4 input clock by Divide 5 input clock by Divide 0xFF 256 input clock by Divide 0xFE 255 input clock by Divide 0xFD 254 input clock by Divide Table 6.2Table CLK_PRESCALE Field of the Encoding Encoding Description PRESCALE_CLK_RATIO_CHANGE_EN CLK_PRESCALE CLK_PRESCALE before the (no between difference input and output frequency of th between the master input clock an between the master orlisd t2, t1 t1, 0x0000_0001 t0, (t1) 0x2008 t0, coherence // Enable plus in t1 address to the base t0 in value // write from CGRBase VA // Create the current clock prescaler ratio. prescaler clock the current 3. Program the The clock prescaler is used to reduce the frequenc is used to reduce clock prescaler The CP0 Interface programmedThe prescaler can be as follows usingglobal the offset address 0x0048. offset 1.the that Verify 2. the the programmer can read Optionally, For an example of how to pr For an example of The 8-bit CLK_PRESCALE field can The 8-bit Change Example — Register Programming Sequence" For more information on this register, refer to theFor more informationthis on register, The base address for the location of th registers. ence, a value ofa value 0x0000_BBDE_0000 ence, is used (MIPS default) Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 6.2.7 Master Prescaler Clock MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 109

CPC CPC Prescale ClockCPC Prescale Clock Domain Change companion document for mplished by programming the by programming mplished MHz,individual program the register located at offset address register located at offset em Programmer’s Guide, Revision 1.00 4 speed. To set the ratio of the clock 4 speed. To the clock prescaler is enabled and the is prescaler the clock I6400 Registers 2 at quarter-speed to save power. the Assume to power. save 2 at quarter-speed te thatisglobal this register and is seenby all shown above, the outputthefrequency of register located at offset address 0x0018. register located For at offset programming the global in the figure. This is acco ce supplied by the clock prescaler, each device can further further can each device prescaler, clock by the ce supplied 1 so it operates at 250 operates so it 1 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ESCALE_CLK_RATIO field in bits 23:16 of this register is a ESCALE_CLK_RATIO cept forbe programmedcan CM, which the frequency with a step 2 of the procedure in the section entitled . // an offset of 0x48 to access the CPC Global Clock Global the CPC access to of 0x48 offset // an register. // Prescale ratio is accomplished by ratio is at full speed. Core 1 is running at 1/ at running Core 1 is at full speed. CALE field is valid. Refer to the CPC Prescale ClockCPC Prescale Change Control ed in the I6400 MPS. In this example, In this I6400 MPS. ed in the CPCLocal Clock ChangeControl its input clock as shown ock Ratio Modification inputthe to clock prescalerAs is 1 GHz. s how to run core 0 at full speed, and core caler for this example: si_ref_clk register located at offset address 0x0048 addressNofollows. register as located at offset generators for core 0 so it operates at 1 GHz, and core 0x0048.the This value CLK_PRESCALE sets to fieldof a value 0x00, indicatingrelationship a 1:1 between the inputbit to clockindi- and the output clock. This value also sets the PRESCALE_CLK_RATIO_CHANGE_EN cate that the value in the CLK_PRES more informationthis on register. lilisd 0x0000_BBDE_0000 t1, 0x0000_0103 t0, t1 into VA value register CPCBase // move 0x48(t1) t0, 4 to ratio divide set and prescaler clock // Enable plus in t1 address to the base in t0 value // write • system 2-core •per 1 VP core • GHz 1 of frequency input si_ref_clk • output Prescaler of 1 GHz • Core 0 input frequency of 1 GHz • Core 1 input frequency of 250 MHz 2.0 is running this example the core In 1. a valueof 0x100 to the global Write cores and all individualcores and all devices (clockin domains) the system. Interface Register program the clock pres To Based on the input clock frequency to each individual devi each individual frequency to Based on the input clock clock divide ratio is set todivide by 4. Note that the PR read-only fieldis updated that hardware by kernel and allows software to quickly this registerread determine to the current clock ratio. In this example this field is ignored. thereducea frequency clock by 1:8,of 1:1 to range ex field (bits 2:0) of each CLK_RATIO By default, the clock prescaler is disabl is prescaler the clock By default, ratio of either 1:1 or 1:2 relative to either ratio of The following example show an example of how to program this field, refer to Programming Sequence Example — Register Change Control following: In this example, the prescaler in this example is also 1 GHz. This also 1 GHz. is example in this prescaler 6.2.8.1 Sequence Programming — Register Example Change Domain Clock Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 6.2.8 Individual Device Cl 110 MIPS64® I6400 Multiprocessing Syst so in this case each so in this CPC Local ClockCPC Local Change register located at offset 0x0028initi- to register located at offset one per clock domain, clock domain, per one tio Core Clock Frequency bit 10 (CLK_CHANGE_ACTIVE)0. If is ine when bit 8 (SET_CLK_RATIO) is 0. If is inebit when 8 (SET_CLK_RATIO) the CPC Local Clock Change Registers Registers Change Clock Local CPC the its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its field (bits 2:0) of the corresponding k changeprogress. is in clock change has completed. At this point, another clock change could // an offset of 0x48 to access the CPC Global Clock Global the CPC to access of 0x48 offset // an register. // Prescale GCRBase from // 0x2018 participatingin3 this example. in the - 0 clock cores is change, which CPC GlobalClock CPC ChangeControl when the clock change has completed. registers. This register is instantiated as This register is registers. shows the programming of the CLK_RATIO shows the programming of the CLK_RATIO CLK_CHANGE_ACTIVEcloc the1, = be requested. SET_CLK_RATIO is 1, the changethestill request is 1, pending. is SET_CLK_RATIO 01 3’b000 3’b100 1:1 4:1 1 GHz 250 MHz register located at offset register locatedaddress 0x0018. at offset Core Value CLK_RATIO Ra Clock •when to determine the CPC_CC_CTL_REG Read • Whenbits are both of these zero, the core has its own register since each core is in its own domain. domain. is in its own core each since register its own has core • to determ register the CPC_CC_CTL_REG Read // Set the core number to 0 in the GCR_CL_REDIRECT register in the GCR_CL_REDIRECT to 0 core number Set the // lisd 0x0000_0000 t2, sync 0x2018 (t0) t2, (1:1 ratio) 0 to field CLOCK_RATIO register CPC_CO_CC_CTL the //Program li to 0 number 0 and VP number to CORE // set sd at register GCR_CL_REDIRECT to contents // store 0x0000_0100 t2, register in the GCR_CL_REDIRECT to 1 core number Set the // 0x4018 (t1) t2, li 0x0000_0001 t2, to 1:1 ratio and set change clock // enable 0x4018 at register CPC_CO_CC_CTL to contents // store to 0 number 1 and VP number to CORE // set #define c0_CMGCRBASE#define $15,3 t0, GCR_BASE_ADDR li t1 into register for CPCBase Store VA // t1, CPC_BASE_ADDR li into t0 value GCRBase // move register CPC_PRESCALE_CC_CTL in the to 1:1 ratio divide clock prescaler Set the // lisd into t1 value CPCBase // move 0x0000_0100 t2, 0x48(t1) t2, to 1:1 ratio set divide and prescaler clock // enable plus in t1 address to the base in t2 value // write Local ClockLocal Change Control ate a clock change for all clock domains all for change clock ate a completed. has clock change the once cleared by hardware is This bit Table 6.3Table Field of CLK_RATIO the Programming 3. the in bit SET_CLK_RATIO the Set Pollfollowing the registers to determine ClockChange Code Example Ratio Table 6.3 Control Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 111

CPC Global companion l state. If the CM is already register is used to optimize used to is register a given device connected to I6400 Registers er. If the CM is not operational at operational If the CM is not er. ersely, the CM is automatically the CM is pow- ersely, wered-upispowered-even if no core em Programmer’s Guide, Revision 1.00 all clock domain ratios are low. For example, if all all clock domain ratios are low. CPC Global Clock Control current state to an operationa ks should not be a problemand leaving itsat this value r Up register (CPC_PWRUP_CTL_REG) located at offset (CPC_PWRUP_CTL_REG) offset located at r Up register its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its e programmer to determine when be used to extend be used todelaystate thedesired. period if the CM by writing a 1 to this regist // 0x2018 copy t2 and 8 with OR bit - logically // register change enable clock the sets t2. This into // back change clock // successful The I6400 allows for the CM to be po ed-up if any core is powered-up.core ed-up if any Conv e value of the delay could The intentbe reduced. is that clock domain changes field29:20 in bits of the more information on this register. on this information more register 0x0040(CPC_ROCC_CTL_REG) at offset is set. Refer to the CPC_CC_CTL_REGCC_DELAY sdsync (t0) 0x2018 t2, (4:1 ratio) to 3 field CLOCK_RATIO register Change Clock Local CPC the //Program lisdat register GCR_CL_REDIRECT to conents // store 0x0000_0103 t2, CPC_CC_CTL_REG 8 of the bit - program change clock based register Initate // (t1) 0x4018 t2, from CPCBase 0x0018 at offset register // ldorito 4:1 ratio and set change clock // enable (t1) 0x0028 t2, at 0x4018 register CPC_CO_CC_CTL contents // store t2, 0x100 t2, sd Loop: 0x0028 (t1) t2, t2 to in register CPC_CC_CTL_REG // load low. 10 are 8 and until bits register control change clock Poll CPC_CC_CTL_REG // REG in the CPC_CC_CTL bit the SET_CLK_RATIO // set ld andi 0x0028 (t1) t2, bne t2, 0x0500 t2, reg CPC_CC_CTL to the t2 back value in new // store r0, loop t2, nop t2 into CPC_CC_CTL_REG of contents // read t2 into copy result 0x0500, t2 and // AND a indicating low, and 10 are bits 8 until // loop Reset Occurred up. This is useful for system debug/setupusefulup. This for system is DBU.the via Interface Register This functionalitycontrolledGlobal is by the CPC Powe address 0x0030. of power-up one-time a execute DBU may The The CM provides a series of read-only bits that allow th Normally, the CM is automatically power CM is the Normally, ered-down if all cores are powered-down. its from transition will it DBU, the by is set bit this time the operational, settingthishas bit meaning no write and the registeris ignored. The document included in the release for the amountofduring delayclock change. a This can be done if current clock ratios are less than 1:4 th do notvery happen setting often, sodefaultthe cloc of 80 defaultcouldrecommended. delay also This register is the CM has including been reset, CPC the itself. Whenever a device is reset, the corresponding bit of the 6.2.8.2 Clock Change Delay Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 6.2.10 Reset Detection 6.2.9 Powerup CM Standalone 112 MIPS64® I6400 Multiprocessing Syst

regis- 0 register. If If a register. register can register VP Run 1 2 VP Run register is set, setting 3 register contains a 4-bit VP Running the type that indicates of register places the VP in register places the VP 4 e bits, kernel software must 5 VP Run s is instantiated per core. per instantiated s is VP Running VP Run e. Writing a 0 to any of the bits in the bits in the of the to any 0 a e. Writing d watchdogd timerfunctionality reset. The ed core is reset. The 16 17 ur. Prior to one of thes setting ur. VP in the system. The I6400Multiprocessingsystem. VP in the system sup- its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its stem. Each of these register Each stem. ains a 2-bit field (RESET_CAUSE) ains to stop a VP. If a given bit in the If a VP. to stop a to set each VP to the run state. The reset whenever the associat register is already set, settingthe corresponding bitthe in running by reading the corresponding bit in the e cold reset, external warm reset, an reset, external warm cold reset, e CPC Global Reset Occurred Register . 31 30 29 register places the VP in the suspend stat VP Running register is cleared, setting the correspondingthe bit in VP Stop , as well as the Debug unit. as well , Figure 6.4 control thisfunctionality: register (RO) Figure Figure 6.4 System Multiprocessing I6400 in the Detection Reset VP Running register (WO) register is a register register used write-only register (WO) register is a Write-only register used register is a Write-only 63 CM Reset DBU Reset DBU CPC Reset CPC register has no effect. register has no effect. CORE4 Reset CORE3 Reset CORE2 Reset CORE1 Reset CORE0 Reset CORE5 Reset VP Stop VP Run VP Stop VP Running VP Run • given bit in the VP Stop Three registerscontrol are used toeach the power state of ports up to four VP’s per core, and up to six cores per sy upandcore,cores per to six per VP’s ports up to four reset for the CPC block.reset foroptions Reset the CPC ar this registerof shown is in In additionIn reset detection,the to this register also cont Register Interface The the corresponding bit in the in the corresponding bit the run state. If a given bitthe run state. If in the also be cleared by hardware also be cleared The • field, where each bit is dedicated to a particular VP, up to fo VP, particular dedicated to a bit is where each field, ensure that the VP in question is already not ter has no meaning.ter in The value this register is • Three registers are used to Three registers Cause of CPC Reset Cause Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 6.2.11 VP Run/Suspend MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 113

a fraction of their register e, kernel software sets e, kernel bit in a given core. These bits are These given core. in a c) to be placed into either Shut- run state using bits 23:16 registers by kernel software as by kernel software registers ks are running at is cleared (logic ‘0’), the RAM’s on is cleared (logic ‘0’), the RAM’s em Programmer’s Guide, Revision 1.00 ty to power cycle its own local RAM VP Stop 2 is already in the Suspend state and no in the Suspend state and 2 is already register. Once awoken, the CPC delays the register. Run state, kernel software sets Run kernel software sets bit state, 2 of the and CPC Local RAM Sleep Control at VP2 is in the Suspend stat VP2 at VP Run the CPC power state the for the device reaches ClockOff the deep sleep state to the its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its CPC_CL_RAM_SLEEP that power running domain at a fraction are of their normal CPC_CL_RAM_SLEEP n state,n kernel software wouldthe do following, at indicates the run state of each VP the run state at indicates e, kernel software wouldfollowing,the do VP’s correspond to the register bits as follows:as to the register correspond bits VP’s f, or Deep Sleep mode where the cloc where the mode or Deep Sleep f, range from 1 to 255 (0xFF) clocks. a given power domain (cores, CM, IOCU, et CM, IOCU, domain (cores, a given power / Shutdown and Wakeup Delay and Wakeup / Shutdown programmed into this field in order to providetime sufficient for the RAMs to domain, so each domain has the abili domain has each domain, so register. If this bit is already If set, is already VP2 runningand no action need be register. cleared, VP this bit is already If register. registerset,indicating is VP2 that is in the register register is cleared, indicating th ) 0x0050located at offset (or 0x2050 relative to the CPC base address). VP Running VP Running register to set VP2 to the Run the state. Run to VP2 to set register VP Running VP Running register is a read-only register th register a read-only register is register to set VP2 to the Suspend state. to the Suspend VP2 register to set VP Run VP Running 2 of the 2 of taken. action need be taken. VP Stop VP Stop CPC_CL_RAM_SLEEP normal frequency. This functionalitycontrolled is the through frequency. normal ( down mode where the clocks are turned of clocks the where down mode 2. If bit 2 of the 2.thebit 2 of If To set VP2 of a given core to the Suspend stat Suspend the to core given a of VP2 set To 1. Read bit 2 of the The CM allows the local RAM’s within local RAM’s allows the The CM The set andby cleared hardware programming based on the the of the local device enter the Deep Sleep low power state when frequency. CPCdelaya way to also providesThe the transition from state. In this state the clocks to the local RAM’s within clockstheIn thisstate to the state. local RAM’s transition to the run state by the value fromwake upcan The delay Deep Sleep. This register is instantiated per power is instantiated This register described above. forthat of these each Note registers, four the •0 = VP0 Bit •1 = VP1 Bit •2 = VP2 Bit •3 = VP3 Bit the Ru to given core of a VP2 For example, to set 1. Read bit 2 of the RAM_DEEP_SLEEP_WAKEUP_DELAY) of the RAM_DEEP_SLEEP_WAKEUP_DELAY) When bitWhen 31 (RAM_DEEP_SLEEP_DISABLE) of the devices. 6.2.12.1 RAM Deep Sleep Mode Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 6.2.12 Local RAM Deep Sleep 114 MIPS64® I6400 Multiprocessing Syst

Rail Core-Local and Core-Local order to meet sys- order to registers. This device reaches the PwrDwn the reaches device e CPC Global Control Block, con- CPC Global Control e is cleared (logic ‘0’), the RAM’s on is ‘0’), the cleared (logic RAM’s domain sequencer machine will state transitioningoperational to the state. ace using the Core number and the Core number and ace using the VP cal and Core-Other s are used to help accommodate a wide used to help accommodate s are tened accordingly in to provide sufficient time for theto RAMs provide sufficient e off. The RAM’s remainShut- in the RAM’s The off. e register. Once awoken, the CPC delays Once awoken, register. re or VP, refer to the section on the to refer VP, re or this register provides the programmer with the ability to counted down to zero. Refer to the section entitled 002, indicating a 2-cycle delay. However, should addi- However, 002, indicating a 2-cycledelay. toexternalassertion delay the of signalsto relative one the CPC power state for the for state power CPC the ese registers to modify the power parameters for a given its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its CPC_CL_RAM_SLEEP CPC_CL_RAM_SLEEP chapter of this manual. this of chapter ) located at offset 0x0008 in th ) located at offset CPC_CL_RAM_SLEEP CPC_CL_RAM_SLEEP registers within the CM sp address ers in Another Power Domain ers in Another ribes the number of each clock cycles wer state changes to ClkOff without wer state changes to ClkOff contains itsown of set CPC Core-Lo lueprogrammed into this field in order stem. Signals can be lengthened or shor n sequencer state machine. These register n sequencer state machine. These and External Signal Delays CM Programming 0x0000x0010x0020x003 delay 1-cycle delay 2-cycle delay 3-cycle delay 4-cycle e local RAM’s within that power domain ar that power domain within RAM’s local e Encoding Description Figure 6.1 CPC_SEQDEL_REG Table 6.4Table MICROSTEP of Field Encoding accessing the CPC registers of another co usage in usage the for more information. Core-Other Register Core-Other The Sequence Delay register ( tem timing. tem number of the device toof be accessed. the devicenumber For more information on domain. This domain. This is accomplished by writing to allows master devices such as a core or IOCU to access th sy constraints in the timing variety of This section describes those register fields that can be used be can that fields register those describes section This another, as well as the internal domai the internal as as well another, Each power domain shown in in shown domain power Each tains a 10-bit MICROSTEP field that desc field a 10-bit MICROSTEP tains take to advance to the next state. to the next state. advance take to 10-bit field MICROSTEP 0x valueThe a default contains of tional delay be required based on the system implementation, increase the sequence delay as necessary. field has Domainsequencing begins RAILDELAY once the The 10-bitfollows:encoded MICROSTEP fieldThe as is Delay the transition to the run state by the va the transition to the fromto wakeupDown the Shut delay The state. from can range(0xFF) 1 to 255 clocks. When bitWhen (RAM_SHUT_DOWN_DISABLE) 15 of the the local device enter the Shutdown low power state when low power state enter the Shutdown the local device state. In the clocks this state to th po if the CPC even down low power state CPCdelaya way to also providesThe the transition from the shutdownthetousingstatestate run 7:0 bits of the RAM_SHUT_DOWN_WAKEUP_DELAY) 6.2.14.1 Global Sequence Delay Count 6.2.12.2 RAM Shut Down Mode Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 6.2.14 Internal Fine Tuning 6.2.13 the Accessing CPC Regist MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 115

signal 1 RailEnable section in the section System should a problem arise where problem arise a should . The power-up sequence starts after . The power-up tuning, the register can be written at for the exact name and usage of this for the em Programmer’s Guide, Revision 1.00 GlobalDelay Sequence Count ount-down to zero has concluded. At IP configuration this register provides this register the programmer with the abilityto to be changed. However, to be changed. However, I6400 Integrator’s Guide Integrator’s I6400 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its for more information. for more information. C to compensateC for slewrail. rates at the gated register are preset. However, for fine register are preset. However, ) located0x010 at offset in the CPC Global Register Block contains a 10-bit ) delays the power-up sequence per domain per sequence power-up the ) delays ...... 0x0000x0010x0020x0030x004 delay 1-cycle delay 2-cycle delay 3-cycle delay 4-cycle delay 5-cycle 0x004 delay 5-cycle 0x3FF delay 1024-cycle 0x3FF delay 1024-cycle 0x3FE delay 1023-cycle 0x3FE delay 1023-cycle 0x3FD delay 1022-cycle 0x3FD delay 1022-cycle Encoding Description Encoding Description Table 6.4Table MICROSTEP of Field Encoding Table 6.5Table Field of RAILDELAY Encoding lustration purposes. Refer to the lustration purposes. Refer to ) used to schedule delayed) used to scheduleof powerstart domain sequencingthe after I6400 Integrator’s Guide I6400 Integrator’s RAILDELAY CPC_RAIL_REG CPC_RAIL_REG RAILDELAY loaded into has been the internal counter and a c signal. has been activated by the CPC. This allows the CP This has been activated by the CPC. RAILDELAY counter field ( time, the contents of the The Rail Delay register ( Rail Delay register The The defaultThe beenregisterfor thisthat valueas the value determined has should by MIPS majority work in theof sys- tem implementations. As such, this value should not need run time. field is encoded as follows: The 10-bit RAILDELAY additionalis delay requiredmeet in order to timing, system increase the delay as necessary. For more informationhow onis refer to the used, this counter Integration chapter of the The 10-bit counterThe 10-bit value ( 6.2.14.2 Rail Delay Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 1. This signal is shown only for il 116 MIPS64® I6400 Multiprocessing Syst

Reset ) at offset at offset rs are used to delay rs are used CPC Global Reset CPC Global Reset CPC_RES_REL_REG section section of the Integra- I6400 should a problem arise where problem arise a should CPC_RESETLEN_REG) . Once this counter reaches 0, the this counter reaches Once . . A series of down-counte register ( nt of delaynt of betweenthe time the configuration Table 6.6 re reset is released. . ed toa secondary internal loaded counter the with value Reset_Hold with a delay value between 1 and 1024 clock The cycles. this register provides this register the programmer with the abilityto reset is active untilreset is activethe domain respondsasserting by the to be changed. However, changed. However, to be Global Sequence Delay Count its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ster — Core Reset Release (RESREL1) Release Reset — Core ster allowing themreset. to comeout of Table 6.6 Table ) at offset 0x0018.programmedregister fieldThis is with a delay ) at offset the CPC Global Reset Release Register ( the CPC Global and the time that the co and the time that the Global Reset Width Counter Global Width Reset and the corresponding hardware signals thatrefer to the can be delayed, ...... 0x0000x0010x0020x0030x004 delay 1-cycle delay 2-cycle delay 3-cycle delay 4-cycle delay 5-cycle 0x3FF delay 1024-cycle 0x3FE delay 1023-cycle 0x3FD delay 1022-cycle to the RESETLENthe to field shown in Encoding Description Table 6.6 Table Field RESETLEN the of Encoding illustration purposes. Refer to the CPC_RESETLEN_REG This register is used to determine the amou ( signal. However, the signal. However, signal is deasserted to the core(s), to signal is deasserted 2 Reset_Hold sectionGuideI6400 the in for more Integrator’s information. Domain_Reset_n encoding of this field is identical encoding located at offset 0x0038. located at offset Bits 9:0 of this register (RES_REL_LEN) are programmed During the power-up sequence, reset is applied. Typically, sequence,is applied. reset Typically, theDuring power-up internal various reset pins usedtovarious reset pinsthe boot CM as described following in the subsections. defaultThe beenregisterfor thisthat valueas the value determined has should by MIPS majority work in theof sys- tem implementations. As such, this value should not need additionalis delay requiredmeet in order to timing, system increase the delay as necessary. For more information on these counters Programming the Global Reset Width Counter Register (RESETLEN) Register Counter Width Reset Global the Programming RESETLEN downThe counter is used to extend the varioussignals resetbits using of the 9:0 Delay Counter Register Width Programming the Global Reset Release Regi Release Reset Global the Programming outputthedescribed RESETLEN counterThe above of is us programmed into the RES_REL_LEN field of value betweenvalue 1 and 1024inas showncycles clock 0x0018 allows reset to be extended beyond the assertion of tor’s Guide for more information of this signal. on the usage information Guide more for tor’s signals are stable signals at the respective core(s), 6.2.14.3 Reset Delay Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 2. only for shown is This signal MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 117

. signal is deas- signal is ) located at offset at offset located ) core(s), allowing the core to core core(s), allowing the load both the RESREL1 and Domain_Reset_n I6400 Integrator’s Guide I6400 Integrator’s ready to begin execution. Note that em Programmer’s Guide, Revision 1.00 has reached zero beforehas reachedcan counting ected once the counts reach zero, refer to reach zero, refer once the counts ected CPC_RES_REL_REG register is used to chapter of the chapter of signal is asserted to the signal is asserted its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ird internalird counter with (RESREL2) value the programmed Domain_Ready System Integration signal, indicating that the core is the core is signal, indicating that CPC_RES_REL_REG requires that the RESREL1 counterrequires that CPC Global Reset Release Register ( Reset Release CPC Global section in the section in Domain_Ready ese counters are loaded and the signals aff and the signals ese counters are loaded d (RES_REL_LEN) of of the d (RES_REL_LEN) GlobalSequence Delay Count begin. Once the RESREL2 counter reaches 0, the begin. Once the RESREL2 the same register fiel the same register into the RES_REL_LEN field of the Programming the Global Reset Release Register (RESREL2) — Ready Register Domain Release Reset Global the Programming outputtheThe counter RESREL1 of a th load is used to RESREL2 counters. counters. RESREL2 The third internal counter (RESREL2) 0x0038. register This to determine is used amount the ofbetween delay the time the serted, and the deassertion of the serted, and the deassertion begin execution. how th on For more information the Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 118 MIPS64® I6400 Multiprocessing Syst

ion in memory, memory, ion in es and code exam- es diate attention. These diate attention. tine is tine is specifically termines the source of the the of source the termines the I6400 MPS for servicing. th register exampl signed its own locat signed its ationvirtualization on as it relatesto ents is accomplished through an inter- is ents rnal events are those that occur within the that occur are those rnal events routine. This rou those whichrequire imme n into then into interrupt controlsystem, allowingseparate ltiples of 8, as well as numerous internal interrupts. The vector and kernel software de numberexternal of interrupts,and configuring individual tointerrupt any within VP and external interruptsthe in I6400 MultiprocessingSys- des in memory. Each time Each an interrupt is detected, event the des in memory. its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its e interrupt service service e interrupt mode, each interrupt type is as each interrupt mode, related Root and in Guest registers. Refer to the chapter on Virtualization ram flowservicing and require to determinethe the typeof for and reason e various elements of the GIC using bo rity events. The servicing of these ev of these servicing The rity events. tting the operatingsetting mode,theup address map, GIC register layout and processes. This chapter contains inform This processes. at originate outside of the I6400 Multiprocessing System and require servicing em incorporates virtualizatio ated from and how they can be resolved. Inte they can be resolved. how ated from and ized by priority. High priority events are High priority events by priority. ized The main difference is that in VI in is that main difference The ples. Some of these elements include se Some of these ples. distribution,settingthe GIC base address, determining the interrupt sources. designed to deal with the interruptevent. chapterThis describesprogram to how th interrupt controllers for guest and root The non-EIC mode includes both Compatibility mode, the most basic type of interrupt mode, and also Vectored Inter- The non-EIC mode includes both Compatibilitythe mode, most basic typeof interrupt mode, also and Vectored mode. rupt (VI) The GIC supports two types of operating modes: • Non-EIC mode • External InterruptController (EIC) mode The I6400 Multiprocessing Syst Multiprocessing The I6400 tem. The system supports up to 256external interrupts in mu GIC is responsible for mapping each internal and external External events are defined as those th The GlobalThe Interrupt Controller (GIC) processes internal to determine whereorigin they I6400 Multiprocessing System. Internal events can include performance counters, watchdog timers, software, and DebugFast Channel (FDC). Interrupts are events which interrupt prog are categor event. Events rupt servicethat resi routine,software a piecewhich of is program flow is interruptedtheandto code branches th interrupts and the programmingthe GIC- of information. more for manual this Compatibilityin whereas mode all interruptstheto go same events are the lower prio handled before events are 7.1.2.1 Non-EIC Mode Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 7.1.2 GIC Operating Modes 7.1.1 GIC Virtualization MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 119 7.1 Overview Global Interrupt Controller Interrupt Global Chapter 7 Chapter

bit. address bit is I6400 Reg- I6400 -Other CONFIG3.VEIC access should be access CONFIG3.VEIC register companion document. rol registers. For more infor- t address spaces as shown in as shown spaces t address her registers her registers are local to a partic- GIC_VL_CTL.EICMODE an index to reference the appropri- to reference an index 0x0018 (physicalof address GIC VP-OTHER ysical address and the VP number of VP number the and ysical address is used to give quick user-mode read to is used give quick user-mode field to select the correct the desired the correct field to select the em Programmer’s Guide, Revision 1.00 overhead of system calls to read GIC overhead of system calls I6400 Registers read-write registerthe bit.to Refer VP-Local GCR GCR Redirect VP-Local of any other VP by using the VP other of any of this pin to set or clear the CP0 field to the and setting the desired VP target e change is reflected in the CP0 e change is number ontointerruptsix the number pinsprior to driving the REDIRECT_VP to access the register set belonging to another VP. The to access the belonging register set to another VP. rruptsencoding by on the value the six interrupt of pins appropriate subset of the conttheappropriate of subset that are located at differen at that are located its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its register located at offset at located register bit to 1 to indicate that a 1 bit to User-Mode Visible Register Visible User-Mode . VP_REDIRECT on on this register. The state of the The state of this register. on on tween all VP’s in the system, while tweenot all VP’s GIC_VL_CTL.EICMODE fied by the VP number information). this section is meant to avoid the to avoid section is meant this re command. The VP number as used is uncached load/store commands. The ph commands. The load/store uncached ve been established through the GIC. Figure 7.1 registers. By using the VP number information, the hardware writes/reads the available to the programmer: GIC_REDIRECT_EN (GCR_CL_REDIRECT) registerinthe em, any VP can access the registers registers the access any VP can em, changes the state of this bit, th the state of changes VP-Local GCR Redirect pin. Hardware then uses the state gisters pertaining to that VP. Software does notneed to explicitly calculate the register gisters pertaining to that VP. ster blocks described above bitboot allowstoup kernel software in non-EIC mode, then switch to EIC mode once by first writing the field to 0x1. Both of these fields are in the of these fields are in the to 0x1. Both field GCR Redirect SI_EICPresent . Some of the registers are . Some of the are shared registers be companion document for more informati companion GIC_VL_CTL.EICMODE “Other” VP is specified is “Other” VP GIC_REDIRECT_EN Also in this manual, an additionalcalled section the of The use registers. specific GIC to access registers. resources, such as counter Figure 7.2 ular VP. This relationship is shown in ular VP. mation, refer to the VP subset of registers and set the subset VP redirected. indexthisvaluetothe register of The is used by hardware The The GIC contains various regi The The GIC address space is accessed with The GIC address space •VP an “Other” VP that allows that for window second A indexthequestion. core in for entirely This donehardware. by “windows” are address made Two The EIC modeprovidesThe support63 for up to individual inte supplied for each load/sto is the requester ate subset the instantiated control of correct subset of the control re •speci “Local” VP (as window for the A error. In addition, for both Compatibility anderror. VI modes, the six interrupt pins on the are usedVP as individual inter- in the EIC mode. an encoded value as not are and rupts, each VP. The GIC is responsible for encodingThe GIC is proper the vector each VP. tovalue VP. the appropriate Software can enable EIC mode by setting the In the I6400 Multiprocessing Syst the I6400 Multiprocessing In 0x1FBF_A018) beforethe accessing address spaces. Set these driven onto the bit. Therefore, if kernel software kernel if bit. Therefore, isters the appropriate interrupt connections ha interrupt connections the appropriate spaces. Software must write the 7.1.2.2 EIC Mode Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 7.1.4 GIC Register Distribution 7.1.3 GIC Register Types 120 MIPS64® I6400 Multiprocessing Syst

often that it makes her VP. by which the VP. her VP23 for more information. for more s in the system using this The use of this sectionThe use of this is address space. address space. Software ular VP, the EIC encoder is the EIC VP, ular GIC_VL23_xxx GIC_VO23_xxx privileged system virtual address Figure 7.1.2 Figure counter registers. Currently, the only counter registers. Currently, ers that are read so by hardware to index the appropriate -Local section of anot registered, masked, and assigned to a particu- to and assigned registered, masked, are registered, masked, and assigned to a par- to a assigned masked, and are registered, an example base address of 0x1BDE_0 is used. out requiring a system call. call. a system out requiring register. Refer to Refer register. IC) mode is used for a partic rticular interrupt pin of another VP. Using the Using VP-other rticular interrupt pinVP. of another her VP by using the VP-Other VP by using her VP can setup the GIC for all VP for more information on the derivation of these addresses. gister block, and the corresponding within register offset VP2 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its GIC_VL2_xxx GIC_VO2_xxx I6400 GIC sections are meant to be located in shared by all VPs and all cores in the system. VPs and all cores all by shared e value of this register is used value of e initialize and update the interrupt controller. interruptthe and controller. update initialize sters. In this figure In sters. (shared among all VPs) register located 0x0018 at offset (physical address of 0x1FBF_A018) calls to read GIC resources, such as calls to read GIC resources, -Other GIC_SH_COUNTER section that contains aliases for GIC regist GICRegisters Shared — GIC_SH_xxxx VP1 , and VP , and Figure 7.1Figure Distribution GIC Register GIC_VL1_xxx GIC_VO1_xxx section in which interrupts local to a VP are -Local section in which the local VP can access the VP section in which the local VP can access ace Configuration sectionwhich in the external interruptsources I6400 Technical Reference Manual Reference I6400 Technical , VP -Local -Other VP-Local GCR Redirect Shared Shared User Mode Visible Mode User VP Shared VP0 shows the mapping of the regi GIC of the the mapping shows GIC_VL0_xxx GIC_VO0_xxx meant to avoid the overhead of system register aliasedintospace this is the Register before accessing these spaces. Th Register before accessing subset of thefor the otherOne controlregisters core(s). sense to make them available to user-mode programs with programs to make them available to user-mode sense must write the the write must section. segment, the "local" VP can access the registers of anot the registers access can VP the "local" segment, interrupt can be registered, masked, and assigned to a pa and masked, registered, can be interrupt ticular VP and interruptThissectionpin. is lar interrupt pin. If External Interrupt Controller Mode (E instantiated here. • A 64 KByte The GIC address space is divided into four blocks: GIC address The • 32 KByte A Figure 7.2 that block. Refer to the space, in whichspace, in only kernel mode software can Each register is mapped using the GIC base address, the re In the GIC, the • A 16 KByte VP •KByte A 16 Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 7.1.5 GIC Address Sp MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 121

GIC Shared Block (32 KBytes) GIC Core-Local Block (16 KBytes) GIC Core-Other Block Core-Other GIC (16 KBytes) GIC User Mode Visible Block GIC User Mode Visible (64 KBytes) em Programmer’s Guide, Revision 1.00 0x0000_1BDC_7FFF 0x0000_1BDC_0000 0x0000_1BDC_8000 0x0000_1BDC_BFFF 0x0000_1BDD_0000 0x0000_1BDC_C000 0x0000_1BDC_FFFF its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its 0x0000_1BDD_FFFF +0x4000 +0x8000 +0x4000 +0x10000 e high or active low ess and Enabling the GIC 17 • dual or singlerising) edge (falling or Edge Sensitivity, • activ Level Sensitivity, GCR_GIC_BASE Register GCR_GIC_BASE 47 This section covers the programming for the following tasks. • SettingBase Addr the GIC • Routingof external interrupts specific to processors • Enablingdisabling or interrupts • Inter-Processor Interrupts (IPI) • Localinterrupt device configuration • Configurationof interrupt sources • External interruptsource configuration Figure Figure 7.2 of 0x1BDC_0 Address Base Example Using an Scheme Addressing Register GIC Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 122 MIPS64® I6400 Multiprocessing Syst 7.2 Programming GIC

. The Shared

a1 GCR_GIC_BASE register. This register is register. companiondocument . This examples assumes . This examples assumes a0 the system. This code reads This the system. number of external interrupts register in CM address space, then instruction. ers. The GIC base address is a 31-bit is a ers. The GIC address base li . For more. information, refer to the GCR GIC Base I6400 Registers instruction. instruction. Then bit 0 is set, which enables sw

li the GIC which is loaded the GIC which into is field in the the in field r of external interruptsr in GIC_SH_CONFIG the value of the register into onfigured at build time. This its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its NTERRUPTS field in bits 23:16. Interrupt sources are config- and sets the GIC enable and sets the GIC bit. GCR_GIC_BASE External Interrupts in the System External Interrupts se Address Register intose Addressa1 using Register the the base address of the GCR_GIC_BASEthe base address of the dress and Enabling the GIC Enabling the dress and address of the Shared section of of the Shared section address address of the GIC memory-mapped regist of the GIC memory-mapped address chapter for more information on this register. register. information on this more chapter for the following defines to make the code easier to read: easier make the code to defines the following hysical address of the GIC using hysical address of the GIC using the at offset 0. The code loads The code 0. at offset the GCR_GIC_BASE register using the GCR_GIC_BASE the (GIC_SH_CONFIG 0x0000) at offset in the terrupts inthe system. register and isolates the NUMI the and isolates register C base address into the register address into C base Coherence Manager li a1, GCR_CONFIG_ADDR + GCR_GIC_BASE GCR_CONFIG_ADDR a1, li li a0, (GIC_P_BASE_ADDR | 1) // Physical address + enable | 1) // Physical (GIC_P_BASE_ADDR a0, li 0(a1) a0, sw in the #define GCR_CONFIG_ADDR 0xffffffffbfbf8000 // KSEG1 address of the GCR registers of the GCR address KSEG1 // 0xffffffffbfbf8000 GCR_CONFIG_ADDR #define the GCR address of Boot // Post 0xffffffffbfbf8000 GCR_CONFIG_ADDR_PB #define registers GIC of the address physical // GIC_P_BASE_ADDR0x000000001bdc0000 #define the GIC of address KSEG1 // GIC_BASE_ADDR0xffffffffbbdc0000 #define GIC of the address Boot Post GIC_BASE_ADDR_PB0xffffffffbbdc0000// #define 16 NUMINTERRUPTS #define 8 NUMINTERRUPTS_S #define GIC_SH_CONFIG GIC_SH_CONFIG value that is programmed into bits 47:17 of the located at offset address 0x0080 Globalthe in offset located Controlregisters.theBlock at to the Refer CM of Global Configuration Register the release. contained in example used to determine the numbe code following is a The the The code then loads a0 with the p Register Interface Register The numbera fixed value of external interrupt c sources is The code loads the address of the GIC Ba of the GIC address the code loads The in the system is stored in the "GIC Configuration Register", Register Interface Register the starting is GIC base address The ured in the core inin groups the core indicatesured field of 8. Thishow many has. groupscore of 8 the The define GIC_BASE_ADDR is the loads loads the physical GI Example Code Address Base GIC the Setting uses example code The following Configuration register is located there are 40 external in The following code example The following code determines Register Register the GIC. This value is stored to is stored value GIC. This the Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 7.2.2 Determining the Number of 7.2.1 Setting Base Ad the GIC MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 123

I6400 Regis- time by processor 0, or by 0, by processor time ared by all processors. While . This example is 40 expecting is . This example a3 em Programmer’s Guide, Revision 1.00 pin. pin Hardware uses this to update the disable this mode. disable (GIC_VL_CTL at offset 0x0000) (GIC_VL_CTL at offset in the gisters in the GIC that are sh SI_EICPresent the EIC_MODE bit in the Local interrupt Control Register, are usually programmed at boot ect this value, it executes a debug breakpoint to stop at a to stop breakpoint debug it executes a value, ect this its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its at is determinedIP configuration during time.GIC The and accordingly to enable or accordingly to enable expected value expected of NUMINTERRUPTS into bit to indicate support for and status of the EIC mode. Local Interrupt Control Register Local Interrupt Control mberof interrupt groups. 8 interrupts giving 40 interrupts. C_MODE bit is driven onto the Config3.VEIC ext a0, NUMINTERRUPTS, NUMINTERRUPTS_S NUMINTERRUPTS, NUMINTERRUPTS_S ext a0, NUMINTERRUPTS //extract 4 a3, li a3, configure_slices a0, bqe nop of 40 interrupts Failed assertion // sdbbp li a1, GIC_BASE_ADDRli (a1) a0, GIC_SH_CONFIG lw // GIC_SH_CONFIG KSEG0 Address GIC // load // Read CTL local Read CTL // a1, GIC_BASE_ADDR li a1, 0x08000 a1, daddiu GIC_VL_CTL // read a0, GIC_VL_CTL(a1) lw KSEG0 Address GIC space // load addressing for local offset // add CP0 Config3 and read 0) in GIC_VL_CTL (bit Set EIC_MODE // li a2, 0x1 0, 1 a0, a2, ins a0, GIC_VL_CTL(a1) sw 3 a2, $16, mfc0 GIC_VL_CTL // write bit 0 // set 3) 16, select (reg Config3 // read companion document in included the release. cores of the system are programmed by hardware are programmed the system of cores Notethat the interrupt mode a systemis setting wideth state of the CP0 The triggering of interrupts is configured through several re EIC Mode Setting Code Code Example Setting EIC Mode Register Interface Register EIC mode is controlled through kernel software by setting Note that the state of the EI GIC_ VPi_CTL. Setting this bit enables EIC mode for that VP. This bit defaults to 0,defaults vectoredto This bitinterrupt For mode. GIC_ VPi_CTL. Setting thisbit enables EIC mode for that VP. more information, refer to the ters Number of External Interrupts Code Example Code Interrupts of External Number "slices"5 of GIC is // Verify all processors can access these registers, in practice they these registers, all processors can access Then the code extracts the nu Then the code loads the example, the code For this pointdebug where can be used to evaluate probe a the problem. interrupt sources (4 + 1 times1interrupt + (4 sources 8). If the code does det not Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 7.2.4 Configuring Interrupt Sources 7.2.3 EICSetting Mode 124 MIPS64® I6400 Multiprocessing Syst

normalized to ng interrupt char- llotted in groups of specific interrupt in :0, the next 127:64, and :0, the next 127:64, d. In this cased. contents the nd active low. In this case the nd active low. ng and falling edges of the edges falling ng and and active high. In this case this high. In active and iggered on the rising edge of edge of the rising on iggered iggered iggered on the falling edge of _DUAL have no meaning ) and dual edge control regis- the interrupts are the interrupts gering is enabled. gering is gering is enabled. gering is at control the interrupttriggering con- n bit, the correspondi oup. Interrupts are a r setting the interrupt register and which dual-edge-sensitive using the polarity con- characteristics of each characteristics negative (asserted low) polarity. Similarly, Similarly, negative low)(asserted polarity. Each register in a group is 64 bits so each reg- 64 bits so each group is in a register Each GIC_SH_TRIGx_y would control interrupts 63 would control interrupts of the GIC_SH_POL have because no meaning inter- the rupts occur on risi both signal. the contents of the GIC_SH_DUAL no meaning have trig level because signal. the signal. the contents of the GIC_SH contents of trig level because and falling edges are used to set the interrupt register. and to the edges are used falling set interrupt register. h bit in the register configures the corresponding interrupt its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its bal Interrupt Trigger Type registers", GIC_SH_TRIGx_y. The registers", GIC_SH_TRIGx_y. Type bal Interrupt Trigger from the GIC to the VP, all of from the GIC to the VP, . The ‘n’ in the table entries denotes that it can be any bit of a given terrupttype supported by the CPU interrupt inputs. bit in each register group. in each bit Single/Dual Edge (GIC_SH_DUAL[n]) Description esponds to an interrupt. So for give a Table 7.1 rst register in each group (OS). There (OS). There are groups three register th l-sensitive, single-edge-sensitive, or register denotes register which edge is used fo terrupt sources there are 4 terrupt sources there in each gr registers ), the trigger typecontrol registers ( Polarity conjunction with one another to define the can have either positive (asserted high) or same bit of each register. same bit of each register. ). When interrupts are driven Trigger (GIC_SH_TRIG[n]) GIC_SH_POLx_y Table 7.1 Triggering and Table Sensitivity, Edge Polarity, Interrupt Selecting GIC_SH_DUALx_y registers (

1x 1 1 0 1 Interrupt is single edge tr Interrupt is dual edge triggere 0 1 0 Interrupt is single edge tr 0 0 x Interrupt is level sensitive a 1 0 x level sensitive Interrupt is positive, level-sensitivethis signals is the in as For single-edge signaling, the the boot code for the operating system figuration. •type register group Trigger • Edge typeregister group •group register Polarity Each interrupt by source is one represented trigger type can be set to level or edge sensitive. Setting eac Register Interface Register The trigger type register group is made up of four "Glo edge is ignored. For double-edgedsignaling,bothrising the work in three registers These corr of each register the system. Each bit register, but must be the but must register, ters ( 8, from256. to 16 8, sources of the interrupt Each leve can be either any of these sources ister controls 64 sources. The fi interrupt so on. Since there can in be 256 trol acteristics would be defined as shown in Polarity 7.2.4.1 Register Group Type Trigger Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies (GIC_SH_POL[n]) MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 125

companion doc- companion rising edge togglerising and set- SH_POLx_y. This register register This SH_POLx_y. bit in this register group in this register bit (GIC_SH_POLx_y) inthe I6400 Registers should be moreset. For informa- em Programmer’s Guide, Revision 1.00 bit configures the corresponding inter- Global InterruptDual Edge Registers Polarity Registers", GIC_ l sensitive. For example, to set the interrupt source 64 to register group has no effect if the edge type was set to dualif the edge type was groupregisterhas no effect ource bit configures the source to its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its single or dual edge. Setting each (GIC_SH_TRIGx_y) in the itive,of the second Global0 bit Dual Edge register GlobalPolarity Interrupt Registers // trigger bits for interrupt sources 0 - 63 sources for interrupt bits // trigger address be uncached NOTE: must // registers. companion document. r more information, refer to the in the previous section is set to if the edge sensitive trigger and has no effect ource to be dualbetoclearing edge anditit to be single configuresource For edge. sensitivity of the source. Setting each source. sensitivity of the of four "Global Interrupt C_SH_TRIG registerC_SH_TRIG (GIC_SH_TRIG127_64) must be uncached address // NOTE: it configures itlow. to be activeit configures I6400 Registers Global Interrupt Trigger Type Registers Type Global Interrupt Trigger companion document. #define GIC_SH_TRIG63_0 0x0180 0x0180 GIC_SH_TRIG63_0 #define for base address GIC the from // offset GIC_BASE_ADDR a1, dli 0x0000000080000000 a0, dli of the GIC address base virtual load // (bit 31) source 31 // interrupt a0, GIC_SH_TRIG63_0(a1) sd sensitive) // (edge ument. Code Example Type Trigger followingThe code exampleprograms interrupt sourcebe edge-sensitive. 31 to to be edge sensitiveconfiguresitclearing and it to be leve edge sensitiveof the second0 bit GI tion, refer to the ting clearing it configure it to be falling edge toggle. This edge sensitive. For more information, refer to the Register Interface Register polarityThe registermade groupup is Register Interface Register This register edgegroupmade typeThe register is up of four "GlobalGIC_SH_DUALx_y. EdgeRegisters", Dual group is used if the trigger type described can be either type The edge set to level sensitive. is type interrupt s configures the corresponding example, to set interrupt source 64 to dual edge sens (GIC_SH_DUAL127_64)Fo should be set. (GIC_SH_DUALx_y) in the CodeEdge Example Type followingThe code exampleprograms interrupt sourcebe 31 to dual-edge sampled. dual for the GIC base address // offset from 0x0200 GIC_SH_DUAL63_0 #define 0 - 63 sources interrupt bits for registers address of the GIC // load virtual base GIC_BASE_ADDR a1, dli 31 (bit 31) // interrupt source 0x0000000080000000 a0, dli // Dual GIC_SH_DUAL63_0(a1) sd a0, group is used to determine the polarity high, and clearing rupt to be active If the interrupt is single-edge-sensitive, then setting the s I6400 Registers 7.2.4.3 Register Group Polarity Type 7.2.4.2 Register Group Edge Type Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 126 MIPS64® I6400 Multiprocessing Syst

external interrupt pins and pins and external interrupt AP_VP —AP_VP maps the interruptspecific to VP number. a terrupt source. The mapping of terrupt source. The mapping its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its External Interrupt Offset Register Name //polarity bits for interrupt sources 0 - 63 sources for interrupt bits //polarity address be uncached NOTE: must //registers . Table 7.2 fic input on a specificfic inputVP is controlled a on registers. 2 by the setting of GIC_SH_MAP8_VP - GIC_SH_MAP8_VP - GIC_SH_MAP8_PIN - GIC_SH_MAP247_VP GIC_SH_MAP247_PIN Table 7.2Table Interrupts on External Based Mapping Register 0x0520 0x05000x0504 GIC_SH_MAP0_PIN0x0508 GIC_SH_MAP1_PIN GIC_SH_MAP2_PIN0x0510 0x08E00x0514 0x08E4 GIC_SH_MAP248_PIN GIC_SH_MAP4_PIN0x0518 0x08E8 GIC_SH_MAP249_PIN GIC_SH_MAP5_PIN GIC_SH_MAP250_PIN GIC_SH_MAP6_PIN 0x08F0 0x08F4 GIC_SH_MAP252_PIN 0x08F8 GIC_SH_MAP253_PIN GIC_SH_MAP254_PIN 0x050C GIC_SH_MAP3_PIN0x051C 0x08EC GIC_SH_MAP251_PIN GIC_SH_MAP7_PIN 0x08FC GIC_SH_MAP255_PIN 0x08DC 0x3EC0 - 0x3EC0 dli a1, GIC_BASE_ADDR a1, GIC_BASE_ADDR dli 0x0000000080000000 a0, dli a0, GIC_SH_POL63_0(a1) sd 31) (bit 31 source // interrupt the GIC of base address virtual //load for 31) // (high/rise #define GIC_SH_POL63_0 0x0100 0x0100 GIC_SH_POL63_0 #define for base address the GIC from //offset 01 0x20002 0x2020 GIC_SH_MAP0_VP3 0x2040 GIC_SH_MAP1_VP 2484 0x2060 GIC_SH_MAP2_VP 249 0x3F005 0x2080 GIC_SH_MAP3_VP 250 0x3F206 GIC_SH_MAP248_VP 0x20A0 GIC_SH_MAP4_VP 251 0x3F407 GIC_SH_MAP249_VP 0x20C0 GIC_SH_MAP5_VP 252 0x3F60 GIC_SH_MAP250_VP 0x20E0 GIC_SH_MAP6_VP 253 0x3F80 GIC_SH_MAP251_VP GIC_SH_MAP7_VP 254 0x3FA0 GIC_SH_MAP252_VP 255 0x3FC0 GIC_SH_MAP253_VP 0x3FE0 GIC_SH_MAP254_VP GIC_SH_MAP255_VP The routing of interrupts to a speci •GIC_SH_M Global InterruptMapto Processor register, •GIC_SH_MAP_PIN maps —signal interrupt a VP. on Global specific to a InterruptMapto Pin Register, in external each for each of these registers There is one of Polarity Type Code Example Type Polarity followingThe code exampleprograms interrupt source 31 for high active or rising edge: the registers that control themin is listed 8 - 247 0x2100 - Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies External Interrupt Offset Name Register 7.2.5 Interrupt Routing MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 127

view of the Root in virtu- Number Core and VP the interruptsthein system, between terrupt to be mapped to up to a maxi- to up mapped be terrupt to . These registers map each external map each external . These registers em Programmer’s Guide, Revision 1.00 the actual core and VP number in the system. Note in the number VP and actual core the ore, and 16 external interrupts, where external interrupt rtualized mode, as well as the tion, refer to the Global refer tion, Interruptto Map VP registers based on the number of VP’s per core. For example, if if example, For core. per VP’s of number the on based its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its P_VP Field to Core and VP Number the system, the mapping would be as follows: companion document. bit field which allows each external in allows each external which field bit to VP Registers", (GIC_SH_MAPi_VP) VP to Number MAP_VP Bit e system. The ‘i’ indicates the number of ‘i’ indicates the number The system. e Core and VP I6400 Registers 0123 VP0 Core0, 4 VP1 Core0, 5 VP2 Core0, 6 VP3 Core0, 7 VP0 Core1, 8 12 VP1 Core1, 9 13 VP2 Core1, 14 VP3 Core1, 15 VP0 Core2, VP0 Core3, 16 VP1 Core2, VP1 Core3, 17 VP2 Core3, 18 VP3 Core3, 19 VP0 Core4, 20 VP1 Core4, 21 VP2 Core4, VP3 Core4, VP0 Core5, VP1 Core5, 11 Core2, VP3 23 VP3 Core5, 10 Core2, VP2 22 VP2 Core5, MAP_VP Bit Table 7.3Table MA of Mapping Physical shows physical the mappingof the MAP_VP field to The following examplesystemshows a with 2 cores, 2 VP’s/c 4 is mapped to core 1, VP1. that thethatthis encodingandfixed field of does not change is there are two VP’s per core and two cores in and two cores per core VP’s two are there •0 in core MAP_VP 0 and 1 would bits therepresent two VP’s • MAP_VP 2 and 3 would bits not be used •1 in core MAP_VP 4 and 5 would bits therepresent two VP’s •bitsother All the in MAP_VP field are unused thatNote thismapping view scheme represents the in non-vi alized mode. interruptspecificsource to a VP in th a 24- contains Each register 256. 16 and Register Interface Register Map "Global Interrupt 256 There are mum of 24 VP’s (6 cores x 4 VP’s/core). For more informamore For (6 cores x 4 VP’s/core). 24mum VP’s of (GIC_SH_MAPx) inthe Table 7.3 7.2.5.1 Mapping an Interrupt Source to a VP Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 128 MIPS64® I6400 Multiprocessing Syst

VP0 VP1 VP0 VP1 CORE 0 CORE 1 CORE individual interrupt can individual interrupt can ofregisters, one per 256 the first example, external external the first example, rrupt pin on a core. The ‘i’ indicates the a core. The ‘i’ indicates on rrupt pin interrupt levels. In non-EIC mode, each interrupt levels. In non-EIC value of 0x00_0020to as 1 select Core 1, VP described above, each external interrupt to be mapped. are interpreted. In External Interrupt are interpreted. Controller (EIC) d 16 externalinterrupts. In Hence there are a maximum are Hence there 0 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its rmation on how the 24-bit refer MAP_VP field is organized, 1 2 3 4 e that can decode up to 64 different up to 64 different e that can decode 5 t pin of a given VP. There are 256 "Global There are 256Interrupt Pin Registers" Map to givenoft pin VP. a 6 p each external interrupt p each external to a inte specific 7 rrupt pin 3 of core 1, VPcorerrupt 1. 1, 3 of pin 8 GIC 9 contains contains 6-bit field which allows a each ng each external interrupt to a particular VP as to ng each external interrupt 10 Hardware Mapper External Interrupts External GIC_SH_MAP4_VP 11 Figure 7.3Figure Core 1, VP a 4 to 1 Interrupt External of Mapping Example 12 , the GIC_SH_MAP4_VP, programmedis register with a . 13 14 24’b0000_0000_0000_0000_0000_0010_0000 Figure 7.3 Table 7.3 15 also be mapped to a specific interrup also be mapped to a specific (GIC_SH_MAPi_PIN) used to ma ofnumber the interrupts the in system, 256.and8 between register interrupt. Each The type of interrupttype of The modehow determines the interrupt pins mode, the 6-bit field is an encoded valu 6-bit field is an the mode, individualinterrupt,an pinallowing is total forof a six interrupts. 1 — Non-EIC Example Mode Configuration Interrupt an followingThe examples show a system with 2 VP’s/core interrupt4 is mappedinte to the destination VP for external interrupt 4. For more info to Register Interface Register In addition to mappi In 7.2.5.2 Mapping an Interrupt Source to a Specific Processor Pin Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 129

VP0 VP1 VP0 VP1 CORE 1 CORE 0 CORE SI_Int[5] SI_Int[4] SI_Int[3] SI_Int[2] SI_Int[1] SI_Int[0] SI_Int[3] SI_Int[2] SI_Int[1] SI_Int[0] SI_Int[5] SI_Int[4] SI_Int[5] SI_Int[4] SI_Int[3] SI_Int[2] SI_Int[1] SI_Int[0] SI_Int[5] SI_Int[4] SI_Int[3] SI_Int[2] SI_Int[1] SI_Int[0] the first example, external external the first example, specific VP up to 24 in the sys- of 0x03, which selects interrupt pin em Programmer’s Guide, Revision 1.00 Core VP 11, as the destination VP for exter- d 16 externalinterrupts. In it 31 ofit 31 the GIC_SH_MAP4_PIN to register 1 to is set bit in this field represent a bit in this field represent Indicatesmode non-EIC its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its 0 0 0 0 0 1 register is programmeda value register with is 2 to 0 to indicateinterrupt non-EIC mode. of core 1, VP 1, interruptVPcore 1, 15. level 1, of 3 4 MAP[5:0] = 6’b000011 5 5 6 7 for the core/VP mapping for the core/VPthis register. of 8 I6400 GIC I6400 9 GIC_VL5_CTL Hardware Mapper 10 External Interrupts External Table 7.3 GIC_SH_MAP4_VP GIC_SH_MAP4_PIN , the GIC_SH_MAP4_VP register is programmed to select 11 12 13 Figure 7.4 14 indicate the external interrupt corresponds toan interrupt and not an NMI. Note that the MAP field is an encoded a binaryrepresentsandvaluevalue for interruptof pin the VP. 3 0 of the GIC_VL5_CTLBit register is set 2 — EIC Mode Example Configuration Interrupt an followingThe examples show a system with 2 VP’s/core The MAP field of the GIC_SH_MAP4_PIN B VPthethe selected by GIC_SH_MAP4_VP3 of register. interrupt4 is mappedinterruptthe to pins tem. Refer to Refer tem. nal interrupt 4. Note that this is not an encoded value. Each In Figure Figure 7.4 1, VP 4 to a Core 1, — Non-EIC Mode Int 3 Interrupt External of Mapping Example 1 24’b0000_0000_0000_0000_0000_0010_0000 23 31 31 Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 15 130 MIPS64® I6400 Multiprocessing Syst

VP0 VP1 VP0 VP1 CORE 1 the GIC by that are shared CORE 0 CORE SI_Int[5] SI_Int[4] SI_Int[3] SI_Int[2] SI_Int[1] SI_Int[0] SI_Int[3] SI_Int[2] SI_Int[1] SI_Int[0] SI_Int[5] SI_Int[4] SI_Int[5] SI_Int[4] SI_Int[3] SI_Int[2] SI_Int[1] SI_Int[0] SI_Int[5] SI_Int[4] SI_Int[3] SI_Int[2] SI_Int[1] SI_Int[0] 31 of the GIC_SH_MAP_PINreg- corresponds to externalinterrupt 15. presents a specific VP up to 24 in the to VP up a specific presents Core VP 11, as the destination VP for exter- 001111 lue in this field represents one less than the actual one less represents field lue in this interruptthis mode. mode, In interrupt the level sent to rammed with a value of 0x0E, which routes the encoded ed through several registers in AP4_VP register. It is important AP4_VPto note register. that when pro- Indicates EIC mode its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ents interrupt levelBit 15. 0 0 0 0 1 1 pt corresponds to an interruptnot and an NMI. 2 the actual interruptthe actualdescribed level as above. 3 indicates the value on the interrupt bus on the interrupt indicates the value encodedEach bitvalue. in this re field 4 MAP[5:0] = 6’b001110 5 5 6 groups for enabling, disabling and polling of interrupts. 7 GIC 8 9 GIC_VL5_CTL 10 Hardware Mapper External Interrupts External GIC_SH_MAP4_VP GIC_SH_MAP4_PIN , the GIC_SH_MAP4_VP register is programmed to select 11 12 13 Figure 7.5 24’b0000_0000_0000_0000_0010_0000 14 The enabling,The polling disabling and is of interrupts configur Bit 0 of the GIC_VL5_CTL register is set to 1 to indicate EIC encodeda 6-bitbetween value VP is 63, 0 and with 0 meaningthe target interrupts.no In this example, a value of 6’b001110 ister is set to 1 to indicateis set to 1 toexternalthe ister interru The value in the register is one less than system. MAPThe field of the GIC_SH_MAP4_PIN register is prog interruptlevelbyselected to of 15 the GIC_SH_M the VP gramming the GIC_SH_MAPi_PIN registers in EIC mode, the va EIC interruptlevel.In this case, a value 0x0E repres of nal interrupt4.that Note is this not an In all VP’s. all VP’s. registers There are 4 shared • Enabling an interruptusing the "GIC Set Mask Registers", GIC_SH_SMASK 1 23 31 31 Figure 7.5Figure 15 — EIC Mode VP 1, a Core Int 4 to Level 1, Interrupt External of Mapping Example Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 15 7.2.6 Enabling, Disabling, and Polling Interrupts MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 131

source bit enables the inter- ding interrupt is disabled. register group. Each register group. Each in register a em Programmer’s Guide, Revision 1.00 the actual number of interrupts may be t to one resets and disables the correspond- ed by reading the NUMINTERRUPTS field of ed by reading the NUMINTERRUPTS sources there are 4 registers in each group. in there are 4 registers sources ite-only register. Setting the ite-onlyregister. interrupts. It is madeinterrupts. up of "GIC is It Registers", Set Mask rst register in each group would controlgroup wouldeachsources interrupts rst register in If it is clear the correspon it is If up of four write-only "GICReset Mask Registers": its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its upt state using "GIC Mask Register", GIC_SH_MASK Register", "GIC Mask upt state using ed to disable external interrupts. The GIC supports a maximum of 256 exter- determineexternal if an interrupt is enabled.madethe following It is up of t source is represented by one bit in each in one bit by is represented t source xed valuexed configured at buildtime, so registers are read-only. registers are read-only.

er of system interrupts can be determin

esponds to external an interrupt. Setting a bi

d so on. Since there be 256 can interrupt GIC_SH_MASK_127_64 GIC_SH_MASK_191_128 GIC_SH_MASK_255_192 GIC_SH_RMASK_191_128 GIC_SH_RMASK_255_192 GIC_SH_RMASK_127_64 Register Interface Register The GIC Reset Mask register group is us rupt. nal interrupts.GIC Mask register Reset The group is made GIC_SH_SMASK. For synchronizationFor GIC_SH_SMASK. purposes a wr is this Register Interface Register external register group is used to enable GIC Set Mask The The number of interrupt sources is a fi is a interrupt sources number of The less than 256.actual The numb Configurationthe "GIC GIC_SH_CONFIG. Register", • • GIC_SH_MASK_63_0 • • GIC_SH_RMASK_63_0 • • Polling the interruptactive state using the "GICPending Register", GIC_PEND_MASK interrup each Like the trigger registers, group is 64 bits so each controls 64 interrupt sources. The fi an 127 - 64 0 - 63, the next • Disabling an interrupt using the "GIC Reset Mask Registers", GIC_SH_RMASK •of an interr state the Enable/Disable Determining Register Interface Register GIC Mask register used tois The group • Each bit in these registers corr ing interrupt. These GIC_SH_MASK register. • • correspondingset thea bit is If interruptenabled.source is 7.2.6.2 Disabling ExternalInterrupts 7.2.6.1 Enabling External Interrupts 7.2.6.3 Determining the Enabled or Disabled Interrupt State Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 132 MIPS64® I6400 Multiprocessing Syst

riate VP. The hard- riate VP. ese bits are set by hardware bits are set ese hardware by rrupt to the approp t to the appropriate VP and the register isset by hardware. If the correspond- register . interrupt level is 15. interrupt level is VP ternal interrupt is active. Th If it is clear the correspondingis clear If itinterrupt inactive. is its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its GIC_SH_PEND up ofup the following GIC_SH_PENDregisters.read-only registerthesend to interrup bit is set, the GIC delivers the inte the delivers GIC the set, is bit register are set, enabling all 64 interrupts. enabling register are set, . ponding bit in the 0, VP 1, and the receiving VP 1, 0,

GIC_SH_MASK

GIC_SH_MAP_VP Figure 7.6 GIC_SH_SMASK register to set the interrupt pins for that that for pins interrupt the set to register upt occurs, the corres •core The receiving VP is • External interrupt 8 is asserted •bits All of the GIC_SH_PEND_255_192 GIC_SH_PEND_127_64 GIC_SH_PEND_191_128 • When an interr • correspondingset thea bit is If interruptactive.source is In the following example: ing interrupt enable bit in the GIC_SH_MAP_PIN • GIC_SH_PEND_63_0 • Register Interface Register GIC PendingThe registerdetermineis to used groupa ex if when an interruptanwhen event occurs. The group is made ware doesby thisthe using This example is shown in This example is 7.2.6.5 Programming Example 7.2.6.4 Polling for an Active Interrupt Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 133

Write-Only Write-Only rtain interrupts rtain interrupts from Software writes a Software writes value of 0x1000_000E toindicate Int[15]interrupt as the of the destinationVP. GIC_SH_RMASK63_0 Interrupt sent to VP1, interrupt 15 in EIC mode. 0x0000_0000_0000_0000 ere are two fields in the two fields are ere em Programmer’s Guide, Revision 1.00 63 0 Writing to the GIC_SH_RMASK to the Writing registersoftware allows reset any to theinbit GIC_SH_MASK 0 as a to way to disablegiven interrupt. a Note: Software can use the RMASK Software Note: register to disable ce this in used not is It generated. being example. GIC_SH_TRIG registers. GIC_SH_TRIG 0 GIC_SH_MAP_PIN 0x1000_000E is a shared registeris a deliver used to an interrupt to 31 0 rnal devices. The interrupt sources chosen for this purpose for this chosen interrupt sources The rnal devices. its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its Hardware reads the GIC_SH_MAP_VP GIC_SH_MAP_PINand registers to determine destinationtheand VP interruptthe pin for interrupt to be processed. Write-Only ed to clear an interrupt. Th ed to clear ng the appropriate bits in ng the appropriate Hardware Check GIC_SH_MASK GIC_SH_SMASK63_0 t any other processor. Each inter-processor interruptconfigured is just like Each inter-processor t any other processor. 0xFFFF_FFFF_FFFF_FFFF 63 63 0 Writing to the GIC_SH_SMASKWriting register allows softwareset any to inbit the GIC_SH_MASKas a 1 to givenenableway to interrupt. a 0x00_0002 Read-Only Figure 7.6Figure GIC in the Interrupts of Mapping and Masking GIC_SH_MAP_VP* Read-Only the interrupt. 23 0 0x00_0002 indicate to GIC_SH_PEND63_0 Interrupt 8 is asserted by external hardware. Register Interface Register in Each processor the system can interrup an external interruptusing sources notused beingby exte must beconfiguredmust to be edge sensitive by setti The "Global Interrupt Write EdgeThe "Global Register", GIC_SH_WEDGE Interrupt Write another (only processor oneIt is also us per system). toused do this.GIC_SH_WEDGE register Interrupt pending status bywritten hardware based on externalinterrupt activity. 0x0000_0000_0000_0100 Software writes a value of Software writes Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 7.2.7 Inter-processor Interrupts Write-Only 63 0 *24’b0000_0000_0000_0000_0000_0010 core 0, VP 1 as the recipient of as the recipient 1 0, VP core 134 MIPS64® I6400 Multiprocessing Syst

Write-Only effect on the state of the state on the effect GIC_SH_WEDGE GIC_SH_WEDGE.INTERRUPT e) usinge) the GIC_SH_MAPi_VP er into the Edge Detect hardware hardware er into the Edge Detect bit in the internal Edge Detect register, bit in the internal Edge Detect register, eated equivalentlyhaving to the edge detection ) or cleared. Setting) or cleared. this bit delivers an interrupt and e Edge register has a direct GIC_SH_WEDGE.INTERRUPT 70 Software write 0x28 to field to bypassinterrupt the detection logic and send an interruptmessagedirectly by toggling a bit in the Edge Detect register. Software can set bits in the in can set bits Software register to bypassregister the interruptdetection logic and send an interrupt message directly by toggling register. in the Edge Detect a bit this case, kernel software writes a value of 0x28 intothe rdware determinesrdware that interrupt being toggled belongs to et (VP1VP in this exampl its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its terrupt to be used is programmed into the GIC_SH_WEDGE ed to be rising, falling,ed to be rising, or dual edge sensitive. terrupt 40 onto the appropriate 1 VP interrupt pins. (WEDGE) registeris tr register be used to bypass can the edgedetectionlogic. it does not Thus, writes the value in the WEDGE regist in the value writes register in turn sets the corresponding the in turn sets register register can be used to bypass the interruptdetection logic and assert interrupt Write Edge Write Edge Write 32 Interrupt sent to the appropriate VP and interrupt pin. Write Edge EDGE_DETECT Mapping Function GIC_SH_PEND Write Edge Write External Interrupts External Interrupt Masking and Interrupt Detection Logic field should be set to the interrupt number to be set or cleared. Figure 7.7Figure GIC the in Interrupts Inter-Processor Sending 63 0 63 0 bitdetermines if the interrupt is being set (delivered shows how the INTERRUPT RW Read-Only clearingbit the clears the interrupt. register). For example, assumeto VP 0 wants toggle interrupt 40. In bypassingedge thedetection effectively logic. Ha register, GIC_SH_WEDGE register. Hardware then Hardware GIC_SH_WEDGE register. 1, notGIC 0. TheVP VP routing then logic routes in Figure 7.7 • The A writesets the R/Wthebit that of directly. Setting a bitSetting in the a directly. matter whether the corresponding interrupt is configur When 0 VP wants to interrupt VP 1, the number of the in to the targ mapped be selected interrupt must The register. • The forcinginterruptinterrupts an withinallowing to be generated and for the GIC. inter-processor internal Edge Detect register, the internal Edge Detect register, logic see an active edge. Because the programming of the Writ Because the programming of the edge. an active logic see 7.2.7.1 Example Programming Register WEDGE Hardware Register Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies (not software visible) MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 135

register. register. of a proces- the interrupt by by the interrupt register. In practical register. GIC_SH_Counter er is also available (read e difference is that the is that difference e interrupt is to be cleared. e interrupt and clear GIC_CORE_COMPARE interruptthey when match. em Programmer’s Guide, Revision 1.00 igured and mapped locally to the pro- GIC_SH_CONFIG have the same time reference. er before executing the ERET instruction. NOTE:ERET instruction.er before executingthe only l compare register, l compare register, er within each processor. Th each processor. er within processor. This counter regist processor. is cleared indicating that the indicating that cleared is its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its 0280)) GIC_WEDGE_REGISTER. the of Address in the GIC and activates an the in inter-processor interrupt.#defines: First theinter-processor register is used to set up the width of the stem so that all processors that all processors stem so is insection the shared is GIC of the memory map. The counter mustbe at compares a loca d do whatever action is intended for th d do whatever action is of the GIC describedofin the GIC this section: the User Mode Visible Section of the GIC. Section User Mode Visible the in the processor and the GIC are conf in the processor and the I number andI number the cpu number and writeGIC_SH_WEDGEthe it to register GIC_SH_CONFIG GIC_SH_COUNTER GIC_SH_COUNTER) register is globalsy to the #define Value #defineValue Description li k0, (GIC_SH_WEDGE | GIC_BASE_ADDR) | (GIC_SH_WEDGE k0, li C0_EBASE k1, mfc0 k1, 0, 10 k1, ext 0x20 k1, addiu 0(k0) k1, sw CP0 EBase // Get CPUNum // Extract IPI interrupts. base of to // Offset IPI. this // Clear GIC_SH_WEDGE = 0x80000000 + FIRST_IPI + cpu_num ; cpu_num + + FIRST_IPI = 0x80000000 GIC_SH_WEDGE sor with a global counter, globalsor with a counter, GIC_SH_WEDGEFIRST_IPI (0xbbdc int*) unsigned *((volatile 32 Source for the number IPI. first Counter Registers counter register ( The use the counter is usuallyOSby set at boot an time by one 0 of located at offset mode user only) in The COUNTBITS field of the GIC_SH_ COUNTER writing the interruptnumber GIC_SH_WEDGE to the regist cessor. that are added as part There are 2 devices • th timer a 64 bit - Interval Timer GIC The GIC also controls how devices with how devices GIC also controls The the interrupt number is set before the write so the R/W bit the R/W the write so before is set number the interrupt Code Example of Clearing an Inter-Processor Interrupt an Inter-Processor of Clearing Code Example the interrupt routine shoul Once received, void set_ipi(int cpu_num) { // Addthe the enablebit, first IP Example of Sending an Inter-Processor Interrupt — C Code Interrupt an Inter-Processor of Sending Example an of sending example C code is a following The In the GICtheInthis design, a default value field of 0x8, has indicating a total counterof 64-bits. size The interval timer is similar to the CP0 Count/Compare tim similar to the CP0 Count/Compare is interval timer The • - a GIC_VO_WD_COUNT. 32 bit decrementing Timer counter, GIC Watchdog stopped before it is set. This is done by setting the COUNTSTOP bit of thebit of stopped settingbyThis is done beforethe COUNTSTOP it is set. 7.2.8.1 GIC Interval Timer Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 7.2.8 Configuration Local Timer 136 MIPS64® I6400 Multiprocessing Syst

GIC_VLi_COMPARE e count value equals the compare value equals the compare e count value and the Compare value is 0x1_FFFF_FFFF 0x1_FFFF_FFFF value is and the Compare sserted) by writing to the its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its is reached, hardware generates an internalhardware generates is reached, interrupt. BITS’ is the value in the COUNTBITS field of the of field in the COUNTBITS BITS’ is the value terrupt is cleared (de-a terrupt is is located in the local section of the GIC memory map makingthe count lue between 32 and 64 bits in increments of 4. For example, 32 bits, 36 bits, s can be written at any time. When be written th s can , the width, 64-bits,counter of the is Figure 7.8 GIC_VLi_COMPARE) register. register. register. Width Counter the Determining the followingused:formula is total derive the the width counter, of To x 4 32 + COUNTBITS Where: ‘32’ is the minimum widthcounter of the and ‘COUNT GIC_SH_CONFIG Compare Registers Compare register ( The compare specific to each processor. These register each processor. specific to an intervaltimer interrupt in asserted. The is For example, if thecontains COUNTBITSfieldofa value 0x8, overall thewidth wouldcounter the of be: = 64 bits x 4 32 + 8 va can be a GIC design, the counter the In 40 bits, etc. Example Interrupt Based Counter In the example shown in which corresponds to 8Gtowhich correspondscount clock cycles. When this Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 137

. When these two . When these GIC_SH_COUNTER Compare register Compare em Programmer’s Guide, Revision 1.00 GIC_VL_Compare register GIC_SH_COUNTER register 0 0 Software writes 0x0000_0000_0000_0000 to set the initialcountzero to GIC_VL_COMPARE 0 Software programs the programs Software 0xFFFF_FFFF_FFFF_FFFF value of with a register for a valuecounts. of 8Gfor its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its In the GIC_SH_CONFIG register, In the GIC_SH_CONFIG a 64-bit counter. hardware sets thehardware value of sets to 0x8COUNTBITS to implement Hardware compares the value in withvalue the in values are equal,internal hardwareare interrupt.values generates an 23 This value is used to determine to used is value This thewidth counter of the Hardware Compare 0x8 Hardwarethebit 1 of sets register GIC_VLi_PEND for further processing. 27 24 GIC_SH_COUNTER 28 writes a 0 to the a 0 to writes 63 Figure 7.8Figure Generation Interrupt Counter-Based of GIC Internal Example 63 63 Software writesSoftware 0x1COUNTSTOP to the bit theof GIC_SH_CONFIG register to stop the counter programming before the register. GIC_SH_COUNTER Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies COUNTSTOP After programmingthe registers, software COUNTSTOP bit to restart the counter. 138 MIPS64® I6400 Multiprocessing Syst

e counter toe counter stop counting each processor and is used to processor each r expires for the second the expires for r register local to each processor to each local register ate controlled by the Cluster Power een a Software hang and a Hardware t signal is sent to all processors in the t is routed to the NMI interrupt. This timer. This is one mode where the inter- timer. xpiration. If the processor itself does processor the xpiration. If med to be a hardware issue. Therefore, d instead be routed to a normal interrupt is reloaded and the time starts counting counting the time starts reloaded and is is what happens in this mode when the boot the kernel software should reload the should reload software the kernel boot d be used in a time slicing OS. wing three wing three registers. (default value) causes th causes (default value) happens when the timer reaches 0. timer If the reloads. time time the counter reaches 0. distinguish betw distinguish again, all cores are reset. reset. are cores all again, ter to stopto counting ter when the VP enters debug mode. ith the value Forin the GIC_VLi_WD_INITIAL register. its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its eristics of the timer. of eristics ", GIC_VLi_WD_CONFIG is local to each processor and each ", GIC_VLi_WD_CONFIG is local to ", GIC_VLi_WD_INITIAL is local to instructionlowor power is in a st controlledby follo the ", GIC_VLi_WD_COUNT is a read only read is a ", GIC_VLi_WD_COUNT WD_CONFIG register this register this register determines what when the count expires the second time a rese system. is An interrupt asserted, the initial count each again interrupting down interval a per-processor This mode provides to NMI. It shoul not be routed should rupt where the interrupt for example coul Watchdog Timer, thus avoiding the second e the second thus avoiding Timer, Watchdog respond to the not interrupt, then it is assu causes the processor to soft reboot. That soft to processor the causes the timer expires first time. the If this during re was a software hang This mode provides a way to a way mode provides This Interrupt Timer the Watchdog Usually hang. time before being time before reloaded Table 7.4Table Modes Timer GIC Watchdog and configures the charact t value oft value the countdown. Interval Timer (PIT) Interval Timer Watchdog Timer Count Register Timer Watchdog GIC Watchdog Timer Configuration Register Timer GIC Watchdog Watchdog Timer Initial Count Register Timer Watchdog when the processor is executing a WAIT is executing when the processor a WAIT Controller (CPC). Setting this bit causes it to continue counting down. thisWhenis set the count bit continues countingdown. that contains the curren set the timer interval. reports information state 0x2 Programmable 0x00x1 Countdown Second One Tripthe asserted and is interrupt An NMI). an (typically stops the timer and is asserted An interrupt Clearing the WD_START bit disables the timer and when it is set it enables the timer. Writing WD_START with a 1 bit disables WD_START the timer and Writing when it is set it enables the timer. Clearing the WD_START triggers a reload of the GIC_VL_WD_COUNT register w • Clearing the Debug bit (default value) causes the coun • of 3:1 The TYPE field in bits GIC Watchdog Timer Configuration Register Configuration Timer Watchdog GIC Configurationcontainsbitsregister control that the function timer. the Timer of GIC Watchdog The • bit of GIC_VLi_ Clearing the WAIT • The " Register Interface Register timer that is VP supportsEach a Watchdog • The " • The " Encoding Mode Behavior 7.2.8.2 Timer GIC Watchdog Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 139

)

reg- register RMASK Watchdog Timer Timer Watchdog GIC_VLi_SMASK register then the counter then the . In non-EIC mode, only register ( Local Interrupt Mask ) at offset address 0x000C. address ) at offset Watchdog Timer Map-to-Pin Timer Watchdog cluded in the release. cluded SI_Int[2] this register and em Programmer’s Guide, Revision 1.00 companion document included in the document included companion For example, if kernel software programs For example, if kernel software Watchdog Timer Initial Count Register Timer Watchdog Local Interrupt Pending pins the interrupt will be driven onto. In non- VLi_WD_INITIAL register each time the VLi_WD_INITIAL register each time GIC_VLi_RMASK a onlyread register that contains the current value NMI Local Interrupt Set Mask Interrupt Local or (GIC_VL_WD_CONFIG) 0x0090 at offset in the companion document in companion register. I6400 Registers register ( interrupts. For example, if kernel software programs this the WatchDog timer is programmed to generate a hardware the WatchDog its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its e counter should be disabled by clearing the WD_START bit e counter should be disabled by clearing the WD_START then reads the state of bit 0 in the then reads SI_Int[5:0] acteristics of theacteristicsthe interrupt, of it uses I6400 Registers er", GIC_VLi_WD_INITIAL is local to each processor and is used to set uded in the release. uded in the and the countdown value loaded into loaded into value and the countdown rated, hardware sets bit 0 of therated, hardware sets bit 0 of Local Interrupt Mask used to select one of 6 VP interrupts. interrupts. 6 VP of one select to used reloaded with the value in the GIC_ is register using the write-only register to enable the Watchdog timer interrupt, or ittheset bit can 0 of to register enable Watchdog the Local Interrupt Reset Mask Interrupt Local Watchdog Timer Config Register Timer Watchdog SMASK . ) at offset address 0x0008 to determine whether the Watchdog timer interrupt has been masked. The address 0x0008) at offset to determine whether the Watchdog ) at offset address 0x0004.) at offset Hardware register is read-only. register is read-only. companion document incl document companion (GIC_WD_COUNT) at offset 0x0094 (GIC_WD_COUNT)the in at offset SI_Int[5:0] GIC_VLi_PEND GIC_VLi_MASK EIC mode, bits 5:0 of thisregister are timer interrupt be driven will onto thisvaluefield witha of 0x2, then the Watchdog ( release. Mapping and Masking Timer Watchdog Interface Register timer interruptis gene Once a Watchdog Watchdog Timer Count Count Register Timer Watchdog Count Register", GIC_VL_WD_COUNT is Timer The "Watchdog countdown. This register is the of the bit in GIC_VLi_WD_CONFIG information,more For register is set. the refer to WD_START Register Count in the GIC_VLi_WD_CONFIGin the register bit. For moreinformation, to the refer enabled by setting the WD_START 0x0098(GIC_WD_INITIAL)thein at offset GIC_VLi_MASK I6400 Registers Register Initial Count Timer Watchdog Interface Register Initial Count Regist Timer The "Watchdog start the counter for the first time th the timer interval. To more information, refer to the at offset address 0x0010 address theand at offset Software can affect the state of th Software can affect ister to disable Watchdog timer interrupts. that Note when toWatchdog disableister reset, the reset cannottheby be masked register at offset address 0x0040register to determine at offset which Once hardware has determined the masking char masking the determined has Once hardware Softwarebit 0 of the sets encodings 0 - 5 are valid. EICencodesInmode, this the VP field support to up to 64 field with a value of 0x20, then the Watchdog timer interrupt to corresponds interrupt 32. This encodedvalue is then fieldvalue with of 0x20, a Watchdog then the driven onto ( Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 140 MIPS64® I6400 Multiprocessing Syst

0 Write-Only Watchdog Timer Watchdog pins Watchdog TimerWatchdog Write-Only NMI GIC_VLi_RMASK InterruptSI_Int[n] sent to or NMI the VP. pins of s s signal. the appropriate 31 0 timer interrupts. If necessary, software can set bit 0 of bit set can software If necessary, Watchdog register to disablethis the DEBUG bit of the the 1 0 1 GIC_VLi_WD_MAP 31 0 t is cleared, countingtis stopped. thatNote the DM bit of its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its Write-Only this register and assert this register and Software writes the GIC_VLi_WD_MAP register to Software writes Hardware set bit 0 of 0Hardware set bit the MASK register. to drive the Watchdog timer interrupt. timer reads Hardware to drive the Watchdog determine on which SI_Int[5:0] or accomplished by clearing the WAIT bit of the bit the WAIT clearing by accomplished s no effect on the Watchdog timer counting timer process. on the Watchdog no effect s Hardware Check accomplished by clearing accomplished GIC_VLi_MASK GIC_VLi_SMASK 0x0090.When bit this is cleared, counting isstopped, including low when timer interrupts. Software 0 of this bit writes enable Watchdog register to 31 31 0 ) must be set to place) device the in debug mode. DM 1 Read-Only a the WAIT instruction. the WAIT a DEBUG register ( Read-Only Figure 7.9Figure GIC the in Mapping and Masking Interrupt Timer Watchdog Debug register located at offset address register located at offset register located at offset address 0x0090. register located Whenat thisoffset bi GIC_VLi_PEND the CP0 Config Config Watchdog Timer and Mode and Debug Timer Watchdog timer operation while the I6400Multipro- Under certain conditions,maysuspendto kernel software want Watchdog can be This System is in debug mode. cessing If thedebugset, entering DEBUG bitIf mode is ha Mode Power and Low Timer Watchdog timer operation while the I6400Multipro- Under certain conditions,maysuspendto kernel software want Watchdog cessing System is in low power mode. This can be powerentered modevi is Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 31 0 Hardware sets bit 0 of thisHardware register of sets bit 0 to timer indicatea Watchdog that interruptoccurred. has MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 141

mode, each bit of each of the local . . . status for each of the local for each of status ing the boot exception vector upt (non-EIC) es the whether a given local inter- a given local whether the es pt Pending bits in the CP0 Cause Cause CP0 the in pt Pending bits at maps the local interrupt to a spe- em Programmer’s Guide, Revision 1.00 er indicates the status this register. If a givenset, interrupt bit If a is are dis- register. this register manages the mask managesmaskreset the function of for each the local is mappedspecificto interrupt a pin using MAP the 6-bit indicates the thatwill interrupt be mapped source to the ecimal). For vectored interr Local Interrupt Routing and Masking Local Interrupt Routing and Masking Local Interrupt Routing and Masking lue of the interrupt (0 - 63) in EIC mode. For example,oflue the interruptFor (0 - 63) in EIC mode. a cause the core to soft boot us ). This write-only registerthe allows programmer to dis- dog timer interrupts described in the previous section. The its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its so providesability the to mask interrupts usingfollowing the ). This read-only regist ). This ). This read-only register indicat read-only register This ). ed as described in the following subsections. bitof (31) the corresponding register is set, indicating thatinterrupt the each interrupt local to Interru source GIC_VL_RMASK gister for each local interrupt source th each local interrupt source for gister GIC_VL_PEND at occurat within I6400the Multiprocessing System. The routingand masking of GIC_VL_MASK Only one of these bits can be set at any one time for each interrupt. time one set at any be bits can these Only one of to interrupt processing. This to interrupt eld indicates interrupt 33 (d There are two bits, MAP_TO_PIN and MAP_TO_NMI There and are twothat MAP_TO_NMI control bits, MAP_TO_PIN the type of input that Routing and Masking Routing interrupts listed at the beginningsection of the entitled rupts has been enabled prior interrupts listed at the beginningsection of the entitled abled for the corresponding interrupt type. This register able one or more of the localor more of theable one interrupts by settingbits the of as the start of the interrupt routine. NMI bit in the CP0 Status register. This in essence will in essence This register. NMI bit in the CP0 Status of the MAPthe six field correspondsof interrupt to one of pins. source is mappedtosource an interrupt pin,actual the interrupt encoded va the field contains This this register. field of the MAP fi of 0x20 in value register of the core. If the MAP_TO_PIN interrupts listed at the beginningsection of the entitled •Reset Mask register ( Local Interrupt registers: • Local Interrupt Pendingregister ( Register Interface Register In addition to the routing of interrupts, the I6400 core al •this registerof bit (30) is set,MAP_TO_NMI thisthe If local interrupts are defined local interrupts are as follows: • Count/Compare interrupt • interrupt timer CPU Local • interrupt counter Performance • software interrupts Two •Debuginterrupt Fast Channel (FDC) mask be routed and can of these interrupts Each There is a Local Interrupt Map-to-Pin Re Local There is a local interrupts is handledin a similar manner to the Watch cific input on the processor. • map registers The local MAP_TO_PIN If this bit is set by the kernel, entering low power mode has no effect on the Watchdog timer counting process. thisiskernel, set by the bitIf low enteringon power the Watchdog no mode effect has events th internal Local interrupts are interrupt source. the to is assigned • Local( InterruptMask register 7.2.9.2 Local Interrupt Masking 7.2.9.1 Local Interrupt Routing Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 7.2.9 Local Interrupt 142 MIPS64® I6400 Multiprocessing Syst

register register are each assigned a bit. The GIC can be e local interrupts to the shared section register VZP GuestID. This GuestID . ively, since the number of since the ively, assigned a GuestID for this . companion documentincluded lows the programmer to enable to lows the programmer = 1) or non-virtualized Local Interrupt Local Interrupt Mask Local InterruptPending VZE GIC_CONFIG tion for each of th tion for each been added for virtualization should be should virtualization for added been number of sources fieldbeen added has rrupt has been masked. If a bit in this register bedisabledby setting appropriate the bits of sources, is a sources, is assigned I6400 Registers Local Interrupt Set Mask GIC_CONFIG. GuestID al interrupt source. Alternat interrupt source. al register. If a givena bit If is set, interrupts are enabled for register. as the Guest Interrupt Bus, is always inactive (alwaysGuestas the Interrupt Bus, is s must be ignored (loads return 0s, stores are dropped). external interrupt source is ) at offset address ) at0x000C. offset its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ). This write-only register al write-only register ). This implementationto may choose group external interrupt fields prior to initializing interrupts in the system. virtualized ( Local Interrupt RoutingMasking and register manages the set mask func manages the set register hardware sets the corresponding of the of hardware sets the corresponding ared and VP-Local sections that have that have sections ared and VP-Local GIC_VLi_RMASK GIC_VL_SMASK rmediate configuration such that some operate in either e per-external-interrupt source e per-external-interrupt ternal Source Interrupts register ( maintained in the fully populated root context. expected to program these expected to program these registers. source, or a logicalofgroup external interruptsource, or = 0) modes. = ) at offset address 0x0010. Conversely, interrupts can address 0x0010.Conversely, at offset ) ) at offset addressinte 0x0008thewhether to determine at offset ) ) at offset address 0x0004. address then reads Hardware of the state the) at offset VZE 0) in both EIC and non-EIC modes. listed at thelistedat beginning section of the entitled considered reserved and read-only. and considered reserved one or more of the local interruptssetting by the bits of this This interrupt type. the corresponding Local InterruptReset Mask GIC_CONFIG. GIC_VLi_PEND GIC_VLi_MASK GIC_VLi_SMASK •guest accesse all virtualization, for the core is enabled If When any of the local interrupts occurs, the local interrupts occurs, of any When • is Core-Local state Any •the core, knownin Theto the guest context GIC interface ( programmed by kernel software to The developer may choose to assign one GuestID to each extern to one GuestID choose to assign developer may The interrupt sources may (up tobe 256large interrupts), an provide an inte GuestID, or by sources ( the is set, the corresponding interruptis ignored. Local interruptsby can be enabled setting of bits the appropriate the Each external interrupt GlobalInterruptPin Map to Register Interface Register supportindicated as logic by a 1 in the The I6400 GIC provides Virtualization The I6400 MPS supportsThe I6400 virtualizationguestconceptthe and and root of The following modes. list shows some of Each madethe changes to the GIC to supportVirtualization. • Local InterruptSet Mask register ( ( Each of the registers listed in the above examples can be found in the in the release. in the ( In the GIC non-virtualized mode, the following rules apply: rules following the mode, non-virtualized GIC the In • Any registers, or any fields in the Sh may be a maximumbemayTh of 8-bits. purpose. The Hypervisor is Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 7.3.2 Routing of Guest Ex 7.3.1 Mode Enabling Virtualization MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 143 7.3 Support Virtualization

each Core- each Such configuration CP0 register field. ID a subset of GIC registers of a subset . Inaddition, a 64-interrupt system where where a 64-interrupt system ered external interrupts. remainingare divided sources GuestCtl1 companion document included in uration registers. Since of a subset GIC registers are em Programmer’s Guide, Revision 1.00 t. The individual interrupts are repre- manage all 256 possible interrupts. manage all st be qualified to avoidthe interrupts effecting 3_0, where n = 63 and m = 0. As such, for all interrupt's type (e.g., polarity, edge/level etc), edge/level polarity, (e.g., interrupt's type I6400 Registers physical core. These resident GuestIDs will be be These resident GuestIDs will core. physical the GuestID value from whatever the next lower next the GuestID value from whatever the PI) and clear EDGE regist EDGE clear PI) and D register. For example, in For example, D register. assigned a GuestID. An example intermediate solution example intermediate a GuestID. An assigned ing interruptsetc. Ds and the remainingDs and thesources are divided into up groups t software may require access to require access may t software e case where a physical GuestIDcase where register doese not exist for assigned GuestIDs, while the while the assigned GuestIDs, its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its n-zero GuestID specifies a guest n-zero GuestID specifies Software Access to GIC registers Software est-specific reads/writes reads/writes mu est-specific or not a physical GuestID register exists ('1' in the bit) or not ('0' in the bit) ere are four registers of each type to of each type registers four are ere for the specification of each ng scheme.Thisbitsvectoris 256 wide, whichthe maximum is of number r example, GIC_SH_PEND_6 w need to be directly accessed by gues by directly accessed be w need to e GuestIDthe resident in ervisor) requires access to the GIC requires ervisor) config ternal interruptsource uses ss than 256 external interrupts. first 32 interrupts [31:0] uting etc. However, the gues the etc. However, uting group with a GuestID. a group with withintended the guest. One guestinterrupts8ID for group[39:32] of One guestinterrupts8ID for group[63:56] of One guestinterrupts8ID for group[55:48] of One guestinterrupts8ID for group[47:40] of One guest ID for each of the guest ID for One 256'h00000000000000000000000000000000000000000000000001010101FFFFFFFF the release. By convention,a GuestIDspecifies0 of root,a no while registers include, but not limited to, are to, registers include, but not limited Core assignment, interrupt ro for reading interruptpending information, masking and clear shared by multiple guests and root, any gu In general, only the root software (hyp listed belo registers shared section The that are not associated sented using the n_m nomenclature, of interrupt. Fo = the range n_m where th the n_m nomenclature, registers using • (I Inter-Processor-Interrupts to cause - GIC_SH_WEDGE Local section in the GIC is aware of th aware is in the GIC Local section brought intothe the GICvia SI*_GID input portsiscores and thisequal core to the GuestID, while the remaining are grouped,remaining is and each group the while GuestID, is one where32 the 1st interrupt sourcesare individually up into8, groups each of Softwarethe can determine GuestID grouping scheme configuredbuildattime by reading this 256-bit GuestID grouping vector registers. For moreto information, the IDGroup refer Configuration Registers (GIC_SH_GID_CONFIG)0x0080 throughoffsets at 0x0098 in the To facilitatethe configuration GuestIDgrouping, of 256-bit a wide vector is provided buildset at toneeds which be To groupi GuestID the required per time as external interrupt sources supported by the I6400 GIC. However, onlyexternal interrupt the relevant sources supported lower by indexed the bits GIC.I6400 will However, take effect is configured for le GIC the when Each bit in this vector whetherrepresents 8, the 256-bitof GuestID grouping vector would be configuredwithvalue the below: shown forbits that correspondingexternal interrupt th source. In indexed external interrupt source which has a physical GuestI interruptthe 1st 32 sources are individually assigned GuestI an external interrupt source, that ex source, an external interrupt Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 7.3.3 Qualification of Root or Guest 144 MIPS64® I6400 Multiprocessing Syst

-Local section. -Local section. ster is added to each Core- is ster rnal interrupt source. Guest rnal interrupt source. al interrupt source valid vec- valid al interrupt source licated for guest licated context reg- gister to each Core ounter value that is commonto root In addition, the compare value registers addition, the compare value registers In ed to be directly accessed by guest soft- accessed to be directly ed st access to this register, but it is safe to do but it is safe this register, to st access register and also cannot disable the counter by register and also cannot disable for guest access to this register, but it is safe to do so. it is safe but to this register, access for guest ompare register bits are rep ompare directly set its compare value after sampling its offset directlysampling set its compareafter its offset value sters contains one bit sters contains one bit per exte of this chapter. of are qualified with a per-extern with a are qualified its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its value, a GIC_VLi_COFFSET regi GIC_SH_COUNTER identified reason for gue field. e (CC) timer interrupts, the global c e (CC) bitsexternal interrupts. for r-external-interrupt source valid vector. source r-external-interrupt a counter which is offset by an n-bit (set to 8 by default) value is used for by an n-bit whichcounteris offset a ites toites WEDGE register the are qualified by gatingthis driving of per-exter- each core to set compare independently. independently. set compare each core to to program this offset value register. to program thisvalue register. offset the previous sections the previous gister, the encoded the interrupt number gets valuedecoded to outgister, drive the per- nd these are added as GIC_VLi_Compare re are added nd these rrently no identified rrently no identified reason COUNTSTOP Compare Timer Interrupts Timer Compare plicated VP-Local section registers may ne plicated VP-Local section registers may GIC_SH_CONFIG so. counter value. writing the to Forthe guest context Count-Comparuse of and all guests cannot be used. Therefore, cannot be used. Therefore, guests and all offset this guest counter specify each guest context. To expected and the root is Local section Apart from the WEDGE register, all of the above listed regi all of the above listed Apart from the WEDGEregister, bits source per external interrupt these each of to accesses 31,to= 0 imax the ofwhere number configured cores. is described in of these registers Each • GIC_SH_PENDn_mdetermine - to whichpending. interrupts external are • GIC_SH_MASKn_mdetermine - to whichmasked. external interrupts are •for external bits- to set mask GIC_SH_SMASKn_minterrupts. • GIC_SH_RMASKn_m - to clear mask • GIC_SH_TRIGn_m - to allowto EDGE forset to guest otherIPI causing cores. •cu is - there GIC_SH_POLn_m • currently no is - there GIC_SH_DUALn_m tor. On guest writes to the WEDGE re guest writes On tor. Guest wr source logic. external interrupt nal-interrupt source logic with the same pe ware. •guest softwarefor GIC_VLi_PEND- to determine which local guestpending.interruptsare • GIC_VLi_MASK software - for guest determine to whichguest local interrupts are masked. • GIC_VLi_SMASKfor guestsoftware - to set mask bitsfor local guest interrupts. • GIC_VLi_RMASK - for guest softwarelocalclear guest interrupts. tofor bits mask • GIC_VLi_Comparetothe - This allows guest software The following context re guest are replicated for the guest context a This allows guest and root contexts in guest and This allows facilitatethis guest context interruptrouting, Count-C the To isters GIC_VL_[PEND/MASK/SMASK/RMASK]_MAP registers and also the GIC_VL_COMPARE register repli- register GIC_VL_COMPARE the also and registers GIC_VL_[PEND/MASK/SMASK/RMASK]_MAP isters cated for guest context. theallowedNotesoftware is not guest to write to Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 7.3.4 Guest Mode Count- MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 145

. interrupts without register. In virtualized register. strictions for guest accesses strictions for guest accesses = 0), this GEN control bit is field. and update the interruptand con- update VZE GNMI are calculated as follows: are calculated as map are meant to be located in priv- ode virtual address space. Within this this ode virtual address space. Within GIC_VLi_[PEND/MASK/RMASK/ are read-only. Currently, the only reg- Currently, are read-only. sense to make sense them available to user- em Programmer’s Guide, Revision 1.00 generated WatchDog generated WatchDog ved for future extensions. future ved for GIC_VLi_WD_CONFIG GIC_VLi_WD_[MAP/CONFIG/COUNT/INITIAL] GIC_SH_CONFIG GIC_SH_CONFIG GEN = 1, there are further re 1, there are GEN = = 1) if the rootGENguest software the= 1, then sets context. The root may allowmayrootcontext. utilize the guest to The this single mode software can initialize = 1 and its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its VZE e Section Register Map Register e Section register: VZP Visible Section of the GIC Section Visible register. ) R alias for GIC Shared Counter. Read-only ftware is given direct access to them. them. to given direct access is ftware ds. further restrictions are as These follows, so that it may be mapped to user-m ther (VO) sections of the GIC register call. The aliases for these registers for these registers The aliases call. that are read so often that it makes are read-only 0 for guest. fields og timer, the guest may handle the guest the og timer, field is further gatedfurther by field is est and Root Interrupts Root Interrupts est and field withfieldvalues 0x2 and 0x0 the and not 0x1. Thus when guestof value GIC_SH_CONFIG GIC_SH_CONFIG WatchDog timer related registers related registers timer WatchDog DEBUG TYPE GIC_VLi_WD_MAP GIC_VLi_WD_CONFIG qualify any GIC register accesses. GICregister any qualify the shared Counter registers. shared Counter the GIC_SH_COUNTER = 1 and and MAP_TO_NMI VZP Table 7.5Table Visibl User-Mode WAIT , WDRESET registers for guest context and guest so guest context and guest for registers GIC_SH_CONFIG - The guest can onlythe set SharedSection_Register_Physical_Address = GIC_baseaddress + = GIC_baseaddress SharedSection_Register_Physical_Address + Register_Offset UMVisible_Section_baseoffset - The writes this 3-bit field, droppedis the 3-bitwrites this and for LSB 0. guest reads, the LSB returns - The guest writes to 0x0000 ( GIC Counter root intervention. To facilitate this, the WatchDog relatedreplicated in bitsare facilitate this, the WatchDog root intervention.To software is allowed to access the to access software is allowed Even when guest is allowed access to WatchDog timer with timer WatchDog to is allowed access guest Even when a don't care and is not a don't care and is not used to SMASK] When guest is allowed access to WatchD • to has limited access Guest Note that register is located at an offset of 1_0000base thatNoteto relative register the GIC address. is locatedoffset an at ileged system virtualwhich address space, in only kernel The Shared (SH), VP-local (VL), and VP-o The Shared (SH), VP-local (VL), troller. allocated space is A separate 64KB address GIC registers for space are aliases address In the GIC, a single WatchDog timer is present for the root In the GIC, a single WatchDog WatchDog timersetting bynewlythe added bit control in the GEN WatchDog mode programs without requiring a system ister aliased into this space is The addresses for the registers within the User-Mode for the registers The addresses within the User-Mode However, in non-virtualized mode ( However, mode ( • to has limited access Guest of certain WatchDog timer related register fiel of certain WatchDog Register Offset Name Type Description Any Other OffsetsAny Other Reserved Reser Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 7.3.5 Gu Timer (WD) Watchdog 146 MIPS64® I6400 Multiprocessing Syst 7.4 Section User-Mode Visible GIC

ry Floating-Point Floating-Point ry Floating Point Con- sters that allow indirect sters ypes of FPU exceptions, rounding floating point exceptions, how to set the IEEE Standard for Bina for Standard IEEE U provides two additionalregi int instructions use the lower 32 bits of the 128 bit regis- between SIMD and FPU instructions (FPU uses only the its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its are updated using a read-modify-write operation. The FSCR such as enabling selected t e following subsections e are controlled by following subsections the (FS) function,a programmingand example. how to enable the FPU, how to handle how to enable the FPU, EEE 754 compliantgeneration 3rd Floating Point(FPU3) Unit with SIMD that (FCSR). These include elements . Most. FPU instructions have a one-cycle throughput. All floating point denormalized inputoperands and is read, the new value is logicallyis read, the new valuesingle into value.the a with OR’d is written existing The result and value merged back to the FCSR. avoidread-modify-write having to use a sequence,FP the To trol and Status Register and Status trol mode, and flush-to-zero operation. Normally these fields The FPU programmable functions described in th Arithmetic 754-2008 defines the following:IEEE Standard The • Floating-point data types •arithmetic,comparison, The basic and conversion operations • A computational model The standard does not define specific processing resources nor does it define an instruction set. toupdates the FCSR fieldssinglein to be performed write operation. a results are fully supported in hardware. The FPU contains thirty-two, 128-bit vector registers shared The FPU supportsThe fusedmultiply-adds theas defined by IEEE 754-2008, lower 64-bits of these registers). Single precision floating po Double precision floating SIMD instructions point instructions use use the lowerter. the 64 bits the 128 of bit register. entirebit 128 register interpretedas multiple vector elements; 16 x 8-bit, 8 x 16-bit, 4 x 32-bit,64-bitx2and vector elements. The I6400 core an optional I features The I6400 allhandles floating point within operations the I6400Multiprocessing upcoreI6400The System. to two can issue to the FPU. cycle instructions per chapter providesThis informationon roundingoperation mode, of the Flush-to-Zero Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 8.1.2 Floating Point Registers 8.1.1 754 IEEE Standard MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 147 8.1 Overview Floating-Point Unit (FPU) Floating-Point Chapter 8 Chapter

register. register. . Status of error once the of Table 8.1 It provides kernel software It provides is an alternative is an alternative way to read and an alternative way to read and write way alternative an determine the type em Programmer’s Guide, Revision 1.00 a floating-point a instruction causes . the fields of the FENR in a singlethe write. Hardwareof fields the FENR in a Cause Enables Flags RM on conditions are enabled. conditions on ngle write operation. In this case the programmer would the programmer case In this operation. ngle write Figure 8.1 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its P1). To enable CP1, set the CU1 bit in the CP0 thethein CU1 bit set enable CP1, P1). To 2008 NAN (FCSRlocatedCP1 at register 31) is used to set and monitor floating (FEXR) located at CP1 Register 26, 26, Register CP1 at (FEXR) located ENR) located at CP1 register 28, is 28, located at CP1 register ENR) unding mode field, andflush-to-zero the field singleFCSR using a the in ABS 2008 use the ‘Cause’ field in bits 17:12 to 17:12 in bits field use the ‘Cause’ ngle write. Hardwarengle write. would thenupdatedthese movefieldsinto FCSR. the Figure 8.1Figure FCSR Format 0 1 1 EVZOU I VZOU I VZOU I 1 is disabled. Any attempt to execute ts 11:7 to enableup to 5 typesexceptions of as described in ts 11:7 at are used for the following purposes. at are used for the following exception. Table 8.1Table Definitions Field Flag and Enable, Cause, This bit exists only in the Cause field. 0FS0 exception occurs. with the ability to check to see the type of error that occurred even though no exception type was enabled and no exception was taken. #define C0_STATUSmfc0li C0_STATUS t0, $t12,0 or 0x20000000 t1, mtc0 t0 register to contents register Status CP0 //move 29 set with bit register into t1 value //load t1, t0 t0, C0_STATUS t0, 29 set bit with Status register CP0 into t0 //write into t0 result and copy and t1 of t0 OR contents //logically the exception type enables field, the ro type enables the exception the programmerIn this case write operation. would update update the fields of the FEXR in a si in a FEXR fields of the update the would then move these updated fields into the FCSR. write the Cause and Flags fields of the FCSR using a si using a of the FCSR and Flags fields write the Cause Floating Point Control and Status Register and Status FloatingPoint Control E Unimplemented Operation. •when no excepti used is in bits 6:2 field the ‘Flags’ Use •the ‘Enables’ programmed, If field is Coprocessor Unusable Example Code Point Enable Floating The This register contains three fields th fields contains three This register • Program the ‘Enables’ field in bi The FloatingUnitPointThe is known as Coprocessor 1 (C When this bit is cleared, Coprocessor is cleared, Coprocessor this bit When • The Floating Point Exceptions register • The Floating Point Enables Register (F point exceptions.format of this register Thein is shown Bit Name Bit Meaning Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 31 252423 20191817161514131211109876543210 148 MIPS64® I6400 Multiprocessing Syst 8.3 Exception a Floating Point Setting 8.2 Unit the Floating-Point Enabling

the destination. the destination. lt.these In cases rounded result is correctly signed according according signed correctly lues are equally near, the equally near, lues are en the Inexact Exception is sig- Inexact Exception en the is delivered to is delivered (continued) that is, it differs from what would is, it differs that were the exponent range not enabled, the not enabled, is defined for an operation on result efully definable resu red to the destination. red to the . The RM field is encoded as follows. RM field is encoded as . The ting-point ting-point result is a quiet NaN. tude than the result. the than tude Figure 8.1 fault result an infinity result is fault stination format’s largest finite number is exceeded exceeded is number finite largest format’s stination a tiny non-zero result is detected after rounding detected after rounding result is tiny non-zero a eld is an underflow condition set. Such has no observ- When two representable va d floating-point d floating-point result, its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its cant bit is zero (that is, even). rounded result is result rounded delive but not greater in magni in but not greater exact bit in the field is set. Cause result result of an operation is inexact -- ed if and only if an exact infinite exact if an only and ed if when the Underflow Exception is the Underflow Exception when enabled, the overflowed rounded result result the overflowed rounded enabled, on is not enabled, the default floa the default enabled, on is not d result is exact or inexact. is exact d result xponent range and precision unbounded -- th unbounded precision and xponent range on is signaled if and only if there is no us is no if there on is signaled if and only e whose least signifi Table 8.2Table Definitions Modes Rounding able effect under default handling. able effect Table 8.1Table Definitions Field and Flag Enable, Cause, finite operands. When enabled, the is by the Divide de not Zero Exception The Invalid Operation Excepti The Invalid Operation performed. be to operation the for invalid are operands the Excepti Operation Invalid When the The by Zero Divide is Exception signal e have been computed were both naled. the enabled, is not Exception Inexact the When in magnitude by been what would have in magnitude the rounde to the operation. The Exception is signaled if and Overflow only if de the is set. Cause field bit in the Inexact the In addition, the If when Underflow Exception is signaled enabled, the regardless of whether rounde default handling, exception Under i.e. delivered to the destination and: • If the rounded result is inexact, the In • If Flags fi no bit the rounded result the in is exact, Unless stated otherwise, if the rounded otherwise, if the stated Unless unbounded. is not Exception Overflow When the Rounds the result to the value closest to, to, closest value the to the result Rounds result. the than less not but to, closest value the to result the Rounds Rounds the result to the nearest representable value. representable nearest the to the result Rounds valu to the is rounded result I Inexact. ZDivide by Zero. Zero. by ZDivide V Invalid Operation. O Overflow. U Underflow. 1zero. toward Round 2plus infinity. / towards positive Round 0even. to / ties to nearest Round To set the rounding mode for floating pointTo operations, program the RM field (bits 1:0) in the Floating Point Control Registerlocated at CP1 register (FCSR 31)and Status shownin Bit Name Bit Meaning RM Field RM Field Encoding Meaning Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 149 8.4 the Rounding Mode Setting

Inexact Exception to be em Programmer’s Guide, Revision 1.00 (FCSR located at CP1 register 31) mod- proximate reciprocals. the varioustheSectionsdescribed inbits of the FCSR as are set to 0, all exception types are enabled, the exception set to 0, all are ons except comparisons causes an causes comparisons ons except its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its Figure 8.1 ructions, except the ap except the ructions, fore rounding.tinynon-zeroFlushingfore of results causes Inexact and Under- ), and 10.6 (Flush-to-Zero), bit). //Enables field = 5’b11111, Flags field = 0, and Rounding = 0, Flags field = 5’b11111, field //Enables = 1 //Mode ubnormal value and tiny non-zero with result is replaced of thezero same sign. In Floating Point Control and Status Register Status and FloatingPoint Control Table 8.2Table Definitions Modes Rounding and Flags fields shown in fields shown Flags and Rounds the result to the value closest to, but not greater than the result. the than greater not but to, closest value the to result the Rounds #definemfc1li C1_FCSR C1_FCSR t0, $31 0x01000F81 t1, or t0 register to = 0, contents field register with Cause CP1 FCSR register //move into t1 value //load mtc1 t1, t0 t0, C1_FCSR t0, register CP1 FCSR the t0 into //write into t0 result and copy and t1 of t0 OR contents //logically flow Exceptions to be signaled for all inst flushed. signaled. 3infinity. / minus towards negative Round 10.4 (Exceptions),10.5 (Rounding Mode In this example, the Cause This section contains a programming example for programming rounding mode‘roundto set is zero’, and thetowards set. FS bit is The Flush to Zero (FS) bit in the the in bit (FS) to Zero Flush The the handlingof denormalizedifies operands. is set, every inputIf Flush to s Zero addition: •be non-zero results are detected Tiny • Flushing of subnormal inputoperands allin instructi • For floating-point comparisons,ExceptionInexact theis not signaledwhen subnormal input operands are RM Field RM Field Encoding Meaning Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 150 MIPS64® I6400 Multiprocessing Syst 8.6 Programming the Floating Point FCSR Register 8.5 FS Bit of the Operation

- TM ure that allow effi- amming example, MSA exception handling, ation data format abbrevia- toscalar floating map point n with leveraging generic ce. Rather than focusing on on Rather than focusing ce. ure, known as MSA (MIPS as (MIPS MSA known ure, and 64-bit integer, 16-and 32-bit fixed- 64-bitand integer, to the MIPS architect to as C or OpenCL, enabling fast and sim- mbly language progr ) data format. e how to enable the MSA, e applications in conjunctio e applications e, all integer, fixed-point, and floating-pointe, all integer, instructions -bit floating-pointdata.16-bit floating-pointfor- storage dicated hardware in the devi the dicated hardware in ent operands. One of the destin ent A architecture, including how including A architecture, the IEEE Standard for Floating-Pointfor Arithmetic 754 Standard IEEE the de written manually in assembly languagebe in order to its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its the I6400 core, MSA implementswide 128-bit vector regis- ltiple-Data (SIMD) architect ltiple-Data (SIMD) Table 9.1 tions to/from 32-bit floating-point data. 32 vector registers of 8-, 16-, 32-, ate compute-intensiv data mining, data feature extraction in video,imagevideo and processing, human- exibilitybysoftware-programmable incorporating a solution for handling Byte, 8-bit .B Data FormatData Abbreviation A Control register (MSACSR) and asse (MSACSR) and register A Control ogramming concepts includ Halfword, 16-bit .H dule adds more than 150 instructions new supported within high-level languages such Table 9.1Table Abbreviations Format Data is appendedinstruction to the that name. Note the data format abbreviationsame theis of vector operations. Table 9.1 a description of each field in the MS of each description a and GNU compiler support. compiler support. Applications such as compiler support. computer interaction,and have others, somebuilt-in data parallelismlendswell that itself SIMD. to The SIMD instructions are easily ple developmentexistingof new code, as wellcode. as leverage of This chapter provides a brief hardware overviewthe MS of registers to MSA vector registers. Pr 2008. All standard operations are provided for 32-bit and 64 is supportedmat through instruc conversion operating on 32-bit elements use the same word (.W in MSA instructions have 2- or 3-register, immediate, or elem MSA instructions have 2- or 3-register, assumed data type. For examplregardless of the instruction’s The MIPS® SIMD instructions operate on tions shown in This chapter describesMIPS Single-Instruction-Mu the The SIMD Architecture). SMA mo narrowly instructions defined that musthaveoptimized co utilized, the MSA is designed to acceler designed utilized, the MSA is point, or 32- and 64-bit floating-pointdata elements. In ters shared with the 64-bit wide floating-point unit registers. (FPU) The MSA floating-point implementation is compliant with cient parallel processing MSAincreased provides systemThe fl CODECs or other functionsemerging covered not the by de Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 9.1.1 MSA Instruction Formats MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 151 9.1 the SIMD Architecture of Overview MIPS® SIMD Architecture (MSA) (MSA) SIMD Architecture MIPS® Chapter 9 Chapter

operate in parallel with operate as follows: Table 9.1 Table d FPU instructions. SIMD instructions em Programmer’s Guide, Revision 1.00 (continued) r a completeallr a instructions, new SIMD list of ionpipelines execution. These e same clock speed as the CPU. The I6400 core can issue The I6400 core can issue e same clock speed as the CPU. stalls. This allows long-runningSIMD operations to be par- its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its on integer, fixed-point and floating-point data. integer, on Vector .V 64® SIMD Architecture. Fo ector registers shared between ector registers shared SIMD an ting-point, and fixed-point data. Word, 32-bitWord, .W Data FormatData Abbreviation Doubleword, 64-bit .D compare and branch instructions with no condition flag. MIPS Architecture for Programmers Volume IV-j; The MIPS64® SIMD Architecture The MIPS64® SIMD Architecture IV-j; Volume for Programmers MIPS Architecture Table 9.1Table Abbreviations Format Data ’. In additionIn to Floating pointinstructions the Floatingpoint Unit (FPU3) containsfull set a of over 150 SIMD instruc- with the MIPS that are compliant tions The FPU contains thirty-two, 128-bit v use the entirebit 128 register interpretedas multiple vectorand elements relateto refer to document MD00868, ‘ Module enable: SIMD instructions • vector parallel Efficient arithmetic operations •on Operations absolute value operands. • Roundingoptions and saturation available. • Fullmultiply precision and multiply-add. • floa Conversions between integer, • Complete set of vector-level • (1D) and arrayoperations. (2D) shuffle Vector •load and store instructions for endian-independent operation. Typed SIMD is fullyThe FPU plus synthesizable and operates at th up to twoup to instructions cycleto per the FPU. FPU containsThe two executionSIMDpipelinesinstruct for tially maskedand/or by system stall other integer unit instructions. The FPU is optimized for SIMD performance. Most SIMD instructions have one cycle throughput. • 16 elements x 8-bits/element (.B) • elements 8 x 16-bits/element (.H) • (.W) elements x 32-bits/element 4 • elements 2 bits/element x 64 (.D) •(.V) element x 128-bits 1 the integer core and do not stall when the integer pipeline Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 9.1.2 SIMD Instructions 152 MIPS64® I6400 Multiprocessing Syst

ar floating-point unit (FPU) 32-bit), doubleword (64-bit). are the 64-bit FPU registers. 64-bit FPU registers. are the ers. If both MSA and the scal the MSA and ers. If both , halfword (16-bit), word ( its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its 64 63 0 ector registers extend and sh ector registers show the vectorelementslayout register all of for four data formats. [1] [0] Figure 9.1Figure x 8) (16 Elements Byte Register MSA Vector Figure 9.3Figure x 32) (4 Elements Word Register MSA Vector Figure 9.4 Figure 9.2Figure x 16) (8 Halfword Elements Register MSA Vector Figure 9.4Figure x 64) (2 Elements Doubleword Register MSA Vector through [3] [2] [1] [0] [7] [6] [5] [4] [3] [2] [1] [0] are the 128-bit MSA present, v MSA vector registers haveregistersvector four MSA formats: data (8-bit) byte Figure 9.1 The MSA operatesThe thirty-two on 128-bit wideregist vector Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies [15] [14] [13] [12] [11] [10] [9] [8] [7] [6] [5] [4] [3] [2] [1] [0] MSB LSB MSB LSB MSB LSB MSB LSB MSB LSB MSB LSB MSB LSB MSB LSB 9.1.4 Registers of MSA Layout 9.1.3 Registers MSA Vector MSB LSB MSB LSB MSB LSB MSB LSB MSB LSB MSB LSB 1271121119695807964634847323116150 2101121140998887776655444333221187 0 127120119112111104103969588878079727164635655484740393231242316158 127 127 96 95 64 63 32 31 0 MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 153

to

V Figure 9.6 ement with and = 0, …, 31, writes el to the to the floating-point r V V Figure 9.5 = …,0, 31, returns the value of , where s. To facilitate register data shar- facilitate To s. r r em Programmer’s Guide, Revision 1.00 where Vector Registers Vector r, = 0, …, 31, writes writes 31, …, 0, = r word element 0, and all remaining elements are 0, and word element after writing a 32-bit value , where = …, 31, returns the 0, of value the element with 32-bitfor rd (single floating-point) precision read or r r r structions, the FPU is requiredfloating-pointthe64-bituse structions, to FPU is preserves . . r r its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its r, d floating-pointd registersas follows: are defined where 64 63 0 r, Registers MSA to ers are mapped on ers are mapped the MSA vector register to the high part of the to thehigh floating-pointpart of register V showsvector the register and all remaining elements are UNPREDICTABLE. are UNPREDICTABLE. elements and all remaining . The element’s format is wo format is . The element’s r r 1 in the vector register 1 in the vector register 1 in the vector register 1 to the floating-point to the register V after32-bit writing a (singlefloating-point) precision 64-bit and a (double precision r to the floating-point register Figure 9.7 V Unpredictable value V Doubleword Figure 9.5Figure Register MSA Vector on the Effect Write FPU Word Figure9.7 Register Effect Write High on FPU the MSAWord Vector . Figure 9.6Figure Register MSA Vector on the Effect Write FPU Doubleword r the word element with index with the word element UNPREDICTABLE. UNPREDICTABLE. register floating-point) value the word element with index the word element with index 0 in the vector register double for(double 64-bitfloating-point) precision read. index 0 in the vector register show the vector register Unpredictable Unpredictable value V Word Unpredictable Unpredictable Unpredictable Unpredictable value V Word •fromread operation A 32-bit high the of part the floating-point register • A 32-bit write operation of value The scalar floating-pointThe scalar unit regist (FPU) • A read operationthe from floating-point register • write A operation of value registers operating in 64-bit mode. mappe the FPU/MSA read and write operations for The ing betweening scalar floating-point instructions and vector in Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 9.1.5 Mapping Scalar Floating-Point of 127 96 95 64 63 32 31 0 127 127 96 95 64 63 32 31 0 154 MIPS64® I6400 Multiprocessing Syst

MSA Disabled the Config3.MSAP state of by the t is not set causes a t is able access to the MSA instructions and MSA access to the able cape functionality as this is already builtthe in to part of the MIPS CodeScape. As such, it is not necessary it is not necessary part of the MIPS CodeScape. As such, enable the MSA block while writing a low-level support its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its MSA is implementedis MSA is set, MSAP bitthe by checking if AP bit is fixedAP hardware by the implementation read-is and ructionMSAEn bi when //and CU1 bits set CU1 bits //and 16, Select 5, bit 27) is used to en 5, 16, Select CP0 register bitsbe programmed:CP0 register must ecture (MSA) implementation is indicated ecture (MSA) . Exception the MSA vector registers. Executing a MSA inst Executing the MSA vector registers. ehb in Config5. bit MSA enable //Set mfc0 C0_CONFIG5 v1, li or CFG5_MSAEN v0, mtc0 v1, v0 v1, C0_CONFIG5 v1, ehb v1 into register CP0 Config5 //move bit the MSAEN that sets into v0 value //load bit set MSAEN with CP0 Config5 result to out //write into v1 back result v0 and place v1 and //OR #include #include #include #define#define C0_STATUS C0_CONFIG5 are set. register CP0 Status the of FR bits CU1 and that the ensures code //this $12,0 mfc0 $16,5 C0_STATUS v1, li or SR_CU1 SR_FR | v0, mtc0 v1, v0 v1, C0_STATUS v1, v0 into place result bits and and CU1 the FR //OR into v1 Register Status of CP0 contents //move FR with register CP0 Status out to result //write into v1 back result v0 and place v1 and //OR •must bit be set. The Status.CU1 Code Example MSA Enabling followingThe code exampleenabletodescribes how block the MSA using register bits described the above. bit (CP0 Register 16, Select 3, bit 28)Select 3, bitMS 16, at reset. The (CP0 Register bit only forkernel the software.Software determinecan if the I6400 core. in always the case which is the MSA block, the following enable To • TheRegister Config5.MSAEn bit (CP0 This example is for a programmer writing their own code to library, RTOS, or their own tool chain. However, this code is tool or their own chain. However, RTOS, library, for the programmermanually to enableusingMSA when CodeS software. CodeScape Register Interface The presence of the MIPS SIMD archit The followingThe subsectionssome describes programming the elements block. MSA of Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 9.2.1 Enabling MSA MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 155 9.2 Programming MSA

. of error once the error once the of Table 9.2 lt.these In cases It provides kernel software It provides enabled, the default float- enabled, the led, the overflowed rounded led, the nd MSA Control and Status Reg- nd MSA Controland Status at controls the operation of the operation the at controls determine the type were the exponent range em Programmer’s Guide, Revision 1.00 n result is is default not enabled, the result is defined for an operation on efully definable resu layout. However, each serves a different functional a different serves each layout. However, stination format’s largest finite number is exceeded exceeded is number finite largest format’s stination Operation Exception is not Operation on conditions are enabled. conditions on rflow Exception is not enab rflow Exception d floating-point d floating-point result, . its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its , CP1 Control31)Register a using the CFCMSA and CTCMSA (Copy From and To Con- usingCFCMSA and CTCMSA (Copy theand From To ) is a 32-bit) is a read/writeth register EVZOU I VZOU I VZOU I FCSR ed if and only if an exact infinite exact only if an and ed if MSACSR MSACSR use the ‘Cause’ field in bits 17:12 to 17:12 in bits field use the ‘Cause’ ng, i.e. when the Invalid ng, i.e. when the Divide by Zero Exceptio MSACSR on is signaled if and only if there is no us is no on is signaled if and only if there ts 11:7 to enableup to 5 typesexceptions of as described in ts 11:7 at are used for the following purposes. at are used for the following Figure Figure 9.8 Format MSACSR Register read and write the FS 0 Impl 0 NX Cause Enables Flags RM shows the shows the of format the Table 9.2Table Definitions Field and Flag Enable, Cause, ) are closely related in their purpose and register This bit exists only in the Cause field. The Invalid Operation Excepti performed. be to operation the for invalid are operands the exception handli default Under in magnitude by been what would have in magnitude the rounde ing-point result is a quiet NaN. result ing-point The by Zero Divide Exception is signal finite operands. handli default exception Under an infinity to the correctly according operation. signed The Overflow Exception is signaled if and only if the de result is delivered to the destination. In addition, the Inexact bit in the Cause field is field set. in the Cause bit Inexact the In addition, destination. the to delivered result is unbounded. handling, i.e. when default exception the Ove Under Figure 9.8 0 MSACSR exception occurs. with the ability to check to see the type of error that occurred even though no exception type was enabled and no exception was taken. 00000000 E Unimplemented Operation. ZDivide by Zero. Zero. by ZDivide V Invalid Operation. O Overflow. •when no excepti used is in bits 6:2 field the ‘Flags’ Use •the ‘Enables’ programmed, If field is trol MSA register) instructions. FloatingThe Register Point Control ( and Status th fields contains three This register • Program the ‘Enables’ field in bi unit exist independently and can of the other. The kernelsoftware can ister ( MSA unit. MSA unit. The MSA Control and Status RegisterMSA ( ControlThe and Status Bit Name Bit Meaning Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 31 25 24 23 22 21 20 19 18 17 12 11 7 6 2 1 0 9.2.2 Setting Exception a MSA 156 MIPS64® I6400 Multiprocessing Syst

MSA Control and Status and Status MSA Control rounded result is lues are equally near, the near, equally lues are en the Inexact Exception is sig- en the (continued) that is, it differs from what would from what would is, it differs that enabled, the rounded result is deliv- enabled, the rounded not enabled, the not enabled, eld (bits 1:0) in the tude than the result. the than tude for more informationmore for on setting the rounding mode. for more information on programming the exception a tiny non-zero result is detected after rounding detected after rounding result is tiny non-zero a eld is set. Such an underflow condition has no observ- When two representable va its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its cant bit is zero (that is, even). but not greater in magni in but not greater exact bit in the field is set. Cause ations, program ations, the RM fi result result is of an operation inexact -- when the Underflow Exception is Exception the Underflow when when the Inexact Exception is not d result is exact or inexact. is exact d result xponent range and precision unbounded -- th unbounded and precision xponent range . The RM fieldencoded. as follows. is for more information on MSA exception types. MSA exception on for more information e whose least signifi ng the MSA CSR Register" ng the MSA CSR Register" Table 9.3 Table 9.3Table Definitions Modes Rounding ) shownin able effect under default handling. able effect Table 9.2Table Definitions Field and Flag Enable, Cause, ered to the destination. ered to the have been computed were both e both have been computed were naled. exception handling, i.e. default Under If enabled, the If Exception is signaled when Underflow enabled, the regardless of whether rounde default exception handling, Under i.e. delivered to the destination and: • If result the rounded is inexact, the In • If Flags fi no bit the rounded result the in is exact, Unless stated otherwise, if the rounded otherwise, if the stated Unless Rounds the result to the value closest to, to, closest value the to the result Rounds result. the than less not but to, closest value the to result the Rounds result. the than greater not but to, closest value the to result the Rounds Rounds the result to the nearest representable value. representable nearest the to the result Rounds result is rounded to the valu to the is rounded result MSACSR Section 9.2.6, "Programmi Section 9.2.6, "Programmi Section 9.3, "MSAExceptions" ( I Inexact. U Underflow. 3infinity. / minus towards negative Round 1zero. toward Round 2plus infinity. / towards positive Round 0even. to / ties to nearest Round Refer to Register To set the rounding mode for floating point oper To Refer to types. Refer to Bit Name Bit Meaning RM Field RM Field Encoding Meaning Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 9.2.3 Setting the Rounding Mode MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 157

. Inexact Exception to be ) modifies the handling of denormal- em Programmer’s Guide, Revision 1.00 proximate reciprocals. MSACSR e same format as the Cause field. The Flags bits Causefield. The Flags the same format as e ( are set to 0, all exception types are enabled, the types set to 0, all exception are for more informationmore for on setting the NX bit. for more informationmore for on setting the FS bit. gister’s elements are set either to the calculated results gister’s signalingNaNvalues the with least significant bits 6 exception condition that does not result in a MSA floating written and the floating point exceptionsbitsset the Cause ons except comparisons causes an comparisons causes ons except its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its Figure 9.8 ock in non-trappinginock floating point exception mode. ructions, except the ap except the ructions, Section 9.3.2 “MSA Non-TrappingExceptions” ed for that element in th for that element ed fore rounding.tinynon-zeroFlushingfore of results causes Inexact and Under- Mode), and 11.6 (Flush-to-Zerobit). Mode), and 11.6 //field = 0, Enables field = 5’b11111, Flags field = 0, field = Flags 5’b11111, field = Enables = 0, //field set and NX bit = 1, Mode //Rounding t0 //into ample for programmingvarious thethe MSACSR bits of as described in Sec- off). This field is encoded as follows: off). ng the MSA CSR Register" ng the MSA CSR Register" ubnormal value and tiny non-zero with result is replaced of thezero same sign. In MSA Control and Status Register and Status Control MSA and Flags fields shown in fields shown Flags and e MSA CSR Register Section 9.2.6, "Programmi Section 9.2.6, "Programmi cfcmsali $1 t0, or 0x01040F81 t1, Cause bit set, with FS register into t1 value //load t0 to register contents register MSA CSR //move t1, t0 t0, result copy and and t1 of t0 OR contents //logically flow Exceptions to be signaled for all inst flushed. signaled. or, if the operation would normallyexception, signalto an or, recording the specific exception type detect are updated for all floating-pointoperation withIEEE an and trap. non-trappingIn exception mode (NXbitthat set), the operations would normallysignal floating point exceptions do writebitsnotthe not and do Cause trap.the All destination re (i.e., the Enable bit is point exception 0: Normal exception mode. 1: Non-trappingexception mode. Refer to (Rounding (Exceptions), 11.5 tions 11.4 In this example, the Cause rounding mode‘roundto is setzero’, and thetowards set. FS bit is This section contains a programming ex contains This section In normalIn exception mode, the destination register is not Settingbl the NX bit CSR sets the MSA in the MSA The Flush to Zero (FS) bit in the the in bit (FS) to Zero Flush The For more informationNX onthe bit, refer to ized operands. ized operands. is set, every inputIf Flush to s Zero addition: •be are detected non-zero results Tiny • Flushing of subnormal inputoperands allin instructi • For floating-point comparisons,ExceptionInexact theis not signaledwhen subnormal input operands are Refer to Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 9.2.6 Programming th 9.2.5 Operation of the NX Bit 9.2.4 Operation FS Bit of the 158 MIPS64® I6400 Multiprocessing Syst

n is either not or with ExcCode and Flags fields of the of fields and Flags e field has e field has an additional e exception vector common element in the MSA vector element in the al an exception according to exception uses the common uses the exception element causes an exception element causes and enable bits control excep- bits control enable and ng in preciseng in exception mode or write privilegedMSA control ssed by the instructio ssed by the 6 bits have the same format as the format the same have 6 bits mmon exception vect mmon exception ster set to 0x0E. The exact reason for to 0x0E. The ster set emulation assistance. If an exception type emulation assistance. indicate Coprocessor indicate Coprocessor 0. . This th exception uses . This ure with the Cause, Enables, ure with the Cause, ect 5, when bitvector registers 27) par- is not set or, 0, bit 29) is set. This 29) is set. 0, bit case on which elements are at fault and the corre- 16, Select 3, bit 28) is not set, or if the usable FPU orSelect 3, bit 28) if 16, is not set, ed if at least one vector least ed if at , the FPU then is operati enables determining which which determining enables ed by the MSA floating point instruction. This exception which would normally sign which would normally exception conditions. The Caus exception its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ception status flags, and the cause flags, and status ception CE field set to 0 to the MSA Control and Status Register (MSACSR). Register and Status Control the MSA CTCMSA instructions attemptread to enabled. This exception uses the co This enabled. set), if any register acce MSA vector FCSR/MSACSR Mnemonic Description ed due to a context switch NaN values, where the least significant where the least significant values, NaN Table 9.4Table Codes Exception precise indication in this each of the five IEEE of the each or with ExcCode field in Cause CP0 regi ExcCode field in Cause CP0 with or ation, to trap used for kernel software in the MIPS FPU/MSA in the MIPS FPU/MSA architect nt instruction. All elements All nt instruction. ode field in Cause CP0 register set to 0x0A. . The flag bits implement IEEE ex takingexception this in the Cause bits of is with ExcCode fieldCause in CP0 register 0x15. set to uses the exception common vect ctcmsa $1 t0, CSR register MSA the into t0 //write operates in 32-bit mode; Status.CU1 (CP Register 12, Select 12, (CP Register mode;operates in 32-bitStatus.CU1 exception vector with ExcC registers without Coprocessor 0 access field in Cause CP0 registerfield in Cause CP0 to setand 0x0B titioning is enabled (i.e. MSAIR.WRP (i.e. is enabled titioning available or needs to be saved/restor spondingexception exception causes. The handlingroutine should set the non-trapping exceptionmode bit NX and MSA floating poi re-execute the to signaling bit-field are set the Enable • MSA Floating Point, a data dependent exception signal enabled by the Enable bit. There is no is There bit. enabled by the Enable caused the floating point exception. normalIn operation mode, floating point exceptionssignal are MSA providesnon-trappingMSA a exception modethat (bit NX) MSA instructions can generate the following exceptions: following the generate can MSA instructions • Reserved Instruction, if bit Config3.MSAP (CP0 Register FPU exceptions are implemented FPU exceptions are •Unusable, Coprocessor if CFCMSA or FCSR/MSACSR tion trapping.a bit fieldEach for has • MSA Disabled, if bit Config5.MSAEn(CP0 Register 16, Sel exception bit, Unimplemented Oper exception bit, Unimplemented for this type of exception. is through the Enables field of enabled the 11 0x0b CPU exception unusable Coprocessor 1421 0x0e 0x15 MSAFPE MSADis Floating Point exception MSA Disabled exception MSA 10 0x0a RI Reserved Instruction exception Exception CodeValue Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies Decimal Hexadecimal 9.3.2 Exceptions MSA Non-Trapping 9.3.1 MSA Types Exception MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 159 9.3 Exceptions MSA

flags are not updated. Enable bit is notEnable bit is ement. The other ele- ement. Flags bits are updated after all ns. If there are enabled excep- are ns. If there MSACSR MSACSR operates on. It is assumedoperateson. It is EVZOU I em Programmer’s Guide, Revision 1.00 MSACSR detected for detected for that el 6543210 Cause bits and setting the destination’s Cause bitssetting anddestination’s the MSACSR results. Regardlessresults.NX the value, if a floating of point Enable bit is 0,Enable bitfloating the is point defaultresult is a value. oating point exceptionwill be taken, not even the always its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its all elements all elements the instruction Cause contains no enabled exceptio no enabled Cause contains MSACSR executing the instruction. The instruction. executing the | c Field Update Pseudocode Field Update MSACSR Cause | E /* Unimplemented (E) is always enabled */ enabled is always (E) Unimplemented | E /* Signaling NaN Bits NaN Signaling Cause Cause ) to record the specific exception or exceptions or the specific exception to record ) MSACSR Enable i.e. the corresponding i.e. the and (enable & U) = 0 and (c & I) = 0 then (c & I) = 0 and & U) (enable and and (enable & O) = 0 then & O) = 0 (enable and  0 0

Cause, a MSA floating-point exception will be signaled and the   Cause Update Pseudocode Update Figure 9.9 Figure 9.9Figure when NX Set is Elements Faulting for Format Output MSACSR c & enable c &  c | I c ^ U (bit E is 0x20, O is 0x04, U is 0x02, and I is 0x01) is and I is 0x02, 0x04, U O is E is 0x20, (bit exception    Cause bits are all cleared before Cause bits are all Cause d: exception a disabled of case used in to be value default e: a non-trapping set, i.e. of NX in case to be used value NaN signaling r: an exception without completed operation if the value result v: element to destination written to be value MSACSR Updated c:I bitfield O, U, V, Z, E, exception(s) element current if c = 0 then if c c */ exceptions current all Cause with MSACSR the update exceptions, No enabled /* MSACSR MSACSR Output enable Input /* Set Inexact (I) when Overflow (O) is not enabled */ is not (O) Overflow (I) when Set Inexact /* (c & O) if endif */ is not enabled (U) Underflow when Underflow Clear Exact /* (c & U) if endif cause if cause = 0 then … value. process is invoked This element-by-element for MSACSR MSACSR enabled UnimplementedException. Operation Notesettingby that the NX bit, the The followingThe pseudocodeprocessthe shows of updating the ments willmentsbecalculated to theset on theirresults based operands. non-trappingtheWhen exception modeNXset, no fl bit is processed and the elements have been tions in Cause field (see Cause field changedstill andgenerate used to is appropriatethe default exception is not enabled, exception is not Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 9.3.3 MSACSR Cause Register 160 MIPS64® I6400 Multiprocessing Syst

*/ */ as possible. In the process as possible. In the process interoperability betweenthe ctions defined by MSA. The MSA. by defined ctions case */ support mixing seamless of scalar and is always enabled */ enabled is always destination ped data transfer instructions.ped data transfer as many of these operations destination gets the result gets destination ngle MSA instructions. the vector registers and instru and the vector registers for SIMD architectures is the for SIMD architectures its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ) tibilities between the base O32 ABI and the corresponding */ Cause Cause | c | E)) = 0 then /* Unimplemented (bit E 0x20) (bit /* Unimplemented = 0 then | E)) the base ABIs in the sense that existing binaries run unchanged on systems | MSACSR | anslate directly in si Cause and the native vector data types. To and the types. To native vector data Enable Flags e MSA provides a rich set of ty provides a e MSA MSACSR 

MSACSR  & (MSACSR = 0 then

Cause NX d r ((e >> 6) << 6) | c 6) << 6) ((e >> destination is not written destination gets the default value for disabled exceptions for disabled value default the gets destination gets the signaling NaN value for non-trapping exception exception non-trapping for NaN value signaling gets the destination Cause Flags    Update and Exception Pseudocode Signaling Exception and Update MSACSR /* Exceptions will trap, update MSACSR Cause with all current exceptions, current with all Cause MSACSR update will trap, Exceptions /* v v enabled, are not exceptions Current /* */ updated are not Flags MSACSR /* Operation completed successfully, completed Operation /* /* No trap on exceptions, element not recorded in MSACSR Cause, in MSACSR recorded not element exceptions, No trap on /* v Flags /* No enabled exceptions, update the MSACSR Flags with all exceptions */ all exceptions with Flags MSACSR the update exceptions, No enabled /* MSACSR endif */ enabled are exceptions Current /* if MSACSR else MSACSR SignalException(MSAFPE, /* Trap on the exceptions recorded in MSACSR Cause, Cause, MSACSR in recorded the exceptions Trap on /* else endif else else if (MSACSR endif MSA ABI extensions are compatible with MSA ABI extensions are compatible supportingIn otherthere MSA. are no incompa words, MSA extended ABI. MSA ABI extensions; In particular, •data not Do base ABI types layout change the / alignment of MSA instruction selection and definition, supporting the standard GCC SIMDpatterns was one of the most impor- tr of these patterns objectives. Most tant The O32 ABIs have been extended to allow efficient use of use efficient have been extended to allow O32 ABIs The The GNUThe C Compiler (GCC) supportfor SIMDoperationsnumberonstandard a is based pattern of names used for the instructionshouldset implement code generation. Ideally, MSACSR Another aspect related to efficient vector code compilation Another aspect to efficient related vectoroperations, data typesth C language arrays (of scalar data types) data (of scalar C language arrays Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 9.4.1 MSA ABI MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 161 9.4 Compiler Support GNU MSA

no-msa. The SIMD ABI can d by the -mmsa command line em Programmer’s Guide, Revision 1.00 one, can be used to disablecan be used toone, passing/ the parameter d (aka saved) status of the aliased floating-point regis- vector registers) is enable vector registers) a can be disabled using -m using disabled be can a one,datavector all types followcalling the conventions of bled at the functionusing level __attribute__()shown as ion for MSA must support for ion 32 64-bit floating point registers bi option. In particular, two SIMD ABIsbi option. are In defined:particular, its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its MSA ABI by defaultMSA ABI by returnedvalueby / and passed without Avectorsvalue by and returning vectorMSA values. c stack re-alignment in this case. in this re-alignment c stack the vector registers. The O32 FR1 ABI permitsregisters.theFR1 ABI The O32 vector use of 64-bit floatingpoint cessor symbols are defined for each option as follows: for each option symbols are defined cessor __attribute__((msa)) ters. • not Do new introduceregisters callee-saved (aka saved) •callee-save or temporary) (aka the call-clobbered Preserve However, vector data types are considered part of the considered part are vector data types However, To be compatible with the MSA hardware, an ABI extens an MSA hardware, be compatible with the To any MSA flags resultswarning. in a compiler of size to the aligned frame and a stack registers. is possibleIt to adjust alignment the stack time at runusing an existing compiler mechanismcalled dynamic stack realignment.ABI Anynotdoes thatthe meet alignment MSAstack will therefore use dynamic stack re-alignment. 128-bit vector registers. How- For example,alignmentstack the 16-byte ofN64 N32 and ABIs is enough for MSA’s mustO32 thedynami perform ABI ever, option. A functiontoMSA is referredfunction. as a MSA compiled for By default, the -mmsa option enables a faster calling convention for thosefunctions passing vectors by value. This is achieved by using the vector registers for passing MS Compiling for MSA (using the MSA defined instructions and A secondcommand MSA-related line -msimd-abi=n argument, -msimd-abi=nreturninginvalues the vector registers. With the base ABI. The use of vector types passed by value without the -mmsa option results in an ABI warning stating that a non-default willbe emitted.ABI becan This warningdisabledexplicitly by passing the -msimd-abi=noneoption. It is illegal to use the -msimd-abi=msa optionwithout -mmsa. -mms option line the command by enabled functionality The below. • -mmsa •__attribute__((no_msa)) -mno-msa • -msimd-abi=none __attribute__((simd_abi_none)) • -msimd-abi=msa __attribute__((simd_abi_msa)) For convenience, pre-pro •__MSA__ -mmsa be controlled by varyinggiven the valuethe to -msimd-a • none - Use calling the base convention •calling - Use the MSA msa convention(default) the same functionality could be enabled/disa Equivalently, 9.4.1.1 ABI Requirements 9.4.1.2 Attributes Function and Line Options Command Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 162 MIPS64® I6400 Multiprocessing Syst

-mno-msa . lue and is compiled with - is lue and Table 9.5 or reg- are returned results via vector e calling conventions of MSA and gisters as specified by the O32 FR1, by the O32 specified gisters as ared with64-bit the wide floating-point -mfp64 -mhard-float 32, and f24,32, ..., f30, f25,N64. f31if For example, for e used to pass vector parameters. This falls back to the back falls vector parameters. This pass e used to or returns a MSA vector by va registers must be saved beforefunction. calling a This d byd non-MSA functions, i.e. a functionscompiled under th GAS (GNU2.22.51 Assembler)The and GCC 4.7.3. its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its /disable MSA are shown in MSA are /disable returned as specified by the particular ABI. specified returned as enforce the compatibility of th the enforce via vector registers w4 to w11 and vect via vector registersw11 w4 to ent release, MSA vector registers are sh MSA vector registers ent release, GAS GCC the aliased callee-savedthe aliased floating-point re mbly directives to enable mbly directives -mhard-float nctionscompiled with -mmsa. temporary, and all live vector temporary, Table 9.5Table MSA GNU and Directives Options Enable Disable Enable Disable and .set msa.set .set nomsa -mmsa -mno-msa -mmsa -mfp64 The GCC options FPU, based on the fact that in the curr the FPU, based on unit (FPU) registers. command line options and asse The MSA is supportedThe by the GNUtoolchain starting wi the base ABI with MSA disabled.the base ABI Any functionAnycompiled with -msimd-abi=nonecan be calle original variable argument passing scheme from passing scheme the particularoriginal ABI. variable argument need to preserve that compilers Note mmsa can be called only by fu A function that takes a MSA vector by value as a parameter a parameter value as by vector a MSA A function that takes For functionsvariable with no vector arguments, registers ar all ABIs. under be preserved has to f30 register point floating aliased the used, w30 is register vector the ensures MSA functionsanycallcan other function and compatibility with future MSA extensions. The first 8 vector parameters are passed ister w0. Floating-point registers are passed and N32,N64 andevenABIs: f30 forf20, f22, N FR1 and O32 ..., The MSA vector registers are are MSA vector registers The •__NO_MSA__ -mno-msa • -msimd-abi=none __SIMD_ABI_NONE___ • -msimd-abi=msa __SIMD_ABI_MSA__ 9.4.1.5 MSA GNU Options and Directives 9.4.1.4 MSA non-MSA Functions and Between Inter-calling 9.4.1.3 and Floating-Point Vector Register Usage for -mmsa and -msimd-abi=msa Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies Assembly Directives Assembly Command Line Options MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 163

and Table 9.6 ; directive, by . ~ , ; >= ; __asm__ , > , ; ; <= ; , < em Programmer’s Guide, Revision 1.00 , != , == rators on vector data types. The list of sup- , >> , tor_size(16))) << __attribute__ ((vector_size(16))) __attribute__ , & __attribute__ ((aligned(16))) __attribute__ , its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its | , __attribute__ ((aligned(16))); __attribute__ ^ __attribute__ ((vector_size(16))) __attribute__ , __attribute__ ((aligned(16))); __attribute__ __attribute__ ((aligned(16))) __attribute__ wu16_t % and compare instructions: __attribute__ ((aligned(16))); __attribute__ , __attribute__ ((aligned(16))) __attribute__ / __attribute__ ((vec , __attribute__ ((vector_size(16))) __attribute__ precision floating-point vectors, as in: wu64_t * programmer either by the inline assembly assembly the inline either by programmer __attribute__ ((aligned(16))) __attribute__ __attribute__ ((vector_size(16))) __attribute__ ((aligned(16))); , __attribute__ ((aligned(16))) __attribute__ __attribute__ ((vector_size(16))) __attribute__ - __attribute__ ((aligned(16))) __attribute__ , wu8_t __attribute__ ((vector_size(16))) __attribute__ __attribute__ ((vector_size(16))) + wu32_t wi8_t __attribute__ ((vector_size(16))) __attribute__ __attribute__ ((vector_size(16))) __attribute__ wi64_t wf64_t wi16_t wi16_t wf32_t wi32_t word floating-point add operators include: intrinsics,whenthemost using or C/C++ ope of typedef unsigned long long typedef unsigned long long typedef unsigned short unsigned typedef typedef float float typedef double typedef Table 9.6 in MSA Table Supported Types Data Vector Integer GCC () Table 9.7Table in MSA Supported Types Data Vector GCC Floating-Point . fadd.w $w3,$w0,$w1 # a is in $w3, b in $w0, c in $w1 in $w0, $w3, b a is in # $w3,$w0,$w1 fadd.w wi32_t t; wi32_t c; a, b, wf32_t b + c; a = b < c; t = mnemonic msa_ MSA instructionsMSA are available to the C/C++ Table 9.7 ported vector C/C++ compiles directly in MSA compiles The GCC integerThe GCC floating-point and vectordata types with generic MSAoperation support listed are in For example, adding or comparing two single- Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies Vector Data Type Data Vector C Definition Vector Data Type Data Vector C Definition Vector of signed bytes signed of Vector char signed typedef Vector of unsigned double- unsigned of Vector words Vector bytes of unsigned Vector typedef unsigned char halfwords signed of Vector short typedef Vector of signed doublewords signed of Vector long long typedef Vector of unsigned halfwords unsigned of Vector Vector of signed words signed of Vector int typedef Vector of unsigned words of unsigned Vector typedef unsigned int Vector of double of precision Vector values floating-point Vector of single precision Vector values floating-point 164 MIPS64® I6400 Multiprocessing Syst

, and . Table 9.8 Figure 9.11 , and W8 after execut- and W8 e. temporary registers are e. temporaryregisters Figure 9.10 Figure ed in vector registers W4 to W11. W4 to W11. registers in vector ed er sizes are shown in are er sizes in assemblylanguage) datafor- based on the n vectors W5, W6, W7, . ws[n] = 0, 1 n ( = 0, …, 7 = 0, …, 3 sters are all caller-saved, i. sters all caller-saved, are = 0, …, 15 …, = 0, n n n ws identical source, target, and destination data types. and destination data identical source, target, Figure 9.12 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its rmats and vector regist E ove instructions with types ofdifferent operands. 31 0 e initializede theto word values shown in first eightfirstvector parameters are pass Figure Figure 9.12 GPR 2 Value Source Byte Word Halfword Table 9.8Table Values Index Element Valid is initialized as shown in have the resulting values of destinatio Figure 9.11Figure W2 Values Vector Source abcd Figure 9.10 W1 Values SourceVector ABCD element in the vector register in element Doubleword Data FormatData Element Index the stack pointer is always aligned to 16 bytes. aligned to pointer is always stack the th n 127127 64 63 64 63 0 0 Figure 9.16 through fclt.w $w4,$w0,$w1 # t is in $w4 t is in # $w4,$w0,$w1 fclt.w . Valid elementvalues indexvarious for data fo Valid . df Regular MSA instructions operate element-by-element with theing following of word additions sequence m and Figure 9.13 Figure that general-purpose register R2 Assume that vector registers W1 and W2 ar W1 and W2 registers that vector Assume not preserved between function between function not preserved calls. The compiled for the MSA, When MSA instructions the select Regarding theRegardingpassing vector parameter conventions, regi MSA mat Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 9.4.3 Examples 9.4.2 Element Selection MSA Vector MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 165

): Figure 9.17 em Programmer’s Guide, Revision 1.00 rating results rating on data formats twice as the destination’s data format.theThe data format of the destination’s Value for ADDV.W Instruction for ADDV.W Value its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its Value for SPLAT.W Instruction for SPLAT.W Value and half the width, i.e. word, in this case. and half the width, i.e. word, //into $w5. The .w indicates at the 128-bit MSA registers MSA at the 128-bit indicates The .w $w5. //into words. 32-bit four into divided //are w7 into result move the //and a * A + b * B * A + b a * D d * + C * c EEEE BBBB e on adjacent odd/even source elements, gene odd/even source on adjacent e a + a A+ + B b c + C + D d a + 17 b + 17 c + 17 d + 17 127 64 63 0 127 64 63 0 127127 64 63 64 63 0 0 127 64 63 0 Figure 9.14Figure Instruction FILL.W for W6 Value Vector Destination Figure 9.13Figure W5 Vector Destination Figure Figure 9.17 DOTP_S Instruction for W9 Value Vector Destination Figure 9.16Figure W8 Vector Destination Figure 9.15Figure ADDVI.W for Instruction W7 Value Vector Destination dotp_s.d $w9,$w1,$w2 dotp_s.d addv.w $w5,$w1,$w2addv.w move and $w2 $w1 and in operands two vector //add $w6,$2fill.w $w7,$w1,17addvi.w $w8,$w2[2]splati.w into w1 17 Add immediate operand. immediate //vector w6 $2 into register of GPR contents //replicate w8 of elements all w2 into word 2 of //replicate Note thatNote the actualinstruction specifies .D (doubleword)as Other MSA instructions operat wide. The signeddoubleword is an instruction such product dot DOTP_S (see being also signed inferred as is operands source

Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 166 MIPS64® I6400 Multiprocessing Syst

e processor is operating ates to the I6400 core to the I6400 core to ates t address translation, t address translation, excep- . The non-guest. The operating ot-mode exception and error Guests tion of guest Systems in Operating a from their guest-mode equivalents. ed for a guest operating system running operating modesbeas canreferred to compatibility is retained for existing kernel kernel existing for retained is compatibility ) to determineth whether and multiple DM bit is used along with ro Root ons, registers, and machine st register sets (or contexts) for guest mode operation, Debug tection of Virtualization, Gues Virtualization, tection of oduction to Root Guest operating and systems, and modes its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its handling, and an overview of Guest debug mode. respectively, to distinguish them respectively, odes guest-kernel, guest-user and guest-supervisor modes. The guest mode er and supervisorer modes to be retain ualization Module allows for the execu for ualization Module allows n of virtualized systems. The Virtualization Module is designedn of virtualized to enablesystems. full virtu- The Virtualization ) and the Debug Mode bit ( bit Mode Debug the and ) . The pre-existing. supervisorand kernel, user ERL root-supervisor and Status all times even when a guest is running. Backward a when even all times , root mode root EXL register contains the GM (Guest Mode) bit. This root-user , Status mode consists of newoperating mode consists of m GuestCtl0 fully virtualizedenvironment. This chapter providesof an overview the VZ module, intr operation,of registerin structuresoftware Guest mode, de tion handling in Root and Guest mode and interrupt mange the efficient implementatio mange the efficient alization of operating systems. The Virt allows theallows separation betweenus kernel, which is physically separate from, and a subset of, the Root Coprocessor 0 context. virtualizationThe moduleoperating contains a modes for one mode is known as root-kernel Guest withinmachine.a virtual The guest-kernel modeinterrupts can handleand and exceptions, managevirtual memory for guest-user mode processes. separationThe mode betweenroot theand limited-privilege guestmode allows root mode softwarecon-full to be in trol of the machine at software running in root mode. The The Virtualization Module defines the followingThe Virtualization elements: • Guest Operating Mode •Guestset (orCP0 Mode use for context) register Partial • Registers for Guest Mode control • Guest interruptsystem • features of Virtualization Detection provides Module separate Coprocessor0 Virtualization The The Virtualization (VZ) Module defines a set of newset of instructi Module a defines (VZ) Virtualization The status bits ( mode. root or mode guest in Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 10.1.1 Root and Guest Operating Modes MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 167 10.1 Overview Virtualization Chapter 10 Chapter

emulates all guest all emulates as a Virtual Machine Mon- Machine Virtual a as exceptions Guest-handled IRQs, ce-specific or board-specific ce-specific all times. When an operating an times. When all is running on a real machine is each user application, inter-process each user application, inter-process em Programmer’s Guide, Revision 1.00 guest-user guest-kernel each VM, and sharing resources between eret ode. The Hypervisor =1 CU0 ther than user applications. or. To ensure that it ensure that remains in control,Hypervisor the To or. ure facilities, plus any devi plus ure facilities, rol of machine resources at rol of machine st operating system as if it system st operating ent is a controlent program known is is intended to allowis VM schedulingtake place while to it becomes a ‘guest’ of the Hypervisor. Allper- operations it becomes a ‘guest’theHypervisor. of Guest.Status , ivileged state, and ensures that the system behaves as behaves the system and ensures that ivileged state, its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its erating system kernel. The hypervisor is responsible for man- eret Root-handled IRQs exceptions hypcall, if ze costs of context switching between VMs. between of context switching ze costs t created and managedHypervisorconsists the by Instructionfull of the ected virtual-memory environment for intaining the expected behavior for intaining the expected Hypercall Root-handled exceptions , which executes in the , which executes privileged m eret ns between operating modes. nts are full operating systems ra nts are full operating systems , the kernel (or ‘supervisor’) typically runs at a higher level of privilege than user ack of the guest view of pr l Privileged Resource Architect IRQs, Exceptions Figure 10.1Figure Modes Operating Between Transitions State root-user root-kernel eret IRQ, Exceptions transitio shows the state Reset peripherals and associated registers. It appears to each gue appears It registers. peripherals and associated control. with full and exclusive enablesand full virtualization, Module Virtualization The meeting real-time requirements, and to minimi In virtualization, the guest operating system operates in unprivileged mode. All privileged operations attempted by Hypervisor the traps back to the guest privileged operations, keeps tr privileged keeps operations, Virtualization is enabled by kernel is software.The key elem Virtualization communications, and I/O device sharing. The hypervisor performs the same basic functions in a virtualized system - clie that the Hypervisor’s except environmen virtual machine execution The Set Architecture, including al itor (VMM)‘Hypervisor’. or The Hypervisor is in full cont run within(OS) kernelsystem is a virtual machine (VM), formedmustguestby be explicitly a permittedHypervis by the always runs at a higheralways runs at a of level privilegea guest than op aging access to sensitive resources, ma aging access multiple VMs. In a traditional operating system a prot The kernel provides applications. Figure 10.1 Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 10.1.2 Introduction Hypervisor to the 168 MIPS64® I6400 Multiprocessing Syst

ations of theguest kernel. the Root TLB maintain an to execute from its original location as described above. mber of entriesmberinto of be equalof the number entries root TLB pages will likely result in better performance. anslation to match the expect match the anslation to time. MIPS recommends that its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its anslation are performed anslation are performed ings. The page sizes used in the root-modepage thesizes used in ings. The TLBmustconsid- be carefully ationallows an unmodified guest kernel root-mode software, while maximizingnumberguest-modetheTLB entries of entries for its own use to avoid cascading TLB evictions (thrashing). shows the outline of address translation in the Virtualization Module. shows the outlineof address translation Virtualization in the adequate amount TLB of reserved Figure 10.2 in memory, and allows and the hypervisor manage to tr address in memory, registerin Moduleand the I6400set execu-separate CP0 provides MMU for guest-mode a core Virtualization The In guest modetion.levels twotr of address nu that the MMU,recommends MIPS guest For the TLB-based control for sufficient allow ered to Larger which are mapped through each root-mode TLB entry. can be active at the same Both the guest and root MMU’s expected by the the expected by transl guest. Full address the root-context TLB used for Guest mapp for Guest used the root-context TLB Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 10.1.4 MMU Considerations 10.1.3 Enabling Mode Translations Guest MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 169

Guest ASID l Guest Virtual l Guest Virtual GuestID = N processor 0 register set are processor Y Guest TLB Y foral and Root ot register space and inaccessible in ot register space and inaccessible em Programmer’s Guide, Revision 1.00 Mapped? Guest CP0 Guest Exception? segmentation Guest exception N Y N ique non-zero GuestID. The GuestID value zero is GuestID The non-zero GuestID. ique l registers and can be accessed only by privileged and can be registers l means that a subset of the Co of the a subset means that Y MMU N enabled? its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ) represents a unique identifier unique ) represents a Guest? CP0 register is unique in the Ro CP0 register is unique in the Root exception RID RID Virtual Address (VA) Address Virtual ace is ace is identified by a un ) contains system contro system ) contains GuestCtl1 Guest Physical Address Physical Guest GuestID=N Root ASID is ignored GuestCtl1

or the Guest Operating System. rtualization in the I6400core ID N Y access to Guest TLB entries. to Guest access Y Figure 10.2Figure Translation Address of Outline Root Virtual Address Root ASID GuestID=0 Root TLB is an optimization,an is designed to minimize TLBinvalidation machine overhead on a virtualcon- ddress (PA) GuestCtl1 N

Mapped? Root CP0 Exception? segmentation Physical A N Guest mode segmentationmoderootthespace. onaddress controls the and guest mode effect have no MMU text switch and simplify Root text In the I6400 core, Coprocessor 0 (CP0I6400 core, Coprocessor the In instructions. The presence of vi The ‘GuestID’ field ( The ‘GuestID’ physically replicated for use by replicated for use physically guest mode. GuestID Address spaces. Each Guest’s address sp Address spaces. Each Guest’s reserved for Root address space. The space. address reserved for Root Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 10.1.6 in Root and Guest Mode CP0 Structure 10.1.5 Guest ID 170 MIPS64® I6400 Multiprocessing Syst

describes CP0 Mode rnal interrupt han- UNDEFINED Table 10.2 KSU 0110 Guest-Supervisor Guest-User s to the guest and root Coproces- guest and root the s to Status translation and exte EXL the minimum the minimum hypervisor intervention, 000 l the guest context. ot Coprocessor 0 are active. The presence of 0 are active. The ot Coprocessor Status for an immediateanswitch for between guest and root ERL Don’t careDon’t 11 UNPREDICTABLE guest mode behavior. mode behavior. guest 0 1 care Don’t are used to enter or exitenter are used to an operating or mode. Status its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its GM lization Module to contro GuestCtl0 systems such as timekeeping, address timekeeping, address such as systems ch to/from memory. Simultaneously accesse Simultaneously memory. ch to/from ileged code accesses to execute with to execute accesses ileged code KSU ssor 0 (CP0) ssor 0 (CP0) registers allows 0110 Root-Supervisor Root-User Don’t careDon’t 1 1 care Don’t Guest-Kernel Status Table 10.1Table Modes and Debug Root Guest, EXL tion, both the guest tion, both 0 Coprocessor the ro and 0000Don’t care 0000Don’t Root Guest Status Table 10.2Table Module Virtualization by the Introduced CP0 Registers ERL 0 1 care Don’t 11 4 GuestCtl0Ext Extension to GuestCtl0 12 6 GuestCtl0 Controls 1010 412 5 GuestCtl1 GuestCtl2 7 GTOffset Guest ID Interrupts Virtual value for guest timer Offset describes the describesvarious how thefields CP0 register Number Sel Register Name Description Register Don’t careDon’t 11 care Don’t Status DM dling continue to operate without major changes during guest execution. Table 10.1 and ensures that key root-mode machine that key root-mode machine and ensures sor 0 registers allows guest-kernel priv guest-kernel allows sor 0 registers Coprocessor 0 registers are added by the Virtua Coprocessor 0 registers are added by the Module. Refer to Chapterthisinformation.more manual 2 of for registers introducedVirtualization the by During guest During guest mode execu two simultaneouslytwoModule. The Coprocessor active is 0 contextsfundamental the operation to Virtualization the of two sets of Coproce of these presence a context swit without requiring modes 10 1 care Don’t care Don’t Debug Root-Kernel Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 10.1.7 New Registers CP0 Debug MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 171

Config3 context, and for a guest context, and R1 Write Reset State Read/ 0 ate of the VZ bit in the ate of the VZ bit em Programmer’s Guide, Revision 1.00 ode access to the guest CP0 the to ode access its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its implemented by checking the st implemented VZ = 1), and GuestID is supported, then explicit invalid TLB entry ll to root mode. VZ Invalidate Flush Invalidate Config3 ented. This bit indicates whether the the Virtualiza- whether indicates This bit ented. w instructions for root m w instructions d. This bit is always 1 I6400 core. for the Figure 10.3Figure Format Register Config3 of Virtualization odule not implemented odule is implemented Table 10.4Table Field Descriptions for Config3 Register Instruction Description tion Module is implemente m 0: Virtualization m 1: Virtualization Table 10.3Table Module Virtualization by the Introduced CP0 Instructions describes CP0 instructions introduced by the Virtualization Module. CP0 describes instructions introducedby the Virtualization TLBGPTLBGRTLBGWITLBGWR Guest TLB Probe TLB Guest Read Guest TLB Write Guest TLB Random to Write HYPCALLMFGC0MTGC0DMFGC0DMTGC0GINVGT - Hypercall ca TLBGINVTLBGINVFfrom Guest CP0 Move to Guest CP0 Move Doubleword Move from Guest CP0 Doubleword Move to Guest CP0 Invalidate Guest Global TLB TLB Guest Invalidate Guest TLB 23 Module implem Virtualization Software can determine if the Virtualization Module is Softwarethe can determine Virtualization if The Virtualization Module introduces ne introduces Module The Virtualization to make a call into root mode - a ‘hypervisor call’. a ‘hypervisor mode - root into call to make a Table 10.3 CP0 register. If Virtualization is supported ( If Virtualization CP0 register. support(EHINV) is requiredinto orderGuest invalid to detect be able for a in the Guest entries TLB. Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 10.1.8 New Instructions CP0 Name Bit(s) Description 31 30 29 28 27 26 25 24 23 22 16 VZ 172 MIPS64® I6400 Multiprocessing Syst 10.2 Detection Software

=1, GM e state which must state which must e ation, instruction Root.GuestCtl0 ed features within guest Coprocessor ed features within t, and then against the root CP0 con- root the t, and then against ese registers, such as TLB entries and entries TLB as such registers, ese it theis entire visibl ftware can access the guest Coprocessor udes address transl address udes r accesses and breakpoints. r accesses =0) and =0) DM =0)) =0) then tingERET instruction an when ine can be used to test whether processor is in guest- EXL EXL s and Guest mode operation the other. The software The visi- Guestand mode s operationthe other. subsetcomplete of the root Coprocessor 0 is implemented. =0) and its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its =0. themselves,also buttheresources which physical are shared DM DM lidity checks, lidity checks, coprocesso Root.Debug e root Coprocessor 0. Root mode so first tested against the guest CP0 contex itch context from one guest to another, itch context to from one guest another, ode accesses toodeor disabled unimplement accesses ich can trigger an exception. This incl This exception. ich can trigger an sters and any state which is accessed via th via which is accessed state any and sters ring Guest mode is mode is by execu ring Guest =0 and the GPRs, FPRs and Hi/Lothe GPRs, FPRs and registers. =0) and (Root.Status =0) and =0) and (Root.Status =0) and =1) and not ((Root.Debug =1) and not ERL =0) or =0) ERL ERL =1) and (Root.Debug =1) and GM GM GM Root.Status =1, ) then return(true) return(false) return(true) return(false) (Root.Status ((GuestCtl0 (GuestCtl0 (Root.Status EXL else endif else endif if (GuestCtl0 if ( endsub endsub subroutine IsGuestMode() IsGuestMode() : subroutine subroutine IsRootMode() : IsRootMode() subroutine The recommended method of ente 0. The guestCoprocessor0. partially 0 is populated - only a fetches, memory accesses for data, instruction va data, for fetches, memory accesses Guest mode software has no to th access and if required can emulate guest-m 0, Root.Status Segmentation Controlconfigurations. For a Hypervisor to save, restore or sw be saved and restored, not solely the replicatedregisters between Root and Guest, such as followingThe subroutine can be used to test whetherroot-mode.processor is in In guest mode, all guest operations are text.‘operation’ An is any process wh Root mode operation uses one set of Coprocessor 0 register of Coprocessor set uses one operation mode Root regi of these ble state is the contents mode. Guest modedetermined operation is as follows. This subrout Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 10.3.2 Guest Mode Operation 10.3.1 Root Mode Operation MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 173 10.3 Of Operation Modes

t operates mode in its own for the I6400 Virtualization Module. Module. Virtualization the I6400 for em Programmer’s Guide, Revision 1.00 , it has full access to all resources that are that to all resources access full it has , =0. Otherwise GuestID =0. Otherwise MMU RAD level addresslevel translation are invoked.isprocess The translation process the Root context, while Gues Root context, while the is operating in debug privileged execution mode (Debug its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its GuestCtl0 ASID ID =4) then // TLB type MT AddressDecode(vAddr, pLevel) AddressDecode(vAddr, sectionsthe describe subroutines called. Guest.TLBLookup(asid, GuestID, addr, IorD, LorS) addr, IorD, GuestID, Guest.TLBLookup(asid, Guest.OtherMMULookup(addr, CCA, LorS, pLevel) LorS, CCA, Guest.OtherMMULookup(addr,    GuestCtl1

Config =0) =0) then  Guest.EntryHi MT

RAD =1 or  =1. If the processor is running in Debug Mode processor is running =1. If the MT DM Config ignored asid asid (addr, CCA) (addr, # MMU=None case is undefined case # MMU=None UNDEFINED use LorS. BAT will or BAT. type, FMT MMU # Other CCA) (addr, GuestID GuestID  Config GuestCtl0 endif if ( if (mapped) then else else else endif CCA) addr, (mapped, if ( // This is a Guest Address translation Guest Address This is a // translation Address Guest Physical -> Virtual step 1: Guest // if ( Root.Debug // Initialization. if is only applicable GuestID // translation. address of in process (not applicable) is ignored // GuestID if (IsGuestMode()) if (IsGuestMode()) then * vAddr* Address - Virtual IorD* LorS* or DATA - INSTRUCTION type - Access pLevel* or STORE - LOAD type KERNEL - Access USER, SUPER, level - - Privilege * * Outputs pAddr* address - physical CCA* * mapped) when (valid attribute - cache functions See called Exceptions: * context. or root guest Called from * */ subroutine AddressTranslation(vAddr, IorD, LorS, pLevel) IorD, LorS, AddressTranslation(vAddr, subroutine /* Inputs described in top-down order - subsequent Segmentation,hardwareTLB lookups, TLB refill and second- The followingpseudocode the complete describes address For processors that implementOCI the debug, processor Mode) when Mode) when availableto Root Kernel Mode operation. RootDebug Mode, mutuallyand Mode Guest Modeare exclusive. At any given the time, processor only canbein one of the modes.three Note that Debug mode operates in unique context. Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 10.3.3 Debug Mode 174 MIPS64® I6400 Multiprocessing Syst 10.4 Pseudocode Address Translation

=0 and =0 and EXL =0, EXL,ERL !=0 and !Instruction) !=0 and =00 and Root.Status =00 RID KSU RID its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its is non-zero,Root.Status is RID GuestCtl1 =0 and GuestCtl1 0 DM AddressDecode(vAddr, pLevel) AddressDecode(vAddr,   =1 and Root.Status =1 and  ASID =0) DRG RAD ASID Root.TLBLookup(asid, GuestID, addr, IorD, LorS) addr, IorD, GuestID, Root.TLBLookup(asid, =1,GuestCtl1 =0, then guest entry ASID is global in Root TLB. in Root is global ASID entry then guest =0, GuestID GuestID GuestID  DRG RAD =0, then all root kernel data accesses are mapped and root are mapped accesses data kernel root all =0, then =0 and Debug =0 and (GuestCtl0 DM ERL Root.EntryHi addr vAddr  endif endif if (Instruction or (!drg_valid)) if (Instruction else 

   Root.TLBLookup(asid, GuestID, addr, IorD, LorS) addr, GuestID, Root.TLBLookup(asid, Root.EntryHi endif

GuestCtl0   mapped mapped addr CCA) (pAddr, (mapped, addr, CCA) addr, (mapped, asid pAddr pAddr endif Exception Guest for Execute-Inhibit Invalid, Refill, include may exceptions TLB // for Data. Read-Inhibit Modified, Invalid, Refill, Instruction, // Error include Address may exceptions related map Guest segment // Exception Root context in guest initiated Root exception This is a // TLB exceptions. all This includes // does not as guest not included, exception Error Address Segment map // map. segment lookup root // endif Exception Root context. Root in exceptions related and Segment all TLB Includes // Error Address an root-kernel,then is not by access and If drg_valid, // is caused. exception // if (drg_valid) if (drg_valid) then else endif if (!mapped) then (GuestCtl0 if else endif if (exception) endif translation Address Root Physical -> Physical step 2: Guest // // if pAddr if (exception) endif translation Root Address This is a // translation Address Physical -> Root Root Virtual // // If GuestCtl0 Root.Status // H/W must set G=1 for guest entry for TLBWI and TLBWR. for TLBWI entry guest for set G=1 H/W must // asid // and Debug guest. were for access the as if set G=1 must ignored.H/W SegCtl is // drg_valid endif if (exception) endif (pAddr,CCA) return else else end Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 175

s are omitted for All exception and Complete operation N el mode software). em Programmer’s Guide, Revision 1.00 odes (supervisor mode odes (supervisor Y Root CP0 Root exception? el throughcomplete the root kernel to the opera- slation rules defined by the root CP0 context are applied, d, and resulting exceptionsare taken in guest mode by the its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its root-kernel d by the root CP0 context is handled in root mode. root in handled is context CP0 the root d by N Y root-user SegmentLookup(vAddr, pLevel) SegmentLookup(vAddr, pLevel) LegacyDecode(, Guest CP0 exception?   context exception,thelayer next to be crossedCP0 contextisroot the (con- are handled in each of the operating m are handled in each of the est CP0 contextest CP0 (controlledguest-kern by in Root and Guest Mode and Guest in Root guest-kernel handler root-kernel handler ) then guest-kernel SC Figure 10.4Figure Mode Handling in and Guest Root Exception guest-user , an operation executed in guest-user mode must trav mode guest-user executed in , an operation // optional Segmentation Control based address decode address based Control Segmentation optional // CCA) addr, (mapped, CCA) addr, (mapped, shows the how exceptions the how shows Operation starting point Operation starting # Determine whether address is mapped address whether # Determine cache attribute and address physical obtain unmapped, if # - if (Config3 else endif CCA) addr, (mapped, return subroutine AddressDecode(vAddr, pLevel) : pLevel) AddressDecode(vAddr, subroutine endsub Figure 10.4 In clarity). Exceptions are handledmode in thecontext whose triggered the exception. An exception triggeredthe by guest CP0 context is handled in guestmode. An exception triggere Figure 10.4 tion. The first layer to be crossed is the gu translation rules defined by the guestCP0 context are applie guest kernel handler. If the operation does not trigger a guest- trolled by root-kernel mode software). All exception and tran and resulting exceptionsin taken root modeby the root kernelshown. handler as Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 176 MIPS64® I6400 Multiprocessing Syst 10.5 Handling Exception

Section cal TLB while the guest index such, the VTLB the VTLB such, must accom-

) RID RID RID RID RID RID RID RID ID ID ID ID ID ID ID RID RID RID RID RID RID RID GuestCtl1 GuestCtl1 GuestCtl1 GuestCtl1 GuestCtl1 GuestCtl1 GuestCtl1 /GuestCtl1 must first be by the permitted guest context ID GuestCtl1 GuestCtl1 GuestCtl1 GuestCtl1 GuestCtl1 GuestCtl1 GuestCtl1 GuestID and guest. The I6400 core contains a TLB structure that else else else else else else else om the bottom of the physi (Fixed page size TLB). As (Fixed page size its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its GuestCtl1 ( bit. However, access of guesttoqualified Coprocessor 0 is not However, bit. not shared with root. CU1 Status B. This is to avoid overlap of root and guest wired entries. On the other 1 (the Floating Point Unit) Point 1 (the Floating and guest in a shared structure. guest in a shared and the root index increases fr TLBP then if RootMode TLBR then if RootMode GINVT then if RootMode TLBWI then if RootMode TLBGP GuestCtl1 TLBGR GuestCtl1 TLBWR then if RootMode Table 10.5Table TLB Use by Instructions GuestID TLBINV then if RootMode GINVGT GuestCtl1 TLBGWI GuestCtl1 TLBGWRTLBINVF GuestCtl1 then if RootMode TLBGINV GuestCtl1 as Coprocessor as Coprocessor 0 state is TLBGINVF GuestCtl1 TLB Operation CU0 Status specifies the association specifies of GuestID with TLBinstructions. For supportinginformation, to refer bit,then and by the context root CU1 . Table 10.5 10.1.8 The I6400 core shares a common physical TLB amongst root In a shared TLB implementation, TLB a shared In incorporates a VTLB (Variable page size TLB) and FTLB page VTLB (Variable incorporates a bothmodateroot wired entries for For example, an access to Coprocessor access to Coprocessor an For example, Status by root context increases from the top of the physical TL of the physical the top from increases 10.5.1.1 Root and Guest Access to the Shared TLB Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 10.5.1 Root and Guest Shared TLB Operation MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 177

Config4 field to guest con- guest and instructions. Guest MMUExtDef Config1 instructions. in the guest CP0 context is st and root TLB operations operations TLBst and root rvisor isrvisor required to initial- d restore, the hypervisor is n, as is the case for any any case for the is n, as MTGC0 rely byrelycontext. the root CP0 TLBGWR and are present in the root context. context. root are present in the ch events can be generated as a by- ch events can be generated as Guest.Config4 inationofCP0 context the guest and and access to a privileged feature triggers privileged access to a em Programmer’s Guide, Revision 1.00 written. Since the entries allocated for written. Since the entries MFGC0 exception handler. The saved state and the The saved state and exception handler. KScratch2 registers determine which features are active and ile the guest is in operatio and guest, root software must be careful not to allocate careful be must and guest, root software exception occurs. Su exception ual boota Guest, of the hype e. On a Guest context an save equate number of scratch registers and restoresave to all eatures. Access to features with eatures. the machine is controlled enti the initialthewritable state of Guest context On registers. e bottome of the FTLB. Both gue KScratch1 events theeventspopulating guest from all remaining non root- its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its The Rootor The onedeconfigure may more registers guest CP0 Guest.Config7 TLBGP, TLBGR, TLBGWI TLBGP, through leaving no entries for non root-wired entries. register controls whether a guest whether a register controls ops processing instructions, saves sufficient state to resumetostate the interruptedops processing saves sufficient instructions, tries for guest. The Root then reads the The Root guest. for tries root mode by using the root-only the machine is controlled by the comb MMU size extension fields need to be MMU size extension GuestCtl0 number of wired entries to itself,andwrites then guest the Guest.Config0 Config4 ception or Error mode,ceptionstarts a software and or itialization and and Control itialization ster Management ster When in guest When in mode, behavior the of controlled from root mode. The must interpret the TLB index accordingly. index accordingly. the TLB interpret must The RootThe allocates the appropriate related fields to set the available VTLB en VTLB the available to set related fields The guest CP0 context consists of a base set plus optional f text. the rootWhen CP0 context. in root mode,of the behavior The Virtualization Module provides a partialModuleuse by the guest.CP0 registers providesa theis known This for of set as Virtualization The guest configurationthat is read-only to guest but writeableroot. by wired entries with its ownitswired entries with guest-wired entries, softwareRoot should change not guest MMU configuration wh determineof which the guest guest use also includeswired nonentriesby shared root both all remaining non root-wired entries to the guest. This pr TLB contents can be accessed by using the root-only the using by can be accessed TLB contents This ensures that hypervisor exceptionhypervisorThis ensures that handlers an ad have general purpose registersguest. in use by the product ofproduct instruction execution(e.g., an integer overflow caused by an add instructionTLB missor caused by a a load instruction), by an illegal attempt to use a privileged instruction (e.g. MTC0 from user mode), or by an event not directly related to instruction execution (e.g., an external interrupt). an exception occurs, the processor st When enters Ex instruction stream, Normal execution of instructions can be interrupted when an Normal execution of instructions hand,root the and guest indices to the FTLB grow from th Guest CP0 registers can be accessed from Guest CP0 registers can be accessed an exception. duringguest mode execution. The Root contextRoot software (hypervisor) is requiredmanageto power-up, the initial the power-up, state defaultshardware to the reset stat by writingguestthe to configuration registers. requires Module that scratch registers Virtualization The required to preserve and re-initializethe virt Guest state. For ize the reset state. Guest state equivalent to the hardware 10.5.1.2 Regi Wired 10.5.1.3 CP0 Register Allocation 10.5.1.4 CP0 Register Access 10.5.1.5 CP0 Register In Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 178 MIPS64® I6400 Multiprocessing Syst 10.6 Exceptions

GuestCtl0 registers, excep- registers, not enabled in root enabled in not . root mode.root External inter- Guest.Config tion, and the current state of the pro- of state the current and tion, ainst the guest CP0 context, and then Section 10.6.2 ode. If an interrupt is an interrupt ode. If mode switch is performed after the excep- switch is mode register, and exceptionsresulting from register, context can be handled entirely within guest be handled entirely within guest can context achine state to be saved to either the root or guest achine state to be saved esent and enabled by the by esent and enabled r, Cache Error are taken in r, d in guestthe CP0 context, the interrupt is taken in guest t-modeCP0 context and then by root modecontext CP0 can be generated - memory accesses, coprocessor coprocessor memory accesses, can be generated - ough the root-mode TLB are taken in root mode. Root.GuestCtl0 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its sultingthe from root-mode CP0 context (including whose CP0 state triggered the exception CP0 state whose e is saved. This allows m are a function of both the type of excep the type of both a function are est-mode operation be checked first ag in the guest context are pr 0 context, and if enabled are taken in root m duringguest mode execution, any required ception to be handledproper in the also mode. See ntext. Exceptions resultingguest from the CP0 Root Guest Guest Root     accesses, breakpoints and so forth. accesses, address translation of guestthraddress translation memory of accesses tions triggeredbytions those features in are taken guest mode. ctx ctx ctx ctx ctx • When architecture features • Exceptions resultingfrom control bits set in the •which exceptions for This operations includes all # Booleans, indicating source of exception: source indicating # Booleans, ## root_async # root_sync# guest_asyncexception context root - Asynchronous # guest_sync Root.Status.EXL, or set Root.Status.ERL context to root directed # Exceptions root context by triggered guest context exception by triggered - Synchronous exception - Asynchronous mode. root in the handler executes the processor that # meaning guest context by triggered exception - Synchronous conditions of exception # Ordering if (root_async) then then (guest_async) elsif then (guest_sync) elsif then (root_sync) elsif tion is detected and before any machine stat tion is detected and before any machine ex the and allows contexts, rupts are received by the root CP by rupts are received enablecontext,is bypassedandmode and is to the guest CP0 mode. When an exception is detected AsynchronousReset, NMI, MemoryErro exceptions such as permissions)a root require mode (hypervisor) handler. guestDuringexecution, mode the in mode exception whichan taken is is determinedfollowing: by the • Guest-modemust operations first be permittedgues by • taken in the mode Exceptions are always cessor. newadds Module rules for theof processing exception conditionsduringVirtualization detected The guest-mode execution. gu every that ‘onion model’ requires The co the root CP0 against address of the software address of exception handler mode without root-mode intervention. Exceptions re Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 10.6.1 Exceptions in Guest Mode MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 179

, refer to a virtual address GuestCtl0, Root.EPC tion was triggered by by root- triggered was tion BadVPN2 address that caused one of the fol- one of the caused that address de execution, the handler executes in is ideally the Guest Physical Address is ideally the Guest Physical Address em Programmer’s Guide, Revision 1.00 ddress (GVA) unmapped by the Guest MMU ddress (GVA) BadVPN2 is is the Guest Physical Address being accessed Root.Context . and BadVPN2 BadVPN2 ion through the guestisa mapped in if itTLB region of sters. The registers affected are sters. The registers affected Root.Context spective of whether the excep the spective of whether its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its not available for write to root then context, and Root.BadVAddr Root.Context Root.Context that captures the most recent virtual virtual recent that captures the most and and sult of a root TLB access during guest-moTLB access duringroot sult of a Root.BadVAddr xceptions from Guest Mode from Guest xceptions the guest context. A Guest Virtual A Virtual context. A Guest the guest Root.Cause , Root.BadVAddr ate is storedinto root CP0 regi after an exception, both indicate this. must null Root.EntryHi , register is a read-only register read-only register is a  GExcCode ctx ctx BadVAddr else endif root mode, and exception st GuestCtl0 stored The value in memory. The GPA presented to the root TLB is the result of translat TLB is the result root the to presented GPA The When an exception is triggered as a re Root.BadVAddr The (GPA) presented to the root TLB by (GPA) perspective. the root’s from GPA a is considered is the GPA is mappedMMU,Guest by the yet a GVA If The faulting address value stored into which is immediately usable by a root-mode handler, irre whichimmediately is usablea root-modeby handler, mode or guest-mode execution. by the guest. by the that ensures This process lowing exceptions. • Address error Modified • TLB Refill • TLB Invalid •TLB • TLBInhibit Execute • TLB Read Inhibit Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 10.6.3 Guest Initiated Root TLB Exception 10.6.2 Faulting Address for E 180 MIPS64® I6400 Multiprocessing Syst

Root Root Root Root Root Guest mode Taken in Taken Reset Debug Debug Debug Synchronous Synchronous Asynchronous Asynchronous Asynchronous Root Asynchronous Guest or Synchronous lowest. The table also lists new lists also The table lowest. mode in which mode in is recommended is recommended other exceptions, other exceptions, anslation (first Guest EXL was one serted. Prioritized its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its e processor Asynchronous so that one can single-step TLB instruction must cause a B. It is recommended that the is recommended It B. occur in guest ite, probe). It the processor. the Asynchronous Root guest (second step) address transla- guest (second step) address relative priority of each, highest to each, priority of relative ons to allow break on illegal detected by the processor. the detected by Root a guest address tr and root TLB operation (write, (write, operation TLB and root d. Prioritized above or or DINT) was asserted. Asynchronous eak condition was asserted. eak condition deferred deferred because deferred because EXL was one when the because EXL deferred upt occurred. Asynchronous Guest break condition was as condition break al was asserted to th s assertedprocessor to the Table 10.6Table of Exceptions Priority An internal inconsistency was that the be Machine-Check synchronous. A TLB instruction must Machine Check. cause a synchronous exception was detected, was asserted after EXL went to zero. A may watch exception root deferred Guest TLB related. Guest TLB related. This can only occur as part of and guest step), TLB operation (wr An imprecise debug data br data An imprecise debug A Machine-Check be synchronous. Check. Machine synchronous into interrupt (or other asynchronous) handlers. to was asserted signal The NMI TL or root guest probe) whether for including asynchronous exceptions, exceptions, including asynchronous part of a occur as can only This tion, root address translation, case it is prioritized higher than a simultaneous occurring guest guest occurring than a simultaneous higher it is case prioritized interrupt. when the exception was detected, was asserted after EXL to EXL went after asserted was detected, was when the exception zero. above instruction fetch excepti above addresses. instruction lists all possible exceptions, and the possible exceptions, all lists Table 10.6 exception conditions introduced by the Virtualization Module, andexception defines whetherconditions a switch to introduced root mode by the Virtualization is required before handlingexception. each Exception Description Type Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 10.6.4 Exception Priority InterruptDeferred Watch exception, Root watch A occurred. interrupt enabled A root Asynchronous Root Debug InterruptDebug Debug Data Imprecise Break interrupt (DbgBrk A debug Nonmaskable Interrupt (NMI) Machine Check related. TLB or Root Root, Soft ResetSoft Debug Single Step occurre A Debug Single Step The Resetwa signal InterruptDeferred Watch Guest A watch exception, enabled interr guest A Debug Instruction Break Instruction Debug instruction debug A ResetReset sign The Cold MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 181

Root Root Root Root Root Root Guest mode Taken in Taken or Debug Hypervisor Synchronous Synchronous Guest Synchronous Guest Synchronous Root Asynchronous em Programmer’s Guide, Revision 1.00 ception is raised ception is exceptions to allow . its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its s detected on an instruction an instruction on detected s n be taken. Instruc- Reserved instruction fetchinstruction Synchronous Guest a Reserved Instruction Excep- Instruction a Reserved loaded into PC. Synchronous Current Reserved Redirect is Instruction Reserved st context TLB entry mapping the entry st context TLB Refer to ‘Watch Registers’ - Registers’ to ‘Watch Refer on fetch exceptions to allow watch to on fetch exceptions valid Root TLB entry which had which entry valid Root TLB oot TLB entry mapping the address oot TLB entry mapping the address Section 10.8 on an instruction fetch. an instruction on otherwise follow standard rules of prior- ion was executed.ion was Synchronous =1, this root-mode ex ddress match was detected on an instruc- tched a valid Guest TLB entry which had which entry tched a valid Guest TLB RI above instruction fetch above instruction t of this processing. ruction addresses. ruction Table 10.6Table of Exceptions Priority . GuestCtl0 context watch address match wa before the ca before guest-mode exception tion Exception processing itization context within a given - taken as a side-effec The valid bit was zero in the gue the in was zero bit valid The fetch. instruction by an referenced address R in the zero was bit valid The fetch. instruction an by referenced to a Root or Guest translation. This can occur due thebit set. XI matched a fetch An instruction thebit set. XI to a Root or Guest translation. This can occur due A on an instruction fetch.cache error occurred instruction fetch. on an occurred error A bus Synchronous A triggers guest-mode instruction When tion. This can occur due to a Root or Guest translation. This can occur due watch on illegal inst watch Registers’ - Refer to ‘Watch was address A non-word-aligned an occurred on miss A Guest TLB occurred miss A Root TLB fetch. Prioritized above instructi above fetch. Prioritized on illegal instruction addresses. Section 10.8 watch a A guest-context Prioritized tion fetch. Exception Description Type Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies TLB Invalid - Instruction - Instruction TLB Invalid fetch TLB Execute-inhibit ma fetch instruction An - Instruction Cache Error fetch - Instruction Bus Error fetch SDBBP Instruc- Reserved Guest tion Redirect A debug SDBBP instruct Address Error - Instruc- fetch tion - Instruction TLB Refill fetch Watch - Instruction fetch Instruction - Watch root A 182 MIPS64® I6400 Multiprocessing Syst

Root Root Root Root Root Root Guest Guest mode Taken in Taken Debug Hypervisor Hypervisor Hypervisor Hypervisor Synchronous Current Synchronous Root Synchronous Synchronous Synchronous Synchronous Current Synchronous Root Synchronous EXL/TS EXL/TS . Status MSAEn was not allowed fer to ‘Watch fer to ‘Watch fer to ‘Watch fer to ‘Watch gal: Coprocessor Unus- Config5 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its re (address match only) or match a re (address ss to a coprocessor was permit- ss to a coprocessor initiated change to certain CP0 to certain change initiated est or Root address est or Root address translation, tch was detected on the address leted because it it because leted disabled. If exceptions occur on occur exceptions If disabled. match) condition was asserted. condition was match) ta addresses. Re ta addresses. Re ptions to allow break on illegal on illegal to allow break ptions Reserved Instruction Exception. Instruction Reserved Guest address translation, or a translation, address Guest st-mode could be completed not the required resources by the resources by required the detected by the processor. the detected by Root bits, but denied by ocessor Unusable, MSA disabled Prioritized above data fetch excep- Prioritized Prioritized above data fetch excep- Prioritized ng point, coprocessor 2 exception. 2 point, coprocessor ng rdware initiated set of rdware initiated ruction was executed. Synchronous CU1-2 . . ception occurred: Integer overflow, trap, trap, Integer overflow, occurred: ception resources, or was ille address match was detected on the address address on the detected was match address , but denied by Root. , but denied bits. register. Table 10.6Table of Exceptions Priority MSAEn CU1-2 Section 10.8 Section 10.8 Guest.Status . e debug data break on load/sto data break e debug Config5 able, Reserved Instruction, MSA able, Reserved Instruction, - Coprocessor unusable guest. Acce the ted by the same instruction, the Copr the instruction, same the over the Exception take priority This can only occur as part of a Gu Guest TLB related. This can only occur as part of a TLBP/TLBWI executed in guest-mode An internal inconsistency was execution, a software During guest occurred. fields register During guest execution, a ha or a TLBP/TLBWI/TLBGP/TLBGWI executed in root-mode. in executed or a TLBP/TLBWI/TLBGP/TLBGWI in gue executing An instruction becausewas it denied access to Root.GuestCtl0 system call, breakpoint, floati breakpoint, system call, occurred MSA disabled - guest. Access to the MSA unit was permitted by Guest. Root.Status access to the required required to the access An could not be comp instruction Registers’ - Registers’ A guest context watch address ma address context watch A guest store. or a load by referenced tions to allow watch on illegal da referenced by a load or store. or a load by referenced tions to allow watch on illegal da - Registers’ data addresses. Prioritized above data fetch exce Prioritized above data break on store (address+data data break on store Exception Description Type Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies Machine Check Root TLB related. HypercallGuest Software Field- Change Guest Hardware Field- Change inst hypercall HYPCALL A Guest Privileged Sensi- Privileged Guest tive Instruction Exception Execution ExceptionExecution Precise Debug Data Break ex An instruction-based precis A Instruction Validity Instruction Validity Exceptions Watch - Data access Data - Watch watch context A root MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 183

Root Root Root Root Root Root mode Taken in Taken ion. These exceptions exceptions ion. These These exceptions have e exceptions always e exceptions or Debug re to enter Debug Mode, even in Mode, even re to enter Debug Synchronous Current Synchronous Guest Synchronous Guest Synchronous Guest Synchronous ways be placed in a state. running Asynchronous em Programmer’s Guide, Revision 1.00 explains the characteristics of each to instruction execut to instruction execution. instruction to Table 10.7 aspects of the onous and synchronous. Thes s+data match only) data reference Synchronous its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its guarantee that the can al processor other exceptions because of the desi other last because all all because last re a switchare to root mode. st TLB entry mapping the address ng root TLB entry was found, and oot entry but the TLB was found, that occurs asynchronously asynchronously that occurs in to do data match. order a matching guest TLB entry was found, and was found, a matching guest TLB entry break on load (addres ed on a load or store to root mode. occurred on a data access.occurred on Synchronous Guest ed on a load or store data reference store or load on a ed a matching guest TLB entry was but the found, t-type exception that occurs asynchronously Table 10.6Table of Exceptions Priority describesexception.type the of Table 10.7 Table Characteristics Type Exception Table 10.6 Table always have the highest priority to requi always exceptions These very high priority with respect to with respect very high priority the presence exceptions, of other both asynchr require a switch An unaligned address, or an address that was inaccessible in the the in inaccessible was that or an address address, An unaligned instruc- store load or a referenced, by mode was processor current tion. A root TLB miss occurred on a data access. occurred on miss TLB A root or Guest translation. to a Root This can occur due valid (V) bit was zero. r matching a On a data access, valid (V) bit was zero. to a Root or Guest translation. This can occur due set. was bit RI the matchi a access, On a data read set. was bit RI the to a Root or Guest translation. This can occur due bit was zero The dirty in the gue referenced by a store instruction The bit dirty was zero in the root entry mapping the address TLB a store instruction. by referenced to a Root or Guest translation. This can occur due condition was asserted. Prioritized was asserted. condition data fetch must complete exception type. The “Type” column of “Type” The Exception Description Type Exception Type Characteristics Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies Asynchronous Asynchronous Reset a rese Denotes Asynchronous Asynchronous Debug Denotes a debug exception Address error - Data access TLB Refill - Data access TLB Refill miss TLB A guest TLB Invalid- Dataaccess On a data access, TLB Read-Inhibit a data On access, read Data TLB Modified - access access - Data Cache Error occurr error A cache Bus Error - Data access- Data Bus Error Precise Debug Data Break occurr error A bus A precise debug data 184 MIPS64® I6400 Multiprocessing Syst

obtained VEIC nd is reported pre- nd is reported precisely reported precisely Config3 and ions always require a ions always exception condition is VS exceptions with each other. In exceptions with each other. ons, they are either the lowest the presence of other exceptions. of other exceptions. presence the =1 (as appropriate) before anybefore =1 (as appropriate) instruction instruction excep- execution. These IntCtl ction execution, a , execution, and is ERL st supply the appropriate value for st supply the appropriate BEV rdware Field Change and Hypercall. instruction execution which requires hyper- which requires execution instruction Status , Root.Status at any time. When an at EXL e second way. These except These e second way. onous exceptions mainly for notational convenience. If notational mainly for onous exceptions =1 or Status , ere the exception was detected. ere EXL isor exceptions Guest PrivilegedSensitive Instruction, its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its is a relative priority of synchronous of priority relative is a ons as occurring between instructi occurring between ons as ecisely with respect to the instruction that caused the exception. the exception. caused that to the instruction respect with ecisely EBase be handled without switching modes. occurs as a result of instruction of occurs as a result tion that occurs as a result of instru re a switchare to root mode. switchare to root mode. to allow entry to Debug Mode, even in Mode, to allow entry to Debug ate in the guest CP0 context is not affected. CP0 context is not ate in the guest ception that occurs asynchronously that occurs ception to Root.Status e conditiona switch requiresmode, to root switch the is made before any , as described in the following pseudo-code. that occurs as a result of guest-mode of guest-mode that occurs as a result ses a guest exit to root, hardware mu hronous Hypervisor Exceptions Guest Software Field Change, Guest Ha Guest Software Field Change, all exception state is stored into root CP0 context, regardless of whether the pro- fined in the base architecture. base fined in the more information on these exceptions. more information on these GExcCode “GE”| guest mode at the point wh Table 10.7 Table Characteristics Type Exception 

minedvalues the from of tion, controltion, returned can be to root mode GuestCtl0 These exceptions always requi always exceptions These reported pr It is intervention. visor requi always exceptions These below prioritized be to tend exceptions These exception. the caused that instruction the to respect with of other types exceptions, but there some cases, these exceptions can priority relative to the previous instruction, or the highest priority relative to the next instruction. The The instruction. to the next relative priority or the highest instruction, to the previous relative priority them in th considers above the table of ordering to root mode. switch cisely with respect to the instruction that caused the exception. These exceptions are prioritized above above are prioritized exceptions These the exception. caused that to the instruction respect with cisely other synchronous exceptions tions are shown with higher priority than synchr shown with higher tions are one thinks of asynchronous excepti of asynchronous one thinks ExcCode and ExcCode Root.Cause Root.Cause Guest Reserved Instruction Redirect, from thefrom context in which the exception is handled. The General Exception entry point is used for new hyperv if guest exception is (GPSI or GSFC or GHFC or HC or GRR or IMP) thenguestor exception GHFC or HCor GRR or IMP) if is (GPSI or GSFC In the case of a guest exception which cau The vector location is deter During guest mode execu Exception vector locations are as de Exception vector locations Refer to the Exceptions chapter for Refer other state is saved. This ensures that that ensures saved. This other state is executing in root or cessor was detected during guest mode execution and th detected during exception state is saved. As a result, exception st a result, As is saved. exception state switchmoderootThe to is achieved by setting Exception Type Characteristics Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 10.6.7 Guest Exception Code in Root Context 10.6.6 Synchronous and Sync 10.6.5 Locations Exception Vector Synchronous Hypervisor Denotes an exception Synchronous Denotes any other exception that Synchronous Debug an debug Denotes excep debug Asynchronous other Denotes any type of ex MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 185

em Programmer’s Guide, Revision 1.00 ecution. Anecution.enabled interrupt source in or “HC” or “GRR” or “IMP” “GRR” or or “HC” or exts to determine when an interrupt should be taken. An its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its rrupt. Guests cannot disable root mode interrupts. “GPA”  “GPA” or GVA” “GPA” “GPA” or “GVA” “GPA” or “GVA” “GPA” or “GVA”

   

“MOD” “TLBL” “TLBXI” “TLBS” or “TLBL” or “TLBS” “TLBRI” ways active, even during guest mode ex active, ways      GExcCode “UNDEFINED” “GPSI” or “GSFC” or “GHFC” or “GSFC” or “GPSI”

  GExcCode GExcCode GExcCode GExcCode

= 0) then = baseline “ExcCode” ExcCode ExcCode ExcCode ExcCode ExcCode  IEC

GExcCode GExcCode ExcCode Root.Cause Root.Cause Root.Cause Root.GuestCtl0 Root.GuestCtl0 Root.GuestCtl0 Root.Cause # loading of GPA for both TLB-RefillTLB-Invalid and is recommended. # loadingof GPA Root.GuestCtl0 Root.GuestCtl0 Root.Cause Root.PageGrain shows how the virtualized interrupts are managedcore.I6400 in the if ( Root.Cause elseif (TLB Execute-Inhibit) elseif (TLB else endif Root.GuestCtl0 Root.GuestCtl0 endif elseif guest exception is (Root TLB-Refill or TLB-Invalid) The Virtualization Module provides a virtualized interruptThe Virtualization system for the guest. The root context interrupt system is al elseif guest exceptionelseif is (Root TLB-Execute_Inhibit or TLB-Read_Inhibit) inte a root-mode results in the root context always interruptenabledcontextroot in the is taken in rootinterruptAn mode. by masked root andin enabled the guest con- is takentext mode. in guest interrupts Roottake priority over guest interrupts. Figure 10.5 Standard interrupt rules are used by bothbyused interrupt root are rules and guest contStandard else elseif guest exception is (TLB Modified) Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 186 MIPS64® I6400 Multiprocessing Syst 10.7 Interrupts

IPTI =1) PIP[n] Guest.IntCtl . register, or direct direct or register, GuestCtl0 N VS via IntCtl No action IPPCI IP[n] or Y IV if implemented. Guest.Cause IntCtl IRQ? PIP Cause Cause Guest Guest handler propriate if both Root and Guest use propriate if both Root , Guest. ces are combined such that both meth- such ces are combined interrupts are not affected by the interrupts are not affected BEV GuestCtl0 or ofthis fieldor is controlledthe from root upt source indicatedupt source by the Status Timer, etc. field in non-EICinfieldor mode, hardwarea capture of Pending VIP nges to the guestnges to the interrupt usingstatus systemfield- the en rootguest and agree interpretation on the of interrupt Y ample, in non-EIC mode, if an interrupt pin (HW[5:0]) is its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its , whether EIC or non-EIC, whether provides EIC or following the capabili- system. Interrupt sour Interrupt system. N Root GuestCtl2 Pass? ys applied to the the interr to ys applied (e.g., Timer interrupts, Software interrupts)Software interrupts, (e.g., Timer No action registers. Direct assignment is ap the interrupt the source indicatedby field. Similarly, Guest software Similarly, field. N om the root context,rootom the enabled by PIP Cause pin to be visible to the Guest, possibly removing Root intervention capability. possiblyGuest,thetopin to be visible removingintervention Root capability. Y in EIC mode. IRQ? and Root can assert IRQ IRQ by assert can Root write to pendingfield GRIPL Root handler GuestCtl0 set by software write to Status field is the source of guestinterrupts. behavi The GuestCtl2 Timer, etc. Figure 10.5Figure l Module Virtualization in the Handling Interrupt Pending RIPL/IP field, and are always applied to PIP Guest.Cause guest interruptin External Sources Root must assign interrupts to Guest with caution. For ex For to caution. with Guest interrupts assign must Root shared by multiplebyshared interrupt enabling sources, then direct guest visibility (in Guest The • context guest the within generated Interrupts • Root asserted interrupts, causes all the interrupt sources on that RootIf Software needsguarantee toRoot intervention capabilityon an interrupt thenthat interrupt shouldnot be directly visible to Guest. non-EICIn mode, guest the timer interruptis alwa ties: A virtualization-based external interrupt delivery system GuestCtl0 assignmentreal interruptsignal of guest to the interrupt ods can be used. and related interrupts are available in bothTimers guestcontexts.rootand ofset pending The interruptsguest seen by thethe context combination is of: (logical OR) • External interrupts passed through fr field and is not affected by the affected field and is not changeexceptionswhich result guest from initiated changes to fields EIC mode, orEIC mode, if bothRoot use non-EIC mode. can track cha context. Two methods can be used to trigger guest interrupts - a root-mode write to thecontext. Two Software should enabledirect interrupt assignment only wh pending/enable fields in the Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 187

The other- PIP. PIP. , , fol- , Root vIP vIP IP[n+2] (Pending Inter- PIP PIP GuestCtl2 GuestCtl2 and GuestCtl0 vIP vIP Guest.Cause ementation. The term HW is GuestCtl0 ]=0)) vIP[n] set earlier. earlier. set em Programmer’s Guide, Revision 1.00 either by device activity or protected activity or device either by IP method to deliver interruptsto guest. so used by Root toso used by Root inject a virtual interrupt Cause which may be problematic since other interrupts IP intervention,followedguest by external servicing of ual drivers. The Rootual drivers. The injects an interrupt into the Guest ) or (MTC0[GuestCtl2 . atively referred to as IRQin other of sectionsModule. the can subsequentlycan interrupt the clear by writing 0 to

its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its handling in a recommended impl It HC[n] . Guest.Cause IP[n+2] vIP withroot-mode servicingof external interrupt. withroot-modeguest to by writing hand-off to handlingrelationis describedinGuestCtl2 to IP then HW[n] then visibleto is guest context through GuestCtl2 , rrupt delivery system also provides support for Virtual Interrupts.can Root rrupt delivery system also provides supportfor Virtual Root.Cause interrupts, then root should use this and in so doing causes a trap toand Root,a trap in so doing causes =1 ent way for Rootent way to save and restore interrupt state of a Guest,interruptan if the interrupt by writing to guest writing to the interrupt by 1 0 ]=1)  

Root.Cause PIP[n]

visible to guest context or root context is dependent on vIP[n] and vIP[n] vIP[n] is discussed below. IP IP HC (Interrupt Pending, Virtual). This method This is al (Interrupt Pending, Virtual). GuestCtl0 GuestCtl2 vIP[n] GuestCtl2 GuestCtl2 =0, but Root needs to transfer the external interrupt to Guest, then it must write to a software visible Guest.Cause Handling: . PIP[n] vIP, vIP, vIP vIP endif else if ((DeassertionHW[n] of and GuestCtl2 if (MTC0[ GuestCtl2 lowed by guest servicingof external interrupt. requiresroot visibility If into guest Hardware delivers interruptHardware delivers to root context, Hardware delivers interrupt deliversHardware to root context, interrupt. The interruptvisible is notmadeas rootroot to has the choice to assign to guest. Hardware delivers interruptHardware delivers to guest context without root GuestCtl0 GuestCtl2 GuestCtl2 into guest context. It is also a conveni into guest context. It is also a had been injected by Root, but needs to be preserved across context switches. In the absence of Virtual Interrupt capability can be used to support guest virt guest to support be used can capability Interrupt Virtual Root may then clear memory access. context. The Guest fields the interrupt, register, register, would need to derive the equivalent of vIP by reading reading by vIP of equivalent the derive to need would If applicationGuestCtl2 of rupt Pass-through).If 2. Guest assignmentof External Interrupt with Root Intervention. 1.Interrupt. External of assignment Root wise it is visible to root context through HW is a set of interruptset of pinscommonHW is a to both root and guestcontext. is Whether an external interrupt used to representused toan external interrupt source. HW is altern This section provides a detailed description of non-EIC could also be present. GuestCtl2 A MIPS enabled virtualized external inte enabled virtualized external A MIPS simulate a guest interrupt by writing 1 to 3. Guest assignmentof External Interrupt without Root Intervention. 10.7.1.1 Handling Interrupt Non-EIC Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 10.7.1 External Interrupts 188 MIPS64® I6400 Multiprocessing Syst

is 0 then is 0 then HC to be cleared. in an implementa- ), and one for the IP[n+2] ) to be cleared, thus thus to be cleared, RIPL . The associated guest GuestCtl2 specific source specific clear- by is fixed - it is either a vec- EID vIP[n] 1, then the deassertion of deassertion then the 1, IP[n+2] =1), two interrupt=1), priority-

. Hardware clear capability is capability is . Hardware clear is VEIC Root.Cause HC Root.Cause If a bit of If a

))) vector number to the core. vector number GuestCtl1 gnals are combined the field is writeable or preset to 1. writeable or the field is GuestCtl2 GuestCtl2 Root.Cause ability to distinguish between the two ability to distinguish HC[n] assignmentexternal of interrupts and GuestCtl2 ble interrupts for that ) or ) or Guest.Config3 to be cleared. CSS is also sent on each of the Root and Guest of the Root each also sent on is CSS shouldcause not vIP PIP[n] If a bit of If a

field. and GuestCtl2 vIP [n] =0, the assumptionisisexternal that nointerrupt there pported in the implementation. This is because the EIC onsible for combining internal and external sources into a tly there is no cap is no tly there RIPL by hardwareis,derived as or from theEICinput. A Gues- its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its bus. Thus the GuestID for the root interrupt bus may be non- lized core can from accept the EIC vIP[n] GuestCtl2 HC[n] causes corresponding to be In cleared. this case, it is the responsibility of root soft- is reset. is this capability to inject interrupts into Guest context for guest

. provides the means to root tosoftware distinguish between the cal virtualized EIC source a cal virtualized EIC source GuestCtl0 Cause vIP vIP ID GuestCtl2 HC vIP[n] vIP[n] est interrupttheest following buses as per rules: GuestCtl2 or (GuestCtl2 or GuestCtl2 ). The root and guest timerguestThe root and ). interrupt si GuestCtl2 GuestCtl1 at can be detected by the presence of by the at can be detected to transfer an external interruptHW[n] for guest servicing. Inthis scenario, GuestCtl2 vIP [n] vIP [n] . GuestCtl2 t context serves two purposes - root s are supported. This capability exists if RIPL PIP[n] ((HW[n] and ((HW[n] =

GuestCtl2 ways causes the associated the associated causes ways GuestCtl0 ned, the typeof vector a virtua e transfer. Otherwise, Root would have to disa e transfer. Guest.Cause n makes EIC mode available (as indicated by IP[n+2] IP[n+2] Onthe other hand,canuse Root . is asserted in bothis cases described. PL, the interrupt Vector (offset or number), and EI or (offset the interrupt Vector PL, =1 and the assertion=1 and the of IM[n] Handling: Handling: is providedto control how IP[n+2] IP[n+2] IP IP (HW[n] and !(

HC[n] HC Root.Cause = Guest.Cause Root.Status also an option,virtual even if interrupt In EIC mode, the external interrupt controller (EIC) is resp single interrupt-prioritysingle level,in which the appears guest context (affecting guest context (affecting with the guest bus is by default equal to ware to clearware to by writing 0 to the deassertion of HW[n] doescause not In the In the architecture as defi both. This is because curren but never offset tor number or Guest.Cause Virtual interrupt handling is an optionth Virtual GuestCtl2 two. Root software can use this facility can use two. Root software GuestCtl2 In summary, interrupt injection in gues summary, In injectionof virtual interrupts to Guest. Guest.Cause Root.Cause tion-dependent way with external inputstheproduce to root and guest interruptpriority levels. In addition to RI a guestinterruptfor GuestID moderoot taken inzero. The must be registeredin tID accompaniesthe onlyproviding root bus, GuestID is su an interruptsendalso forcan interrupt guest on the root interrupt buses. The Vector from the EIC is either utilized interrupt buses. The Vector When an implementatio tied totied the injected interrupt, and thus assertion of transparently affecting th transparently affecting level signals must be generated within the EIC - one for the root context (affecting related external interrupts al virtual device drivers, as an e.g. In this case, In this an e.g. as device drivers, virtual types, intentionally so.is Itrecommended a typi that EIC shouldassignThe interrupts to root and gu ing 10.7.1.2 EIC Interrupt Handling Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 189

to the EIC. EIC. the to

ID , the other for guest. GuestCtl1 ftware programs the EIC to context, followed by a write detects a change in RIPL on change detects a active guest is posted active guest on the command for the EIC. The EIC for the command the requested interruptpriority level changes and when it is visible to the it is visible changes and when em Programmer’s Guide, Revision 1.00 ID can enable guest modeIf an EIC operation. s are used, one for root one for s are used, GuestCtl1 nt on rootthe interrupt bus. An interrupt for the res- terrupt bus, then an implementation of thecore is st be presented onst be presented the root interruptbusEIC. by the and thusandpresented be on root interruptthebybus EIC. the guest interrupt bus. If so then and save restore guest tothe avoid above mentioned race. On a guest context t vector, depending on whether the EIC provides a vector t vector, An interrupt for theformerly its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ot context whenever hardware ot context whenever . This is equivalent to a STOP to equivalent is . This ID and non-resident must guests, the core send 0x200 EIC_vectorOffset EIC_vectorOffset 0x200 0x200 + (EIC_vectorNumber x (IntCtl.VS || 0b00000)) GuestCtl1 C. It is possibleIt is for an EIC implementationC. haveto active interruptsboth on     send interruptsguestto context by setting r the delay between when the r the delay . Once the write is complete, root software root context,root interrupt two vector different ID vectorOffset vectorOffset vectorOffset vectorOffset vectorOffset vectorOffset vectorOffset vectorOffset ector number (vectorNumber). Hardware shouldnotHardware interrupt stall the until the processor entersguest mode. . IPL GuestCtl1 endif else //GuestIDnon-zerois endif relative0x200to// EIC provides (GuestID=0) an offset if else //GuestIDnon-zerois if (GuestID=0) Config3.VEIC=1IntCtl.VS!=0 && &&&& Cause.IV=1Status.BEV=0  endif else // EIC provides vectorOffset if (EIC providesif vectorNumber) send an interrupt for a non-resident guesttheonin guest not required to respond to this interrupt. identonbe guest mayalsosent root the interruptbus. A guest interrupt while the processor is in root mode can cause an interrupt immediately unless masked by Root.Status And interrupt for a non-resident guest must always be se • Only an interrupt for a resident guest can be sent on • Rootinterrupts musttakenalways be in root context •guest interrupt a If requires root intervention, then it mu endif if EIC_mode Root interrupt bus. interrupt Root An EIC mode interruptguesteitheris generated in or ro EIC_mode EIC_mode offset (vectorOffset) or v (vectorOffset) or offset implementation and root software follow this recommendation, then this prevents loss of an interrupt posted to the guest interrupt bus while root is switching guest context. the respective interruptbusesthe from EI In thiscase the root interruptbus. priorityis alwayshigher the then guest interrupt. interrupt in of an For the case distinguishHardware is able to between the two by checking the GuestIDthe on root interrupt bus.following The pseudo-code describes how hardware generates the interrup EIC to avoid a spuriousEIC to avoid a interruptnon-resident for a from guestonsent beinginterrupt the guest bus. processorandThe EIC are required to implement a protocol switch,must first root software write 0 to To allow the EIC to distinguishresident between To An implementation must account fo An implementation recognizes this as a stall and does not a stall as recognizes this guesttheto 0 on interrupt buscan the tosoftware core. Root new GuestID to of Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 190 MIPS64® I6400 Multiprocessing Syst

is GRIPL

ss is disabled in EIC when the inter- GuestCtl2 providing if the interrupton the ESS EICSS cause a loss of an interrupt. cause d on this bus. Hardware can this bus. d on ion. Therebe maysome arbi- e root ERET causes instruction tCtl1ID. If they are nottCtl1ID. equal, thenare If they a STOP command on interface to on command a STOP returned to the rootinterrupt or guest context. If ser- Root.SRSCtl guest context. If a return to root mode appropriately. If acce If appropriately. Guest.SRSCtl ot context after having been received on ot context after having been received on SS and RIPL storage in root CP0 state is RIPL storage in root CP0 state and SS to to rrupt was received on the root interrupt bus, received on the root interrupt rrupt was ware must always send an acknowledgement into guest CP0 context on guest entry. intocontextCP0 on guest guest entry. CU[2:1] EICSS EICSS ed to the EIC as this may to the EIC as this may ed mode and hardware detects that Saving the fields as root CP0 register allows for nesting re may execute whatever software sequence it needs to. it needs sequence whatever software execute may re on of preventing hardware transfer by clearing the interrupt was not receive the interrupt was not , then the root ERET, then instruct g by a guest interruptg by a providing GuestCtl2GRIPL is non- GuestCtl2 Root.Status a change a to guestmode.meanwhile, In the another root iggered in root context, then the use of these fields does not iced in guest context, then th in guest context, iced ID its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its s the appropriate exception (Coprocessor Unusable in root con- , and and GuestCtl2 , root software must first insert ID RIPL eans, and be saved/restored must if necessary. providing GuestCtl2GRIPL is non-zero. GuestCtl1 GRIPL GRIPL . Otherwise, hardware forces use of hardware . Otherwise, because processor is in root mode. terrupt servicing either may continue in d ERET instruction where can respond EIC interruptan with on guestbus, e from the interrupt is transferred ro would be a non-zero value for the case of root injection. GuestCtl1 ndler must ndler must GuestCtl1EID compare Gues to d implementation, an acknowledgement is d implementation, EICSS ntext. The root ERET instructionto guest mode causes a transfer (when to root mode even if there was a change in to root mode even if there was Guest.Cause GuestCtl2 where the root interrupt bus Vector, EIC where the root interruptbus Vector, GRIPL to to erwrite the fields on the bus. on the bus. the fields erwrite D are equal, then save and restoreD Interruptneeded. is not servicing may either con- . Once quiescent, root softwa . Once quiescent, root when the processor enters guest enters when the processor on the guest interrupt bus as guest interrupt bus the on is received on the root interrupt bus, hard the root interrupt bus, on received is ID an interrupt into guest context after the inte GRIPL Root.SRSCtl e not protected by any m by not protected e GuestCtl2 GuestCtl2, GuestCtl1 GuestCtl2 is non-zero. Root software thus has the opti non-zero. Root software is before guest entry. , and not transfer back GRIPL GRIPL EPC hardware must ensure thathardware must twoacknowledgements ensure are not return In the case where an interrupt on the root interrupt bus. But in the case where the interrupt was injected into guest context by root, hardware should not send an acknowledgement determine this because determine this the root interrupt bus is caused trary delay betweentrary delay write of GuestID an hardwarebut trigger does not an interrupt A root interruptmust use EIC by writingto0 root context, then it is also disabled in guest and cause ar Hi/Lo registers text). This is followed by a write of newfolloweda write of GuestID to byThis is Hardware writes interrupt can occur which can ov which can can occur interrupt GuestCtl2 GuestCtl2 root injects the case where In is required, theninstructionthe HYPERCALL be used. must register, CP0 root The required because in a typical EIC-base rupt is triggered. If an interrupt for the guest is initially tr executed to effect is ERET instruction until the root occur Once in guest mode,guestOnce in the guestinterrupt handler completeswith an ERET instruction. The guest continues execution from its If GuestCtl1EID and GuestCtl1I intinue root or guest context.interrupt If theis be serv to interruptis for non-resident guest, and in vicingcontinueto is guest in context, thenmust the handlersave the first resident state guest (CP0, GPRs etc) follow- co restorebying of the new guest’s a GuestCtl0GM=1), followedinterrupt by a guest change to guesta mode(when GuestCtl0GM=1), followin zero. As describedany above, change in for If the interruptIf ha guest, then the is for non-zero. Access to COP1 FPR and COP2 may be protected setting root interruptbusis for any guest. wher the scenario in guest interrupt The of these fields, and thus supports nesting of of interrupts. nesting supports thus and fields, these of of optimizes the transfer Hardware Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 191

) PIP[5:0] terrupts. Hence the value on a read is generated on a from eudocode example. The example. eudocode result RIPL IP / em Programmer’s Guide, Revision 1.00 field and is the value used to determine value used to determine the is field and Guest.Cause RIPL/ IP rectly assigned external in is source of interrupt. source is as shown in the following ps its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its Guest.Cause ) GRIPL OR (irq[7:2] AND Root.GuestCtl0 AND OR (irq[7:2] IP[1:0] GRIPL is however preserved. is however vIP[5:0] IP/RIPL is not supported, then root writes Guest.Cause.IP writes then root not supported, is and from the status of di of from the status and IP[1:0] IP[1:0] vIP[5:0] vIP is incorporated in EIC. is incorporated GRIPL = 1) and = =1) =1)  G2 G2 = 0)) then = 0)) does not apply in EIC mode. in EIC not apply does != 0) and != 0) VEIC VEIC =0) = 1) and IP[1:0] IP[1:0] PT VS BEV PIP GRIPL GRIPL IV HW[5:0] GuestCtl2 Root_HW_VIP[5:0] Root.GuestCtl2     EICGuestLevel GuestCtl2   # if GuestCtl2 # if # root software injects interrupts. injects software # root r in a the write captures H/W context. in guest interrupt inject # to Root_HW_VIP. called register # shadow r r irq << 2) OR Guest.Cause 2) OR irq << irq irq irq GuestCtl2  else # h/w must clear if GuestCtl2 if must clear # h/w # All interrupts processed first by root. by first processed interrupts # All if (GuestCtl0 endif supported. passthrough interrupt # Guest if (GuestCtl0 - FDCI is only visible in root context. in root visible is only - FDCI else else r (Guest.Cause (Guest.IntCtl # State of Guest.Cause of # State # Guest in non-EIC mode in # Guest enabled. passthrough if guest in factored interrupts External # - if implemented here, applied interrupts Internal # - root. by injection interrupt for guest support Includes # - irq[7:2] if (GuestCtl0 else endif # Guest.Cause (Guest.Status mode. in EIC required GuestCtl2 # - if (EICGuestLevel > GuestCtl2 # - EIC must include guest interrupt sources in the EICGuestLevel signal EICGuestLevel the in sources guest interrupt include EIC must # - # if implemented. and PCI IP0 IP1, TI, Guest’s includes - This # Guest in EIC mode in # Guest GuestCtl0 # - else # Returns: # Non-EIC -# EIC IP7..0. : GuestInterruptPending() subroutine + IP1..0 << 2) (RIPL if ((Guest.Config3 written bywritten the root not maytheto be equalback. value read the value originallythe value written the root by value can be read bycan be readvaluefrom guest (and the root) the the The interrupt pending value seen pending value seen The interrupt by the guest is calculated whether interrupt a guest is taken. Notevalue that the returnedfrom Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 10.7.2 Derivation of Guest.Cause 192 MIPS64® I6400 Multiprocessing Syst

. This Compare Compare is allowed is writtenis by IP[IPTI] TI TI ) Compare Cause in non-EIC mode or PIP[5:0] IM and Guest ) Status IPTI Count and indirectly Guest . ble toble the guest. TI that is set when Guest.Cause that is set ) TI TI Cause IPPCI . Once Root transitionstothen guest mode, guest if there are multipleif there if Otherwise, guests in operation. . MIPS per baseline As architecture, a write to IP[IPTI] Section 10.7.2 TI its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its options are availa Cause Cause IP[1:0] or “Other External and Internal interrupts”. and Internal External or “Other t has been switched out, but its virtual timer, maintainedtrig- by root,hast is switched beenbut out, its virtual timer, TI ng() is subsequently qualified by Guest and Guest 1 0 is in root mode, then a match on Guest TI   before enteringthe guest mode guest. Guest would for takea timer interrupt, TI TI ]=1) TI Cause TI Cause is a hardware shadow copy of Guest.Cause TI TI r OR (PCIEvent << Guest.IntCtl << OR (PCIEvent r Root_HW_VIP[5:0] OR (irq[7:2] AND Root.GuestCtl0 AND (irq[7:2] OR Root_HW_VIP[5:0] r << 2 << r << Guest.IntCtl OR (GuestTimerInterrupt r r OR Guest.Cause r and Internal interrupts”is defined in      registers indicate which interrupt Root.Guest.Cause Guest.Cause , which would then clear Guest , would then clear which = Root.Guest.Cause Root.Guest.Cause

r Config in EIC mode, as per the base architecture. in EIC mode, as . else endif IP[IPTI] IPL TI Compare if (MTGC0[ r else if ((MTC0[Guest.Compare])) endif endif r r r Cause Status Cause endif return(r) endsub Root. Guest. timer interrupt can be signaled in guest mode. TI: of Guest Injection Root where “Other External Root mayRoot timer inject a interruptguest in context by setting Guest The value returned by GuestInterruptPendi The in an implementationtoGuest set clear Guest Fields in Guest Fields in Guest clears Guest may happen under the scenario where a gues scenario where a the happen under may Guest set would Root gered. Root maintainingRoot a guesttimer a virtualis recommended for one guest, but the processor only there is where Root.Guest.Cause Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 10.7.3 Interrupts Timer MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 193

Perf- is is and . PCI UNPRE- PCI as follows: registers but is registers but as 2 or 3. as 2 or PC and Root EC Cause PerfCnt(N+1) registers in Root CP0 tializes guest context. tializes guest register pair(s). These ed, encodings of the PCI ounter register. Root soft- ounter register. WatchLo WatchLo Config1 y Watch register support, or y Watch PerfCnt Guest.Cause Cause PerfCnt WatchLo WatchLo and = 1. to indicate to register interruptandmasks WR and EC[1] em Programmer’s Guide, Revision 1.00 WatchHi Status =1. If virtually shar tlythe manages performance counters, Config1 WatchHi PC performanceinterrupt counter by clearing the Guest access when it ini when Guest access ead of the last M 0. Guest bit must return ead of the last est context via Guest est context as such for duration of guest. for duration of such as t. A Guest may to virtual perfor- access have direct PerfCnt(N+1) Config1 shared by root and guest contexts. by root and guest shared emented in the guest context, access is access in the guest context, emented us range, starting from the least significant pair. It is It range, startingus the from least significant pair. h register support, Root-onl to a guest is not dynamicto a - once established initial after Guest access to a performanceGuest access to a c enabled context enabled , its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its IE Otherwise, counter overflow sets Otherwise, counter overflow sets Guest. a GPSI Exception. PerfCnt = 1 and in Guest if Guest does not require the ability to read/write fields is taken fieldsthe is from current Root or Guestcontext. lly choosellytotalthe subset to assign a of nce countersGu in the should returnroot WR M EXL lt) context, that context independen context lt) context, that ormance counter interrupt into Guest context, it must so by do setting the most- and en the allotment must remain en the allotment must Config1 PerfCnt(N) access does not cause pairs are allocated to guest, then guest r pairs are allocated to K, S, U, counter. Similarly Root may clear a guest counter. oot and Guest. Root typically configures the PerfCnt =0, then performance counters are unimpl are counters performance then =0, counters are virtually performance the =1, register PerfCnt in Root if Root . PC PC . Status field is Rootfield only virtualizationis notand control visible to the Guest. are set by hardware on counter overflow. overflow. counter on by hardware are set

IP[IPPCI] register(s) are never implemented in the Guest contex in the implemented register(s) are never EC Config1 Config1 as 0 or 1 cause a GPSI Exception to be raised on raised beException to a GPSI 0 or 1 cause as use of use interrupt behaviorgoverned is solely by EC IP[IPPCI] Cause PerfCnt DICTABLE PerfCnt pairs assignedGuest to in this mannermust be a contiguo • Guest The ware may choose to configure performance counters for legal Guest access by encodingby access legal Guest may choose to configurecounters for ware performance mance counter registers under root software management when A virtualized implementation may choose to provide no Watc handling applies to both Virtualized support. register andRoot Guest Watch generically referred to as “Watch” registers. registers. “Watch” to as generically referred Root and Guest Watchpoint debug support is providedCoprocessor by 0 Root and Guest Watchpoint PerfCnt PerfCnt Cause PerfCnt Root can configure the definition of performa registers are present registers are present Guest. enable. • Guest owned by guest. If all Root (defau Guest or to Once assigned includingenabledthen interrupts. for Root, Rootare ifcounters the performance E.g., further assumed that the allotmentperformance of counters guest GPSI), access (which caused th Cnt most-significant bit of the counter. Thus, Rootmost-significant bit of the counter. If Root software needs to inject a perf If Root software needs significant bit of the Software may choose to assign all performance counters to Guest or Root, butboth. not This is the recommended pol- icy for sharing between R Guest then assigned to Guest If an implementation may optionaAlternatively, context to Guest. Read of guest Read of context to Guest. The Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 10.7.4 Performance Counter Interrupts 194 MIPS64® I6400 Multiprocessing Syst 10.8 Debug Support Watchpoint

= WR WR WR Config1 Root Config1 Exception ers under Guest control. If ers under Guest = 1), then Root and Guest and = 1), then Root , and an MFC0 will return 0s Guest Match WR [1:0] writes 2 in2 this configura-[1:0] writes WM 0s on MFC0. If Guest to determine whether virtual register pair is assigned to Guest, is pair register Exception on on Exception WM WR GPSI exception. Root may choose to may choose GPSI exception. Root [1:0], will write 0, defaulting to RVA [1:0],defaulting will write 0, to RVA are enabled for Guest watch. A Guest for are enabled Config1 present. WM WatchHi port provided. port WatchHi Config1 [0] determines whetheris Root watching = 0) allows for Root Watch of Root Virtual of Root Virtual allows for Root= 0) Watch WM Virtual Guest Watch sup- Watch Guest Virtual WatchHi WR e enabled foror Root Guest watch. Referring to Access an MTC0 and return an MTC0 ss of a shared pair of regist pair of ss of a shared WatchHi Config1 Guest Exception on on Exception Guest gisters. Root emulation of Guest watch registers would C0 cannot modify its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its virtual Guest Watch via Root Watch registers (Guest via Root Watch virtual Guest Watch [1]. Root R/W StateR/W Function R/W (Root) return to the guestinstruction followingmove. the WM = 0, then no watch register pairs = 1 and Guest . conveys what support is available to Guest. to available is what support . conveys [1]=0 then a Guest access results in a results Guest access then a [1]=0 WR WR t Watch registers are present (Guest (Guest registers are present t Watch ap to Root. In sharing mode, once a watch ap to Root. In sharing mode, once a watch WR The I6400 will no-op The I6400 Table 10.9Table Control Watch WatchHi WM Function rmally except a MT rmally except Software can test R/W state of Guest can test R/Wstate Software Value Config1 Config1 Config1 WR Table 10.8Table Support Watchpoint Guest WatchHi 01 R R registers. No Guest Watch registers Watch Guest 0/1 R (Guest) Config1

, if Guest, if WM[1:0] = 1, then selected Root Watch register pairs ar = 1, then selected Root Watch Guest Root WR , the state of Guest Table 10.9 WatchHi WM. , this is determined by Root

Config1 WR WatchHi Table 10.8 0111 X0 00 10 11 RVA Root Watch RVA Watch Root GVA Watch Guest UNPREDICTABLE Reserved None GPSI None Watch None - Watch Watch None - - access is treated as UNPREDICTABLE. access assign this register pairGuestassignto register this at this point, or RVA or GPA. Root Watch of GPA is optional. A write of 1 to Root Root to 1 of write A optional. is GPA of Watch Root GPA. or RVA watch. Root3 to A write of Guestunder control, Guest can only If GVA. watch Watch operates independently. operates independently. Watch Debug definition also allows for The Virtualization = 1, then a access is treated Guest no If Guest In Table 10.9 0/1). This feature is optional. Root for Guest can setup registers withoutregisters Guest can setup Root intervention. Referring to There is no support for RootThere is no support emulationfor watchGuest of re Guest Watch registers are supported. Guest Watch require that every and write tr guest read tion, defaulting to GVA watch. Root can take away privilege from regis- Guest at any time by writingtion, defaulting to Root to Watch GVA on acce not take an exception does thus Root access The ters. undercontrol Root with Root Root-only Watch registers (Root Root-only Watch Addresses (RVA). If bothGuesand Root Addresses (RVA). Config1

Value Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies (in R/W State) Guest MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 195

10.1.8 Section Debug Technical Debug Technical Debug Technical Reference Manual Reference Manual = 0) = is, events occurring when = 1))

DM DM MIPS On-Chip Instrumentation On-Chip MIPS Instrumentation On-Chip MIPS ged execution modes; kernel, cessor is running in debug privileged cessor is em Programmer’s Guide, Revision 1.00 Root.Debug Root.Debug the Root context. context. the Root = 0 and = 1 or

ERL ERL not in guest mode. Root That intervention events. se a GPSI exception. are used to transfer to are used data est, there are three privile there are three est, its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its gisters (GPRs) and coproces- gisters (GPRs) and step process is required - to the provided to ini- instructions root context - TLBGP, TLBGR, context root - TLBGP, st use the Virtualization Module st use the Virtualization estCP0 context. accessing When 0 context and guest TLB, and to and guest TLB, and 0 context Debug mode, it has Debug mode, access to full ions in the processor pipeline. ions in the pipeline. processor Root.Status Root.Status data between the root and guest and the root data between that are available in CP0 context and guest CP0 context. e to Root-Kernel e to Root-Kernel mode operation. clusive with Debug mode. When in mode. Debug with clusive lication to the Virtualization Module. lication to the Virtualization = 1 or = 1 =1), the processor is processor =1), the

ere are four; kernel, supervisor, user andere are four; debug.kernel, supervisor, = 0 and = 0 and running in Debug mode, the memory map is the memory in Debug mode, running DM EXL access to debug facilities. When the pro to debug facilities. access EXL est access does not cau does est access Debug Root.Status Root.Status bug tools access general purpose re access general tools bug to Guest or Root,to but not Guest or both.recommended is the This betweensharingpolicy Rootfor and When the processor is in running availabl are that resources all determined by the root context. context. root the by determined instruct executing sor registers by Access mu to the CP0 context guest transfer to provided instructions operations These TLBGWR. TLBGWI, contexts - MTGC0 and MFGC0. Accessesuse must guest to the TLB tiate guest TLB operations the from between the guest TLB and the gu the guest TLB the in guest debug mode, a two- guest CP the to/from data transfer root the to/from data transfer Debug mode ( Debug mode = 1 and = 1 and ( = 1 , The debug privileged execution mode exists incontext. the rootA processor supportingvirtualiza- GM GM WatchLo Table 10.10Table Module Virtualization to Application and Features Debug lists debug features lists debug features and their app and Table 10.1 Feature Description Reference Debug mode is Guest mode ex mutually Root.GuestCtl0 Root.GuestCtl0 As per Table 10.10 supervisor and user, and in Root and in context, th and user, supervisor There is no facility for Guesttorelatedaddresses facilityThere is no to watchfor tion operates in two contexts, Root and Guest. Within Gu intion two contexts,Within Guest. operates and Root execution mode, it has full access to all resources execution mode, it has full access The Virtualization Module provides full The Virtualization Guest watch is enabledGuest watchin strictlyby is defined mode guest the equation: as ( The I6400 supportsThe I6400 virtual sharingRoot betweenGuest. and Rootmay As such, software chooseallassign to WatchHi Guest. If assigned to Guest then Gu assigned to Guest Guest. If the following equation is true: ( Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies Debug Segment (dseg) When the processor is Access to guest CP0 context De 196 MIPS64® I6400 Multiprocessing Syst 10.9 Mode and Debug Features Guest

10.6.4 Section 10.8 Section Refer to Refer guest software. debug are part of the root con- of the are part In both cases, the excep- cases, the both In its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its e Guest is optionally pro- optionally e Guest is ve during guest mode execution, guest mode during ve re breakpoints breakpoints are of type re Syn- kpoints to be used kpoints to be to used e of watch-point from th from of watch-point e mented, hardware breakpoints mented, hardware vided. chronous Debug or Asynchronous Debug. Asynchronous Debug or chronous mode. handled in Debug tions are Exceptions resulting from Exceptions resulting from hardwa text. The root remains acti context hardware brea allowing Table 10.10Table Module Virtualization to Application and Features Debug Feature Description Reference Watch registersWatch for us Support Hardware Breakpoints When imple Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 197 em Programmer’s Guide, Revision 1.00 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 198 MIPS64® I6400 Multiprocessing Syst

. started by the con- Table 11.1 Cells can be configured to Cells can be types appear as a single a single appear as types index into the SAAR reg- SAAR the into index The ITU memory region is a fixed is region memory The ITU on between threads. It is achieved achieved is It threads. on between ovides an alternative to Linked-Load/ an alternative ovides d if necessary and re necessary d if r block is being accessed. There is one r block . There is one SAAR register per core. per is one SAAR register . There storage cells. Both cell store instructions. number of extra instructions required to perform them Register Index. Provides an eading utilizing by Gating gating storage. storage is used formation and associated data. mix of single element cells and multi-element FIFO cells. its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its d to facilitate access to the ITU as shown in to the ITU as access to facilitate d uration based on customer requirements. customer on based uration TU) is a configurationis optionTU) that pr SAARI register per VP. size block the as well as located, be ister to indicate whether the ITU or othe ister to indicate whether multi-element FIFO data eby allowing Inter-thread communicati allowing Inter-thread eby or store request can be precisely aborte request can be precisely or store ) is checked against base and range to validate access. validate range to and ) is checked against base these registers is these registers is as follows. Table 11.1 Table Module ITU the Accessing for Used Registers CP0 9 7 SAAR ITU will where the address the base Stores Register. Access Address Special 9 6 SAARI Address Special Access The bit assignments for each of bit assignments The Two new Coprocessor 0 registers have been adde 0 registers Coprocessor new Two The ITU module provides the provides followingThe ITU module features: • 64-bit data path •read and write operations8 Byte or 4 • Eliminateslocking spin encounteredthe with LL/SC instructions • Provides support for semaphorewhich types reduces the • Root physical address (RPA The Inter-thread Communication Unit (I Communication The Inter-thread trolling operating system. The ITU is memory mapped and is accessed through load and to synchronize execution streams, ther streams, to synchronize execution Store-Conditional synchronization Store-Conditional for fine grained multithr may whichrequests be blocked through untilofstate the Load location the storage changes, and Store allowing a given. A blocked load to be response addressable memorylocation,and the ITU can contain a The cells are configured during IP config size of 128 KBytes. Each cell contains tag state in made up of cells. ITU is The be single element data cells, storage or Number Sel Register Name Description Register Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 11.1.1 New CP0 Registers MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 199 11.1 Overview Inter-Thread Communication Unit Communication Inter-Thread Chapter 11 Chapter

programmed into the 65 0 ers is indirectly accessible indirectly accessible ers is R/W 0 for more information. em Programmer’s Guide, Revision 1.00 Thread Communication Unit (ITU). The Thread Communication ce this information is eld is encoded as 2^SIZE to indicate the size P0 Register 9, Select 6) 9, Select P0 Register e ITU. This field is stored in bits 43:12is stored in bitsfieldThis e ITU. of the SAAR rdware register. This regist register. rdware Accessing the ITU Module Accessing more for information on howthe SARRI register is for more informationmore for how on this register is used. h corresponds to a decimal value of 17. This means that the iple SAARI registers use the same SAAR register to access to access register SAAR the same SAARI use iple registers its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its d by the SAAR register. d by the SAAR register. nd stores the base addresstheof andnd stores base block. the ITU size itten as zero. Reads are undefined. R 0 per-VP and provides an index provides and that value ITU.is used to select the Since per-VP Reserved TARGET e of the TARGET field is always 0x00DSPRAM module as theis not present. TARGET the e of it TARGET field that the selects Inter- it TARGET s Register Index — SAARI (C ogic block to be accesse block to be ogic Figure Figure 11.2 Format SAAR Register Figure Figure 11.1 Format SAARI Register 0 of this register enables ITU accesses. On 0 of this register enables ITU accesses. bits should be wr Register Programming Sequence Register Register Programming Sequence Programming Register register encodes the size of the ITU. This fi of the ITU. This the size register encodes Table 11.2Table SAARI for Register Field Descriptions This field is encoded as is follows: field encoded This ITU (default) 0x00: 0x03F: Reserved - 0x01 values will to reserved be dropped. Writes 5:0 l the between Selects 31:6 This Reserved. Reserved ADDR[47:17] Reserved SIZE EN CP0 SAAR,CP0 is it moved hardware by into the SAAR0_ITU ha sectionRefer to the entitled through the CP0 SAARregister. toentitled the sectionRefer used. default ITU128size is 2^17, KBytes. or An ENABLE bit located in bit The 32-bit ADDR[47:16]The indicates field th the of base address register. 5-bitThe SIZE field of this of the ITU. The default for this field is 128 KBytes, whic The 64-bit SAAR register is instantiated per-core a per-core instantiated is register 64-bit SAAR The Note that in the I6400 MPS, the valu the ITUblock. SAARI register contains a 6-b Each The 32-bit The 32-bit register SAARI is instantiated this means mult that SAARI there is one register per VP, is set to 0x00valueblock.ITU to select the toentitled the sectionRefer 11.1.1.2 7) Select 9, — SAAR (CP0 Register Register Address Access Special 11.1.1.1Addres Access Special Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies Name Bit(s) Description Read/ Write Reset State TARGET Reserved 63 44 43 13 12 6 5 1 0 31 200 MIPS64® I6400 Multiprocessing Syst

. Note EXEC ERROR_ Table 11.6 PARITY ERROR_ R IP Config R/W 0 R/W 0 R/WR/W 0x11 0 R/W1R/W1 0 0 AXI ERROR_ . is memory is memory mapped. when PA[6:3] contains a when PA[6:3] at least 128 KB- at least 128 Section 11.2.5 ITU. This register ITU. This register formation about the ITU configuration and the ITU configuration about formation its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its address must be ons for SAAR Register Register SAAR ons for te an error te an error on the AXI bus. Software s out the cell addressing by virtually the cell addressing by out s cell 0, but can be accessed the base physical address for the loca- the for address physical the base cate a parity error occurred when occurred error a parity cate 0 of the ITU using cell view 0xF as described in 0xF using cell view of the ITU 0 itten as zero. Reads are undefined. zero. Reads itten as Rare undefined. zero. Reads itten as 0 R 0 itten as zero. Reads are undefined. zero. Reads itten as R 0 itten as zero. Reads are are undefined. Reads zero. as itten R 0 access to cell 0. 0. to cell access rol by mapping individual or groups of d contains the number of cells configured in the in cells configured of the number contains d Encoded bytes. as 2^SIZE Encoded Figure 11.3 ICR0 Format Register controls for the global controls for the global operation of the It is not physically It is part of not value of 0xF during an is implementation dependent. Table 11.5Table ICR0 Register for Field Descriptions Table 11.3Table Descripti Field ITU. Maximum number of cells is 32. number ITU. Maximum in examples the Refer to pages. different across cells replicating the cells, giving multiple addresses to access the same cell. cell. same the access to addresses multiple giving cells, the replicating This is to allow cell access cont error. the clear to bit this set must accessing the ITU module. Software mustset this bit to clear the error. tion of the ITU block in memory. The The memory. in block ITU the of tion aligned (ADDR[47:17] = PA[47:17]). aligned (ADDR[47:17] = PA[47:17]). For SIZE field should be 0x11. 128 KB, the rent value of the bit. the rent value of Table 11.4Table the ITU Module Accessing for Used Registers CM3 ICR0 0. Contains in register ITU Control 2hardware to indica This bit is set by 1hardware to indi This bit is set by 0 cur- the A gives read accesses. ITU allow to be set must bit enable This 7:3 This bits should be Reserved. wr 5:1 Size of the device. 10:8 This field spread granularity. Block 12:6 as zero. be written Reserved. bits should Reads are This undefined. R 0 15:11 This bits should be wr Reserved. 23:16 This fiel Cell number. 63:24 This bits should be Reserved. wr 63:44 be wr Reserved. should This bits 43:13 specifies This field Address. Base Register Name Description Reserved CELL_NUM Reserved BLK_GRAIN Reserved that the total number of cells This register can be modified when accessing cell accessing can be modified when This register EN SIZE Name Bit(s) Description Read/ Write Reset State Name Bit(s) Description Read/ Write Reset State Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies Reserved Reserved Reserved Reserved Reserved 11.1.2 ITU Control Register CELL_NUM BLK_GRAIN ERROR_AXI ADDR[47:16] 63 24 23 16 15 11 10 8 7 3 2 1 0 ERROR_PARITY MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 201

es the FIFO depth,es the R/W1 0 Cell

0 1 2 3 l. The exact operation is l. The exact operation

Tag

Storage

Entry Entry Entry Entry

State

em Programmer’s Guide, Revision 1.00 FIFO FIFO FIFO FIFO Cell (continued) element ngle-element cell. In a 4-entry FIFO as ngle-element cell. In a 4-entry ‐ state information includ state information Multi full state of the target cel target full state of the ng cell data update,datang celland cell state update, load/store its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its e first entrywrittenfirstfirst. read outFIFO is to the e g cell types, view, state, and indexing. state, view, cell types, g y 0 of the FIFO. Subsequent writes cause theSubsequent data to be shifted of writes cause 0 the FIFO. y te an execution error during an access an access error during te an execution set this bit to clear the error. clear the to set this bit Data

Cell bit data value and associated state information. Multi-element FIFO data ry cells. Two cell types are supported; cell Two ry cells. l structure, includin Cell

64-bit data values in a FIFO The format. FO cell is accessed in the same manner as a si as a same manner accessed in the cell is FO Tag

Storage

Data Figure 11.4Figure Cells Storage and Multi-Element Single-

, a write operation writes to entr to writes , a write operation State to the ITU module. Software must module. Software ITU the to

FIFO data storage cells Cell Table 11.5Table for ICR0 Register Field Descriptions Cell element ‐ 0to indica hardware This bit is set by Figure 11.4 Single storage cells contains multiple contains storage cells full/empty indication, and read pointer to indicate which entry in the isFIFO being read. Note that a multi-element FI ITU supportsThe numerouskinds of cell operations, includi Single element storage cells contain a 64- storage cells element Single •storage cells data Single-element • Multi-element throughread operation, th On the entries in the a FIFO. blocking/nonblocking semphore requests on the empty/ based The ITU consists of a series of memo a series of The ITU consists of This section describes the ITU cel describes This section shown in Name Bit(s) Description Read/ Write Reset State Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 11.2.1 ITU Cell Types ERROR_EXC 202 MIPS64® I6400 Multiprocessing Syst 11.2 ITU Cell Structure

ITU Cell AddressingITU Cell e requesting thread. e requesting for more information. type of cell can be accessed only in only in cell can be accessed of type on multi-entryon FIFO cells.theIn undefined when using this view. undefined when using using P/V view. All views are legal All views are view. P/V using ad the headad theFIFO. of the Bypass l information. neither depending on the number of the number on depending neither tion is undefined when using this undefined when using is tion for hardware messaging queue sup- hardware messaging queue for pass a single value between threads value between pass a single effect on cell state/control informa- cell state/control on effect will block the will block thread. requesting Cell Views empty cell blocks th ailable stored value. cell state/contro above. Each of these views is described in the following of these views is described in Each above. its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its tion. Operation of the SC instruc view. if it is on completion, flag, empty cell set the will A Load request av the last returning to a full cell request A Store is the cell if completion, on flag, full cell set the will request A Store after storing value. the last full possible is the SC instruction of Operation Table 11.6 the physicalto address. the section Refer entitled entries already pushed onto the queue. onto the pushed entries already Note, P/V type views are not allowed will re operation a read view, Bypass tail. the overwrite will stores In the multi-entry FIFO cell type a single memory mapped word contains contains word mapped memory single a FIFO type cell multi-entry In the allowing FIFO, a in entries multiple or full can be empty, The FIFO port. In the single entry cell type a single cell type a memory mapped word contains entry In the single to This can be used single entry. counter using E/F view as an event or for single entry cells. for a definition of E/F and P/V. a definitionP/V. and for of E/F Table 11.7Table Cell Views . Refer to the section to theRefer entitled . 2 1 address when ITU is accessed. the Each Table 11.6 Table 11.6 E/F try E/F E/F try E/F P/V try Table 11.6Table by Cell Type Supported Views Cell Control Control E/F synchronized E/F synchronized P/V synchronized 0000 Bypass No of cell data, no or Store Load 00010010synchronized E/F Control Yes to an Load request No A of or Store Load Cell TypeView Cell Description Single Entry Bypass Addr[6:3] Cell View Blocking Description for more information.to Refer The cell can operate in any of the views listed listed the views in any of cell can operate The table. The cell viewis indicatedin bits 6:3 of certain cell views as shown in as shown views cell certain embedded into bits 6:3 of the physical the 6:3 of into bits embedded Multi-entry FIFOMulti-entry Bypass 1. E/F = Empty/Full 2. (increment) (try)/Verhoog P/V = Probeer Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 11.2.2 Cell Views MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 203

be cleared as part of the initial- the of part as cleared be be cleared as part of the initial- the of part as cleared be em Programmer’s Guide, Revision 1.00 undefined when using this view. undefined when using undefined when using this view. using undefined when ons referencing the E/F Try view ons Try the E/F referencing ll affectnot cell state. omic post decrement by 1. (Decre- by 1. omic post decrement based on whether the ITU store suc- based the on whether turates at 16 bits (0xFFFF) turates at 16 bits (0xFFFF) atomic post decrement by 1. atomic post decrement cell will fail, returning a value of 0. of value a returning fail, will cell not affect cellstate. not affect cellstate. cell value is not 0 ) cell value is not 0 will block if the cell value is 0. the cell value is if block will its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ization ization for P/V semaphore use. is the SC instruction of Operation A Store request will never block. request A Store by write request's 1 data increment and an is ignored A Store occurs. sa increment request's A Store will requests Load/Store The empty and full state bits should A Load which wi request, fails, silently. fail will it not block, will to a full cell request A Store state. cell affect will not fails, which request, A Store instructi Conditional) SC (Store failure or success indicate will fails.ceeds or value; cell of bits significant least 16 the return will request A Load causes an extended, and zero will never block. request A Store by write request's 1 data increment and an is ignored A Store occurs. sa increment request's A Store will requests Load/Store The empty and full state bits should ization for P/V semaphore use. is the SC instruction of Operation value; cell of bits significant least 16 the return will request A Load zero extended,causes and an at occurs if ment only A Load request from reads the ICR0 register. ICR0 register. the to writes request A Store Table 11.7Table Cell Views 1111 ICR Register No (ICR0) access register ITU control 0011 E/F/try Noempty to an Load request A 0100 P/V synchronized Yes Load request A 0101 P/V try Noblock. never will Load request A Addr[6:3] Cell View Blocking Description 0110 - 11100110 Reserved No Behaves the same as the Bypass view. Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 204 MIPS64® I6400 Multiprocessing Syst

0 1 - 3 1 - (Multi-element) (Single-element) . The cell state The cell . R IP Value Config R0 R IP Config Value R IP Config Value R/W 0 Table 11.7 e will be returned returned be will e encoded encoded as follows: its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its , where n is the value in this field. field. in this value n is where the , e even FIFO entry. In this configura- In this entry. FIFO e even igured as multi-element with an 8- n oldest oldest entry whose valu pends on the size of the FIFO_RD_PTR pointer depends on the state of the ew when PA[6:3]) = in 0001 as described ew when PA[6:3]) address that is organized as follows. is organized address that ts 29:28. This field is Table 11.8Table State Cell ion bit 20 is reserved. ion bit ta (single-entry cell type) cell (single-entry ta = 1 entry = 2 entries = 4 entries = 8 entries 0 1 2 3 Notethatsingle-element cells this field isalways0. field. See below Indicates that is the cell FIFO depth 2 01: 2 10: 2 11: 2 This means that he FIFO can be encoded as as follows: be encoded can FIFO that he means This 00: 2 on The the next size of this read. FIFO multi-entry 1 = FIFO FIFO (multi-entry in the of entries number maximum the contains Cell type) cell FIFO_DEPTH field in bi Bit 18: Used when the cell is configured as single-element, or as a multi-ele- single-element, as configured is cell the when Used Bit 18: ment cell with a 2-entry FIFO. A logic 0 bit 18 on selects the odd FIFO and a logic 1 bit 18 on selects th entry, reserved. bits 20:19 are tion 4-entry a with multi-element as configured is cell the when Used Bits 19:18: configurat FIFO. In this Bits 20:18. Used when the cell is conf entry FIFO. R de field this of size The Reserved. FIFO_RD_PTR Indicates the pointer value for the The ITU is accessed using a 48-bitaccessed using physical The ITU is provides the following information. The cell state can be accessed in the Control vi in the accessed can be state The cell Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 1 Full Cell contains valid da 0 Empty data.no valid Cell contains R/W 1 11.2.4 ITU Cell Addressing 11.2.3 Cell State 18 17 FIFO_TYPEcell entry = single 0 16:2 R Reserved. R 0 Bits Field Description Type Reset 19:18 20:18 27:19 27:20 27:21 63:3029:28 FIFO_DEPTH R Reserved. R 0 MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 205

. Section 11.1.1.2 Section ds of the ICR0 register register ICR0 the of ds . . cells are used, there are no are cells are used, there the followingsubsections. . Table 11.7 em Programmer’s Guide, Revision 1.00 FO cells. The value configured in the value configured in The FO cells. Figure 11.6 Table 11.7 ss bits are required to access these 32 cells, the these 32 cells, to access bits are required ss e always 0. Bit 2 is used to select between the upper and 0. Bit 2 is used to e always GRAIN field as describedGRAINin field as its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its :12 are don’t care. Since all 32 care. Since :12 are don’t , the cell indexwithinsomewhere falls 15:7 bits of the physi- e CELL_NUMfiel and BLK_GRAIN ts43:12of the CP0 SAAR register described in mmedintoBLK_GRAIN the field is zero. Figure 11.5 on the programming of these fields. on the programming into 16 single-element cells and 16 FI 16 and cells 16 single-element into dress Care Don’t Index Cell View Cell Offset No Index Shift and No Invalid Cells No Index Shift Figure 11.5Figure Format Addressing Cell ITU . As shown in . As , the cell index starts at PA[7]. Since 5 addre PA[7]. at index starts , the cell Figure 11.6 Cells with Example 1: 32 No Index Shift Section 11.1.2 , the cell state is accessed when PA[6:3] = 0001 as described0001in as = , the cell state is accessed when PA[6:3] Figure 11.5 Figure withinbitsisindicated 15:7 the BLK_ by the state of lowerwords. 32-bit Reserved ITU Base Ad Reserved Address Base ITU Index Cell View Cell Offset Figure 11.7 •as described in of the cell views one This field encodes View: Cell • fieldIn this wide. Each cell is 64-bits 1:0 bits ar Offset: The address is dividedaddress is The into the followingfields: • ITU Base Address:This field is taken from bi • Cell Index: The size of the cell index varies depending on the number of cells. The position of the cell index The CellThe Index is derivedinformation from storedin th are divided the 32 cells this example In as described in in as described location depends exact The cal address. CELL_NUM fieldprogra valueThe is 32 (0x20). as shownin would be organized invalidcells.address The physical PA[63:0] As shown in Cell Index would reside at PA[11:7]. In this case, bits 15In thiscase, bits Index wouldCellreside at PA[11:7]. In 63 48 47 16 15 12 11 7 6 3 2 1 0 63 48 47 16 15 7 6 3 2 1 0 11.2.5.1 Example 1: 32 Cells with Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 11.2.5 Cell Indexing Examples 206 MIPS64® I6400 Multiprocessing Syst

d contains a valued contains of a . no invalidno physicalcells. The Cell View Offset Table 11.7 FO cells. The value configured in the value configured in The FO cells. are required to these 32 cells, the access VA Index

Cells

Tag

Structure

ample the BLK_GRAINample the fiel Cells ITC

.

State 1

Element Cell Index Cell

‐ 14 are don’t care. Bits 8:7 (VA Index) to map can be used (VA care. Bits 8:7 14 are don’t FIFO its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its our. In this example there are there this example In our. Cell dex Shift and No Invalid and Cells Shift dex Multi_Element Single Care Don’t Don’t Figure 11.8 Example mmedintoBLK_GRAIN the field is two. to the left 2 bits. Since 5 address bits Bit Index Shift and No Invalid Cells Bit Index Shift FI 16 and cells 16 single-element into 0x1F 0x00 0x0F 0x10

= = = =

, the cell index starts at PA[7]. In this ex In this PA[7]. at index starts , the cell PA[11:7] PA[11:7] PA[11:7] PA[11:7] Figure 11.8Figure Shift Index Cells with 2-bit 1: 32 Example Figure 11.7 32 Cells with No In , the cell state is accessed when PA[6:3] = 0001 as described0001in as = , the cell state is accessed when PA[6:3] Figure 11.5 Figure Reserved ITU Base Address Figure 11.9 63 48 47 16 15 14 13 9 8 7 6 3 2 1 0 In this example the 32 cells are divided the 32 cells this example In CELL_NUM fieldprogra valueThe is 32 (0x20). As shown in 0x2, causing the cell index to be shifted to cell index the 0x2, causing In Cell Index would reside at PA[13:9]. In this case, bits 15: case, In this Index wouldCellreside at PA[13:9]. multiplethe ITUto virtual addresses, up to f case in this would beas shown organized in address PA[63:0] 11.2.5.2 Example 2: 32 Cells with 2- Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 207

. d contains a valued contains of a tothe map ITU to multiple ss bits are required to access to access bits are required ss Table 11.7 nvalid cells. The physical address em Programmer’s Guide, Revision 1.00 cells and 10 FIFO cells. The value pro- The FIFO cells. 10 and cells

Cells

Tag Structure

ample the BLK_GRAINample the fiel Cells ITC

is case, bits 10:7 can be used be can case, bits 10:7 is State

Element 2: ‐

FIFO its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its Cell memory space. Since 5 addre value programmed into the BLK_GRAIN field is four. programmedvalue intoBLK_GRAINthe four. field is . Cells with 4-bit Index Shift Multi_Element ere are 20 valid cells and 12 i and 12 ere are 20 valid cells Single Example sed when PA[6:3] = 0001 as described in described 0001 as = sed when PA[6:3] Figure 11.10 4-Bit Index Shift and Invalid Cells 4-Bit Index Shift 0x1F 0x00 0x0F 0x10

at are divided into 10 single-element 10 single-element into at are divided = = = =

, the cell index starts at PA[7]. In this ex In this PA[7]. at index starts , the cell PA[13:9] PA[13:9] PA[13:9] PA[13:9] Figure 11.10 Example 1: 20 Figure 11.9Figure Cells No Invalid and Shift Index 2-bit with 32 Cells Figure 11.5 Figure Reserved ITUBase Address Cell Index Index VA CellView Offset 63 48 47 16 15 11 10 7 6 3 2 1 0 grammedthe into CELL_NUM field20 (0x14). is The As shown in 0x4, causing the cell indexleftthe to be shifted to 4 bits. Even thoughrequired,onlyof the number memoryare cells 20 locations allocatedof 2, whichpowerbeon must a this to access still required are bits that 5 address means In this example there are 20 cells th 20 cells are there this example In these 20 cells, the Cell Index would reside at PA[15:11]. In th would reside at PA[15:11]. Index the Cell these 20 cells, virtual address, in thisto case upthis 16. In example th PA[63:0] would beas organized shown in PA[63:0] In the diagram below, the cell state is acces diagram below, the In 11.2.5.3 Example 2: 20 Cells with Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 208 MIPS64® I6400 Multiprocessing Syst

tes). Any violation of structions are non-specula- are structions tiated. Refer to the section thetiated. Refer to ) located at CP0 (Reg 9, Sel 6). ARI register per VP. This means that means This VP. per ARI register ss (i.e. 4 bytes or 8 by or 8 4 bytes (i.e. ss behavior is undefined.

ddress (PA) matches that in the in matches that (PA) ddress Special Address Access Register (SAAR) Register Access Address Special Cells

ecuted independent of the core's CP0 core's ecuted independent of the ted on the address space of the ITU. The address ITU. The address of the space address ted on the ligned loads/storessupported).ITU are notthe to Tag Structure

e translated address. e SAAR0_ITU address match overwrites and e SAAR0_ITU address Cells

Cells ITC

the number of VP's instan State

Element ns. Write requests for store in for store requests ns. Write 3: ‐

FIFO its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its Cell Invalid Multi_Element Single Example associated with th with associated other views, SC pass/fail other views, structions are not suppor are not structions ess error exception (i.e. una error exception ess Special Address Access Register Index (SAARI Special Address sed using two CP0 registers; the the CP0 registers; sed using two 0x1F 0x14 one SAAR register per core, and one SA and core, per one SAAR register ys uncached. When the physical a ys uncached. 0x13 0x00 0x09 0x0A

======

for more information. the ITU module is via a 64-bit wide data path. 64-bit wide module is via a ITU the field, then Th the access is uncached. to the ITU must be aligned to the of acce size ructions referencing the ITU space are ex ITU the ructions referencing viewis used, the SCinstruction dependingsuccess or failure indicates thewhether on ITU PA[15:12] PA[15:12] PA[15:12] PA[15:12] PA[15:12] PA[15:12] Figure 11.11Figure Cells Invalid and Shift Index 4-bit with Cells 20 E/F try ITU Software Interface ITU of the load/store instruction store succeeds or fails due to the Full state. In the Full state. due to fails or succeeds store Cache Coherency Attributes (CCA) that may be in and SYNCI PREF, GINV*, The CACHE, the address alignment can cause an addr alignment the address Conditional (SC) inst Store LLAddr.LLbit. If If LLAddr.LLbit. tive. The read/write access to tive. The read/write access SAAR0_ITU.ADDR[47:16] From the core, ITU accesses are alwa accesses the core, ITU From located at CP0 (Reg 9, Sel 7), and the CP0 (Reg 9, located at As mentioned above, the ITU is acces As mentioned From a software perspective, there is perspective, a software From there can be multiple SAARI registers per core depending on The ITU is accessed by Load (LD) and Store (ST) instructio accessed by Load (LD) and Store The ITU is entitled Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 209 11.4 Accessing the ITU Module 11.3 Interface ITU Software

register that is dedicated register that value, such as 0xA0000, ster. Therefore, if one VP one if Therefore, ster. to the following constraints: to the ster with and size ster the base address her VP at any time. For example, in ze of 128K,the ITU resides soze of from em Programmer’s Guide, Revision 1.00 fieldsSAAR, the programming of the lue of 0x00 to indicate an ITU access. lue se address at a different se address at a different 47:16] field of the CP0 SAAR register. This the sets 47:16]the fieldSAAR CP0register. of can write to the SAAR regi can write to the overwritten by anot overwritten by operating system software if Virtualization is not imple- operating system software if Virtualization a result, the SIZE field always contains a value of 0x11 alwaysfieldvalue containsa result, of 0x11 SIZE a the bit is not set, the ITU is not present in the system. the in present not is ITU the set, not is bit sses sses to the ITU must adhere programthe internal SAAR0_ITU its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its field resides in bits 43:12 of the register. e ITU at a base of 0x40000si at with a ITU e register to enable ITU accesses. ITU accesses. enable register to 0x00, softwarethe programs SAAR regi \\ from the base address of 40000 base address the \\ from following assembly language code, along with an example transfer of data: and set the address, size, and enable and size, and set the address, address and the size of the ITU block. of the ITU and the size address of the CP0 SAARI register with a va with register CP0 SAARI of the \\is stored to ITU address 42000 to ITU address stored \\is mented or by hypervisor if Virtualization is implemented). bymented hypervisor or if Virtualization base address for the ITU module.base that Note this sd t0, 0(t1) \\Store the data in t0 to the address in t1; address offset = 0; Data = 0; offset t1; address in to the address in t0 the data \\Store t0, 0(t1) sd #define c0_SAARI $9,6 c0_SAARI #define $9,7 c0_SAAR #define t0, 000000000 li t0, c0_SAARI mtc0 ITU enable to 128K; set size 40000; to address base \\Set t1, 0000000040000031 dli t1, c0_SAAR dmtc0 the target as ITU 0 to select bit \\Clear ITU 0 to select a value of with register SAARI \\Write t0, 5555AAAA5555AAAA dli ITU to be transferred value to data \\load t1, 0000000000042000 dli 2000 is offset t1; this of 42000 into address \\load t1 register of with contents register SAAR \\Write and need notand need be programmed. Note thatNote the ITUmodule a fixed As size of 128 is KB. sets the base address for the ITU, that information can be the code example above, VP0 places th the ba if VP1 sets However, 0x40000 - 0x5FFFF in memory. moved. be the ITU will the location of then These steps can be represented by the by can be represented steps These 2.addressthe Program location base the of the ITU in ADDR[ sequence is as follows: 1. field Program the TARGET 5. Programming is done by theprivileged software (i.e. by 4.If the EN register. the SAAR of contents Read back the The ITU is shared across all VP's in the core. As such, acce As in the core. all VP's across shared The ITU is of the ITU block.of then uses thisinformation Hardware to 3.CP0 SAAR the bit in the ENABLE Set to the ITU, thereby settingthe base the ITU block For example, to select 1. The SAARI register must be programmed before the SAAR register. 2.VP per core, each only one SAAR register is Since there When the SAARI.TARGET field is set to field is set When the SAARI.TARGET It isIt incumbent upon softwareensure to that these conditions do not occur. Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 11.4.2 Programming Constraints 11.4.1 Sequence Programming Register 210 MIPS64® I6400 Multiprocessing Syst

d cell address d cell address This range is only is This range waysa power2 regard- of to an invali ccurs during an access to the an during ccurs the AXI bus. The ITU only ree error bits that are used by that are used bits ree error bothor set both cleared. ing invalidoccurs: accesses lid cell address range. cell address lid transactions. As such, the SIZE field of the transaction The cell address range is al ware when a parity error o a parity ware when e value of this field is any other value, hardware sets the If a normal request is made If a normal re when an error occurs on an error re when its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its en the E and F flags either the E and F en when either of the follow the when either of the ICR0.ERROR_EXECbit. For moreto information, refer , the ICR0 register contains th ICR0 register , the ess variesICR0.BLK_GRAINess based on theandfield the num- when cell does not have both the E and F cleared. not have both the E does cell when ity checked whenity checked is received. their data the cell address range is part of the inva part is the cell address range a multi-element FIFO cell a multi-element FIFO cell ITU ControlRegister ITU module. Write transactions are even-par ITU module. Write range, an error response is returnedrange,hardwareand an error sets Examplewith 2: 20 CellsShift and InvalidIndex 4-Bit Cells Register Access Invalid ICR0 The ERROR_EXECthe set underis bitThe following any of conditions: • cell, wh to a single-element Store a E/F Load or On The ERROR_PARITY bit in the ICR0 register is set by hard bit in the ICR0 is set register ERROR_PARITY The The ERROR_AXI bit in the ICR0 register is set by hardwa • cell a single-element to or Store Load a P/V On As shown in the section entitled section the in As shown •a multi-element toflags both set.E and F FIFO cell whenits or cellLoad Store has E/F On • to Store P/V Load or any On addition,In the ERROR_EXEChardwareset by bit is supports 32- 64-bit and transactions. It does not support burst must be equal to either 3'b010 [64bit]. [32bit] If th or 3'b011 ICR0.ERROR_AXI bit. • address Invalid •ICR0 Invalid register access Invalid Address decode the cell addr used to bits of address range The ber of cells configured based on the ICR0.CELL_NUMof cells configured based on the field. ber less of how many cells are configured. many cells of how less space within The remaining address accessed when the total number of cells is not a power of 2. not a power of cells is number when the total accessed hardware to report the followingtohardware error types. • AXI bus error • error Parity • Execution error Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 11.5.2 Parity Error 11.5.3 Execution Error 11.5.1 AXI Bus Error MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 211 11.5 Reporting ITU Error

= 0xF) to a cell = 0xF) to execution error, causing causing error, execution request (PA[6:3] request (PA[6:3] em Programmer’s Guide, Revision 1.00 l 0) will generate an an generate will 0) l such, any ICRsuch, any view its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its gister (any cell except cel cell except gister (any accessed only in cell 0. As only accessed hardware to set the ICR0.ERROR_EXEC bit. Note that software will see this condition as a Bus Error. address which corresponding has no ICR re In the ITU, the ICR0 register can be the ICR0 register In the ITU, Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 212 MIPS64® I6400 Multiprocessing Syst

ltiple threads in C) for all four threads. Simultaneous Multithread- Simultaneous s are used to store the dou- s are used to store slated to a Guest Physical orms instruction translation, address the larger Variable TLB (VTLB) and Variable the larger ter (PC)thread. for that ter pair This of eading that executes mu that executes eading is performed entirely in and hardware does the VP. Conversely, if there are four VPs, if Conversely, the VP. In addition, In instructionsdifferent from ructions in a round-robin manner. ructions ina round-robin manner. Each VP contains a complete system state (Gen- state system complete Each VP contains a shared Instruction Cache (I shared Instruction exception model). In addition, each thread has its In addition, each thread has exception model). resolvesand all datamanages resource conflicts and pre- ssue machine, allowing up to two allowing up to ssue machine, threads to in a execute . The I6400 MPS implements ical address (RPA). The The ITLB ical address (RPA). ly intended to provide an overview of multithreading and its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its n Virtual Address (IVA) is tran is (IVA) Address n Virtual can be fetched, decoded,graduated executed, issued, and in ies depends on the number of VPs implemented. threads. This ITLBs are backed up by are ITLBs threads. This tches instructions from a ion TLB (ITLB) structure. The ITLB structure. perf ion TLB (ITLB) thread) in a cycle, using a program coun a cycle, using in thread) to the ExecutionUnit (EXU), whichisresponsible decoding, issuing, for exe- B mappings, interrupt and B mappings, interrupt tion The IFU fetches(EXU). Unit inst e shared between all of the VPs. between all e shared em (MPS) incorporates hardware em (MPS) incorporates multithr multiple threads in parallel every cycle. ear to be run in parallel. This functionality , the instructions are issued and graduated in order. and issued are , the instructions one VP, all entries of the ITLB are used by one VP, read is referred to as a Virtual Processor (VP). Processor a Virtual read is referred to as Fixed TLB (FTLB). The number of shared ITLB entr ITLB shared of number The (FTLB). TLB Fixed •= 6 entries VP 1 • VPs = 12 entries 2 • VPs = 18 entries 4 only is For example, if there there are 18 ITLB that entries ar Instructio toorderIn support virtualization, the thread’s instructions are sent to the Execu the instructions are sent to a The IFU also shared manages Instruct is translated to Root Phys and then the GPA Address (GPA) ble translation to minimize the number of entries and more importantly improve performance by doing the double cycle. translation in a single instructions are passed translated The cutinggraduating and the instructions. In addition, the EXU cise exceptions. In the I6400 not requirenot any software control. Hencethis chapteron is implementedhowit is I6400 in the MPS. In the I6400, each th allowing complete independence amongst parallel. such a way that the threads app TL MSA registers, and eral, CP0, FP, own system debug, reset and various boot and exception vectors, and memory coherency. There are multipleof types multithreading implementations can execute ing, where the core executethreads can at the same time pipelinesame in the stage.maximum Thisallows for throughput and minimiza- tion of idle hardware during execution. The I6400 is a dual-i pipelinesingleIn the I6400, stage. threads all (up to 4) It fetches two instructions (for a single The I6400 Instruction Fetch Unit (IFU) fe The I6400 Instruction Fetch Unit (IFU) The I6400 Multiprocessing Syst Multiprocessing The I6400 Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 213 12.1 Instruction Flow Multithreading Chapter 12 Chapter

TLB (VTLB)and execution, each instruc- multiple loads and stores r instruction from the same the from r instruction to a Guest Physical Address Address Physical to a Guest itvisible becomesother to data address translation, allowing In other words, data stored by one ta cache to perform loads ta cache to perform and stores lability and data dependencies. It is are available to issue, to issue, are available the EXU uses a em Programmer’s Guide, Revision 1.00 the larger Variable the larger units. units. During its ed by the VP. Conversely, if there are four if there are Conversely, the VP. ed by (up to) four threads and determines which e thread in a cycle, but a cycle, e thread in ss (RPA). The DTLBs operate much like the ITLBs to the operate much like The DTLBs (RPA). ss d in a very controlled manner. The MIPS R6 Instruction Instruction The MIPS R6 very controlled manner. d in a data buffers) between threads to prevent starvation and between threads to prevent buffers) data s in the graduations queuetomaintain in-ordercompletion. le instructions (>2) le instructions antiated on a per-VP basis. The 512 basis. The antiated dual-entry on a per-VP Fixed TLB exactly the same point that to maintain cache coherency between threads. The data maintain cache to l Address (DVA) is translated is translated (DVA) l Address e some common resources. However, there are times when common resources. However, e some hence the term Simultaneousterm hence the Multithreading (SMT). Note ruction completes, but an earlie an ruction completes, but its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its ies depends on the number of VPs implemented. which most eliminates conflicts. of the cache (DTLB) structure. The DTLB The DTLB performs structure. (DTLB) The shared DTLB is backed up by is backed up DTLB The shared load or a store for a singl store for a or a load -Store Unit (LSU) manages Unit (LSU) manages a shared da -Store entification and instruction order. This allows the properentification instruction and instruction order to order. ciated cache line (for cacheable accesses). are avai based on resource ready to issue e shared between all of the four VPs. between all of e shared to a Root Physical addre to a , they are executed in one of the functional one of the in executed , they are the top two instructions from each of the each of two instructions from the top r threads in the same core at the same in threads r one VP, all eight entries of the DTLB are us one VP, tion is appropriately tagged for thread id be maintained at graduation (completion)If an inst time. thread has not graduated,thread has completed the instructionremain thread becomes visible to othe the processor needs to make sure the system is being accesse is being to make sure the system the processor needs Each of the threads operate independently, except to shar except of the threads operate independently, Each that the I6400 always issues instructions in order. If multip If thethat I6400 always issues instructions in order. capable of issuing instructionsof issuingcapablefrom any of the four VPs, policygetallfair issue threads equal representation. to makesure Once the instructions are issued Like the IFU mentioned above, the Load above, the Like the IFU mentioned Every cycle, the EXU decodes Every cycle, EXU decodes the perform a double translation in a single cycle. threadData stored by one does not becomevisiblethreadsotherto until the store instructionhas graduated and the has obtained ownership of the asso core cores in the system. cores (such as resources shared allocation of manages The I6400 that all threads can make forward progress. ensure two (of the possible eight) instructions for all The threads. a performs LSU also the data cache. to access queued up be threads can differing from received in the order stores and loads LSU processes The as 4-way set cache, associative cache is organized The LSU also manages a shared Data TLB complete independence amongst threads. independence complete entr ITLB shared of number The (FTLB). TLB Fixed •= 8 entries VP 1 • VPs = 14 entries 2 • VPs = 20 entries 4 only is For example, if there is translated then and the GPA (GPA) (FTLB) is shared between all VPs. shared between (FTLB) is Data Virtua toorderIn support virtualization, the thread’s VPs, there are 20 DTLB entries that ar entries 20 DTLB are VPs, there TLB (VTLB) is inst In addition, 16the dual-entry Variable Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 214 MIPS64® I6400 Multiprocessing Syst 12.3 Thread Management 12.2 Flow Data

formationlost is be re-enabled via the EVP EVP the via be re-enabled reads, no state in e instructions allow a thread, operat- allow a thread, e instructions read is suspended, hence the system hence the read is suspended, her threads to achieve that goal. to achieve her threads There is no loss of informationlossno There of for the resumed is thread. (Overflow, TLB Miss, etc.), Asynchronous Interrupts (Overflow, e, the suspended threads can e, the read can be reset to read can be reset reboot, while the other are threads f. DVPprivilegedf. is a instruction and is only available to its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its all other threads. For thesuspended th instructions to manage threads. Thes threads. manage to instructions stem timer continues to count while a th end or resume the execution theresume end or of ot erating mode. When appropriat erating instruction. ing in privileged state, to susp privileged state, in ing Sincethread eachhas a completely independentexception model,block one thread cannotthread.another This inde- pendent exception model includes: Synchronous Exceptions completely unaffected. The EVP instruction re-enables execution in all other threads. execution in all other re-enables instruction EVP The The DVP instruction suspends execution in suspends instruction DVP The be the might sy The only visible effect timing could changed.be This is a privileged instruction and is only available to Root (Hypervisor) Kernel operating mode. Debug(Int, NMI, etc.), Exceptions (DIint),Reset. A th and Set Architecture (ISA) includes specialized (ISA) includes Architecture Set Root (Hypervisor) Kernel op Kernel (Hypervisor) Root and the thread can be restarted exactly where they left of they where thread can be restarted exactly and the Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 12.3.2 Processor (EVP) Instruction Enable Virtual 12.3.1 Instruction (DVP) Processor Disable Virtual MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 215 12.4 Model Independent Exception em Programmer’s Guide, Revision 1.00 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 216 MIPS64® I6400 Multiprocessing Syst

ssors (VP)per core or mul- ese are connected by a Regis- single-cluster core. single-cluster two or four Virtual Proce twoVirtual or four h debug data is gathered. Th n on IMG Codescapeprobes, debug tools, SDKs and doc- rs (VPs) in a system.inrs (VPs)several functions The DBU provides implemented in a typical ide comprehensive debugging and performance-monitoring its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its of the interface and external debuggingenvironmentdebug torequiredMIPS com/developers/mips/tools/ On-Chip Instrumentation(OCI) debug system for multi-coredesigns. Please shows the OCI system as OCI system the shows cessor designs where there can be can where there designs cessor Figure 13.1 https://community.imgtec. tiple cores per cluster. There is one DBU per cluster of cores or Virtual Processo Virtual of cores or per cluster There is one DBU capabilitiesfor multi-core pro The MIPS OCI Debug System comprises a dedicated on-chip module called the Debug Unit and various on-chip components that have dedicateddebug whic resources from to assist debugging. This chapter provides a This chapter provides brief overview the MIPS that incorporate processors followingrefer to the communitypageslink for informatio umentation: The MIPS OCI debug system has been developed to prov ter Bus (RRB). Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 13.1.1 Debug (DBU) Unit MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 217 13.1 System Overview OCI Debug MIPS On-Chip Instrumentation On-Chip MIPS Chapter 13 Chapter

em Programmer’s Guide, Revision 1.00 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its Figure 13.1 OCI System Block Diagram Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 218 MIPS64® I6400 Multiprocessing Syst

iguration registers ers reside in the DBU and in ers reside by the VP when running in via the coherence manager g a packet-based protocol. a packet-based g s for Fast Debug Channels. ch as breakpoint conf abled on-chip debug controller or emulator transac- . The JTAG TAP data regist data TAP JTAG The . ite these registers indirectlyite theseregisters e processor in debug mode only. It contains the com- debugin modeprocessor only. e monitor code and contains the memory mapped area, contains the memory mapped monitor code and gment, dmseg, and is accessed and is gment, dmseg, the Register Bus (RRB) usin the Register its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its RAM also contains the FIFO contains RAM also e DBU during normal and debug mode execution. debug normal and e DBU during control logic and configuration registers. connection to an APB en connection s that are mapped to debug resources su resources to debug that are mapped s eakpoints eakpoints are shared between all VPs. ed by dmxseg in DBU RAM. in DBU dmxseg ed by The I6400 MPS implementsThe I6400 of4 8 instruction whichlower the triggers,triggering have range have4upper and the equality/mask, and 4 data triggers.Br The DBU connects to VPs and other coherent devices on devices to VPs and other coherent DBU connects The debug modeandprobeThis when a debug is attached. The debug memory segment of dseg that is accessed by the core when running in debug mode when a debug probe is area is mirror This attached. bined dmseg and drseg areas. A memory mapped area of main memory, accessible from th accessible A memory mapped area of main memory, Each VPindependent has its ownEach breakpoint A dedicated blockDBUthe of RAM inthehosts that debug is a dmxseg. Dmxseg mapped to the VPs memory debug se A serial JTAG TAP allows connection to a JTAG debug probe connection allows to a JTAG TAP JTAG A serial Debug monitor code is loadedintoRAMin schedules the DBUand commands debug VPs to the via the Register Bus. tor interface. allow read/write requeststhe to being VPs debuggeddebug via monitorcode. An APB Slave Port in the DBU provides Port in the DBU An APB Slave A region of dseg that includesdseg thatregister A region of and sampling registers.The VP and the DBU read and wr can th from accessed Drseg registers can be (CM). 13.1.4.3 Dmseg 13.1.4.2 Dseg 13.1.4.1 Breakpoint Controller 13.1.1.4 RAM 13.1.1.2 TAP JTAG 13.1.1.3 Debug Monitor 13.1.1.1 APB Slave Port 13.1.4.4 Drseg Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 13.1.3 Number of Breakpoints 13.1.4 Per Core/VP Resources 13.1.2 Register Bus MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 219

the debug process. For the debug process. es signals to put VPs into to put signals es em Programmer’s Guide, Revision 1.00 rious aspects of the CM, the coherence the of the CM, aspects rious can be used to aid in used to be can ous aspects ous aspects of a features. VP's debug a cluster or core and provid or core a cluster for the DBU, CM and VPs; registers for determiningthe its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its CMTrace functionality that CMTrace ains theains following devices. coherent and power up and clock gating of the CM.clock gating of the and power up and are used to configure and control va ion PDtrace Specification ers that facilitate and configure vari configure and that facilitate ers nce Manager) (v3) (v3) Manager) nce Power Controller) Power Interrupt Controller) more information, refer to the document entitled; MIPS On-Chip Instrumentat For more information on the MIPS OCI debug system, refer to the document entitled; Manual Reference Technical Debug Instrumentation; On-Chip MIPS and also contains PDTrace MPS The I6400 The I6400 Multiprocessing System cont System Multiprocessing The I6400 Provides stop/run signals for VPs; reset occurred signals state of each VPs power and clock rate; CP0 contains specific CP0 contains regist This handles the distributiontheThis handles of interrupts between the VPs in A set of memory mapped registers that of memory A set Debug Mode. The GIC also providesGIC also TheDebug Mode. debug mode status monitoring and controlsdebug team assignments for syn- chronous stop/go of multiple VPs. Connects coherent devices to the Coherence Manager. Coherence to the devices coherent Connects Controlsthe global orderingdevices. of requests and responses across core An optional block of custom registersbecontrolcan used to thatlevel system functions. scheme and CM performance counters. performance CM scheme and 13.1.4.5 CP0 Registers 13.1.5.1 GIC (Global 13.1.5.2 CPC (Cluster 13.1.5.3 Registers) Configuration (Global GCR 13.1.5.6 IOCU (I/O Coherence Unit) 13.1.5.5 CM - (Cohere 13.1.5.4 Registers) Configuration Global CGCR - (Custom Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 13.1.5 Devices Coherence 220 MIPS64® I6400 Multiprocessing Syst 13.2 More Information

its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 221 em Programmer’s Guide, Revision 1.00 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 222 MIPS64® I6400 Multiprocessing Syst

the document since its last document since its the Cluster Programmers Guide Cluster Programmers t indicate significant changes in t indicate significant s that are more than one revision old. itial version of I6400 Single- of I6400 itial version Copyright © 2017 MIPS Technologies Inc. All rights reserved. 01.002017 March 29, In Revision Date Description Change bars (vertical lines) in the margins of this documen of the margins lines) in bars (vertical Change release. Change bars are removed for are release. change Change bars MIPS64® I6400MIPS64® Multiprocessing System Programmer’s Guide, Revision 1.00 223 Revision History Revision Appendix A Appendix em Programmer’s Guide, Revision 1.00 its Affiliated Group Companies. All rights reserved. All rights Affiliated Group Companies. its Copyright © 2017 Imagination Technologies Ltd. and/or Ltd. CopyrightImagination © 2017 Technologies 224 MIPS64® I6400 Multiprocessing Syst