4/26/2015 Lab #6, Homework #4 Lab #6 is now posted on the course website Adding I/O support to access SRAM memory in your processor Can be completed in teams of 2 Due May 8th or later?? Lecture 35, 36, 37: Parallel Processing HW #4 will be posted by Wednesday Reading: Chapter 4, 6 April 24, 27, 29, 2015 Covers virtual memory, I/O, superscalar design Patterson & Hennessey textbook Prof. R. Iris Bahar Due May 11th ?? Class final was scheduled for May 15th (last day of finals!) Willing to extend deadlines, but prefer not give until the 15th © 2015 R.I. Bahar since grading takes time for both assignments Portions of these slides taken from Professors S. Reda 2 and D. Patterson Parallel Processor Design 1. SIMD architectures Single Instruction Multiple Data (SIMD) Single instruction acts on multiple pieces of data at once (aka vector instructions) Common application: scientific computing, graphics Intel Core i7 AMD A10 IBM Power 8 Requires vector register file and multiple execution units 1. SIMD (data-level parallelism) 2. Superscalar (instruction-level parallelism) 3. Multi-cores (thread-level parallelism) 3 4 1 4/26/2015 2. Superscalar architectures Superscalar vs. VLIW VLIW: Compiler groups instructions to be issued together into Very Large Instruction Words (VLIW) Packages them statically into “issue slots” Compiler detects and avoids hazards Superscalar: CPU examines instruction stream and chooses instructions to issue each cycle Compiler can help by reordering instructions CPU resolves hazards using advanced techniques at runtime Can be static in order or dynamic out-of-order [from Fisher et al.] 5 6 Instruction-Level Parallelism (ILP) Multiple Issue Pipelining: executing multiple instructions in parallel Static multiple issue To increase ILP Compiler groups instructions to be issued together Deeper pipeline Packages them into “issue slots” Less work per stage shorter clock cycle Compiler detects and avoids hazards Multiple issue Replicate pipeline stages multiple pipelines Dynamic multiple issue Start multiple instructions per clock cycle CPU examines instruction stream and chooses instructions to CPI < 1, so use Instructions Per Cycle (IPC) issue each cycle E.g., 4GHz 4-way multiple-issue Compiler can help by reordering instructions 16 BIPS, peak CPI = 0.25, peak IPC = 4 But dependencies reduce this in practice CPU resolves hazards using advanced techniques at runtime 7 8 2 4/26/2015 Scheduling Static Multiple Issue MIPS with Static Dual Issue Two-issue packets Compiler must remove some/all hazards One ALU/branch instruction Reorder instructions into issue packets One load/store instruction No dependencies within a packet 64-bit aligned Possibly some dependencies between packets ALU/branch, then load/store Varies between ISAs; compiler must know! Pad an unused instruction with nop Address Instruction type Pipeline Stages Pad with nop if necessary n ALU/branch IF ID EX MEM WB n + 4 Load/store IF ID EX MEM WB n + 8 ALU/branch IF ID EX MEM WB n + 12 Load/store IF ID EX MEM WB n + 16 ALU/branch IF ID EX MEM WB n + 20 Load/store IF ID EX MEM WB 9 10 Pipeline Design for Dual Issue Hazards in the Dual-Issue MIPS More instructions executing in parallel EX data hazard Forwarding avoided stalls with single-issue Now can’t use ALU result in load/store in same packet add $t0, $s0, $s1 load $s2, 0($t0) Split into two packets, effectively a stall Load-use hazard Still one cycle use latency, but now two instructions More aggressive scheduling required 11 12 3 4/26/2015 Scheduling Example Dynamic Multiple Issue Schedule this for dual-issue MIPS Loop: lw $t0, 0($s1) # $t0=array element “Superscalar” processors add $t0, $t0, $s2 # add scalar in $s2 sw $t0, 0($s1) # store result CPU decides whether to issue 0, 1, 2, … each cycle addi $s1, $s1,–4 # decrement pointer Avoiding structural and data hazards bne $s1, $zero, Loop # branch $s1!=0 Avoids the need for compiler scheduling ALU/branch Load/store cycle Though it may still help Loop: nop lw $t0, 0($s1) 1 Code semantics ensured by the CPU nop nop 2 add $t0, $t0, $s2 nop 3 addi $s1, $s1,–4 sw $t0, 0($s1) 4 bne $s1, $zero, Loop nop 5 IPC = 5/5 = 1 (peak dual issue IPC = 2 and single issue IPC=5/6 = .83) 14 13 Dynamic Pipeline Scheduling Name dependency: WAR lw $s0, 0($t0) … Allow the CPU to execute instructions out of order to avoid add $t0, $s1, $s2 stalls But commit result to registers in order add $s4, $s2, $s0 ….. Example sub $s2, $s1, $s3 lw $t0, 20($s2) addu $t1, $t0, $t2 lw $t2, 0($s2) sub $s4, $s4, $t3 …. slti $t5, $s4, 20 lw $s2, 4($t0) Just name dependency --- no values being transmitted Can start sub while addu is waiting for lw Dependency can be removed by renaming registers 15 (either by compiler or HW) 16 4 4/26/2015 Name dependency: WAW Register Renaming With only 32 architectural registers defined in MIPS ISA, lw $s0, 0($t0) compiler needs to reuse same registers repeatedly …. leads to potential WAR, WAW hazards if instructions are add $s0, $s1, $s2 reordered. HW Solution: Allocated extra physical registers for add $s2, $s1, $s0 additional temporary storage …. Reservation stations: all instructions are allocated space in the sub $s2, $t2, $t3 reservation station after they are decoded Only need to preserve RAW dependencies If operand(s) available (in RF or elsewhere in pipeline), copy to reservation station entry If operand(s) not immediately available, RS entry will be updated by Just name dependency --- no values being transmitted a functional unit Use in conjunction with a Reorder Buffer to write values back to Dependency can be removed by renaming registers architectural registers in order (thus avoiding WAW, WAR hazards) (either by compiler or HW) 17 18 Overall superscalar organization • Dynamic Out-of-order (OoO) execution Register Renaming • Multiple instructions fetched and decoded in parallel. Reservation stations and reorder buffer effectively provide • Decoder checks for dependencies and register renaming rename registers to avoid WAW and On instruction issue to reservation station WAR hazards. If operand is available in register file or reorder buffer • Instructions wait in a dispatch buffer until Copied to reservation station their operands are available (avoids No longer required in the register; can be overwritten RAW hazards). If operand is not yet available • When ready, instructions are dispatched It will be provided to the reservation station by a function unit to the execution units. Register update may not be required • Re-order buffer puts back instructions in program order for WB 19 20 5 4/26/2015 Dynamic Rescheduling Example Exposing ILP using Loop Unrolling Reschedule code for dual-issue MIPS Loop: lw $t0, 0($s1) # $t0=array element Replicate loop body to expose more parallelism add $t0, $t0, $s2 # add scalar in $s2 sw $t0, 0($s1) # store result Reduces loop-control overhead addi $s1, $s1,–4 # decrement pointer Use different registers per replication bne $s1, $zero, Loop # branch $s1!=0 register renaming taken care of with reservation station and ALU/branch Load/store cycle reorder buffer Loop: nop lw $t0, 0($s1) 1 Avoids loop-carried WAR, WAW dependencies addi $s1, $s1,–4 nop 2 add $t0, $t0, $s2 nop 3 bne $s1, $zero, Loop sw $t0, 4($s1) 4 IPC = 5/4 = 1.25 21 22 Loop unrolling example Speculation ALU/branch Load/store cycle “Guess” what to do with an instruction Loop: addi $s1, $s1,–16 lw $t0, 0($s1) 1 Start operation as soon as possible nop lw $t1, 12($s1) 2 add $t0, $t0, $s2 lw $t2, 8($s1) 3 Check whether guess was right add $t1, $t1, $s2 lw $t3, 4($s1) 4 If so, complete the operation add $t2, $t2, $s2 sw $t0, 16($s1) 5 If not, roll-back and do the right thing add $t3, $t3, $s2 sw $t1, 12($s1) 6 Common to static and dynamic multiple issue nop sw $t2, 8($s1) 7 bne $s1, $zero, Loop sw $t3, 4($s1) 8 Examples Speculate on branch outcome Roll back if path taken is different IPC = 14/8 = 1.75 Speculate on load Closer to 2, but a cost of registers and code size Roll back if location is updated (with store) or if data was not in cache 23 24 6 4/26/2015 Compiler/Hardware Speculation Cortex A8 and Intel i7 Processor ARM A8 Intel Core i7 920 Market Personal Mobile Device Server, cloud Compiler can reorder instructions Thermal design power 2 Watts 130 Watts e.g., move load before branch Clock rate 1 GHz 2.66 GHz Cores/Chip 1 4 Can include “fix-up” instructions to recover from incorrect Floating point? No Yes guess Multiple issue? Dynamic Dynamic Hardware can look ahead for instructions to execute Peak instructions/clock cycle 2 4 Buffer results until it determines they are actually needed Pipeline stages 14 14 Pipeline schedule Static in-order Dynamic out-of-order Reorder buffer takes care of this with speculation Flush buffers on incorrect speculation Branch prediction 2-level 2-level 1st level caches/core 32 KiB I, 32 KiB D 32 KiB I, 32 KiB D 2nd level caches/core 128-1024 KiB 256 KiB 3rd level caches (shared) - 2- 8 MB 25 26 ARM Cortex-A8 Pipeline Core i7 Pipeline 27 28 7 4/26/2015 Summary of superscalar architectures Pros: Improved single-thread throughput: hide memory latency; avoid or reduces stalls; and ability to fetch and execute multiple instructions per cycle. Cons: impacts silicon area impacts power consumption impacts design complexity SW compilation techniques enable more ILP from the same HW simplify HW (e.g., VLIW) at the expense of code portability Conclusion: great single-thread performance but at the expense of energy efficiency.