CS420/520 Computer Architecture I Instruction-Level Parallelism Dr. Xiaobo Zhou Department of Computer Science CS420/520 pipeline.1 UC. Colorado Springs Adapted from ©UCB97 & ©UCB03 Re: Pipeline Data Hazard Detection (Delay Load) ° Comparing the destination and sources of adjacent instructions CS420/520 pipeline.2 UC. Colorado Springs Adapted from ©UCB97 & ©UCB03 What is Instruction-level Parallelism (ILP) ° ILP: the potential overlap among instruction executions due to pipelining • The instructions can be executed and evaluated in parallel ° How to exploit ILP • Hardware stall • Software NOP • Hardware forwarding/bypassing ° Two MORE largely separable approaches to exploiting ILP • Static/compiler pipeline scheduling by the compiler tries to minimize stalls by separating dependent instructions so that they will not lead to hazards • Dynamic hardware scheduling tries to avoid stalls when dependences, which could generate hazards, are present. CS420/520 pipeline.3 UC. Colorado Springs Adapted from ©UCB97 & ©UCB03 Software (compiler) Static Scheduling / ILP ° Software scheduling: the goal is to exploit ILP by preserving program order only where it affects the outcome of the program Try producing fast code for a = b + c; d = e – f; assuming a, b, c, d, e, and f in memory. Slow code: Fast code: LW Rb,b LW Rb,b LW Rc,c LW Rc,c ADD Ra,Rb,Rc LW Re,e SW a,Ra ADD Ra, Rb, Rc LW Re,e LW Rf,f LW Rf,f SW a,Ra SUB Rd,Re,Rf SUB Rd,Re,Rf SW d,Rd SW d,Rd CS420/520 pipeline.4 UC. Colorado Springs Adapted from ©UCB97 & ©UCB03 Compiler Avoiding Load Stalls: scheduled unscheduled 54% gcc 31% 42% spice 14% 65% tex 25% 0% 20% 40% 60% 80% % loads stalling pipeline CS420/520 pipeline.5 UC. Colorado Springs Adapted from ©UCB97 & ©UCB03 Data Dependences ° Data dependence • Instruction i produces a result that may be used by instruction j • Instruction j is data dependent on instruction k, and instruction k is data dependent on instruction i ( a chain of dependences) loop: LD F0, 0(R1) DADD F4, F0, F2 SD F4, 0(R1) DAADI R1, R1, -8 BNE R1, R2, Loop CS420/520 pipeline.6 UC. Colorado Springs Adapted from ©UCB97 & ©UCB03 Name Dependences ° Name dependence (not-true-data-hazard) • Occurs when two instructions use the same register or memory location, called a name, but there is no flow of data between the instructions associating with that name • Remember: do not be restricted to the 5-staggppe pipeline! Anti-dependence (WAR) j writes a register or memory location that i reads: ADD $1, $2, $4 What if SUB does earlier than ADD? SUB $4, $5, $6 Is there a data flow? Outppput dependence ( WAW) i and j write the same register or memory location SUB $4, $2, $7 What if SUBI does earlier than SUB? SUBI $4, $5, 100 Is there a data flow? How many ways for data to flow between instructions? CS420/520 pipeline.7 UC. Colorado Springs Adapted from ©UCB97 & ©UCB03 Data Hazards - RAW ° Data hazards may be classified, depending on the order of read and write accesses in the instructions ° RAR (read after read) is not a hazard, nor a name dependence ° RAW (read after write): • j tries to read a source before i writes it, so j incorrectly gets the old value; most common type – true data hazards Example? CS420/520 pipeline.8 UC. Colorado Springs Adapted from ©UCB97 & ©UCB03 Data Hazards - WAW ° WAW (write after write): • Output dependence of name hazards: j tries to write an operand before it is written by i. Can you nominate an example? Short/long pipelines MULTF F4, F5, F6 LD F4, 0(F1) Is WAW possible in the MIPS classic five-stage integer pipelining? Why? CS420/520 pipeline.9 UC. Colorado Springs Adapted from ©UCB97 & ©UCB03 Data Hazards - WAR ° WAR (write after read): • Anti-dependence of name hazards: j tries to write a destination before it is read by i, so i incorrectly gets the new value Example? 1) Due to re-ordering DIV $0, $2, $4 (add is stalled waiting DIV, ADD $6, $0, $8 if SUB is done before ADD, SUB $8, $10, $14 anti-dependence violated). 2) One writes early in the pipeline and some others read a source late in the pipeline Is WAR possible in the MIPS classic five-stage integer pipelining? CS420/520 pipeline.10 UC. Colorado Springs Adapted from ©UCB97 & ©UCB03 ILP and Dynamic Scheduling ° Dynamic scheduling: the goal is to exploit ILP by preserving program order only where it affects the outcome of the program Out-of-order execution DDIV F0, F2, F4 DADD F10, F0, F8 DSUB F12, F8, F14 // DSUB not dependent on // anything in the pipeline // can its order be exchanged // with DADD? Out-of-order execution may introduce WAR and WAW hazards DDIV F0,F2,F4, F2, F4 DADD F6, F0, F8 // anti-dependence between DADD DSUB F8, F10, F14 // and DSUB; if out-of-order, WAR DMUL F6, F10, F8 // register renaming helps! CS420/520 pipeline.11 UC. Colorado Springs Adapted from ©UCB97 & ©UCB03 Dynamic Scheduling – Tomasulo’ Register Renaming Before: DDIV F0, F2, F4 //anti-dependence DSUB – F8, WAR DADD F6, F0, F8 //output dependence DMUL-F6, WAW SD F6, 0(R1) DSUB F8, F10, F14 // How manyyp true data dependences? DMUL F6, F10, F8 After: DDIV F0, F2, F4 DADD S, F0, F8 SD S, 0(R1) DSUB T, F10, F14 DMUL F6, F10, T What are dependencies there? What dependencies disappear? And what are still there? What to do with the subsequence use of F8? Finding the subsequent use of F8 requires compiler analysis or hardware support CS420/520 pipeline.12 UC. Colorado Springs Adapted from ©UCB97 & ©UCB03 Concurrency and Parallelism ° Concurrency in software is a way to manage the sharing of resources efficiently at the same time • When multiple software threads of execution are running concurrently, the execution of the threads is interleaved onto a single hardware resource - Why not schedule another thread while one thread is on a cache miss or even page fault? - Time-sliced multi-threading increases concurrency, overlapping CPU time with I/O time - Pseudo-parallelism ° True ppqparallelism requires multiple hardware resources • Multiple software threads are running simultaneously on different hardware resources/processing elements • One approach to addressing thread-level true parallelism is to increase the number of physical processors / execution elements in the computer CS420/520 pipeline.13 UC. Colorado Springs Adapted from ©UCB97 & ©UCB03 ILP and Super-Scalar Processors ° ILP: the potential overlap among instruction executions due to pipelining • To increase the number of instructions that are executed by the processor on a single clock cycle • A processor that is capable of executing multiple instructions in a single clock cycle is known as a super-scalar processor • http://en.wikipedia.org/wiki/Superscalar (a) A three-stage path (b) A superscalar CPU CS420/520 pipeline.14 UC. Colorado Springs Adapted from ©UCB97 & ©UCB03 Multi-Threading ° Process: for resource grouping and execution ° Thread: a finer-granularity entity for execution and parallelism • Lightweight processes, multithreading ° Multi-threadinggp: Operating gy system supp orts multip le threads of execution within a single process • At hardware level, a thread is an execution path that remains independent of other hardware thread execution paths User-level Threads Used by applications and handled by user-level runtime al flow Kernel-level Threads n Used and handled by OS kernel Hardware Threads Operatio Used by each processor CS420/520 pipeline.15 UC. Colorado Springs Adapted from ©UCB97 & ©UCB03 Simultaneous Multi-threading (Hyper-Threading) ° Simultaneous multi-threading (SMT) • Create multiple logical processors with a physical processor • One logical processor for a thread, which requires an architecture state consisting of the GPRs and Interrupt logic •Duppplicate multiple architecture states ( (),,CPU state), and, let other CPU resources, such as caches, buses, execution units, branch prediction logic shared among architecture states • Multi-threading at hardware level, instead of OS switching • Intel’s SMT implementation is called Hyper-Threading Technology (HT Technology) What is next? CS420/520 pipeline.16 UC. Colorado Springs Adapted from ©UCB97 & ©UCB03 Multi-Core CS420/520 pipeline.17 UC. Colorado Springs Adapted from ©UCB97 & ©UCB03 Limitations of Sing-core Technology CS420/520 pipeline.18 UC. Colorado Springs Adapted from ©UCB97 & ©UCB03 Multi-Processor and Multi-Core ° multi-core processors use chip multi-processing (CMP) • Cores are essentially two individual processors on a single die • May or may not share on-chip cache • True parallelism, instead of high concurrency CS420/520 pipeline.19 UC. Colorado Springs Adapted from ©UCB97 & ©UCB03 Stanford HYDRA Multi-core Structure CS420/520 pipeline.20 UC. Colorado Springs Adapted from ©UCB97 & ©UCB03 Multi-core Processors as a Solution a) Advantages and improvements • Performance increase by means of parallel execution • Power and Energy efficient cores (Dynamic power coordination) • Minimized wire lengths and interconnect latencies • Both ILP and thread level parallelism • Reduced design time and complexity b) Task management and parceling • Usually operating system distributes the application to the cores • Can be done according to resource requirements c) Multicore examples AMD Opteron Enterprise, Athlon64 Intel Pentium Extreme edition, Pentium D IBM Power 4, Power 5 and Sun Niagara CS420/520 pipeline.21 UC. Colorado Springs Adapted from ©UCB97 & ©UCB03 Multi-core vs Superscalar vs SMT Multicore VS Superscalar VS SMT(Hammond, 1997) CS420/520 pipeline.22 UC. Colorado Springs Adapted from ©UCB97 & ©UCB03 Multi-core vs Superscalar • Superscalar processors differ from multi-core processors in that the redundant functional units are not entire processors. • A single superscalar processor is composed of finer-grained functional units such as the ALU, integer shifter, etc. There may be multiple versions of each functional unit to enable execution of many instructions in parallel.