Lecture 2: Review of Instruction Sets, Pipelines, and Caches Prof. David A. Patterson Computer Science 252 Spring 1998 DAP Spr.‘98 ©UCB 1 Review, #1 • Designing to Last through Trends Capacity Speed Logic 2x in 3 years 2x in 3 years DRAM 4x in 3 years 2x in 10 years Disk 4x in 3 years 2x in 10 years Processor ( n.a.) 2x in 1.5 years • Time to run the task – Execution time, response time, latency • Tasks per day, hour, week, sec, ns, … – Throughput, bandwidth • “X is n times faster than Y” means ExTime(Y) Performance(X) --------- = -------------- ExTime(X) Performance(Y) DAP Spr.‘98 ©UCB 2 Review, #2 • Amdahl’s Law: 1 ExTimeold Speedupoverall = = (1 - Fractionenhanced) + Fractionenhanced ExTimenew • CPI Law: Speedupenhanced CPU time = Seconds = Instructions x Cycles x Seconds Program Program Instruction Cycle • Execution time is the REAL measure of computer performance! • Good products created when have: – Good benchmarks – Good ways to summarize performance • Die Cost goes roughly with die area4 DAP Spr.‘98 ©UCB 3 Review, #3: Price vs. Cost 100% 80% Average Discount 60% Gross Margin 40% Direct Costs 20% Component Costs 0% Mini W/S PC 5 4.7 3.8 4 3.5 Average Discount 3 2.5 Gross Margin 2 1.8 Direct Costs 1.5 1 Component Costs 0 Mini W/S PC DAP Spr.‘98 ©UCB 4 Computer Architecture Is … the attributes of a [computing] system as seen by the programmer, i.e., the conceptual structure and functional behavior, as distinct from the organization of the data flows and controls the logic design, and the physical implementation. Amdahl, Blaaw, and Brooks, 1964 SOFTWARE DAP Spr.‘98 ©UCB 5 Computer Architecture’s Changing Definition • 1950s to 1960s: Computer Architecture Course = Computer Arithmetic • 1970s to mid 1980s: Computer Architecture Course = Instruction Set Design, especially ISA appropriate for compilers • 1990s: Computer Architecture Course = Design of CPU, memory system, I/O system, Multiprocessors DAP Spr.‘98 ©UCB 6 Instruction Set Architecture (ISA) software instruction set hardware DAP Spr.‘98 ©UCB 7 Interface Design A good interface: • Lasts through many implementations (portability, compatability) • Is used in many differeny ways (generality) • Provides convenient functionality to higher levels • Permits an efficient implementation at lower levels use imp 1 time Interface use imp 2 use imp 3 DAP Spr.‘98 ©UCB 8 Evolution of Instruction Sets Single Accumulator (EDSAC 1950) Accumulator + Index Registers (Manchester Mark I, IBM 700 series 1953) Separation of Programming Model from Implementation High-level Language Based Concept of a Family (B5000 1963) (IBM 360 1964) General Purpose Register Machines Complex Instruction Sets Load/Store Architecture (Vax, Intel 432 1977-80) (CDC 6600, Cray 1 1963-76) RISC (Mips,Sparc,HP-PA,IBM RS6000,PowerPC . .1987) DAP Spr.‘98 ©UCB 9 LIW/”EPIC”? (IA-64. .1999) Evolution of Instruction Sets • Major advances in computer architecture are typically associated with landmark instruction set designs – Ex: Stack vs GPR (System 360) • Design decisions must take into account: – technology – machine organization – programming langauges – compiler technology – operating systems • And they in turn influence these DAP Spr.‘98 ©UCB 10 A "Typical" RISC • 32-bit fixed format instruction (3 formats) • 32 32-bit GPR (R0 contains zero, DP take pair) • 3-address, reg-reg arithmetic instruction • Single address mode for load/store: base + displacement – no indirection • Simple branch conditions • Delayed branch see: SPARC, MIPS, HP PA-Risc, DEC Alpha, IBM PowerPC, CDC 6600, CDC 7600, Cray-1, Cray-2, Cray-3 DAP Spr.‘98 ©UCB 11 Example: MIPS (≈ DLX) Register-Register 31 26 25 21 20 16 15 11 10 6 5 0 Op Rs1 Rs2 Rd Opx Register-Immediate 31 26 25 21 20 16 15 0 Op Rs1 Rd immediate Branch 31 26 25 21 20 16 15 0 Op Rs1 Rs2/Opx immediate Jump / Call 31 26 25 0 Op target DAP Spr.‘98 ©UCB 12 Pipelining: Its Natural! • Laundry Example • Ann, Brian, Cathy, Dave A B C D each have one load of clothes to wash, dry, and fold • Washer takes 30 minutes • Dryer takes 40 minutes • “Folder” takes 20 minutes DAP Spr.‘98 ©UCB 13 Sequential Laundry 6 PM 7 8 9 10 11 Midnight Time 30 40 20 30 40 20 30 40 20 30 40 20 T a A s k B O r d C e r D • Sequential laundry takes 6 hours for 4 loads • If they learned pipelining, how long would laundry take? DAP Spr.‘98 ©UCB 14 Pipelined Laundry Start work ASAP 6 PM 7 8 9 10 11 Midnight Time 30 40 40 40 40 20 T a A s k B O r d C e r D • Pipelined laundry takes 3.5 hours for 4 loadsDAP Spr.‘98 ©UCB 15 Pipelining Lessons 6 PM 7 8 9 • Pipelining doesn’t help latency of single task, it Time helps throughput of entire workload T a 30 40 40 40 40 20 • Pipeline rate limited by s slowest pipeline stage k A • Multiple tasks operating simultaneously O • Potential speedup = r B Number pipe stages d e • Unbalanced lengths of r C pipe stages reduces speedup D • Time to “fill” pipeline and time to “drain” it reduces speedup DAP Spr.‘98 ©UCB 16 Computer Pipelines • Execute billions of instructions, so throughout is what matters • DLX desirable features: all instructions same length, registers located in same place in instruction format, memory operands only in loads or stores DAP Spr.‘98 ©UCB 17 5 Steps of DLX Datapath Figure 3.1, Page 130 Instruction Instr. Decode Execute Memory Write Fetch Reg. Fetch Addr. Calc Access Back IR L M D DAP Spr.‘98 ©UCB 18 Pipelined DLX Datapath Figure 3.4, page 137 Instruction Fetch Instr. Decode Execute Reg. Fetch Addr. Calc. Write Back Memory Access • Data stationary control – local decode for each instruction phase / pipeline stage DAP Spr.‘98 ©UCB 19 Visualizing Pipelining Figure 3.3, Page 133 Time (clock cycles) I n s t r. O r d e r DAP Spr.‘98 ©UCB 20 Its Not That Easy for Computers • Limits to pipelining: Hazards prevent next instruction from executing during its designated clock cycle – Structural hazards: HW cannot support this combination of instructions (single person to fold and put clothes away) – Data hazards: Instruction depends on result of prior instruction still in the pipeline (missing sock) – Control hazards: Pipelining of branches & other instructions that change the PC – Common solution is to stall the pipeline until the hazard is resolved, inserting one or more “bubbles” in the pipeline DAP Spr.‘98 ©UCB 21 One Memory Port/Structural Hazards Figure 3.6, Page 142 Time (clock cycles) Load I n s Instr 1 t r. O Instr 2 r d Instr 3 e r Instr 4 DAP Spr.‘98 ©UCB 22 One Memory Port/Structural Hazards Figure 3.7, Page 143 Time (clock cycles) Load I n s t Instr 1 r. O Instr 2 r d e stall r Instr 3 DAP Spr.‘98 ©UCB 23 Speed Up Equation for Pipelining CPIpipelined = Ideal CPI + Pipeline stall clock cycles per instr Speedup = Ideal CPI x Pipeline depth Clock Cycle x unpipelined Ideal CPI + Pipeline stall CPI Clock Cyclepipelined Speedup = Pipeline depth Clock Cycle x unpipelined 1 + Pipeline stall CPI Clock Cyclepipelined DAP Spr.‘98 ©UCB 24 Example: Dual-port vs. Single-port • Machine A: Dual ported memory • Machine B: Single ported memory, but its pipelined implementation has a 1.05 times faster clock rate • Ideal CPI = 1 for both • Loads are 40% of instructions executed SpeedUpA = Pipeline Depth/(1 + 0) x (clockunpipe/clockpipe) = Pipeline Depth SpeedUpB = Pipeline Depth/(1 + 0.4 x 1) x (clockunpipe/(clockunpipe / 1.05) = (Pipeline Depth/1.4) x 1.05 = 0.75 x Pipeline Depth SpeedUpA / SpeedUpB = Pipeline Depth/(0.75 x Pipeline Depth) = 1.33 • Machine A is 1.33 times faster DAP Spr.‘98 ©UCB 25 Data Hazard on R1 Figure 3.9, page 147 Time (clock cycles) IF ID/RF EX MEM WB I add r1,r2,r3 n s t sub r4,r1,r3 r. O and r6,r1,r7 r d e or r8,r1,r9 r xor r10,r1,r11 DAP Spr.‘98 ©UCB 26 Three Generic Data Hazards InstrI followed by InstrJ • Read After Write (RAW) InstrJ tries to read operand before InstrI writes it DAP Spr.‘98 ©UCB 27 Three Generic Data Hazards InstrI followed by InstrJ • Write After Read (WAR) InstrJ tries to write operand before InstrI reads i – Gets wrong operand • Can’t happen in DLX 5 stage pipeline because: – All instructions take 5 stages, and – Reads are always in stage 2, and – Writes are always in stage 5 DAP Spr.‘98 ©UCB 28 Three Generic Data Hazards InstrI followed by InstrJ • Write After Write (WAW) InstrJ tries to write operand before InstrI writes it – Leaves wrong result ( InstrI not InstrJ ) • Can’t happen in DLX 5 stage pipeline because: – All instructions take 5 stages, and – Writes are always in stage 5 • Will see WAR and WAW in later more complicated pipes DAP Spr.‘98 ©UCB 29 CS 252 Administrivia • Too many students with too varied background? – In past, CS grad students took written prelim exams on undergraduate material in hardware, software, and theory – Prelims were dropped => some unprepared for CS 252? • In class exam on Wednesday January 28 – Improve 252 experience if recapture common background – Bring 1 sheet of paper with notes on both sides – Doesn’t affect grade, only admission into class – 2 grades: Admitted or audit/take CS 152 1st (before class Friday) • Review: Chapters 1- 3, CS 152 home page, maybe “Computer Organization and Design (COD)2/e” – If did take a class, be sure COD Chapters 2, 6, 7 are familiar – Copies in Bechtel Library on 2-hour reserve DAP Spr.‘98 ©UCB 30 CS 252 Administrivia • Too many students? • 61 students at 1st lecture – To give proper attention to projects (as well as homeworks and quizes), I can handle up to 36 students • Limiting Number of Students – First priority is first year CS/ EECS grad students (32) – Second priority is N-th year CS/ EECS grad students (21) – Third priority is College of Engineering grad students (1) – Fourth priority is CS/EECS undegraduate seniors (7) (Note: 1 graduate course unit = 2 undergraduate course units) – All other categories • If not this semester, 252 is offered regularily (Fall) DAP Spr.‘98 ©UCB 31 Forwarding to Avoid Data Hazard Figure 3.10, Page 149 Time (clock cycles) I n s add r1,r2,r3 t r.