Computer Architecture Spring 2016

Lecture 11: Dynamic Scheduling: Scoreboarding Shuai Wang

Department of Computer Science and Technology Nanjing University [Slides adapted from CS252, UC Berkeley and CS 246, Harvard University] Dynamic Scheduling • Dynamic scheduling: rely on hardware to rearrange inst. execution to reduce the stalls while maintaining exception behavior and data flow

• Advantages of dynamic scheduling: – Handling cases when dependences are unknown at compile time, e.g., involving memory reference – Simplifies compiler – Allow code that compiled for one pipeline to run efficiently on a different pipeline – Hardware speculation, a technique with significant performance advantages, that builds on dynamic scheduling Dynamic Scheduling: The Idea • Key idea: allow instructions behind stall to proceed DIVD F0 ,F2,F4 ADDD F10, F0 ,F8 SUBD F12,F8,F14

• Enables out-of-order execution and allows out-of-order completion - But • Dynamic scheduling introduces the possibility of WAW and WAR hazards • Dynamic scheduling processor may generate imprecise exceptions Precise vs. Imprecise Exceptions • Precise exception: if the pipeline can be stopped so that the inst. before the faulting inst. are completed and those after it can be restarted from scratch.

• Imprecise exception: if the processor state when an exception is raised does not look like exactly as if the inst. were executed sequentially in strict program order – Inst. later in program order than the faulting inst. have completed – Inst. earlier in program order than the faulting inst. have not yet completed Dynamic Scheduling with a Scoreboard • Scoreboard : allow inst. to execute out-of-order when there are sufficient resources and no data dependences

• Named after CDC6600 scoreboard

• The scoreboard takes full responsibility for instruction issue and execution, including all hazard detection Supercomputer (CDC6600) Definitions of a supercomputer: • Fastest machine in world at given task • A device to turn a compute-bound problem into an I/O bound problem • Any machine costing $30M+ • Any machine designed by Seymour Cray

• CDC6600 (Cray, 1963) regarded as first supercomputer

Control Data Corporation (CDC), 1957-1992 CDC 6600 Seymour Cray , 1963 • A fast pipelined machine with 60-bit words – 128 Kword main memory capacity, 32 banks • Ten functional units (parallel, unpipelined) – Floating Point: adder, 2 multipliers, divider – Integer: adder, 2 incrementers, ... • Hardwired control (no microcoding) • Scoreboard for dynamic scheduling of instructions • Ten Peripheral Processors for Input/Output – a fast multi-threaded 12-bit integer ALU • Very fast clock, 10 MHz (FP add in 4 clocks) • >400,000 transistors, 750 sq. ft., 5 tons, 150 kW, novel freon-based technology for cooling • Fastest machine in world for 5 years (until 7600) – over 100 sold ($7-10M each) CDC 6600: A Load/Store Architecture (RISC) • Separate instructions to manipulate three types of reg. – 8x60-bit data registers (X) – 8x18-bit address registers (A) – 8x18-bit index registers (B)

• All arithmetic and logic instructions are register-to-register 6 6 6 6 opcode i j k Ri ← Rj op Rk

• Only Load and Store instructions refer to memory! 6 6 6 6 opcode i j disp Ri ← M[Rj + disp] Touching address registers 1 to 5 initiates a load 6 to 7 initiates a store -very useful for vector operations CDC6600 Scoreboard • Instructions dispatched in-order to functional units provided no structural hazard or WAW – Stall on structural hazard, no functional units available – Only one pending write to any register • Instructions wait for input operands (RAW hazards) before execution – Can execute out-of-order • Instructions wait for output register to be read by preceding instructions (WAR) – Result held in functional unit until register free Scoreboarding

Centralized scheme – No bypassing – WAR/WAW hazards are problems

• Originally proposed in CDC6600 (S. Cray,1963) First Stage of Scoreboarding – Issue (or Dispatch) • Issue replaces first half of ID stage – Check structural hazard • All function units for this instruction are busy – Check WAW hazards • Active instruction wants to write the same destination register – Stall all current (future) inst. issuing until none of these hazards remain (In-Order) – Issue the instruction and update scoreboard Second Stage of Scoreboarding – Read Operands (or Issue!) • Read operands replaces the second half of ID – Check for RAW hazards: are all source operands available? • If NO, hold instruction in a pre-execution buffer (I- Buffer) • If YES, scoreboard tells functional unit to read operands from registers and begin execution Third Stage of Scoreboarding – Execution • Execution replaces EX in MIPS – Begin execution upon receiving operands – Notify scoreboard when it completes execution Fourth Stage of Scoreboarding – Write Result • Write Result replaces WB in MIPS – Checks for WAR hazards • Any preceding instruction has not read its operands, and • One of operand is the same register as the destination register of the completing inst. • Stalls the completing inst. If WAR detected • Otherwise, functional unit writes its result to destination register Scoreboard Implementation • Instruction status: which of the stages the inst. is in • Functional unit status: state of the functional unit – Busy : indicates whether the unit is busy or not – Op : operation to perform in the unit – Fi : destination register – Fj , Fk : source registers – Qj , Qk : functional units producing Fj, Fk – Rj , Rk : flags indicating when Fj, Fk are ready and not yet read. Set to NO after operands are read. • Register result status: indicates which functional unit will write each register Scoreboard Example Scoreboard Example: Cycle 1 Scoreboard Example: Cycle 2

Issue 2nd LD? Scoreboard Example: Cycle 3

Issue MULT? Scoreboard Example: Cycle 4 Scoreboard Example: Cycle 5 Scoreboard Example: Cycle 6 Scoreboard Example: Cycle 7

Read MULT operands? Scoreboard Example: Cycle 8a (First half of clock cycle) Scoreboard Example: Cycle 8b (Second half of clock cycle) Scoreboard Example: Cycle 9

Read operands for MULT & SUB? Issue ADD? Scoreboard Example: Cycle 10 Scoreboard Example: Cycle 11 Scoreboard Example: Cycle 12

Read Operands for DIV? Scoreboard Example: Cycle 13 Scoreboard Example: Cycle 14 Scoreboard Example: Cycle 15 Scoreboard Example: Cycle 16 Scoreboard Example: Cycle 17

Why not write result of ADD? Scoreboard Example: Cycle 18 Scoreboard Example: Cycle 19 Scoreboard Example: Cycle 20 Scoreboard Example: Cycle 21

WAR Hazard is now gone… Scoreboard Example: Cycle 22 Skip Couple of Cycles Scoreboard Example: Cycle 61 Scoreboard Example: Cycle 62 Review Scoreboard Example: Cycle 62

In-order issue; out-of-order execute & commit Scoreboarding Limitations • Number and type of functional units

• Number of scoreboard entries

• Amount of application ILP (RAW hazards)

• The presence of anti and output dependence – Lead to WAR and WAW stalls