Hot Chips 2008 August 2008 © 2008 IBM Corporation Systems and Technology Group + Research Division Outline

Systems Technology Group and Research Division Power-Performance Comparative Evaluation of Alternate Microarchitectures Rick Eickemeyer, Michael Floyd, John Griswell, Alex Mericas, Balaram Sinharoy (IBM Systems and Technology Group) Pradip Bose, Soraya Ghiasi, Hendrik Hamann, Hans Jacobson, Tom Keller, Victor Zyuban (IBM Research Division)

Acknowledgements: Brad McCredie & many others in processor/system design teams within IBM STG + members of the power-aware microarchitectures and systems departments within IBM Research

 Power Dissipation and Efficiency Basics  POWER4 vs. POWER5  POWER5 vs. POWER6  Roadrunner and Blue Gene System Efficiency  Conclusion  BACKUP: Looking Ahead: A Few Key Research Issues

Clock Tree Other IDU FXU 10% 10% 3% 4% IFU Issue L3 Tags 6% Queues 32% 2% ISU 10% Map Tables Completion 43% Table L2 9% 23% Dispatch 6%

FPU ISU ISU FPU FXU FXU LSU IDU IDU IFU LSU LSU IFU 19% BXU BXU CIU 4% GX FBC ZIO RAS Core Buffer FPU L2 L2 L2 1% 3% 4% 5% 1% 5% L3Directory/Control Pre-silicon, POWER4-like superscalar design

D. Brooks, et. al. MICRO-03 (tutorial)

Processor Power Pie-Chart: Another View

. High performance processors (prior/current generation) typically burn most of their power in the clocked latches and arrays (registers, caches).

(taken from: Bose, Martonosi, Brooks: Sigmetrics-2001 Tutorial) 1% 9%

28% Clks Dist Latches Logic IO Arrays Example data 4% 46% other 12% Pre-silicon ckt-sim based; assumes: no clock-gating

In addition: Non-uniform power distribution or hotspots aggravate challenges significantly:

Hotspots limit performance, reliability & increase costs As we go forward (towards decreasing technology nodes): - Hotspot factor (overhead) is likely continue to increase - predictability of hotspots will be more difficult (multicore, SoC, power / thermal management, variability etc.)

16 Source: Berkeley CPU Center, spec.org, etc. 30

14 us3-1.7ghz . Performance metrics: power5 25 us3-1.4ghz 12 us2-480 delay (execution time) per power4+ – IBM 20 10 instruction; MIPS power3-ii us3-1ghz • CPI (cycles per instr): abstracts out 8 us2 power4 15 the MHz specint/watt 6 • SPEC (int or fp); TPM: factors in us1 10

4 us1+ specint**3/w(millions) benchmark, MHz ppc620us1 power3+ power3-200mhz Sun 5 2 p2sc . energy and power metrics: P. Bose, VLSI Design Conf. 2004 0 0 – joules (J) and watts (W) 1996 1998 2000 2002 1996 1998 2000 2002 1997 1999 2001 2003 . joint metric possibilities (perf and 1997 1999 2001 2003 power or temperature) Single-core regime, through start of multi-cores – watts (W): for ultra LP processors; also, thermal issues Jan. 2002 – MIPS/W or SPEC/W ~ energy per estimated from published instruction data for particular • CPI * W: equivalent inverse metric assumed system 2 2 – MIPS /W or SPEC /W ~ configurations energy*delay (EDP) – MIPS3/W or SPEC3/W ~ energy*(delay)2 (ED2P) (TPC-C)/System tpm Watts – (Peak Temp) * (Execution Time)

Sun E10K Sun F15K System-level perf/watt for commercial OLTP IBM 24-way p680 IBM 32-way p690 HP SuperDome-64

is quite different from processor-level SPECint/watt ! Compaq AlphaServer GSFujitsu PrimePower2000

. Fundamental microarchitectural knobs that determine efficiency – optimal pipeline depth at the core-level – optimal core complexity and number of cores – type of clock-gating and power-gating (if applicable): coarse-grain vs. fine-grain – adaptive microarchitectures: [to control unnecessary energy waste] – etc…

. Fundamental logic/circuit-level efficiency features – support for clock-gating (area and verification-efficient) – support for voltage and frequency scaling (performance, reliability and verification-friendly) – (Near)-optimal mix of low, medium and high-Vt devices – area and power efficient latch design – etc..

Power-performance optimal Performance optimal 1 SPEC2000 suite 0.9 Direction of shift of optimal point 0.8 if leakage power component were to escalate 0.7

0.6 Direction of shift of optimal point 0.5 for commercial OLTP workloads 0.4 bips 0.3 bips^3/W 0.2

Relative to Optimal FO4 0.1 0 37 34 31 28 25 22 19 16 13 10 7 Total FO4 Per Stage

. With uniprocessor performance improvements slowing, multiple cores per chip (socket) will help continue the exponential system performance growth . Exploit performance through higher levels of integration in chips, modules, and systems . Invest power in chip-level performance rather than core performance

FPU ISU ISU FPU FXU FXU

IDU IDU IFU LSU LSU IFU POWER 4: 2001 BXU BXU 180 nm, Cu, SOI POWER 5: 2004 2 cores / chip 130 nm, Cu, SOI 2 cores / chip L2 L2 L2 POWER 4+: 130 nm 2 way SMT / core

L3Directory/Control POWER5+: 90nm

. Basic concepts: – use (i.e. power or clock) a storage, compute or interconnect (e.g. bus) resource only to the extent needed: adapt or reconfigure dynamically in tune with workload resources • Predictive power-gating to reduce leakage • Dynamic resizing of queues, buffers, caches – dynamically change a bandwidth parameter to conserve power • Adaptive fetch to minimize speculative waste • Adaptive prefetch to conserve bus bandwidth and prefetch logic usage; reduce speculative waste (cache pollution) • etc…. . Issues that prevent widespread adoption in high-end processors: – complexity (verification cost, overhead area/power) – relatively small power savings, if performance loss is not tolerable In general, dynamic voltage-frequency scaling (DVFS) offers the most efficient knob for power management

10 | Hot Chips 2008 August 2008 © 2008 IBM Corporation Systems and Technology Group + Research Division Multithreaded Instruction Flow in Processor Pipeline (transition from POWER4 to POWER5) Out-of-Order Processing Branch Redirects

Instruction Fetch BR MP ISS RF EX WB Xfer LD/ST IFIF IC BP MP ISS RF EA DC Fmt WB Xfer CPCP D0D0 D1 D2 D3 Xfer GD MP ISS RF EX WB Xfer FX Group Formation and MP ISS RF F6F6 Instruction Decode F6F6 FP F6F6 WB Xfer Interrupts & Flushes

Branch Prediction Dynamic Instruction Shared Selection Execution Program Branch Return Target Cache Shared Issue Units Counter History Stack Queues Tables LSU0 Alternate FXU0 Instruction LSU1 Buffer 0 Group Formation, Instruction Instruction Decode, FXU1 Group Store Cache Completion Instruction Dispatch FPU0 Queue Instruction Buffer 1 FPU1 Translation BXU Thread Data Data Priority Shared CRL Translation Cache Read Shared Write Shared Register Register Files Register Files Mappers L2 Cache Shared by two threads Resource used by thread 0 Resource used by thread 1 11 | Hot Chips 2008 August 2008 © 2008 IBM Corporation Systems and Technology Group + Research Division

Energy Per Useful Instruction: POWER4+ vs. POWER5

50 0

40 0

30 0

20 0

RelativeEnergy 10 0 PerUseful Instruction 0 ORACLE(p) TPCC(i) SAP(i) NOTES(i)

P4 + P5 ST P5 SM T (no gating) P5 SMT (clock-gated)

.Steady growth in single-thread performance: • POWER4  POWER4+  POWER5 .Efficient throughput increase in POWER5: • typical OLTP: 40 % IPC growth at 20 % more power

80 70 SMT ST IFU 60 50 IDU % power savings 40 ISU over baseline 30 FXU 20 LSU 10 FPU 0 Core Notes (i) SAP (i) TPC-C (i) TPC-C(p) DAXPY SparseMV TPP

Note: post-silicon hardware-based analysis shows good agreement at the chip level

P. Chaudhary, SPSICOMP, Mar. 2008

15 | Hot Chips 2008 August 2008 © 2008 IBM Corporation Systems and Technology Group + Research Division Benefit of Fine-Grain Clock Gating in POWER6 pre-silicon simulation unconstrained, max (normalized to 1)

1 . Power reduction due to 0.9 clock-gating (average) SMT runs ST runs 0.8

0.7 – Floating point kernels 19% – SPEC2K 24% 0.6 – Commercial 26% 0.5

– RFCTH 20% 0.4 – POP 25% 0.3 – LBMHD 23 % 0.2

RFCTH is as “hot” as 0.1 0 daxpy; POP is similar gc swim amm ej tpccora fft perlbm dot pop ST(ST ST(ST To SPEC2K average rfcth Leakage power daxpy lbmhd trade2 pop )rfcth() )lbmhd()daxpy Daxpy clock-gating factors – validated via direct post-silicon measurements 16 | Hot Chips 2008 August 2008 © 2008 IBM Corporation Systems and Technology Group + Research Division

Clock Gating – Temperature Benefit

Temperature Versus Frequency

DAXPY with Clock Gating DAXPY without Clock Gating DAXPY without Clock Gating (estimated) NULL INS with Clock Gating NULL INS without Clock Gating NULL INS without Clock Gating (estimated)

150

140

130

120 Tcrit 110

100 Twarn 90

Temperature(Celsius) 80

60 3000 3200 3400 3600 3800 4000 4200 4400 4600 4800 5000 Frequency (MHz)

Prototype hardware, both cores good, real h/w measurements (POWER6)

. Clock gating benefit – POWER4: performance-centric, with minimal clock-gating – POWER5: SMT throughput boost, matched with fine-grain clock-gating to manage power – POWER6: High frequency performance boost with aggressive, fine-grain clock-gating to manage power and thermals

. Net: progressive improvement with POWER6 being the best

SMT: area-efficient, thermally-efficient 3 heat-up mechanisms 365 . Unit self heating 360 determined by the power density of the 355 unit 350 . Lateral thermal 345 coupling between 340 AveragePeak Temperature (K) neighboring units 335

ST . Global heating through SMT CMP TIM (thermal interface ST (area enlarged)

CMP (one core rotated) material), heat SMT(only activity factor) spreader, and heat sink

P. Bose, VLSI Design 2005, quoted from Y. Li, Z. Hu et al. 2004

A brief look now at a different system product space ….

D. Grice, SCICOMP-14, March 2008; http://www.spscicomp.org/

• IBM CU-11, 0.13 µm • 11 x 11 mm die size • 25 x 32 mm CBGA • 474 pins, 328 signal • 1.5/2.5 Volt

Integrated functionality • Two PPC 440 cores • Two “double FPUs” • L2 and L3 caches • Torus network • Tree network • JTAG • Performance counters • EDRAM

22 | Hot Chips 2008 August 2008 © 2008 IBM Corporation Systems and Technology Group + Research Division The Green500 Top 10 (http://www.green500.org) Green500 MFLOPS/Watt Computer Total Power Top 500 Rank Rank (all IBM) (kW)

1 488.14 Roadrunner 22.76 324 BladeCenter QS2 – PowerXCell 8i

1 488.14 Roadrunner 18.97 464 3 437.43 Roadrunner 2345.50 1 4 371.75 Blue 31.50 304 Gene/P 4 371.75 BG/P 31.50 305 4 371.75 BG/P 94.50 306 7 371.67 BG/P 63.00 52 7 371.67 BG/P 94.50 75 7 371.67 BG/P 126.00 51 7 371.67 BG/P 63.00 37

23 | Hot Chips 2008 August 2008 © 2008 IBM Corporation Systems and Technology Group + Research Division Concluding Remarks [Based on POWER Systems experiences so far] . Power-performance tradeoff analysis must be integral part of early-stage definition of microprocessors – Fundamental design decision errors can lead to post-silicon power overruns and/or performance shortfalls – Pre-silicon power-performance modeling and validation methodology: key investment needed to prevent post-silicon surprises – Power analysis and tuning must percolate through all stages of design, with closed loop feedback to higher levels. – Temperature-aware vs. power-aware: needs careful balance

. Power-aware microarchitecture techniques: can be a key lever in future power reduction at the chip and system level – But, co-design with circuit/technology and software groups is key

. Power “optimization” in server-class, high-end systems can be quite different from that in embedded systems – System-level power-performance efficiency requires careful separation of emphasis on efficiency at the processor, memory and system sub-components – IBM’s POWER Systems microprocessors have been designed with system-level efficiencies in mind and have proven to be very successful offerings in the Green Computing Era.

BACKUP: Some Key Research Issues of the Future

. M1-level simulation (FPU) FPU0 with Shallow Pipeline Morph (TSP)

– Transparent clock gated pipeline 100 90 80 70 60 50 40 30 Clock Power in % in vs.Clock Power 20

traditional stage-level clockstage-level traditional gating 10 0

fldaxpy_fldfft_TSP5 fldaxpy_flddot_TSP5 fldaxpy_fldsqrt_TSP5 fldaxpy_flssqrt_TSP5 fldaxpy_fldi4l01_TSP5fldaxpy_fldi4l02_TSP5fldaxpy_fldi4l20_TSP5fldaxpy_fldi4l23_TSP5fldaxpy_fldi4l29_TSP5fldaxpy_fldi4l35_TSP5fldaxpy_fldi4l41_TSP5fldaxpy_fldi4l42_TSP5fldaxpy_fldi4l43_TSP5fldaxpy_fldi4l59_TSP5fldaxpy_fldi4l63_TSP5 fldaxpy_flki4l04_TSP5fldaxpy_flki4l05_TSP5fldaxpy_flki4l09_TSP5fldaxpy_flmi4ld1_TSP5 fldaxpy_flsi4l41_TSP5fldaxpy_flsi4l42_TSP5 fldaxpy_flddivide_TSP5 fldaxpy_fldi4matb_TSP5fldaxpy_flinti4l02_TSP5 fldaxpy_flsdivide_TSP5 fldaxpy_fldi4l59_p6_TSP5 fldaxpy_flmi4ld4st2_TSP5 fldaxpy_flmi4ld0stm2_TSP5fldaxpy_flmi4ld16st1_TSP5fldaxpy_flmi4ld3stm1_TSP5 fldaxpy_flmi4ld4st2_p6_TSP5 fldaxpy_flmi4ld4st2_stonly_TSP5 FLTLOOPS

FPU0 FPU1

100 100 90 90 80 80 70 70 60 60 50 50 40 40 30 30 Clock pulses inpulses % Clock for

TCGopaq gating vs. clk 20 20 10 10 0 0

as400_testt_TSP7 as400_testt_TSP6 aix_tpccora51_TSP5as400_cpw252_TSP5tpcc_db2-tpcc_TSP5as400_testn3_TSP7 aix_tpccora51_TSP5as400_cpw252_TSP5tpcc_db2-tpcc_TSP5as400_testn3_TSP6 as400_trade2ejb52_TSP7as400_trade2ejb52j_TSP7 as400_trade2ejb52_TSP6as400_trade2ejb52j_TSP5

Commercial / TPC-C Ref: H. Jacobson et al., ISLPED04, HPCA-05

Dynamic mgmt of power, temperature, noise, reliability, performance….

Across-die monitored variability )in perf, power, temp, Vdd, …) will increase in the multi/many-core area. Effective control and management will require integrated, hierarchical, closed-loop feedback control mechanisms

•On-chip controller can also serve Application Virtual as static (v,f) setting device for Machine Thread/Resource effective yield and good baseline OS scheduling Chip power Load balancing performance Global budget power/perf Thread-core •Per-core DFVS: costly and requires information affinities async interfaces to bus/fabric; also GLOBALGlobal MANAGEMENTMonitoring further exacerbates soft error rates and ControlPer-core power •Control loop stability issues must Per-core power/perf modes be analyzed (pre-silicon) information Local Monitoring and Control •Simple, scalable global DVFS control algorithms: optimize perf for given power budget

C. Isci, A. Buyuktosunoglu, C-Y-Cher, P. Bose, M. Martonosi, MICRO-39, 2006

. Ultra-high frequency dual-core chip RU – 7-way superscalar, 2-way SMT core – 9 execution units L2 Core L2 • 2LS, 2FP, 2FX, 1BR, 1VMX,1DFU Data Data – 790M transistors L3 – Up to 64-core SMP systems L2 Ctrl Ctrl – 2x4MB on-chip L2 – 32MB On-chip L3 directory and controller Mem SMP I/O Mem – Two memory controllers on-chip Ctrl Interconnect Ctrl Ctrl – Recovery Unit . Technology L2 L3 Ctrl – CMOS 65nm lithography, SOI Ctrl . High-speed elastic bus interface at 2:1 freq L2 L2 – I/Os: 1953 signal, 5399 Power/Gnd Data Core Data

Research Issue: power-efficient RAS microarchitecture RU in the face of increasing SER and other sources of unreliability 28 | Hot Chips 2008 August 2008 © 2008 IBM Corporation Systems and Technology Group + Research Division POWER5 Hotspot Patterns

Thermal map Power map

-50 different workloads for POWER5 imaged & analyzed •HotGen microbenchmark generator tool - observed significant differences in circuit utilization

(H. Hamann et al., ISSCC-2006)

Leveraging Spatial Heat Slack J. Choi, C-Y, Cher et al., ISLPED07 Activity Migration reduces Hotspots

Summary: Core-hopping (4ms) reduces maximum on-chip temperature 10.0 9.0 8.0 maximum delta temperature 7.0

6.0 5.5 4.9 5.1 5.0 4.2

4.0 3.3 3.5 (Celsius) 3.0 2.3 2.2 2.0 2.0 1.0

Reduction In TemperaturesReductionIn 0.0

vpr apsi lucas swim bzip2 twolf daxpy fma3d vortex Workloads

% slow down 0.1 -1.1 -0.5 0.4 1.0 1.1 1.6 0.9 2.5

A Page from IBM EnergyScale for POWER6 Systems

http://www-03.ibm.com/systems/power/hardware/whitepapers/energyscale.html