EECS Electrical Engineering and Computer Sciences BERKELEY PAR LAB P A R A L L E L C O M P U T I N G L A B O R A T O R Y The Roofline Model: A pedagogical tool for program analysis and optimization Samuel Williams1,2, David Patterson1, Leonid Oliker1,2, John Shalf2, Katherine Yelick1,2 1University of California, Berkeley 2Lawrence Berkeley National Laboratory [email protected] 1 EEElectrical EngCineeringS and Outline Computer Sciences BERKELEY PAR LAB Motivation, Goals, Audience, etc… Survey of multicore architectures Description of the Roofline model Introduction to Auto-tuning Application of the roofline to auto-tuned kernels . Example #1 - SpMV . Example #2 - LBMHD Conclusions 2 EEElectrical EngCineeringS and Motivation Computer Sciences BERKELEY PAR LAB Multicore guarantees neither good scalability nor good (attained) performance Performance and scalability can be extremely non-intuitive even to computer scientists Success of the multicore paradigm seems to be premised upon their programmability To that end, one must understand the limits to both scalability and efficiency. - How can we empower programmers? 3 EEElectrical EngCineeringS and Primary Focus Computer Sciences BERKELEY PAR LAB Throughput-oriented kernels (rather than time) Our performance metrics are: Gflop/s and % of peak (efficiency) for purposes of this talk, I will focus on memory-intensive 64b floating-point SPMD kernels. Not focused on algorithmic innovation or computational complexity 4 EEElectrical EngCineeringS and Goals & Audience Computer Sciences BERKELEY PAR LAB Goals for Roofline: . Provide everyone (especially undergrads) with a graphical aid that provides: realistic expectations of performance and productivity . Show inherent hardware limitations for a given kernel . Show potential benefit and priority of optimizations Who’s not the audience for the Roofline: . Not for those interested in fine tuning (+10%) . Not for those challenged by parallel kernel correctness 5 EECS Electrical Engineering and Computer Sciences BERKELEY PAR LAB P A R A L L E L C O M P U T I N G L A B O R A T O R Y Multicore SMPs of Interest (used throughout the rest of the talk) 6 EEElectrical EngCineeringS and Multicore SMPs Used Computer Sciences BERKELEY PAR LAB Intel Xeon E5345 (Clovertown) AMD Opteron 2356 (Barcelona) Core Core Core Core Core Core Core Core Opteron Opteron Opteron Opteron Opteron Opteron Opteron Opteron 4MB 4MB 4MB 4MB 512KB 512KB 512KB 512KB 512KB 512KB 512KB 512KB shared L2 shared L2 shared L2 shared L2 victim victim victim victim victim victim victim victim 2MB Shared quasi-victim (32 way) 2MB Shared quasi-victim (32 way) FSB FSB 4GB/s (each direction) 10.66 GB/s 10.66 GB/s SRI / crossbar HyperTransport HyperTransport SRI / crossbar Chipset (4x64b controllers) 2x64b memory controllers 2x64b memory controllers 21.33 GB/s(read) 10.66 GB/s(write) 10.66 GB/s 10.66 GB/s 667MHz FBDIMMs 667MHz DDR2 DIMMs 667MHz DDR2 DIMMs Sun T2+ T5140 (Victoria Falls) IBM QS20 Cell Blade VMT VMT SPE SPE SPE SPE SPE SPE SPE SPE SPE SPE SPE SPE SPE SPE SPE SPE PARC PARC PARC PARC PARC PARC PARC PARC PARC PARC PARC PARC PARC PARC PARC PARC PPE PPE S S S S S S S S S S S S S S S S MT MT MT MT MT MT MT MT MT MT MT MT MT MT MT MT 256K 256K 256K 256K 256K 256K 256K 256K 256K 256K 256K 256K 256K 256K 256K 256K 512K 512K BIF BIF Crossbar Crossbar irection) L2 d L2 MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC <20GB/s 179 GB/s 90 GB/s B/s 179 GB/s 90 GB/s G (each 4MB Shared L2 (16 way) 4MB Shared L2 (16 way) EIB (ring network) EIB (ring network) (64b interleaved) (64b interleaved) 8 x 6.4 (1 per hub direction) 4 Coherency Hubs 4 Coherency Hubs XDR memory controllers XDR memory controllers 2x128b controllers 2x128b controllers 25.6 GB/s 25.6 GB/s 21.33 GB/s 10.66 GB/s 21.33 GB/s 10.66 GB/s 512MB XDR DRAM 512MB XDR DRAM 667MHz FBDIMMs 667MHz FBDIMMs 7 EEElectrical EngCineeringS and Multicore SMPs Used Computer Sciences BERKELEY PAR LAB Intel Xeon E5345 (Clovertown) AMD Opteron d2356 (Barcelona) se Core Core Core Core Core Core Core Core Opteron Opteron OpteronaOpteron Opteron Opteron Opteron Opteron 4MB 4MB 4MB 4MB 512KB 512KB 512KB 512KB 512KB 512KB 512KB 512KB shared L2 shared L2 shared L2 shared L2 victim -victimbvictim victim victim victim victim victim 2MB Shared quasi-victim (32 way) 2MB Shared quasi-victim (32 way) FSB FSB e 4GB/s (each direction) 10.66 GB/s 10.66 GB/s h SRI / crossbar HyperTransport HyperTransport SRI / crossbar Chipset (4x64b controllers) y c 2x64b memory controllers 2x64b memory controllers 21.33 GB/s(read) 10.66 GB/s(write) a h c 10.66 GB/s 10.66 GB/s 667MHz FBDIMMs C r l a 667MHz DDR2 DIMMs 667MHz DDR2 DIMMs na er io Hi nt y Sun T2+ eT5140 (Victoriaor Falls) IBM QS20 Cell Blade nv m re VMT oVMT SPE SPE SPE SPE SPE SPE SPE SPE SPE SPE SPE SPE SPE SPE SPE SPE PARC PARC PARC PARC PARC PARC PARC PARC PARC PARC PARC PARC e PARC PARC PARC PARC t o PPE PPE S S S S S S S S S S S S S S S S S MT MT MT MT MT MT MT MT MT MT MT MT MT MT MT MT 256K 256K 256K 256K 256K 256K 256K 256K C M 256K 256K 256K 256K 256K 256K 256K 256K l y 512K 512K BIF BIF Crossbar Crossbar irection) d h L2 a L2 MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC <20GB/s B/s c c 179 GB/s 90 GB/s 179 GB/s 90 GB/s G r o(each 4MB Shared L2 (16 way) 4MB Shared L2 (16 way) EIB (ring network) EIB (ringa network) (64b interleaved) (64b interleaved) L r 8 x 6.4 (1 per hub direction) t e 4 Coherency Hubs 4 Coherency Hubs XDR memory controllersn XDRi memory controllers 2x128b controllers 2x128b controllers i H jo25.6 GB/s y 25.6 GB/s 21.33 GB/s 10.66 GB/s 21.33 GB/s 10.66 GB/s is512MB XDR DRAM or 512MB XDR DRAM 667MHz FBDIMMs 667MHz FBDIMMs D m Me 8 EEElectrical EngCineeringS and Multicore SMPs Used Computer Sciences BERKELEY PAR LAB Intel Xeon E5345 (Clovertown) AMD Opteron 2356 (Barcelona) Core Core Core Core Core Core Core Core Opteron Opteron Opteron Opteron Opteron Opteron Opteron Opteron 4MB 4MB 4MB 4MB 512KB 512KB 512KB 512KB 512KB 512KB 512KB 512KB shared L2 shared L2 shared L2 shared L2 victim victim victim victim victim victim victim victim 2MB Shared quasi-victim (32 way) 2MB Shared quasi-victim (32 way) FSB FSB 4GB/s (each direction) 10.66 GB/s 10.66 GB/s SRI / crossbar HyperTransport HyperTransport SRI / crossbar Chipset (4x64b controllers) 2x64b memory controllers 2x64b memory controllers 21.33 GB/s(read) 10.66 GB/s(write) 10.66 GB/s 10.66 GB/s 667MHz FBDIMMs 667MHz DDR2 DIMMs 667MHz DDR2 DIMMs Sun T2+ T5140 (Victoria Falls) IBM QS20 Cell Blade es VMT VMT SPE SPE SPE SPE SPE SPE SPE SPE r SPE SPE SPE SPE SPE SPE SPE SPE PARC PARC PARC PARC PARC PARC PARC PARC PARC PARC PARC PARC PARC PARC PARC PARC PPE PPE S S S S S S S S S S S S S S S S o MT MT MT MT MT MT MT MT MT MT MT MT MT MT MT MT 256K 256K 256K 256K 256K 256K 256K 256K c 256K 256K 256K 256K 256K 256K 256K 256K 512K 512K BIF BIF Crossbar Crossbar irection) d L2 d L2 MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC <20GB/s 179 GB/s 90 GB/s B/s 179e GB/s 90 GB/s G d (each 4MB Shared L2 (16 way) a 4MB Shared L2 (16 way) EIB (ring network) EIB (ring network) (64b interleaved) (64b interleaved) e 8 x 6.4 (1 per hub direction) 4 Coherency Hubsr 4 Coherency Hubs XDR memory controllers XDR memory controllers 2x128b controllersh 2x128b controllers tit 25.6 GB/s 25.6 GB/s 21.33 GB/s 10.66 GB/s 21.33 GB/s 10.66 GB/s ul 512MB XDR DRAM 512MB XDR DRAM m 667MHz FBDIMMs 667MHz FBDIMMs 9 EECS Multicore SMPs Used Electrical Engineering and Computer Sciences (peak double precision flops) BERKELEY PAR LAB Intel Xeon E5345 (Clovertown) AMD Opteron 2356 (Barcelona) Core Core Core Core Core Core Core Core Opteron Opteron Opteron Opteron Opteron Opteron Opteron Opteron 4MB 4MB 4MB 4MB 512KB 512KB 512KB 512KB 512KB 512KB 512KB 512KB shared L2 shared L2 shared L2 shared L2 victim victim victim victim victim victim victim victim 2MB Shared quasi-victim (32 way) 2MB Shared quasi-victim (32 way) FSB FSB 4GB/s (each direction) 10.66 GB/s 10.66 GB/s SRI / crossbar HyperTransport HyperTransport SRI / crossbar Chipset (4x64b controllers) 75 GFlop/s 2x64b memory74 controllers Gflop/s2x64b memory controllers 21.33 GB/s(read) 10.66 GB/s(write) 10.66 GB/s 10.66 GB/s 667MHz FBDIMMs 667MHz DDR2 DIMMs 667MHz DDR2 DIMMs Sun T2+ T5140 (Victoria Falls) IBM QS20 Cell Blade VMT VMT SPE SPE SPE SPE SPE SPE SPE SPE SPE SPE SPE SPE SPE SPE SPE SPE PARC PARC PARC PARC PARC PARC PARC PARC PARC PARC PARC PARC PARC PARC PARC PARC PPE PPE S S S S S S S S S S S S S S S S MT MT MT MT MT MT MT MT MT MT MT MT MT MT MT MT 256K 256K 256K 256K 256K 256K 256K 256K 256K 256K 256K 256K 256K 256K 256K 256K 512K 512K BIF BIF Crossbar Crossbar irection) L2 d L2 MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC MFC <20GB/s 179 GB/s 90 GB/s B/s 179 GB/s 90 GB/s G (each 4MB Shared L2 (16 way) 4MB Shared L2 (16 way) EIB (ring network) EIB (ring network) (64b interleaved) (64b interleaved) 19 GFlop/s8 x 6.4 (1 per hub direction) 29* GFlop/s 4 Coherency Hubs 4 Coherency Hubs XDR memory controllers XDR memory controllers 2x128b controllers 2x128b controllers 25.6 GB/s 25.6 GB/s 21.33 GB/s 10.66 GB/s 21.33 GB/s 10.66 GB/s 512MB XDR DRAM 512MB XDR DRAM 667MHz FBDIMMs 667MHz FBDIMMs *SPEs only 10 EECS Multicore SMPs Used Electrical Engineering and Computer Sciences (total DRAM bandwidth) BERKELEY PAR LAB Intel Xeon E5345 (Clovertown) AMD Opteron 2356 (Barcelona) Core Core Core Core Core Core Core Core Opteron Opteron Opteron Opteron Opteron Opteron Opteron Opteron 4MB 4MB 4MB 4MB 512KB 512KB 512KB 512KB 512KB 512KB 512KB 512KB shared