High Performance Computing: Concepts, Methods, & Means SMP Node Architecture
Prof. Thomas Sterling Department of Computer Science Louisiana State University February 1, 2007 Topics
• Introduction • SMP Context • Performance: Amdahl’s Law • SMP System structure • Processor core • Memory System • Chip set • South Bridge – I/O • Performance Issues • Summary – Material for the Test
2 2 Topics
• Introduction • SMP Context • Performance: Amdahl’s Law • SMP System structure • Processor core • Memory System • Chip set • South Bridge – I/O • Performance Issues • Summary – Material for the Test
3 3 Opening Remarks
• This week is about supercomputer architecture – Last time: major factors, classes, and system level – Today: modern microprocessor and multicore SMP node • As we’ve seen, there is a diversity of HPC system types • Most common systems are either SMPs or are ensembles of SMP nodes • “SMP” stands for: “Symmetric Multi-Processor ” • System performance is strongly influenced by SMP node performance • Understanding structure, functionality, and operation of SMP nodes will allow effective programming • Next time: making SMPs work for you! 4 4 The take-away message
• Primary structure and elements that make up an SMP node • Primary structure and elements that make up the modern multicore microprocessor component • The factors that determine microprocessor delivered performance • The factors that determine overall SMP sustained performance • Amdahl’s law and how to use it • Calculating cpi • Reference: J. Hennessy & D. Patterson, “Computer Architecture A Quantitative Approach” 3rd Edition, Morgan Kaufmann, 2003
5 5 Topics
• Introduction • SMP Context • Performance: Amdahl’s Law • SMP System structure • Processor core • Memory System • Chip set • South Bridge – I/O • Performance Issues • Summary – Material for the Test
6 6 SMP Context
• A standalone system – Incorporates everything needed for • Processors • Memory • External I/O channels • Local disk storage • User interface – Enterprise server and institutional computing market • Exploits economy of scale to enhance performance to cost • Substantial performance – Target for ISVs (Independent Software Vendors) • Shared memory multiple thread programming platform – Easier to program than distributed memory machines – Enough parallelism to • Building block for ensemble supercomputers – Commodity clusters – MPPs 7 7 Topics
• Introduction • SMP Context • Performance: Amdahl’s Law • SMP System structure • Processor core • Memory System • Chip set • South Bridge – I/O • Performance Issues • Summary – Material for the Test
8 8 Performance: Amdahl’s Law
Baton Rouge to Houston • from my house on East Lakeshore Dr. • to downtown Hyatt Regency • distance of 271 • in air flight time: 1 hour • door to door time to drive: 4.5 hours • cruise speed of Boeing 737: 600 mph • cruise speed of BMW 528: 60 mph 9 9 Amdahl’s Law: drive or fly?
• Peak performance gain: 10X – BMW cruise approx. 60 MPH – Boeing 737 cruise approx. 600 MPH • Time door to door – BMW • Google estimates 4 hours 30 minutes – Boeing 737 • Time to drive to BTR from my house = 15 minutes • Wait time at BTR = 1 hour • Taxi time at BTR = 5 minutes • Continental estimates BTR to IAH 1 hour • Taxi time at IAH = 15 minutes (assuming gate available) • Time to get bags at IAH = 25 minutes • Time to get rental car = 15 minutes • Time to drive to Hyatt Regency from IAH = 45 minutes • Total time = 4.0 hours • Sustained performance gain: 1.125X
10 10 Amdahl’s Law
TO start end
TA TF
TO ≡ time for non - accelerated computation start end TA ≡ time for accelerated computation
TF ≡ time of portion of computation that can be accelerated T /g F g ≡ peak performance gain for accelerated portion of computation f ≡ fraction of non - accelerated computation to be accelerated S ≡ speed up of computation with acceleration applied
S = TO TA
f = TF TO f TA = ()1− f ×TO + ×TO g TO S = f ()1− f ×TO + ×TO g 1 S = f 1− f + g 11 11 Amdahl’s Law with Overhead
TO start end
tF tF tF tF TA
start end n TF = ∑tFi v + tF/g i v ≡ overhead of accelerated work segment n V ≡ total overhead for accelerated work = ∑vi i f TA = ()1− f ×TO + ×TO + n×v g
TO TO S = = TA f ()1− f ×TO + ×TO + n×v g 1 S = f n×v ()1− f + + g TO 12 12 Amdahl’s Law and Parallel Computers
• Amdahl’s Law (FracX: original % to be speed up) Speedup = 1 / [(FracX/SpeedupX + (1 FracX)] • A portion is sequential => limits parallel speedup – Speedup <= 1/ (1 FracX) • Ex. What fraction sequential to get 80X speedup from 100 processors? Assume either 1 processor or 100 fully used 80 = 1 / [(FracX/100 + (1 FracX)] 0.8*FracX + 80*(1 FracX) = 80 79.2*FracX = 1 FracX = (80 1)/79.2 = 0.9975 • Only 0.25% sequential!
13 13 Topics
• Introduction • SMP Context • Performance: Amdahl’s Law • SMP System structure • Processor core • Memory System • Chip set • South Bridge – I/O • Performance Issues • Summary – Material for the Test
14 14 SMP Node Diagram
MP MP MP MP Legend : L1 L1 L1 L1 MP : MicroProcessor L2 L2 L2 L2 L1,L2,L3 : Caches L3 L3 M1.. : Memory Banks S : Storage NIC : Network Interface Card
M1 M1 Mn 1
S PCI e Controller JTAG Ethernet Peripherals S USB NIC NIC
15 15 SMP System Examples
Vendor & Processor Number Cores Memory Chipset PCI slots name of cores per proc. IBM eServer IBM Power5 64 2 2 TB Proprietary GX+, ≤240 PCI X p5 595 1.9 GHz RIO 2 (20 standard) Microway AMD Opteron 16 2 128 GB Nvidia nForce Pro 6 PCIe QuadPuter 8 2.6 Ghz 2200+2050 Ion M40 Intel Itanium 2 8 2 128 GB Hitachi CF 3e 4 PCIe 1.6 GHz 2 PCI X Intel Server Intel Itanium 2 8 2 64 GB Intel E8870 8 PCI X System 1.6 GHz SR870BN4 HP Proliant Intel Xeon 7040 8 2 64 GB Intel 8500 4 PCIe ML570 G3 3 GHz 6 PCI X Dell PowerEdge Intel Xeon 5300 8 4 32 GB Intel 5000X 3 PCIe 2950 2.66 GHz
16 16 Sample SMP Systems
DELL PowerEdge
HP Proliant
Intel Server System
Microway Quadputer
IBM p5 595 17 17 HyperTransport based SMP System
Source: http://www.devx.com/amd/Article/17437 18 18 Comparison of Opteron and Xeon SMP Systems
Source: http://www.devx.com/amd/Article/17437 19 19 Multi Chip Module (MCM) Component of IBM Power5 Node
20 20 Major Elements of an SMP Node
• Processor chip • DRAM main memory cards • Motherboard chip set • On board memory network – North bridge • On board I/O network – South bridge • PCI industry standard interfaces – PCI, PCI X, PCI express • System Area Network controllers – e.g. Ethernet, Myrinet, Infiniband, Quadrics, Federation Switch • System Management network – Usually Ethernet – JTAG for low level maintenance • Internal disk and disk controller • Peripheral interfaces 21 21 Topics
• Introduction • SMP Context • Performance: Amdahl’s Law • SMP System structure • Processor core • Memory System • Chip set • South Bridge – I/O • Performance Issues • Summary – Material for the Test
22 22 Itanium™ Processor Silicon (Copyright: Intel at Hotchips ’00)
IA-32 FPU Control
IA-64 Control TLB
Integer Units Cache Instr. Fetch & Decode Cache Bus
Core Processor Die 4 x 1MB L3 cache
23 23 Multicore Microprocessor Component Elements
• Multiple processor cores – One or more processors • L1 caches – Instruction cache – Data cache • L2 cache – Joint instruction/data cache – Dedicated to individual core processor • L3 cache – Shared among multiple cores – Often off die but in same package • Memory interface – Address translation and management (sometimes) – North bridge • I/O interface – South bridge 24 24 Comparison of Current Microprocessors
Processor Clock Caches ILP Cores Process & Power Linpack rate (per core) (each core) per die size TPP chip (one core) AMD Opteron 2.6 GHz L1I: 64KB 2 FPops/cycle 2 90nm, 95W 3.89 L1D: 64KB 3 Iops/cycle 220mm 2 Gflops L2: 1MB 2* LS/cycle IBM Power5+ 2.2 GHz L1I: 64KB 4 FPops/cycle 2 90nm, 180W (est.) 8.33 L1D: 32KB 2 Iops/cycle 243mm 2 Gflops L2: 1.875MB 2 LS/cycle L3: 18MB Intel 1.6 GHz L1I: 16KB 4 FPops/cycle 2 90nm, 104W 5.95 Itanium 2 L1D: 16KB 4 Iops/cycle 596mm 2 Gflops (9000 series) L2I: 1MB 2 LS/cycle L2D: 256KB L3: 3MB or more Intel Xeon 3 GHz L1I: 32KB 4 Fpops/cycle 2 65nm, 80W 6.54 Woodcrest L1D: 32KB 3 Iops/cycle 144mm 2 Gflops L2: 2MB 1L+1S/cycle
25 25 Processor Core Micro Architecture
• Execution Pipeline – Stages of functionality to process issued instructions – Hazards are conflicts with continued execution – Forwarding supports closely associated operations exhibiting precedence constraints • Out of Order Execution – Uses reservation stations – hides some core latencies and provide fine grain asynchronous operation supporting concurrency • Branch Prediction – Permits computation to proceed at a conditional branch point prior to resolving predicate value – Overlaps follow on computation with predicate resolution – Requires roll back or equivalent to correct false guesses – Sometimes follows both paths, and several deep 26 26 Topics
• Introduction • SMP Context • Performance: Amdahl’s Law • SMP System structure • Processor core • Memory System • Chip set • South Bridge – I/O • Performance Issues • Summary – Material for the Test
27 27 Recap: Who Cares About the Memory Hierarchy?
Processor-DRAM Memory Gap (latency)
Proc 1000 CPU 60%/yr. “Moore’s Law” (2X/1.5yr) 100 Processor-Memory Performance Gap: (grows 50% / year) 10 DRAM
Performance DRAM 9%/yr. 1 (2X/10 yrs) 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1980 1981 1982 1983 1984 1985 1986 1999 2000
Time Copyright 2001, UCB, David Patterson28 28 What is a cache?
• Small, fast storage used to improve average access time to slow memory. • Exploits spacial and temporal locality • In computer architecture, almost everything is a cache! – Registers a cache on variables – First level cache a cache on second level cache – Second level cache a cache on memory – Memory a cache on disk (virtual memory) – TLB a cache on page table – Branch prediction a cache on prediction information?
Proc/Regs L1 Cache Bigger L2 Cache Faster Memory
Disk, Tape, etc. Copyright 2001, UCB, David Patterson29 29 Levels of the Memory Hierarchy Capacity Upper Level Access Time Staging Cost Xfer Unit faster CPU Registers 100s Bytes Registers <1s ns Instr. Operands prog./compiler 1-8 bytes Cache 10s-100s K Bytes 1-10 ns Cache $10/ MByte cache cntl Blocks 8-128 bytes Main Memory M Bytes 100ns- 300ns Memory $1/ MByte OS Pages 512-4K bytes Disk 10s G Bytes, 10 ms (10,000,000 ns) Disk $0.0031/ MByte user/operator Files Mbytes Larger Tape infinite Tape Lower Level sec-min $0.0014/ MByte Copyright 2001, UCB, David Patterson30 30 Cache performance • Miss oriented Approach to Memory Access:
MemAccess CPUtime === IC ××× CPI +++ ××× MissRate ××× MissPenalty ××× CycleTime Execution Inst MemMisses CPUtime === IC ××× CPI +++ ××× MissPenalty××× CycleTime Execution Inst
– CPI Execution includes ALU and Memory instructions
• Separating out Memory component entirely – AMAT = Average Memory Access Time
– CPI ALUOps does not include memory instructions AluOps MemAccess CPUtime === IC ××× ××× CPI +++ ××× AMAT ××× CycleTime Inst AluOps Inst AMAT === HitTime +++ MissRate ××× MissPenalty
=== (((HitTime Inst +++ MissRateInst ××× MissPenaltyInst )))+++ Copyright 2001, UCB, David Patterson31 31 (((HitTime Data +++ MissRateData ××× MissPenaltyData ))) Memory Hierarchy: Terminology • Hit : data appears in some block in the upper level (example: Block X) – Hit Rate : the fraction of memory access found in the upper level – Hit Time : Time to access the upper level which consists of RAM access time + Time to determine hit/miss • Miss : data needs to be retrieve from a block in the lower level (Block Y) – Miss Rate = 1 (Hit Rate) – Miss Penalty : Time to replace a block in the upper level + Time to deliver the block the processor • Hit Time << Miss Penalty (500 instructions on 21264!)
Lower Level To Processor Upper Level Memory Memory Blk X From Processor Blk Y Copyright 2001, UCB, David Patterson32 32 Cache Measures • Hit rate : fraction found in that level – So high that usually talk about Miss rate – Miss rate fallacy: as MIPS to CPU performance, miss rate to average memory access time in memory • Average memory access time = Hit time + Miss rate x Miss penalty (ns or clocks) • Miss penalty : time to replace a block from lower level, including time to replace in CPU – access time : time to lower level = f(latency to lower level) – transfer time : time to transfer block =f(BW between upper & lower levels)
Copyright 2001, UCB, David Patterson33 33 34 34 Topics
• Introduction • SMP Context • Performance: Amdahl’s Law • SMP System structure • Processor core • Memory System • Chip set • South Bridge – I/O • Performance Issues • Summary – Material for the Test
35 35 Motherboard Chipset
• Provides core functionality of motherboard • Embeds low level protocols to facilitate efficient communication between local components of computer system • Controls the flow of data between the CPU, system memory, on board peripheral devices, expansion interfaces and I/O susbsystem • Also responsible for power management features, retention of non volatile configuration data and real time measurement • Typically consists of: – Northbridge (or Memory Controller Hub, MCH), managing traffic between the processor, RAM, GPU, southbridge and optionally PCI Express slots – Southbridge (or I/O Controller Hub, ICH), coordinating slower set of devices, including traditional PCI bus, ISA bus, SMBus, IDE (ATA), DMA and interrupt controllers, real time clock, BIOS memory, ACPI power management, LPC bridge (providing fan control, floppy disk, keyboard, mouse, MIDI interfaces, etc .), and optionally Ethernet, USB, IEEE1394, audio codecs and RAID interface
36 36 Major Chipset Vendors
• Intel – http://developer.intel.com/products/chipsets/index.htm • Via – http://www.via.com.tw/en/products/chipsets • SiS – http://www.sis.com/products/product_000001.htm • AMD/ATI – http://ati.amd.com/products/integrated.html • Nvidia – http://www.nvidia.com/page/mobo.html
37 37 Chipset Features Overview
38 38 Other Common Chipsets
Intel AMD • Desktop chipsets • Desktop chipsets – 975X Express Chipset – AMD 580X CrossFire™ Chipset – 955X Express Chipset – ATI Radeon™ Xpress 200 – 845E Chipset – ATI SB600 Series • Laptop chipsets • Laptop chipsets – Intel® 855GME Chipset – ATI Radeon™ Xpress 1100 Series – Mobile Intel® 945GM Express Chipset – ATI Radeon™ Xpress 200M – Mobile Intel® 945PM Express Chipset
39 39 Motherboard
• Also referred to as mainboard or system board • Provides mechanical and electrical support for pluggable components of a computer system • Constitutes the central circuitry of a computer, distributing power and clock signals to target devices, and implementing communication backplane for data exchanges between them • Defines expansion possibilities of a computer system through slots accommodating special purpose cards, memory modules, processor(s) and I/O ports • Available in many form factors and with various capabilities to match particular system needs, housing capacity and cost 40 40 Motherboard Form Factors
• Refer to standardized motherboard sizes • Most popular form factor used today is ATX, evolved from now obsolete AT (Advanced Technology) format • Examples of other common form factors: – MicroATX, miniaturized version of ATX – WTX, large form factor designated for use in high power workstations/servers featuring multiple processors – Mini ITX, designed for use in thin clients – PC/104 and ETX, used in embedded systems and single board computers – BTX (Balanced Technology Extended), introduced by Intel as a possible successor to ATX
41 41 Motherboard Manufacturers
• Abit • IBM • Albatron • Intel • Aopen • Jetway • ASUS • MSI • Biostar • Shuttle • DFI • Soyo • ECS • SuperMicro • Epox • Tyan • FIC • VIA • Foxconn • Gigabyte
42 42 Populated CPU Socket
Source: http://www.motherboards.org 43 43 DIMM Memory Sockets
Source: http://www.motherboards.org 44 44 Motherboard on Celeritas
45 45 SuperMike Motherboard: Tyan Thunder i7500 (S720)
Source: http://www.tyan.com 46 46 Topics
• Introduction • SMP Context • Performance: Amdahl’s Law • SMP System structure • Processor core • Memory System • Chip set • South Bridge – I/O • Performance Issues • Summary – Material for the Test
47 47 PCI enhanced systems
48 48 http://arstechnica.com/articles/paedia/hardware/pcie.ars/1 PCI express
Lane Clock Throughput Throughput Initial expected uses width speed (duplex, (duplex, bytes) bits) x1 2.5 GHz 5 Gbps 400 MBps Slots, Gigabit Ethernet x2 2.5 GHz 10 Gbps 800 MBps x4 2.5 GHz 20 Gbps 1.6 GBps Slots, 10 Gigabit Ethernet, SCSI, SAS x8 2.5 GHz 40 Gbps 3.2 GBps x16 2.5 GHz 80 Gbps 6.4 GBps Graphics adapters
49 49 http://www.redbooks.ibm.com/abstracts/tips0456.html PCI X
Bus Width Clock Speed Features Bandwidth
PCI X 66 64 Bits 66 MHz Hot Plugging, 3.3 V 533 MB/s
PCI X 64 Bits 133 MHz Hot Plugging, 3.3 V 1.06 GB/s 133 133 MHz PCI X 64 Bits, optional Hot Plugging, 3.3 & 1.5 V, ECC Double Data 2.13 GB/s 266 16 Bits only supported Rate 133 MHz PCI X 64 Bits, optional Hot Plugging, 3.3 & 1.5 V, ECC Quad Data 4.26 GB/s 533 16 Bits only supported Rate
50 50 Bandwidth Comparisons
CONNECTION BITS BYTES PCI 32 bit/33 MHz 1.06666 Gbit/s 133.33 MB/s PCI 64 bit/33 MHz 2.13333 Gbit/s 266.66 MB/s PCI 32 bit/66 MHz 2.13333 Gbit/s 266.66 MB/s PCI 64 bit/66 MHz 4.26666 Gbit/s 533.33 MB/s PCI 64 bit/100 MHz 6.39999 Gbit/s 799.99 MB/s PCI Express (x1 link) [6] 2.5 Gbit/s 250 MB/s PCI Express (x4 link) [6] 10 Gbit/s 1 GB/s PCI Express (x8 link) [6] 20 Gbit/s 2 GB/s PCI Express (x16 40 Gbit/s 4 GB/s link) [6] PCI Express 2.0 (x32 80 Gbit/s 8 GB/s link) [6] PCI X DDR 16 bit 4.26666 Gbit/s 533.33 MB/s PCI X 133 8.53333 Gbit/s 1.06666 GB/s PCI X QDR 16 bit 8.53333 Gbit/s 1.06666 GB/s PCI X DDR 17.066 Gbit/s 2.133 GB/s PCI X QDR 34.133 Gbit/s 4.266 GB/s 51 51 AGP 8x 17.066 Gbit/s 2.133 GB/s HyperTransport : Context
• Northbridge Southbridge device connection facilitates communication over fast processor bus between system memory, graphics adaptor, CPU • Southbridge operates several I/O interfaces, through the Northbridge operating over another proprietary connection • This approach is potentially limited by the emerging bandwidth demands over inadequate I/O buses • HyperTransport is one of the many technologies aimed at improving I/O. • High data rates are achieved by using enhanced, low swing, 1.2 V Low Voltage Differential Signaling (LVDS) that employs fewer pins and wires consequently reducing cost and power requirements. • HyperTransport also helps in communication between multiple AMD Opteron CPUs
52 52
http://www.amd.com/us en/Processors/ComputingSolutions/0,,30_288_13265_13295%5E13340,00.html Hyper Transport (continued)
• Point to point parallel topology uses 2 unidirectional links (one each for upstream and downstream) • HyperTransport technology chunks data into packets to reduce overhead and improve efficiency of transfers. • Each HyperTransport technology link also contains 8 bit data path that allows for insertion of a control packet in the middle of a long data packet, thus reducing latency. • In Summary : “HyperTransport™ technology delivers the raw throughput and low latency necessary for chip-to-chip communication. It increases I/O bandwidth, cuts down the number of different system buses, reduces power consumption, provides a flexible, modular bridge 53 53 architecture, and ensures compatibility with PCI.http://www.amd.com/us en/Processors/ComputingSoluti“ ons /0,,30_288_13265_13295%5E13340,00.html Topics
• Introduction • SMP Context • Performance: Amdahl’s Law • SMP System structure • Processor core • Memory System • Chip set • South Bridge – I/O • Performance Issues • Summary – Material for the Test
54 54 Performance Issues
• Cache behavior – Hit/miss rate – Replacement strategies
• Prefetching • Clock rate • ILP • Branch prediction • Memory – Access time – Bandwidth
55 55 Topics
• Introduction • SMP Context • Performance: Amdahl’s Law • SMP System structure • Processor core • Memory System • Chip set • South Bridge – I/O • Performance Issues • Summary – Material for the Test
56 56 Summary – Material for the Test
• Please make sure that you have addressed all points outlined on slide 5 • Understand content on slide 7 • Understand concepts, equations, problems on slides 11, 12, 13 • Understand content on 21, 24, 26, 29 • Understand concepts on slides 32, 33 • Understand content on slides 36, 55
• Required reading material :
http://arstechnica.com/articles/paedia/hardware/pcie.ars/1
57 57 58 58