High Performance Computing Dav Id B

High Performance Computing Dav id B. Kir k, Chie f Sci enti s t, NVIDIA

• Invented the Graphics Processing Unit (GPU) • Pioneered ppgrogrammable shadin g • Over 2000 patents*

1995 1999 2002 2003 2004 2005 2006-2007 NV1 GeForce 256 GeForce4 GeForce FX GeForce 6 GeForce 7 GeForce 8 1 Million 22 Million 63 Million 130 Million 222 Million 302 Million 754 Million Transistors Transistors Transistors Transistors Transistors Transistors Transistors 2008 GeForce GTX 200 1.4 Billion Transistors

• Real system • NVSG-driven animation and interaction • Programmable shading • Modeled in Maya, imported through COLLADA • Fully ray traced

2 million polygons Bump-mapping Movable light source 5 bounce reflection/refraction Adaptive antialiasing

• Parallel portions of an application are executed on the GPU as kernels – One kernel is executed at a time – Many threads execute each kernel

• Differences between CUDA and CPU threads – CUDA threads are extremely lightweight • Very little creation overhead • Instant switching – CU DA uses 1000s of thhdreads to achieve effiffiiciency • Multi-core CPUs can use only a few

void saxpy_serial(int n, float a, float *x, float *y) { for (int i = 0; i < n; ++i) y[i] = a*x[i] + y[i] ; } Standard C Code // Invoke serial SAXPY kernel saxpy_serial(n, 2.0, x, y);

__global__ void saxpy_parallel(int n, float a, float *x, float *y) { int i = blockIdx.x*blockDim.x + threadIdx.x; if (i < n) y[i] = a*x[i] + y[i]; Parallel C Code } // Invoke parallel SAXPY kernel with 256 threads/block int nbl ock s = (n + 255) / 256; saxpy_parallel<<>>(n, 2.0, x, y);

• Standard high level language support – C, soon C++ and Fortran – Standard and domain specific libraries • Hardware Thread Management – No switc hing over hea d – Hide instruction and memory latency • Shared memory – User-managed data cache – Thread communication / cooperation within blocks • Runtime and tool support – Loader, Memory Allocation

Heterogeneous Computing

Oil & Gas Finance Medical Biophysics Numerics Audio Video Imaging

• Duke • Kent State • Santa Clara • Erlangen • Kyoto • Stanford • ETH Zurich • Lund • Stuttgart • Georgia Tech • Maryland • Suny • Grove City College • McGill • Tokyo • Harvard • MIT • TU-Vienna • IIIT • North Carolina - Chapel Hill • USC • IIT • North Carolina State • Utah • Illinois Urbana-Champpgaign • Northeastern • Virginia • INRIA • Oregon State • Washington • Iowa • Pennsylvania • Waterloo • ITESM • Polimi • Western Australia • Johns Hopkins • Purdue • Williams College • Wisconsin

146X 36X 19X 17X 100X

Interactive Ionic placement for Transcoding HD video Simulation in Matlab Astrophysics NN--bodybody visualization of molecular dynamics stream to H.264 using .mex.mexfile CUDA simulation volumetric white simulation on GPU function matter connectivity

149X 47X 20X 24X 30X

Financial simulation of GLAME@labGLAME@lab:: An MM-- Ultrasound medical Highly optimized Cmatch exact string LIBOR model with script API for linear imaging for cancer object oriented matching to find swaptions Algebra operations on diagnostics molecular dynamics similar proteins and GPU gene sequences

800 746 186x Faster Than CPU 600

400 220 200 100 4 0 CPU PS3 RdRadeon GFGeForce HD 4870 GTX 280

• 2-3X faster is “just faster” • Do a little more, wait a little less • Doesn’t change how you work • 5-10x faster is “significant” • Worth upgrading • Worth re-writing (parts of) the application • 100x+ faster is “fundamentally different” • Worth considering a new platform • Worth re-architecting the application • Makes new applications possible • Drives “time to discovery” and creates fundamental changes in Science

Thread Processor Cluster (()TPC)

Thread Processor Array (TPA)

Thread Processor

Die Photo of Tesla T10

1 Teraflop

500 Gigaflops 4 Gigabytes 1.5 Gigabytes

Tesla 8 Tesla 10 Tesla 8 Tesla 10 Double Precision

Finance Science Design

Rounding modes for FADD and FMUL All 4 IEEE, round to nearest, zero, inf, -inf

Denormal handling Full speed

NaN support Yes

Overflow and Infinity support Yes

Flags No

FMA Yes

Square root Software with low-latency FMA-based convergence

Division Software with low-latency FMA-based convergence

Reciprocal estimate accuracy 24 bit

Reciprocal sqrt estimate accuracy 23 bit

log2(x) and 2^x estimates accuracy 23 bit

DNA Sequence Dynamics of Black Video Application AlignmentG80 holes

T10P Reverse Time Migration

Cholesky Factorization LB Flow Lighting Ray Tracing T10P

G80

Traditional Data Center Cluster

1000’s of cores QdQuad-core 1000’s of servers CPU

8 cores per server More Servers To Get More Performance

• Hess 10,000’s processors per cluster • NCSA / UIUC • JFCOM • SAIC • University of North Carolina • Max Plank Institute • Rice University • University of Maryland • GusGus • Eotvas University • University of Wuppertal • IPE/Chinese Academy of Sciences • Cell phone manufacturers

1928 processors 1928 processors

CPU 1U Server 4 CPU cores 4 GPUs: 960 cores Tesla 1U System 0.07 Teraflop 4 Teraflops $ 2000 $ 8000 400 W 800 W 1429 CPU servers 25 CPU servers 25 Tesla systems $3.1M$ 3.1 M $0.31M$ 0.31 M 10x lower cost 571 KW 27 KW 21x lower power

4 Teraflops1

800 watts2

1 single precision 2 typical power

957 Giggpaflops1

160 watts2

1 single precision 2 typical power

System CUDA Compiler CUDA Tools 1U PCI‐E Switch C FortranMulti‐core Debugger Profiler

4 cores

IdIndust ry StdStandar d Librar ies

CUDA Compiler Standard C Fortran DbDebugger Pro filer

Multi-core

C CUDA Application

NVCC NVCC --multicore

Many-core Multi-core PTX code CPU C code

PTX to Target gcc and Compiler MSVC

Many-core Multi-core