CUDA

GPU Computing

K. Cooper1

1Department of Mathematics Washington State University

2020 CUDA Review of Parallel Paradigms

MIMD Computing Multiple Instruction– Multiple Data –Several separate program streams, each executing possibly different sets of instructions –Each instruction stream operates on different data – each instruction stream may only have access to a fragment of data – MPI – Python parallel package CUDA Review of Parallel Paradigms

SIMD Computing Single Instruction– Multiple Data –Only one program stream, though that may launch multiple threads –The instruction stream may be applied simultaneously to many different data elements. CUDA Review of Parallel Paradigms

Advantages of MIMD Advantages • Instructions can be wildly different for individual streams • Instructions can be separated – even to different nodes • Memory is distributed – becomes limited only by number of nodes • Nodes can be unsophisticated – cheap Disadvantages • Communication • Really only at best for simulations CUDA Review of Parallel Paradigms

Advantages of SIMD Disadvantages • All computations must happen on single machine – limited memory, processors • Hardware might be very complex, therefore expensive Advantages • All computations happen on a single machine – fast CUDA Review of Parallel Paradigms

SIMT Historically, SIMD computing involved vastly complex CPUs with many ALUs, with complicated switch architectures. This is, in some sense, a description of a modern video card. Ever since SGI, video cards have had small specialized processors designed for arithmetic involved in 3-d projections. Single Instruction - Multiple Thread We start one program – that program can launch many threads to perform small tasks in parallel on a Graphics Processing Unit. CUDA CUDA Computing

NVidia – The company that really drives this is NVidia – Makes video cards for 3-d games – API for programmers to send instructions to card:

– CUDA - Compute Unified Device Architecture

AMD/ATI is playing too, but uses different API CUDA CUDA Computing

Model 1 – Start one program 2 – Write function(s) to handle core of computation in parallel – kernel 3 – Allocate memory for CPU – and also on video card 4 – Copy data from CPU to video card 5 – Run the kernel on the card 6 – Copy data back from card to CPU CUDA CUDA Computing

Automation This has been around long enough that people have developed frameworks to handle much of the complication. Anaconda – Numba • CUDA for Conda. . . • translates your code into CUDA for you • make CUDA code from simple scalar code easily CUDA CUDA Computing

CUDA

1 Must install NVidia drivers for your video card. 2 Must install CUDA libraries – depends on OS 3 We have all this on three public machines, not listed here. 4 nvidia-smi gives info 5 Demo. . . CUDA CUDA Computing

Numba To install on your own machine, note that Numba is available in many repositories now.

pip3 install numba or conda install numba conda install cudatoolkit[=10.1]

CUDA communicates with firmware in card, so it is important to install the right toolkit to work with the card. As of this writing, CUDA 10.2 is most recent version. CUDA CUDA Computing

Example Consider the simplest collection of ODEs:

0 y = −y, y(0) = y0,

where y is a vector.

We can hit this with Euler’s method.

def euler(y0,T,h): nSteps = int(T/h) for i in range(nSteps): y0 = (1.-h)*y0 return y0 CUDA CUDA Computing

Results

Method Time(sec) Scalar 15.0 Vectorized w/ cpu option 27.2 Vectorized w/ parallel option 6.9 Vectorized w/ cuda option 2.3