April 4-7, 2016 | Silicon Valley

CUDA ON MOBILE Yogesh Kini, GTC 2016 Typical pipeline ABSTRACT CUDA Interop Unified Memory on Tegra

2 TYPICAL USE CASES

Automobiles: Autonomous Cars Mobile Devices: Consoles, Tablets

Embedded: Drones, Robots, Smart-Surveillance

3 TYPICAL PIPELINE

Graphics/ CameraCamera Graphics Display

ISP/DSPISP/DSP CUDA

Actuators SensorSensor Actuators

CUDA

CAPTURE PROCESS DISPLAY 4/1/2016 4 CUDA OPENGL(ES) INTEROP

5 CUDA–OPENGL(ES)

• Provide access to OpenGL-ES resources in CUDA • Support for EGL • Supported on Android, L4T, Vibrante- Linux, QNX • Implicit synchronization support • Useful for graphics applications and games

4/1/2016 6

EGL IMAGE INTEROP

7 EGL IMAGE

Source for EGL image • GStreamer • OpenGL ES • OpenMAX • Android - GraphicBuffer

Khronos EGL_image_base: https://www.khronos.org/registry/egl/extensions/KHR/EGL_KHR_image_base.txt

4/1/2016 8 EGL IMAGE

cudaArray

EGLimage cudaDevicePointer

cuGraphicsEGLRegisterImage() cuGraphicsResourceGetMappedPointer() cuGraphicsResourceGetMappedArray()

Begin resource Begin resource usage in Other API Usage in CUDA

Other API code CUDA code

End resource End resource Usage in Other API synchronize Usage in CUDA

4/1/2016 9 EGL STREAMS INTEROP

10 EGL STREAMS • Producer-Consumer architecture

• EGL streams spec: https://www.khronos.org/registry/egl/extensions/KHR/EGL_KHR_stream.txt • Implicit Synchronization • Cross Process support • Supports YUV formats cuDNN

ISP CUDA cuBLAS CUDA EGL stream Producer Consumer Visionworks Producer

CUDA

OpenGL EGL stream Consumer

4/1/2016 11 EGL STREAMS CUDA CUDA cuEGLStreamProducerConnect() EGL Stream cuEGLStreamConsumerConnect() Producer Consumer

Frame 1 cuEGLStreamConsumerAcquireFrame(frame) cuEGLStreamProducerPresentFrame(frame) Frame 2 Use Frame in CUDA 3

cuEGLStreamConsumerReleaseFrame(frame) cuEGLStreamProducerReturnFrame(frame) Frame

Frame 4

12 INTEROP SUMMARY

EGL STREAMS EGL IMAGE CUDA-OPENGL

• Producer-Consumer • Easy setup • EGL support

• Implicit- • Works with several EGL • OpenGL-ES support Synchronization client API • Portable across Tegra • Cross-Process support • YUV Planar Image and discrete GPU support • YUV Planar Image support

4/1/2016 13 CUDA UNIFIED MEMORY ON TEGRA

• Helps take advantage of unified DRAM on Tegra

• Easier to program, Unified allocator: cudaMallocManaged TEGRA • Programming model enforced through memory access protection CPU GPU

• Memcpy not needed, migration managed by CUDA driver Memory- DRAM • Saves memory consumption and power

• Attach API will help achieve optimal performance

4/1/2016 14 CUDA MEMORY TYPES

Traditional Zero Copy Unified Memory

malloc() Allocate cudaMallocHost() cudaMallocManaged() cudaMalloc()

CPU use CPU use CPU use CPU use

Migrate cudaMemcpyHtoD() NA cudaMemAttach[Optional]

CUDA kernel Kernel_launch<<<>>>() Kernel_launch<<<>>>() Kernel_launch<<<>>>()

Migrate cudaMemcpyDtoH() NA cudaMemAttach[Optional]

CPU use CPU use CPU use CPU use

15 CUDA MEMORY TYPES

Traditional Zero Copy Managed

• Easy portability from • Cache is bypassed by • Memory access by CPU existing desktop both GPU and CPU and GPU is through programs while accessing these cache. allocations • Faster for some small • Faster for larger allocations • Suitable when memory allocations access is not affected • Suitable for GPU by caching • Suitable when memory intermediate buffers, used on both host and tables, etc GPU

Time taken(ms) by the Matrix Multiply CUDA kernel with different allocation types: TRADITIONAL ZERO COPY MANAGED MEMORY 16KB 0.617 0.544 0.644 1MB 9.723 11.119 7.093 4MB 59.37618 62.232 46.42551 16MB 377.9244 403.2382 344.926

16 April 4-7, 2016 | Silicon Valley

THANK YOU

JOIN THE DEVELOPER PROGRAM AT developer.nvidia.com/join