USING CONTAINERS for GPU APPLICATIONS Jonathan Calmels, Felix Abecassis, 05/08/2017 OUR TEAM Core Container Technologies

USING CONTAINERS FOR GPU APPLICATIONS Jonathan Calmels, Felix Abecassis, 05/08/2017 OUR TEAM Core Container Technologies

Enable GPUs in the container ecosystem: • Monitoring • Orchestration • Images • Runtime • OS

2 CHALLENGES A Typical Cluster

Ubuntu 14.04 CentOS 7 Ubuntu 16.04 Drivers 367 Drivers 361 Drivers 375 4x Maxwell 4x Kepler 8x Pascal

CUDA 7.5 CUDA 7.0 cuDNN 4 CUDA 7.5 CUDA 8.0 cuDNN 6

Patches

3 CONTAINERS To the rescue

Portable and reproducible builds Ease of deployment Isolation of resources Run across heterogeneous CUDA toolkit environments (sharing the host driver) Bare Metal Performance Facilitate collaboration

4 VIRTUAL MACHINES VS CONTAINERS Not so similar

5 6 7 NVIDIA-DOCKER github.com/NVIDIA/nvidia-docker

8 DOCKERHUB IMAGES Multiple flavors

Ubuntu Ubuntu 14.04 16.04

CUDA 8.0 CUDA 8.0 runtime runtime

cuDNN v5 CUDA 8.0 cuDNN v5 CUDA 8.0 runtime devel runtime devel

NVIDIA/Caffe cuDNN v5 cuDNN v5 cuDNN v6 0.15.13 devel devel devel

DIGITS CNTK TensorFlow PyTorch 5.0

9 CONTAINERS ON NVIDIA DGX-1 NVIDIA’s Deep Learning System

10 NVIDIA-DOCKER 1.0 Internals

GPU information

http cuda+nvml nvidia driver nvidia-docker nvidia-docker-plugin

http+unix docker dockerd http+unix

container process

$ NV_GPU=0 nvidia-docker run -ti nvidia/cuda 11 LIMITS OF NVIDIA-DOCKER 1.0 Limited scope

Only for Docker CLI Docker plugins are difficult to manage Not extensible (OpenGL, Vulkan, InfiniBand, KVM, etc.) Challenging to support new architectures (Power, ARM) Difficult to integrate into the container ecosystem

12 DOCKER COMPONENTS

13 WHAT’S A CONTAINER Kernel building blocks

Init system Netlink Namespaces Capabilities Cgroups Seccomp BPF UnionFS LSM KVM Netfilter ...

14 LIBNVIDIA-CONTAINER github.com/NVIDIA/libnvidia-container

Integrates with the container internals Agnostic of the container runtime Drop-in GPU support for runtime developers Better stability, follows driver releases Brings features seamlessly (Graphics, Display, Exclusive mode, VM, etc.)

15 NVIDIA-DOCKER 2.0 Internals

docker nvidia-runc nvidia-oci-runtime

http(s)(+unix)

container process libnvidia-container dockerd

cuda+nvml grpc+unix nvidia driver docker-containerd + shim

$ docker run -ti -e NVIDIA_VISIBLE_DEVICES=0 --runtime=nvidia nvidia/cuda 16 DEMO!

17 CONTAINER FUTURE Enable GPUs everywhere nvidia-docker 2.0 release Multi-arch support (Power, ARM) Support other container runtimes (LXC/LXD, Rkt) Additional Docker images Additional features (OpenGL, Vulkan, InfiniBand, KVM, etc.) Support for GPU monitoring (cAdvisor)

18 QUESTIONS?