Turbocharging the AI Pipeline with Python and Anaconda

Stan Seibert Director of Community Innovation @ Anaconda

!1 What is AI? Augmented AI = Artificial Intelligence?

• Create insight by combining: • Data • Software • Domain Expertise

A mature system might end up being (at “ most) 5% machine learning code and (at least) 95% glue code – Google AI Researchers ”

Source: “Machine Learning: the High-Interest Credit Card of Technical Debt”, Google Inc, 2015

• Open Source • Build on cutting-edge innovation in the community • Reproducible Environments • Manage the data science software environment • Team Collaboration • Share and collaborate with fellow data scientists and other stakeholders

!5 Examples: Image Captioning

van Gogh

https://research.googleblog.com/2016/09/show-and-tell-image-captioning-open.html

• Want to model and predict behavior in foreign exchange markets • GPU-trained RNN using previous 20 values to predict next value

http://on-demand.gputechconf.com/gtc/2017/presentation/s7625-daniel-egloff-going-deeper-in-finance.pdf

Picasso van Gogh Monet

http://genekogan.com/works/style-transfer/

(Much) Bigger Training Sets Faster & Specialized Hardware

Monet Open Source Tools Algorithm Research

TensorFlow Keras PyTorch

GPU CPU

× 80

!14 What Else Do Data Scientists Want?

• Data Preparation / ETL • Feature Engineering • Simulation • Scalable Analytics • Fast Visualization

• ...ability to integrate tools together in a complete pipeline

• An open-source, function-at-a-time compiler library for Python • Compiler toolbox for different targets and execution models: • single-threaded CPU, multi-threaded CPU, GPU • regular functions, “universal functions” (array functions), etc • Speedup: 2x (compared to basic NumPy code) to 200x (compared to pure Python) • Combine ease of writing Python with speeds approaching FORTRAN • BSD licensed (including GPU compiler) • Goal is to empower scientists who make tools for themselves and other scientists

Work with NumPy array elements and attributes on the GPU

Special CUDA function for atomic addition

Launch CUDA kernel with 32 blocks of 32 threads each

Simulation

Causes Effects

Machine Learning

Simulation

Cars on a Road Pixels in a Camera

Machine Learning

Numba

Cars on a Road Pixels in a Camera

TensorFlow

Modifying these projects * to share GPU tensors through a common Python interface

* PyTorch support in community PR

ETL/Data Machine Data Data Prep Learning

Database Data Data Visualization

GPU

ETL/Data Machine Data Data Prep Learning

CPU transfer CPU transfer

Database Data Data Visualization

GPU

ETL/Data Machine Data Data Prep Learning

Database Data Data Visualization

GPU

• Rapidly becoming the standard for many kinds of structured data • Can Arrow work on the GPU? • Answer: Yes!

• GPU Dataframes are Arrow-format data structures on the GPU • Designed to be passed between different applications, languages and runtimes: • Example: MapD database -> Python notebook -> XGBoost • RAPIDS dataframe support includes 3 tier library structure (names to be changed soon): • libgdf: C library of GPU dataframe functions • pygdf: Python wrapper around GDF + JIT functionality • dask_gdf: Distributed GPU dataframes

• Scalable execution task graphs of task graphs from single computers to 1000+ node clusters • Orchestrate CPU and GPU tasks on data structures distributed across many nodes

!28 Moving From Tools to Projects

• Every data science project is a software integration problem • Need to bring together many tools, plus their dependencies • Important to be able to record state of environment for future reproducibility • Often latest versions of libraries • Other times want older versions of libraries • Want to be able to easily compare different versions

B • Language independent A Numba • Platform independent NumPy NumPy C • No special privileges required v1.10 v1.11 Pandas cudatoolkit R • No VMs or containers v0.16 9.2 Essentials

• Enables: Python v2.7 Python v3.6 R • Reproducibility • Collaboration conda • Scaling

NEURAL NETWORKS CHAINER KERAS

TENSOR MATH CUPY MXNET TENSORFLOW ...and many others PRIMITIVES MKL-DNN CUBLAS CUDNN NCCL

OS/DRIVERS

MANY-CORE CPU HARDWARE MULTI-CORE CPU GPU (XEON PHI)

NEURAL NETWORKS CHAINER KERAS

TENSOR MATH CUPY MXNET TENSORFLOW

PRIMITIVES MKL-DNN CUBLAS CUDNN NCCL

OS/DRIVERS

MANY-CORE CPU HARDWARE MULTI-CORE CPU GPU (XEON PHI)

conda create -n deeplearn python=3.6 notebook keras tensorflow-gpu conda activate deeplearn jupyter notebook

Force a particular CUDA toolkit: conda create -n deeplearn python=3.6 notebook keras tensorflow-gpu \ cudatoolkit=9.0

TensorFlow Keras My Model

Pandas My App

Tornado

!38 GPU Needs for Data Science Teams

• Centralized access • High end GPUs are most easily managed in the data center • Data scientists connect remotely to train and run their GPU-accelerated models • Resource management • Best practice is to allow only one application to use a GPU at a time. • Need to reserve GPU exclusively for a user when they are running their GPU code • Support for varying compute needs • Many projects don't need a GPU, some need one, a very small number need two or more • Most cost-efficient approach is a heterogenous cluster with mixture of GPU and non-GPU nodes

From One Data Scientist to Thousands From One Machine to Thousands

Anaconda Enterprise 5

• Anaconda is the industry’s trusted distributor for all core Core AI AI technologies Technologies Governance • Simplifies and automates AI governance and reproducibility Cloud Native • Cloud Native: Modern, Approach dynamic, API-oriented, and container-based

AE Worker (CPU) AE Worker (GPU) AE Master (Node 1) AE Services Editor Sessions Object Storage Database Git UI Session 2 (TensorFlow)

Persistent Disk kube-apiserver Deploy Storage etcd Sessions docker Editor Sessions Session 4 (PyTorch) kubelet Session 1 (Py)

Session 3 (Py) Deployments

Deployments AE Kubernetes Masters (Nodes 2 and 3) Deployment 2 (TF) Deployment 1 (Py) kube-apiserver Deployment 3 (Py) etcd docker docker docker kubelet kubelet kubelet

Inference in batches, or one at a time?

Do you need GPU- GPU only 30% faster w/ batch size = 1 GPU 10x faster when levels of throughput batch size >= 32 in production?

• GPU-accelerated AI with Anaconda brings together many parts: • Open source technology: Deep learning, GPU dataframes, Numba for custom algorithms and simulation • Reproducible environments: Conda package manager, Anaconda Distribution • Team collaboration: Resource management and deployment with Anaconda Enterprise • Learn more at: https://www.anaconda.com/