Deep Learning at Scale

Dr. Nathalie Rauschmayr Applied Scientist at AWS

Data and annotations

Compute and scale

Hyperparameter tuning

Design Experiment

Reproducibility

Image taken from https://shaoanlu.wordpress.com/2017/05/07/vihicle-detection-using-ssd-on-floybhub-udacity-self-driving-car-nano-degree/ IoT and real time

Debuggability

30 25 SOTA 20

ILSVRC classification error error classification ILSVRC 5

0 2011 XRCE 2012 AlexNet 2013 ZF 2014 2015 ResNet 2016 GoogLeNet GoogLeNet-v4 © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark How to tackle Deep Learning Challenges

Vision Speech Language Chatbots & Forecasting Recommendations Contact Centers

A I SERVICES

(App Developers with no REKOGNITION REKOGNITION POLLY TRANSCRIBE TRANSLATE COMPREHEND LEX FORECAST PERSONALIZE knowledge of ML) IMAGE VIDEO

AMAZON SAGEMAKER ML SERVICES

(ML developers and data BUILD | TRAIN | DEPLOY scientists)

Frameworks Interfaces Infrastructure ML FRAMEWORKS & INFRASTRUCTURE EC2 P3 EC2 C5 FPGAs GREENGRASS (ML researchers and academics)

CLOUD DEPLOYMENT

Endpoints

LABEL BUILD & TRAIN TUNE COMPILE

Ground Truth Notebooks & Jobs Automatic Model Neo Tuner EDGE DEPLOYMENT

Greengrass

Automatic Annotations

Raw Data Human Active Learning Training Annotations Model Data

Human Annotations

NEO

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Amazon Elastic Inference Inference amounts to 90% of production costs. Elastic inference helps to reduce those costs up to 75%

Available between Lower inference costs Match capacity to 1 to 32 TFLOPS per demand accelerator

KEY FEATURES

Integrated with Support for TensorFlow, Apache Single and Amazon EC2 and MXNet, and ONNX mixed-precision Amazon SageMaker with PyTorch coming soon operations

A flexible and scalable library for deep learning

Optimized 7 frontend Scalable Debuggable Flexible libraries languages Portable

Choose your most preferred language

Frontend Backend

Nearly linear scaling across hundreds of GPUs

16 x p2.16 xlarge

https://mxnet.incubator.apache.org/versions/master/tutorials/vision/large_scale_classification.html © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark MXNet Toolkits - GluonCV

Quickly produce state of the art results with just a few lines of codes

50+ Pre-trained models, with training scripts, datasets, tutorials

Quickly produce state of the art results with just a few lines of codes

Features (as of 0.3.2)

- Pre-trained models: over 300 word-embedding - 5 language models - Neural Machine Translation (Google NMT, Transformer) - Flexible data pipeline tools and many public datasets. - NLP examples such as sentiment analysis. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark MXNet Toolkits - GluonTS

Quickly produce state of the art results with just a few lines of codes

Amazon SageMaker: https://aws.amazon.com/sagemaker/

MXNet https://mxnet.apache.org/

Gluon https://gluon.mxnet.io/

GluonCV https://gluon-cv.mxnet.io/

GluonNLP https://gluon-nlp.mxnet.io/

Gluon-TS https://gluon-ts.mxnet.io/

Deep learning book http://www.d2l.ai/