SUMMIT B e r l i n Apache MXNet (Incubating) and Gluon: What’s in it for You? Constantin Gonzalez Principal Solutions Architect AWS [email protected] | @zalez SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Our mission at AWS Put machine learning in the hands of every developer SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. The Amazon Machine Learning stack Vision Speech Language Chatbots Forecasting Recommendations NEW NEW NEW A I SERVICES A m a z o n A m a z o n A m a z o n A m a z o n A m a z o n A m a z o n A m a z o n A m a z o n A m a z o n A m a z o n Rekognition Rekognition T e x t r a c t P o l l y T r a n s c r i b e T r a n s l a t e C o m p r e h e n d L e x F o r e c a s t Personalize I m a g e V i d e o N o t e b o o k s S u p e r v i s e d L e a r n i n g NEW T r a i n i n g D e p l o y m e n t A m a z o n ML SERVICES Ground Truth A l g o r i t h m s Unsupervised S a g e M a k e r L e a r n i n g NEW NEW NEW Optimization H o s t i n g ( N e o ) Marketplace Reinforcement L e a r n i n g Frameworks Interfaces Infrastructure NEW NEW ML FRAMEWORKS & INFRASTRUCTURE A m a z o n A m a z o n F P G A s AWS Greengrass A m a z o n A m a z o n E C 2 P 3 NEWE C 2 C 5 E l a s t i c I n f e r e n t i a & P 3 D N I n f e r e n c e SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. ML FRAMEWORKS & INFRASTRUCTURE SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. What you’ll learn about today History & Motivation Being more productive as a developer Getting more training done, in less time Deployment: Easy, efficient and scalable Portability and flexibility: Languages, devices, and frameworks Getting started and Resources SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. History imperative symbolic SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Goals Developer Productivity Inference Training Efficiency Efficiency SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Computational dependency graph 3 2 푢 푡 + x 푧 휆 k x x x y a 1 b 1 SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. MXNet computational dependency graph net = mx.sym.Variable('data') net = mx.sym.FullyConnected(net, name='fc1', num_hidden=64) net = mx.sym.Activation(net, name='relu1', act_type="relu") net = mx.sym.FullyConnected(net, name='fc2', num_hidden=10) net = mx.sym.SoftmaxOutput(net, name='softmax') SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Training import logging logging.getLogger().setLevel(logging.DEBUG) # logging to stdout # create a trainable module on compute context mlp_model = mx.mod.Module(symbol=mlp, context=ctx) mlp_model.fit(train_iter, eval_data=val_iter, optimizer='sgd', optimizer_params={'learning_rate':0.1}, eval_metric='acc', batch_end_callback = mx.callback.Speedometer(batch_size, 100), num_epoch=10) SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. MXNet computational dependency graph net = mx.sym.Variable('data') net = mx.sym.FullyConnected(net, name='fc1', num_hidden=64) net = mx.sym.Activation(net, name='relu1', act_type="relu") net = mx.sym.FullyConnected(net, name='fc2', num_hidden=10) net = mx.sym.SoftmaxOutput(net, name='softmax') SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Multi-language support Java Perl Julia Clojure Python Scala C++ R Frontend Backend C++ SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Simple, Easy-to- Flexible, Imperative Dynamic Graphs High Performance Understand Code Structure SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Network definition in Gluon net = gluon.nn.HybridSequential() with net.name_scope(): net.add(gluon.nn.Dense(units=64, activation='relu')) net.add(gluon.nn.Dense(units=10)) softmax_cross_entropy = gluon.loss.SoftmaxCrossEntropyLoss() net.initialize(mx.init.Xavier(magnitude=2.24), ctx=ctx, force_reinit=True) trainer = gluon.Trainer(net.collect_params(), 'sgd', {'learning_rate': 0.02}) SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Training in Gluon smoothing_constant = .01 for e in range(10): cumulative_loss = 0 for i, (data, label) in enumerate(train_data): data = data.as_in_context(model_ctx).reshape((-1, 784)) label = label.as_in_context(model_ctx) with autograd.record(): output = net(data) loss = softmax_cross_entropy(output, label) loss.backward() trainer.step(data.shape[0]) SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Imperative API Debuggable Flexible Scalable SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Gluon Model Zoo SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. GluonCV: a deep learning toolkit for computer vision 50+ Pre-trained models, with training scripts, datasets, tutorials https://gluon-cv.mxnet.io SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. GluonCV pre-trained models SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. GluonCV example code SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Chick-fil-A keeps waffle fries fresh using MXNet • Track waffle fry freshness • Identify fries that have exceeded hold time • Gluon Computer vision model for object detection and tracking • A team of students with no ML expertise • 12 months from no ML knowledge to completion SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. GluonNLP: a deep learning toolkit for natural language processing • 300+ word embedding pre-trained models • 5 language models • Neural Machine Translation (Google NMT, Transformer) • Flexible data pipeline tools • Public datasets • NLP examples, e.g. sentiment analysis SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Training efficiency – 92% https://mxnet.incubator.apache.org/tutorials/vision/large_scale_classification.html SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. TuSimple uses MXNet for autonomous vehicles • Deep Learning algorithms built with MXNet • Computer vision and driving simulation • MXNet teaches computers to Recognize and track objects • Avoid collisions and prioritize safety • Largest simulation of its kind • Simulated a billion miles of road driving with wide range of variables and driving conditions SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon SageMaker: Build, train, and deploy ML models at scale 1 2 3 Collect and Choose and Set up and Train and Deploy models Scale and manage prepare training optimize your manage Tune ML Models in production the production data ML algorithm environments environment for training SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. ONNX SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Portability with ONNX Model Parameters Hyper Parameters SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Open Neural Network eXchange - Overview Many Frameworks MXNet SNPE Caffe2 Many Platforms ONNX: Common NGraph PyTorch O(n2) Intermediate Representation (IR) Pairs • Open source • Community driven TensorRT TF • Simple CoreML CNTK SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Supported tools SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Supported runtimes SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Supported compilers and visualizers SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. KerasKeras – MXNetMXNet hthttps://github.com/awslabs/kerastps://github.com/awslabs/keras-apacheapache-mxnet SUMMITSUM M I T © 2019,2019, AmazonAmazon WebWeb Services,Services, Inc.Inc. oror itsits affiliates.affiliates. AllAll rightsrights reserved.reserved. Keras – Apache MXNet • Deep Learning for Humans • 2nd most popular Deep Learning framework • Now Keras users can leverage MXNet’s performance SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Keras benchmarks Setup: https://github.com/awslabs/keras-apache-mxnet/tree/master/benchmark Training Inference Instance P3.8x Large, P3.16x Large C5.xLarge, C4.8xLarge Network ResNet50v1 ResNet50v1 Batch size 32 * Num of GPUs 32 Image size 3*256*256 3*256*256 ResNet50-Synthetic data 1200 Keras-MXNet Keras-TensorFlow 1000 GPUs [ Image/sec ] [ Image/sec ] Speed Up 800 1 194 184 600 1.05 400 4 764 393 200 1.94 0 1 GPU 4 GPU 8 GPU 8 1068 261 4.09 Keras-MXNet Keras-TensorFlow SUMMIT © 2019, Amazon Web Services, Inc.