Deep Learning in MATLAB

2 AI, Machine Learning, and Deep Learning

Artificial Machine Deep Learning Intelligence Learning Neural networks with many layers that Any technique Statistical methods learn representations and tasks that enables enable machines to “directly” from data machines to “learn” tasks from data mimic human without explicitly intelligence programming

1950s 1980s 2015

FLOPS Thousand Million Quadrillion 3 What is can Deep Learning do for us? (An example)

4 Example 1: Object recognition using deep learning

5 Object recognition using deep learning

Training Millions of images from 1000 (GPU) different categories

Real-time object recognition using Prediction a webcam connected to a laptop

6 WhatWhat is isMachine Deep Learning? Learning?

7 Machine Learning vs Deep Learning

We specify the nature of the features we want to extract… …and the type of model we want to build. Machine Learning

8 Machine Learning vs Deep Learning

We need only specify the architecture of the model… Deep Learning

9 ▪ Deep learning is a type of machine learning in which a model learns to perform tasks like classification – directly from images, texts, or signals. ▪ Deep learning performs end-to-end learning, and is usually implemented using a neural network architecture. ▪ Deep learning algorithms also scale with data – traditional machine learning saturates.

10 Why is Deep Learning So Popular Now?

AlexNet

Human Accuracy

Source: ILSVRC Top-5 Error on ImageNet 11 Two Approaches for Deep Learning

1. Train a Deep Neural Network from Scratch

2. Fine-tune a pre-trained model (transfer learning)

12 Pains In Deep Learning

Expertise Time to Train Data

13 Example: Vehicle recognition using deep transfer learning

Cars Trucks 5 Category SUVs Classifier Big Trucks Vans

14 Import the Latest Models for Transfer Learning Pretrained Models* ▪ AlexNet Import Models from Frameworks ▪ VGG-16 ▪ Caffe Model Importer ▪ VGG-19 ▪ TensorFlow-Keras Model Importer ▪ GoogLeNet ▪ Onnx - Importer/ Exporter (Coming Soon) ▪ Inception-v3 ▪ ResNet50 AlexNet VGG-16 ResNet-50 ResNet-101 PRETRAINED PRETRAINED PRETRAINED MODEL PRETRAINED MODEL ▪ ResNet-101 MODEL MODEL TensorFlow- Caffe GoogLeNet ▪ Inception-resnet-v2 Inception-v3 IMPORTER PRETRAINED Keras MODELS MODEL IMPORTER ▪ SqueezeNet ▪ MobileNet(coming soon)

* single line of code to access model 15 Detection and localization using deep learning

Regions with Convolutional Neural Network Features (R-CNN) 16 What is semantic segmentation?

17 Localization using deep learning

Original Image ROI detection

Pixel classification

18 Semantic Segmentation Network

Boat Airplane

Other classes

19 Semantic Segmentation Network

20 Semantic Segmentation Demo

CamVid Dataset 1. Segmentation and Recognition Using Structure from Motion Point Clouds, ECCV 2008 2. Semantic Object Classes in Video: A High-Definition Ground Truth Database ,Pattern Recognition Letters 21 Semantic Segmentation

~ Said no engineer, ever.

23 Ground truth Labeling

“How do I label my data?”

New App for Ground Truth Labeling Label pixels and regions for semantic segmentation

Data

24 Attributes and Sublabels NEW in

25 Types of Datasets

Numeric Time Series/ Image Data Text Data Data

ML or LSTM CNN LSTM or CNN

26 Analyzing signal data using deep learning

Signal Classification using LSTMs Speech Recognition using CNNs 27 Deep learning features overview

▪ Classification ▪ Data augmentation ▪ Regression ▪ Hyperparameter tuning ▪ Semantic segmentation – Bayesian optimization ▪ Object detection ▪ Python MATLAB interface ▪ Scalability ▪ LSTM networks – Multiple GPUs – Time series, signals, audio – Cluster or cloud ▪ Custom labeling ▪ Custom network layers – API for ground-truth labeling automation ▪ Import models – Superpixels – Caffe ▪ Data validation – Keras/TensorFlow – Training and testing

28 Prediction Performance: Fast with GPU Coder

Images/Sec Why is GPU Coder so fast? – Analyzes and optimizes network architecture – Invested 15 years in code generation

TensorFlow MATLAB MXNet GPU Coder

Using CUDA v9 AlexNet ResNet-50 VGG-16 and cuDNN v7

29 Overview of deep learning deployment options

“How do I deploy my model?” ▪ Create Desktop Apps GPU Coder

▪ Run Enterprise Solution Introducing: GPU Coder- Convert to ▪ Generate C and C++ Code NVIDIA CUDA code

Deploy / Share ▪ Target GPUs ▪ Generate C and C++ Code 30 GPU Coder Fills a Gap in Our Deep Learning Solution

Training Inference

Access Data Preprocess Select Network Train Deploy

Image Neural GPU Image Acq. PCT Processing Network Coder Computer Vision

31 Deploying to CPUs

Intel MKL-DNN Library

NVIDIA GPU TensorRT & Coder cuDNN Libraries Deep Learning Networks ARM Compute Library

32 MATLAB products for deep learning

Required products Recommended products

▪ Neural Network Toolbox ▪ Statistics and Machine Learning ▪ Parallel Computing Toolbox Toolbox ▪ Image Processing Toolbox ▪ MATLAB Coder ▪ Computer Vision System Toolbox ▪ GPU Coder ▪ Automated Driving System Toolbox

33 Deep learning features overview

▪ Classification ▪ Data augmentation * ▪ Regression * ▪ Hyperparameter tuning * ▪ Semantic segmentation – Bayesian optimization ▪ Object detection * ▪ Python MATLAB interface * ▪ Scalability * ▪ LSTM networks * – Multiple GPUs – Time series, signals, audio – Cluster or cloud ▪ Custom labeling * ▪ Custom network layers * – API for ground-truth labeling automation ▪ Import models * – Superpixels – Caffe ▪ Data validation * – Keras/TensorFlow – Training and testing

* We can cover in more detail outside this presentation 34 Thank you!

Deep Learning Onramp – MATLAB

35 Deep learning in automated driving…

36 Deep Learning Onramp

▪ Get started using deep learning methods to perform image recognition. ▪ Free access for everyone ▪ Interactive exercises and short video demonstrations ▪ Work on real-life image recognition problems ▪ Topics include: – Convolutional neural networks – Working with pre-trained networks – Transfer learning – Evaluating network performance

37 Convolutional Neural Networks (CNN)

Edges Shapes Objects

38 Deep Reinforcement Learning (E.g. Deep Q Network)

AGENT ▪ Policy is a sequence of actions to observations to get maximum reward POLICY

▪ Reinforcement Learning finds the optimal policy maximizing the reward OBSERVATION ACTIONS S ▪ Reinforcement Learning adapts to REWARD changes in environment by improving the policy

▪ No need for explicit model (model-free)

39 Google Deepmind’s Deep Q Learning playing Atari Breakout

40 41