AI Techniques in MATLAB for Signal, Time-Series, and Text Data

Matthias Sommer

2 The Use of Deep Learning is Growing Across Industries Aerospace, Defense and Communications Consumer Electronics and Digital Health

Communications devices, Multi-standard communications security receivers, drone recognition Voice assistants Digital health

Automotive Industrial Automation

Sensor processing, Voice control enabled automated driving Condition monitoring Predictive maintenance Infotainment

3 Application Examples Using MATLAB – Audio and Speech

Speech Command Recognition Music Genre Classification (a.k.a. "Keyword Spotting") https://www.mathworks.com/help/audio/examples/music- https://www.mathworks.com/help/deeplearning/e genre-classification-using-wavelet-time-scattering.html xamples/deep-learning-speech-recognition.html 4 Application Examples Using MATLAB – Industrial and Radar sensors

ECG Signal Classification Modulation Classification https://www.mathworks.com/help/signal/examples/classify- ecg-signals-using-long-short-term-memory-networks.html http://www.mathworks.com/help/comm/examples/ modulation-classification-with-deep-learning.html 5 Agenda

▪ Deep Learning – Basic ideas

▪ Deep Learning Model Development for Signals, Time Series, and Text

▪ Conclusions

6 What is Deep Learning?

Deep learning is a type of machine learning in which a model learns from examples.

7 Common Network Architectures - Signal Processing

Convolutional Neural Networks (CNN)

Time-Frequency Transformation Long Short Term Memory (LSTM) Networks

Feature Engineering

8 Deep Learning Workflow

CREATE AND ACCESS PREPROCESS AND DEVELOP PREDICTIVE ACCELERATE AND DATASETS TRANSFORM DATA MODELS DEPLOY

Data sources Pre-Processing Import Reference Models/ Desktop Apps Design from scratch

Simulation and Transformation Hardware-Accelerated Enterprise Scale Systems augmentation Training

Analyze and tune Data Labeling Feature extraction Embedded Devices and hyperparameters Hardware

9 Deep Learning Workflow Challenges – Signals and Time Series

Limited Data Data Labeling Availabiliy

Deep learning models only as good as training data

Deployment and Domain-specific data Limited reference Scaling to various processing desirable research platforms

10 Agenda

▪ Deep Learning – Basic ideas

▪ Deep Learning Model Development for Signals, Time Series, and Text – Data – Processing and transformation – Model design and optimization – Acceleration, prototyping, and deployment

▪ Conclusions

11 Agenda

▪ Deep Learning – Basic ideas

▪ Deep Learning Model Development for Signals, Time Series, and Text – Data – Processing and transformation – Model design and optimization – Acceleration, prototyping, and deployment

▪ Conclusions

12 Current Investments – Models vs. Data

Research Industry

Models and Algorithms

Datasets

From "Troubleshooting deep neural networks" (Josh Tobin et al., Jan 2019) 13 What does a large dataset look like?audioDatastore How to navigate, index, read (aka "ingest") a largeimageDatastore dataset? fileDatastore Custom Datastores How to...

▪ Build a list of all data and labels?

▪ Review basic statistics about available data?

▪ Select data subsets without nested for loops, dir, ls, what, ... aplenty?

▪ Jointly read data and labels?

▪ Automatically distribute computations?

14 Use appropriate tools to help you label signals

▪ Programmatically… ▪ … or via Apps

15 What if available data isn't enough?

PitchOriginal shift

Augmented Dataset

N times as Original much data Dataset

Data augmentation allows building more complex and more robust models

16 Simulation is key if recording and labelling real-world data is impractical or unreasonable – Communications Signals

17 Simulation is key if recording and labelling real-world data is impractical or unreasonable – Radar Signals

Radar Target Simulation Micro-Doppler Analysis

18 Agenda

▪ Deep Learning – Basic ideas

▪ Deep Learning Model Development for Signals, Time Series, and Text – Data – Processing and transformation – Model design and optimization – Acceleration, prototyping, and deployment

▪ Conclusions

19 Common types of network architectures used in signal processing and text analytics applications

Convolutional Neural Networks (CNN)

Time-Frequency Transformation Long Short Term Memory (LSTM) Networks

Feature Engineering

20 Time-Frequency Transformations

Reframe To frequency

(e.g. Buffer) ... (e.g. FFT)

frequency (bin frequency #) Time (samples) Time

Time (samples) Time (frame #) Time (frame #)

Easiest to More compact for Best resolution, for understand and speech & audio non-periodic Better resolution at implement applications signals low frequencies

Basic spectrogram Perceptually-spaced (e.g. Constant Q transform Mel, Bark) Spectrogram Wavelet scalogram 21 Extracting Features from Signals: Application-Agnostic Examples

BW measurements Spectral statistics Harmonic analysis Octave spectrum

Frequency domain Time domain

22 Automated Feature Extraction: Wavelet Scattering

Deep Feature Extraction

Averaging

Nonlinearity

Classifier

Nonlinearity

Wavelet Convolution Wavelet

Wavelet Convolution Wavelet Wavelet Convolution Wavelet

Fixed Fixed Fixed

▪ Can relieve requirements on amount of data and model complexity – Featured in leader-boards a number of research competitions

▪ Framework for extracting features [1]

[1] Joan Bruna, and Stephane Mallat, P. 2013. Invariant Scattering Convolution Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 35, No. 8, pp. 1872-1886.

24 Agenda

▪ Deep Learning – Basic ideas

▪ Deep Learning Model Development for Signals, Time Series, and Text – Data – Processing and transformation – Model design and optimization – Acceleration, prototyping, and deployment

▪ Conclusions

25 Developing Deep Learning Models

Design Network

Accelerate Training MATLAB as a Train container on NGC

Design Optimize Model Exchange Pre-trained Networks

Bayesian Hyperparameter Optimization

26 Exchange Models With Deep Learning Frameworks

TensorFlow- PyTorch TensorFlow Keras

MXNet ONNX MATLAB

Core ML Chainer Caffe

ONNX = Open Neural Network Exchange Format

27 Exchange Models With Deep Learning Frameworks

TensorFlow- PyTorch TensorFlow Deployment Keras

Keras importer

MXNet Augmentation ONNX MATLAB and Optimization

Caffe importer

Core ML Chainer Caffe Visualization and Debugging

ONNX = Open Neural Network Exchange Format

28 Agenda

▪ Deep Learning – Basic ideas

▪ Deep Learning Model Development for Signals, Time Series, and Text – Data – Processing and transformation – Model design and optimization – Acceleration, prototyping, and deployment

▪ Conclusions

29 Deployment and Scaling for A.I.

MATLAB

Embedded Devices Enterprise Systems

30 Deploying Deep Learning Models for Inference

Intel MKL-DNN Library

Application logic NVIDIA Code TensorRT & Generation cuDNN Libraries Auto-generated Code (C/C++/CUDA) ARM Compute Library

31 Enterprise Deployment

Deployment to the cloud with MATLAB Compiler and MATLAB Production Server

32 Agenda

▪ Deep Learning – Basic ideas

▪ Deep Learning Model Development for Signals, Time Series, and Text – Data – Processing and transformation – Model design and optimization – Acceleration, prototyping, and deployment

▪ Conclusions

33 Deep Learning Workflow Challenges – Signals and Time Series

Data-labeling Apps AugmentationLimited data and Data Labeling Deployment and and Examples simulationavailability algorithms Scaling to various platforms

Deep learning models only as good as training data

Domain-specific data Limited reference Collaboration processingApplication desirable-specific algorithmsresearch and tools in the AI ecosystem

34 Demo Station

Find out more: Deep Learning from Idea to Embedded Hardware

Matthias Sommer Technology Showcase

35 Related Trainings

▪ Time series analysis – Signal Preprocessing and Feature Extraction for Data Analytics with MATLAB

▪ Deep-learning (including non-vision applications) – Deep Learning with MATLAB

https://nl.mathworks.com/services/training.html