John Donnelly

Yann LeCun Geoff Hinton, MSR & University Handwriting recognition (ZIP codes) Yoshua Bengio of Toronto Backpropagation algorithm Deep Belief Networks Speech Recognition

1965 1989 1993 2009 2012

Alexey Ivakhnenko Jürgen Schmidhuber Andrew Ng Supervised deep feedforward Recurrent long and Deep Learning multilayer perceptrons short term memories renaissance with cats

FUEL + SPARK + ENGINE MASSIVE + NEW + COMPUTER DATA MATH HORSEPOWER

Deep Learning Demystified (NVIDIA), GTC 2017

http://nikhilbuduma.com/2014/12/29/deep-learning-in-a-nutshell/ http://www.theprojectspot.com/tutorial-post/introduction-to-artificial-neural-networks-part1/7

160 ResNet152, 152 layers 140

120

100

80

60

40

GoogLeNet, 22 layers 20 (VGG @ 19/7.3) AlexNet, 8 layers 8 layers 0 shallow shallow 2010 2011 2012 2013 2014 2015

layers error https://xkcd.com/1425/ https://leonardoaraujosantos.gitbooks.io/artificial-inteligence/content/object_localization_and_detection.html

* Feature visualization images from “Visualizing and Understanding Convolutional Neural Networks”, Zeiler and Fergus, ECCV 2014. -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 1 -1 1 1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 x -1 -1 -1 1 -1 -1 -1 1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1

http://brohrer.github.io/how_convolutional_neural_networks_work.html 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

1 -1 -1 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 -1 1 -1 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11

-1 -1 1 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

-1 -1 -1 -1 -1 -1 -1 -1 -1 0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33

-1 1 -1 -1 -1 -1 -1 1 -1 -0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55 -1 -1 1 -1 -1 -1 1 -1 -1 1 -1 1 0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -0.11 0.33 -0.77 1.00 -0.77 0.33 -0.11 -1 -1 -1 1 -1 1 -1 -1 -1 0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11 -1 -1 1 -1 -1 -1 1 -1 -1 1 -1 1 -0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

-1 -1 1 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 -1 1 -1 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

1 -1 -1 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

http://brohrer.github.io/how_convolutional_neural_networks_work.html 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 1.00 0.33 0.55 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 0.33 1.00 0.33 0.55 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 0.55 0.33 1.00 0.11

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.33 0.55 0.11 0.77 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33 0.55 0.33 0.55 0.33 -0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55

0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11 0.33 1.00 0.55 0.11 -0.11 0.33 -0.77 1.00 -0.77 0.33 -0.11

0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11 0.55 0.55 0.55 0.11

-0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55 0.33 0.11 0.11 0.33 0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77 0.33 0.55 1.00 0.77 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 0.55 0.55 1.00 0.33 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 1.00 1.00 0.11 0.55

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.77 0.33 0.55 0.33 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

http://brohrer.github.io/how_convolutional_neural_networks_work.html 0.77 0 0.11 0.33 0.55 0 0.33 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

0 1.00 0 0.33 0 0.11 0 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.11 0 1.00 0 0.11 0 0.55 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.33 0.33 0 0.55 0 0.33 0.33

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 0.55 0 0.11 0 1.00 0 0.11

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0 0.11 0 0.33 0 1.00 0

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77 0.33 0 0.55 0.33 0.11 0 0.77

0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33 0.33 0 0.11 0 0.11 0 0.33

-0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55 0 0.55 0 0.33 0 0.55 0

0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11 0.11 0 0.55 0 0.55 0 0.11

-0.11 0.33 -0.77 1.00 -0.77 0.33 -0.11 0 0.33 0 1.00 0 0.33 0

0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11 0.11 0 0.55 0 0.55 0 0.11

-0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55 0 0.55 0 0.33 0 0.55 0

0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33 0.33 0 0.11 0 0.11 0 0.33

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77 0.33 0 0.55 0.33 0.11 0 0.77

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0 0.11 0 0.33 0 1.00 0

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 0.55 0 0.11 0 1.00 0 0.11

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.33 0.33 0 0.55 0 0.33 0.33

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 0.11 0 1.00 0 0.11 0 0.55

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0 1.00 0 0.33 0 0.11 0

0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 0.77 0 0.11 0.33 0.55 0 0.33

http://brohrer.github.io/how_convolutional_neural_networks_work.html 1.00 0.55

-1 -1 -1 -1 -1 -1 -1 -1 -1 0.55 1.00 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 1.00 0.55 -1 -1 -1 -1 1 -1 -1 -1 -1 0.55 0.55 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 0.55 1.00 -1 -1 -1 -1 -1 -1 -1 -1 -1 1.00 0.55

http://brohrer.github.io/how_convolutional_neural_networks_work.html -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 X -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 O

http://brohrer.github.io/how_convolutional_neural_networks_work.html

https://aka.ms/leak_detection

train_test_split train_keras score_keras

Selective Search

Rich feature hierarchies for accurate object detection and semantic segmentation, https://arxiv.org/abs/1311.2524

Tutorial on CNTK https://www.youtube.com/watch?v=Khuj4ASldmU

https://arxiv.org/pdf/1609.04802.pdf its own zoo Quora session Credit: Bruno Gavranović https://www.microsoft.com/developerblog/2017/06/12/learning-image-image-translation-cyclegans/ https://www.cs.cmu.edu/~sbhagava/papers/face-rec-ccs16.pdf AI’s White Guy Problem FaceApp “Whitewashing” https://medium.com/@ricardo.guerrero/deep-learning-frameworks-a-review-before-finishing-2016-5b3ab4010b06 https://github.com/zer0n/deepframeworks

GitHub Azure Notebooks arXiv arXiv Sanity Preserver

The Neural Network Zoo Brandon Rohrer‘s Blog Machine Learning is Fun Brief History of Image segmentation Brief History of Neural Nets and Deep Learning

Deep Learning Toolkit (DSVM)