John Donnelly Yann LeCun Geoff Hinton, MSR & University Handwriting recognition (ZIP codes) Yoshua Bengio of Toronto Backpropagation algorithm Deep Belief Networks Speech Recognition 1965 1989 1993 2009 2012 Alexey Ivakhnenko Jürgen Schmidhuber Andrew Ng Supervised deep feedforward Recurrent long and Deep Learning multilayer perceptrons short term memories renaissance with cats FUEL + SPARK + ENGINE MASSIVE + NEW + COMPUTER DATA MATH HORSEPOWER Deep Learning Demystified (NVIDIA), GTC 2017 http://nikhilbuduma.com/2014/12/29/deep-learning-in-a-nutshell/ http://www.theprojectspot.com/tutorial-post/introduction-to-artificial-neural-networks-part1/7 160 ResNet152, 152 layers 140 120 100 80 60 40 GoogLeNet, 22 layers 20 (VGG @ 19/7.3) AlexNet, 8 layers 8 layers 0 shallow shallow 2010 2011 2012 2013 2014 2015 layers error https://xkcd.com/1425/ https://leonardoaraujosantos.gitbooks.io/artificial-inteligence/content/object_localization_and_detection.html * Feature visualization images from “Visualizing and Understanding Convolutional Neural Networks”, Zeiler and Fergus, ECCV 2014. -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 1 -1 1 1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 x -1 -1 -1 1 -1 -1 -1 1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 http://brohrer.github.io/how_convolutional_neural_networks_work.html -1 -1 -1 -1 -1 -1 -1 -1 -1 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 -1 1 -1 -1 -1 -1 -1 1 -1 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 1 -1 -1 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 -1 1 -1 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 -1 -1 1 -1 -1 -1 1 -1 -1 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 -1 -1 1 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 -1 -1 -1 1 -1 1 -1 -1 -1 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77 0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33 -1 -1 -1 -1 1 -1 -1 -1 -1 -0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55 1 -1 1 0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11 -1 -1 -1 1 -1 1 -1 -1 -1 -1 1 -1 -0.11 0.33 -0.77 1.00 -0.77 0.33 -0.11 0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11 1 -1 1 -0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55 -1 -1 1 -1 -1 -1 1 -1 -1 0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33 -1 1 -1 -1 -1 -1 -1 1 -1 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 -1 -1 1 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 1 -1 -1 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 http://brohrer.github.io/how_convolutional_neural_networks_work.html 1.00 0.33 0.550.77 -0.110.330.11 0.33 0.55 -0.11 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.33 1.00 0.33 0.55 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.55 0.33 1.00 0.11 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.33 0.55 0.11 0.77 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77 0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33 0.55 0.33 0.55-0.55 0.550.33-0.55 0.33 -0.55 0.55 -0.55 0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11 0.33 1.00 0.55-0.11 0.330.11-0.77 1.00 -0.77 0.33 -0.11 0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11 0.55 0.55 0.55-0.55 0.550.11-0.55 0.33 -0.55 0.55 -0.55 0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33 0.33 0.11 0.11 0.33 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.33 0.55 1.000.55 -0.110.770.11 -0.33 1.00 -0.11 0.11 0.33 0.33 -0.33 0.55 -0.33 0.33 0.33 0.55 0.55 1.000.11 -0.110.331.00 -0.33 0.11 -0.11 0.55 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 1.00 1.00 0.110.77 -0.110.550.11 0.33 0.55 -0.11 0.33 0.77 0.33 0.55 0.33 http://brohrer.github.io/how_convolutional_neural_networks_work.html 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 0.77 0 0.11 0.33 0.55 0 0.33 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0 1.00 0 0.33 0 0.11 0 0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55 0.11 0 1.00 0 0.110.33 0.330 -0.330.55 0.55 -0.33 0.33 0.33 0.33 0.33 0 0.55 00.55 0.33-0.11 0.110.33 -0.33 1.00 -0.11 0.11 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.55 0 0.11 0 1.00 0 0.11 0.33 -0.11 0.55 0.33 0.11 -0.11 0.77 0 0.11 0 0.33 0 1.00 0 0.33 0 0.55 0.33 0.110.33 -0.550 0.110.77 -0.11 0.11 -0.55 0.33 -0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55 0.33 0 0.11 0 0.110.11 -0.550 0.550.33 -0.77 0.55 -0.55 0.11 0 0.55 0 0.33 -0.110 0.550.33 -0.770 1.00 -0.77 0.33 -0.11 0.11 0 0.55 0 0.550.11 -0.550 0.550.11 -0.77 0.55 -0.55 0.11 -0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55 0 0.33 0 1.00 0 0.33 0 0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33 0.11 0 0.55 0 0.55 0 0.11 0 0.55 0 0.33 00.33 0.55-0.11 0.550 0.33 0.11 -0.11 0.77 -0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11 0.33 0 0.11 0 0.11 0 0.33 0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11 0.33 0 0.55 0.33 0.110.33 0.330 -0.330.77 0.55 -0.33 0.33 0.33 0 0.11 0 0.33 00.11 1.00-0.11 1.000 -0.33 0.11 -0.11 0.55 -0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11 0.55 0 0.11 0 1.00 0 0.11 0.77 -0.11 0.11 0.33 0.55 -0.11 0.33 0.33 0.33 0 0.55 0 0.33 0.33 0.11 0 1.00 0 0.11 0 0.55 0 1.00 0 0.33 0 0.11 0 0.77 0 0.11 0.33 0.55 0 0.33 http://brohrer.github.io/how_convolutional_neural_networks_work.html -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 1.00 0.55 -1 -1 -1 1 -1 1 -1 -1 -1 0.55 1.00 -1 -1 -1 -1 1 -1 -1 -1 -1 1.00 0.55 -1 -1 -1 1 -1 1 -1 -1 -1 0.55 0.55 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 0.55 1.00 -1 -1 -1 -1 -1 -1 -1 -1 -1 1.00 0.55 http://brohrer.github.io/how_convolutional_neural_networks_work.html -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 X -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 O -1 -1 -1 -1 -1 -1 -1 -1 -1 http://brohrer.github.io/how_convolutional_neural_networks_work.html https://aka.ms/leak_detection train_test_split train_keras score_keras Selective Search Rich feature hierarchies for accurate object detection and semantic segmentation, https://arxiv.org/abs/1311.2524 Tutorial on CNTK https://www.youtube.com/watch?v=Khuj4ASldmU https://arxiv.org/pdf/1609.04802.pdf its own zoo Quora session Credit: Bruno Gavranović https://www.microsoft.com/developerblog/2017/06/12/learning-image-image-translation-cyclegans/ https://www.cs.cmu.edu/~sbhagava/papers/face-rec-ccs16.pdf AI’s White Guy Problem FaceApp “Whitewashing” https://medium.com/@ricardo.guerrero/deep-learning-frameworks-a-review-before-finishing-2016-5b3ab4010b06 https://github.com/zer0n/deepframeworks GitHub Azure Notebooks arXiv arXiv Sanity Preserver The Neural Network Zoo Brandon Rohrer‘s Blog Machine Learning is Fun Brief History of Image segmentation Brief History of Neural Nets and Deep Learning Deep Learning Toolkit (DSVM) .