AI in 10 minutes – How Shapes our Future of Production

Industrial Advisory Board Cluster of Excellence „Integrative Production Technology for High-Wage Countries”

February 23th, 2016

Univ.-Prof. Dr. rer. nat. Sabina Jeschke

IMA/ZLW & IfU Faculty of Mechanical Engineering RWTH Aachen University

www.ima-zlw-ifu.rwth-aachen.de Towards Machine Learning Machines and Learning 2

Can machines learn? ? Can they learn to predict future states and to do tasks optimized and in the right way? And if so, how can they do it?  This is what this talk is about!

 How do machines learn?

A – B – Learning by observations Learning by doing and explanations

 Data-driven  Trial-and-error learning learning

Let us take a look into a first example of ! data-driven learning! ! Later…

24.02.2016 S. Jeschke Data-driven learning - supervised A first example – learning from guided observations 3 ! Do you remember your childhood heroes – “The Mario Brothers” by Nintendo? So let us write down our observations (and gather some training data)

pos_x on_ground action status jump (B) 563 yes jump (B) alive (1) 571 yes jump (A) alive (1) 580 yes walk right dead (0) walk right 582 no jump (A) dead (0) … … … …

We want to learn general rules how to survive in this situation - by using data –  and visualize it in a decision tree yes > 560 and < 575 jump (B) on_ground pos_x action no c = 100% jump (A) confidence (c) = 50%

c = 75% c = 75%

24.02.2016 S. Jeschke Data-driven learning - supervised Supervised learning down-to-earth 4 Can we predict the result of a HPDC (high-pressure die casting) process – ? by using historical data? - YES WE CAN! … in cooperation with

IO NIO (Outbreak) NIO (Cold shot)

NIO (Blowhole)

HPDC process Historic data Prediction model Visualization of prediction Process and Modelling and Inline and web-based quality data training (result NIO|IO with reason)

We extended the prediction model by integrating  . mechanical vibration (using solid-borne sound sensors) . weather data. Acoustic measurements Extended model Weather data Fourier transformation & k-nearest clustering and Temporal correlation of feature extraction random forest tree weather (and circumstances)

24.02.2016 S. Jeschke Data-driven learning - unsupervised A second example – what if we do not tell what is right 5 What if we do not know if an observation belongs to a specific category? ? Or, if an observation is good or bad?  Finding the hidden structure in data! “Although it may seem somewhat mysterious to imagine what a machine could possibly learn given that it doesn't get any feedback from its environment, it is possible to find patterns in image data using probabilistic techniques.” Zoubin Ghahramani, Professor of Information Engineering at the , Machine Learning

Cleansing, preprocessing and Batch of unlabeled pictures hierarchical clustering

 Unsupervised. Human factor is reduced to modeling. (however a certain bias survives…)

24.02.2016 S. Jeschke Data-driven learning - unsupervised Unsupervised learning “down-to-earth” 6 Finding hidden relations in our data, we were not aware of, e.g. ! understanding failures or bad quality of products and processes … in cooperation with

Data about chemical compositions of [Ruiz, 2014] steel (identified as low quality - example)

. Sulfur (S) > 0.04% and heat treatment  fragile structure Searching for hidden relations in data . Phosphorus (P) > 0.04%  reduced plasticity by applying subgroup mining . Chrome (Cr) > 16%, Molybdenum (Mo) > 13%, Nickel (Ni) > 56%  no findings . …

24.02.2016 S. Jeschke Learning by doing – reinforcement learning The next step: Using rewards to learn actions 7 Remember Mario: What if the machine could learn, how to solve a level? ? Why not use a some kind of intelligent trial-and-error?

Neuroevolution of augmenting

topologies (NEAT) [Stanley, 2002] . Genetic algorithms on top of neural networks . At each state the system decides what action to do . Actions are rewarded if Mario do not die in return . Level progress by evolving neural [SethBling, 2015] networks

Reinforcement learning (R-learning) Human factor is “very small” is inspired by behaviorist psychology – . reduced to very general, mainly  maximizing the expected return by applying  formal specifications of the neural a sequence of actions at a current state. network… . However, human still influences the  can be applied to broad variety of problems underlying representation model

24.02.2016 S. Jeschke Learning by doing Reinforcement learning “down-to-earth” 8 Obviously: Super-Mario can easily be extended ! towards intralogistics scenarios… [TU Delft, 2012] [MiorSoft (reexre), 2014]

[UC Berkeley, 2015]

… for learning and optimization of motions [UC Berkeley, 2015]

[Intelligent Autonomous Systems, 2015] … for learning and executing complete assembly tasks Should have crashed 10.000 cars before coming up with first „ok- solutions“ for autonomous driving?  Coupling to embodiment theory

… as “pro-training” for human- Avoiding “nonsense solutions” by machine interaction using simulation environments

24.02.2016 S. Jeschke Deep learning – can to used “everywhere” The age of deep learning (deep neural networks) 9 “Today, computers are beginning to be able to generate human-like insights into ! data…. Underlying … is the application of large artificial neural networks to machine learning, often referred to as deep learning.” [Cognitive Labs, 2016] Deep Q-Networks (also "deep reinforcement learning“,  Q refers to the mathematical action-prediction-function behind the scenes….): Learning directly from high-dimensional sensory input

[Minh, 2015] [nature, 2015]  AI starts to develop strategies to beat the game  Signs of “body cousciousness”  Human factor practically zero.  …

24.02.2016 S. Jeschke Deep learning – can to used “everywhere” Deep learning “down-to-earth” 10 ! … a variety of practical applications

Face/picture/object recognition  Central part Important feature for autonomous driving etc. of “cognitive computing”

Handwriting recognition Natural language processing Anomaly recognition Automated translation

24.02.2016 S. Jeschke Deep learning Where the story goes: AlphaGo 11

Go originated in China more than 2,500 years ago. Confucius wrote about it. As simple as the rules are, Go is a game of ! profound complexity. This complexity is what makes Go hard for computers to play, and an irresistible challenge to artificial intelligence (AI) researchers. [adapted from Hassabis, 2016]

The problem: 2.57×10210 possible positions - that is more than the number of atoms in the  universe, and more than a googol times (10100) larger than chess.  Bringing it all together! Training set Learning non-human strategies 30 million moves recorded from AlphaGo designed by Google DeepMind, games played by humans experts played against itself in thousands of games

and evolved its neural networks; Monte Carlo 2016] [Hassabis, tree search

Creating deep neural networks January 2016: 12 network layers with millions of Beating Fan Hui (triple EU champion)

driven learning driven neuron-like connections

- AlphaGo won 5 games to 0.

(5 years before time)

Data Reinforcement learning Reinforcement Predicting the human move (57% of time) ! Achieving one of the grand challenges of AI

24.02.2016 S. Jeschke Summary What a zoo! Get me out of here… :) 12 ? Relax. Order is half of life. Machine learning Brain projects A – B – C– Learning by observations Learning by doing Using “brain and explanations  Trial-and-error structures”  Data-driven learning learning

reinforcement supervised… learning

un-supervised… neuroevolution Human brain k-nearest clustering genetic algorithms neural networks project (EU) deep learning decision trees Brain project (US) Monte Carlo tree search random forest trees Q-Learning SARSA … Neuromorphic … computing WANT BETTER RESULT? – Just shake it!!

24.02.2016 S. Jeschke Summary When do you start embracing artificial intelligence? 13

Waiting for Google to take over? – Google is addressing fields as gaming, mobility, language … ! Because there, they do get the data they need. They do not have the data for production lines. – So far….

Production engineering is still somewhat hesitating and waiting, but you have the data and we have the domain ! experts (from production engineering as well as data science), so - let’s get started!

[Blomberg, 2016]

24.02.2016 S. Jeschke Thank you!

Univ.-Prof. Dr. rer. nat. Sabina Jeschke Head of Institute Cluster IMA/ZLW & IfU [email protected]

Co-authored by:

Prof. Dr.-Ing. Tobias Meisen Juniorprofessor, Managing director IMA [email protected]

Dr.-Ing. Christian Büscher Research group leader „Production Technology“ [email protected]

Thorsten Sommer, M. Eng. Team „Knowledge Engineering“ [email protected]

www.ima-zlw-ifu.rwth-aachen.de Prof. Dr. rer. nat. Sabina Jeschke 15

1968 Born in Kungälv/Schweden 1991 – 1997 Studies of Physics, Mathematics, Computer Sciences, TU Berlin 1994 NASA Ames Research Center, Moffett Field, CA/USA 10/1994 Fellowship „Studienstiftung des Deutschen Volkes“ 1997 Diploma Physics 1997 – 2000 Research , TU Berlin, Institute for Mathematics 2000 – 2001 Lecturer, Georgia Institute of Technology, GA/USA 2001 – 2004 Project leadership, TU Berlin, Institute for Mathematics 04/2004 Ph.D. (Dr. rer. nat.), TU Berlin, in the field of Computer Sciences 2004 Set-up and leadership of the Multimedia-Center at the TU Berlin 2005 – 2007 Juniorprofessor „New Media in Mathematics & Sciences“ & Director of the Multimedia-center MuLF, TU Berlin 2007 – 2009 Univ.-Professor, Institute for IT Service Technologies (IITS) & Director of the Computer Center (RUS), Department of Electrical Engineering, University of Stuttgart since 06/2009 Univ.-Professor, Head of the Institute Cluster IMA/ZLW & IfU, Department of Mechanical Engineering, RWTH Aachen University since 10/2011 Vice Dean of the Department of Mechanical Engineering, RWTH Aachen University since 03/2012 Chairwoman VDI Aachen since 05/2015 Supervisory Board of Körber AG, Hamburg

24.02.2016 S. Jeschke References 16

[Bloomberg, 2016] Why 2015 Was a Breakthrough Year in Artificial Intelligence, http://www.bloomberg.com/news/articles/2015-12-08/why-2015-was-a- breakthrough-year-in-artificial-intelligence, last visited 18.02.2016 [Cognitive Labs, 2016] http://cognitivlabs.com/the-age-of-neural-networks/, last visited 18.02.2016 [Hassabis, 2016] AlphaGo: using machine learning to master the ancient game of Go, https://googleblog.blogspot.de/2016/01/alphago-machine-learning- game-go.html, last visited 18.02.2016 [Intelligent Autonomous Systems, 2015] Collaborative assembly with phase estimation (ISRR 2015), https://www.youtube.com/watch?v=4qDFv02xlNo, last visited 18.02.2016 [Minh, 2015] Mnih, Volodymyr; et al. (2015). “Human-level control through deep reinforcement learning”, 518: 529–533. [MiorSoft (reexre), 2014] Neuroevolution - Car learns to drive, https://www.youtube.com/watch?v=5lJuEW-5vr8, last visited 18.02.2016 [nature, 2015] http://www.nature.com/nature/journal/v518/n7540/fig_tab/nature14236_SV1.html, last visited 18.02.2016 [Ruiz, 2014] Ruiz, Paula Andrea Rotes; et al. (2014). “An Interactive Approach for the Post-processing in a KDD Process”. In: Advances in Production Management Systems. Innovative and Knowledge-Based Production Management in a Global-Local World”, pp 93-100 [SethBling, 2015] MarI/O - Machine Learning for Video Games, https://www.youtube.com/watch?v=qv6UVOQ0F44, last visited 18.02.2016 [Stanley, 2002] Stanley, Kenneth O. and Miikkulainen, Risto (2002). “Evolving Neural Networks through Augmenting Topologies”. In: Evolutionary Computation: 10(2) 2002, pp 99–127. [TU Delft, 2012] TU Delft robot Leo learns to walk, https://www.youtube.com/watch?v=SBf5-eF-EIw, last visited 18.02.2016 [UC Berkeley, 2015] BRETT the Robot learns to put things together on his own, https://www.youtube.com/watch?v=JeVppkoloXs, last visited 18.02.2016 [Lee et.al. 2011) Unsupervised Learning of Hierarchical Representations with Convolutional Deep Belief Networks, http://www.cs.princeton.edu/~rajeshr/papers/cacm2011-researchHighlights-convDBN.pdf ; last visited 23.02.2016, doi :10.1145/2001269.2001295

24.02.2016 S. Jeschke