Introduction to Deep Learning and Tensorflow

Introduction to Deep Learning and Tensorflow Day 1 MR ST LF FS SO IB CINECA Roma - SCAI Department Marco Rorro Stefano Tagliaventi Luca Ferraro Francesco Salvadore Sergio Orlandini Isabella Baccarelli Rome, 7th-9th Nov 2018 Outline Intro and ML Basics Intro The Task The Performance The Experience ML 1 Intro and ML Basics ingredients Data Model Objective f 2 Introduction Optimization Tensorflow Graph and Session 3 A simple ML ingredients example 4 Tensorflow Outline Intro and ML Basics Intro The Task The Performance 1 The Experience Intro and ML Basics ML ingredients Data 2 Introduction Model Objective f Optimization The Task Tensorflow The Performance Measure Graph and Session The Experience A simple example 3 ML ingredients 4 Tensorflow The Big Picture Intro and ML Basics Intro The Task The Performance The Experience ML ingredients Data Model Objective f Optimization Tensorflow Graph and Session A simple example Machine learning basics Intro and ML Basics Intro The Task Machine learning is a form of applied statistic with increased The Performance The Experience emphasis on the use of computers to statistically estimate ML ingredients complicated functions and decreased emphasis on proving Data Model confidence intervals around these functions. Objective f Optimization • Machine learning is a subset of Artificial Intelligence that Tensorflow Graph and uses statistics to build models θ of data from data. Session A simple example • Machine learning systems use frequentist statistics, i.e estimate the true fixed models from uncertain observations. • Machine learning systems use Bayesian statistics, i.e find best model from families of models. • Machine learning is a matter of statistics Machine learning definition Intro and ML Basics Intro The Task The Performance The Experience ML • Arthur Samuel (1959). Machine Learning: Field of study that ingredients Data gives computers the ability to learn without being explicitly Model Objective f Optimization programmed. Tensorflow • Tom Mitchell (1998) Well-posed Learning Problem: A Graph and Session A simple computer program is said to learn from experience E with example respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E. Learning problems Intro and ML Basics Playing checkers Intro The Task The Performance • The task T : playing checkers. The Experience ML • The performance measure P : percent of games won against ingredients Data opponents. Model Objective f Optimization • The training experience E : playing practice games against Tensorflow itself. Graph and Session A simple example Recognize human faces in images • The task T : automatic human face recognition. • The performance measure P : percent of faces recognized. • The training experience E : learn from a database of face images. The Task Intro and ML Basics Intro • Machine learning enable us to tackle tasks that are too The Task The Performance The Experience difficult to solve with fixed programs written and designed by ML human beings. ingredients • Data fixed is referred to rule-based system where the decision Model Objective f process is defined at the design level of the algorithm. In such Optimization system output is deterministic with respect to the input. Tensorflow • Graph and probabilistic systems are in contrast to fixed programs. Session A simple • Machine learning deals with probabilistic systems. example • Time series forecasting • Computational linguistics • Game palying • Driving • The task shall not be confused with the learning phase, which instead is the procedure used to achieve the task. Tasks: Classification I Intro and ML Basics • In this type of task, the computer program is asked to specify Intro The Task which of k categories some input belongs to. The Performance n The Experience f : R ! f1; :::; kg. ML ingredients Map f could be a probability distribution over classes. Data Model A typical classification task is object recognition, where the Objective f Optimization input is an image. Tensorflow Graph and Session A simple example Tasks: Classification II Intro and ML Basics Intro The Task The Performance • The Experience Classification with missing inputs: It is possible to ML generalize the classification task to take into account ingredients Data incomplete inputs or some missing variable. Instead of Model Objective f learning a single function, the system can learn a probability Optimization Tensorflow distribution over all relevant variables, then solve the Graph and Session classification by marginalizing out the missing variables, i.e A simple example summing up over the unknown distributions to extract information on known ones. • A typical use case is related to medical diagnosis where some tests could be invasive or expensive. Tasks: Classification III Intro and ML Basics Intro The Task • Regression: In this type of task the program is asked to The Performance n The Experience predict a numerical value given some input. f : R ! R Used ML ingredients for example in algorithmic trading. Data Model Objective f Optimization Tensorflow Graph and Session A simple example Tasks: Structured output I Intro and ML Basics • In structured output tasks output values are related to each Intro The Task The Performance other. One example is parsing a natural language sentence The Experience where the output is the tree that describes its grammatical ML ingredients structure as for example tagging verbs, nouns, adverbs and Data Model so on. Objective f Optimization • Transcription: In this type of task, the machine learning Tensorflow Graph and Session system is asked to observe a relatively unstructured A simple example representation of some kind of data and transcribe the information into discrete textual form. Examples are optical character recognition and speech recognition, where the computer program is asked to map a raw data (image pixels or waveform audio data ) to a sequence of characters. • Machine translation: Translate from symbols in some language to symbols in another language. TasksI Intro and ML Basics • Anomaly detection: In this type of task, the computer Intro The Task program sifts through a set of events or objects and flags The Performance The Experience some of them as being unusual or atypical. One example is ML ingredients credit card fraud detection, by modeling user purchasing Data Model habits. Objective f Optimization • Synthesis and sampling: In this type of task, the machine Tensorflow Graph and learning algorithm is asked to generate new examples that Session A simple example are similar to those into the training data. For example video games can use synthesis to automatically generate textures for large objects. Another example is the speech synthesis task, where given a written sentence an audio waveform is produced containing the spoken version with the requirements that there is no single correct output for each input. A large amount of variation in the output produce a more natural and realistic system. TasksII Intro and • Denoising: Predict a clean output from a corrupted input. ML Basics • Intro Density estimation: In this kind of tasks the machine The Task The Performance learning algorithm is asked to learn a function The Experience n pmodel ! where pmodel (x) can be interpreted as a ML R R ingredients probability density function(if x is continuous) or a probability Data Model Objective f mass function (if x is discrete) on the space that the Optimization examples where drawn from. In density estimation tasks the Tensorflow Graph and algorithm needs to learn the structure of the data. Session A simple example The Performance Measure,P Intro and ML Basics Intro • To evaluate the abilities of a machine learning algorithm, we The Task The Performance The Experience must design a quantitative measure of its performance, ML usually related to the task T. ingredients Data • Model For classification tasks we can use the accuracy/error rate, Objective f Optimization i.e. the proportion of examples for which the model produces Tensorflow the correct/incorrect output. Graph and Session A simple • In machine learning field we are interested in evaluating the example performance on data that is not seen before. • We evaluate these performance measures using a test set of data that is separate from the data used for training the machine learning system. • It is often difficult to choose a performance measure corresponding well to the desired behavior of the system. The Experience, E I Intro and ML Basics Intro The Task The Performance The Experience ML Two main categories: ingredients Data Model • Unsupervised learning algorithms experience a dataset Objective f Optimization containing many features, then learn useful properties of the Tensorflow structure of this dataset. Graph and Session • In some contexts we wish to learn the entire probability A simple example distribution that generated a dataset • Implicitly : denoising, synthesis • Explicitly : density estimation The Experience, E II Intro and ML Basics Intro The Task The Performance The Experience ML ingredients Data Model Objective f Optimization Tensorflow Graph and Session A simple example • Supervised learning algorithm experience a dataset containing features, but each example is also associated with a label or target • Regression, classification, structured output The Experience, E III Intro and ML Basics Intro The Task • Unsupervised and supervised are not formally defined terms, The Performance The Experience many ML technologies can be used to perform both. ML ingredients

Load more