DOCSLIB.ORG

Artificial Intelligence Machine Learning and Deep Learning

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Artificial Intelligence Machine Learning and Deep Learning ARTIFICIAL INTELLIGENCE MACHINE LEARNING AND DEEP LEARNING LICENSE, DISCLAIMER OF LIABILITY, AND LIMITED WARRANTY By purchasing or using this book and its companion files (the “Work”), you agree that this license grants permission to use the contents contained herein, but does not give you the right of ownership to any of the textual content in the book or ownership to any of the information, files, or products contained in it. This license does not permit uploading of the Work onto the Internet or on a network (of any kind) without the written consent of the Publisher. Duplication or dissemination of any text, code, simu- lations, images, etc. contained herein is limited to and subject to licensing terms for the respective products, and permission must be obtained from the Publisher or the owner of the content, etc., in order to reproduce or network any portion of the textual material (in any media) that is contained in the Work. MERCURY LEARNING AND INFORMATION (“MLI” or “the Publisher”) and anyone involved in the creation, writing, production, accompanying algorithms, code, or computer programs (“the software”), and any accompanying Web site or software of the Work, cannot and do not warrant the performance or results that might be obtained by using the contents of the Work. The author, developers, and the Publisher have used their best efforts to insure the accuracy and functionality of the textual material and/or pro- grams contained in this package; we, however, make no warranty of any kind, express or implied, regarding the performance of these contents or programs. The Work is sold “as is” without warranty (except for defective materials used in manufacturing the book or due to faulty workmanship). The author, developers, and the publisher of any accompanying content, and anyone involved in the composition, production, and manufacturing of this work will not be liable for damages of any kind arising out of the use of (or the inability to use) the algorithms, source code, computer programs, or textual material contained in this publication. This includes, but is not limited to, loss of revenue or profit, or other incidental, physical, or consequential damages arising out of the use of this Work. The sole remedy in the event of a claim of any kind is expressly limited to replace- ment of the book and only at the discretion of the Publisher. The use of “implied warranty” and certain “exclusions” vary from state to state, and might not apply to the purchaser of this product. Companion files also available for downloading from the publisher by writing to [email protected]. ARTIFICIAL INTELLIGENCE MACHINE LEARNING AND DEEP LEARNING Oswald Campesato MERCURY LEARNING AND INFORMATION Dulles, Virginia Boston, Massachusetts New Delhi Copyright © 2020 by MERCURY LEARNING AND INFORMATION LLC. All rights reserved. This publication, portions of it, or any accompanying software may not be reproduced in any way, stored in a retrieval system of any type, or transmitted by any means, media, electronic display or mechanical display, including, but not limited to, photocopy, recording, Internet postings, or scanning, without prior permission in writing from the publisher. Publisher: David Pallai MERCURY LEARNING AND INFORMATION 22841 Quicksilver Drive Dulles, VA 20166 [email protected] www.merclearning.com 1-800-232-0223 I’d like to dedicate this book to my parents – O. Campesato. Artificial Intelligence, Machine Learning and Deep Learning. may this bring joy and happiness into their lives. ISBN: 978-1-68392-467-8 The publisher recognizes and respects all marks used by companies, manufacturers, and developers as a means to distinguish their products. All brand names and product names mentioned in this book are trademarks or service marks of their respective companies. Any omission or misuse (of any kind) of service marks or trademarks, etc. is not an attempt to infringe on the property of others. Library of Congress Control Number: 2019957226 202122321 Printed on acid-free paper in the United States of America. Our titles are available for adoption, license, or bulk purchase by institutions, corporations, etc. For additional information, please contact the Customer Service Dept. at 800-232-0223(toll free). All of our titles are available in digital format at authorcloudware.com and other digital vendors. The sole obligation of MERCURY LEARNING AND INFORMATION to the purchaser is to replace the book, based on defective materials or faulty workmanship, but not based on the operation or functionality of the product. I’d like to dedicate this book to my parents – may this bring joy and happiness into their lives. CONTENTS Preface xv Chapter 1: Introduction to AI 1 What is Artificial Intelligence? 2 Strong AI versus Weak AI 4 The Turing Test 5 Definition of the Turing Test 5 An Interrogator Test 6 Heuristics 6 Genetic Algorithms 8 Knowledge Representation 8 Logic-based Solutions 9 Semantic Networks 9 AI and Games 10 The Success of AlphaZero 11 Expert Systems 12 Neural Computing 13 Evolutionary Computation 14 Natural Language Processing 14 Bioinformatics 17 Major Parts of AI 18 Machine Learning 18 Deep Learning 19 Reinforcement Learning 19 Robotics 20 Code Samples 21 Summary 22 viii • CONTENTS Chapter 2: Introduction to Machine Learning 23 What is Machine Learning? 24 Types of Machine Learning 24 Types of Machine Learning Algorithms 26 Machine Learning Tasks 28 Feature Engineering, Selection, and Extraction 30 Dimensionality Reduction 31 PCA 32 Covariance Matrix 33 Working with Datasets 33 Training Data Versus Test Data 34 What Is Cross-validation? 34 What Is Regularization? 34 ML and Feature Scaling 35 Data Normalization vs Standardization 35 The Bias-Variance Tradeoff 35 Metrics for Measuring Models 36 Limitations of R-Squared 36 Confusion Matrix 37 Accuracy vs Precision vs Recall 37 The ROC Curve 38 Other Useful Statistical Terms 38 What Is an F1 Score? 38 What Is a p-value? 39 What Is Linear Regression? 39 Linear Regression vs Curve-Fitting 40 When Are Solutions Exact Values? 40 What Is Multivariate Analysis? 41 Other Types of Regression 42 Working with Lines in the Plane (optional) 43 Scatter Plots with NumPy and Matplotlib (1) 46 Why the “Perturbation Technique” Is Useful 48 Scatter Plots with NumPy and Matplotlib (2) 48 A Quadratic Scatterplot with NumPy and Matplotlib 49 The Mean Squared Error (MSE) Formula 51 A List of Error Types 51 Non-linear Least Squares 52 Calculating the MSE Manually 52 Approximating Linear Data with np.linspace() 54 Calculating MSE with np.linspace() API 55 Linear Regression with Keras 57 Summary 62 CONTENTS • ix Chapter 3: Classifiers in Machine Learning 63 What Is Classification? 64 What Are Classifiers? 64 Common Classifiers 65 Binary vs MultiClass Classification 65 MultiLabel Classification 66 What Are Linear Classifiers? 66 What Is kNN? 67 How to Handle a Tie in kNN 67 What Are Decision Trees? 68 What Are Random Forests? 73 What Are SVMs? 74 Tradeoffs of SVMs 74 What Is Bayesian Inference? 75 Bayes Theorem 75 Some Bayesian Terminology 76 What Is MAP? 77 Why Use Bayes’ Theorem? 77 What Is a Bayesian Classifier? 77 Types of Naïve Bayes Classifiers 78 Training Classifiers 78 Evaluating Classifiers 79 What Are Activation Functions? 80 Why do We Need Activation Functions? 81 How Do Activation Functions Work? 81 Common Activation Functions 82 Activation Functions in Python 83 Keras Activation Functions 84 The ReLU and ELU Activation Functions 84 The Advantages and Disadvantages of ReLU 85 ELU 85 Sigmoid, Softmax, and Hardmax Similarities 86 Softmax 86 Softplus 86 Tanh 86 Sigmoid, Softmax, and HardMax Differences 87 What Is Logistic Regression? 87 Setting a Threshold Value 88 Logistic Regression: Important Assumptions 89 Linearly Separable Data 89 Keras, Logistic Regression, and Iris Dataset 89 Summary 93 x • CONTENTS Chapter 4: Deep Learning Introduction 95 Keras and the XOR Function 96 What Is Deep Learning? 98 What Are Hyper Parameters? 100 Deep Learning Architectures 101 Problems that Deep Learning Can Solve 101 Challenges in Deep Learning 102 What Are Perceptrons? 103 Definition of the Perceptron Function 104 A Detailed View of a Perceptron 104 The Anatomy of an Artificial Neural Network (ANN) 105 Initializing Hyperparameters of a Model 107 The Activation Hyperparameter 107 The Loss Function Hyperparameter 108 The Optimizer Hyperparameter 108 The Learning Rate Hyperparameter 109 The Dropout Rate Hyperparameter 109 What Is Backward Error Propagation? 109 What Is a Multilayer Perceptron (MLP)? 110 Activation Functions 111 How Are Datapoints Correctly Classified? 112 A High-Level View of CNNs 113 A Minimalistic CNN 114 The Convolutional Layer (Conv2D) 114 The ReLU Activation Function 115 The Max Pooling Layer 115 Displaying an Image in the MNIST Dataset 118 Keras and the MNIST Dataset 119 Keras, CNNs, and the MNIST Dataset 122 Analyzing Audio Signals with CNNs 125 Summary 126 Chapter 5: Deep Learning: RNNs and LSTMs 127 What Is an RNN? 128 Anatomy of an RNN 129 What Is BPTT? 130 Working with RNNs and Keras 130 Working with Keras, RNNs, and MNIST 132 Working with TensorFlow and RNNs (Optional) 135 What Is an LSTM? 139 Anatomy of an LSTM 139 Bidirectional LSTMs 140 CONTENTS • xi LSTM Formulas 141 LSTM Hyperparameter Tuning 142 Working with TensorFlow and LSTMs (Optional) 142 What Are GRUs? 147 What Are Autoencoders? 147 Autoencoders and PCA 150 What Are Variational Autoencoders? 150 What Are GANs? 151 Can Adversarial Attacks Be Stopped? 152 Creating a GAN 153 A High-Level View of GANs 156 The VAE-GAN
Load more

Recommended publications
  • Intro to Tensorflow 2.0 MBL, August 2019
    Intro to TensorFlow 2.0 MBL, August 2019 Josh Gordon (@random_forests) 1 Agenda 1 of 2 Exercises ● Fashion MNIST with dense layers ● CIFAR-10 with convolutional layers Concepts (as many as we can intro in this short time) ● Gradient descent, dense layers, loss, softmax, convolution Games ● QuickDraw Agenda 2 of 2 Walkthroughs and new tutorials ● Deep Dream and Style Transfer ● Time series forecasting Games ● Sketch RNN Learning more ● Book recommendations Deep Learning is representation learning Image link Image link Latest tutorials and guides tensorflow.org/beta News and updates medium.com/tensorflow twitter.com/tensorflow Demo PoseNet and BodyPix bit.ly/pose-net bit.ly/body-pix TensorFlow for JavaScript, Swift, Android, and iOS tensorflow.org/js tensorflow.org/swift tensorflow.org/lite Minimal MNIST in TF 2.0 A linear model, neural network, and deep neural network - then a short exercise. bit.ly/mnist-seq ... ... ... Softmax model = Sequential() model.add(Dense(256, activation='relu',input_shape=(784,))) model.add(Dense(128, activation='relu')) model.add(Dense(10, activation='softmax')) Linear model Neural network Deep neural network ... ... Softmax activation After training, select all the weights connected to this output. model.layers[0].get_weights() # Your code here # Select the weights for a single output # ... img = weights.reshape(28,28) plt.imshow(img, cmap = plt.get_cmap('seismic')) ... ... Softmax activation After training, select all the weights connected to this output. Exercise 1 (option #1) Exercise: bit.ly/mnist-seq Reference: tensorflow.org/beta/tutorials/keras/basic_classification TODO: Add a validation set. Add code to plot loss vs epochs (next slide). Exercise 1 (option #2) bit.ly/ijcav_adv Answers: next slide.
    [Show full text]
  • Ways to Use Machine Learning Approaches for Software Development
    Ways to use Machine Learning approaches for software development Nicklas Jonsson Nicklas Jonsson VT 2018 Examensarbete, 30 hp Supervisor: Eddie wadbro Extern Supervisor: C4 Contexture Examiner: Henrik Bjorklund¨ Master of Science Programme in Computing Science and Engineering, 300 hp Abstract With the rise of machine learning and in particular deep learning enter- ing all different types of fields, including software development. It could be a bit hard to know where to begin to search for the tools when some- one wants to use machine learning for a one’s problems. This thesis has looked at some available technologies of today for applying machine learning to one’s applications. This thesis has looked at some of the available cloud services, frame- works, and libraries for machine learning and it presents three different implementation structures that can be used with these technologies for the problem of image classification. Acknowledgements I want to thank C4 Contexture for giving me the thesis idea, support, and supplying me with a working station. I also want to thank Lantmannen¨ for supplying me with the image data that was used for this thesis. Finally, I want to thank Eddie Wadbro for his guidance during this thesis and of course a big thanks to my family and friends for their support during this period of my life. 1(45) Content 1 Introduction 3 1.1 The Client 3 1.1.1 C4 Contexture PIM software 4 1.2 The data 4 1.3 Goal 5 1.4 Limitation 5 2 Background 7 2.1 Artificial intelligence, machine learning and deep learning.
    [Show full text]
  • Keras2c: a Library for Converting Keras Neural Networks to Real-Time Compatible C
    Keras2c: A library for converting Keras neural networks to real-time compatible C Rory Conlina,∗, Keith Ericksonb, Joeseph Abbatec, Egemen Kolemena,b,∗ aDepartment of Mechanical and Aerospace Engineering, Princeton University, Princeton NJ 08544, USA bPrinceton Plasma Physics Laboratory, Princeton NJ 08544, USA cDepartment of Astrophysical Sciences at Princeton University, Princeton NJ 08544, USA Abstract With the growth of machine learning models and neural networks in mea- surement and control systems comes the need to deploy these models in a way that is compatible with existing systems. Existing options for deploying neural networks either introduce very high latency, require expensive and time con- suming work to integrate into existing code bases, or only support a very lim- ited subset of model types. We have therefore developed a new method called Keras2c, which is a simple library for converting Keras/TensorFlow neural net- work models into real-time compatible C code. It supports a wide range of Keras layers and model types including multidimensional convolutions, recurrent lay- ers, multi-input/output models, and shared layers. Keras2c re-implements the core components of Keras/TensorFlow required for predictive forward passes through neural networks in pure C, relying only on standard library functions considered safe for real-time use. The core functionality consists of ∼ 1500 lines of code, making it lightweight and easy to integrate into existing codebases. Keras2c has been successfully tested in experiments and is currently in use on the plasma control system at the DIII-D National Fusion Facility at General Atomics in San Diego. 1. Motivation TensorFlow[1] is one of the most popular libraries for developing and training neural networks.
    [Show full text]
  • The Ethics of Machine Learning
    SINTEZA 2019 INTERNATIONAL SCIENTIFIC CONFERENCE ON INFORMATION TECHNOLOGY AND DATA RELATED RESEARCH DATA SCIENCE & DIGITAL BROADCASTING SYSTEMS THE ETHICS OF MACHINE LEARNING Danica Simić* Abstract: Data science and machine learning are advancing at a fast pace, which is why Nebojša Bačanin Džakula tech industry needs to keep up with their latest trends. This paper illustrates how artificial intelligence and automation in particular can be used to en- Singidunum University, hance our lives, by improving our productivity and assisting us at our work. Belgrade, Serbia However, wrong comprehension and use of the aforementioned techniques could have catastrophic consequences. The paper will introduce readers to the terms of artificial intelligence, data science and machine learning and review one of the most recent language models developed by non-profit organization OpenAI. The review highlights both pros and cons of the language model, predicting measures humanity can take to present artificial intelligence and automatization as models of the future. Keywords: Data Science, Machine Learning, GPT-2, Artificial Intelligence, OpenAI. 1. INTRODUCTION Th e rapid development of technology and artifi cial intelligence has al- lowed us to automate a lot of things on the internet to make our lives eas- ier. With machine learning algorithms, technology can vastly contribute to diff erent disciplines, such as marketing, medicine, sports, astronomy and physics and more. But, what if the advantages of machine learning and artifi cial intelligence in particular also carry some risks which could have severe consequences for humanity. 2. MACHINE LEARNING Defi nition Machine learning is a fi eld of data science which uses algorithms to improve the analytical model building [1] Machine learning uses artifi cial Correspondence: intelligence (AI) to learn from great data sets and identify patterns which Danica Simić could support decision making parties, with minimal human intervention.
    [Show full text]
  • Tensorflow, Theano, Keras, Torch, Caffe Vicky Kalogeiton, Stéphane Lathuilière, Pauline Luc, Thomas Lucas, Konstantin Shmelkov Introduction
    TensorFlow, Theano, Keras, Torch, Caffe Vicky Kalogeiton, Stéphane Lathuilière, Pauline Luc, Thomas Lucas, Konstantin Shmelkov Introduction TensorFlow Google Brain, 2015 (rewritten DistBelief) Theano University of Montréal, 2009 Keras François Chollet, 2015 (now at Google) Torch Facebook AI Research, Twitter, Google DeepMind Caffe Berkeley Vision and Learning Center (BVLC), 2013 Outline 1. Introduction of each framework a. TensorFlow b. Theano c. Keras d. Torch e. Caffe 2. Further comparison a. Code + models b. Community and documentation c. Performance d. Model deployment e. Extra features 3. Which framework to choose when ..? Introduction of each framework TensorFlow architecture 1) Low-level core (C++/CUDA) 2) Simple Python API to define the computational graph 3) High-level API (TF-Learn, TF-Slim, soon Keras…) TensorFlow computational graph - auto-differentiation! - easy multi-GPU/multi-node - native C++ multithreading - device-efficient implementation for most ops - whole pipeline in the graph: data loading, preprocessing, prefetching... TensorBoard TensorFlow development + bleeding edge (GitHub yay!) + division in core and contrib => very quick merging of new hotness + a lot of new related API: CRF, BayesFlow, SparseTensor, audio IO, CTC, seq2seq + so it can easily handle images, videos, audio, text... + if you really need a new native op, you can load a dynamic lib - sometimes contrib stuff disappears or moves - recently introduced bells and whistles are barely documented Presentation of Theano: - Maintained by Montréal University group. - Pioneered the use of a computational graph. - General machine learning tool -> Use of Lasagne and Keras. - Very popular in the research community, but not elsewhere. Falling behind. What is it like to start using Theano? - Read tutorials until you no longer can, then keep going.
    [Show full text]
  • Deep Learning with Keras I
    Deep Learning with Keras i Deep Learning with Keras About the Tutorial Deep Learning essentially means training an Artificial Neural Network (ANN) with a huge amount of data. In deep learning, the network learns by itself and thus requires humongous data for learning. In this tutorial, you will learn the use of Keras in building deep neural networks. We shall look at the practical examples for teaching. Audience This tutorial is prepared for professionals who are aspiring to make a career in the field of deep learning and neural network framework. This tutorial is intended to make you comfortable in getting started with the Keras framework concepts. Prerequisites Before proceeding with the various types of concepts given in this tutorial, we assume that the readers have basic understanding of deep learning framework. In addition to this, it will be very helpful, if the readers have a sound knowledge of Python and Machine Learning. Copyright & Disclaimer Copyright 2019 by Tutorials Point (I) Pvt. Ltd. All the content and graphics published in this e-book are the property of Tutorials Point (I) Pvt. Ltd. The user of this e-book is prohibited to reuse, retain, copy, distribute or republish any contents or a part of contents of this e-book in any manner without written consent of the publisher. We strive to update the contents of our website and tutorials as timely and as precisely as possible, however, the contents may contain inaccuracies or errors. Tutorials Point (I) Pvt. Ltd. provides no guarantee regarding the accuracy, timeliness or completeness of our website or its contents including this tutorial.
    [Show full text]
  • Supervised Machine Learning
    Tikaram Acharya Supervised Machine Learning Accuracy Comparison of Different Algorithms on Sample Data Metropolia University of Applied Sciences Bachelor of Engineering Information Technology Bachelor’s Thesis 11 January 2021 Author Tikaram Acharya Title Supervised Machine Learning: Accuracy Comparison of Differ- ent Algorithms on Sample Data Number of Pages 39 pages Date 11 January 2021 Degree Bachelor of Engineering Degree Programme Information Technology Professional Major Software Engineering Instructors Janne Salonen, Head of School The purpose of the final year project was to discuss the theoretical concepts of the analyt- ical features of supervised machine learning. The concepts were illustrated with sample data selected from the open-source platform. The primary objectives of the thesis include explaining the concept of machine learning, presenting supportive libraries, and demon- strating their application or utilization on real data. Finally, the aim was to use machine learning to determine whether a person would pay back their loan or not. The problem was taken as a classification problem. To accomplish the goals, data was collected from an open data source community called Kaggle. The data was downloaded and processed with Python. The complete practical part or machine learning workflow was done in Python and Python’s libraries such as NumPy, Pandas, Matplotlib, Seaborn, and SciPy. For the given problem on sample data, five predictive models were constructed and tested with machine learning with different techniques and combinations. The results were highly accurately matched with the classification problem after evaluating their f1-score, confu- sion matrix, and Jaccard similarity score. Indeed, it is clear that the project achieved the main defined objectives, and the necessary architecture has been set up to apply various analytical abilities into the project.
    [Show full text]
  • Intro to Keras
    Intro to Keras Justin Zhang July 2017 1 Introduction Keras is a high-level Python machine learning API, which allows you to easily run neural networks. Keras is simply a specification; it provides a set of methods that you can use, and it will use a backend (TensorFlow, Theano, or CNTK, as chosen by the user) to actually run your code. Like many machine learning frameworks, Keras is a so-called define-and-run framework. This means that it will define and optimize your neural network in a compilation step before training starts. 2 First Steps First, install Keras: pip install keras We'll go over a fully-connected neural network designed for the MNIST (clas- sifying handwritten digits) dataset. # Can be found at https://github.com/fchollet/keras # Released under the MIT License import keras from keras.datasets import mnist from keras .models import S e q u e n t i a l from keras.layers import Dense, Dropout from keras.optimizers import RMSprop First, we handle the imports. keras.datasets includes many popular machine learning datasets, including CIFAR and IMDB. keras.layers includes a variety of neural network layer types, including Dense, Conv2D, and LSTM. Dense simply refers to a fully-connected layer, and Dropout is a layer commonly used to ad- dress the problem of overfitting. keras.optimizers includes most widely used optimizers. Here, we opt to use RMSprop, an alternative to the classic stochas- tic gradient decent (SGD) algorithm. Regarding keras.models, Sequential is most widely used for simple networks; there is also a functional Model class you can use for more complex networks.
    [Show full text]
  • Auto-Keras: an Efficient Neural Architecture Search System
    Auto-Keras: An Efficient Neural Architecture Search System Haifeng Jin, Qingquan Song, Xia Hu Department of Computer Science and Engineering, Texas A&M University {jin,song_3134,xiahu}@tamu.edu ABSTRACT epochs are required to further train the new architecture towards Neural architecture search (NAS) has been proposed to automat- better performance. Using network morphism would reduce the av- ically tune deep neural networks, but existing search algorithms, erage training time t¯ in neural architecture search. The most impor- e.g., NASNet [41], PNAS [22], usually suffer from expensive com- tant problem to solve for network morphism-based NAS methods is putational cost. Network morphism, which keeps the functional- the selection of operations, which is to select an operation from the ity of a neural network while changing its neural architecture, network morphism operation set to morph an existing architecture could be helpful for NAS by enabling more efficient training during to a new one. The network morphism-based NAS methods are not the search. In this paper, we propose a novel framework enabling efficient enough. They either require a large number of training Bayesian optimization to guide the network morphism for effi- examples [6], or inefficient in exploring the large search space [11]. cient neural architecture search. The framework develops a neural How to perform efficient neural architecture search with network network kernel and a tree-structured acquisition function optimiza- morphism remains a challenging problem. tion algorithm to efficiently explores the search space. Intensive As we know, Bayesian optimization [33] has been widely adopted experiments on real-world benchmark datasets have been done to to efficiently explore black-box functions for global optimization, demonstrate the superior performance of the developed framework whose observations are expensive to obtain.
    [Show full text]
  • Wavefront Parallelization of Recurrent Neural Networks on Multi-Core
    The final publication is available at ACM via http://dx.doi.org/10.1145/3392717.3392762 Wavefront Parallelization of Recurrent Neural Networks on Multi-core Architectures Robin Kumar Sharma Marc Casas Barcelona Supercomputing Center (BSC) Barcelona Supercomputing Center (BSC) Barcelona, Spain Barcelona, Spain [email protected] [email protected] ABSTRACT prevalent choice to analyze sequential and unsegmented data like Recurrent neural networks (RNNs) are widely used for natural text or speech signals. The internal structure of RNN inference and language processing, time-series prediction, or text analysis tasks. training in terms of dependencies across their fundamental numeri- The internal structure of RNNs inference and training in terms of cal kernels complicate the exploitation of model parallelism, which data or control dependencies across their fundamental numerical has not been fully exploited to accelerate forward and backward kernels complicate the exploitation of model parallelism, which is propagation of RNNs on multi-core CPUs. the reason why just data-parallelism has been traditionally applied This paper proposes W-Par, a parallel execution model for RNNs to accelerate RNNs. and its variants LSTMs and GRUs. W-Par exploits all possible con- This paper presents W-Par (Wavefront-Parallelization), a com- currency available in forward and backward propagation routines prehensive approach for RNNs inference and training on CPUs of RNNs. These propagation routines display complex data and that relies on applying model parallelism into RNNs models. We control dependencies that require sophisticated parallel approaches use ne-grained pipeline parallelism in terms of wavefront com- to extract all possible concurrency. W-Par represents RNNs forward putations to accelerate multi-layer RNNs running on multi-core and backward propagation as a computational graph [37] where CPUs.
    [Show full text]
  • Seinfeld's Recent Comments on Zoom…. Energy
    Stanford Data Science Departmental 2/4/2021 Seminar Biomedical AI: Its Roots, Evolution, and Early Days at Stanford Edward H. Shortliffe, MD, PhD Chair Emeritus and Adjunct Professor Department of Biomedical Informatics Columbia University Department of Biomedical Data Science Seminar Stanford Medicine Stanford, California 2:30pm – 4:50pm – February 4, 2021 Seinfeld’s recent comments on Zoom…. Energy, attitude and personality cannot be “remoted” through even the best fiber optic lines Jerry Seinfeld: So You Think New York Is ‘Dead’ (It’s not.) Aug. 24, 2020 © E.H. Shortliffe 1 Stanford Data Science Departmental 2/4/2021 Seminar Disclosures • No conflicts of interest with content of presentation • Offering a historical perspective on medical AI, with an emphasis on US activities in the early days and specific work I know personally • My research career was intense until I became a medical school dean in 2007 • Current research involvement is largely as a textbook author and two decades as editor-in-chief of a peer- reviewed journal (Journal of Biomedical Informatics [Elsevier]) Goals for Today’s Presentation • Show that today’s state of the art in medical AI is part of a 50-year scientific trajectory that is still evolving • Provide some advice to today’s biomedical AI researchers, drawing on that experience • Summarize where we are today in the evolution, with suggestions for future emphasis More details on slides than I can include in talk itself – Full deck available after the talk © E.H. Shortliffe 2 Stanford Data Science Departmental
    [Show full text]
  • Introduction to Keras Tensorflow
    Introduction to Keras TensorFlow Marco Rorro [email protected] CINECA – SCAI SuperComputing Applications and Innovation Department 1/33 Table of Contents Introduction Keras Distributed Deep Learning 2/33 Introduction Keras Distributed Deep Learning 3/33 I computations are expressed as stateful data-flow graphs I automatic differentiation capabilities I optimization algorithms: gradient and proximal gradient based I code portability (CPUs, GPUs, on desktop, server, or mobile computing platforms) I Python interface is the preferred one (Java, C and Go also exist) I installation through: pip, Docker, Anaconda, from sources I Apache 2.0 open-source license TensorFlow I Google Brain’s second generation machine learning system 4/33 I automatic differentiation capabilities I optimization algorithms: gradient and proximal gradient based I code portability (CPUs, GPUs, on desktop, server, or mobile computing platforms) I Python interface is the preferred one (Java, C and Go also exist) I installation through: pip, Docker, Anaconda, from sources I Apache 2.0 open-source license TensorFlow I Google Brain’s second generation machine learning system I computations are expressed as stateful data-flow graphs 4/33 I optimization algorithms: gradient and proximal gradient based I code portability (CPUs, GPUs, on desktop, server, or mobile computing platforms) I Python interface is the preferred one (Java, C and Go also exist) I installation through: pip, Docker, Anaconda, from sources I Apache 2.0 open-source license TensorFlow I Google Brain’s second generation
    [Show full text]