DOCSLIB.ORG

Neural Networks & Convolutional Neural Networks

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Neural Networks & Convolutional Neural Networks In The Name of Allah ImageNet Classification with Deep Convolutional – AlexNet Alex Krizhevsky - Ilya Sutskever - Geoffrey E. Hinton University of Toronto Iman Jami Moghaddam Digital Image processing Dey 96 3 Outline Problem Definition: Image classification Artificial Neural Networks Convolutional Neural Networks Alex net Problem Definition: Image 4 classification Detailed explanation 5 Classification Problem Given: Training set labeled set of 푁 Input-output pairs 퐷 = 푥푖, 푦푖 , 푦푖 ∈ {1, … , 퐾} , Goal: Given an input 풙, assign it to one of 퐾 classes 6 ImageNet LSVRC-2010 ImageNet is a dataset of over 15 million labeled high-resolution(variable resolutions) images belonging to roughly 22,000 categories. LSVRC uses a subset of ImageNet. The dataset has 1.2 million high- resolution training images and150,000 testing images . The classification task: Get the “correct” class in your top 5 bets. There are 1000 classes. 7 Artificial Neural Networks Detailed explanation 8 A typical neuron There is one axon that branches There is a dendritic tree that collects input from other neurons. A spike of activity in the axon causes charge to be injected into the post-synaptic neuron There is an axon hillock that generates outgoing spikes whenever enough charge has flowed in at synapses to depolarize the cell membrane 9 Synapse When a spike of activity travels along an axon and arrives at a synapse it causes vesicles of transmitter chemical to be released. There are several kinds of transmitter. The transmitter molecules diffuse across the synaptic cleft and bind to receptor molecules in the membrane of the post-synaptic neuron thus changing their shape. 10 How the brain works Each neuron receives inputs from other neurons The effect of each input line on the neuron is controlled by a synaptic weight The synaptic weights adapt so that the whole network learns to perform useful computations You have about 1011neurons each with about 104 weights. 11 Artificial neuron A set of weight W Y = 푗 푤푗푥푗 Output is a function of Y 12 Perceptrons They were popularised by Frank Rosenblatt in the early 1960’s Binary threshold neurons (McCulloch-Pitts) 13 Perceptrons Learning Pick training cases using any policy that ensures that every training case will keep getting picked. If the output unit is correct, leave its weights alone. If the output unit incorrectly outputs a zero, add the input vector to the weight vector. If the output unit incorrectly outputs a 1, subtract the input vector from the weight vector. 14 Limitation of perceptron So “multi-layer” neural networks do not use the perceptron learning procedure. The perceptron convergence procedure works by ensuring that every time the weights change, they get closer to every “generously feasible” set of weights 15 A different look at learning procedure instead of showing the weights get closer to a good set of weights, show that the actual output values get closer the target values. The simplest example is a linear neuron with a squared error measure. 16 Linear neurons y = b + 푖 푥푖푤푖 1 푛 푛 퐸 = (푡 − 푦 )2 2 푛 ∈푡푟표푖푛푖푛 17 Changing weights 18 Non-Linear neurons Sigmoid Tanh ReLU 19 Learning with hidden units Networks without hidden units are very limited in the input-output mappings they can model. Randomly perturb one weight and see if it improves performance. If so, save the change. Very inefficient 20 Backpropagation algorithm We don’t know what the hidden units ought to do, but we can compute how fast the error changes as we change a hidden activity. Instead of using desired activities to train the hidden units, use error derivatives w.r.t. hidden activities. We can compute error derivatives for all the hidden units efficiently at the same time. 21 Changing weights 22 Multi class classification - SVM 23 Multi class classification - Softmax 24 Convolutional Neural Networks Introduction 25 Viewpoint invariance methods Use redundant invariant features: Extract a large, redundant set of features that are invariant under transformations Put a box around the object and use normalized pixels: But choosing the box is difficult because of Segmentation errors, occlusion, unusual orientations. We need to recognize the shape to get the box right! The replicated feature approach (CNN) 26 CNN Convolutional Neural Networks take advantage of the fact that the input consists of images and they constrain the architecture in a more sensible way In particular, unlike a regular Neural Network, the layers of a ConvNet have neurons arranged in 3 dimensions: width, height, depth 27 Example Architecture INPUT [32x32x3] will hold the raw pixel values of the image, in this case an image of width 32, height 32, and with three color channels R,G,B. CONV layer will compute the output of neurons that are connected to local regions in the input, each computing a dot product between their weights and a small region they are connected to in the input volume. This may result in volume such as [32x32x12] if we decided to use 12 filters. RELU layer will apply an elementwise activation function, such as the max(0,x)max(0,x) thresholding at zero. This leaves the size of the volume unchanged ([32x32x12]). POOL layer will perform a downsampling operation along the spatial dimensions (width, height), resulting in volume such as [16x16x12]. FC (i.e. fully-connected) layer will compute the class scores, resulting in volume of size [1x1x10], where each of the 10 numbers correspond to a class score, such as among the 10 categories of CIFAR-10. As with ordinary Neural Networks and as the name implies, each neuron in this layer will be connected to all the numbers in the previous volume. 28 Architecture Overview 29 Convolutional Layer The CONV layer’s parameters consist of a set of learnable filters. Every filter is small spatially (along width and height), but extends through the full depth of the input volume As we slide the filter over the width and height of the input volume we will produce a 2-dimensional activation map that gives the responses of that filter at every spatial position For example, suppose that the input volume has size [32x32x3], (e.g. an RGB CIFAR-10 image). If the receptive field (or the filter size) is 5x5, then each neuron in the Conv Layer will have weights to a [5x5x3] region in the input volume, for a total of 5*5*3 = 75 weights (and +1 bias parameter) 30 Convolution Layer 31 Convolution Layer 32 Convolutional layers 33 Convolutional Layer Three hyperparameters control the size of the output volume: Depth Stride Zero-padding Size of output of each filter: (W−F+2P)/S+1 Backpropagation learning 34 Pulling Layer Its function is to progressively reduce the spatial size of the representation to reduce the amount of parameters and computation in the network, and hence to also control overfitting. The Pooling Layer operates independently on every depth slice of the input and resizes it spatially, using the MAX operation Recall from the backpropagation chapter that the backward pass for a max(x, y) operation has a simple interpretation as only routing the gradient to the input that had the highest value in the forward pass 35 Pulling Layer 36 Compare with fully connected network compared to standard feedforward neural networks with similarly-sized layers, CNNs have much fewer connections and parameters and so they are easier to train. their theoretically-best performance is likely to be only slightly worse. 37 Alex Net 38 Architecture of AlexNet 39 Architecture of AlexNet The first convolutional layer filters the 224×224×3 input image with 96 kernels of size 11×11×3 with a stride of 4 pixels The second convolutional layer takes as input the (response-normalized and pooled) output of the first convolutional layer and filters it with 256 kernels of size 5 × 5 × 48. The third, fourth, and fifth convolutional layers are connected to one another without any intervening pooling or normalization layers The third convolutional layer has 384 kernels of size 3 × 3 × 256 connected to the (normalized, pooled) outputs of the second convolutional layer. The fourth convolutional layer has 384 kernels of size 3 × 3 × 192. the fifth convolutional layer has 256 kernels of size 3 × 3 × 192. The fully-connected layers have 4096 neurons each. 40 ReLU Nonlinearity The standard way to model a neuron’s output f as a function of its input x is with f(x) = tanh(x) these saturating nonlinearities are much slower than the non-saturating nonlinearity f(x) = max(0, x). 41 Training on multiple GPU A single GTX 580 GPU has only 3GB of memory, which limits the maximum size of the networks that can be trained on it. It turns out that 1.2 million training examples are enough to train networks which are too big to fit on one GPU. Therefore we spread the net across two GPUs. 42 Overlapping pooling If we set s = z, we obtain traditional local pooling as commonly employed in CNNs. If we set s < z, we obtain overlapping pooling. This is what we use throughout our network, with s = 2 and z = 3. This scheme reduces the top-1 and top-5 error rates by 0.4% and 0.3%, respectively, as compared with the non- overlapping scheme s = 2, z = 2, which produces output of equivalent dimensions. 43 Intuition of first layer features 44 Results 45 Qualitative Evaluations 46 After AlexNet … 47 Now - Dynamic Routing Between Capsules Geoffrey Hinton and Sara Sabour, holding a two-piece pyramid puzzle, are researching a system that could let computers see more like humans at a Google laboratory in Toronto.
Load more

Recommended publications
  • Backpropagation and Deep Learning in the Brain
    Backpropagation and Deep Learning in the Brain Simons Institute -- Computational Theories of the Brain 2018 Timothy Lillicrap DeepMind, UCL With: Sergey Bartunov, Adam Santoro, Jordan Guerguiev, Blake Richards, Luke Marris, Daniel Cownden, Colin Akerman, Douglas Tweed, Geoffrey Hinton The “credit assignment” problem The solution in artificial networks: backprop Credit assignment by backprop works well in practice and shows up in virtually all of the state-of-the-art supervised, unsupervised, and reinforcement learning algorithms. Why Isn’t Backprop “Biologically Plausible”? Why Isn’t Backprop “Biologically Plausible”? Neuroscience Evidence for Backprop in the Brain? A spectrum of credit assignment algorithms: A spectrum of credit assignment algorithms: A spectrum of credit assignment algorithms: How to convince a neuroscientist that the cortex is learning via [something like] backprop - To convince a machine learning researcher, an appeal to variance in gradient estimates might be enough. - But this is rarely enough to convince a neuroscientist. - So what lines of argument help? How to convince a neuroscientist that the cortex is learning via [something like] backprop - What do I mean by “something like backprop”?: - That learning is achieved across multiple layers by sending information from neurons closer to the output back to “earlier” layers to help compute their synaptic updates. How to convince a neuroscientist that the cortex is learning via [something like] backprop 1. Feedback connections in cortex are ubiquitous and modify the
    [Show full text]
  • The Deep Learning Revolution and Its Implications for Computer Architecture and Chip Design
    The Deep Learning Revolution and Its Implications for Computer Architecture and Chip Design Jeffrey Dean Google Research [email protected] Abstract The past decade has seen a remarkable series of advances in machine learning, and in particular deep learning approaches based on artificial neural networks, to improve our abilities to build more accurate systems across a broad range of areas, including computer vision, speech recognition, language translation, and natural language understanding tasks. This paper is a companion paper to a keynote talk at the 2020 International Solid-State Circuits Conference (ISSCC) discussing some of the advances in machine learning, and their implications on the kinds of computational devices we need to build, especially in the post-Moore’s Law-era. It also discusses some of the ways that machine learning may also be able to help with some aspects of the circuit design process. Finally, it provides a sketch of at least one interesting direction towards much larger-scale multi-task models that are sparsely activated and employ much more dynamic, example- and task-based routing than the machine learning models of today. Introduction The past decade has seen a remarkable series of advances in machine learning (ML), and in particular deep learning approaches based on artificial neural networks, to improve our abilities to build more accurate systems across a broad range of areas [LeCun et al. 2015]. Major areas of significant advances ​ ​ include computer vision [Krizhevsky et al. 2012, Szegedy et al. 2015, He et al. 2016, Real et al. 2017, Tan ​ ​ ​ ​ ​ ​ and Le 2019], speech recognition [Hinton et al.
    [Show full text]
  • ARCHITECTS of INTELLIGENCE for Xiaoxiao, Elaine, Colin, and Tristan ARCHITECTS of INTELLIGENCE
    MARTIN FORD ARCHITECTS OF INTELLIGENCE For Xiaoxiao, Elaine, Colin, and Tristan ARCHITECTS OF INTELLIGENCE THE TRUTH ABOUT AI FROM THE PEOPLE BUILDING IT MARTIN FORD ARCHITECTS OF INTELLIGENCE Copyright © 2018 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. Acquisition Editors: Ben Renow-Clarke Project Editor: Radhika Atitkar Content Development Editor: Alex Sorrentino Proofreader: Safis Editing Presentation Designer: Sandip Tadge Cover Designer: Clare Bowyer Production Editor: Amit Ramadas Marketing Manager: Rajveer Samra Editorial Director: Dominic Shakeshaft First published: November 2018 Production reference: 2201118 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK ISBN 978-1-78913-151-2 www.packt.com Contents Introduction ........................................................................ 1 A Brief Introduction to the Vocabulary of Artificial Intelligence .......10 How AI Systems Learn ........................................................11 Yoshua Bengio .....................................................................17 Stuart J.
    [Show full text]
  • Fast Neural Network Emulation of Dynamical Systems for Computer Animation
    Fast Neural Network Emulation of Dynamical Systems for Computer Animation Radek Grzeszczuk 1 Demetri Terzopoulos 2 Geoffrey Hinton 2 1 Intel Corporation 2 University of Toronto Microcomputer Research Lab Department of Computer Science 2200 Mission College Blvd. 10 King's College Road Santa Clara, CA 95052, USA Toronto, ON M5S 3H5, Canada Abstract Computer animation through the numerical simulation of physics-based graphics models offers unsurpassed realism, but it can be computation­ ally demanding. This paper demonstrates the possibility of replacing the numerical simulation of nontrivial dynamic models with a dramatically more efficient "NeuroAnimator" that exploits neural networks. Neu­ roAnimators are automatically trained off-line to emulate physical dy­ namics through the observation of physics-based models in action. De­ pending on the model, its neural network emulator can yield physically realistic animation one or two orders of magnitude faster than conven­ tional numerical simulation. We demonstrate NeuroAnimators for a va­ riety of physics-based models. 1 Introduction Animation based on physical principles has been an influential trend in computer graphics for over a decade (see, e.g., [1, 2, 3]). This is not only due to the unsurpassed realism that physics-based techniques offer. In conjunction with suitable control and constraint mechanisms, physical models also facilitate the production of copious quantities of real­ istic animation in a highly automated fashion. Physics-based animation techniques are beginning to find their way into high-end commercial systems. However, a well-known drawback has retarded their broader penetration--compared to geometric models, physical models typically entail formidable numerical simulation costs. This paper proposes a new approach to creating physically realistic animation that differs Emulation for Animation 883 radically from the conventional approach of numerically simulating the equations of mo­ tion of physics-based models.
    [Show full text]
  • Neural Networks for Machine Learning Lecture 4A Learning To
    Neural Networks for Machine Learning Lecture 4a Learning to predict the next word Geoffrey Hinton with Nitish Srivastava Kevin Swersky A simple example of relational information Christopher = Penelope Andrew = Christine Margaret = Arthur Victoria = James Jennifer = Charles Colin Charlotte Roberto = Maria Pierro = Francesca Gina = Emilio Lucia = Marco Angela = Tomaso Alfonso Sophia Another way to express the same information • Make a set of propositions using the 12 relationships: – son, daughter, nephew, niece, father, mother, uncle, aunt – brother, sister, husband, wife • (colin has-father james) • (colin has-mother victoria) • (james has-wife victoria) this follows from the two above • (charlotte has-brother colin) • (victoria has-brother arthur) • (charlotte has-uncle arthur) this follows from the above A relational learning task • Given a large set of triples that come from some family trees, figure out the regularities. – The obvious way to express the regularities is as symbolic rules (x has-mother y) & (y has-husband z) => (x has-father z) • Finding the symbolic rules involves a difficult search through a very large discrete space of possibilities. • Can a neural network capture the same knowledge by searching through a continuous space of weights? The structure of the neural net local encoding of person 2 output distributed encoding of person 2 units that learn to predict features of the output from features of the inputs distributed encoding of person 1 distributed encoding of relationship local encoding of person 1 inputs local encoding of relationship Christopher = Penelope Andrew = Christine Margaret = Arthur Victoria = James Jennifer = Charles Colin Charlotte What the network learns • The six hidden units in the bottleneck connected to the input representation of person 1 learn to represent features of people that are useful for predicting the answer.
    [Show full text]
  • Lecture 10: Recurrent Neural Networks
    Lecture 10: Recurrent Neural Networks Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 10 - 1 May 4, 2017 Administrative A1 grades will go out soon A2 is due today (11:59pm) Midterm is in-class on Tuesday! We will send out details on where to go soon Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 10 - 2 May 4, 2017 Extra Credit: Train Game More details on Piazza by early next week Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 10 - 3 May 4, 2017 Last Time: CNN Architectures AlexNet Figure copyright Kaiming He, 2016. Reproduced with permission. Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 10 - 4 May 4, 2017 Last Time: CNN Architectures Softmax FC 1000 Softmax FC 4096 FC 1000 FC 4096 FC 4096 Pool FC 4096 3x3 conv, 512 Pool 3x3 conv, 512 3x3 conv, 512 3x3 conv, 512 3x3 conv, 512 3x3 conv, 512 3x3 conv, 512 Pool Pool 3x3 conv, 512 3x3 conv, 512 3x3 conv, 512 3x3 conv, 512 3x3 conv, 512 3x3 conv, 512 3x3 conv, 512 Pool Pool 3x3 conv, 256 3x3 conv, 256 3x3 conv, 256 3x3 conv, 256 Pool Pool 3x3 conv, 128 3x3 conv, 128 3x3 conv, 128 3x3 conv, 128 Pool Pool 3x3 conv, 64 3x3 conv, 64 3x3 conv, 64 3x3 conv, 64 Input Input VGG16 VGG19 GoogLeNet Figure copyright Kaiming He, 2016. Reproduced with permission. Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 10 - 5 May 4, 2017 Last Time: CNN Architectures Softmax FC 1000 Pool 3x3 conv, 64 3x3 conv, 64 3x3 conv, 64 relu 3x3 conv, 64 3x3 conv, 64 F(x) + x 3x3 conv, 64 ..
    [Show full text]
  • Image Retrieval Algorithm Based on Convolutional Neural Network
    Advances in Intelligent Systems Research, volume 133 2nd International Conference on Artificial Intelligence and Industrial Engineering (AIIE2016) Image Retrieval Algorithm Based on Convolutional Neural Network Hailong Liu 1, 2, Baoan Li 1, 2, *, Xueqiang Lv 1 and Yue Huang 3 1Beijing Key Laboratory of Internet Culture and Digital Dissemination Research, Beijing Information Science & Technology University, Beijing 100101, China 2Computer School, Beijing Information Science and Technology University, Beijing 100101, China 3Xuanwu Hospital Capital Medical University, 100053, China *Corresponding author Abstract—With the rapid development of computer technology low, can’t meet the needs of people. In order to overcome this and the increasing of multimedia data on the Internet, how to difficulty, the researchers from the image itself, and put quickly find the desired information in the massive data becomes forward the method of image retrieval based on content. The a hot issue. Image retrieval can be used to retrieve similar images, method is to extract the visual features of the image content: and the effect of image retrieval depends on the selection of image color, texture, shape etc., the image database to be detected features to a certain extent. Based on deep learning, through self- samples for similarity matching, retrieval and sample images learning ability of a convolutional neural network to extract more are similar to the image. The main process of the method is the conducive to the high-level semantic feature of image retrieval selection and extraction of features, but there are "semantic using convolutional neural network, and then use the distance gap" [2] between low-level features and high-level semantic metric function similar image.
    [Show full text]
  • Lecture Notes Geoffrey Hinton
    Lecture Notes Geoffrey Hinton Overjoyed Luce crops vectorially. Tailor write-ups his glasshouse divulgating unmanly or constructively after Marcellus barb and outdriven squeakingly, diminishable and cespitose. Phlegmatical Laurance contort thereupon while Bruce always dimidiating his melancholiac depresses what, he shores so finitely. For health care about working memory networks are the class and geoffrey hinton and modify or are A practical guide to training restricted boltzmann machines Lecture Notes in. Trajectory automatically learn about different domains, geoffrey explained what a lecture notes geoffrey hinton notes was central bottleneck form of data should take much like a single example, geoffrey hinton with mctsnets. Gregor sieber and password you may need to this course that models decisions. Jimmy Ba Geoffrey Hinton Volodymyr Mnih Joel Z Leibo and Catalin Ionescu. YouTube lectures by Geoffrey Hinton3 1 Introduction In this topic of boosting we combined several simple classifiers into very complex classifier. Separating Figure from stand with a Parallel Network Paul. But additionally has efficient. But everett also look up. You already know how the course that is just to my assignment in the page and writer recognition and trends in the effect of the motivating this. These perplex the or free liquid Intelligence educational. Citation to lose sight of language. Sparse autoencoder CS294A Lecture notes Andrew Ng Stanford University. Geoffrey Hinton on what's nothing with CNNs Daniel C Elton. Toronto Geoffrey Hinton Advanced Machine Learning CSC2535 2011 Spring. Cross validated is sparse, a proof of how to list of possible configurations have to see. Cnns that just download a computational power nor the squared error in permanent electrode array with respect to the.
    [Show full text]
  • The Dartmouth College Artificial Intelligence Conference: the Next
    AI Magazine Volume 27 Number 4 (2006) (© AAAI) Reports continued this earlier work because he became convinced that advances The Dartmouth College could be made with other approaches using computers. Minsky expressed the concern that too many in AI today Artificial Intelligence try to do what is popular and publish only successes. He argued that AI can never be a science until it publishes what fails as well as what succeeds. Conference: Oliver Selfridge highlighted the im- portance of many related areas of re- search before and after the 1956 sum- The Next mer project that helped to propel AI as a field. The development of improved languages and machines was essential. Fifty Years He offered tribute to many early pio- neering activities such as J. C. R. Lick- leiter developing time-sharing, Nat Rochester designing IBM computers, and Frank Rosenblatt working with James Moor perceptrons. Trenchard More was sent to the summer project for two separate weeks by the University of Rochester. Some of the best notes describing the AI project were taken by More, al- though ironically he admitted that he ■ The Dartmouth College Artificial Intelli- Marvin Minsky, Claude Shannon, and never liked the use of “artificial” or gence Conference: The Next 50 Years Nathaniel Rochester for the 1956 “intelligence” as terms for the field. (AI@50) took place July 13–15, 2006. The event, McCarthy wanted, as he ex- Ray Solomonoff said he went to the conference had three objectives: to cele- plained at AI@50, “to nail the flag to brate the Dartmouth Summer Research summer project hoping to convince the mast.” McCarthy is credited for Project, which occurred in 1956; to as- everyone of the importance of ma- coining the phrase “artificial intelli- sess how far AI has progressed; and to chine learning.
    [Show full text]
  • LSTM-In-LSTM for Generating Long Descriptions of Images
    Computational Visual Media DOI 10.1007/s41095-016-0059-z Vol. 2, No. 4, December 2016, 379–388 Research Article LSTM-in-LSTM for generating long descriptions of images Jun Song1, Siliang Tang1, Jun Xiao1, Fei Wu1( ), and Zhongfei (Mark) Zhang2 c The Author(s) 2016. This article is published with open access at Springerlink.com Abstract In this paper, we propose an approach by means of text (description generation) is a for generating rich fine-grained textual descriptions of fundamental task in artificial intelligence, with many images. In particular, we use an LSTM-in-LSTM (long applications. For example, generating descriptions short-term memory) architecture, which consists of an of images may help visually impaired people better inner LSTM and an outer LSTM. The inner LSTM understand the content of images and retrieve images effectively encodes the long-range implicit contextual using descriptive texts. The challenge of description interaction between visual cues (i.e., the spatially- generation lies in appropriately developing a model concurrent visual objects), while the outer LSTM that can effectively represent the visual cues in generally captures the explicit multi-modal relationship images and describe them in the domain of natural between sentences and images (i.e., the correspondence language at the same time. of sentences and images). This architecture is capable There have been significant advances in of producing a long description by predicting one description generation recently. Some efforts word at every time step conditioned on the previously rely on manually-predefined visual concepts and generated word, a hidden vector (via the outer LSTM), and a context vector of fine-grained visual cues (via sentence templates [1–3].
    [Show full text]
  • CNN Encoder to Reduce the Dimensionality of Data Image For
    CNN Encoder to Reduce the Dimensionality of Data Image for Motion Planning Janderson Ferreira1, Agostinho A. F. J´unior1, Yves M. Galv˜ao1 1 1 2 ∗ Bruno J. T. Fernandes and Pablo Barros , 1- Universidade de Pernambuco - Escola Polit´ecnica de Pernambuco Rua Benfica 455, Recife/PE - Brazil 2- Cognitive Architecture for Collaborative Technologies (CONTACT) Unit Istituto Italiano di Tecnologia, Genova, Italy Abstract. Many real-world applications need path planning algorithms to solve tasks in different areas, such as social applications, autonomous cars, and tracking activities. And most importantly motion planning. Although the use of path planning is sufficient in most motion planning scenarios, they represent potential bottlenecks in large environments with dynamic changes. To tackle this problem, the number of possible routes could be reduced to make it easier for path planning algorithms to find the shortest path with less efforts. An traditional algorithm for path planning is the A*, it uses an heuristic to work faster than other solutions. In this work, we propose a CNN encoder capable of eliminating useless routes for motion planning problems, then we combine the proposed neural network output with A*. To measure the efficiency of our solution, we propose a database with different scenarios of motion planning problems. The evaluated metric is the number of the iterations to find the shortest path. The A* was compared with the CNN Encoder (proposal) with A*. In all evaluated scenarios, our solution reduced the number of iterations by more than 60%. 1 Introduction Present in many daily applications, path planning algorithm helps to solve many navigation problems.
    [Show full text]
  • The History Began from Alexnet: a Comprehensive Survey on Deep Learning Approaches
    > REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 1 The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches Md Zahangir Alom1, Tarek M. Taha1, Chris Yakopcic1, Stefan Westberg1, Paheding Sidike2, Mst Shamima Nasrin1, Brian C Van Essen3, Abdul A S. Awwal3, and Vijayan K. Asari1 Abstract—In recent years, deep learning has garnered I. INTRODUCTION tremendous success in a variety of application domains. This new ince the 1950s, a small subset of Artificial Intelligence (AI), field of machine learning has been growing rapidly, and has been applied to most traditional application domains, as well as some S often called Machine Learning (ML), has revolutionized new areas that present more opportunities. Different methods several fields in the last few decades. Neural Networks have been proposed based on different categories of learning, (NN) are a subfield of ML, and it was this subfield that spawned including supervised, semi-supervised, and un-supervised Deep Learning (DL). Since its inception DL has been creating learning. Experimental results show state-of-the-art performance ever larger disruptions, showing outstanding success in almost using deep learning when compared to traditional machine every application domain. Fig. 1 shows, the taxonomy of AI. learning approaches in the fields of image processing, computer DL (using either deep architecture of learning or hierarchical vision, speech recognition, machine translation, art, medical learning approaches) is a class of ML developed largely from imaging, medical information processing, robotics and control, 2006 onward. Learning is a procedure consisting of estimating bio-informatics, natural language processing (NLP), cybersecurity, and many others.
    [Show full text]