Artificial Intelligence in Robotics
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Backpropagation and Deep Learning in the Brain
Backpropagation and Deep Learning in the Brain Simons Institute -- Computational Theories of the Brain 2018 Timothy Lillicrap DeepMind, UCL With: Sergey Bartunov, Adam Santoro, Jordan Guerguiev, Blake Richards, Luke Marris, Daniel Cownden, Colin Akerman, Douglas Tweed, Geoffrey Hinton The “credit assignment” problem The solution in artificial networks: backprop Credit assignment by backprop works well in practice and shows up in virtually all of the state-of-the-art supervised, unsupervised, and reinforcement learning algorithms. Why Isn’t Backprop “Biologically Plausible”? Why Isn’t Backprop “Biologically Plausible”? Neuroscience Evidence for Backprop in the Brain? A spectrum of credit assignment algorithms: A spectrum of credit assignment algorithms: A spectrum of credit assignment algorithms: How to convince a neuroscientist that the cortex is learning via [something like] backprop - To convince a machine learning researcher, an appeal to variance in gradient estimates might be enough. - But this is rarely enough to convince a neuroscientist. - So what lines of argument help? How to convince a neuroscientist that the cortex is learning via [something like] backprop - What do I mean by “something like backprop”?: - That learning is achieved across multiple layers by sending information from neurons closer to the output back to “earlier” layers to help compute their synaptic updates. How to convince a neuroscientist that the cortex is learning via [something like] backprop 1. Feedback connections in cortex are ubiquitous and modify the -
Memory-Prediction Framework for Pattern
Memory–Prediction Framework for Pattern Recognition: Performance and Suitability of the Bayesian Model of Visual Cortex Saulius J. Garalevicius Department of Computer and Information Sciences, Temple University Room 303, Wachman Hall, 1805 N. Broad St., Philadelphia, PA, 19122, USA [email protected] Abstract The neocortex learns sequences of patterns by storing This paper explores an inferential system for recognizing them in an invariant form in a hierarchical neural network. visual patterns. The system is inspired by a recent memory- It recalls the patterns auto-associatively when given only prediction theory and models the high-level architecture of partial or distorted inputs. The structure of stored invariant the human neocortex. The paper describes the hierarchical representations captures the important relationships in the architecture and recognition performance of this Bayesian world, independent of the details. The primary function of model. A number of possibilities are analyzed for bringing the neocortex is to make predictions by comparing the the model closer to the theory, making it uniform, scalable, knowledge of the invariant structure with the most recent less biased and able to learn a larger variety of images and observed details. their transformations. The effect of these modifications on The regions in the hierarchy are connected by multiple recognition accuracy is explored. We identify and discuss a number of both conceptual and practical challenges to the feedforward and feedback connections. Prediction requires Bayesian approach as well as missing details in the theory a comparison between what is happening (feedforward) that are needed to design a scalable and universal model. and what you expect to happen (feedback). -
Identificação De Textos Em Imagens CAPTCHA Utilizando Conceitos De
Identificação de Textos em Imagens CAPTCHA utilizando conceitos de Aprendizado de Máquina e Redes Neurais Convolucionais Relatório submetido à Universidade Federal de Santa Catarina como requisito para a aprovação da disciplina: DAS 5511: Projeto de Fim de Curso Murilo Rodegheri Mendes dos Santos Florianópolis, Julho de 2018 Identificação de Textos em Imagens CAPTCHA utilizando conceitos de Aprendizado de Máquina e Redes Neurais Convolucionais Murilo Rodegheri Mendes dos Santos Esta monografia foi julgada no contexto da disciplina DAS 5511: Projeto de Fim de Curso e aprovada na sua forma final pelo Curso de Engenharia de Controle e Automação Prof. Marcelo Ricardo Stemmer Banca Examinadora: André Carvalho Bittencourt Orientador na Empresa Prof. Marcelo Ricardo Stemmer Orientador no Curso Prof. Ricardo José Rabelo Responsável pela disciplina Flávio Gabriel Oliveira Barbosa, Avaliador Guilherme Espindola Winck, Debatedor Ricardo Carvalho Frantz do Amaral, Debatedor Agradecimentos Agradeço à minha mãe Terezinha Rodegheri, ao meu pai Orlisses Mendes dos Santos e ao meu irmão Camilo Rodegheri Mendes dos Santos que sempre estiveram ao meu lado, tanto nos momentos de alegria quanto nos momentos de dificuldades, sempre me deram apoio, conselhos, suporte e nunca duvidaram da minha capacidade de alcançar meus objetivos. Agradeço aos meus colegas Guilherme Cornelli, Leonardo Quaini, Matheus Ambrosi, Matheus Zardo, Roger Perin e Victor Petrassi por me acompanharem em toda a graduação, seja nas disciplinas, nos projetos, nas noites de estudo, nas atividades extracurriculares, nas festas, entre outros desafios enfrentados para chegar até aqui. Agradeço aos meus amigos de infância Cássio Schmidt, Daniel Lock, Gabriel Streit, Gabriel Cervo, Guilherme Trevisan, Lucas Nyland por proporcionarem momentos de alegria mesmo a distância na maior parte da caminhada da graduação. -
Unsupervised Anomaly Detection in Time Series with Recurrent Neural Networks
DEGREE PROJECT IN TECHNOLOGY, FIRST CYCLE, 15 CREDITS STOCKHOLM, SWEDEN 2019 Unsupervised anomaly detection in time series with recurrent neural networks JOSEF HADDAD CARL PIEHL KTH ROYAL INSTITUTE OF TECHNOLOGY SCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE Unsupervised anomaly detection in time series with recurrent neural networks JOSEF HADDAD, CARL PIEHL Bachelor in Computer Science Date: June 7, 2019 Supervisor: Pawel Herman Examiner: Örjan Ekeberg School of Electrical Engineering and Computer Science Swedish title: Oövervakad avvikelsedetektion i tidsserier med neurala nätverk iii Abstract Artificial neural networks (ANN) have been successfully applied to a wide range of problems. However, most of the ANN-based models do not attempt to model the brain in detail, but there are still some models that do. An example of a biologically constrained ANN is Hierarchical Temporal Memory (HTM). This study applies HTM and Long Short-Term Memory (LSTM) to anomaly detection problems in time series in order to compare their performance for this task. The shape of the anomalies are restricted to point anomalies and the time series are univariate. Pre-existing implementations that utilise these networks for unsupervised anomaly detection in time series are used in this study. We primarily use our own synthetic data sets in order to discover the networks’ robustness to noise and how they compare to each other regarding different characteristics in the time series. Our results shows that both networks can handle noisy time series and the difference in performance regarding noise robustness is not significant for the time series used in the study. LSTM out- performs HTM in detecting point anomalies on our synthetic time series with sine curve trend but a conclusion about the overall best performing network among these two remains inconclusive. -
The Deep Learning Revolution and Its Implications for Computer Architecture and Chip Design
The Deep Learning Revolution and Its Implications for Computer Architecture and Chip Design Jeffrey Dean Google Research [email protected] Abstract The past decade has seen a remarkable series of advances in machine learning, and in particular deep learning approaches based on artificial neural networks, to improve our abilities to build more accurate systems across a broad range of areas, including computer vision, speech recognition, language translation, and natural language understanding tasks. This paper is a companion paper to a keynote talk at the 2020 International Solid-State Circuits Conference (ISSCC) discussing some of the advances in machine learning, and their implications on the kinds of computational devices we need to build, especially in the post-Moore’s Law-era. It also discusses some of the ways that machine learning may also be able to help with some aspects of the circuit design process. Finally, it provides a sketch of at least one interesting direction towards much larger-scale multi-task models that are sparsely activated and employ much more dynamic, example- and task-based routing than the machine learning models of today. Introduction The past decade has seen a remarkable series of advances in machine learning (ML), and in particular deep learning approaches based on artificial neural networks, to improve our abilities to build more accurate systems across a broad range of areas [LeCun et al. 2015]. Major areas of significant advances include computer vision [Krizhevsky et al. 2012, Szegedy et al. 2015, He et al. 2016, Real et al. 2017, Tan and Le 2019], speech recognition [Hinton et al. -
Neuromorphic Architecture for the Hierarchical Temporal Memory Abdullah M
IEEE TRANSACTIONS ON EMERGING TOPICS IN COMPUTATIONAL INTELLIGENCE 1 Neuromorphic Architecture for the Hierarchical Temporal Memory Abdullah M. Zyarah, Student Member, IEEE, Dhireesha Kudithipudi, Senior Member, IEEE, Neuromorphic AI Laboratory, Rochester Institute of Technology Abstract—A biomimetic machine intelligence algorithm, that recognition and classification [3]–[5], prediction [6], natural holds promise in creating invariant representations of spatiotem- language processing, and anomaly detection [7], [8]. At a poral input streams is the hierarchical temporal memory (HTM). higher abstraction, HTM is basically a memory based system This unsupervised online algorithm has been demonstrated on several machine-learning tasks, including anomaly detection. that can be trained on a sequence of events that vary over time. Significant effort has been made in formalizing and applying the In the algorithmic model, this is achieved using two core units, HTM algorithm to different classes of problems. There are few spatial pooler (SP) and temporal memory (TM), called cortical early explorations of the HTM hardware architecture, especially learning algorithm (CLA). The SP is responsible for trans- for the earlier version of the spatial pooler of HTM algorithm. forming the input data into sparse distributed representation In this article, we present a full-scale HTM architecture for both spatial pooler and temporal memory. Synthetic synapse design is (SDR) with fixed sparsity, whereas the TM learns sequences proposed to address the potential and dynamic interconnections and makes predictions [9]. occurring during learning. The architecture is interweaved with A few research groups have implemented the first generation parallel cells and columns that enable high processing speed for Bayesian HTM. Kenneth et al., in 2007, implemented the the HTM. -
Vikas Sindhwani Google May 17-19, 2016 2016 Summer School on Signal Processing and Machine Learning for Big Data
Real-time Learning and Inference on Emerging Mobile Systems Vikas Sindhwani Google May 17-19, 2016 2016 Summer School on Signal Processing and Machine Learning for Big Data Abstract: We are motivated by the challenge of enabling real-time "always-on" machine learning applications on emerging mobile platforms such as next-generation smartphones, wearable computers and consumer robotics systems. On-device models in such settings need to be highly compact, and need to support fast, low-power inference on specialized hardware. I will consider the problem of building small-footprint non- linear models based on kernel methods and deep learning techniques, for on-device deployments. Towards this end, I will give an overview of various techniques, and introduce new notions of parsimony rooted in the theory of structured matrices. Such structured matrices can be used to recycle Gaussian random vectors in order to build randomized feature maps in sub-linear time for approximating various kernel functions. In the deep learning context, low-displacement structured parameter matrices admit fast function and gradient evaluation. I will discuss how such compact nonlinear transforms span a rich range of parameter sharing configurations whose statistical modeling capacity can be explicitly tuned along a continuum from structured to unstructured. I will present empirical results on mobile speech recognition problems, and image classification tasks. I will also briefly present some basics of TensorFlow: a open-source library for numerical computations on data flow graphs. Tensorflow enables large-scale distributed training of complex machine learning models, and their rapid deployment on mobile devices. Bio: Vikas Sindhwani is Research Scientist in the Google Brain team in New York City. -
ARCHITECTS of INTELLIGENCE for Xiaoxiao, Elaine, Colin, and Tristan ARCHITECTS of INTELLIGENCE
MARTIN FORD ARCHITECTS OF INTELLIGENCE For Xiaoxiao, Elaine, Colin, and Tristan ARCHITECTS OF INTELLIGENCE THE TRUTH ABOUT AI FROM THE PEOPLE BUILDING IT MARTIN FORD ARCHITECTS OF INTELLIGENCE Copyright © 2018 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. Acquisition Editors: Ben Renow-Clarke Project Editor: Radhika Atitkar Content Development Editor: Alex Sorrentino Proofreader: Safis Editing Presentation Designer: Sandip Tadge Cover Designer: Clare Bowyer Production Editor: Amit Ramadas Marketing Manager: Rajveer Samra Editorial Director: Dominic Shakeshaft First published: November 2018 Production reference: 2201118 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK ISBN 978-1-78913-151-2 www.packt.com Contents Introduction ........................................................................ 1 A Brief Introduction to the Vocabulary of Artificial Intelligence .......10 How AI Systems Learn ........................................................11 Yoshua Bengio .....................................................................17 Stuart J. -
Memory-Prediction Framework for Pattern Recognition: Performance
Memory–Prediction Framework for Pattern Recognition: Performance and Suitability of the Bayesian Model of Visual Cortex Saulius J. Garalevicius Department of Computer and Information Sciences, Temple University Room 303, Wachman Hall, 1805 N. Broad St., Philadelphia, PA, 19122, USA [email protected] Abstract The neocortex learns sequences of patterns by storing This paper explores an inferential system for recognizing them in an invariant form in a hierarchical neural network. visual patterns. The system is inspired by a recent memory- It recalls the patterns auto-associatively when given only prediction theory and models the high-level architecture of partial or distorted inputs. The structure of stored invariant the human neocortex. The paper describes the hierarchical representations captures the important relationships in the architecture and recognition performance of this Bayesian world, independent of the details. The primary function of model. A number of possibilities are analyzed for bringing the neocortex is to make predictions by comparing the the model closer to the theory, making it uniform, scalable, knowledge of the invariant structure with the most recent less biased and able to learn a larger variety of images and observed details. their transformations. The effect of these modifications on The regions in the hierarchy are connected by multiple recognition accuracy is explored. We identify and discuss a number of both conceptual and practical challenges to the feedforward and feedback connections. Prediction requires Bayesian approach as well as missing details in the theory a comparison between what is happening (feedforward) that are needed to design a scalable and universal model. and what you expect to happen (feedback). -
BRKIOT-2394.Pdf
Unlocking the Mystery of Machine Learning and Big Data Analytics Robert Barton Jerome Henry Distinguished Architect Principal Engineer @MrRobbarto Office of the CTAO CCIE #6660 @wirelessccie CCDE #2013::6 CCIE #24750 CWNE #45 BRKIOT-2394 Cisco Webex Teams Questions? Use Cisco Webex Teams to chat with the speaker after the session How 1 Find this session in the Cisco Events Mobile App 2 Click “Join the Discussion” 3 Install Webex Teams or go directly to the team space 4 Enter messages/questions in the team space BRKIOT-2394 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 3 Tuesday, Jan. 28th Monday, Jan. 27th Wednesday, Jan. 29th BRKIOT-2600 BRKIOT-2213 16:45 Enabling OT-IT collaboration by 17:00 From Zero to IOx Hero transforming traditional industrial TECIOT-2400 networks to modern IoT Architectures IoT Fundamentals 08:45 BRKIOT-1618 Bootcamp 14:45 Industrial IoT Network Management PSOIOT-1156 16:00 using Cisco Industrial Network Director Securing Industrial – A Deep Dive. Networks: Introduction to Cisco Cyber Vision PSOIOT-2155 Enhancing the Commuter 13:30 BRKIOT-1775 Experience - Service Wireless technologies and 14:30 BRKIOT-2698 BRKIOT-1520 Provider WiFi at the Use Cases in Industrial IOT Industrial IoT Routing – Connectivity 12:15 Cisco Remote & Mobile Asset speed of Trains and Beyond Solutions PSOIOT-2197 Cisco Innovates Autonomous 14:00 TECIOT-2000 Vehicles & Roadways w/ IoT BRKIOT-2497 BRKIOT-2900 Understanding Cisco's 14:30 IoT Solutions for Smart Cities and 11:00 Automating the Network of Internet Of Things (IOT) BRKIOT-2108 Communities Industrial Automation Solutions Connected Factory Architecture Theory and 11:00 Practice PSOIOT-2100 BRKIOT-1291 Unlock New Market 16:15 Opening Keynote 09:00 08:30 Opportunities with LoRaWAN for IOT Enterprises Embedded Cisco services Technologies IOT IOT IOT Track #CLEMEA www.ciscolive.com/emea/learn/technology-tracks.html Cisco Live Thursday, Jan. -
Fast Neural Network Emulation of Dynamical Systems for Computer Animation
Fast Neural Network Emulation of Dynamical Systems for Computer Animation Radek Grzeszczuk 1 Demetri Terzopoulos 2 Geoffrey Hinton 2 1 Intel Corporation 2 University of Toronto Microcomputer Research Lab Department of Computer Science 2200 Mission College Blvd. 10 King's College Road Santa Clara, CA 95052, USA Toronto, ON M5S 3H5, Canada Abstract Computer animation through the numerical simulation of physics-based graphics models offers unsurpassed realism, but it can be computation ally demanding. This paper demonstrates the possibility of replacing the numerical simulation of nontrivial dynamic models with a dramatically more efficient "NeuroAnimator" that exploits neural networks. Neu roAnimators are automatically trained off-line to emulate physical dy namics through the observation of physics-based models in action. De pending on the model, its neural network emulator can yield physically realistic animation one or two orders of magnitude faster than conven tional numerical simulation. We demonstrate NeuroAnimators for a va riety of physics-based models. 1 Introduction Animation based on physical principles has been an influential trend in computer graphics for over a decade (see, e.g., [1, 2, 3]). This is not only due to the unsurpassed realism that physics-based techniques offer. In conjunction with suitable control and constraint mechanisms, physical models also facilitate the production of copious quantities of real istic animation in a highly automated fashion. Physics-based animation techniques are beginning to find their way into high-end commercial systems. However, a well-known drawback has retarded their broader penetration--compared to geometric models, physical models typically entail formidable numerical simulation costs. This paper proposes a new approach to creating physically realistic animation that differs Emulation for Animation 883 radically from the conventional approach of numerically simulating the equations of mo tion of physics-based models. -
Artificial Intelligence and Big Data – Innovation Landscape Brief
ARTIFICIAL INTELLIGENCE AND BIG DATA INNOVATION LANDSCAPE BRIEF © IRENA 2019 Unless otherwise stated, material in this publication may be freely used, shared, copied, reproduced, printed and/or stored, provided that appropriate acknowledgement is given of IRENA as the source and copyright holder. Material in this publication that is attributed to third parties may be subject to separate terms of use and restrictions, and appropriate permissions from these third parties may need to be secured before any use of such material. ISBN 978-92-9260-143-0 Citation: IRENA (2019), Innovation landscape brief: Artificial intelligence and big data, International Renewable Energy Agency, Abu Dhabi. ACKNOWLEDGEMENTS This report was prepared by the Innovation team at IRENA’s Innovation and Technology Centre (IITC) with text authored by Sean Ratka, Arina Anisie, Francisco Boshell and Elena Ocenic. This report benefited from the input and review of experts: Marc Peters (IBM), Neil Hughes (EPRI), Stephen Marland (National Grid), Stephen Woodhouse (Pöyry), Luiz Barroso (PSR) and Dongxia Zhang (SGCC), along with Emanuele Taibi, Nadeem Goussous, Javier Sesma and Paul Komor (IRENA). Report available online: www.irena.org/publications For questions or to provide feedback: [email protected] DISCLAIMER This publication and the material herein are provided “as is”. All reasonable precautions have been taken by IRENA to verify the reliability of the material in this publication. However, neither IRENA nor any of its officials, agents, data or other third- party content providers provides a warranty of any kind, either expressed or implied, and they accept no responsibility or liability for any consequence of use of the publication or material herein.