Neuromorphic Engineering - Wikipedia
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Neural Network Transformation and Co-Design Under Neuromorphic Hardware Constraints
NEUTRAMS: Neural Network Transformation and Co-design under Neuromorphic Hardware Constraints Yu Ji ∗, YouHui Zhang∗‡, ShuangChen Li†,PingChi†, CiHang Jiang∗,PengQu∗,YuanXie†, WenGuang Chen∗‡ Email: [email protected], [email protected] ∗Department of Computer Science and Technology, Tsinghua University, PR.China † Department of Electrical and Computer Engineering, University of California at Santa Barbara, USA ‡ Center for Brain-Inspired Computing Research, Tsinghua University, PR.China Abstract—With the recent reincarnations of neuromorphic (CMP)1 with a Network-on-Chip (NoC) has emerged as a computing comes the promise of a new computing paradigm, promising platform for neural network simulation, motivating with a focus on the design and fabrication of neuromorphic chips. many recent studies of neuromorphic chips [4]–[10]. The A key challenge in design, however, is that programming such chips is difficult. This paper proposes a systematic methodology design of neuromorphic chips is promising to enable a new with a set of tools to address this challenge. The proposed computing paradigm. toolset is called NEUTRAMS (Neural network Transformation, One major issue is that neuromorphic hardware usually Mapping and Simulation), and includes three key components: places constraints on NN models (such as the maximum fan- a neural network (NN) transformation algorithm, a configurable in/fan-out of a single neuron, the limited range of synaptic clock-driven simulator of neuromorphic chips and an optimized runtime tool that maps NNs onto the target hardware for weights) that it can support, and the hardware types of neurons better resource utilization. To address the challenges of hardware or activation functions are usually simpler than the software constraints on implementing NN models (such as the maximum counterparts. -
Image Recognition by Spiking Neural Networks
TALLINN UNIVERSITY OF TECHNOLOGY Faculty of Information Technology Nikolai Jefimov 143689IASM IMAGE RECOGNITION BY SPIKING NEURAL NETWORKS Master's thesis Supervisor: Eduard Petlenkov PhD Tallinn 2017 TALLINNA TEHNIKAÜLIKOOL Infotehnoloogia teaduskond Nikolai Jefimov 143689IASM KUJUTISE TUVASTAMINE IMPULSI NÄRVIVÕRKUDEGA Magistritöö Juhendaja: Eduard Petlenkov PhD Tallinn 2017 Author’s declaration of originality I hereby certify that I am the sole author of this thesis. All the used materials, references to the literature and the work of others have been referred to. This thesis has not been presented for examination anywhere else. Author: Nikolai Jefimov 30.05.2017 3 Abstract The aim of this thesis is to present opportunities on implementations of spiking neural network for image recognition and proving learning algorithms in MATLAB environment. In this thesis provided examples of the use of common types of artificial neural networks for image recognition and classification. Also presented spiking neural network areas of use, as well as the principles of image recognition by this type of neural network. The result of this work is the realization of the learning algorithm in MATLAB environment, as well as the ability to recognize handwritten numbers from the MNIST database. Besides this presented the opportunity to recognizing Latin letters with dimensions of 3x3 and 5x7 pixels. In addition, were proposed User Guide for masters course ISS0023 Intelligent Control System in TUT. This thesis is written in English and is 62 pages long, including 6 chapters, 61 figures and 4 tables. 4 Annotatsioon Kujutise tuvastamine impulsi närvivõrkudega Selle väitekiri eesmärgiks on tutvustada impulsi närvivõrgu kasutamisvõimalused, samuti näidata õpetamise algoritmi MATLAB keskkonnas. -
Real-Time System for Driver Fatigue Detection Based on a Recurrent Neuronal Network
Journal of Imaging Article Real-Time System for Driver Fatigue Detection Based on a Recurrent Neuronal Network Younes Ed-Doughmi 1,* , Najlae Idrissi 1 and Youssef Hbali 2 1 Department Computer Science, FST, University Sultan Moulay Sliman, 23000 Beni Mellal, Morocco; [email protected] 2 Computer Systems Engineering Laboratory Cadi Ayyad University, Faculty of Sciences Semlalia, 40000 Marrakech, Morocco; [email protected] * Correspondence: [email protected] Received: 6 January 2020; Accepted: 25 February 2020; Published: 4 March 2020 Abstract: In recent years, the rise of car accident fatalities has grown significantly around the world. Hence, road security has become a global concern and a challenging problem that needs to be solved. The deaths caused by road accidents are still increasing and currently viewed as a significant general medical issue. The most recent developments have made in advancing knowledge and scientific capacities of vehicles, enabling them to see and examine street situations to counteract mishaps and secure travelers. Therefore, the analysis of driver’s behaviors on the road has become one of the leading research subjects in recent years, particularly drowsiness, as it grants the most elevated factor of mishaps and is the primary source of death on roads. This paper presents a way to analyze and anticipate driver drowsiness by applying a Recurrent Neural Network over a sequence frame driver’s face. We used a dataset to shape and approve our model and implemented repetitive neural network architecture multi-layer model-based 3D Convolutional Networks to detect driver drowsiness. After a training session, we obtained a promising accuracy that approaches a 92% acceptance rate, which made it possible to develop a real-time driver monitoring system to reduce road accidents. -
Deep Learning with Tensorflow 2 and Keras Second Edition
Deep Learning with TensorFlow 2 and Keras Second Edition Regression, ConvNets, GANs, RNNs, NLP, and more with TensorFlow 2 and the Keras API Antonio Gulli Amita Kapoor Sujit Pal BIRMINGHAM - MUMBAI Deep Learning with TensorFlow 2 and Keras Second Edition Copyright © 2019 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. Commissioning Editor: Amey Varangaonkar Acquisition Editors: Yogesh Deokar, Ben Renow-Clarke Acquisition Editor – Peer Reviews: Suresh Jain Content Development Editor: Ian Hough Technical Editor: Gaurav Gavas Project Editor: Janice Gonsalves Proofreader: Safs Editing Indexer: Rekha Nair Presentation Designer: Sandip Tadge First published: April 2017 Second edition: December 2019 Production reference: 2130320 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-83882-341-2 www.packt.com packt.com Subscribe to our online digital library for full access to over 7,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. -
A Sparsity-Driven Backpropagation-Less Learning Framework Using Populations of Spiking Growth Transform Neurons
ORIGINAL RESEARCH published: 28 July 2021 doi: 10.3389/fnins.2021.715451 A Sparsity-Driven Backpropagation-Less Learning Framework Using Populations of Spiking Growth Transform Neurons Ahana Gangopadhyay and Shantanu Chakrabartty* Department of Electrical and Systems Engineering, Washington University in St. Louis, St. Louis, MO, United States Growth-transform (GT) neurons and their population models allow for independent control over the spiking statistics and the transient population dynamics while optimizing a physically plausible distributed energy functional involving continuous-valued neural variables. In this paper we describe a backpropagation-less learning approach to train a Edited by: network of spiking GT neurons by enforcing sparsity constraints on the overall network Siddharth Joshi, spiking activity. The key features of the model and the proposed learning framework University of Notre Dame, United States are: (a) spike responses are generated as a result of constraint violation and hence Reviewed by: can be viewed as Lagrangian parameters; (b) the optimal parameters for a given task Wenrui Zhang, can be learned using neurally relevant local learning rules and in an online manner; University of California, Santa Barbara, United States (c) the network optimizes itself to encode the solution with as few spikes as possible Youhui Zhang, (sparsity); (d) the network optimizes itself to operate at a solution with the maximum Tsinghua University, China dynamic range and away from saturation; and (e) the framework is flexible enough to *Correspondence: incorporate additional structural and connectivity constraints on the network. As a result, Shantanu Chakrabartty [email protected] the proposed formulation is attractive for designing neuromorphic tinyML systems that are constrained in energy, resources, and network structure. -
Multilayer Spiking Neural Network for Audio Samples Classification Using
Multilayer Spiking Neural Network for Audio Samples Classification Using SpiNNaker Juan Pedro Dominguez-Morales, Angel Jimenez-Fernandez, Antonio Rios-Navarro, Elena Cerezuela-Escudero, Daniel Gutierrez-Galan, Manuel J. Dominguez-Morales, and Gabriel Jimenez-Moreno Robotic and Technology of Computers Lab, Department of Architecture and Technology of Computers, University of Seville, Seville, Spain {jpdominguez,ajimenez,arios,ecerezuela, dgutierrez,mdominguez,gaji}@atc.us.es http://www.atc.us.es Abstract. Audio classification has always been an interesting subject of research inside the neuromorphic engineering field. Tools like Nengo or Brian, and hard‐ ware platforms like the SpiNNaker board are rapidly increasing in popularity in the neuromorphic community due to the ease of modelling spiking neural networks with them. In this manuscript a multilayer spiking neural network for audio samples classification using SpiNNaker is presented. The network consists of different leaky integrate-and-fire neuron layers. The connections between them are trained using novel firing rate based algorithms and tested using sets of pure tones with frequencies that range from 130.813 to 1396.91 Hz. The hit rate percentage values are obtained after adding a random noise signal to the original pure tone signal. The results show very good classification results (above 85 % hit rate) for each class when the Signal-to-noise ratio is above 3 decibels, vali‐ dating the robustness of the network configuration and the training step. Keywords: SpiNNaker · Spiking neural network · Audio samples classification · Spikes · Neuromorphic auditory sensor · Address-Event Representation 1 Introduction Neuromorphic engineering is a discipline that studies, designs and implements hardware and software with the aim of mimicking the way in which nervous systems work, focusing its main inspiration on how the brain solves complex problems easily. -
Neuromorphic Computing Using Emerging Synaptic Devices: a Retrospective Summary and an Outlook
electronics Review Neuromorphic Computing Using Emerging Synaptic Devices: A Retrospective Summary and an Outlook Jaeyoung Park 1,2 1 School of Computer Science and Electronic Engineering, Handong Global University, Pohang 37554, Korea; [email protected]; Tel.: +82-54-260-1394 2 School of Electronic Engineering, Soongsil University, Seoul 06978, Korea Received: 29 June 2020; Accepted: 5 August 2020; Published: 1 September 2020 Abstract: In this paper, emerging memory devices are investigated for a promising synaptic device of neuromorphic computing. Because the neuromorphic computing hardware requires high memory density, fast speed, and low power as well as a unique characteristic that simulates the function of learning by imitating the process of the human brain, memristor devices are considered as a promising candidate because of their desirable characteristic. Among them, Phase-change RAM (PRAM) Resistive RAM (ReRAM), Magnetic RAM (MRAM), and Atomic Switch Network (ASN) are selected to review. Even if the memristor devices show such characteristics, the inherent error by their physical properties needs to be resolved. This paper suggests adopting an approximate computing approach to deal with the error without degrading the advantages of emerging memory devices. Keywords: neuromorphic computing; memristor; PRAM; ReRAM; MRAM; ASN; synaptic device 1. Introduction Artificial Intelligence (AI), called machine intelligence, has been researched over the decades, and the AI now becomes an essential part of the industry [1–3]. Artificial intelligence means machines that imitate the cognitive functions of humans such as learning and problem solving [1,3]. Similarly, neuromorphic computing has been researched that imitates the process of the human brain compute and store [4–6]. -
Using Memristors in a Functional Spiking-Neuron Model of Activity-Silent Working Memory
Using memristors in a functional spiking-neuron model of activity-silent working memory Joppe Wik Boekestijn (s2754215) July 2, 2021 Internal Supervisor(s): Dr. Jelmer Borst (Artificial Intelligence, University of Groningen) Second Internal Supervisor: Thomas Tiotto (Artificial Intelligence, University of Groningen) Artificial Intelligence University of Groningen, The Netherlands Abstract In this paper, a spiking-neuron model of human working memory by Pals et al. (2020) is adapted to use Nb-doped SrTiO3 memristors in the underlying architecture. Memristors are promising devices in neuromorphic computing due to their ability to simulate synapses of ar- tificial neuron networks in an efficient manner. The spiking-neuron model introduced by Pals et al. (2020) learns by means of short-term synaptic plasticity (STSP). In this mechanism neu- rons are adapted to incorporate a calcium and resources property. Here a novel learning rule, mSTSP, is introduced, where the calcium property is effectively programmed on memristors. The model performs a delayed-response task. Investigating the neural activity and performance of the model with the STSP or mSTSP learning rules shows remarkable similarities, meaning that memristors can be successfully used in a spiking-neuron model that has shown functional human behaviour. Shortcomings of the Nb-doped SrTiO3 memristor prevent programming the entire spiking-neuron model on memristors. A promising new memristive device, the diffusive memristor, might prove to be ideal for realising the entire model on hardware directly. 1 Introduction In recent years it has become clear that computers are using an ever-increasing number of resources. Up until 2011 advancements in the semiconductor industry followed ‘Moore’s law’, which states that every second year the number of transistors doubles, for the same cost, in an integrated circuit (IC). -
Neuromorphic Processing and Sensing
1 Neuromorphic Processing & Sensing: Evolutionary Progression of AI to Spiking Philippe Reiter, Geet Rose Jose, Spyridon Bizmpikis, Ionela-Ancuța Cîrjilă machine learning and deep learning (MLDL) developments. Abstract— The increasing rise in machine learning and deep This accelerated pace of innovation was spurred on by the learning applications is requiring ever more computational seminal works of LeCun, Hinton and others in the 1990s on resources to successfully meet the growing demands of an always- convolutional neural networks, or CNNs. Since then, numerous, connected, automated world. Neuromorphic technologies based on more advanced learning models have been developed, and Spiking Neural Network algorithms hold the promise to implement advanced artificial intelligence using a fraction of the neural networks have become integral to industry, medicine, computations and power requirements by modeling the academia and commercial devices. From fully autonomous functioning, and spiking, of the human brain. With the vehicles to rapid facial recognition entering popular culture to proliferation of tools and platforms aiding data scientists and enumerable innovations across almost all domains, it is not an machine learning engineers to develop the latest innovations in exaggeration to claim that CNNs and their progeny have artificial and deep neural networks, a transition to a new paradigm will require building from the current well-established changed both the technological and physical worlds. foundations. This paper explains the theoretical workings of CNNs can be exceedingly computationally intensive, neuromorphic technologies based on spikes, and overviews the however, as will be explained, often limiting their use to high- state-of-art in hardware processors, software platforms and performance computers and large data centres. -
Neuromorphic Computing in Ginzburg-Landau Lattice Systems
Neuromorphic computing in Ginzburg-Landau polariton lattice systems Andrzej Opala and Michał Matuszewski Institute of Physics, Polish Academy of Sciences, Warsaw Sanjib Ghosh and Timothy C. H. Liew Division of Physics and Applied Physics, Nanyang Technological University, Singapore < Andrzej Opala and Michał Matuszewski Institute of Physics, Polish Academy of Sciences, Warsaw Sanjib Ghosh and Timothy C. H. Liew Division of Physics and Applied Physics, Nanyang Technological University, Singapore < Machine learning with neural networks What is machine learning? Modern computers are extremely efficient in solving many problems, but some tasks are very hard to implement. ● It is difficult to write an algorithm that would recognize an object from different viewpoints or in di erent sceneries, lighting conditions etc. ● It is difficult to detect a fradulent credit card transaction. !e need to combine a large number of weak rules with complex dependencies rather than follow simple and reliable rules. "hese tasks are often relatively easy for humans. Machine learning with neural networks Machine learning algorithm is not written to solve a speci#c task, but to learn how to classify / detect / predict on its own. ● A large number of examples is collected and used as an input for the algorithm during the teaching phase. ● "he algorithm produces a program &eg. a neural network with #ne tuned weights), which is able to do the job not only on the provided samples, but also on data that it has never seen before. It usually contains a great number of parameters. Machine learning with neural networks "hree main types of machine learning: 1.Supervised learning – direct feedback, input data is provided with clearly de#ned labels. -
Integer SDM and Modular Composite Representation Dissertation V35 Final
INTEGER SPARSE DISTRIBUTED MEMORY AND MODULAR COMPOSITE REPRESENTATION by Javier Snaider A Dissertation Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy Major: Computer Science The University of Memphis August, 2012 Acknowledgements I would like to thank my dissertation chair, Dr. Stan Franklin, for his unconditional support and encouragement, and for having given me the opportunity to change my life. He guided my first steps in the academic world, allowing me to work with him and his team on elucidating the profound mystery of how the mind works. I would also like to thank the members of my committee, Dr. Vinhthuy Phan, Dr. King-Ip Lin, and Dr. Vasile Rus. Their comments and suggestions helped me to improve the content of this dissertation. I am thankful to Penti Kanerva, who introduced the seminal ideas of my research many years ago, and for his insights and suggestions in the early stages of this work. I am grateful to all my colleagues at the CCRG group at the University of Memphis, especially to Ryan McCall. Our meetings and discussions opened my mind to new ideas. I am greatly thankful to my friend and colleague Steve Strain for our discussions, and especially for his help editing this manuscript and patiently teaching me to write with clarity. Without his amazing job, this dissertation would hardly be intelligible. I will always be in debt to Dr. Andrew Olney for his generous support during my years in the University of Memphis, and for being my second advisor, guiding me academically and professionally in my career. -
The Cognitive Computing Era Is Here: Are You Ready?
The Cognitive Computing Era is Here: Are You Ready? By Peter Fingar. Based on excerpts from the new book Cognitive Computing: A Brief Guide for Game Changers www.mkpress.com/cc Artificial Intelligence is likely to change our civilization as much as, or more than, any technology thats come before, even writing. --Miles Brundage and Joanna Bryson, Future Tense The smart machine era will be the most disruptive in the history of IT. -- Gartner The Disruptive Era of Smart Machines is Upon Us. Without question, Cognitive Computing is a game-changer for businesses across every industry. --Accenture, Turning Cognitive Computing into Business Value, Today! The Cognitive Computing Era will change what it means to be a business as much or more than the introduction of modern Management by Taylor, Sloan and Drucker in the early 20th century. --Peter Fingar, Cognitive Computing: A Brief Guide for Game Changers The era of cognitive systems is dawning and building on today s computer programming era. All machines, for now, require programming, and by definition programming does not allow for alternate scenarios that have not been programmed. To allow alternating outcomes would require going up a level, creating a self-learning Artificial Intelligence (AI) system. Via biomimicry and neuroscience, Cognitive Computing does this, taking computing concepts to a whole new level. Once-futuristic capabilities are becoming mainstream. Let s take a peek at the three eras of computing. Fast forward to 2011 when IBM s Watson won Jeopardy! Google recently made a $500 million acquisition of DeepMind. Facebook recently hired NYU professor Yann LeCun, a respected pioneer in AI.