DOCSLIB.ORG

Exploiting Neuron and Synapse Filter Dynamics in Spatial Temporal Learning of Deep Spiking Neural Network

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Exploiting Neuron and Synapse Filter Dynamics in Spatial Temporal Learning of Deep Spiking Neural Network Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI-20) Exploiting Neuron and Synapse Filter Dynamics in Spatial Temporal Learning of Deep Spiking Neural Network Haowen Fang , Amar Shrestha , Ziyi Zhao and Qinru Qiu Syracuse University fhfang02, amshrest, [email protected], [email protected] Abstract spatial temporal patterns with an ability superior to conven- tional artificial neural network (ANN) [Wu et al., 2018b]. The recently discovered spatial-temporal informa- The potential of SNNs has not been fully explored. First of tion processing capability of bio-inspired Spiking all, due to the lack of unified and robust training algorithms, neural networks (SNN) has enabled some interest- the performance of SNNs is still not comparable with deep ing models and applications. However designing neural networks (DNN). Directly adapting backpropagation large-scale and high-performance model is yet a is not feasible because their output is a sequence of Dirac challenge due to the lack of robust training algo- delta functions, hence is non-differentiable. Secondly, most rithms. A bio-plausible SNN model with spatial- SNN models and training algorithms use rate coding, repre- temporal property is a complex dynamic system. senting a numerical value in DNN by spike counts in a time Synapses and neurons behave as filters capable of window, and consider only the statistics of spike activities. preserving temporal information. As such neuron Temporal structure of spike train and spike timing also con- dynamics and filter effects are ignored in existing vey information [Mohemmed et al., 2012]. Spike trains with training algorithms, the SNN downgrades into a similar rates may have distinct temporal patterns representing memoryless system and loses the ability of tem- different information. To detect the temporal pattern in the poral signal processing. Furthermore, spike tim- spike train, novel synapse and neuron models with temporal ing plays an important role in information repre- dynamics are needed. However, synapse dynamics are often sentation, but conventional rate-based spike coding ignored in the computational models of SNNs. models only consider spike trains statistically, and To address the problem with non-differentiable neuron out- discard information carried by its temporal struc- put, one approach is to train an ANN such as a multi-layer tures. To address the above issues, and exploit the perceptron (MLP) and convert the model to an SNN. This temporal dynamics of SNNs, we formulate SNN as method is straightforward, but it requires additional fine- a network of infinite impulse response (IIR) filters tuning of weights and thresholds [Diehl et al., 2015]. There with neuron nonlinearity. We proposed a training are also works that directly apply backpropagation to SNN algorithm that is capable to learn spatial-temporal training by approximating the gradient of the spiking func- patterns by searching for the optimal synapse fil- tion [Lee et al., 2016; Esser et al., 2015; Shrestha et al., ter kernels and weights. The proposed model and 2019], or utilizing gradient surrogates [Wu et al., 2018b; training algorithm are applied to construct associa- Shrestha and Orchard, 2018]. Other approaches include us- tive memories and classifiers for synthetic and pub- ing derivatives of soft spike [Neftci et al., 2019] or membrane lic datasets including MNIST, NMNIST, DVS 128 potential [Zenke and Ganguli, 2018]. etc. Their accuracy outperforms state-of-the-art ap- proaches. The ability of capturing temporal patterns relies on neu- ron and synapse dynamics [Gutig¨ and Sompolinsky, 2006]. Synapse function can be modeled as filters, whose states pre- 1 Introduction serve rich temporal information. The challenge is how to cap- ture the dependencies between the current SNN states and Spiking neural networks have demonstrated their capability previous input spikes. This challenge has been addressed in signal processing and pattern detection by mimicking the by some existing works. [Gutig¨ and Sompolinsky, 2006] behavior of biological neural systems. In SNNs, informa- and [Gutig,¨ 2016] train individual neuron to classify different tion is represented by sparse and discrete spike events. The temporal spike patterns. [Mohemmed et al., 2012] is capable sparsity of spike activities can be exploited by event driven to train neurons to associate an input spatial temporal pattern implementation for energy efficiency. In a more bio-realistic with a specific output spike pattern. However, the aforemen- neuron and synapse model, each neuron is a dynamic system, tioned works cannot be extended to multiple layers and there- which is capable of spatial temporal information processing. fore are not scalable. Some recent works utilize backpropaga- The network made of such neurons can memorize and detect tion through time (BPTT) to address the temporal dependency 2799 Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI-20) problems. [Wu et al., 2018b] proposed simplified iterative The ODE system is linear time invariant (LTI). It can also leaky integrate and fire (LIF) neuron model. [Gu et al., 2019] be interpreted as the convolution of an impulse response of a derived an iterative model from a current based LIF neu- filter with the input spike train, which leads to the spike re- ron. Based on the iterative model, network can be unrolled sponse model [Gerstner et al., 2014]. The relation between hence BPTT is possible. However, these works only consider the v(t), O(t) and the historical spike input can clearly be the temporal dynamics of membrane potential, the synapse seen in the spike response model. We denote the input spike dynamics and the filter effect of SNN are ignored. There trains as a sequence of time-shifted Dirac delta functions, are also works that introduced the concept of IIR and FIR P j j Si(t) = δ(t − ti ), where ti denotes the jth spike ar- [ j into Multi Layer Perceptron (MLP) Back and Tsoi, 1991; rival time from the ith input synapse. Similarly, output spike Campolucci et al., 1999], which enabled MLP to model time train can be defined as O(t) = P δ(t − tf ); tf 2 ftf : f series. v(t ) = Vthg. To simplify the discussion, we consider only In this work, our contributions are summarized as follows: one synapse. The impulse response kernel k(t) of a neuron 1. The dynamic behavior of LIF neuron is formulated by described by above ODE system is obtained by passing a sin- infinite impulse response (IIR) filters. We exploit the gle spike at time 0 at the input, such that the initial condi- synapse and neuron filter effect, derive a general repre- tions are x(0) = 1 and v(0) = 0. By solving equation 1a η −t −t sentation of SNN as a network of IIR filters with neuron and 1b, we have k(t) = η η−1 (e τm − e τs ). Given the gen- non-linearity. eral input S(t), PSP is the convolution of k(t) and S(t). For 2. A general algorithm is proposed to train such SNN to a neuron with M synapses, without reset, the sub-threshold learn both rate-based and spatial temporal patterns. The membrane potential is the summation of all PSPs, such that PM R 1 algorithm does not only learn the synaptic weight, but v(t) = i wi 0 k(s)Si(t − s)ds. is also capable to optimize the impulse response kernel In hybrid model, the reset is modeled by simply setting v of synapse filters to improve convergence. The similar to vrest, and regarding the reset as the start of the next evalua- learning behavior has been discovered in biological sys- tion and discarding the neuron’s history information. A more tems [Hennig, 2013]. Our training algorithm can be ap- biological way is to treat reset as a negative current impulse plied to train simple LIF, and neurons with more com- applied to the neuron itself [Gerstner et al., 2014]. The re- plex synapses such as alpha synapse, dual-exponential −t set impulse response is h(t) = −Vthe τr , where τr controls synapse etc. the decay speed of reset impulse. Such that the membrane 3. Our algorithm is tested on various datasets includ- potential is the summation of all PSPs and reset voltage: ing MNIST, neuromorphic MNIST, DVS128 gesture, Z 1 M Z 1 TIDIGITS and Australian Sign Language dataset, and X v(t) =− h(t)O(t− s)ds+ wi k(s)Si(t− s)ds (2) outperform state of the art approaches. 0 i 0 Treating reset as a negative impulse enables adaptive 2 Neuron Model threshold, which is observed in biological neurons. Neu- Without loss of generality, we consider a LIF neuron with ron’s threshold depends on its prior spike activity. With dual exponential synapse for its biological plausibility. The adaptation, frequent spike activity increases the reset volt- neuron can be described as a hybrid system, i.e. the mem- age, which inhibits the neuron activity, preventing SNNs from brane potential and synapse status evolve continuously over over-activation. Such that additional tuning methods such as [ et al. ] time, depicted by ordinary differential equations (ODE), weight-thresholds balancing Diehl , 2015 is not neces- while a spike event triggers the update of the state variables sary. as the following [Brette et al., 2007]: Above equations reveal the filter nature of the biologically realistic neuron model. Each synapse act like a low pass fil- M ter. Synapse filter is causal, and the kernel is defined to decay dv(t) η X over time, hence the current state of the PSP is determined by τm = −(v(t) − vrest) + η η−1 wixi(t) (1a) dt all previous input spikes up to current time. The temporal de- i pendency calls for temporal error propagation in the training. dx (t) τ i = −x (t) (1b) s dt i 3 Neuron and Synapse as IIR Filters xi(t) xi(t) + 1; upon receiving spike (1c) In practice, for computational efficiency, spiking neural net- v(t) vrest, if v(t) = Vth (1d) work are usually simulated in discrete time domain and net- work states are evaluated for every unit time.
Load more

Recommended publications
  • Spiking Neural Networks: Asurvey Oscar T
    Oscar T. Skaug Spiking Neural Networks: A Survey Bachelor’s project in Computer engineering Supervisor: Ole Christian Eidheim June 2020 Bachelor’s project NTNU Engineering Faculty of Information Technology and Electrical Faculty of Information Technology Norwegian University of Science and Technology Abstract The purpose of the survey is to support teaching and research in the department of Computer Science at the university, through fundamental research in recent advances in the machine learning field. The survey assumes the reader have some basic knowledge in machine learning, however, no prior knowledge on the topic of spiking neural networks is assumed. The aim of the survey is to introduce the reader to artificial spiking neural networks by going through the following: what constitutes and distinguish an artificial spiking neural network from other artificial neural networks; how to interpret the output signals of spiking neural networks; How supervised and unsupervised learning can be achieved in spiking neural networks; Some benefits of spiking neural networks compared to other artificial neural networks, followed by challenges specifically associated with spiking neural networks. By doing so the survey attempts to answer the question of why spiking neural networks have not been widely implemented, despite being both more computationally powerful and biologically plausible than other artificial neural networks currently available. Norwegian University of Science and Technology Trondheim, June 2020 Oscar T. Skaug The bachelor thesis The title of the bachelor thesis is “Study of recent advances in artificial and biological neuron models” with the purpose of “supporting teaching and research in the department of Computer Science at the university through fundamental research in recent advances in the machine learning field”.
    [Show full text]
  • Spiking Neural Networks: Neuron Models, Plasticity, and Graph Applications
    Virginia Commonwealth University VCU Scholars Compass Theses and Dissertations Graduate School 2015 Spiking Neural Networks: Neuron Models, Plasticity, and Graph Applications Shaun Donachy Virginia Commonwealth University Follow this and additional works at: https://scholarscompass.vcu.edu/etd Part of the Theory and Algorithms Commons © The Author Downloaded from https://scholarscompass.vcu.edu/etd/3984 This Thesis is brought to you for free and open access by the Graduate School at VCU Scholars Compass. It has been accepted for inclusion in Theses and Dissertations by an authorized administrator of VCU Scholars Compass. For more information, please contact [email protected]. c Shaun Donachy, July 2015 All Rights Reserved. SPIKING NEURAL NETWORKS: NEURON MODELS, PLASTICITY, AND GRAPH APPLICATIONS A Thesis submitted in partial fulfillment of the requirements for the degree of Master of Science at Virginia Commonwealth University. by SHAUN DONACHY B.S. Computer Science, Virginia Commonwealth University, May 2013 Director: Krzysztof J. Cios, Professor and Chair, Department of Computer Science Virginia Commonwealth University Richmond, Virginia July, 2015 TABLE OF CONTENTS Chapter Page Table of Contents :::::::::::::::::::::::::::::::: i List of Figures :::::::::::::::::::::::::::::::::: ii Abstract ::::::::::::::::::::::::::::::::::::: v 1 Introduction ::::::::::::::::::::::::::::::::: 1 2 Models of a Single Neuron ::::::::::::::::::::::::: 3 2.1 McCulloch-Pitts Model . 3 2.2 Hodgkin-Huxley Model . 4 2.3 Integrate and Fire Model . 6 2.4 Izhikevich Model . 8 3 Neural Coding Techniques :::::::::::::::::::::::::: 14 3.1 Input Encoding . 14 3.1.1 Grandmother Cell and Distributed Representations . 14 3.1.2 Rate Coding . 15 3.1.3 Sine Wave Encoding . 15 3.1.4 Spike Density Encoding . 15 3.1.5 Temporal Encoding . 16 3.1.6 Synaptic Propagation Delay Encoding .
    [Show full text]
  • Simplified Spiking Neural Network Architecture and STDP Learning
    Iakymchuk et al. EURASIP Journal on Image and Video Processing (2015) 2015:4 DOI 10.1186/s13640-015-0059-4 RESEARCH Open Access Simplified spiking neural network architecture and STDP learning algorithm applied to image classification Taras Iakymchuk*, Alfredo Rosado-Muñoz, Juan F Guerrero-Martínez, Manuel Bataller-Mompeán and Jose V Francés-Víllora Abstract Spiking neural networks (SNN) have gained popularity in embedded applications such as robotics and computer vision. The main advantages of SNN are the temporal plasticity, ease of use in neural interface circuits and reduced computation complexity. SNN have been successfully used for image classification. They provide a model for the mammalian visual cortex, image segmentation and pattern recognition. Different spiking neuron mathematical models exist, but their computational complexity makes them ill-suited for hardware implementation. In this paper, a novel, simplified and computationally efficient model of spike response model (SRM) neuron with spike-time dependent plasticity (STDP) learning is presented. Frequency spike coding based on receptive fields is used for data representation; images are encoded by the network and processed in a similar manner as the primary layers in visual cortex. The network output can be used as a primary feature extractor for further refined recognition or as a simple object classifier. Results show that the model can successfully learn and classify black and white images with added noise or partially obscured samples with up to ×20 computing speed-up at an equivalent classification ratio when compared to classic SRM neuron membrane models. The proposed solution combines spike encoding, network topology, neuron membrane model and STDP learning.
    [Show full text]
  • Spiking Neural Network Vs Multilayer Perceptron: Who Is the Winner in the Racing Car Computer Game
    Soft Comput (2015) 19:3465–3478 DOI 10.1007/s00500-014-1515-2 FOCUS Spiking neural network vs multilayer perceptron: who is the winner in the racing car computer game Urszula Markowska-Kaczmar · Mateusz Koldowski Published online: 3 December 2014 © The Author(s) 2014. This article is published with open access at Springerlink.com Abstract The paper presents two neural based controllers controller on-line. The RBF network is used to establish a for the computer car racing game. The controllers represent nonlinear prediction model. In the recent paper (Zheng et al. two generations of neural networks—a multilayer perceptron 2013) the impact of heavy vehicles on lane-changing deci- and a spiking neural network. They are trained by an evolu- sions of following car drivers has been investigated, and the tionary algorithm. Efficiency of both approaches is experi- lane-changing model based on two neural network models is mentally tested and statistically analyzed. proposed. Computer games can be perceived as a virtual representa- Keywords Computer game controller · Spiking neural tion of real world problems, and they provide rich test beds network · Multilayer perceptron · Evolutionary algorithm for many artificial intelligence methods. An example which can be mentioned here is successful application of multi- layer perceptron in racing car computer games, described for 1 Introduction instance in Ebner and Tiede (2009) and Togelius and Lucas (2005). Inspiration for our research were the results shown in Neural networks (NN) are widely applied in many areas Yee and Teo (2013) presenting spiking neural network based because of their ability to learn. controller for racing car in computer game.
    [Show full text]
  • Computing with Spiking Neuron Networks
    Computing with Spiking Neuron Networks Hel´ ene` Paugam-Moisy1 and Sander Bohte2 Abstract Spiking Neuron Networks (SNNs) are often referred to as the 3rd gener- ation of neural networks. Highly inspired from natural computing in the brain and recent advances in neurosciences, they derive their strength and interest from an ac- curate modeling of synaptic interactions between neurons, taking into account the time of spike firing. SNNs overcome the computational power of neural networks made of threshold or sigmoidal units. Based on dynamic event-driven processing, they open up new horizons for developing models with an exponential capacity of memorizing and a strong ability to fast adaptation. Today, the main challenge is to discover efficient learning rules that might take advantage of the specific features of SNNs while keeping the nice properties (general-purpose, easy-to-use, available simulators, etc.) of traditional connectionist models. This chapter relates the his- tory of the “spiking neuron” in Section 1 and summarizes the most currently-in-use models of neurons and synaptic plasticity in Section 2. The computational power of SNNs is addressed in Section 3 and the problem of learning in networks of spiking neurons is tackled in Section 4, with insights into the tracks currently explored for solving it. Finally, Section 5 discusses application domains, implementation issues and proposes several simulation frameworks. 1 Professor at Universit de Lyon Laboratoire de Recherche en Informatique - INRIA - CNRS bat. 490, Universit Paris-Sud Orsay cedex, France e-mail: [email protected] 2 CWI Amsterdam, The Netherlands e-mail: [email protected] 1 Contents Computing with Spiking Neuron Networks :::::::::::::::::::::::::: 1 Hel´ ene` Paugam-Moisy1 and Sander Bohte2 1 From natural computing to artificial neural networks .
    [Show full text]
  • Neural Networks Indirect and Direct Training of Spiking Neural Networks for End-To-End Control of a Lane-Keeping Vehicle
    Neural Networks 121 (2020) 21–36 Contents lists available at ScienceDirect Neural Networks journal homepage: www.elsevier.com/locate/neunet Indirect and direct training of spiking neural networks for end-to-end control of a lane-keeping vehicle ∗ Zhenshan Bing a,b, Claus Meschede b, Guang Chen c, Alois Knoll b, Kai Huang a, a School of Data and Computer Science, Sun Yat-Sen University, China b Department of Computer Science, Technical University of Munich, Germany c School of Automotive Studies, Tongji University, China article info a b s t r a c t Article history: Building spiking neural networks (SNNs) based on biological synaptic plasticities holds a promising Available online 9 July 2019 potential for accomplishing fast and energy-efficient computing, which is beneficial to mobile robotic applications. However, the implementations of SNNs in robotic fields are limited due to the lack of Keywords: practical training methods. In this paper, we therefore introduce both indirect and direct end-to-end Spiking neural network End-to-end learning training methods of SNNs for a lane-keeping vehicle. First, we adopt a policy learned using the Deep R-STDP Q-Learning (DQN) algorithm and then subsequently transfer it to an SNN using supervised learning. Lane keeping Second, we adopt the reward-modulated spike-timing-dependent plasticity (R-STDP) for training SNNs directly, since it combines the advantages of both reinforcement learning and the well-known spike- timing-dependent plasticity (STDP). We examine the proposed approaches in three scenarios in which a robot is controlled to keep within lane markings by using an event-based neuromorphic vision sensor.
    [Show full text]
  • Error-Backpropagation in Temporally Encoded Networks of Spiking Neurons
    Neurocomputing 48 (2002) 17–37 www.elsevier.com/locate/neucom Error-backpropagation in temporally encoded networks of spiking neurons Sander M. Bohtea;∗, Joost N. Koka;c , Han La PoutrÃea;b aCWI, Kruislaan 413, 1098 SJ Amsterdam, The Netherlands bSchool of Technology Management, Eindhoven University of Technology, The Netherlands cLIACS, Leiden University, P.O. Box 9512, 2300 RA Leiden, The Netherlands Received 27 October 2000; accepted 6 June 2001 Abstract For a network of spiking neurons that encodes information in the timing of individual spike times, we derive a supervised learning rule, SpikeProp, akin to traditional error- backpropagation. With this algorithm, we demonstrate how networks of spiking neurons with biologically reasonable action potentials can perform complex non-linear classiÿcation in fast temporal coding just as well as rate-coded networks. We perform experiments for the classical XOR problem, when posed in a temporal setting, as well as for a number of other benchmark datasets. Comparing the (implicit) number of spiking neurons required for the encoding of the interpolated XOR problem, the trained networks demonstrate that temporal coding is a viable code for fast neural information processing, and as such requires less neurons than instantaneous rate-coding. Furthermore, we ÿnd that reliable temporal computation in the spiking networks was only accomplished when using spike response functions with a time constant longer than the coding interval, as has been predicted by theoretical considerations. c 2002 Elsevier Science B.V. All rights reserved. Keywords: Spiking neurons; Temporal coding; Error-backpropagation 1. Introduction Due to its success in artiÿcial neural networks, the sigmoidal neuron is reputed to be a successful model of biological neuronal behavior.
    [Show full text]
  • Training a Digital Model of a Deep Spiking Neural Network Using Backpropagation
    E3S Web of Conferences 224, 01026 (2020) https://doi.org/10.1051/e3sconf/202022401026 TPACEE-2020 Training a digital model of a deep spiking neural network using backpropagation V Bondarev1,* 1Sevastopol State University, Sevastopol, 299053, Russia Abstract. Deep spiking neural networks are one of the promising event- based sensor signal processing concepts. However, the practical application of such networks is difficult with standard deep neural network training packages. In this paper, we propose a vector-matrix description of a spike neural network that allows us to adapt the traditional backpropagation algorithm for signals represented as spike time sequences. We represent spike sequences as binary vectors. This enables us to derive expressions for the forward propagation of spikes and the corresponding spike training algorithm based on the back propagation of the loss function sensitivities. The capabilities of the proposed vector-matrix model are demonstrated on the problem of handwritten digit recognition on the MNIST data set. The classification accuracy on test data for spiking neural network with 3 hidden layers is equal to 98.14%. 1 Introduction In recent years, conventional deep neural networks have been widely used in solving various artificial intelligence problems: image classification, object detection, speech recognition, natural language processing and much more [1]. Training such deep neural networks typically requires powerful GPUs and computing clusters. Therefore, the application of conventional deep neural networks in areas with critical power consumption is problematic. An alternative are biologically inspired spiking neural networks (SNNs). SNNs had long shown great theoretical potential as efficient computing units [2], and recent advances in SNN hardware have renewed research interest in this area [3].
    [Show full text]
  • Gradient Descent for Spiking Neural Networks
    Gradient Descent for Spiking Neural Networks Dongsung Huh Terrence J. Sejnowski Salk Institute Salk Institute La Jolla, CA 92037 La Jolla, CA 92037 [email protected] [email protected] Abstract Most large-scale network models use neurons with static nonlinearities that pro- duce analog output, despite the fact that information processing in the brain is predominantly carried out by dynamic neurons that produce discrete pulses called spikes. Research in spike-based computation has been impeded by the lack of efficient supervised learning algorithm for spiking neural networks. Here, we present a gradient descent method for optimizing spiking network models by in- troducing a differentiable formulation of spiking dynamics and deriving the exact gradient calculation. For demonstration, we trained recurrent spiking networks on two dynamic tasks: one that requires optimizing fast (≈ millisecond) spike-based interactions for efficient encoding of information, and a delayed-memory task over extended duration (≈ second). The results show that the gradient descent approach indeed optimizes networks dynamics on the time scale of individual spikes as well as on behavioral time scales. In conclusion, our method yields a general purpose supervised learning algorithm for spiking neural networks, which can facilitate further investigations on spike-based computations. 1 Introduction The brain operates in a highly decentralized event-driven manner, processing multiple asynchronous streams of sensory-motor data in real-time. The main currency of neural computation is spikes: i.e. brief impulse signals transmitted between neurons. Experimental evidence shows that brain’s architecture utilizes not only the rate, but the precise timing of spikes to process information [1]. Deep-learning models solve simplified problems by assuming static units that produce analog output, which describes the time-averaged firing-rate response of a neuron.
    [Show full text]
  • Introduction to Spiking Neural Networks: Information Processing, Learning and Applications
    Review Acta Neurobiol Exp 2011, 71: 409–433 Introduction to spiking neural networks: Information processing, learning and applications Filip Ponulak1,2* and Andrzej Kasiński1 1Institute of Control and Information Engineering, Poznan University of Technology, Poznan, Poland, *Email: [email protected]; 2Princeton Neuroscience Institute and Department of Molecular Biology, Princeton University, Princeton, USA The concept that neural information is encoded in the firing rate of neurons has been the dominant paradigm in neurobiology for many years. This paradigm has also been adopted by the theory of artificial neural networks. Recent physiological experiments demonstrate, however, that in many parts of the nervous system, neural code is founded on the timing of individual action potentials. This finding has given rise to the emergence of a new class of neural models, called spiking neural networks. In this paper we summarize basic properties of spiking neurons and spiking networks. Our focus is, specifically, on models of spike-based information coding, synaptic plasticity and learning. We also survey real-life applications of spiking models. The paper is meant to be an introduction to spiking neural networks for scientists from various disciplines interested in spike-based neural processing. Key words: neural code, neural information processing, reinforcement learning, spiking neural networks, supervised learning, synaptic plasticity, unsupervised learning INTRODUCTION gates (Maass 1997). Due to all these reasons SNN are the subject of constantly growing interest of scientists. Spiking neural networks (SNN) represent a special In this paper we introduce and discuss basic con- class of artificial neural networks (ANN), where neu- cepts related to the theory of spiking neuron models.
    [Show full text]
  • Improving Spiking Neural Networks Trained with Spike Timing Dependent Plasticity for Image Recognition Pierre Falez
    Improving Spiking Neural Networks Trained with Spike Timing Dependent Plasticity for Image Recognition Pierre Falez To cite this version: Pierre Falez. Improving Spiking Neural Networks Trained with Spike Timing Dependent Plasticity for Image Recognition. Artificial Intelligence [cs.AI]. Université de Lille, 2019. English. tel-02429539 HAL Id: tel-02429539 https://hal.archives-ouvertes.fr/tel-02429539 Submitted on 6 Jan 2020 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. École doctorale des Sciences Pour l’Ingénieur de Lille - Nord De France Improving Spiking Neural Networks Trained with Spike Timing Dependent Plasticity for Image Recognition Doctoral Thesis Computer Science Pierre Falez 10 October 2019 Supervisors Jury Director President Pr. Pierre Boulet Pr. Nathalie H. Rolland Université de Lille CNRS – Université de Lille Referees Co-director Dr. Timothée Masquelier Dr. Pierre Tirilly CERCO UMR 5549, CNRS – Univer- Université de Lille sité de Toulouse 3 Pr. Benoît Miramond Co-supervisors LEAT / CNRS UMR 7248 – Université Dr. Ioan Marius Bilasco Côte d’Azur Université de Lille Dr. Philippe Devienne CNRS Examiner Pr. Sander M. Bohte Centrum Wiskunde and Informatica 2 École doctorale des Sciences Pour l’Ingénieur de Lille - Nord De France Amélioration des réseaux de neurones impulsionnels entraînés avec la STDP pour la reconnaissance d’images Thèse en vue de l’obtention du titre de docteur en Informatique et applications Pierre Falez 10 Octobre 2019 Superviseurs Jury Directeur Présidente Pr.
    [Show full text]
  • A Low-Power Spiking Neural Network Chip Based on a Compact LIF Neuron and Binary Exponential Charge Injector Synapse Circuits
    sensors Article A Low-Power Spiking Neural Network Chip Based on a Compact LIF Neuron and Binary Exponential Charge Injector Synapse Circuits Malik Summair Asghar 1,2, Saad Arslan 3 and Hyungwon Kim 1,* 1 Department of Electronics Engineering, Chungbuk National University, Chungdae-ro 1, Seowon-gu, Cheongju 28644, Korea; [email protected] 2 Department of Electrical and Computer Engineering, Abbottabad Campus, COMSATS University Islamabad, University Road, Tobe Camp, Abbottabad 22044, Pakistan 3 Department of Electrical and Computer Engineering, COMSATS University Islamabad, Park Road, Tarlai Kalan, Islamabad 45550, Pakistan; [email protected] * Correspondence: [email protected] Abstract: To realize a large-scale Spiking Neural Network (SNN) on hardware for mobile applications, area and power optimized electronic circuit design is critical. In this work, an area and power opti- mized hardware implementation of a large-scale SNN for real time IoT applications is presented. The analog Complementary Metal Oxide Semiconductor (CMOS) implementation incorporates neuron and synaptic circuits optimized for area and power consumption. The asynchronous neuronal circuits implemented benefit from higher energy efficiency and higher sensitivity. The proposed synapse circuit based on Binary Exponential Charge Injector (BECI) saves area and power consumption, and provides design scalability for higher resolutions. The SNN model implemented is optimized for 9 × 9 pixel input image and minimum bit-width weights that can satisfy target accuracy, occupies Citation: Asghar, M.S.; Arslan, S.; less area and power consumption. Moreover, the spiking neural network is replicated in full digital Kim, H. A Low-Power Spiking implementation for area and power comparisons. The SNN chip integrated from neuron and synapse Neural Network Chip Based on a circuits is capable of pattern recognition.
    [Show full text]