Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18)

Brain-inspired Balanced Tuning for Spiking Neural Networks

Tielin Zhang1,3, Yi Zeng1,2,3,4,5,∗, Dongcheng Zhao1,2,3 and Bo Xu1,2,3,5 1 Institute of Automation, Chinese Academy of Sciences (CAS), China 2 University of Chinese Academy of Sciences, China 3 Research Center for Brain-inspired Intelligence, Institute of Automation, CAS, China 4 National Laboratory of Pattern Recognition, Institute of Automation, CAS, China 5 Center for Excellence in Brain Science and Intelligence Technology, CAS, China {tielin.zhang, yi.zeng}@ia.ac.cn

Abstract from the brain have contributed to the research of Artificial Intelligence (AI). For example, Hopfield network with recur- Due to the nature of Spiking Neural Network- rent connections is inspired by the Hippocampus; Hierarchi- s (SNNs), it is challenging to be trained by bio- cal temporary memory (HTM) network with micro-column logically plausible learning principles. The multi- structures is inspired by the neocortex; Convolutional neu- layered SNNs are with non-differential neurons, ral network (CNN) with hierarchical perception is inspired temporary-centric , which make them n- by the primary visual cortex; with early impossible to be directly tuned by back prop- dynamic acquisition of online rules is inspired by the basal agation. Here we propose an alternative biological ganglia centric pathway. inspired balanced tuning approach to train SNNs. The approach contains three main inspirations from Many Artificial Neural Network (ANN) models are with the brain: Firstly, the biological network will usu- brain-inspirations at different level of details. And they have ally be trained towards the state where the temporal shown their power on various tasks, such as image cap- update of variables are equilibrium (e.g. membrane tion, language translation [LeCun et al., 2015] and the game potential); Secondly, specific proportions of exci- Go [Hassabis et al., 2017]. tatory and inhibitory neurons usually contribute to However, the tuning methods of back propagation in ANNs stable representations; Thirdly, the short-term plas- are facing challenges on preventing overfitting, improving ticity (STP) is a general principle to keep the in- transferability, and increasing convergence speed. The fire- put and output of synapses balanced towards a bet- rate models in ANNs are also short at processing tem- ter learning convergence. With these inspirations, poral information which makes them hard to be with good we train SNNs with three steps: Firstly, the SNN self-stability. The principles of neurons, synapses, and net- model is trained with three brain-inspired princi- works in biological systems are far more complex and pow- ples; then weakly is used to erful than those used in ANNs [Hassabis et al., 2017]. It has tune the in the final layer for been proved that even a single biological neuron with dendrit- network classification; finally the learned informa- ic branches needs a three-layered ANN for finer simulation- tion is consolidated from membrane potential into s [Hausser¨ and Mel, 2003]. the weights of synapses by Spike-Timing Depen- The intelligence of biological systems is based on multi- dent Plasticity (STDP). The proposed approach is scale complexities, from microscale of neurons and synaps- verified on the MNIST hand-written digit recog- es to the macroscale of brain regions and their interactions. nition dataset and the performance (the accuracy At the microscale, the neurons in biological systems repre- of 98.64%) indicates that the ideas of balancing sent or process information by discrete action potentials (or state could indeed improve the learning ability of spikes). It raises an open question that how these discrete neu- SNNs, which shows the power of proposed brain- ron activities interpret continuous functions, or from a similar inspired approach on the tuning of biological plau- point of view, how these network with non-differential neu- sible SNNs. rons could be successfully tuned by biological learning prin- ciples. Understanding these mechanisms of biological sys- 1 Introduction tems will give us hints on the new biological-plausible tuning methods [Abbott et al., 2016]. Decoding brain on both structural and functional perspectives Compared to other neural network models, Spiking Neu- has lasted for centuries. In this procedure, many inspirations ral Networks (SNNs) are generally more solid on biological ∗Tielin Zhang and Yi Zeng contribute equally to this article and plausibility. The SNNs are considered to be the third gen- should be considered as co-first authors. eration of neural network models, and are powerful on the

1653 Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18) processing of both spatial and temporal information [Maass, 2 Related Works 1997]. Neurons in SNNs communicate with each other by discon- Zenke et al. showed that the interaction of Hebbian homosy- tinuous spikes, which raises gaps between spikes and behav- naptic plasticity with rapid non-Hebbian heterosynaptic plas- iors but also narrow down the multi-level integration chal- ticity would be sufficient for memory formation, and then lenges since the spikes could be considered as naturally in- memory could be recalled after a brief stimulation of a sub- teractive signals. In SNNs, the neurons will not be activated set of assembly neurons in a spiking recurrent network mod- [ ] until the membrane potentials reach thresholds. This makes el Zenke et al., 2015 . them energy efficient. Alireza et al. proposed a local learning rule supported by The diversity of neuron types (e.g. excitatory and inhibito- the theory of efficient, balanced neural networks (EBN) for ry neurons) also enables SNNs to keep balance, which will, the tuning of recurrent spiking neural networks. An addition- in turn, help SNNs on efficient learning and forming specific al tight excitatory and inhibitory balance is maintained for the [ ] functions. In addition, different computing costs in neurons spiking efficiency and robustness Alemi et al., 2018 . and synapses cause various kinds of time delays which will Zeng et al. proposed seven biologically plausible rules to also contribute to the asynchronous computation of SNNs, s- train multi-layer SNNs with Leaky-Integrated and Fire (LIF) ince these kinds of delays will open up a new temporal dimen- neurons, which includes more local principles such as dy- sion on SNN for better representation capacity. SNNs have namic neuron allocations, formation and elimination, been well applied on XOR problem [Sporea and Gruning,¨ various kinds of STDPs, and also more global learning prin- 2013], visual pattern recognition [Diehl and Cook, 2015; ciples such as background noise influence and the proportion Zeng et al., 2017], probabilistic inference [Soltani and Wang, of different kinds of neurons [Zeng et al., 2017]. It has been 2010] and planning tasks [Rueckert et al., 2016]. proved that the synaptic weights in first few layers of SNNs Although SNNs have shown more biological plausibility could be dynamically updated by STDP rules without any su- than conventional ANNs, from the computational perspec- pervision, and the weights between the final two layers could tive, lack of efficient and biological plausible learning meth- be supervised and learned by Hebb’s law. ods in the current SNN models limits their values to support Diehl et al. trained an SNN with conductance-based understanding the nature of intelligence and potential appli- synapses, STDP, lateral inhibition, and adaptive spiking cations. threshold, and used an scheme to train With respect to this, some efforts have been made to train a two-layered SNN. Finally, the accuracy reached 95% on the the networks by biological plausible principles. Long-Term MNIST benchmark [Diehl and Cook, 2015]. Potentiation (LTP), Long-Term Depression (LTD), Short Ter- Some other efforts get around of the direct training of m Plasticity (STP) which includes Short Term Facilitation SNNs by equivalent converting of learned synaptic weight- (STF) and Depression (STD), Hebbian learning, Spike Tim- s from ANNs into SNNs. Diehl et al. try to convert deep ing Dependent Plasticity (STDP), lateral inhibition, synap- ANNs into SNNs directly and keep the minimum perfor- tic scaling, synaptic redistribution, and many other brain- mance loss in the conversion process, the key techniques in- inspired learning principles from biological nervous system- clude the limitation of rectified linear units (ReLUs) with zero s are proposed and applied on the training procedure of bias and weight normalization into a linear range [Diehl et al., SNNs [Abraham and Bear, 1996]. Nevertheless, there is still 2015]. Although this method could achieve the performance gaps for SNNs in specific applications when compared with of 98.48% on 10-class hand-written digit MNIST classifica- ANN models. More efficient and comprehensive learning tion task, the performance of SNN is actually contributed by frameworks for SNNs need to be proposed and applied. ANN from instead of pure biological plau- In this paper, we propose brain-inspired balanced tuning sible SNN learning. for SNNs (Balanced SNN for short), we will tune the SNNs Lee et al. argued that the discontinuities between spikes based on three inspirations from the brain: Firstly, the bi- could be considered as noises, and the SNN without nois- ological network will be trained towards the equilibrium s- es are continuous and could use backpropagation for train- tates for the membrane potentials. Secondly, the proportion ing. A new architecture based on this idea is tested on M- of excitatory and inhibitory neurons need to make a balance NIST and N-MNIST dataset, and a better performance and a and cooperate together for better tuning. Thirdly, the STP faster convergence are achieved compared with conventional is used to keep the synaptic input-output balanced towards SNNs [Lee et al., 2016]. a better learning convergence. These inspirations are intro- SpikeProp is an error-backpropagation based supervised duced as three balance principles, namely, the Membrane Po- learning algorithm for the training of spiking networks which tential (MP) based balance, the Excitatory-Inhibitory neuron could actually be considered as equivalent exchanges from type (E-I) based balance and the STP based balance. With spatial information (i.e. fire rates) in ANN to the temporal in- these inspirations, we start to train SNNs with three steps: formation (i.e. timing of inter-spike intervals) in SNN [Bohte training the membrane potential of SNNs with three brain- et al., 2002]. inspired principles; then a weakly supervised learning is used In our efforts, we aim to minimize artificial rules and to tune the membrane potential in the final layer; finally con- principles, and incorporate more biological plausible tuning solidating the learned membrane potential information into mechanisms. We want to understand whether more solid bi- synaptic weights by STDP. The MNIST benchmark is used ological principles could bring current brain-inspired SNN to test the performance of the proposed model. models to the next level and computationally enhance our un-

1654 Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18)

Hidden layers with post-synaptic LIF Model Input layer with excitation or inhibition spiking signals Output layers with Pre-synaptic Spikes Post-synaptic lateral inhibition Spikes Spikes

Postsynaptic Membrane Input Potential Output

Non-linear Current leakage ` Integration in refractory time

Excitatory neurons Inhibitory neurons Figure 1: The LIF neuron with discontinuous spikes Excitatory synapses Inhibitory synapses Winner-Take-All Inhibition derstanding of learning in the brain, and be more practically Figure 2: The architecture of a feed forward multi-layered Balanced efficient in AI applications. SNN 3 The Architecture of Balanced SNNs non-differential potential Vj). The fsyn is the function from Different SNNs are with different structures (e.g. recurren- membrane potential (or spikes) to currents, which could be a t, or feed-forward connections) and learning methods. The decay factor or an STP function. recurrent networks are usually designed for temporal infor- mation processing and the multi-layered networks are mainly   X for the abstraction of spatial information. Isyn = fsyn  wj,iδt (2)

The simplest version of feed-forward multi-layered SNNs j∈NE is with two-layered architecture, which is also the first suc- cessful paradigm which could be tuned well by biological- 3.2 The Multi-layered Balanced SNN ly plausible learning principles [Diehl and Cook, 2015].A As shown in Figure 2, a multi-layered feed-forward SNN (for three layered SNN is constructed and trained by seven brain- simplicity here we use three-layered SNN) is constructed. inspired learning principles [Zeng et al., 2017], which shows The neurons in the first layers are non-LIF neurons which unique contributions of different brain-inspired principles. only receive the signals from inputs and output signals to the In this paper, we use similar building blocks as in [Zhang next layer directly without decay. et al., 2018] which uses the LIF neuron model for temporal The neurons in the second layers are LIF neurons with information processing and feedforward three-layered SNN both excitatory and inhibitory types. For the excitatory neu- for information integration as the basic structure. rons, all of the output synapses are excitatory (with posi- tive values). On the contrary, for the inhibitory neurons, all 3.1 The Basic LIF Neuron Model of the output synapses are inhibitory (with negative values). The LIF neuron model is the basic building block of the SNN Synapses will receive spikes from pre-synaptic neurons and model in this paper. It’s function is for non-linear information then send positive (or negative) spikes to postsynaptic neu- integration and non-differential spikes generation. rons after firing. The proportion of inhibitory neurons in the As shown in Figure 1, when pre-synaptic neuron fires, the second layer will be predefined. spikes are generated and propagated into post-synaptic neu- The neurons in the final layer are all of excitatory LIF neu- rons. We use V to represent V (t) for simplicity. The dynamic rons which could receive both inputs from pre-synaptic neu- function of membrane potential in LIF will be integrated with rons and also the additional teaching signals. The teaching dt. The Cm is the membrane capacitance, the gL is the leaky signals will be updated synchronously with the network in- conductance, VL is the leaky potential, and Isyn is the input puts. stimulus (converted from spikes by synapses) from presynap- tic neurons. 4 Brain inspired Balanced Tuning Most of the cognitive and functional neural systems tend dV gE/I  Isyn to keep balanced states for better adaptability and stability. τm = − (V − VL) − V − VE/I + (1) dt gL gL Here we introduce three brain-inspired balance principles: the Membrane Potential (MP) based balance, the Excitatory- The gE is the excitatory conductance, gI is the inhibitory Inhibitory neuron type (E-I) based balance and the STP based conductance, VE and VI are the reversal potentials for excita- balance. Cm tory and inhibitory neurons respectively, and τm = . gL The value of Isyn in Equation (2) will be updated by the 4.1 Membrane Potential based Balance pre-synaptic spikes. The wj,i is the connection weight from Here we focus on the membrane potential Vi as one of a main pre-synaptic neuron j to the target neuron i, δt denotes the balanced variable for tuning. Membrane potential is a tempo- pre-synaptic spikes (in the next step it will be updated into ral dynamical variable which works for the function of infor-

1655 Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18)

dEi mation integration or abstraction. Here we define ∆Ei = dt could be considered as a condition to the of wj,i, as the energy representation of temporal differential states of as shown in Equation (6). neurons. (   ηw = η0 if (wj,i∆wj,i ≥ 0) N   X ∆wj,i (6) ηw = η0 −η1 if (wj,i∆wj,i < 0) ∆Ei = Vi −  wj,iVj − Vth,i (3) wj,i j where ηw is the learning rate for each synapse, η0 is the As shown in Equation (3), the first term after equality sign predefined initial learning rate, η1 is the variable in the range is the current membrane potential of neuron i, and the sec- −∆w of (0, 1) which makes the condition of j,i ≤ 1 works ond term is the future membrane potential of neuron i which wj,i 1 integrates all of the inputs from pre-synaptic Vj. The net- (here we use η1 = 2 for simplicity). Finally, we will have the work will learn dynamically towards network convergence, rule for excitatory and inhibitory synapses update which also and with training time going by, the current states and nex- works as an alternative balanced principle for network tuning. t states of neurons will become equivalent, which means the 4.3 Short-Term Plasticity based Balance ∆Ei will be around zero. Considering that in our work, the membrane potential Vi To a certain extent, the firing frequencies of neurons in SNNs has already taken the place of wj,i on network tuning (the are kept balanced by STP. The spiking frequency will increase information will be consolidated from Vi to wj,i in final step- by STF when the frequency is low, while will decrease by s), we update Equation (3) into Equation (4) according to the STD when the frequency is high [Zucker and Regehr, 2002]. differential chain rule. For STF, the release of Ca2+ from synapses will increase the probability of the neuron firing next time. For STD, the high- frequency firing will cost too much energy to support spike PN  dE dE dt Vi − j wj,iVj − Vth,i generation next time which is very near for the last spike. i = i × = (4) dV dt dV dVi i i dt du u = − + U(1 − u)δ(t − tsp) (7) dt τf PN PN  Vi − wj,iVj − Vth,i MP j j dx 1 − x = − uxδ(t − tsp) (8) ∆Vi = −ηMP gE (5) − (Vi − VL) − (Vi − VE) dt τd gL As shown in Equation (7) and Equation (8), u and x are the Finally Equation (5) describes the detailed update mecha- normalized variables which represent the dynamical charac- nisms of Vi based on the membrane potential based balance. teristics of STF and STD respectively. δ(t − tsp) is the input 4.2 Excitatory-Inhibitory Neuron Type based of spikes on time tsp, τf and τd are the recover time constants of STF and STD respectively. Balance In the biological brains, the proportion of excitatory neurons ISTP I syn = − syn + Aw uxδ(t − t ) (9) is larger compared to inhibitory neurons. These two types of dt τ j,i sp neurons are integrated together in a interactive way to make s A the network in balanced states [Okun and Lampl, 2009]. As shown in Equation (9), is the maximal connection τ I I Different with conventional ANNs, in which the weights weight, s is the recover time constants for syn. syn will u x of output synapses from a single neuron could be both posi- be updated based on the in Equation (7) and in Equa- I tive or negative, in our model, we follow the biological sys- tion (8), then the syn will be combined with Equation (1) for tem that normally, weights of synapses have to be positive for the balanced tuning. excitatory neurons while being negative for inhibitory neu- 4.4 Supervised Learning in the Final Layer rons. Considering that the initial weights of neurons have al- ready fit for the biological conditions, we separate the proce- As shown in Figure 2, the neurons in the final layer of the dure of weights update into two situations: the first situation network receive inputs from both the pre-synaptic neurons and also the teaching signals. The teaching signal is a kind of is when wj,i∆wj,i ≥ 0, in which the weights of synapses will not change their symbols after update (i.e. the weights very high-frequency stimulus (the frequency will be the same of synapses will increase for the excitatory type and will de- as the input signals of the first layer) to the neurons in the crease for the inhibitory type); the second situation is when final layer. wj,i∆wj,i < 0, where the updated weights of synapses may N 1 X 2 change their symbols. C = (V − δ (t − t )) (10) The first situation has already fit for the biological brain- 2 i sp i s. For the second situation, we make another condition as SUP c wj,i (wj,i + ∆wj,i) ≥ 0 to limit the update range of synapses dVi = −η (Vi − δ (t − tsp)) (11) (i.e. wj,i +∆wj,i) which distinguish the types of neurons (ex- As shown in Equation (10), N is the number of neurons citatory or inhibitory). The equation w (w + ∆w ) ≥ 0 j,i j,i j,i in the last layer, we set the differences of Vi and δ (t − tsp) could also be converted into the form of −∆wj,i ≤ 1 which as the cost of the network. Then it could be converted into wj,i

1656 Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18)

c SUP 0.9 Equation (11), where the η is the learning rate. Vi will be calculated only one time in the final layer which contains 0.8 the divergence of realistic prediction and supervised teaching signals. 0.7

0.6 Train 4.5 Equivalent Conversion from Membrane Test 0.5

Potential to Synaptic Weights based on STDP Accuracy 0.4 The tuning of Vi and its relationship with network outputs has been discussed. However, the network has to save the learned 0.3 knowledge by consolidating them from membrane potential 0.2 Vi into synapses wj,i. Here we use STDP-like rules [Bi and Poo, 2001; Bengio et al., 2015] to realize this function. 0 5 10 15 20 Iteration Times t+1 t+1 t t ∆wj,i = ηSTDP Vj Vi − Vj Vi (12) Figure 3: The training and test accuracy with MP balanced tuning The Equation (12) is a integration of two different types of STDP rules, one is ∆w ∝ V dVi [Bengio et al., 2015] and j,i j dt 10 classes of handwritten digits with 60, 000 training samples dVj [ ] another is ∆wj,i ∝ dt Vi Bi and Poo, 2001 . wj,i is the and 10, 000 test samples. synaptic weights between neuron i and neuron j, η is STDP In order to understand the mechanisms and individual con- the learning rate of STDP rule, V t and V t+1 are the different j j tributions, we will add each individual principle gradually to temporal states of neuron j. compare and then integrate them together for the ultimate per- 4.6 The Learning Procedure of the Balanced SNN formance. The training and test procedure of the balanced SNN model 5.1 Performance of the Membrane Potential based is shown in Algorithm 1. Balanced Tuning on SNN Algorithm 1 The Balanced SNN Learning Algorithm. We construct an SNN and train it with only LIF based feed 1. Convert the spatial inputs into temporary inputs with random forward (FF) architecture and the membrane potential bal- anced principle based on Equation (3), Equation (4) and E- sampling. Initialize weights wj,i with random uniform distribu- tion, membrane potential states Vi with leaky potential VL, iter- quation (5). ation time Iite, simulation time T , differential time ∆t, learning As shown in Figure 3, the number of neurons in the hidden c rates ηMP , η0, η1, ηSTDP and η ; layer are 100, and the x-axis of the figure is the iteration time, 2. Start Training procedure: the y-axis is the accuracy of SNN on the MNIST classification 2.1. Load training samples; task. We could conclude from the figure that the proposed 2.2. Update Vi by feed forward propagation with Equation (1) membrane potential (MP) balanced tuning principle is work- and Equation (2); MP ing for the network convergence, and the SNN could form the 2.3. Update Vi by the condition of membrane potential based balance with Equation (5); classification ability after the MP based balanced tuning. STP 2.4. Update Isyn by STP based balance with Equation (7), E- quation (8) and Equation (9); 5.2 Performance of the Excitatory and Inhibitory SUP 2.5. Update Vi by weak supervised learning in final layer Neuron Type based Balance Tuning on SNN with Equation (11); 2.6. Equivalent conversion from membrane potential to synaptic The function of inhibitory neurons in the biological system weights based on integrated STDP, and passively update synaptic is a mystery. Some of them play the role on the anti post- weights wj,i based on Equation (12); synaptic membrane potentials (anti-E), and some of them 2.7. Update wj,i by excitatory-inhibitory neuron type based bal- work on blocking activities of other neurons (Block-E) [Zeng ance with Equation (6); ] 2.8. Iteratively train SNNs from Step 2.2 to Step 2.7 and finally et al., 2017 . Here we test the anti-E type of inhibitory neurons and also the proportion of them on SNNs with E- save tuned wj,i; 3. Start test procedure: quation (6) based on the tuned result of membrane potential 3.1. Load test samples; based balanced tuning. 3.2. Test the performance of trained balanced SNN with feed for- The test accuracies of E-I based balanced tuning on SNN ward propagation based on saved wj,i; is shown in Figure 4, from which we could see that the 3.3. Output test performance without cross validation; SNNs with a proper proportion of inhibitory neurons could be trained for the function of classification. When the pro- portion is too big (e.g. bigger than 80%), the network will be failed to learn. More concretely, as shown in Figure 5, SNN 5 Experiments will be well tuned when the proportion of inhibitory neurons We use the standard MNIST [LeCun, 1998] to test the pro- is smaller than 60%. The best proportion of inhibitory neu- posed brain-inspired Balanced SNN model. MNIST contains rons is 30%.

1657 Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18)

0.9 1.0 u 0.8 x spike 0.7 0% 0.6 20% 40% 0.5 60% 0.5 0.4 80% 100% RelativeValues 0.3 Accuracy Performance 0.2

0.1 0.0

0 200 400 600 800 1000 0 5 10 15 20 25 30 35 40 Time (ms) Iteration Times

Figure 4: The test accuracy of E-I based balance tuning Figure 6: The dynamic variables of u and x with input spikes

5.3 Performance of the STP based Balanced Some state of the art performance of SNNs on the MNIST Tuning on SNN benchmark with different structures is shown in Table 1. For two-layer SNNs, the accuracy of 95% is achieved with 6, 400 The STP principle will keep the neurons firing towards a sta- output neurons [Diehl and Cook, 2015]. For three-layered ble frequency. Figure 6 shows the dynamic changes of vari- SNNs, VPSNN gets the accuracy of 98.52% with 4, 800 hid- ables of u and x with the input of spikes. den neurons [Zhang et al., 2018]. While in our work (Bal- For the different frequencies of spikes, the u and x will be anced SNN), we reach the accuracy of 97.90% with only 100 tuned automatically towards the stable output of Isyn which hidden neurons and also reach 98.64% with 6, 400 hidden will be updated by the product of u and x in Equation (9). neurons. To the best of our knowledge, our result is the s- As shown in Figure 7, when the iteration time is 100, the tate of the art performance of pure biological plausible SNNs test accuracy performance on pure MP balanced tuning could on the MNIST benchmark. reach 58%, the MP with feedforward LIF neuron model could reach 89%, and the integration of MP, FF, STP, and 30% in- 6 Conclusion hibitory neurons could reach 97.9%. As a conclusion, the M- P, the proportion of E-I neurons and STP are contributing the There are various learning principles from the brain which balanced effects to SNNs for a better classification accuracy. may help to design better spiking neural network models for Artificial Intelligence. However, how to integrate these brain 5.4 Comparative Studies inspirations together properly for optimal model are still with big challenges. Here we focus on the research of balanced The training of SNNs is very different with DNNs which states of SNNs and try to integrate three kinds of balanced are trained by backpropagation. Here we exclude these none learning principles together. They are the membrane potential biological-plausible tuning methods which firstly train DNNs based balance, the excitatory-inhibitory neuron type based by backpropagation and then convert into SNNs (e.g. the balance and the STP based balance. The model analysis sup- Convolutional SNN in Table 1 with the accuracy of 99.1%), ports the hypothesis that the balanced state of the network is since these efforts are not biological plausible, and the contri- important for network training, and the experimental result butions are mainly from backpropagation. also proves that, even without backpropagation, a better SNN

0.9

0.8 1.0

0.7 0.8 0.6

0.5 0.6

0.4 0.4 0.3 TestAccuracy Accuracy Performance 0.2 MP 0.2 FF+MP 0.1 Best FF+MP+E/I+STP

0.0 0.0 0 20 40 60 80 100 0 20 40 60 80 100 Proportion of Inhibitory Neurons Iteration Times

Figure 5: The test accuracy on different proportion of inhibitory Figure 7: The test accuracy of balanced SNNs with the integration neurons of different rules

1658 Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18)

Architecture Preprocessing (Un-)Supervised Training Type Learning Principles Accuracy Convolutional SNN [Diehl et al., 2015] None Supervised Rate-based Backpropagation 99.1% Two-layer SNN [Diehl and Cook, 2015] None Unsupervised Spike-based Exponential STDP 95% Voltage-driven Plasticity-centric SNN (VPSNN) [Zhang et al., 2018] None Both Spike-based Equilibrium learning + STDP 98.52% Balanced SNN (with 100 hidden neurons) None Both Spike-based Balanced learning + STDP 97.90% Balanced SNN (with 6,400 hidden neurons) None Both Spike-based Balanced learning + STDP 98.64%

Table 1: Classification accuracies of different SNNs on the MNIST task. performance could be achieved based on a deeper integration [Hausser¨ and Mel, 2003] Michael Hausser¨ and Bartlett Mel. of brain-inspired principles. Dendrites: bug or feature? Current opinion in neurobiolo- gy, 13(3):372–383, 2003. Acknowledgments [LeCun et al., 2015] Yann LeCun, Yoshua Bengio, and Ge- This work is supported by Beijing Natural Science Foun- offrey Hinton. . Nature, 521(7553):436– dation (4184103) and the CETC Joint Fund (Grant No. 444, 2015. 6141B08010103). [LeCun, 1998] Yann LeCun. The mnist database of hand- written digits. http://yann. lecun. com/exdb/mnist/, 1998. References [Lee et al., 2016] Jun Haeng Lee, Tobi Delbruck, and [Abbott et al., 2016] LF Abbott, Brian DePasquale, and Michael Pfeiffer. Training deep spiking neural network- Raoul-Martin Memmesheimer. Building functional net- s using backpropagation. Frontiers in Neuroscience, 10, works of spiking model neurons. Nature neuroscience, 2016. 19(3):350–355, 2016. [Maass, 1997] Wolfgang Maass. Networks of spiking neu- [Abraham and Bear, 1996] Wickliffe C Abraham and rons: the third generation of neural network models. Neu- Mark F Bear. Metaplasticity: the plasticity of synaptic ral Networks, 10(9):1659–1671, 1997. plasticity. Trends in neurosciences, 19(4):126–130, 1996. [Okun and Lampl, 2009] Michael Okun and Ilan Lampl. [Alemi et al., 2018] Alireza Alemi, Christian Machens, So- Balance of excitation and inhibition. Scholarpedia, phie Deneve,` and Jean-Jacques Slotine. Learning nonlin- 4(8):7467, 2009. ear dynamics in efficient, balanced spiking networks using [Rueckert et al., 2016] Elmar Rueckert, David Kappel, local plasticity rules. In The 32th AAAI Conference on Ar- Daniel Tanneberg, Dejan Pecevski, and Jan Peters. Re- tificial Intelligence (AAAI-2018), 2018. current spiking networks solve planning tasks. Scientific [Bengio et al., 2015] Yoshua Bengio, Thomas Mesnard, As- reports, 6:21142, 2016. ja Fischer, Saizheng Zhang, and Yuhuai Wu. Stdp as presy- [Soltani and Wang, 2010] Alireza Soltani and Xiao-Jing naptic activity times rate of change of postsynaptic activi- Wang. Synaptic computation underlying probabilistic in- ty. arXiv preprint arXiv:1509.05936, 2015. ference. Nature neuroscience, 13(1):112–119, 2010. [Bi and Poo, 2001] Guo-qiang Bi and Mu-ming Poo. Synap- [Sporea and Gruning,¨ 2013] Ioana Sporea and Andre´ tic modification by correlated activity: Hebb’s postulate Gruning.¨ Supervised learning in multilayer spiking neural revisited. Annual review of neuroscience, 24(1):139–166, networks. Neural computation, 25(2):473–509, 2013. 2001. [Zeng et al., 2017] Yi Zeng, Tielin Zhang, and Bo Xu. Im- [Bohte et al., 2002] Sander M Bohte, Joost N Kok, and Han proving multi-layer spiking neural networks by incorpo- La Poutre. Error-backpropagation in temporally encoded rating brain-inspired rules. Science China Information Sci- networks of spiking neurons. Neurocomputing, 48(1):17– ences, 60(5):052201, 2017. 37, 2002. [Zenke et al., 2015] Friedemann Zenke, Everton J Agnes, [Diehl and Cook, 2015] Peter U Diehl and Matthew Cook. and . Diverse synaptic plasticity mecha- Unsupervised learning of digit recognition using spike- nisms orchestrated to form and retrieve memories in spik- timing-dependent plasticity. Frontiers in Computational ing neural networks. Nature Communications, 6:6922, Neuroscience, 9, 2015. 2015. [Diehl et al., 2015] Peter U Diehl, Daniel Neil, Jonathan Bi- [Zhang et al., 2018] Tielin Zhang, Yi Zeng, Dongcheng nas, Matthew Cook, Shih-Chii Liu, and Michael Pfeif- Zhao, and Mengting Shi. A plasticity-centric approach to fer. Fast-classifying, high-accuracy spiking deep networks train the non-differential spiking neural networks. In The through weight and threshold balancing. In The 2015 In- 32th AAAI Conference on Artificial Intelligence (AAAI- ternational Joint Conference on Neural Networks (IJCNN- 2018), 2018. 2015), pages 1–8. IEEE, 2015. [Zucker and Regehr, 2002] Robert S Zucker and Wade G [Hassabis et al., 2017] Demis Hassabis, Dharshan Kumaran, Regehr. Short-term synaptic plasticity. Annual review of Christopher Summerfield, and Matthew Botvinick. physiology, 64(1):355–405, 2002. Neuroscience-inspired artificial intelligence. Neuron, 95(2):245–258, 2017.

1659