Stochastic Resonance and Finite Resolution in a Network of Leaky
Total Page:16
File Type:pdf, Size:1020Kb
Stochastic resonance and finite resolution in a network of leaky integrate-and-fire neurons Nhamoinesu Mtetwa Department of Computing Science and Mathematics University of Stirling, Scotland Thesis submitted for the degree of Doctor of Philosophy 2003 ProQuest Number: 13917115 All rights reserved INFORMATION TO ALL USERS The quality of this reproduction is dependent upon the quality of the copy submitted. In the unlikely event that the author did not send a com plete manuscript and there are missing pages, these will be noted. Also, if material had to be removed, a note will indicate the deletion. uest ProQuest 13917115 Published by ProQuest LLC(2019). Copyright of the Dissertation is held by the Author. All rights reserved. This work is protected against unauthorized copying under Title 17, United States C ode Microform Edition © ProQuest LLC. ProQuest LLC. 789 East Eisenhower Parkway P.O. Box 1346 Ann Arbor, Ml 48106- 1346 Declaration I declare that this thesis was composed by me and that the work contained therein is my own. Any other people’s work has been explicitly acknowledged. i Acknowledgements This thesis wouldn’t have been possible without the assistance of the people acknowledged below. To my supervisor, Prof. Leslie S. Smith, I wish to say thank you for being the best supervisor one could ever wish for. I am specifically thankful for giving me the freedom to walk into your office anytime. This made it easy for me to approach you and discuss any research issues as they arose. Dr. Amir Hussain, a note of thanks for advice on the signal processing aspects of my research. Dr. Mike Roberts and Dr. Douglas McLean, I appreciate your significant help on the mathematical aspects of this thesis. Katie Howie, I am grateful for your help with the statistical analysis of the results. CSG (Sam, Graham and Claire) and Frank, you were excellent on assisting me with the technical aspects of my work. Dr. David Sterratt and Dr. Catherine Breslin, thank you for taking time to read the thesis and providing useful comments. Laura Kelling, you are a genius at proofreading, much appreciated. Dr. Hans Plesser, your work was the original inspiration to this thesis. I thank you for your readiness to answer any questions during the conducting of this research. CCCN, thank you for useful feedback during research seminars. Stirling University, thank you for funding my research. This work would not have been possible without the funding. Last but not least, I wish to thank Thandi for being there for me during the highs and the lows of this work. ii A bstract This thesis is a study of stochastic resonance (SR) in a discrete implementation of a leaky integrate-and-fire (LIF) neuron network. The aim was to determine if SR can be realised in limited precision discrete systems implemented on digital hardware. How neuronal modelling connects with SR is discussed. Analysis techniques for noisy spike trains are described, ranging from rate coding, statistical measures, and signal pro cessing measures like power spectrum and signal-to-noise ratio (SNR). The main problem in computing spike train power spectra is how to get equi-spaced sample amplitudes given the short duration of spikes relative to their frequency. Three different methods of comput ing the SNR of a spike train given its power spectrum are described. The main problem is how to separate the power at the frequencies of interest from the noise power as the spike train encodes both noise and the signal of interest. Two models of the LIF neuron were developed, one continuous and one discrete, and the results compared. The discrete model allowed variation of the precision of the simulation values allowing investigation of the effect of precision limitation on SR. The main difference between the two models lies in the evolution of the membrane potential. When both models are allowed to decay from a high start value in the absence of input, the discrete model does not completely discharge while the continuous model discharges to almost zero. The results of simulating the discrete model on an FPGA and the continuous model on a PC showed that SR can be realised in discrete low resolution digital systems. SR was found to be sensitive to the precision of the values in the simulations. For a single neuron, we find that SR increases between 10 bits and 12 bits resolution after which it saturates. For a feed-forward network with multiple input neurons and one output neuron, SR is stronger with more than 6 input neurons and it saturates at a higher resolution. We conclude that stochastic resonance can manifest in discrete systems though to a lesser extent compared to continuous systems. Contents 1 Introduction 1 1.1 Aim of the thesis ...................... ........................................................................... 2 1.2 Methodology ........................................................................................................ 2 1.3 Contribution to knowledge .................................................................................. 3 1.4 Outline of thesis chapter by chapter ................................................................... 4 2 Background 6 2.1 Neurophysiology of neurons ........................................................................... 7 2.2 Neuronal modelling ............................................................................................... 8 2.2.1 Detailed m odels .......................................................................................... 9 2.2.2 Simple m odels ............................................................................................. 11 2.3 Noise in neurons .................................................................................................. 18 2.3.1 Origins of noise in neurons ....................................................................... 18 2.3.2 Some benefits of noise ............................................................................. 19 2.3.3 Noise and signal processing .................................................................... 20 2.3.4 Noise and neural c o d in g .......................................................................... 21 2.4 Models of n o is e ..................................................................................................... 23 2.4.1 Noisy th resh o ld .......................................................................................... 24 2.4.2 Noisy reset ................................................................................................ 25 2.4.3 Noisy in teg ratio n ....................................................................................... 26 2.5 Artificial neural networks in hardw are ................................................................ 27 iv 2.5.1 Mapping ANN onto hardware ............................................................... 28 2.5.2 Quantisation and limited precision in DSPs ...................................... 28 2.6 Field Programmable Gate A rrays ..................................................................... 31 2.6.1 FPGA A rchitectures ............................................................................... 33 2.6.2 Static RAM Technology ......................................................................... 34 2.6.3 Fuse and A n ti-fu se .................................................................................. 35 2.6.4 Hardware description languages ............................................................ 36 2.6.5 Applications ............................................................................................... 39 2.7 Summary .............................................................................................................. 42 3 Stochastic Resonance 44 3.1 What is Stochastic Resonance? ........................................................................ 45 3.2 History of stochastic resonance ........................................................................ 48 3.3 Quantifying stochastic resonance in neurons .................................................. 50 3.3.1 Power spectrum ......................................................................................... 51 3.3.2 Interspike interval h isto g ram s ............................................................... 51 3.3.3 Signal-to-noise ratio ............................................................................... 53 3.3.4 Other m easures ......................................................................................... 55 3.4 Aperiodic vs periodic stochastic reso n an ce ..................................................... 55 3.5 Stochastic resonance in neuronal m odels ......................................................... 56 3.5.1 Threshold d e te c to r .................................................................................. 56 3.5.2 Stochastic resonance in n eu ro n s ............................................................ 57 3.6 Tools for modelling S R ........................................................................................ 60 3.6.1 Digital simulation ...................................................................................... 61 3.6.2 Analogue sim u latio n ............................................................................... 61 3.6.3 Physiological experiments ...................................................................... 62 3.7 Some of the benefits of stochastic re so n a n c e .................................................. 62