2 the Cerebral Cortex of Mammals
Total Page:16
File Type:pdf, Size:1020Kb
Abstract This thesis presents an abstract model of the mammalian neocortex. The model was constructed by taking a top-down view on the cortex, where it is assumed that cortex to a first approximation works as a system with attractor dynamics. The model deals with the processing of static inputs from the perspectives of biological mapping, algorithmic, and physical implementation, but it does not consider the temporal aspects of these inputs. The purpose of the model is twofold: Firstly, it is an abstract model of the cortex and as such it can be used to evaluate hypotheses about cortical function and structure. Secondly, it forms the basis of a general information processing system that may be implemented in computers. The characteristics of this model are studied both analytically and by simulation experiments, and we also discuss its parallel implementation on cluster computers as well as in digital hardware. The basic design of the model is based on a thorough literature study of the mammalian cortex’s anatomy and physiology. We review both the layered and columnar structure of cortex and also the long- and short-range connectivity between neurons. Characteristics of cortex that defines its computational complexity such as the time-scales of cellular processes that transport ions in and out of neurons and give rise to electric signals are also investigated. In particular we study the size of cortex in terms of neuron and synapse numbers in five mammals; mouse, rat, cat, macaque, and human. The cortical model is implemented with a connectionist type of network where the functional units correspond to cortical minicolumns and these are in turn grouped into hypercolumn modules. The learning-rules used in the model are local in space and time, which make them biologically plausible and also allows for efficient parallel implementation. We study the implemented model both as a single- and multi-layered network. Instances of the model with sizes up to that of a rat-cortex equivalent are implemented and run on cluster computers in 23% of real time. We demonstrate on tasks involving image-data that the cortical model can be used for meaningful computations such as noise reduction, pattern completion, prototype extraction, hierarchical clustering, classification, and content addressable memory, and we show that also the largest cortex equivalent instances of the model can perform these types of computations. Important characteristics of the model are that it is insensitive to limited errors in the computational hardware and noise in the input data. Furthermore, it can learn from examples and is self-organizing to some extent. The proposed model contributes to the quest of understanding the cortex and it is also a first step towards a brain-inspired computing system that can be implemented in the molecular scale computers of tomorrow. The main contributions of this thesis are: (i) A review of the size, modularization, and computational structure of the mammalian neocortex. (ii) An abstract generic connectionist network model of the mammalian cortex. (iii) A framework for a brain-inspired self- organizing information processing system. (iv) Theoretical work on the properties of the model when used as an autoassociative memory. (v) Theoretical insights on the anatomy and physiology of the cortex. (vi) Efficient implementation techniques and simulations of cortical sized instances. (vii) A fixed-point arithmetic implementation of the model that can be used in digital hardware. Keywords: Attractor Neural Networks, Cerebral Cortex, Neocortex, Brain Like Computing, Hypercolumns, Minicolumns, BCPNN, Parallel Computers, Autoassociative Memory ISBN 91-7178-461-6 • TRITA-CSC-A-2006:14 • ISSN-1653-5723 • ISRN-KTH/CSC/A--06/14--SE iii iv Acknowledgements First of all, I want to express my utmost gratitude to my primary supervisor professor Anders Lansner for his support and enthusiasm, and also to my secondary supervisor Dr. Örjan Ekeberg for insightful comments and enlightening discussions. I also want to thank Dr. Anders Sandberg, Mikeal Djurfeldt, Peter Raicevic, and Martin Rehn for productive collaborations and inspiring discussions. I am very grateful for the many invaluable advises I have received from Dr. Erik Fransén and Dr. Jeanette Hellgren Kotaleski. Great thanks to all former and present members of the SANS research group for a nice and pleasant atmosphere and lively discussions. I am very grateful for the support and encouragement I have received from Sofia and my family. This work was partly supported by grants from the Swedish Science Council (Vetenskapsrådet) grant No. 621-2001-2548, 621-2003-6256, 621-2004-3807, and from the European Union (FACETS project, FP6-2004-IST-FETPI-015879). v vi Contents 1 INTRODUCTION ........................................................................................................ 1 1.1 CONTRIBUTIONS ...................................................................................................... 3 1.2 THESIS STRUCTURE ................................................................................................. 4 1.3 ARTICLES................................................................................................................. 5 2 THE CEREBRAL CORTEX OF MAMMALS ......................................................... 7 2.1 AN OVERVIEW OF THE MAMMALIAN NERVOUS SYSTEM......................................... 7 2.2 AN OVERVIEW OF THE CEREBRAL CORTEX ............................................................. 8 2.3 NEURONS AND SYNAPSES IN THE CEREBRAL CORTEX............................................. 9 2.4 CORTICAL LAYERS ................................................................................................ 10 2.4.1 The Layers of Neocortex ............................................................................... 10 2.4.2 Neuron Distribution within Layers ............................................................... 12 2.5 CORTICOCORTICAL CONNECTIONS ........................................................................ 13 2.5.1 Intracortical Connections ............................................................................. 13 2.5.2 Intercortical Connections and Cortical Areas.............................................. 14 2.6 VISUAL CORTEX .................................................................................................... 15 2.7 CORTICAL COLUMNS ............................................................................................. 16 2.7.1 Developmental and Evolutionary Aspects..................................................... 17 2.7.2 Minicolumns.................................................................................................. 18 2.7.3 Hypercolumns ............................................................................................... 20 2.7.4 The Hypercolumn as a Module of Minicolumns ........................................... 21 2.8 THE SIZE OF CORTEX............................................................................................. 21 2.8.1 Volume of Gray and White Matter................................................................ 21 2.8.2 Cortical Area ................................................................................................ 22 2.8.3 Cortical Thickness ........................................................................................ 22 2.8.4 Neuron Numbers ........................................................................................... 23 2.8.5 Synaptic Numbers ......................................................................................... 25 2.8.6 Summary of Anatomical Data ....................................................................... 26 2.8.7 Analysis of Anatomical Data......................................................................... 28 3 AN ABSTRACT MODEL OF NEOCORTEX......................................................... 29 3.1 ALLOMETRIC AND THEORETICAL STUDIES OF CORTEX ......................................... 29 3.1.1 Scaling of Neocortex..................................................................................... 30 3.2 HEBBIAN LEARNING AND ATTRACTOR STATES IN CORTEX ................................... 30 3.3 MODELING CORTEX............................................................................................... 31 3.4 A GENERIC MODEL OF CORTEX ............................................................................ 32 3.4.1 Connectivity of the Minicolumns................................................................... 34 3.4.2 Scaling the Model ......................................................................................... 35 3.5 RELATED WORK .................................................................................................... 35 3.6 INFORMATION CHARACTERISTICS OF A SYNAPSE .................................................. 36 3.7 TIME-SCALES OF CORTICAL OPERATION ............................................................... 37 3.8 COMMUNICATION BETWEEN CORTICAL NEURONS ................................................ 38 vii 4 ATTRACTOR NEURAL NETWORKS................................................................... 39 4.1 ARTIFICIAL NEURAL NETWORKS AND CONNECTIONISM........................................ 39 4.2 ATTRACTOR NETWORKS AND AUTOASSOCIATIVE MEMORY ................................. 40 4.2.1 Performance Measures ................................................................................. 40 4.2.2 Experimental Investigation of