Artificial Neural Nets
Total Page:16
File Type:pdf, Size:1020Kb
Artificial Neural Nets: a critical analysis of their effectiveness as empirical technique for cognitive modelling Peter R. Krebs A thesis submitted in fulfillment of the requirements for the degree of Doctor of Philosophy. University of New South Wales August 2007 COPYRIGHT, AUTHENTICITY, and ORIGINALITY STATEMENT I hereby grant the University of New South Wales or its agents the right to archive and to make available my thesis or dissertation in whole or part in the University libraries in all forms of media, now or here after known, subject to the provisions of the Copyright Act 1968. I retain all proprietary rights, such as patent rights. I also retain the right to use in future works (such as articles or books) all or part of this thesis or dissertation. I also authorise University Microfilms to use the 350 word abstract of my thesis in Dissertation Abstract International. I have either used no substantial portions of copyright material in my thesis or I have obtained permission to use copyright material; where permission has not been granted I have applied/will apply for a partial restric- tion of the digital copy of my thesis or dissertation, and I certify that the Library deposit digital copy is a direct equivalent of the final officially approved version of my thesis. No emendation of content has occurred and if there are any minor variations in formatting, they are the result of the conversion to digital format, and I hereby declare that this submission is my own work and to the best of my knowledge it contains no materials previously published or written by another person, or substantial proportions of material which have been accepted for the award of any other degree or diploma at UNSW or any other educational institution, except where due acknowledgment is made in the thesis. Any con- tribution made to the research by others, with whom I have worked at UNSW or elsewhere, is explicitly acknowledged in the thesis. I also declare that the intellectual content of this thesis is the product of my own work, except to the extent that assistance from others in the project’s design and conception or in style, presentation and linguistic expression is acknowledged. Signed ................................... Date ............................... iii Abstract This thesis is concerned with the computational modelling and sim- ulation of physiological structures and cognitive functions of brains through the use of artificial neural nets. While the structures of these models are loosely related to neurons and physiological structures ob- served in brains, the extent to which we can accept claims about how neurons and brains really function based on such models depends largely on judgments about the fitness of (virtual) computer experi- ments as empirical evidence. The thesis examines the computational foundations of neural models, neural nets, and some computational models of higher cognitive functions in terms of their ability to provide empirical support for theories within the framework of Parallel Dis- tributed Processing (PDP). Models of higher cognitive functions in this framework are often presented in forms that hybridise top-down (e.g. employing terminology from Psychology or Linguistics) and bottom-up (neurons and neural circuits) approaches to cognition. In this thesis I argue that the use of terminology from either approach can blind us to the highly theory-laden nature of the models, and that this tends to produce overly optimistic evaluations of the empirical value of com- puter experiments on these models. I argue, further, that some classes of computational models and simulations based on methodologies that hybridise top-down and bottom-up approaches are ill-designed. Conse- quently, many of the theoretical claims based on these models cannot be supported by experiments with such models. As a result, I question the effectiveness of computer experiments with artificial neural nets as an empirical technique for cognitive modelling. Contents 1 Introduction 1 1.1 Research and Thesis Outline . ........ 1 1.1.1 Part One . ............... 2 1.1.2 Part Two . ...................... 3 1.1.3 Part Three . ................. 4 1.2AShortHistoryofIdeas.................... 6 1.3 Setting the Stage . ................ 9 1.4 Minds and Models ........................ 13 1.5 Summary . ................... 18 I Representation and Computing 21 2 Models and Simulations 23 2.1TheConceptofModel...................... 23 2.2 Models as Representations . ................. 30 2.2.1 Representations ..................... 32 2.2.2 Simulations ....................... 38 2.3Methodologies.......................... 39 2.4 Summary . ......................... 42 3 Computational Foundations 43 3.1 Symbol Systems . ...................... 46 3.1.1 Formal Computation . ....... 46 3.1.2 Limitations of Turing Machines . .... 56 3.1.3 Digital Computing . ........ 59 3.1.4 Parallel Computing . ............... 62 3.2 Other Computing systems . ................. 67 v vi CONTENTS 3.2.1 Analog Computing . ................ 68 3.2.2 Neural Computing . .............. 69 3.2.3 Other Forms of Computing . 70 3.2.4 Implementation Independence . 70 3.3 Computation as Interpretation . ......... 71 3.4 Machines and Semantics .................... 78 3.5 Summary . ............................ 79 II Models and Reality 81 4 Virtual Models 83 4.1 Virtual Scientific Experimentation . 85 4.1.1 Mathematical Models . .......... 87 4.1.2 Methodologies ...................... 92 4.2 Implementation . ........................ 94 4.3 Computer experiments . .................. 97 4.3.1 Computer models as scientific experiments . 99 4.3.2 Levels of Explanation . ..............101 4.3.3 Virtual Models . .........102 4.4 Summary . .................103 5 Models of Neurons 105 5.1BiologicalModels........................106 5.1.1 The Hodgkin and Huxley Model . .........107 5.1.2 Neural coding . ....................109 5.2 Mathematical Neurons . ...............111 5.3 Relation to Reality . ................118 5.4 Summary . ........................120 6 Artificial Neural Nets 123 6.1Background...........................124 6.1.1 The limitations . ....................126 6.1.2 Plausibility . ..............129 6.1.3 Unsupervised Learning . .........131 6.1.4 Architectures . ...............132 6.1.5 Implementation . ...................136 CONTENTS vii 6.2 Universal Frameworks . ....................140 6.2.1 Labels ...........................142 6.2.2 Simplicity . ....................144 6.3 Summary . ............................147 III Models and Explanation 149 7 Models and Evidence 151 7.1 Verification and Validation . ..................153 7.2 Validation of Models ......................156 7.2.1 Visualization . ......................156 7.2.2 Measuring and Imaging . .........159 7.3 Limits of Technology . ....................163 7.4 Other Evidence . ....................167 7.5 Summary . .........................168 8 Models of Cognition 171 8.1 Higher Level Models . .................172 8.2 Interpreting Results . ..................174 8.2.1 Pronouncing English . ........175 8.2.2 Structure in time . ........178 8.2.3 Moral virtues . .......185 8.2.4 Cluster Analysis . ..............188 8.3 Summary . ......................191 9 Conclusion 193 9.1NeuralNetsandPDP......................194 9.1.1 Neurological Inspiration . ..............194 9.1.2 Holding out Hope . ...............195 9.1.3 Computational Sufficiency ...............196 9.1.4 Psychological Accuracy . ..........203 9.1.5 Parallel Processing . ..............204 9.1.6 Distributed Representation . ......205 9.1.7 Learning . .......................206 9.2 Bridging the Explanatory Gap . .....207 9.3 Neurological Plausibility . .................212 viii CONTENTS 9.4ClosingRemarks........................212 Bibliography 215 Author Index 231 List of Figures 235 Chapter 1 Introduction Because mind has shown itself to behave as a nearly decomposable system, we can model thinking at the symbolic level, with events in the range of hundreds of milliseconds or longer, without concern for details of implementation at the ‘hardware’ level, whether the hardware be brain or computer (Simon, 1995, 83). Many of the current philosophical issues in Cognitive Science that are concerned with inquiries into human cognition and intelligence involve computation in some form. These inquiries may include the search for an artificial intelligence, but are mainly concerned with understanding and explaining real human intelligence. Computation is not only at the core of the philosophical foundations of Cognitive Science, but is also the primary tool for models and simulations in Cognitive Science. This thesis concerns computational models and simulations of cognitive func- tions and processes of the human mind. There are many approaches to modelling and I will focus on models based on artificial neural nets. The questions I will investigate are (1) how much can we infer about the human mind from models, and (2) are some of the approaches and architectures of models really suitable as empirical devices? 1.1 Research and Thesis Outline In this introduction, I will outline the fundamental philosophical ideas, theories and assumptions that form the framework for Cognitive Sci- ence and the field of artificial intelligence (AI). First I will re-trace 1 2 CHAPTER 1. INTRODUCTION the various ideas of the computational theory of mind and the appli- cation of this theory as the philosophical foundation of an AI. A short review of the major ideas that led to the development of the computing machinery