University of Alberta Istvh Stephen Norman Berkeley O A thesis submitted to the Faculty of Graduate Studies and Research in partial fulfillment of the requirements for the degree of Doctor of Philosophy Department of Philosophy Edmonton, Alberta Spring 1997 The author has granted a non- L'auteur a accordé une licence non exclusive iicence allowing the excImhe pumettant à la National Lr'brary of Canada to Bibliothèque nationale du Canada de reproduce, loan, districbute or selI nprodirm, prtkr, distnIbuerou copies of hismcr thesis by any means vmdn des copies de sa thèse de and in any fonn or fommt, maliiog 9ue1que manière et sow quelque this thesis adable to interested forme que ce soit pour mettre des persons. exemplaires de cette thèse a la disposition des personnes intéressées. The author retains ownership of the L'auteur conseme la propriété du copyrightin hislhtr thesis. Neither droit d'auteur qui protège sa thèse. Ni the thesis nor substantiai extracts la th& ni &s extraitssubstantieis de fiom it may be printed or othexwïse celle-ci ne doivent être imprimés ou reproduced with the author's autrement reproduits sans son permission. autorisation. Absttact This dissertation opens with a discussion and clarification of the Classical Computatioaal Theory of Miad (CCTM). An alleged challenge to this theory, which derives fiom research into connectionist systems, is then briefly outIined and connectionist systems are introduced in some detail. Several claims which have been made on behaif of connectiooist systems are then examined, these purport to support the conclusion that connectionist systems provide better models of the mind and cognitive functioning than the CCTM. It is argued that most claims made on behalf of connectionism often amount to litde more than myths. A significant dificuity with standard comectionist research methodology is then describeci. This dificuity derives hmthe fact that connectionist systems are seldom subject to detailed analysis and hterpretabion der tcaining and has serious consequemes for the plausibility of connectionist research in cognitive science and as the bais for a challenge to the CCTM. A technique for network anaiysis is described in detail and the results brn the anaiysis of a particuiar network are then introduced, in order to show how these difficdties can be overcome. The analyzed network exhiibited a number of surpnsing and intriguing propdes. These properties provide the basis for a detailed assessrnent of the network with respect to the CCTM. There then follows a discussion of the results of the network analysis, with respect to two feahues which are commonly associated with the CCTM, namely systematicity and compositionality. It is argued that the network has some properties similar to those associated with the CCTM and exhibits, in a weak seme, compositionality and systematicity. It is aiso argued that the network amounts to a genuinely cognitive theory. This suggests that there is insufncient evidence at the current tirne to determine whether or not comectionism presents a genuine challenge to the CCTM. The plausibility of the clai.that networks pmvide the basis of an alternative to the CCTM is then traced, in part, to a revisionkt tendency in some contempomy histories of comectionist research. Findy, fiture research directions in phiiosophy and cognitive science are suggested. 1 wouid like to thank the many pemple who assisted me with this dissertation. First and foremost, 1 would like to thank my Supavisors. Jeff Pelletier time and thne again went well above and beyond the cdof duty on di bnts, in addition to pmviding insightnil guidance and encouragement. Mïke Dawson (in conjunction with the various members of his Bio1ogica.i Computational Roject Iab) taught me how to be a cognitive scientist and provideci much of the inspiration for the empiricai work (and insights into it) dacniin this dissertation, as weil as providing the necessary facilties. 1 wouid also lüre to thank my examination committee, Wes Cooper, Robert Hadley, Bruce Hunter, Mohan Matthen and Gary Prideaux for thek many helpfûi commentS. In addition, 1 must thank the many people who gave me th& time and insight whilst 1 was writing this dissertation. There are too many to Est, but Ken Aizawa, Bill Bechtel, Andy Brook, Mason Cash, Andy Clark, Jerry Feldmau, Jeff Foss, Steve Giambroney Charles Ling, Tony Maida, Dave Medlery Ruth Michaels, Mawh MiaskyyZenon Pylyshyn, Don Ross, David Rumeihart, Paul Smolensky and Chris Thomton aii deserve special mention. AU the errors which remain are ail my own work. Table of Contents 1. Intniduction 1 IL The Classicd Cornputatbaai Tbeory of Mind (CCTM) 7 ID. Mat1s Coaaectionbm? 26 Introduction 26 Philosophical Objections to The Classicai Computatiod Theory of Mind 26 What 1s Co~mectionism? ProceSSitlg Units Leaming Rules Conclusion IV. The Myths of Coanectionism Introduction Myth 1 :Connectionist systems me biologikally plausible Processing Units Connections Myth 2: Connectionisl systems me consktent with mltime comtraints vonprocessing Myth 3: Connectionist Systems exhibit gracefl degradation Myîh 4: Connectionist systems are good generalizers Myth 5: Recent connectionist systenr have shown that Minsky und Papert were wrong Conclusion V. An Empirical Study: The Interpretation of The Logic Network, LI0 Introduction Ruies, Representations and Comectionist Systems McCloskey's Critique of the Comectionist Raearch Program Understanding Trained Comectionist Networks Bechtel and Abrahamsen's Logic Problem The Network L 10 The hoblem Encoding Scheme of LI0 The Training of LlO The Network Analysis Technique The Analysis of Network LI0 DefZnite Features and Inference 'Rules' Conclusion VI. Connectionist Networks and The CCTM Introduction The CCTM Again S ystematicity Compositionalîty Ruies and Cognitive Systems Conclusion VII. Conclusion: Connectionhm, Prcsent, Past and L;btPre Introduction: The Present History: The Past Furtber Research Directions: The Future Cognitive Science Philosophy List of Tables Table 2-1: Example of a Turing Machine Table 12 Table 5-1 :Examples of Valid Infetences hmBechtel and Abrabunsen's (1991) Logic Problem Set. Table 52: The Unary Definite Features Found in Band B of Hïdden Unit 4 of Network LI0 Table 13: The Binary Dennite Features Found in Band B of Hidden Unit 4 of Network LI0 Table 5-4: Interpretations of the Bands of the Hidden Units of Network LI0 97-98 Table 5-5: The Patterns of Bands Produced by LI0 for each Pmblem Type and the Properties Associated with each Band, as compared to the Properties Associated with the Inférence Rules of Natural Deduction Table 61: Network Rule for Modus Ponens, as Compared to the Traditional Rule Table 62: Network Rule for Type (i) Aitemative Syllogism, as Compared to the Traditional Rule List of Fimires Figure 2-1: Diagram of a Turing Machine Figure 2-2: Example of a Maze Figure 3-1: Detail of a Connectionist RocdgUnit Figure 3-2: A Step Activation Function Figure 3-3: A Sigrnoidal Activation Function Figure 3-4: A Gaussian Activation Function Figure 3-5: Detail of a Connection Weight Figure 3-6: Layen of Connections and Weights in a Connectionist Network Figure 5-1: The Structure and Encoding Scheme of the Network LI0 Figure 5-2: An Example of a Jittered Density Plot Figun 5-3: An Example Jittered Density Plot of a Hidden Value Unit Figmre 54: Jittered Density Plots for the 10 Hidden Units of Network LI0 Figure 5-5: The Jittered Density Plot of Hidden Unit 4 of Network L 10 1 introduction Issues arising hmthe computationai view ofthe mind will be central to this dissertation. There is a long tradition of philosophem taking inspiration for th& philosophicai theorizing hmmechanical devices. This was as tme at the time of Descartes, as it is toâay (see Meyering 1989). One particular contemporary instance of this tendency is the consideration of computational systems as a stnicturing metaphor for the mind (see Boden 1981). This strucnuing metaphor has inspimi an ambitious research program in cognitive science which, in turn, has inspird a considerable body of philosophical theorking about the mind. Central to both avenues of research are questions concerning representatiom or tokens. These questions arise as a direct consequeme of the adoption of the computationai metaphor. There are certain presumptions that have been made about the nature of representation or tokens, which have been centrai to both the philosophicai and tecbnical research programs. These presumptions define what has been called (e-g. by Fodor and Pylyshyn 1988) 'classical' cognitive science and philosophical theorizing about the mind. However, recently these suppositions have corne under attack. Advocates of the 'comectionist' approach have claimed that these suppositions are incorrect. Although it is usually assumed that there is some kind of incompatibility between the connectionist and classical positions, I will argue in this dissertation that these incompatibilities are not as great as has been assumed. In the chapter which immediately follows this one, introduce a paradigm exampie of a computational device, the Turing machine. Turing machines have played a signifîcant role in philosophical theorizing about the mind, especidy with respect to fùnctionalism. This Turing machine-like conception of the mind has (in conjmction with other factors, such as Fodor's (1975) language of thought hypothesis) underWntten a view of the mind, which I term the 'classical computationai theory of mind' (CCTM), which is central to much of both contemporary philosophy of mind and cognitive science. In this chapter 1 atternpt to consider in detail the properties wbich are supposeci by the CCTM to be shared by both Turing machina and minds. The purpose here is to develop a clear picture of the position, so that the alleged challenge to the CCTM posed by connectionist systems can be assesseci. Although the CCTM is weU established in the philosophical literature, it tas not been without its critics. Philosophers, such as Searle (1980), and Dreyfh (1991), have argued on a number of grounds that this position is deeply flawed.