AI Magazine Volume 9 Number 4 (1988) (© AAAI) Connectionism and Information- B. Chandrasekaran, Ashok Goel, and Dean Allemang 24 AI MAGAZINE Processing Abstractions The Message Still Counts More Than the Medium Challenge to modeling human cognition and per- Connectionism challenges a the Symbolic View ception. basic assumption of much of AI, that Connectionism and symbolicism mental processes are best viewed as algo- Much of the theoretical and empirical both agree on the idea of intelligence rithmic symbol manipulations. Connec- research in AI over the past 30 years as information processing of tionism replaces symbol structures with has been based on the so-called sym- representations but disagree about the distributed representations in the form of bolic paradigm—the thesis that algo- medium in which the representations weights between units. For problems close rithmic processes which interpret dis- reside and the corresponding process- to the architecture of the underlying crete symbol systems provide a good machines, connectionist and symbolic ing mechanisms. We believe that basis for modeling human cognition. approaches can make different representa- symbolicism and connectionism carry tional commitments for a task and, thus, Stronger versions of the symbolic a large amount of unanalyzed assump- can constitute different theories. For com- paradigm were proposed by Newell tional baggage. For example, it is not plex problems, however, the power of a (1980) and Pylyshyn (1984). Newell’s clear if many of the theories cast in system comes more from the content of physical symbol system hypothesis is the symbolic mode really require this the representations than the medium in an example of the symbolic view. form of computation and what role which the representations reside. The con- Pylyshyn argues that symbolicism is the connectionist architecture plays nectionist hope of using learning to obvi- not simply a metaphoric language to in a successful connectionist solution ate explicit specification of this content is talk about cognition but that cogni- to a problem. We examine the undermined by the problem of program- tion literally is computation over ming appropriate initial connectionist assumptions and the claims of con- symbol systems. It is important to architectures so that they can in fact nectionism in this article to better learn. In essence, although connectionism note that the symbolic view does not understand the nature of representa- is a useful corrective to the view of mind imply a restriction to serial computa- tions and information processing in as a Turing machine, for most of the cen- tion or a belief in the practical general. tral issues of intelligence, connectionism sufficiency of current von Neuman is only marginally relevant. computer architectures for the task of The Nature of Representations: understanding intelligence. Often, dis- agreements about symbolicism turn Roots of the Debate out to be arguments for computer The symbolic versus connectionist architectures that support some form debate in AI today is the latest version of parallel and distributed processing of a fairly classic contention between rather than arguments against compu- two sets of intuitions, each leading to tations on discrete symbolic represen- a weltanschauung about the nature of tations. intelligence. The debate can be traced In spite of what one might regard as in modern times at least as far back as significant AI achievements in provid- Descartes (to Plato if one wants to go ing a computational language to talk further back) and the mind-brain dual- about cognition, recurring challenges ism known as Cartesianism. In the have been made to the symbolic Cartesian world view, the phenomena paradigm. A number of alternatives of the mind are exemplified by lan- have been proposed whose basic guage and thought. These phenomena This article is an expanded version of part of a larger mechanisms are not in the symbol- can be implemented by the brain but paper entitled “What Kind of Information Processing Is interpretation mode. Connectionism are seen to have a constituent struc- Intelligence? A Perspective on AI Paradigms and a Pro- posal” that appears in Foundations of Artificial Intelli- is one such alternative. It revives the ture in their own terms and can be gence: A Source Book, eds. Derek Partridge and Yorick basic intuitions behind the early per- studied abstractly. Symbolic logic and Wilks, Cambridge University Press, 1988. A much ceptron theory (Rosenblatt 1962) and other symbolic representations are abridged version appears in Brain and Behavioral offers largely continuous, nonsymbol- often advanced as the appropriate Sciences 11(1) as a commentary on a paper on interpreting processes as a basis for tools for studying these phenomena. connectionism. WINTER 1988 25 Functionalism in philosophy, infor- ther an inference nor a product of any ences also exist between them in mation-processing theories in psy- kind of information processing; computational terms. As the size of chology, and the symbolic paradigm rather, it is a one-step mapping from the multiplicands increases, the algo- in AI all share these assumptions. stimuli to categories of perception rithmic solution suffers in the Although most of the intuitions that made possible by the inherent proper- amount of time it takes to complete drive this point of view arise from a ties of the perceptual architecture. All the solution, and the slide rule solu- study of cognitive phenomena, the the needed distinctions are already tion suffers in the amount of precision thesis is often extended to include directly in the architecture, and no it can deliver. perception; for example, for Bruner processing over representations is Let us call the algorithmic and slide (1957), perception is inference. In its needed. rule solutions S1 and S2. Consider modern version, the Cartesian view- We note that the proponents of the another solution, S3, which is the point appeals to the Turing-Church symbolic paradigm can be happy with simulation of S2 by an algorithm. S3 hypothesis as a justification for limit- the proposition that mental phenome- can simulate S2 to any desired accura- ing attention to symbolic models. na are implemented by the brain, cy. However, S3 has radically different These models ought to suffice, the which might or might not have a properties from S1 in terms of the argument goes, because even continu- symbolic account. However, the anti- information that it represents. S3 is ous functions can be computed to Cartesian theorists cannot accept this closer to S2 representationally. Its arbitrary precision by a Turing duality. They want to show the mind symbol-manipulation character is at a machine. as epiphenomenal. To put it simply, lower level of abstraction altogether. The opposing view springs from the brain is all there is, and it isn’t a Given a black-box multiplier, ascrip- skepticism about the separation of the computer. tion of S1or S2 (among others) about mental from the brain-level phenome- Few people in either camp subscribe what is really going on results in dif- na. The impulse behind anti-Carte- to all the features in our descriptions. ferent theories about the process. sianism appears to be a reluctance to Connectionism is a less radical mem- Each theory makes different represen- assign any kind of ontological inde- ber of the anti-Cartesian camp tational commitments. Further, pendence to the mind. In this view, because many connectionists do not although S2 is analog, the existence of the brain is nothing like the symbolic have any commitment to brain-level S3 implies that the essential character- processor of Cartesianism. Instead of theory making. Connectionism is also istic of S2 is not continuity but a radi- what is seen as the sequential and explicitly representational—its main cally different sense of representation combinational perspective of the sym- argument is only about the medium and processing than S1. bolic paradigm, some of the theories of representation. The purpose of the The connectionist models relate to in this school embrace parallel, holis- preceding account is to help in the symbolic models in the same way tic (that is, they cannot be explained understanding the philosophical S2 relates to S1. An adequate discus- as compositions of parts), nonsymbol- impulse behind connectionism and sion of what makes a symbol requires ic alternatives; however, others do not the rather diverse collection of bedfel- more space and time than we current- even subscribe to any kind of informa- lows that it has attracted. ly have (Pylyshyn [1984] provides a tion processing or representational thorough and illuminating discus- language in talking about mental phe- Symbolic and NonSymbolic sion), but the following points are use- nomena. Those who do accept the ful. A type-token distinction exists: Representations need for information processing of Symbols are types about which some type nevertheless reject process- To better understand the difference abstract rules of behavior are known ing of labeled symbols and look to between the symbolic and nonsym- and can be brought into play. This dis- analog, or continuous, processes as bolic approaches, let us consider the tinction leads to symbols being labels the natural medium for modeling the problem of multiplying two positive that are interpreted during the pro- relevant phenomena. In contrast to integers. We are all familiar with algo- cess; however, no such interpretations Cartesian theories, most of