Associationism: Not the Cliff Over Which To Push Connectionism

RENE J. JORNA and WILLEM F G. HASELAGER

Management Informatics, University of Groningen, Groningen, The Netherlands and Department of Psychology, Free University Amsterdam Amsterdam, The Netherlands

CONTENTS Page Synopsis 279 1. Introduction 280 2. Associationism: a conceptual structure 284 2.1 Couples and couple signs 286 2.2 The nature of the relationship 288 2.3 Two couple elements 290 2.4 Similarity and contiguity 291 2.5 Recursive associations 297 3. Conclusions concerning the analysis and interpretation of association 300 Acknowledgements 304 References 305

SYNOPSIS

In the present discussion in artificial intelligence and intelligent architectures several authors have come to the conclusion that the overall domination of classical cognitive science has gone. One of the arguments

279 Volume 4. Nos. 3-4, 1994 Associationism: Not the Cliff Over Which to Push Connectionism in this discussion concerns the interpretation and status of the notion of association. Cognitive scientists equated association with behaviorism and because of the downfall of behaviorism they rejected the history of associationism. Opponents of symbolic A.I., especially connectionists, favoured associationism as necessary in explaining learning and reasoning. In this article we consider the notion of association to be ambivalent. In order to clear the ground we treat associationism as a conceptual structure consisting of five main principles, dealing with the interpretation of (couple) elements (judgements, ideas, stimuli, neurons, symbols, etc.) and the systematicity in the construction of elements and relations. On the basis of these principles historical material will be reviewed. We conclude that associationism should not be used as a simplistic label (positive or negative) in the debate between classical cognitivists and connectionists.

1. INTRODUCTION

In the present discussion in artificial intelligence and intelligent architectures several authors have come to the conclusion that the overall domination of classical cognitive science has gone (Ramsey, Stich & Rumelhart, 1991; Bechtel & Abrahamsen, 1991). The most extreme position holds that cognitive science started in 1956 (Gardner, 1985) and stopped to be interesting in 1986 with the publication of the book on Parallel Distributed Processing (McClelland, Rumelhart & the PDP-Group, 1986). This does not imply that PDP or connectionist models do not study cognition. Their theoretical background, however, is different from the approach of classical cognitive science. In the extreme it is stated by some connectionists that future research on cognitive modelling can only be fruitful within a PDP or connectionist mode. This claim is a very strong one and in several articles it has been defended (Bechtel, 1985; Smolensky, 1988) and attacked (Fodor & Pylyshyn, 1988; Levelt, 1989). One of the arguments in this discussion concerns the interpretation and status of the notion of association. The problem with this notion and its implications for cognition is that its relevance is interpreted into two opposite directions.

280 R.J. Joma and W.F.G. Haselager Journal of Intelligent Systems

The first direction is developed by Levelt (1989) who claims that connectionism has to be rejected, because this endeavour is just a new version of the old doctrine of association, called, associationism. In this he follows Fodor & Pylyshyn who say amongst others: "the idea that the brain is a neural network motivates the revival of a largely discredited Associationist psychology." (1988, p. 63). In the history of psychology, behaviorism is a version of associationism. In the 1950s cognitive science succeeded in defeating behaviorism. One of the weak points in behaviorism is that it rests on the notion of association. Connectionism as a new research area in the mind-and-brain sciences also rests on the notion of association. Because cognitive science already showed the untenability of behaviorism that is based on the notion of association as fundamental for research on mind-and-brain, connectionism rests on a mistake. A similar negative evaluation of associationism with respect to neurophysiological theories can be found in Buckingham (1984), who considers associationism as a price to pay for a 'cerebral connectionistic' approach.

The second, opposite, direction is worked out by Bechtel (1985). He claims that cognitive science does not take into account learning. According to him one of the reasons for this omission is the neglect of the important role of association. Because intelligence is based among other things upon association and because the only true follow-up of associationism is to be found in connectionist networks, the high-days of classical cognitive science are over. There is no cognition without association. In our opinion Bechtel is right with this last statement but for reasons different from the ones he offers.

Both opposite perspectives emphasize the role of association. This is done, however, without giving a detailed characterization of this notion. In this article our goal is twofold. First of all we will give a rather detailed analysis of historical, 'classical' associationism. We think associationism is a rather complex conceptual structure, not just a simplistic label. In the second place we will use this analysis to assess the value of associationism in the evaluation of connectionism as a modelling approach to the mind.

281 Volume 4, Nos. 3-4, 1994 Associationism: Not the Cliff Over Which to Push Connectionism We hope to show that associationism should not be seen as a 'mistake' of which connectionism is to be accused.

Several attempts have been made to incorporate the history of associationism in (cognitive) psychological research (Robinson, 1932; Deese, 1966). Besides the comments of Bechtel, Fodor & Pylyshyn and Levelt a recent and influential one is from Anderson & Bower (1973). They tried to link their Human Associative Memory (HAM) model to the history of (classical) associationism. They argue that associationism does not represent a specific position, but only a theoretical tradition. According to Anderson & Bower this tradition is characterized by four features, namely associationism is connectionistic, it is reductionistic, it is sensationalistic and it is mechanistic. The problem with this description is that the characterization is not quite satisfactory. We will discuss the four features in brief.

The connectionistic feature implies that "ideas, sense data or similar mental elements are associated together in the mind through experience. This means that it is connectionistic" (Anderson & Bower, 1973, p. 10). The essence of associationism is of course that connections are made. If this definition is used, nearly all (cognitive) psychology is connectionistic. The important point, of course, is how and why connections arise. The history of associationism and present day psychology show that different answers have been given to these questions.

The second feature says that associationism is reductionistic, which means that "ideas can ultimately be decomposed into a basic stock of 'simple ideas'" (Anderson & Bower, p. 10). This feature is rather ambiguous. Reductionist might refer to decomposition and to composition. Warren (1921) shows that many associationists, for example John Stuart Mill, Tucker, and Brown, start with simple elements that can be combined into complex ones, but it is also argued that sometimes the decomposition from complex ideas to simple ones is not possible, because the complex ones show characteristics that cannot be reduced to the simple ones. It is,

282 R.J. Jorna and W.F.G. Haselager Journal of Intelligent Systems therefore, not precisely to the point to conclude that reductionism is a defining feature of associationism. A feature of associationism is that it studies compounds or composites. The research question in this case is by what regularities and under what preconditions compounds are likely to occur.

According to Anderson & Bower (1973, p. 10): "The simple ideas are to be identified with elementary, unstructured sensations. [...] Because it identifies the basic components of the mind with sensory experience, associationism is sensationalistic." This feature is questionable. It seems to mean that those philosophers and psychologists who only study the connections and combinations of the mind and leave perception and sensation outside the discussion are not associationists. However, it is one issue to study the relation between elements in perception and elements in cognition, but it is another issue to study the train of thought, reasoning and memory as such. One does not necessarily have to be an empiricist, not in philosophy nor in psychology, to be an investigator of the mind in the associationistic tradition.

The last feature of associationism implies that "simple, additive rules serve to predict the properties of complex associative configurations from the properties of underlying simple ideas." (Anderson & Bower, 1973, p. 10). According to Anderson & Bower associationism is mechanistic (i.e., non-intentional). Thinking, reasoning and memory processes are only to be conceived in mechanical terms. However, it is not self-evident that the elements in the association process are all like the bricks that constitute a house or the particles such as described in classical mechanics (which heavily influenced British associationism) to be connected in a simple, straightforward way. Ideas, concepts or symbols in the mind are different from bricks and particles in that they have a syntax.

We have to conclude that Anderson & Bower's characterization of the theoretical tradition of associationism is strongly influenced by their attempts to connect the Human Associative Memory model with classical

283 Volume 4, Nos. 3-4, 1994 Associationism: Not the Cliff Over Which to Push Connectionism associationists (especially Aristotle; e.g. Anderson & Bower, 1973, p. 186). The description does not do justice to the attempts of many philosophers and psychologists in the distant and recent past to describe, analyze and conceptualize the phenomenon of the combination of elements in perception, thinking and remembering. Anderson & Bower try to give characteristics of what has been called classical associationism. In this stereotype view of the history of philosophy and psychology Leibniz, Descartes and Kant are not associationists. But it is also possible to take the phenomenon known as association and study the ways in which it is defined, analyzed and explained.

Because association in all its shades of meaning still plays a very important role in cognitive science and connectionism we will first try to give a more balanced overview of associationism. We will illustrate this description with several historical examples. After that we will discuss the claims of Fodor & Pylyshyn, Levelt and Bechtel and make some conclusions about the role of association in the study of intelligent systems.

2. ASSOCIATIONISM: A CONCEPTUAL STRUCTURE

Research on association has a very long and very rich historical and conceptual tradition. Because various other names have been given to association and various related concepts have been formulated in relation to the notion, we suggest conceiving the intellectual tradition concerning association to be a conceptual structure consisting of several points of departure and working with and based upon entities and relations. A conceptual structure can be compared with a paradigm in the Kuhnian sense (1962) with one big difference. A cornerstone in Kuhn's view is the incommensurability of a similar notion in different paradigms. In the perspective of a conceptual structure the emphasis is on the continuity of a larger intellectual tradition (Hacking, 1975; Barth, 1977; Laudan, 1977).

In the conceptual structure of associationism the explanadum is the simultaneous appearance of mental or psychological elements, or the so

284 R.J. Jorna and W.F.G. Haselager Journal of Intelligent Systems called train of ideas, such as is the case in that thinking about cars brings me to think about travelling or that seeing that it starts to rain, brings me to think about my umbrella which at the same time could mean that I think about the house where I left the umbrella. A conceptual structure might be the starting point of a theory or even a whole discipline. This has been the case with association, because, according to J.S. Mill, the study of the train of ideas is the core subject of psychology, that is to say that psychology studies the laws of the succession (and simultaneous appearance) of the states of mind.

The reason it is sometimes fruitful to use a conceptual structure instead of a psychological theory is that one wants to take into account the historical continuity of the description of phenomena and that one wants to neglect operationalization of concepts and empirical and experimental procedures. This does not imply a total exclusion of empirical matters, only a temporary one.

In order to understand what the coherence is in the historical monuments of associationism, we will firstly present the main principles of the conceptual structure. Then we will discuss the principles in greater detail. The five principles are (see Linschoten, 1957; Jorna, 1981):

1. "Couples" and "couple signs" can be found with respect to our physiological and mental activity.

2. It can be stated that the relation between the members of the couple (the associates) is contingent or determined.

3. At least two couple elements are present, that is to say judgements, ideas, neurons or otherwise.

4. The coupling of the elements takes place based on a relation of similarity or contiguity or based on a relation that can be reduced to one of these.

285 Volume 4, Nos. 3-4, 1994 Associationism: Not the Cliff Over Which to Push Connectionism 5. One couple enables the (passive or active) realization of new couples.

It should be noted that this way of formulating the conceptual structure implicitly rejects the "Terminal Meta Postulate" (TMP; Bever, Fodor & Garrett, 1968; Anderson & Bower, 1973) as an adequate characterisation of associationism. This postulate has played, and still seems to play, a prominent role in the evaluation of associationism. It states that associative principles are rules defined over the vocabulary of a theory in which behavior is described. It does not matter whether the relations are between ideas, as in classical British associationism, or between stimuli and responses, as in behaviorism. The point is that the psychological theory will not contain any element which is only abstractly related to the elements of behavior (Bever et al., 1969, p. 583).

A, for this paper, very significant consequence of the TMP is that contiguity is seen as the important associative principle; "every associative relation is a relation between left and right elements of a sequence." (Ibidem). A more specific interpretation of the TMP as a manifestation of associationism's meta-features is given by Anderson & Bower: all observable behavior can be explained by concatenating links, purely based on contiguity, between elements directly from experience or derivable from observation (Anderson & Bower, 1973, p. 12). Thus contiguity is taken as the only associationistic principle and the only elements allowed in a psychological explanation have to be put into a one to one correspondence with potentially observable elements. In this form the postulate has haunted behaviorism and it seems to form the basis of many modern critiques of connectionism as well. Yet it is important to note that the TMP as described above cannot be taken as an adequate description of associationism. A major thesis of this paper is that any negative evaluation of associationism based on this restricted interpretation should be rejected.

2.1 Couples and couple signs

The first principle starts from the beginning and just mentions the

286 R.J. Jorna and W.F.G. Haselager Journal of Intelligent Systems

phenomenon of connection, the union or the link between units, elements or events in very general form. It does not say anything about the nature of the connection or of the elements. It looks much like Anderson & Bower's connectionistic feature and needs to be supplemented by the other principles.

In the principle couples and couple signs are mentioned. This illustrates the fact that material things as well as thought-oriented objects fall under the description of association. Even names or concepts for things in the sense of signs and symbols may be subjected to association. This principle tries to catch in conceptual terms the dynamic character of many processes going on in the external (physiological) and internal (mental) world, such as the train of thoughts, the series of movements, the fusion of ideas and the connection of neurons. Things are continuously evolving, which is described in terms of successive combinations. The question, of course, is why and how.

In general the term "association" was primarily applied to the sequences that occur in trains of thought, although the synchronous association of ideas into complex ones was thought possible as well by many writers. Aristotle and Hobbes were mainly concerned with successive association. Locke, who introduced the term "association" in relation to false ideas, did pay attention to the simultaneous union of ideas, and saw them as a consequence of necessary connections. Yet he declined to speak of association in this respect, reserving the term solely for sequence of ideas (Warren, 1921, p. 274). Like the other writers before him, Hume (1738, I, i, 4) mainly concentrated on the successive train of ideas although he devoted some attention to simultaneous association.

Hartley (1749, Prop. X, p. 65) made a basic distinction between synchronous and successive association and attributed to the former a crucial role in the formation of "proper" ideas: "we could have no proper idea of a horse, unless the particular ideas of the head, neck body, legs and tail, peculiar to this animal, stuck to each other in the fancy, from frequent joint impression." (Ibidem, Prop. XI, p. 71). He applied the same

287 Volume 4, Nos. 3-4, 1994 Associationism: Not the Cliff Over Which to Push Connectionism associative law to successive and to synchronous experiences, and he explicitly separated it from his physical theory. His theory of vibrations could be wrong, but the doctrine of association was a certain foundation (Ibidem, Prop. XI, p. 72).

Likewise, James Mill (1829, Ch. 3, p. 53) made an explicit distinction between the synchronous and successive order of ideas, although the former kind is far more common than the latter. He equated synchronous associations with an order in space, the successive with an order in time. In both kinds of associations, ideas follow to a large extent the association of sensations (Ibidem, p. 56). The strength of the association can be attributed to two causes; the vividness of the associated feelings and the frequency of the association. (Ibidem, p. 61).

2.2 The nature of the relationship

The second principle focuses upon the nature of the combination. It says that the association is in the first place the result of the togetherness by chance, as a consequence of contingency or contiguity in time, in place or otherwise. In the second place, however, it also may be the result of necessity, such as is true in natural, logical or mathematical matters.

Locke distinguished between natural and customary connections between ideas. The natural connections are closely linked to the operation of reason: "Some of our Ideas have a natural Correspondence and Connexion one with another: It is the Office and Excellency of our Reason to trade these, and hold them together in that Union and Correspondence which is founded in their peculiar Beings." (1690, II, xxxiii, 5, p. 395). Customary connections are the basis of all error for Locke, and the bonds between ideas thus made can be so strong that reason has not the power to break them (1690, II, xxxiii, 13, p. 398).

The distinction between contingency and necessity can be found again in several other categories that have been suggested in the past (Warren, 1921), for example in objective and logical, and mediated and immediated.

288 R.J. Jorna and W.F.G. Haselager Journal of Intelligent Systems

It can even be defended that the reasoning process in logic is nothing more than a very strong form of association. Indications in favour of this position go back as far as Aristotle's On Memory and Recollection, where he discusses the kinesis in remembering and thinking. Aristotle even explicitly says that recollecting is a sort of syllogism (453a, 11). In the same way that the copula is the uniting principle in combining concepts in judgement and reasoning, the principles of similarity and closely connectedness — what later has been called: association — are the uniting elements in remembering. The beginning of the recollection process could be a deliberate probe based on the human capacity to make inferences. This probe, distinctive of recollection, starts the automatic process of remembering (451b, 30; 453a, 5-15).

The combination of contingency and necessity covers all occurrences of connections of states of affairs, objects, concepts, thoughts and signs. It has always been the purpose of several associationists to describe and "explain" as many phenomena as possible.

However, if one wants to put things to the extreme, two further reductions are possible. The first reduction takes the view that real combinations are natural and logical and all other sorts of combinations are less and less stringent with as a bottom case the contiguous combination. This means that logic and mathematics are the standard, or, in other words, that psychological processes are viewed as a very weak form of mathematics. The second reduction goes the other way around and takes combination by necessity as the upper case of the normal situation which are contingent combinations. In this case the empirical environment determines how and why certain combinations occur, and the laws of mathematics are thought to be based on psychological laws.

In the case of contingent combinations as well as in the case of necessary combinations the question, of course, is which laws, which regularities and which pre-conditions govern the appearance of association. We will discuss this in relation to the fourth principle.

289 Volume 4, Nos. 3-4, 1994 Associationism: Not the Cliff Over Which to Push Connectionism 2.3 Two couple elements

The third principle expresses the fact that an association is realized between at least two elements. If association takes place not only the nature of the relation itself, but also the characteristics of the elements at their special level of description are important. If we distinguish between the levels of description formulated by Dennett (1978) and Newell (1990), we can find association between elements at the physical level (vibrations (Hartley), traces, neurons or networks of neurons), elements at the functional level (ideas, representations, processes or symbols) and at the intentional level (units of behavior, beliefs or judgements). Successive association between more than two elements is possible, but rarely is found in the literature (Warren, 1921). Simultaneous or synchronous associations can, of course, occur between more than two elements.

In the history of the conceptual structure of associationism the assessment at which level of description the associative elements are localized is very often related to discussions about the mind-body problem. For example, how are associated ideas connected to associated patterns of neurons? Several positions have been defended in relation to this: parallelist, interactionist, monist and functionalist positions.

Illustrative of his time Hume suggested a physiological explanation of association: spirits, motioned by the will, go to the proper region of the brain and "rummage that cell, which belongs to the idea". Because these animal spirits sometimes deviate a little from the right track other ideas beside the wanted one are activated as well, which can cause mistakes in reasoning (1738,1, ii, 5, p. 61).

One of the most outspoken parallelist positions with respect to association can be found in Hartley's Observations on Man (1749) in his famous statement: "If any sensation A, idea B, or muscular motion C, be associated for a sufficient number of times with any other sensation D, idea E, or muscular motion F, it will at last excite d, the simple idea belonging to the sensation D, or the very idea Ε or the very muscular motion F."

290 R.J. Joma and W.F.G. Haselager Journal of Intelligent Systems

(Prop. XX, cor. 7). From the point of view of twentieth century cognitive science Hartley's statement is rather simple, but it contains the issue of association in brief. Hartley does not discuss the way in which sensations relate to ideas and, furthermore, to muscular motions. Yet he did try to give a physiological underpinning of the mechanism of association. Simultaneous sensations A and Β will produce corresponding miniature vibrations in the medullary substance. These miniature vibrations leave somewhat different traces than had they not been simultaneously excited, such that they "lean toward each other" (1749, Prop. XI, p. 68; he gives the same physical explanation of successive association on p. 69). Although his theory is not particularly trustworthy and he himself separated it from his associationism (see above), one can ask the question whether modern cognitive science or connectionism is is able to give a more profound and a more enlightening answer with regard to the mind-body issue (P.S. Churchland, 1986; P.M. Churchland, 1989; Clark, 1989).

2.4 Similarity and contiguity

The fourth principle is very important and tries to explain how and why association takes place. It is based on the assumptions that association works with couples (first principle), involves some sort of relation (second principle) and consists of elements (third principle). In the fourth principle it is stated that association is the result of the resemblance or similarity of two elements or the result of the adjacency of two elements in time or place. Although the history of associationism shows that the explanation patterns are not always integrated, both patterns have been formulated in one statement by Aristotle and by Hume. In his On Memory Aristotle says about recollecting: "If the succession be necessary, it is plain that when the one is stimulated it will stimulate the other; if not necessary but habitual, then it will stimulate it only in most instances. [...] Hence, when we are recollecting we keep stimulating certain earlier experiences until we have stimulated one which the one in question is wont to succeed. And just so we hunt through the sequence, thinking along from the present or some other thing and from similar or contrasted or contiguous." (451b, 12-32,

291 Volume 4, Nos. 3-4, 1994 Associatiotiism: Not the Cliff Over Which to Push Connectionism our emphasis). About two thousand years later, Hume, thinking that he is is very original, says in his Inquiry concerning Human Understanding (1748): "Though it be too obvious to escape observation that different ideas are connected together, I do not find that any philosopher has attempted to enumerate or class all the principles of association — a subject, however, that seems worthy of curiosity. To me there appear to be only three principles of connection among ideas, namely, resemblance, contiguity in time or place and cause or effect. [...] That this enumeration is complete [.. .] may be difficult to prove to the satisfaction of the reader, or even to a man's own satisfaction." (1748,1, i, 4). Hume's three principles seem to be different from Aristotle's in that he respectively mentions the principles of causation and contrast. But Hume said: "Contrast, or contrariety, also is a species of connexion among ideas. But it may perhaps be considered as a mixture of causation and resemblance. Where two objects are contrary, the one destroys the other, i.e., is the cause of its annihilation, and the idea of the annihilation of an object implies the idea of its former existence." (1748, Section III). So, it can be said that Hume reduces Aristotle's contrast principle to his own causation and resemblance principles. Although he is severely taken to task by Thomas Brown (1820, Lecture XXXIV, p. 221) for this rather awkward reduction, at least it makes his claim to originality somewhat more understandable.

Hume, after reducing Aristotle's principle of contrast to the principles of causation and resemblance, tried to reduce the relation of causation to that of contiguity ("constant conjunction") in experience. He based this reduction of causality to contiguity on the frequency principle; an accidental contiguity becomes a necessity by continuous repetition (1738,1, iii, 6, p. 86-94; Linschoten, 1957, p. 10).

Linschoten points to a very notable difference between association by contiguity and association by similarity. Association by contiguity takes place independently of the consciousness in which the associated ideas reside, whereas with respect to similarity a subject who notices the similarity is presupposed. This point bears emphasis: contiguity based association has nothing to do with the content or structure of the associated

292 R.J. Jorna and W.F.G. Haselager Journal of Intelligent Systems ideas, whereas similarity based association depends on it. In strongly mechanistic versions of associationism, the principle of similarity is rather neglected. In a way, however, it returns in cognitive science and it is challenged again as a sound principle in connectionism (Fodor & Pylyshyn, 1988; Levelt, 1989; Bechtel, 1985).

Interesting in this respect is Hume's statement that the association among impressions is solely governed by resemblance (1738, II, i, 4, p. 283). It would seem to follow, since ideas are only faint copies of the impressions, that association by resemblance has some priority in his philosophical system. Warren, however, (1921, p. 45-46) treats this as a mere casual statement not belonging to Hume's general analysis, and it is true that in this particular case Hume seems to be speaking specifically of the passions rather than of perceptual impressions in general.

Hume thought that ideas are like images, and that they have structure. He furthermore assumed an active, structure sensitive faculty of the mind which he called imagination. Fodor & Pylyshyn call this 'cheating' because "qua associationist Hume had, of course, no right to active mental faculties" (1988, p. 49-50). Yet, in our view, Hume had every right, because his associationism was based on more than pure contiguity and active mental faculties are to be found in classical associationists like Aristotle, Locke, J.S. Mill and of course Hume himself, to name but a few.

Although more examples can be found where the principles are formulated in combination, a division in associationism seems to be present in which the more empiristically minded philosophers and psychologists emphasize association by contiguity, whereas association by resemblance or similarity is favoured by those who are concerned with the meaningful content of the mind in which syntax and semantics play an important role.

Hartley tried to expand associationism into a complete psychology. He insisted that similarity was a derivative of contiguity. Furthermore, he reduced contiguity in space to time contiguity. He thought he could do without the law of resemblance as the basis of his analysis of complex ideas

293 Volume 4, Nos. 3-4, 1994 Associationism: Not the Cliff Over Which to Push Connectionism and sensations into simpler elements. What would be experienced as two similar ideas were in fact complexes in which indeed the very same basic elements were present in a timely contiguity (Warren, 1921, p. 55; it should be noted that this very same idea forms the basis of the connectionistic treatment of similarity). All this is in complete agreement with Hartley's mechanistic-physiological position; no consciousness is needed if contiguity is the only association principle (Linschoten, 1957, p. 12).

Hartley strongly stressed the function of habit or repeated experience in the formation of connections. It is the frequency of combinations that decides which connection ultimately becomes established. He even stated that changes in non-essential characteristics of the phenomena help to strengthen the bond between essential ones (1749, Prop. LXXX).

Association, as explanans, now covered the entire field of human phenomena. Hartley argued that sensation, thinking, imagining, remembering, language and also movement could be explained by pure contiguity based associative analysis. It is interesting to note, however, that Hartley does not give any analysis of reasoning. Instead he just considers abnormalities of reason, which may be a sign of Locke's influence. Still, according to Warren (1921, p. 63), this omission is probably the greatest defect in Hartley's work. Kant objected to the idea that the train of thought could be based on a mechanism of contiguity in time or space. Our experiences are just too varied and disorganised, to result in anything else but accidental heaps (Kant, 1787, A 121). Tetens (1777, p. 1088; p. 266) agreed with Kant and stated that contiguity based associations can merely repeat past experiences and never create new representations.

According to Thomas Brown (1820), Aristotle's three principles of contiguity, similarity and contrast are the crucial ones. Yet he tried to reduce direct resemblance to the influence of mere contiguity (Lecture, XXXV, p. 223), just as contrast can be reduced to former proximity (Lecture, XXXVI, p. 232). Hume's principle of causation could, according to Brown, be reduced to contiguity in time and place (constant proximity) (Lecture, XXXV, p. 222). 294 R.J. Joma and W.F.G. Haselager Journal of Intelligent Systems

James Mill (1829) stated that contiguity in time formed the basis of successive associations, whereas contiguity in space underlay synchronous associations. He equated causation with contiguity in time, causation being only a name for the order established between an antecedent and a consequent (Ch. 3, p. 79). Resemblance is also not a separate principle of association, for it can be attributed to the fact that we usually see the same things together. Resemblance can therefore be seen as being nothing more than a particular case of the law of frequency underlying contiguity (Ibidem, p. 80).

Very interesting is James Mill's treatment of ratiocination. He notices that the ancient writers, following Aristotle, did nothing more than enumerating and classifying terms, propositions and syllogisms, and tried to give the rules for making correct and detecting incorrect syllogisms. Mill, however, tried to explain syllogisms in terms of the underlying associations. Seeing the truth of a premise, e.g., "All men are animals" is merely the recognition that the meaning of "all men" is included in the term "animals", and that recognition is nothing more than a case of association. The same goes for the premise "kings are men". The conclusion "kings are animals" is also reached through association: "In each of the two preceding propositions, two terms or names are compared. In the last proposition, a third name is compared with both the other two; immediately with the one, and, through that, with the other; the whole, obviously, a complicated case of association." (1829, Ch. 12, p. 311).

John Stuart Mill (1843) criticized this foundation of ratiocination on association and he reintroduced the idea of association by similarity. According to John Stuart Mill the principle of similarity was the first law of association, contiguity being second (VI, iv, 3, p. 557).

Association on the basis of similarity or resemblance implies that two concepts or two ideas are combined because they share a common characteristic. In a strong sense association on the basis of similarity also could include rules of inference. In formulating theories of knowledge representation as semantic networks, as frames and schemata and as

295 Volume 4, Nos. 3-4, 1994 Associationism: Not the Cliff Over Which to Push Connectionism production rules cognitive scientists have tried to operationalize in one sense or the other the train of ideas, the proceeding of thought on the basis of association by similarity and resemblance (Jorna, 1990). Most cognitive scientists, however, do not explicitly use the terms of the classical associationists (see for an exception: Anderson & Bower, 1973).

Association on the basis of contiguity implies that two items are combined because they accompany one another either in space or time. Behaviorists and for the last ten years also connectionists have tried to explain the combination of stimuli and responses or the connections of computational units or neurons on the basis of the consecutive or simultaneous appearance of elements for which the frequency of appearance is the decisive factor to result in an associative combination. Operant behaviorism as well as connectionism take number and quantity, frequency, inhibition and excitation of elements (stimuli and responses) to be the basic constituents for theories about the succession of behavior and the processes of thinking.

Association by contiguity and association by resemblance are both formulated in order to express the regularities of association as a relation. As an interesting detour it is notable that associationists normally use metaphors or models that proved to be successful in other areas to make the relation of association more clear. Warren (1921) distinguishes association as a force, association as a fact or a sequence and association as a process.

Association as a force starts from the position that a special power is active upon data that unites elements. A slightly different view of this "power" does not consider it to be outside the data, but to be inside, to be part of the data itself. Just as two balls bump together in which the first one pushes the other, in the same way two concepts push against one another in such a way that it can be said that the first concept moves or stimulates the other. Aristotle already uses this metaphor to explain the fact that in the material implication an antecedence arouses or revives a consequence.

In association as a sequence it is denied that any inherent force connects two elements with one another. The only thing there is, is the fact

296 R.J. Jorna and W.F.G. Haselager Journal of Intelligent Systems

of sequence or succession and because of the repetition of a sequence a certain habit is established. "The frequent repetition of any given succession, a-b, makes it easier for the mind to pass from a to b than from a to c, if the latter sequence has occurred less frequently in the past." (Warren, p. 261). This line of thinking can be found in Berkeley (1710, par. 25) who stated that association is not a power or strength resident in the ideas themselves, but that it is the frequency of contiguity that basically determines the strength of their connections.

All of this very much reminds us of Hume's analysis of the notion of causation which he reduced to mere succession. If one ball pushes another ball the first ball does not cause the second to move. What we see, when we look in greater detail, is that the first ball moves and then stops, whereas the second ball first lies still and then moves. Causation is not in the things themselves, but only in the eye of the beholder. However, this position concerning the interpretation of association is not defended by Hume himself. He holds a more moderate view; he talks about association as a process.

Association as a process lies between association as a force and as a sequence. It is granted that succession and simultaneous manifestation imply some sort of union, but this union is not the result of an inherent principle in consciousness. The term Hume uses in this context is "a gentle force" and this force is not in analogy with atomistic mechanics but with a sort of magnetism in which two ideas are attracted, which might be the result of a repetitive sequence but also of some sort of resemblance. The viewpoint that association is some sort of process is in complete agreement with the view that thinking and remembering are information processing activities (Anderson & Bower, 1973).

2.5 Recursive associations

The fifth principle more or less expresses the recursive step in associationism. The four foregoing principles describe the phenomenon in all its aspects, whereas this principle says that associations themselves may

297 Volume 4, Nos. 3-4, 1994 Associationism: Not the Cliff Over Which to Push Connectionism be elements in a further process of association. The same principles concerning the association relation as well as the explanation patterns, that were discussed earlier, are applicable, but this principle says that more is possible concerning the elements. Simple ideas that are associated in complex ideas may in their turn be associated in hyper complex ideas and sensations that are associated in the experience of seeing a small thing may in their turn be associated into seeing an object or state of affairs and so on.

Two possible positions of the association process can be discerned, not only in relation to the recursive form of association, but also concerning elementary association. The first views association in analogy with a mechanical process (e.g., James Mill). The main characteristic of the mechanical process is that the elements in the association remain unchanged. The result of the association process is a cluster of elements. The elements in the cluster may be very close to each other, but they remain qualitatively the same. The inseparability of strongly connected ideas is, according to James Mill, in many cases only appearance, although it is only with effort that these complex ideas can be decomposed into their compounds (1829, Ch. 3, p. 69). In some cases, however, the associative bond cannot be broken, as in the case of colour and extension.

The second position assumes it to be the analogue of a chemical process. From Abraham Tucker (1768) on the idea of emergent properties of complexes of elements appeared in the history of associationism. Tucker spoke of combinations as junctures of ideas, and acknowledged the possibility of a fusion, a kind of connection in which ideas more or less melt together. In this he was a forerunner of the later theory of mental chemistry. The same line of thinking can be found in the work of Brown. The chemical analogy indicates that a transformation takes place which results in a melting or fusion of two elements into one new element. This means that the old elements are lost after the association process. This also means that the compound element might have new features which were not present in and cannot be carried back to either of the old ones.

Both metaphors are used in associationism and it is hard to say which

298 R.J. Joma and W.F.G. Haselager Journal of Intelligent Systems one is more promising and psychologically more relevant. Although in more general terms almost all main British writers (except James Mill) thought that chemical analogies were possible, the mechanical view was still quite dominant.

Both metaphors have their advantages. The advantage of the mechanical metaphor is that it permits a more insightful knowing of the composition and decomposition of elements and that it also allows one to do rather simple mental arithmetics which in the history of psychology has been suggested and carried out by Herbart (1824) and Beneke in the middle of the nineteenth century. The advantage of the chemical metaphor is that it takes into consideration the psychological evidence concerning thinking as a fusion process. No matter how simple therefore the mental composition may look like, there is evidence which indicates that once combined elements cannot be dissolved back again.

What we showed in the foregoing is that to talk about association implies more than just being pro or contra a position in the history of philosophy and psychology. The phenomenon described as association both as explanandum and as explanans is a central theme in almost every part of the study of mind and brain. Associationism is not a simple, univocal term but a long-standing conceptual structure. We gave some historical evidence in favour of our position that the program to study intelligent systems advocated by connectionism has in this broad sense its roots in the philosophy of the seventeenth and eighteenth century (the same can be said for classical cognitive science in relation to the rationalistic tradition; Haselager, 1992). Let us now return to Levelt and Fodor & Pylyshyn, who as defenders of classical cognitive science reject association and to Bechtel, who as a defender of connectionism embraces association (albeit somewhat more cautiously).

299 Volume 4, Nos. 3-4, 1994 Associationism: Not the Cliff Over Which to Push Connectionism 3. CONCLUSIONS CONCERNING THE ANALYSIS AND INTERPRETATION OF ASSOCIATION

In the foregoing we showed that being an associationist does not mean that one holds a very restricted position in the debate on the explanation of reasoning, perception and memory processes. So, if Levelt (1989, p. 212) says that associationism is an old intellectual tradition, he is right, but he is wrong in equating associationism with connectionism and behaviorism, and in concluding that both therefore should be rejected. The same caution must be exercised in relation to Bechtel's 1985) statement that association is fundamental in learning and that learning mainly is studied in behaviorism and connectionism.

The equation between connectionism and associationism is evident in the work of many more writers. In discussing Smolensky's article on connectionism (1988) Dellarosa says that the appeal of connectionism has its roots in associationism, in the idea that cognition is characterized by the probabilistic construction and activation of associations among units. The success of connectionist models will mean a "reinstatement of associationism as the cornerstone of cognition" (Dellarosa, 1988, p. 28-29). In the same context, Lindsay agrees with Dellarosa's explanation of the appeal of connectionism, although he doesn't share the enthusiasm, and he underlines that the models of classical cognitive scientists can be seen as "explicitly eschewing the desire to be reductions-to-associationism and one- principle learning" (Lindsay, 1988, p. 41-42). Smolensky's reaction to this exemplifies the underlying narrow interpretation of associationism. He refuses to be identified as a "simple" or "pure" associationist: "simple associationism is a particularly impoverished and impotent corner of the connectionist universe. It may well be that the attraction a number of people feel to connectionism is an attraction to neo-associationism; but it is nonetheless a serious mistake to presume connectionism to be committed to simple associationist principles." (Smolensky, 1989, p. 5). He prefers to speak of statistical inference, instead of pure association, as the basic processing principle of connectionist networks (Smolensky, 1988, p. 63).

300 R.J. Jorna and W.F.G. Haselager Journal of Intelligent Systems

The same kind of refusal to equate connectionism directly with associationism is evident in Bechtel's work. He finds his main reason for this in assuming associationism to be far more sensationalistic than connectionism (Bechtel, 1985, p. 57), and therefore failing to pay attention to the importance of the structure of the mind. Bechtel explicitly rejects the idea that all elements in the association have to be directly derivable from sense elements. He states that in connectionistic models many units are only remotely connected to sensory input and to behavior (Ibidem). Especially hidden units are not directly connected to, nor have a one to one correspondence with observables. They rather can be taken to represent sub-symbolic sub-features of the experienced input, which form the basis for the processing of the output.

We have the impression that associationism, in the sense used above (whether called "simple" or not) is interpreted as being exclusively contiguity based and extremely sensationalistic, like the Terminal Meta Postulate. We have shown, conceptually and historically, that this restricted image of associationism is incorrect. Especially the principle of similarity underlying the work of many classical associationists (Aristotle, Hume, J.S. Mill) and its connection with inference and reasoning makes associationism a lot more elaborate and encompassing than usually thought. Also, many classical associationists have acknowledged the existence of elements in the mind whose relations with sense elements are at least as complicated as the relations between activation patterns over hidden units and those over input and output units in connectionist networks. Nor is it the case that associationists have declined to acknowledge the importance of the structure of the mind. On the contrary, the study and explanation of the structure of the mind has always been a dominant theme in associationist literature.

An important thing that can be learned from the discussion of the phenomenon of association is that holding a position in the debate on association must be seen apart from positions holding in the mind-body issue and in the relation between perception and cognition. Association occurs and might be studied within cognitive elements or perceptive

301 Volume 4, Nos. 3-4, 1994 Associationism: Not the Cliff Over Which to Push Connectionism elements, but also between perceptive and cognitive elements. The same holds whether one has an identity-theory position in the mind-body debate or a dualist or functionalist position. It is therefore interesting to consider Smolensky's unreserved statement that "it is likely that connectionist models will offer the most significant progress of the past several millennia on the mind/body problem." (Smolensky, 1988, p. 2). It should be clear that, whether or not Smolensky is right, it has very little, if anything, to do with connectionism being associationistic or not.

In our opinion one gains nothing by saying that the enemy is associationistic or that the enemy is bad because he is not an associationist. All in all, there is no good reason to use associationism as a contemptuous term or to see it as a price to pay. As always, things are more complicated if one takes a closer look. Then it becomes clear that the history of philosophy and psychology has a more subtle structure with respect to association. The conclusions we like to draw from our analysis are the following.

In the first place it is clear that there is a division in association based on similarity and resemblance and association based on contiguity and frequency. If one says that the law of similarity is an irreducible law of association besides the law of contiguity, this means that the association of elements depends on their structure. If, however, contiguity is taken to be the only fundamental law of association (e.g., Hartley, James Mill), then there is no need for structure or structure-sensitive operations. This particular point of view is very vulnerable to a Kantian critique. Fodor & Pylyshyn (1988, p. 27) don't fail to point to this, and their "modern paraphrase" stresses the fact that it is the simultaneous entertainment of concepts that is inadequate. But the division is not in the sense of an exclusive "or". That is, it is not only possible but a historical fact that both laws can be combined in an associationistic program, thereby escaping Fodor & Pylyshyn's and Levelt's negative evaluation.

In the second place it is claimed (Bechtel, 1985) that the classical associationist picture conforms very well with the connectionist program

302 R.J. Jorna and W.F.G. Haselager Journal of Intelligent Systems because it includes features such as that it gives an explanation for learning, that it rejects innate ideas and that it entangles data and procedures (symbols and processes). It is said that these are things not to be found in cognitive science. Yet, while it is indeed true that Fodor (1981) defends innate ideas and has outspoken opinions about learning, it is almost rhetorical to equate Fodor's extreme position with the whole field of cognitive science — although Fodor claims that this is the case. Doing cognitive science does not imply a philosophically rationalist (nativist) position. Newell (1990), for example, does not support the innate ideas position. Recent work shows that SOAR is extended with sensory and motor components. Newell also gives an explanation for learning in the framework of cognitive science by using chunking of production rules into more compiled production rules which, in more associationist terms, means a regulated "fusion" of elements (production rules). Thus, the title of Bechtel's (1985) article; "Contemporary connectionism: are the new parallel distributed processing models cognitive or associationist?" is too suggestive in its supposed contradiction. On the one side PDP-models are used to describe and explain cognition and are associationist, on the other hand models of cognitive science are also cognitive and also try to describe and explain why and how (meaningful) associations are formed. If one really wants to attach labels to cognitive science and connectionism we suggest the following. PDP-models are associationist and non-symbolic or sub-symbolic. Furthermore, they are cognitive, and (philosophically) empiricist. Information processing models, however, also deal with combinations and associations of representations. Furthermore, they are symbolic, cognitive, and (philosophically) empiricist (Newell) or rationalist (Fodor).

In the third place the analysis of association, conceived as a conceptual structure, makes clear that the difference between connectionism and classical cognitive science consists not only of the rejection of the symbol level by connectionists. The difference has very much to do with the positions they hold concerning the structure of the content of the associated elements. Symbols (as in Newell & Simon's

303 Volume 4, Nos. 3-4, 1994 Associationism: Not the Cliff Over Which to Push Connectioni.im 'physical symbol hypothesis'; 1976) express content in a way regulated by syntax and semantics. The distributed representations utilized in connectionist models differ from this to a substantial degree.

The history of association psychology shows that several classical associationists, for example Hartley and James Mill, had difficulties explaining reasoning and thinking in terms of association based on contiguity and frequency. In the same sense Levelt (1989) asserts that connectionist networks have difficulties to explain reasoning adequately. It is indeed true that the main research topic of connectionists is pattern recognition and not the reasoning process. We think that this is a consequence of the emphasis on association based on contiguity. Cognitive scientists, on the contrary, have mainly been interested in reasoning and memory. In our opinion this is the case because they work with semantically contentful elements and implicitly accept association based on similarity and resemblance. Here, we see again that an interpretation of association has consequences for other components of a theory about intelligent systems.

The phenomenon of association was, is and will be a main research topic in the study of intelligent systems. The phenomenon has to be tackled and this is complicated and difficult enough. This has to be the realistic conclusion if one looks at the fact that association has been conceived as a force, as mere succession, as a process or as the cement or the glue of the mind. It is normal to put labels on points of view concerning association, but if one really wants to make progress in the study of natural and artificial cognition one does not reach anything by attacking schools or trends through a negative evaluation of associationism as a conceptual structure.

ACKNOWLEDGEMENT

We would like to thank Hans van Rappard (Free University of Amsterdam) for his critical remarks on an earlier draft of this article.

304 R.J. Jorna and W.F.G. Haselager Journal of Intelligent Systems

REFERENCES

Anderson, J.R. and Bower, G.E., 1973, Human Associative Memory, Washington, Winston and Sons. Aristotle, On Memory and Recollection, in: Parva Naturalia, Loeb VIII. London, Heinemann (1975). Barth, E M., 1977, The logical Paradigm in dialectical Philosophy and Science, Erkenntnis, 11, 291-322. Bechtel, W., 1985, Contemporary connectionism: are the new parallel distributed processing models cognitive or associationist?, Behaviorism, 13, 1, 53-63. Bechtel, W. and Abrahamsen, A, 1991, Connectionism and the Mind, Cambridge, Basil Blackwell. Berkeley, G., 1710, Treatise Concerning the Principles of Human Knowledge. Dublin, Lasalle, The Open Court (1963). Bever, T.G., Fodor, J.A. and Garrett, M., 1969, A formal limitation of associationism, in: Dixon, T.R. and Horton, D.L., eds, Verbal behavior and general behavior theory, New Jersey, Prentice-Hall. Brown, T., 1820, Lectures on the Philosophy of the Human Mind, Edinburgh, William Tait (8th edition 1834). Buckingham, H.W., 1984, Early development of association theory in psychology as a forerunner to connection theory, Brain and Cognition, 3, 19-34. Churchland, P.M., 1989, A Neurocomputational Perspective, Cambridge, the MIT Press. Churchland, P.S., 1986, Neurophilosophy: Toward a Unified Science of the Mind-Brain, Cambridge, the MIT Press. Clark, Α., 1989, Microcognition: Philosophy, Cognitive Science, and Parallel Distributed Processing, Cambridge, The MIT Press. Deese, J., 1966, The Structure of Associations in Language and Thought, Baltimore, John Hopkins Press. Deilarosa, D., 1988, The psychological appeal of connectionism, Behavioral and Brain Sciences, 11, 28-29. Dennett, D.C., 1978, Brainstorms, Brighton, Sussex, The Harvester Press. Fodor, J.Α., 1981, Representations, Brighton, Sussex, The Harvester Press.

305 Volume 4, Nos. 3-4, 1994 Associationism: Not the Cliff Over Which to Push Connectionism Fodor, J. A. and Pylyshyn, Z.W., 1988, Connectionism and Cognitive Architecture: A Critical Analysis, Cognition, 28, 3-71. Gardner, H., 1985, The Mind's New Science, New York, Basic Books. Hacking, I., 1975, Why does Language matter to Philosophy?, Cambridge, Cambridge University Press. Hartley, D., 1749, Observations on Man, his Frame, his Duty, and his Expectations, Vol. 1 & 2, Hildesheim, Georg Olms Verlag (1967). Haselager, W.F.G., 1992, Historical Foundations of the Computational Theory of Mind, Proceedings of the Cheiron Conference, Madrid 1991. Herbart, J.F., 1824, Psychologie als Wissenschaft, Theil I & II, Königsberg. Amsterdam, Bonset (1968). Hume, D., 1738, Treatise of Human Nature, Second Selby-Bigge edition (1978) Oxford, Clarendon Press. Hume, D., 1748, Inquiry concerning Human Understanding, Indianapolis, Bobbs-Merrill Edition (1955). Jorna, R.J., 1981, Associatie, van Copula naar Functie, Centrale Interfaculteit, Universiteit van Groningen. Jorna, R.J., 1990, Knowledge Representation and Symbols in the Mind, Tübingen: Stauffenburg Verlag. Kant, I., 1787, Kritik der reinen Vernunft, Hamburg, Felix Meiner Verlag Edition (1956). Kuhn, T., 1962, The Structure of Scientific Revolutions, Chicago, The University of Chicago Press. Laudan, L., 1977, Progress and its Problems, Berkeley, University of California Press. Linschoten, J., 1957, A Gentle Force, Academische Rede, Groningen, Walters. Lindsay, R.K., 1988, Can this treatment raise the dead?, Behavioral and Brain Sciences, 11, 41-42. Levelt, W., 1989, De connectionistische mode, in: C. Brown, P. Hagoort and T. Meijering, eds, Vensters op de geest, Utrecht, Stichting Grafiet. Locke, J., 1690, An Essay Concerning Human Understanding, Nidditch edition (1975), Oxford, Clarendon Press.

306 R.J. Jorna and W.F.G. Haselager Journal of Intelligent Systems

McClelland, J.L., Rumelhart, D.E. and the PDP-Group, G.E., 1986, Parallel Distributed Processing, Vol. I & II, Cambridge, The MIT Press. Mill, J., 1829, Analysis of the Phenomena of the Human Mind, Vol. 1 & 2, London, Baldwin and Cradock. Mill, J.S.,1843,^4 System of Logic, London, Longman (1970). Newell, Α., 1990, Unified Theories of Cognition, Cambridge, Harvard University Press. Newell, A. and Simon, H.A., 1976, Computer Science as Empirical Inquiry: Symbols and Search, Communications of the ACM, 19, 3, 113-126. Ramsey, W., Stich, S P. and Rumelhart, D.E., 1990, Philosophy and Connectionist Theory, Hillsdale, Lawrence Erlbaum Associates. Robinson, E.S., 1932, Association Theory Today, Reprinted 1964, New York, Hafner Publishing Cie. Smolensky, P., 1988, On the proper treatment of connectionism, Behavioral and Brain Sciences, 11, 1-74. Tetens, J.N., 1777, Philosophische Versuche über die mensliche Natur und ihre Entwicklung, Vol. 1 & 2, Leipzig, Weidmann. Tucker, Α., 1768, The light of the Nature pursued, New York, Garland Edition (1977). Warren, H.C., 1921, History of the Association Psychology, New York, Scribner's Sons, Reprinted in 1967.

307