Connectionism and the Commitments of Folk Psychology

PhilosophicalPerspectives, 9, AI, Connectionism,and PhilosophicalPsychology, 1995

CONNECTIONISM AND THE COMMITMENTS OF FOLK PSYCHOLOGY

TerenceHorgan and JohnTienson The Universityof Memphis

Many philosophersbelieve thatconnectionism is incompatiblewith folk psychology,and hence that the success of connectionismwould support elimina- tivistconclusions about propositional attitudes. Some philosophers actually argue that(certain brands of) connectionismhas sucheliminativist implications. In this paperwe examinetwo sucharguments, due to Ramsey,Stich, and Garon(1990; hereafter,RSG). Their principalargument centers around a featureof the propositionalattitudes they call functionaldiscreteness. RSG's secondargument centerson thequestion whether the predicates of commonsense psychology are projectable. We concludethat their arguments are not successful.Common sense psychology,properly understood, is notincompatible with connectionism, properly understood-atleast not in the way thatRSG allege. But we are not simply concernedto refuteanother eliminativist argument, even though RSG's paperhas been quiteinfluential.' RSG's argumentsare a usefulvehicle for getting clearer aboutthe issues on whichtheir arguments turn, concerning both folk psychology and connectionism.

1. The Functional DiscretenessArgument.

RSG's mainargument is thatcommon sense psychology has commitments thatare notsatisfied by an importantclass of connectionistmodels. Thus, if the correctmodels of human cognitionlie withinthat class, common sense psychologywill be shownto be seriouslyin error. In thisSection we lay outRSG's argumentin foursteps: the commitment of commonsense psychology to functionaldiscreteness, a class of connectionist modelsthat are held to lackfunctional discreteness, an exampleof a modelfrom thatclass, and theexplicit argument that such models lack functionaldiscreteness. In Section2 we look at commonsense functionaldiscreteness in more detail.We arguethat common sense is committedonly to certainparadigm cases of functionaldiscreteness, and that connectionistmodels of the class RSG identifydo exhibitdiscreteness of that kind. We also pointout that connectionist modelsin this class can exhibitother kinds of functional discreteness which com- 128 / TerenceHorgan and JohnTienson mon senserecognizes as possible,but to whichit is not committed.

1.1 A Commitmentof CommonSense Psychology.

RSG base theirmain argumenton threecommitments of commonsense psychologyconcerning propositional attitudes: propositional attitudes are semanticallyinterpretable; they have a causal role; and theyare functionally discrete.RSG call thiscluster of featurespropositional modularity (504). The firsttwo are familiar.Propositional attitudes are thesorts of thingsthat can be trueor false,satisfied or unsatisfied,and the like; in thecurrent term of art,they havecontent. And propositional attitudes influence behavior, belief fixation, etc. in waysthat are appropriateto theircontent. To say thatpropositional attitudes are functionallydiscrete is to saythat they can haveeffects singly (or in content- based structures,as whena conclusionis drawnfrom two premises,with no otherpropositions playing a role).RSG holdthat distributed connectionist models do notsatisfy the common sense demand for functionally discrete states because in suchmodels all informationis encodedholistically-hence inseparably- throughoutthe network. They mentiontwo differentways in whichcommon sense propositional attitudesare functionally discrete. First, they can be acquiredor lostindividually (nearlyenough). For example,"Henry...had completely forgotten that the car keyswere hidden in therefrigerator," (504-5) althoughhe had forgottennothing else. And if you are toldthat the keys are in therefrigerator, you will acquirea small clusterof new beliefs,but most of yourbeliefs will be notbe altered. The second kind of functionaldiscreteness is more importantin the argument.Sometimes a personhas a totalset of beliefsand desiresthat provide morethan one reasonfor performing an action,A. And sometimesit happens thatthe person does A forone of thosereasons, with the other possible reason notfiguring in theetiology of the action at all. Likewise,sometimes a personhas severalsets of beliefsthat could lead herto infera particularnew belief,p, and she infersp fromone of thosesets, with the others not figuring in herthinking at all. Thus,according to commonsense psychology, it is a determinatequestion whichpotential reasons for an actionor changein beliefwere the actual or operativereasons. Accordingto commonsense psychology, then, the same state is semantically evaluableand has a content-appropriate,functionally discrete, causal role. Such stateshave what RSG call propositionalmodularity. Functional discreteness is the featureon whichthe argument turns. Since semanticevaluability and somekind of causal role are takenfor granted for the most part, we will usuallyspeak of functionaldiscreteness, reserving 'propositional modularity' for contexts in which semanticevaluability (or causal role) mightbe an issue. Connectionismand theCommitments of Folk Psychology/ 129

1.2 A Class of ConnectionistModels.

RSG claim that distributedconnectionism must deny propositional modularity.They characterize a class ofconnectionist models, which, they claim, are incompatiblewith propositional modularity, in particularwith functional discretenessof semanticallyevaluable states. The models in this class are characterizedby threeproperties:

i. Theirencoding of informationin theconnection weights and in the biases on unitsis highlydistributed rather than localist. ii. Individualhidden units in the networkhave no comfortable symbolicinterpretation; they are subsymbolic.... iii. The modelsare intendedas cognitive,not merely as implementa- tionsof cognitivemodels. (p. 508)

Features(i) and(ii) aremeant to insurethat it is notpossible to associatespecific informationwith particular local partsof themodel. Connections and nodesare notto be semanticallyevaluable individually or in smallsets. Information in the modelis encodedholistically throughout the network or throughout large portions of the network.Furthermore, each node contributesto representingmany differentpropositions, and each connectionweight contributes to storingmany differentpropositions. Thus, information is contained in thenetwork holistically and globally,not locally. And this means, RSG argue,that all ofthe information in thenetwork is involvedin all of its processing,so thatit is not possibleto singleout certain bits of informationas operative-andothers as inoperative-in a tokenprocess, as folkpsychology requires. As RSG note,feature (iii) is notabout the network as such,but about how it is to be interpreted.The idea is thatthe model is supposedto tellus something about how the mindworks, not how it mightbe embodied.Consider, for instance,a classicalparser-a classicalcomputer program which is meantto take naturallanguage sentences as inputand yieldstructural descriptions of theinput sentencesas output.Such a programcan be considereda hypothesisabout the cognitiveprocesses, knowledge structures, and so forth,involved in recognizing the grammaticalstructure of sentences.The programcan be run on many differentcomputers, with differentmachine languages; the hypothesisabout cognitionis thesame in each case. The machinelanguage of thecomputer that the programhappens to be runningon is irrelevantto the cognitivestory the programproposes. One could attemptto use a connectionistnetwork to implementthe opera- tionof sucha classicalprogram. This would be to attemptto use thenetwork as an implementationofthe classical model-as an alternative,unorthodox kind of machinelanguage. There would still be no differencein the hypothesesput forwardabout cognition. This is thekind of construalof connectionistmodels that(iii) rulesout. 130 / TerenceHorgan and JohnTienson

But a connectionistmodel-for instance,a parsingmodel such as Berg (1992)-can also be construedas offeringan alternativestory about the cognitive processesinvolved in recognizingthe grammatical structure of sentences,a story thatis in competitionwith the classicalmodel. This would be to construethe modelas a cognitivemodel, as requiredby (iii). Whenunderstood in thisway, RSG hold, distributedconnectionist models are incompatiblewith the propositionalmodularity of folk-psychologicalstates.

1.3 An Example.

RSG describea simplethree-layered, feedforward connectionist network, whichthey describe as "a connectionistmodel of memory." The network,called NetworkA, has sixteeninput nodes, one outputnode, and a hiddenlayer of four nodes. Inputconsists of encodingsof sixteenpropositions, for example,

Dogs have fur. Cats have fur. Dogs have gills. Fish have gills.

Eight inputnodes are used to encodethe subjectof the proposition,eight to encodethe predicate. NetworkA was trainedup so thatits outputnode is on (> .9) whenthe inputproposition is true,and off (< .1) whenthe input proposition is false.Thus, the networkhas memorizedthe answersto a true/falsetest. The networkis capable of generalizing;it respondedcorrectly to encodingsof 'cats have legs' and 'Cats have scales,' whichwere not in thetraining set.

1.4 TheArgument

RSG observethat

[t]heinformation encoded in Network A is storedholistically and distributed throughoutthe network. Whenever information is extracted from Network A, bygiving it an inputstring and seeing whether it computesa highor a low valuefor the output unit, many connection strengths, many biases and many hiddenunits play a rolein the computation. And any particular weight or unit or biaswill help encode information about many different propositions. (513)

This is certainlya correctdescription of the workings of thenetwork. Whenever thetruth or falsehoodof a proposition(i.e., highor low outputnode activation) is computedfrom an inputproposition, all of thehidden units and manyof the weights are involved in the computation.RSG argue that this holistic computationis incompatible(they say, "radically incongruent") "with the propo- Connectionismand theCommitments of Folk Psychology/ 131 sitionalmodularity of commonsense psychology."

Foras wesaw in Section 3, common sense psychology seems to presuppose that thereis generally[sic] some answer to thequestion of whethera particular beliefor memoryplayed a causalrole in a specificcognitive episode. But if beliefand memory are subserved by a connectionistnetwork like ours, such questionsseem to haveno clearmeaning. (513)

2. Critique of the FunctionalDiscreteness Argument.

The overallstructure of our critiqueis as follows.We distinguishthree differentways thatintentional mental properties (state-types) can be possessed by a cognitivesystem (section 2.1). On thebasis of thistripartite distinction we distinguishseveral different possible forms of functionaldiscreteness; we then arguethat common sense psychology is committedto onlyone of theseforms of functionaldiscreteness, and thatcommon sense psychology leaves it an open empiricalquestion whether or not any of the otherforms are manifestedby propositionalattitudes in humans (section 2.2). With this discussion as background,we pressthree separate replies to RSG's argument. First,connectionist models, including RSG's NetworkA, typicallyexhibit theonly kind of functionaldiscreteness to whichcommon sense psychology is committed.Thus, even if connectionismdoes precludesome or all of theother kinds,this fact would not generatean incompatibilitywith commonsense psychology;rather, it would meanthat connectionism answers in thenegative certainempirical questions about functionaldiscreteness that common sense psychologyitself leaves open (section2.3). Second,even if commonsense psychology were committed to thoseother kindsof functionaldiscreteness, and even if humancognition failed to exhibit them,these facts would only show that common sense psychology is somewhat mistakenabout propositional attitudes; they would not showthat propositional attitudesdon't exist(section 2.4). Third,we arguethat connectionism does notreally preclude any of the other kindsof functionaldiscreteness anyway. In principle,any or all of themcould be manifestedin a connectionistsystem in which informationis embodied holisticallyand distributedlyin weightsand in activationpatterns across nodes. Such functionaldiscreteness normally would not involve distinctphysical componentsof thenetwork's causal evolution;instead it would be discernible onlyat a moreabstract, mathematical, level of descriptionin whichthe network is characterizedas a high-dimensional"dynamical system" (section 2.5).

2.1 PsychologicalStates: Occurrent,Dispositional, and Morphological.

Thereare threedifferent ways in whichintentional content or intentional state-typescan be possessedby humansand othercognizers. The firsttwo are 132 / TerenceHorgan and JohnTienson standardlymentioned in the literatureof philosophyof mind;the third is not. First,a personcan possessthe intentional property occurrently. This means,at leastroughly, that a tokenof therelevant psychological state-type occurs as a concreteevent or state within the person. In connectionistmodels, representations are certainpatterns of activation-thosepatterns that are interpretedas having intentionalcontent. Patterns of actual, occurrent activation in thenetwork are the concretestates that constitute its token representations, and that thus play the role (in themodel) of occurrentmental states. Second,a personcan possessthe intentional state-type dispositionally. This means,at leastroughly, that the cognitive system is disposedto generatea token of thattype underappropriate circumstances, a tokenwhich will thenhave suitablycontent-appropriate effects on the system'scognitive processing and behavior.In connectionistmodels, dispositional intentional state-types thus are a matterof a network's(weights being set to producea) tendencyto generate occurrentrepresentations under appropriate circumstances. Third,a personcan possessintentional content morphologically (as we will put it). Morphologicalpossession of intentionalcontent M is a matterof the cognitivesystem's being disposed, by virtue of its persisting structure rather than byvirtue of anyoccurrent states that are tokens of M, to undergostate transitions thatare systematicallyappropriate to contentM-and to do so, at leastmuch of the time,without generating a tokenof M duringthe process. Morphological contentdiffers from occurrent representational content (e.g., occurrentbelief) because it involves the cognitivesystem's persisting structure, rather than occurrenttokening of M. Morphologicalcontent differs from dispositional repre- sentationalcontent (e.g., dispositionalbelief) as standardlyunderstood (and as characterizedinthe preceding paragraph) because the relevant dispositions associ- ated withmorphological content involve tendencies other than the tendency to generatetoken representations with that content. Consider,for example, a cognitivesystem which treats all membersof a certainclass, R, of representationssimilarly. It tendsto makethe same kinds of inferencesfrom representations in R. Whenit acquiresa new representationin the way it acquiredthe membersof R, it tendsto make the same kinds of inferenceswith that new representation.But it has otherrepresentations from whichit does notmake similarinferences. The systemthus treats members of R as representationsof thesame type; in effect,it treats the kinds represented by the membersof R as speciesof the same genus.But the systemmay have no representationfor the genus itself. Thus, the system cannot represent the fact that (the kindsrepresented by) two membersof R are species of the same genus, thoughit treatsthem as such. (We can say R is a class of representations;the systemcannot.) This is thekind of thing that would count as (in thiscase purely) morphologicalcontent. In connectionistmodels, morphological possession of intentionalcontent is a matterof informationbeing embodied"in the weights."(The tendencyto generateoccurrent mental states when appropriate is, of course,also something Connectionismand theCommitments of Folk Psychology/ 133 that's"in theweights." So dispositionalintentional states can be seenas a special case of morphologicalcontent.) Commonsense psychology attributes both occurrent and dispositional modes of possessionfor the various kinds of state-typesit posits,including beliefs and desires.Dispositional beliefs, desires, etc., as such,are unconsciousstates; any consciousmental state is an occurrentstate. On theother hand, common sense psychologydoes leave open theconceptual possibility of occurrentbeliefs and desiresthat are unconscious.There is no apparentreason why an intentional psychologicaltheory could not posit morphological content, in additionto occurrentand dispositionalstate types. As faras we can see, morphologicalcontent is consistentwith common sense psychology, but common sense psychology is not committedto morphologicalcontent.2

2.2 Typesof FunctionalDiscretenes, and theirStatus withinCommon Sense Psychology.

Withthese distinctions in mind,let us reconsidercommon sense psychology's commitmentto functionaldiscreteness. Consider, for instance,RSG's exampleof Clouseau.Clouseau has heardthat the hotel is closed forthe season and thatthe train is outof service.The Butlersays that he spentthe night at the hoteland took thetrain back to townin themorning. Common sense reckons thatClouseau might have inferredthat the butler is lyingfrom his beliefthat the hotelis closedfor the season, or fromhis beliefthat the morning train has been takenout of service,or fromboth. From the perspective of commonsense there is-often-a determinateanswer to thequestion which it was. RSG believethat no determinateanswer to thisquestion is possibleif human cognitive systems are relevantlylike theirNetwork A. Why does commonsense reckonthis a determinatequestion? The first thoughtof the common sense psychologist is thatit depends upon which relevant beliefsconsciously occurred to Clouseau(and whichlogical connections he was awareof). If he consciouslythought of the hotel closing and consciously realized thatits being closed meant that the Butler couldn't have spend the evening in the hotel,but didn't remember the train at all at thetime, well-its obviouswhich one was operative. Consideralso RSG's exampleof Alice theE-mailer. Alice had tworeasons to go to heroffice. She wantedto talkto herresearch assistant, and believed that he wouldbe at theoffice. And shewanted to sendsome E-mail messages, which she believedshe coulddo fromthe office. "Common sense psychology assumes thatAlice's goingto her officemight have been caused by eitherone of the belief/desirepairs, or by both,and thatdetermining which of these options obtainsis an empiricalmatter." (p. 505) In RSG's rendition,Alice's desireto send some E-mail messageswas causallyinert. Why might that be? The most naturalexplanation is thatit did notconsciously occur to herin the relevant time framewhile her desireto talkto herresearch assistant did. The relevanttime 134 / TerenceHorgan and JohnTienson frameis not,of course,just theperiod immediately preceding her departure for the office.She mighthave had a thoughtearly in themorning which she could have expressedout loud by saying,"Oh, I've gotto talkto Fredtoday about...." She mightthen have taken care of somehousehold chores, read the paper, gotten readyto go to the officeand departed,without Fred ever again enteringher consciousness. Thus, the paradigmaticcases of propositionalmodularity recognized by commonsense psychology are cases in whichthe causally active mental state is occurrentand conscious,whereas the causallydormant mental state is dispositionalbut not occurrent.That is, the typeof functionaldiscreteness to which commonsense psychology is clearlycommitted is thefollowing.

1. SI is occurrent;S2 is dispositionalbut not occurrent. (Subcase: SI conscious/S2unconscious) (Here and below,SI is thestate that is causallyactive; S2 is a statethat could have led to thesame actionor thoughtbut did not do so in this case.)

We will call thissubcase of type1 functionaldiscreteness paradigmatic functionaldiscreteness. Commonsense also recognizesthat one mightmake use of informationthat does not rise to consciousness,or arriveat a conclusionwithout conscious inference,especially in rapidphysical activity.3 Thus we shouldadd a second subcaseto type1 functionaldiscreteness: unconscious/unconscious. It is not contraryto commonsense to considerpossible complications of paradigmaticfunctional discreteness, especially in the case of explanationof actions,decisions, choices, etc. Perhaps Alice is moredeeply interested in herE- mail conversationsthan she caresto admitto herself.So her"real" reasonfor goingto theoffice is to sendsome E-mailmessages, but she "tellsherself' (as we mightsay) thatshe is goingto theoffice to talkto Fred.Her desireto send E-mail messageswas occurrentand causallyefficacious, but she suppressed awarenessof itsefficacy, and perhaps of the desire itself. This seems to be a case in whichboth desires are tokened,but in whichonly one, the one thatis not consciouslyconsidered, is the actual cause. Thus, common sense clearly recognizesthe possibility of a secondtype of functionaldiscreteness.

2. S1 is occurrentand S2 is occurrent. (Conscious/conscious;conscious/unconscious; unconscious/conscious; unconscious/unconscious.)

All foursubcases are conceptually possible although the first is perhapsquestion- able fromthe pointof view of commonsense. It seems odd to supposethat Clouseau thoughtof thehotel closing, thought of thetrain being taken out of service,understood that each was incompatiblewith something the Butler had Connectionismand theCommitments of Folk Psychology/ 135 said,and inferredthat the Butler was lyingfrom one of thesebeliefs but not the other. Dispositionalpossession of an intentionalstate-type does notmake a direct causalcontribution to an outcomethe state-type could cause; rather, dispositional statesenter the causal frayindirectly via theexercise of thedisposition, i.e., via the occurrenceduring processing of an occurrenttoken of thatstate-type. Case 1 involvesa situationwhere the disposition to producea tokenof S2 does not get exercisedduring processing, so it is, in a sense,a degeneratetype of functionaldiscreteness. It needsto be statedbecause it is theone case of functional discretenessto whichcommon sense is clearlycommitted. Given that intentional state-typesthat remain merely dispositional do not play a causal role, three furthercases are worthdistinguishing.

3. SI is occurrent;S2 is morphological. (Conscious/unconscious;unconscious/unconscious.) 4. SI is morphological;S2 is occurrent. (Unconscious/conscious;unconscious/unconscious.) 5. SI is morphological;S2 is morphological.

Commonsense allows forthe conceptualpossibility of each of Cases 3 through5, becauseone can makeintelligible, from the point of viewof common sense,the idea thatthere is morphologicalcontent that has a causal role. For instance,here is thetranscript of an actualconversation:

J: The Parkersare at theirplace; thered flagis up on theirmailbox. N: Yeah, I saw theirgolden [retriever dog] lastnight. J: Oh, yeah;you toldme that.I forgot.

Clearly,J inferredthe Parkers'presence from the flag,not fromthe dog. The flag is the thinghe thoughtof. Had he rememberedthe dog, he mighthave realizedthat he did not have to tell N about the Parkers.So far,this story illustratesType 1 functionaldiscreteness. But it illustratessomething more. To inferthat the Parkers were home from theraised flag on theirmailbox, J must rely on somethinglike (F) theflag is up on the Parkersmailbox only when they are hereto raiseit. Likewise,when N inferredthat the Parkerswere home, she relied on somethinglike (D) the Parkers'golden is hereonly when they are here. Thus, we have instancesof (F) and (D) exhibitingfunctionally discrete causal roles. But it is quiteunlikely that either (F) or (D) consciouslyoccurred to either Jor N. And,we submit,it is quiteintelligible from the point of view of common sense to supposethat neither (F) nor (D) was tokenedsubconsciously either; indeed,that neither was tokenedsubconsciously seems more likely to us thannot. If it was not,this is not a case of Type 1 or of Type2 functionaldiscreteness. If the information(F) thatplayed a role in J's inferencewas not tokened 136 / TerenceHorgan and JohnTienson consciouslyor unconsciously,then it was morphologicalrather than occurrent or merelydispositional.4 This seems,then, to be construableby commonsense as a case of Type5 functionaldiscreteness. So commonsense evidently does permit thepossibility of morphologicalcontent and Type5 functionaldiscreteness. (In Section5 we offeran exampleconstruable as Type 4 functionaldiscreteness, plus variantsof RSG's Clouseau examplefor each of Types2-5.) But ourmain point in thisSection is thatcommon sense is onlycommitted to theparadigm case of functionaldiscreteness, Type 1 functionaldiscreteness wherethe causallyactive stateis conscious.The othercases we have distin- guishedare recognizedby commonsense as possibilities,some as the quite seriouspossibilities. In Section2.5 below,we arguethat all fivetypes of functionaldiscreteness arepossible in modelsthat fall within the class ofmodels characterized by RSG.

2.3. First Reply: Folk Psychologyand DistributedConnectionism Are Compatible.

The mostimmediate reply to RSG's functionaldiscreteness argument is now quitestraightforward. The onlykind of functional discreteness to whichcommon sensepsychology is committedis paradigmaticfunctional discreteness. Connec- tionistmodels have no troubleat all manifestingthis degenerate kind of functionaldiscreteness. For, on one hand,the occurrentbeliefs of commonsense psychologycorrespond most naturally to certaintokened activation patterns in a connectionistnetwork; and activation patterns have a causal influenceon processing.Processing in a connectionistnetwork is spreadingactivation. On theother hand,the dispositionalbeliefs of commonsense psychologycorrespond most naturallyto a connectionistnetwork's dispositions to generatethe activation patternsthat function in the systemas representation-tokens;and whensuch a dispositionremains dormant, so thatthe relevant activation pattern does notget tokened duringprocessing, then the patterndoes not affectthe system's processing(since it is not there).Thus, paradigmatic functional discreteness is easily accommodatedwithin the relevantclass of connectionistmodels: an activationpattern can occurthat constitutes a token representation that causes a certainsubsequent outcome, while at the same time the system has a dispositional representationwhich would bring about the same outcome were it activatedand yet remainsdormant on thisparticular occasion. Therefore,the connectionist modelsconsidered by RSG do notpreclude the kind of functionaldiscreteness to whichcommon sense psychology is committed. In theremainder of this subsection we will amplifythis reply, by discussing (i) the NetworkA describedby RSG and theirremarks about it, (ii) RSG's repliesto certainobjections they themselves consider, and (iii) recentremarks aboutRSG's modularityargument by Stichand Warfield(forthcoming). Thereare two kindsof representationsin NetworkA. The inputlayer is interpretedas representingquestions concerning the truthor falsityof certain Connectionismand theCommitments of Folk Psychology/ 137 propositions.And thetrained up networkrepresents answers to thosequestions. Thatis, it representscertain propositions as beingtrue or falsein thepattern of activationin theset of nodes consistingof inputnodes plus theoutput node. Activationof the encodingof a propositionin the inputlayer causes the outputnode to recordtrue or false.The occurrentrepresentation of theproposi- tioncauses the recollection of itstruth value. Potential but non-occurrent repre- sentationsdo notexert any such influence.Thus Network A, like connectionist modelsgenerally, exhibits paradigmatic functional discreteness: occurrent repre- sentationshave an effect,whereas non-occurrent representations that the system is disposedto produce,but has notproduced on a givenoccasion, do nothave an effect. Of course,what RSG actuallyhave in mind as analoguesof beliefs,in NetworkA, arenot occurrent representations in the input layer (or in thehidden layer),but rather the propositional information which is holisticallyembodied "in theweights". They argue,

Sinceinformation is encoded in a highlydistributed manner...with information regardingany given proposition scattered throughout thenetwork, the system lacks functionallydiscrete, identifiable sub-structures thatare semantically interpretable as representationsof individual propositions. (514.)5

Functionaldiscreteness does not obtain, they contend,for propositional informationembodied holistically in thenetwork's weights (and biases). Ratier,

Thereis a real sensein whichall the informationencoded in thenetwork's connectivitymatrix [weights] is causallyimplicated inany processing inwhich the networkengages. (O'Brien, 1991, p. 173;quoted with endorsement byStich, 1991, p. 180)6

Butthe common-sense psychologist can grantthat all theinformation inthe weightsis implicatedin anyprocessing in whichthe network engages, and deny thatthis goes contraryto thekind of functionaldiscreteness to whichcommon sensepsychology is committed(viz., theparadigmatic kind). Information in the weightsis, if anything,the connectionist analog of morphologicalcontent. And commonsense psychology simply is notcommitted to claimingthat morphological contentin humansexhibits functional discreteness. Whether it does or not is an open empiricalquestion, as faras commonsense psychology is concerned. Thus,if (some) distributedconnectionist models do nothave functionally discrete morphologicalcontent, that doesn't make them incompatible with common sense psychology;rather, it answersthe empiricalquestion negatively (for those models). RSG do discussthe objection that (i) connectionistrepresentations are pat- ternsof activation,and (ii) activationpatterns are functionallydiscrete states. Theyreply that the identification ofbeliefs with activation patterns is "singularly 138 / TerenceHorgan and JohnTienson implausible,"because "in commonsense psychology beliefs and propositional memoriesare typicallyof substantialduration; and theyare the sortsof things thatcognitive agents generally have lotsof evenwhen they are not using them" (p. 518). But theappropriate counterreply is straightforward: it is onlyoccurrent beliefs that are appropriatelyregarded, within connectionist modeling, as activationpatterns. RSG also discuss the suggestionthat "long standingbeliefs might be identifiednot with activation patterns, but with dispositions to produce activation patterns,"and therelated suggestion that "the familiar philosophical distinction betweendispositional and occurrentbeliefs might be captured,in connectionist models,as thedistinction between dispositions to produceactivation patterns and activationpatterns themselves" (pp. 518-9). They replythat dispositionsto produceactivation patterns "are notthe discrete,independently causally active statesthat folk psychology requires" (p. 519). Once again the counterreplyis straightforward:folk psychology recognizes a distinctionbetween occurrent and dispositionalbelief, and is notcommitted to thefunctional discreteness of dispositionalbeliefs qua dispositional;it is onlycommitted to paradigmaticfunctional discreteness.(Dispositions to re-createbeliefs, memories, etc. are in theweights holistically.But the (recreatable)activation pattern is not in theweights when it is notactive. It is nowhere.Thus, there really is no questionof thefunctional discretenessof dispositionalbeliefs. It is notthat all ofthe dispositional beliefs are directlyimplicated in processing;none of themare.7) So connectionistrepresentations have, by and large,functionally discrete causal roles.Those representationsthat get activatedin a processplay a causal role in thatprocess; those not activated do not.And thespecific causal rolesof the ones thatget activateddepend upon patterns of spreadingactivation.8 Stichand Warfield(forthcoming) reply to a similarobservation by Andy Clark(1990). Clarksuggests that it is onlya "belief-in-action"(as opposedto a long-standingbelief, which may be just a dispositionto producean occurrent belief-in-action)that needs to be capableof functionallydiscrete causal potency (p. 96). Stichand Warfield'srelevant argument is thatthe proposal is too weak,

foron the interpretation ofpropositional modularity..., nodeterministic system that storespropositional information could fail to satisfy propositional modularity. Ifthis is right,there is nothingat all wecould learn about the workings of such a system thatwould show that it violates modularity and thus does not really have beliefs. (Section2.2)

We take 'and thus' in the last line to mean 'and forthis [lack of propositional modularity]reason'. Otherdeep commitmentsof commonsense psychology mightbe violatedeven if propositionalmodularity is not. Thus, the operativecomplaint in this passage is that,on the proposed interpretationofpropositional modularity (which requires discrete causal potency onlyfor tokened representations), no systemcould failto satisfypropositional Connectionismand theCommitments of Folk Psychology/ 139

modularity,inparticular, no systemcould fail to exhibit functional discreteness. Ourreply is fourfold.First, Network A is a modelof a single,immediate cognitivestep-in thiscase, roterecall. There are no representation-level intermediaries.Given that the input and output of sucha one-stepprocess are (interpretedas) representations,and thatrepresentations are enteredsingly, nothingcould show that the system lacks Type 1 functionaldiscreteness. The occurrentrepresentation in the inputlayer is causallyactive. Dispositional representations-onesthat could be in theinput layer but are not-are not causallyactive. Butsecond, this is surelynothing to complain about. Any cognitive process whichis immediatefor a system,9and which receives only one relevant input at a timemust, obviously, exhibit Type 1 functionaldiscreteness. (It does not even dependupon the systembeing deterministic.)This just meansthat the commitmentof commonsense psychologyto functionaldiscreteness of propositionalattitudes is a veryweak commitment. However,third, it is easyto imagine other sorts of connectionist models that mightnot exhibit functional discreteness for tokened representations. A model ofsome task that involves multiple simultaneous soft constraint satisfaction, for instance,must allow many representations to be activeat once.It mightoften, perhapseven typically, be impossibleto determinewhich representations were causallyresponsible for the solution to a problem,especially if the representa- tionsare widely distributed. Furthermore, some systems with distributed repre- sentationsinwhich each node contributes tomany different representations can havemany representations active at once by superposition ofrepresentations, in whichcase it may be-though it need not be (cf. section 2.5 below)-impossible toseparate the causal contribution ofdistinct representations. Thus, there may be modelsof these kinds, with sensible representation level interpretations thatdo notexhibit Type 2 functionaldiscreteness. Finally,fourth, common sense psychology is notcommitted to Type2 functionaldiscreteness anyway. The only kind of functional discreteness towhich it is actuallycommitted is trivially satisfiable.

2.4 SecondReply: Ontologically Conservative Theory Change.

Suppose,for the sake of argument,that we are wrongin claimingthat commonsense psychology is only committed tothe kind of functional discrete- nessthat involves a consciousoccurrent belief and a non-activateddispositional belief,and thatit is actuallycommitted to someor all of theother kinds delineatedin section 2.1 above.Suppose too (although we will argue against this inthe next subsection) that distributed connectionist models of the sort sketched by RSG areincompatible with the further kinds of functionalmodularity. Do thesesuppositions sanction RSG's keyclaim, viz., "If connectionist hypotheses ofthe sort we willsketch turn out to be right,so toowill eliminativism about propositionalattitudes" (p. 500)? 140/ Terence Horgan and John Tienson

Surelynot. RSG themselves draw a distinctionbetween "ontologically conservative"theory changes (which preserve the key theoretical entities posited by ofan originaltheory while altering or replacing that theory's claims about those entities),and "ontologically radical" theory changes (in which the entities posited by theold theoryare repudiated as well).Even if commonsense psychology happensto be committedtoone or more kinds of functional discreteness ofType 2 throughType 5, alteringthe theory by dropping this commitment would be a ratherconservative change-especially since these are not paradigmatic cases of functionaldiscreteness. Such a changewould not even approach entailing that thereare no beliefs.'0

2.5 ThirdReply: Strong Forms of Functional Discreteness Are Not Precluded.

Distributedconnectionist models embody information holistically, rather thancontaining discrete items of propositional information inphysically discrete internalstates, structures, orprocesses. Information that is notoccurrently repre- sentedis distributedthroughout the network, and each partof thenetwork contributesto storing much or all ofits information. So it appears plausible that distributedconnectionist models are incompatible with functional discreteness of Types3-5. Thisappearance motivates RSG's argument.But the appearance is misleading,as we now briefly explain. We begin by describing a way of thinking aboutmorphological content in connectionism.We thendiscuss a common phenomenonthat is plausiblyregarded as involvingType 4 functionaldiscreteness-thatis, a situationin whichmorphological content trumps occurrent content.Then we revisitClouseau and the butler. Thenatural mathematical framework fordescribing connectionist networks is thebody of mathematical concepts, techniques, and results known as dynamical systemstheory. To describea networkas a dynamicalsystem is to specify ina certainway its temporal evolution, both actual and hypothetical. Each node inthe network is assigneda separatedimension, or axis,in a high-dimensional hyper-space;the possible activation values the node can take are points along that axis.Each possible total state of the system is thusrepresented bya uniquepoint inthe system's "state space" (in the case of connectionist networks, often called "activationspace"). The dynamicalsystem, as such,is essentiallythe full collectionof temporaltrajectories the network would follow through its state space-witha trajectoryemanating from each point it can occupy in state space. Thedynamical system can be thoughtof as a high-dimensionalgeometri- cal/topologicalobject. A usefulgeometrical metaphor for dynamical systems is thenotion of a landscape(in thecase of networks,an activationlandscape). Thinkof the network'sn-dimensional activation space as a contouredn- dimensionalsurface, oriented "horizontally" in(n+1)-dimensional space. For each pointp, thetemporal trajectory that the network itself would follow through its activationspace if it wereto evolve(without perturbation) from p is thepath "downhill"along the landscape that a ballwould follow if positioned at p and Connectionismand theCommitments of Folk Psychology/ 141

thenallowed to "roll." Eachpoint on the activation landscape corresponds toa totalactivation state ofthe network. Certain of these points are representation-realizing points:when thenetwork is inthe total activation state corresponding tothe given point, one ormore representations aretokened as activationpatterns. In general,represen- tationsare multiplyrealizable in connectionistmodels. Representations are identifiedwith activation vectors, typically with activation vectors that specify activationvalues for only a relativelysmall portion of the nodes of the network. Sucha vectorthus specifies values for some, but not all dimensionsofactivation space.All pointsthat satisfy these values will realize the given representation." Also,several distinct representations can,in general, be realized by a singlepoint in activationspace: the point'scoordinates simultaneously satisfying the coordinate-specificationsofseveral different vectors, each of which is identified witha distinctrepresentation.'2 Fromthe dynamical systems point of view, cognitive-level state transitions in a connectionistnetwork are trajectories along the activation landscape from onerepresentation-realizing pointto another. These transitions depend jointly on twointerrelated factors: (i) therelative positions on theactivation landscape of therepresentation-realizing points,and (ii) thetopography ofthe landscape itself. Landscapetopography is determined bythe connections among the nodes and the weightson thosenodes. "Training up" a network,by progressively altering its weightsin accordancewith some learning algorithm (e.g., backpropagation of error),amounts to theprogressive molding of the activation landscape in a way thatresults in systematicallycontent-appropriate trajectories from one representation-realizingpoint to another.'3Learning thus involves modification ofthe existingactivation landscape, in a waythat accommodates new information while leavingintact the information to which what is learnedis irrelevant. Learningto makea certainclass of inferences,for instance, produces a slopeor incline on the activation landscape. From every point realizing a (possi- blycomplex) representation ofa certainkind, the system is inclinedto proceed to a pointrealizing a corresponding representation ofa different kind. The land- scapealso has other inclines, subserving potentially conflicting inferences one has learnedto make.And many other inclines too, subserving various potentially conflictingtendencies to evolvefrom one representationalpoint to anotherin variousnon-inferential content-appropriate ways. So theactivation landscape is a veryhigh dimensional, subtly contoured, space with inclines upon inclines upon inclines.(Think of the disorientation and contortion ofearlier geological strata byrising new land.) Thus, being inclined to makean inferencedoes not mean thatone will make the inference. In any particularcognitive trajectory along the activationlandscape, informationthat is partof the content of representation-realizing pointsalong that trajectoryis the informationthat becomes occurrent-i.e., gets explicitly represented-duringthecognitive process corresponding to that trajectory. On theother hand, information that is accommodatedbythe trajectory without being 142/ Terence Horgan and John Tienson

partof the content of any representation-realizing pointon itis (relativeto that trajectory,anyway) morphologically embodied rather than explicitly represented. The local topographicalfeatures of thelandscape-i.e., the various different, superimposedinclines present in the immediate vicinity of a givenrepresentation- realizingpoint-are what determine the content-appropriate trajectory from any suchpoint to anotherone. One specialcase of inclinesin activationspace comes up fairlyoften in connectionistdiscussions. Representations arethought of as attractorpoints or regionsin activation space, and one speaks of the basin of the attractor-viz, the setof all pointsin activationspace from which the system will evolve to the attractor.A basin is, ofcourse, an inclineall individualslopes of which lead to thesame place. Fora differentkind of exampleof an incline,consider a systemthat has learnedto makea classof Humean inferences. Whenever itencounters an A, it expectsa B. Butit has never occurrently represented the proposition that A's are B's, and it is notcurrently disposed to do so. Perhapsif it is sophisticated enough,it could come to occurrently believe that A's areB's byreflecting onits owninferential tendencies. But it has not reflected in thatway. In sucha case,the information that A's areB's is containedin the system morphologically(but not dispositionally or occurrently). That is, thereis an inclinein its activation space connecting A-realizing points to B-realizing points, butthere is no pointin itscurrent activation space that realizes the belief that A's areB's. Considernow the phenomenon ofprejudice. A personis stronglyinclined tocome to certain kinds ofjudgments, J,about anyone (or anything) heclassifies as beingof a certaintype, K.14 On occasionhe feelsa needto explainone of thosejudgments, sometimes from external prodding, sometimes not. On these occasionshe comesup withan explanationof the particular judgment that does notrefer to typeK. Andtypically, the explanations he givesare rather different in differentcases. Theprejudice consists in an inclinein thatperson's activation space from pointsrealizing representations ofindividuals asbeing of kind K topoints realiz- ingJjudgments about those individuals. The person may have little or no inclin- ationto (occurrently)believe the generalization connecting K to J. (Beinga humanbeing, he has,of course, the capacity to entertainthat generalization.) Oftenwhen a Jjudgmentis made, it is precededby an occurrent representation,R, thatthe person puts forward-to himself or others-as the reason for his Jjudgmenton a particularoccasion. But in fact,R is causallyinert. There is no pathin activation space from points realizing R butnot realizing a representation ofan individualas ofkind K to theJ judgment. At the network level, there is no spreadingof activation from R toJ. Theincline (in activation space), i.e., morphological content, plays an actual causalrole in bringingabout the J judgment; the occurrent representation, R, doesnot. Thus, we havehere a conceptuallypossible case of Type 4 functional Connectionismand theCommitments of Folk Psychology/ 143

discreteness.Representation R might be a complexrepresentation that fully justifiesJ-in theeasiest case, the premises of a validargument for J. Yet K leads to J onlyin conjunctionwith the (mis)informationmorphologically embodiedin the incline from K-realizing points to J-realizing points; and K leads toJ inthat case whether or not the inferential trajectory commences from an R- realizingpoint. Furthermore, theperson need not consciously represent the fact thatthe individual is ofkind K forhis prejudice concerning K's to comeinto play.(It is an interestingempirical question whether such a representationmust be occurrentat all, even unconsciously.) Consider,in light of the foregoing discussion, RSG's exampleof Clouseau. Supposethat Clouseau's internal network is ata pointp inactivation space that realizesthe state-type B:

(B) believingthat the butler said he spent the night at the village hotel, andthat he saidhe arrivedback on themorning train.

Supposethat Clouseau's activation landscape has distinct, determinate, inclines withinit that respectively subserve trajectories appropriate to belief-types H and T, respectively:

(H) believingthat the village hotel is closedfor the season. (T) believingthat the morning train has been taken out of service.

(We willcall these inclines the H-incline and the T-incline, respectively.) In the immediatevicinity of thepoint p on Clouseau'sactivation landscape that his cognitivesystem currently occupies, the local topography is a complexcontour- ingconsisting of the superposition ofvarious different inclines, including the H- inclineand the T-incline. Suppose that at point p, the T-incline and certain other inclines(not including the H-incline) effectively "cancel each other out"; i.e., whensuperimposed together, the T-incline and these other inclines jointly make no netcontribution tothe local topography inthe vicinity of p. Finally,suppose thatthe dominant net effect, locally at point p, is contributedbythe H-incline. So an inferentialtrajectory commences, emanating from p andterminating at a pointp' whichrealizes the state-type L:

(L) believingthat the butler is lying.

Thisis a scenarioin which Clouseau believes that the village hotel is closedfor theseason, he also believesthat the morning train has been take out of service, andhe infersthat the butler is lyingon thebasis of the first belief but not the second. Thisscenario can be furtherelaborated in severalways, corresponding to Type2 throughType 5 functionaldiscreteness. If the content of both belief H andbelief T is onlyembodied morphologically in the H-incline and T-incline 144/ Terence Horgan and John Tienson respectively,but neither content gets occurrently represented during Clouseau's inferentialprocess, then we get morphological/morphological functional discreteness:Type 5. Butthere are three other variants or the scenario, where the content ofone or both beliefs also becomes occurrent, i.e., is partof the representational contentof pointp, or of someother point along the inferentialtrajectory commencingfrom p: H andT bothoccurrent (Type 2); H occurrentbut not T (Type3); T occurrentbut not H (Type4). So theupshot of this subsection is that all fourof these kinds of functional discretenessare open conceptual possibilities, under distributed connectionism. RSG aremistaken to supposethat functional discreteness ofcognitive states can onlyoccur if thecontent of thosestates is embodied,in weightsand/or in activationpatterns, in a physicallydiscrete way.'5

3. The Projectable Predicates Argument.

RSG brieflyoffer a secondargument for the radical incompatibility of connectionismand commonsense psychology.Network A learnedthe truth valuesof sixteenpropositions. RSG describea secondmodel, Network B, which learnedthe truth values of those sixteen propositions plus one more. The weights, biases,and internalactivation values in processingare not similarin Networks A and B, and thedifferences between them do notcorrelate in anyway with the differencein whatthey have "learned".Both of thesenetworks represent the propositionthat dogs have fur,among others. There are indefinitelymany other connectionistnetworks that represent the information that dogs have fur, which differin indefinitelymany ways from Networks A andB. Fromthese observations, RSG argueas follows.

...commonsense psychology treats predicates expressing the semantic properties of propositionalattitudes as projectable.Thus 'believesthat dogs have fur'or 'remembersthat dogs have fur' will be projectablepredicates in commonsense psychology....[But] though there are indefinitely many connectionist networks that representthe information that dogs have fur just as wellas NetworkA does,these networkshave no projectablefeatures in commonthat are describablein the languageof connectionisttheory. (514)

Thus, we take it, the conclusionis thatcommon sense psychologytreats as projectablea huge class of predicates that connectionism renders non-projectable. (Projectablepredicates, say RSG, are"the sort of predicates that are appropriately used in nomological or law-like generalizations"(p. 504).) Hence, if connectionismturns out to be correct,there will be no statesof the kind that thesecommon-sense psychological predicates purport to ascribe.'6 RSG professnot to find "featuresin commonthat are describablein connectionisttheory" in NetworksA and B. We supposethat they are thinking Connectionismand theCommitments of Folk Psychology/ 145

of connectionisttheory as thetheory of networks:activation levels and the equationsthat determine them, weights, biases, and learning algorithms. There are indeedno projectablepredicates here that correspond to theprojectable predicatesof common sense psychology. Butthe projectable predicates of common sense psychology are "predicates expressingthe semanticproperties of propositionalattitudes." Connectionist theoryalso has predicates expressing the semantic properties ofrepresentations- e.g., the predicate,'representation that dogs have fur.' A largepart of connectionisttheorizing consists of talk about representations. Read the descrip- tionof any connectionist model! Read RSG's description ofNetwork A! Promin- entin the description ofany connectionist model is an accountof the representationsin themodel and of how they are realized in the network. The connectionistmodels on which RSG's argument centers are, they insist, to be construed as cognitivemodels. Surely, if a modelis construedas a cognitivemodel, then representationswill be a centralpart of the theory of that model. Thus,Network A andNetwork B do havea featurein commonthat is describablein thelanguage of connectionisttheory: they both represent the propositionthat dogs have fur. And, of course, they share this feature with all thoseother actual and potential connectionist models that have a representation ofthe proposition that dogs have fur. Furthermore,connectionist representation predicates are projectable. For any reasonablysuccessful connectionist model that has a representationofthe propositionthat dogs have fur, 'representation thatdogs have fur' will be a projectable predicate.If thealleged representations of a modelare not projectable relative to thecognitive task being modeled, the model doesn't work. Exactly how representationpredicates are projectable relative to a modelwill depend upon the cognitivetask being modeled. But that is whatone would expect, since cognitive modelstend to be aimedat modelinga single cognitive task or small cluster of tasks,and the causal role of a representationrelative to onecognitive task will be differentfrom its causal role relative to differentcognitive tasks.'7 Whenthe same network is trainedup on thesame task more than once, thereare differences inweights, biases, and activation levels of hidden nodes, but generalizationsinvolving representation-level connectionist predicates are typi- callyprojectable from one trained up networkto theother. Representation-level generalizationsaresimilarly projectable when distinct networks are implementa- tionsof the same cognitive model, and when similar networks are trained up on differentbut similar tasks (as wereRSG's NetworkA andNetwork B). Generalizationsinvolving representation-level connectionist predicates are not,in general,projectable from one connectionist model to othersdevoted to differentcognitive tasks; the terms of sucha generalizationare often not even applicableto theother model. But that is a resultof thenature of cognitive modeling.In anycase, it is nota differencefrom common sense. From the point ofview of common sense psychology, the degree of projectability ofgeneraliza- tionsinvolving propositional attitude predicates to othercognizers and other 146 / TerenceHorgan and JohnTienson kindsof cognizersis quitevariable and contextdependent. Thus, connectionism has projectablepredicates-predicates assigning representations to network models-that line up quite nicelywith the projectablepredicates of common sensepsychology. WhenRSG say "thesenetworks have no projectablefeatures in common" (our emphasis),there is a sensein whichwhat they say is true.The networksas such have no projectablefeatures in common.The models-the networks interpretedas performinga cognitivetask-do have projectablefeatures in common.Cognitive science is a branchof scientifictheory that spans and interconnectsseveral levels of description.Within classical, pre-connectionist, cognitivescience, the canonicalarticulation of the multi-levelnature of the enterprisewas givenby David Marr,who wrote:

Atone extreme, the top level, is theabstract computational theory of the device, in whichthe performance ofthe device is characterizedas a mappingfrom one kind of informationto another, the abstract properties of thismapping are defined precisely,and its appropriateness andadequacy for the task are demonstrated. Inthe centeris thechoice of representationforthe input and output and algorithm to transformone intothe other. At theother extreme are thedetails of howthe algorithmand representationare realizedphysically-the detailed computer architecture,so to speak.(1982, 24-25)

Thus,Marr identifies three theoretically significant levels of description. The top level,the level ofthe mental qua mental,specifies a cognitivefunction: a transition-functionthat pairs cognitive states with the appropriate cognitive successor- states.'8The middle level specifiesthe algorithmby whichthat function is computed.And the lowest level specifiesthe physicaldevice in which the algorithmis implemented.'9 An algorithm,or program,is a mathematicalobject, a set of rules for manipulatingsymbols or data-structurespurely on the basis of theirfor- mal/structuralproperties, independent of any intentional content they might have. Symbolsand data-structures,so described, are also mathematicalobjects. Thus, themiddle level in Marr'stypology is a mathematicallevel of organization. This level of organizationmediates between intentional mental states and their physicalrealization. Intentional mental states and state-transitions arerealized by certainmathematical states and state-transitions,which in turnare realizedby certainphysical states and state-transitions.20The mathematicallevel is the appropriateone for characterizing the abstract system of functional/organizational featuresthat constitutes Nature's engineering design for human cognition. However,the discretemathematics of algorithmsis not commonto all approachesto cognition.As discussedin Section2.5, thenatural mathematical frameworkfor connectionism is thetheory of dynamicalsystems. And if cogni- tivetransitions are not determined by algorithmsover symbols, then it neednot be assumedthat the potential cognitive transitions of a cognitivesystem consti- Connectionismand theCommitments ofFolk Psychology/ 147 tutea tractablycomputable function. Marr's tri-leveltypology for cognitive sciencecan thusbe seen as a speciesof a moregeneric tri-level typology:

CognitiveState-Transitions. The level of themental qua mental. MathematicalState-Transitions. The level of functionalorganization. Physical Implementation.The level of thephysical qua physical.

Connectionistcognitive models are another species of this generic typology, with themathematics of dynamicalsystems as thenatural mathematical framework at the middle level of description,and with connectionistnetworks (often as simulatedon conventionalcomputers) as theprototypical devices for physical implementation.2" In both classical and connectionistcognitive science, then,theorizing involvesthe cognitive, the mathematical, and thephysical levels of description andthe interconnections among them. In bothclassical and connectionistcogni- tivescience, predicates at each level of descriptionare projectable,even though the state-typesthey express are multiplyrealizable at lower levels of description.22To claim,as RSG do, thatconnectionist models that differ in the mannerof theirNetwork A and NetworkB "have no projectablefeatures in commonthat are describablein the languageof connectionisttheory," is to ignore the fact that connectionistcognitive science includestwo levels of descriptionabove thelevel of thephysical qua physical.

4. Conclusion.

RSG arguethat common sense psychology is incompatiblewith a certain brandof connectionismbecause common sense psychology is committedto the functionaldiscreteness of propositionalattitudes, while thatbrand of connec- tionismprecludes functional discreteness. We distinguishedthree ways in which a cognitivesystem may possess intentional content: occurrently, dispositionally, ormorphologically. Mixing and matching these ways of possessing content leads to severalconceptually possible types of functionaldiscreteness. We arguedthat commonsense psychologyis committedto onlythe mostinnocuous kind of functionaldiscreteness-Type 1 functionaldiscreteness, in which occurrent representationsmake a causalcontribution and merely dispositional ones do not. Virtuallyany systemthat has representations,including the systemsof RSG's brandof connectionism,will exhibitType 1 functionaldiscreteness. Commonsense also recognizesthe possibility of the othertypes of functionaldiscreteness that we distinguish,and some of thesepossibilities suggest interestingways to thinkabout cognition. We suggested(in Section2.5) that theseother types of functional discreteness could be foundin distributedconnec- tionistmodels. Thus, even if common sense psychologyis more deeply committedto functionaldiscreteness than we believe,RSG would not have shown that common sense psychologyis incompatiblewith distributed 148 / TerenceHorgan and JohnTienson

connectionism. We also argued(Section 3), contraryto RSG, thatconnectionist theory does haveprojectable predicates comparable to thepropositional attitude predicates of commonsense psychology. Even if we have succeededin showingthat RSG's argumentsare not successful,this constitutes only a limiteddefense of the compatibility of connectionismand commonsense psychology. There are otherarguments afoot (e.g., Davies 1991) thatpurport to demonstratean incompatibilitybetween connectionismand commonsense." But addressingsuch arguments and the issues they raise is a taskfor another occasion.

Notes

1. The paper has been anthologizedat least fourtimes (see the entryfor it in the bibliography),and has beenwidely discussed in therecent philosophical literature. 2. We leave open the question of what sorts of conditionsmust be met by morphologicalcontent in orderfor it to countas themorphological possession of a belief(or of some otherstate-type of commonsense psychology).Here is one plausible-lookingrequirement: in order to be a belief,the state-type must be onethat can becomean occurentthought within the cognitive system. Morphological content onlycounts as a beliefif thatcontent is also possesseddispositionally. 3. Whetherarriving at a conclusionwithout conscious inference is properlycalled inferenceat all fromthe point of view ofcommon sense is notclear. But itdoes not matterfor present purposes. 4. This sortof thing-non-tokenedinformation playing an essentialrole in cognitive processing-appearsto be ubiquitous.If itreally does occur,as thephenomenology ofconscious experience suggests, then any adequate theory of cognition needs to be able to accountfor it. 5. This argumentas statedis clearlyfallacious. It has thesame logicalform as: since thereare non-cowsin thepasture, there are no cows in thepasture. (Since network A has statesthat embody propositional information indiscretely, it lacks states that embodypropositional information discretely.) But what RSG obviouslymean is that thepropositional information embodied in thesystem's weights is notembodied by functionallydiscrete substructures of weightedconnections. 6. O'Brien says herethat the informationis encodedin the network'sconnectivity matrix,and thisis notan uncommonway to speak.We wouldchoose, however, not to saythat the weights themselves encode information, though they may be said to embody(RSG's word)information. Information is notrepresented in theweights; it's representedin the representations.The weightsdo not constitutea code. 7. In traditional,pre-connectionist, models of memory in cognitivescience, memories are full-fledgedrepresentations; they reside in a mental"file cabinet," in thesame formas whenthey become occurrent by being fetched back into the system's central processingunit. But the idea ofstored (in-the-head, or in-the-soul)memories is not presupposedby common sense psychology (even though if it might seem so to some philosopherseducated in classical cognitivescience). Common sense is not committedto any particularview aboutthe ontologyof memory;hence it is not contraryto commonsense to say thatbeliefs and memoriesare no wherewhen not Connectionismand theCommitments ofFolk Psychology/ 149

active.Cf. Locke: "Memory,signifies no morebut this, that the Mind has a Power, in manycases to revivePerceptions... . And in thisSense it is, thatour Ideas are said to be in ourMemories, when indeed, they are actually no where,but only there is an abilityin the Mind,when it will, to revivethem again.... (Essay, II.X.ii.) 8. There are manyways in whichactivated representations might be involvedin a process. Some might,for example, be false startsthat are overruledby further informationor processing.But in suchcases it is oftenpossible, by followingout pathsof spreadingactivation, to determinewhich active representations made a positivecasual contributionin bringingabout the end result,and whichactive representationsgot overruled. 9. A particulartype of cognitive transition might be traversedin several cognitive steps byone connectionistsystem but leapt in a singlebound by another. Cf. Lloyd 1991. 10. Stichand Warfield (forthcoming) make essentially the same point (without, however, distinguishingbetween paradigmatic functional discreteness and theother kinds). Needless to say, thisis a changeof positionfor Stich. 11. A morefamiliar source of multiple realizability in connectionist models is thatoften manydifferent vectors-all vectorsmeeting some condition-count as realizingthe same representation.E.g., the representationis consideredactivated when all membersof a certainset of nodes have activationlevel > .85. 12. Sometimeswhen two or moredistinct representations are realizedby a singlepoint on the activationlandscape, each occurrentrepresentation will correspondto a physicallydiscrete sub-pattern of activation within the overall activation state of the network;but sometimes the total complex representation will insteadbe a physical superpositionof physicalsub-patterns, with certain nodes participatingsimulta- neouslyin severalsub-patterns. (In Bach's piano music,often a noteplayed on a singlekey belongs simultaneously to severalsuperimposed, contrapuntal, melodies.) 13. Certainsophisticated learning techniques employ what is calledthe "moving target" strategy,which in effectbrings about a controlledco-evolution of weightsand representations;in dynamicalsystems terms, this amountsto the simultaneous moldingof the activation landscape and re-positioning of representational points on thatlandscape. We discussspecific examples in Horgan and Tienson (1992a, 1993). 14. The term'prejudice' has negativeassociations. But there are prejudices that incline one to make positivejudgments, as well as evaluativelyneutral judgments. 15. Forsterand Saidel (forthcoming)present a simplenetwork that arguably exhibits some of the kindsof functionaldiscreteness we have been describing(and also illustratessome of the ways that functionaldiscreteness is relatedto various counterfactualconditionals true of the network'sperformance). 16. RSG statethe alleged connectionism/commonsense contrastin anotherway:

Fromthe point of view of the connectionist model builder, the class of networks thatmight model a cognitiveagent who believesthat dogs have furis not a genuinekind at all, but simplya chaoticallydisjunctive set. Commonsense psychologytreats the class of people who believe thatdogs have furas a psychologicallynatural kind; connectionism does not.(514-5)

This suggestsa rhetoricallystronger eliminativist conclusion: if connectionismis correct,then certain natural kinds of common sense psychology do notexist (though it seemsodd to say thatcommon sense psychology treats the class of people who 150 / TerenceHorgan and JohnTienson

believethat dogs have fur, as opposedto the belief itself, as a naturalkind). We will discussthe argument in termsof projectablepredicates, although what we say appliesequally to naturalkinds. 17. Most any connectionistcognitive model will exhibitcertain non-accidental generalizationsatthe representational level of description. For instance, in Chapter 1 ofRumelhart and McClelland (1986), one finds the following remarks about a modelof memory retrieval that embodies information about the members of two gangs,the Sharks and the Jets:

Themodel...tends toretrieve what is commonto those memories which match a retrievalcue which is toogeneral to capture any one memory.... In thisway themodel can retrieve the typical values that members of the Jets has on each dimension-eventhough no oneJet has these typical values. (p. 30)

18. Marrlabels the top level 'the theory of the computation'. This refers to whatis to be computed.How it is computedis to be determinedat the middle level. 19. Ingeneral, the interconnections between these three levels involve a variable number ofintervening levels. Flow charts specify levels of description between the top and themiddle-increasingly specific determinations of the algorithm by whichthe cognitivefunction iscomputed. There may be several levels between an Al program in a familiarprogramming language-Marr's middle level-and themachine languageof thecomputer on whichit is running-whichis not yet a physical implementation,but ratheris the abstractspecification of the computational processesthat literally get physically implemented. The specification ofa network, includingweights and activation equations and levels, occupies a similarrole in connectionism. 20. Multiplerealizability is possible between each level and one below it. This point is commonlyrecognized with respect to thephysical realization of computational processes,but it is equallytrue for the computational realization of intentional state- transitions.For further elaboration of this point, see Horgan(1992, pp.454-6) and Horganand Tienson (1993, pp. 160).On p. 162of the latter we quotea passage fromMarr (1977) indicatingthat he himselfevidently appreciated the point quite clearly. 21. Thisgeneric framework and the possible virtues of itsconnectionist species are discussedin detailin Horganand Tienson (1994, forthcoming a). 22. For a discussionof multiplerealizability of higher-levelstates and processes in connectionistmodels, see Bickle(in preparation). 23. Daviesargues (i) thatcommon sense psychology requires a language of thought, in orderto accommodatethe distinctive causal roles that common sense posits for semanticconstituents of propositional attitudes, but (ii) thatconnectionism with distributedrepresentations is incompatible with a languageof thought. We agree withDavies that syntax is necessaryto supportthe kind of causal role of semantic constituentsthat common sense psychologyimplies. But we maintainthat connectionismdoes not preclude either syntactic structure inmental representations or structure-sensitiveprocessing; nor does it becomemere implementation of classicismby incorporating these features. Cf. Horgan and Tienson (1988, 1989, 1992a,1992b, forthcoming a, forthcoming b). Connectionismand theCommitments of Folk Psychology/ 151

References

Berg,G. (1992). "A ConnectionistParser with Recursive Sentence Structure and Lexical Disambiguation,"Proceedings of the American Association for Artificial Intelligence. Bickle, J. (in preparation)."Connectionism, Reduction, and MultipleRealizability." Clark,A. (1990). "ConnectionistMinds," Proceedings of theAristotelian Society, 90. Reprintedin C. andG. MacDonald(eds.), ThePhilosophy ofPsychology (Blackwell, forthcoming). Davies, M. (1991). "Concepts,Connectionism, and the Languageof Thought,"in W. Ramsey,S. Stich,and D. Rumelhart(eds.), Philosophyand ConnectionistTheory, Earlbaum. Forster,M. and Saidel, E. (forthcoming)."Connectionism and the Fate of Folk Psychology:A Replyto Ramsey,Stich, and Garon,"Philosophical Psychology. Horgan,T. (1992). "FromCognitive Science to Folk Psychology:Computation, Mental Representation,and Belief,"Philosophy and PhenonemologicalResearch, 52. Horgan,T. and Tienson,J. (1988), "Settlinginto a New Paradigm,"Southern Journal of Philosophy,24, Supplement.Reprinted in T. Horgan and J. Tienson (eds.), Connectionismand thePhilosophy of Mind (Earlbaum,1991). Horgan,T. andTienson, J. (1989). "Representationswithout Rules," Philosophical Topics, 17. Horgan,T. and Tienson,J. (1992a). "CognitiveSystems as DynamicalSystems," Topoi, 11. Horgan, T. and Tienson, J. (1992b) "StructuredRepresentations in Connectionist Systems?"in S. Davis (ed.), Connectionism:Theory and Practice,Oxford. Horgan,T. and Tienson,J. (1993). "Levels of Descriptionin NonclassicalCognitive Science,"in C. Hookwayand D. Peterson(eds.), Philosophy and CognitiveScience, Cambridge. Horgan,T. and Tienson,J. (forthcoming a). "A NonclassicalFramework for Cognitive Science,"Synthese, issue on connectionismand philosophy. Horgan,T. and Tienson,J. (forthcomingb). Connectionismand the Philosophyof Psychology:Representational Realism without Rules. M.I.T. Lloyd, D. (1991). "Leaping to Conclusions:Connectionism, Consciousness, and the ComputationalMind," in T. Horganand J.Tienson (eds.), Connectionismand the Philosophyof Mind,Earlbaum. Marr,D. (1977). "ArtificialIntelligence-A PersonalView," ArtifcialIntelligence, 9. Reprintedin J.Haugeland (ed.), Mind Design,(M.I.T., 1981). Marr,D. (1988). Vision,Freeman. O'Brien,G. (1991). "Is ConnectionismCommonsense?", Philosophical Psychology, 4. Ramsey,W., Stich,S., and Garon,J. (1990). "Connectionism,Eliminativism and the Futureof Folk Psychology,"Philosophical Perspectives, 4, Action Theoryand Philosophyof Mind, 1990, editedby J.E. Tomberlin.Reprinted in W. Greenwood (ed.), TheFuture of Folk Psychology(Cambridge, 1991); in W. Ramsey,S. Stich, and D. Rumelhart(eds.), Philosophyand ConnectionistTheory (Earlbaum,); in S. Christensenand D. Turner(eds.), Folk Psychologyand thePhilosophy of Mind (Earlbaum, 1993); and in C. and G. MacDonald (eds.), The Philosophyof Psychology(Blackwell, forthcoming). 152 / TerenceHorgan and JohnTienson

Rumelhart,D. and McClelland,J. (1986). Parallel DistributedProcessing: Explorations in theMicrostructure of Cognition.Volume 1: Foundations,M.I.T. Stich,S. (1991). "Causal Holismand CommonsensePsychology: A Replyto O'Brien," PhilosophicalPsychology, 4. Stich,S. and Warfield,T. (forthcoming)."Do ConnectionistMinds Have Beliefs?-A Replyto Clarkand Smolensky,"in C. and G. MacDonald(eds.), ThePhilosophy of Psychology,Blackwell.