Psychology Jerry Fodor & Zenon Pylyshyn – Connectionism and Cognitive Architecture Notes A Critical Analysis
1. Introduction
• Fodor and Pylyshyn (F&P) are not impressed by the wave of enthusiasm for connectionism, Parallel Distributed Processing – PDP. • Classical AI = Operation on Symbols, but connectionism is strikingly unlike this. • F&P’s account of connectionist architecture seems to differ from Churchland’s for whom the initial state is limited to the weights on the connections rather than including the initial unit activation (maybe I got this wrong). • The term “connectionist model” is a vague term for a galaxy of architectural commitments. • Connectionist networks can recognise patterns, exhibit rule-like behavioural regularities, and map virtually any input pattern to an output one, though this can require a very large number of units. • Connectionist models have neural plausibility. • F&P consider that the theoretical issues favour classical AI. • F&P’s approach in this paper is (1) discuss levels of explanation, (2) discuss what makes connectionist and classical architectures incompatible and (3) revise some arguments in favour of the classical approach.
1.1 Levels of explanation
• Representational (semantic / intentional) versus eliminativist (neurological / behaviourist) theories of mind. • Surprisingly, connectionists are on the representationalist side. Example of connectionist account of the bi-stability of the Necker cube. • But talk of sub-symbolic processes (presumably not representational) confuses the issue. But sub-symbolic states do have semantics – “sub-symbolic theories slice representational states thinner”. • Quarks to galaxies: different levels. Analogy between different levels of scientific and psychological explanation. • The cognitive level is the level at which there is property-encoding. • Architecture of cognitive system at level of representational states of the organism. • It’s the interconnection of representational states that’s important. • Introduction of thought that classical cognitive architecture can be implemented on connectionist networks. • The lower level of sub-symbolic processes produces rule-like behaviour. This isn’t a problem for classical AI. • Everyone who’s a materialist agrees that there are sub-symbolic interactions that implement both rule-like and rule-violating behaviour. • Classical architecture not committed per se to explicit rules.
2. The Nature of the Dispute
• Connectionists assign content to nodes (units or aggregates of units). • Storage versus relations; causal (connectionist) versus causal plus structural (classical) relations. • Architectural differences: defining characteristics of classical models are:-
[email protected] Page 1 of 5
Psychology Jerry Fodor & Zenon Pylyshyn – Connectionism and Cognitive Architecture Notes A Critical Analysis
1). Combinatorial syntax and semantics form mental representations: classical theories have a language of thought, missing from connectionist theories. The semantic content of a molecular representation is a function of the semantic contents of its syntactic parts. 2). Structure sensitivity of processes: transformations defined over structural properties of mental representations. Example of (P & Q) Q. Correspondence to real physical structure in the brain. • Object-oriented or message-passing architectures OK classically. Executive control isn’t the issue either, nor is synchronicity versus asynchronicity.
2.1 Complex mental representations
• In a classical machine, the objects to which content is ascribed contain as proper parts objects to which the component parts of the complex object are ascribed (eg. A & B). The semantics of a complex are determined in a uniform way by the semantics of its constituents. • None of this is true of connectionist architectures. Causal, but no structural, connections.
2.1.4 Representations as distributed over microfeatures
• Connectionist attempt to explain representation as conceptual-level units corresponding to vectors in a sub-conceptual space of microfeatures. Microfeature vectors replace macro-level concepts. • Suggestion to replace classical distinction between complex symbols and their constituents by feature-sets and their subsets, with role-relations captured by features (instead of by relations amongst constituents). • Feature-sets are of the form <+John-subject; +Mary-object; +loves>. • Various problems with these ideas – not all subsets of features correspond to genuine constituents. Also issues about the order of the constituents. •
2.2 Structure-sensitive operations
• The classical treatment of mental ideas rests on two ideas: 1. Syntax encodes meaning, with semantic inference from syntax alone. 2. It is possible to build machines that transform symbols in a way sensitive to their syntactical structure. • The brain is a semantic engine. •
2.2.1 Learning
• Boltzmann machines – adjustment of weights between units so that behaviour models statistical properties of inputs. • Associationist principle; Pavlov. • Statistical versus logical / structural. •
[email protected] Page 2 of 5
Psychology Jerry Fodor & Zenon Pylyshyn – Connectionism and Cognitive Architecture Notes A Critical Analysis
2.2.2 Reasoning
• Associative strength is not sensitive to content or structure. Structure is not relevant to the output. • Classical and connectionist theories disagree about the nature of mental representation and processes. Classical but not connectionist theories exhibit combinatorial constituent structures and semantics. Again, for classical but not connectionist theories, mental processes are sensitive to the combinatorial structure of the representations on which they operate.
3. The need for symbol systems: productivity, systematicity and inferential coherence
• Arguments for the classical theory.
3.1 Productivity of thought
• Finite means to indefinite representational capacity. •
3.2 Systematicity of cognitive representation
• Systematicity with respect to linguistic capacity is that the ability to produce or understand some sentences is intrinsically connected to the ability to produce or understand others. F&P consider systematicity to be a pervasive feature of human thought, not just on language.
3.4 The systematicity of inference
• The syntax of mental representations mediates between their semantic properties and their causal roles in mental processes. So, inferences of similar logical type ought to elicit similar cognitive capacities. •
3.5 Summary
• What’s wrong with connectionist architectures is that they deal with lists rather than structure. So, there’s nothing to prevent minds that are arbitrarily unsystematic – an idea F&P take to be preposterous, because the systematicity of cognitive capacities is pervasive.
4. The allure of connectionism
• Rapidity of cognitive processes relative to neural speeds: the “hundred step” constraint. The brain cannot work serially because there’s only time for 100 neural firings in the second required for (say) face recognition. F&P note that the brain is only massively parallel at certain things – such as image-recognition; it’s dreadful at logic!
[email protected] Page 3 of 5
Psychology Jerry Fodor & Zenon Pylyshyn – Connectionism and Cognitive Architecture Notes A Critical Analysis
• Difficulty of large-capacity pattern recognition and content-based retrieval using conventional architectures. Pattern recognition again. • Lack of progress with non-verbal or intuitive processes. But non-verbal and intuitive processes are very different. • Acute sensitivity of conventional architectures to damage and noise. • Conventional rule-based systems depict cognition as all-or-none (continuous variation or degrees of relevance of principles, rules or procedures; non- determination of human behaviour; failure of classical systems to display graceful degradation). • Conventional models take no account of the facts of neuroscience, but are dictated by current computer technology.
4.1 Replies: why the usual reasons for preferring connectionist architectures are invalid.
• F&P think here are fundamentally two sorts of defects in these arguments against classical architectures: 1. Not intrinsic to classical architectures. 2. Addressed at the implementation level. These arguments would have less force directed at classical AI implemented on other than contemporary computers.
4.1.1 Parallel computation and the issue of speed
• This is an implementation issue – it’s absurd to think of the brain as approximating to a classical computer. • Parallel execution of multiple symbolic manipulations isn’t ruled out by classical architectures.
4.1.2 Resistance to noise and physical damage (the argument for distributed representation)
• This is an implementation-level issue solved by distributed representation. Functional proximity does not imply physical proximity.
4.1.3 “Soft” constraints, continuous magnitudes and stochastic mechanisms
• Addressed by classical computers. Can cover graceful degradation – a programming issue.
4.1.4 Explicitness of rules
•
4.1.5 On “brain-style” modelling
•
4.2 Concluding comments: connectionism as a theory of implementation [email protected] Page 4 of 5
Psychology Jerry Fodor & Zenon Pylyshyn – Connectionism and Cognitive Architecture Notes A Critical Analysis
•
5. Conclusion
• There are four possibilities open to connectionists: 1). Carry on as normal. 2). Keep associationist mental processing, but adopt structural representation. 3). Implementation 4).
[email protected] Page 5 of 5