Things and Places: How the Mind Connects with the World
Total Page：16
File Type：pdf, Size：1020Kb
Things and Places: How the Mind Connects With the World Zenon W. Pylyshyn Rutgers Center for Cognitive Science Forthcoming, 2007, MIT Press (Jean Nicod Lecture Series) i 11/26/2006 Table of contents Preface & Acknowledgements Chapter 1. Introduction to the Problem: Connecting Perception and the World 1.1 Background 1.2 What’s the problem of connecting the mind with the world? Doesn’t every computational theory of vision do that ? 1.3 The need for a direct way of referring to certain individual tokens in a scene 1.3.1 Incremental construction of representations (and a brief sketch of FINSTs) 1.3.2 Using descriptions to pick out individuals 1.3.3 The need for demonstrative reference in perception 1.4 Some empirical phenomena illustrating the role of Indexes 1.4.1 Tagging/marking individual objects for attentional priority 1.4.2 Argument binding 1.4.3 Subitizing 1.4.4 Subset selection 1.5 What are we to make of such empirical demonstrations? Chapter 2. Indexing and tracking individuals 2.1 Individuating and tracking 2.2 Indexes and primitive tracking 2.3 What goes on in MOT? 2.3.1 FINSTs and Object Files 2.3.2 The explanation of tracking 2.4 Other empirical and theoretical issues surrounding MOT 2.4.1 Do we track by keeping a record of locations? 2.4.2 Can we select objects voluntarily? 2.4.3 Tracking without keeping track of labels 2.4.4 Nonconceptual individuation without reference? 2.5 Infants capacity for individuating and tracking objects 2.6 Summary and Implications for the Foundations of Cognitive Science 2.6.1 Review: Nonconceptual functions and Natural Constraints 2.6.2 Summary: Why are FINSTs needed? Chapter 3. Selection: The key to linking representations and things 3.1 Selection: The role of focal attention 3.1.1 Allocating and shifting attention: The role of objects and places 3.1.2 Attention selects and adheres to objects 3.2 Selection and demonstrative reference: The role of FINSTs 3.2.1 Causes and codes 3.2.2 Conceptual and nonconceptual representations and quasi-representations 3.3 Problems with selection-by-location 3.3.1 A note on the role of location in selection and tracking 3.3.2 Selection and the binding problem 3.3.3 Selection and the causal link 3.3.4 Selection as nonconceptual access 3.4 Feature placing and sentience 3.5 What do FINSTs select? Some consequences of the present view ii 11/26/2006 Chapter 4. Conscious contents and nonconceptual representation 4.1 Nonconceptual representation and perceptual beliefs 4.2 The role of conscious experience in the study of perception and cognition 4.2.1 The Contents question 4.2.2 Conscious experience and public report 4.3 What subjective experience reveals about psychological processes 4.3.1 The illusion of conscious will 4.3.2 Conscious Experience, Interpretation and Confabulation 4.3.3 Failures of Conscious Access: Split Brains and Split Visual Systems 4.4 The phenomenal experience of seeing 4.4.1 Nonconceptual representation in visual perception 4.5 The phenomenal experience of mental imagery 4.5.1 Phenomenal experience and explanation: The role of tacit knowledge 4.5.2 Does the architecture of visual cortex matter to explanations of imagery? 4.5.3 Problems in accounting for phenomenal space by appealing to brain-space 4.6 Does phenomenal appearance correspond to a level of representation? Chapter 5. How we represent space: Internal vs External Constraints 5.1 What does it mean to represent space? 5.2 Internalizing general spatial constraints 5.2.1 Marr’s principle of Natural Constraints 5.2.2 Shepard’s psychophysical complementarity 5.3 Internalizing spatial properties by mapping them onto an inner space 5.3.1 Brain Space 5.3.2 Functional space and principled constraints 5.3.3 Internalizing by incorporating visuomotor experience: Poincaré’s insights 5.4 What is special about representing space? 5.4.1 Conditions on a theory of Active Spatial Representation (ASPAR) 5.4.2 Maps and map-like navigation behavior 5.5 Externalizing spatial properties: The projection hypothesis 5.5.1 Where is our spatial representation? 5.5.2 FINST Indexes can anchor spatial representations to real space 5.6 Projection in non-visual modalities 5.6.1 The unitary frame of reference assumption 5.6.2 The role of coordinate transformations in externalizing space 5.6.3 Proprioceptive FINSTs? The projection hypothesis in nonvisual modalities 5.6.4 Motor control, proprioception and intention define places in allocentric space 5.6.5 Summary of evidence in support of the projection hypothesis Conclusions References iii 11/26/2006 Preface & Acknowledgements This book is based in part on the Jean Nicod Lectures that I delivered in Paris in May-June of 2004. The temporal gap between the lectures and the publication is not due entirely to my slow typing, but arose from my need to assimilate the rather wide range of publications that are relevant to the thesis I am presenting. The thesis, it turns out, is one that I have been gestating over many years and hints of it occur in fragmentary form in a number of my publications. Many of these are reports of experimental work carried out with graduate students over the years and whose contribution is much appreciated. The thesis rests on a growing appreciation of an idea I first learned from David Marr who refers to it as the principle of Natural Constraints. The mind has been tuned over its evolutionary history so that it carries out certain functions in a modular fashion, without regard for what an organism knows or believes or desires, but because it is in its nature, or as I more often put it, because of its architecture. So far this is an innocent enough idea that fits many different schools of psychology (and in fact is at the heart of J.J.Gibson’s Direct Realism Theory). The particular constraint I am interested in here takes the form of a mechanism that allows the modular perceptual system to do things that many philosophers have said (correctly) can only be done by using the sophisticated machinery of concepts and the logical machinery of induction, deduction and what Charles Sander Peirce called abduction. The mechanism includes the capacity to select individual things in one’s field of view, to re-identify each of them under certain conditions as the same individual thing that was seen before and to keep track of their enduring individuality despite radical changes in their properties. I claim that so long as we are in the kind of world we live in there are mechanisms that allow the visual system to do these things without using the heavy equipment of concepts, identity, and tenses (which really are needed in the general case). For example this is a world in which most surfaces that we can see are surfaces of physical objects, so that most of the texture elements we see move coherently as the object moves, almost all elements nearby on the proximal image are at the same distance from the viewer, and when objects disappear they frequently reappear nearby and often with a particular pattern of occlusion and disocclusion at the edges of the occluding opaque surfaces, and so on. Identifying things as ones we have seen before and keeping track of them as being the same individual things over time is at the heart of the research I have been doing and it has shown that we are very good at doing it in a way that does not use encoded properties (nor the conceptual category) of the things that are tracked and re-identified. This mechanism is important to us because if it were not for the existence of such nonconceptual processes, our concepts would not be grounded in experience so they would not have the meaning that they do. I have proposed that the capacity to individuate and track several independently-moving things is accomplished by a mechanism in the early vision module that I have called FINSTs (for historical reasons I called them “Fingers of INSTantiation” because they were initially viewed as a mechanism for instantiating or binding the arguments of visual predicates to objects in the world). This primitive nonconceptual mechanism functions to identify, reidentify and track distal objects. It is an ability that we exercise every waking minute, and has also been understood to be fundamental to the way we see and understand the world. I came upon these ideas in quite a different context, initially one in which I (along with my colleague Edward Elcock) attempted to develop a computer system for reasoning about diagrams and later when I was carrying out experimental research on vision, visual attention and mental imagery. This may seem like a very circuitous route, but it has turned out that all these endeavors involved the same puzzles, which I later discovered were also the puzzles that preoccupied many philosophers: The puzzle of how concepts are grounded in experience, how iv 11/26/2006 we manage to encode and represent properties of the world when there are so many of them, why we feel that we are conscious of seeing an enormous number of things but are unable to report most of them, while at the same time a great deal of information of which we are not conscious can affect our behavior. These puzzles appear in their most striking form in discussions of two related problems: What are properties of mental images that allows them to function in thought, and How do certain kinds of thoughts – thoughts about spatial layouts – manage to display properties very similar to those of perceived space.