S EEING AND VISUALIZING : I T ’ S N OT W HAT Y OU T HINK * An Essay On Vision and Visual Imagination ZENON PYLYSHYN, RUTGERS CENTER FOR COGNITIVE SCIENCE Table of Contents Preface ........................................................................................................................................... v 1. The Puzzle of Seeing .......................................................................................................1-1 1.1 Why do things look the way they do?...................................................................................1-1 1.2 What is seeing? ..................................................................................................................1-2 1.3 Does vision create a “picture” in the head? ...........................................................................1-3 1.3.1 The richness of visual appearances and the poverty of visual information.................................1-3 1.3.2 Some reasons for thinking there may be an inner display..............................................................1-6 1.4 Some problems with the Inner Display Assumption: Part I: What’s in the display?..................1-10 1.4.1 How is the master image built up from glances?............................................................................1-10 1.4.2 What is the form of non-retinal information?.................................................................................1-11 1.4.3 How “pictorial” is information in the “visual buffer”?................................................................1-17 Some further problems with the Inner Display Assumption Part II: seeing or figuring out? ..............1-22 1.4.4 A note about terminology...................................................................................................................1-23 1.4.5 “Seeing X” versus “believing that what you saw is X”.................................................................1-24 1.4.6 Reports of what something “looks like”: What do they mean?....................................................1-25 1.4.7 Vision and conscious appearance: Can they be separated? ........................................................1-27 1.5 Where do we go from here?..............................................................................................1-28 2. The Independence of Vision and Cognition ....................................................................2-1 2.1 Is Vision distinct from reasoning?.........................................................................................2-1 2.1.1 What determines what we see? Do we see what we expect to see?...............................................2-1 2.2 The case for the continuity of vision and cognition ................................................................2-2 2.2.1 The “New Look” in the psychology of perception ............................................................................2-2 2.2.2 The perspective of neuroscience..........................................................................................................2-4 2.2.3 The perspective of robot vision ...........................................................................................................2-5 2.2.4 Seeing and knowing: Where do we stand?........................................................................................2-8 2.3 Some reasons for questioning the continuity thesis.................................................................2-9 2.3.1 Evidence of the cognitive impenetrability of illusions.....................................................................2-9 2.3.2 Evidence of the independence of principles of visual organization and of inference...............2-10 2.3.3 Neuroscience evidence for top-down attenuation and gating of visual signals........................2-11 2.3.4 Evidence of dissociation of visual and cognitive functions in the brain.....................................2-12 2.3.5 Evidence that cognitive penetration occurs in pre-perceptual and post-perceptual stages....2-13 2.4 Distinguishing perceptual and decision stages: Some methodological issues ............................2-14 2.5 Some examples in which knowledge is claimed to affect perception ......................................2-15 2.5.1 “Intelligent” interpretations of inherently ambiguous information............................................2-15 2.5.2 Experience and “hints” in perceiving ambiguous figures and stereograms..............................2-15 2.5.3 Learning to “see” differently: A case of controlling focal attention?.........................................2-18 2.5.3.1 Cultural and linguistic effects on perception............................................................................................................................2-19 2.5.3.2 The case of “expert” perceivers ..................................................................................................................................................2-19 2.5.3.3 Attention and perceptual learning................................................................................................................................................2-21 * Manuscript of a forthcoming book from MIT Press. Please do not quote without permission. Ó 1998 by Zenon Pylyshyn 7/1/2002 i 2.5.4 Focal attention as an interface between vision and cognition.....................................................2-22 2.6 Conclusions: Early vision as a cognitively impenetrable system.............................................2-22 3. The Architecture of the Early Vision System: Components and Functions.................3-1 3.1 Some intrinsic architectural factors determining the visual percept...........................................3-1 3.1.1 Natural Constraints in Vision..............................................................................................................3-1 3.1.1.1 An example of the use of natural constraints in feature labeling .......................................................................................... 3-4 3.1.1.2 Other constraints: The “rules” of visual analysis ....................................................................................................................... 3-9 3.1.2 On “Intelligence” and “Problem-solving” in vision......................................................................3-13 3.1.3 Constraints do not reflect high-frequency or physically permissible real-world principles...3-17 3.2 What is computed by the encapsulated early vision system?.................................................3-19 3.2.1 What are the inputs to the visual system?........................................................................................3-20 3.2.1.1 Cross-modal visual processing.....................................................................................................................................................3-21 3.2.1.2 Synesthesia: Cross-Modal perception?.......................................................................................................................................3-21 3.2.1.3 Should “visual input” include actual and potential motor actions?....................................................................................3-23 3.2.2 What are the outputs of the visual system? .....................................................................................3-25 3.2.2.1 The visual output: Categories and surface layouts..................................................................................................................3-25 3.3 Some subprocesses of the visual system.............................................................................3-30 3.3.1 Stages in early vision: Microgenesis of vision................................................................................3-33 3.4 Visual control of action: An encapsulated system?................................................................3-35 3.4.1 Evidence from studies of eye movements and reaching.................................................................3-36 3.4.2 Evidence from clinical neurology......................................................................................................3-37 3.4.3 Evidence from animal studies............................................................................................................3-37 3.5 Is “vision” one system or many?........................................................................................3-38 4. Focal Attention: How cognition Influences Vision .........................................................4-1 4.1 Focal attention in visual perception.......................................................................................4-1 4.2 Focal attention as selective filtering ......................................................................................4-2 4.3 Allocation of visual attention................................................................................................4-4 4.4 Individuation: A precursor to attentional allocation?................................................................4-8 4.4.1 Individuation is different from discrimination..................................................................................4-8 4.4.2 Individuating an object is distinct from encoding its location or visual properties..................4-10 4.5 Visual attention is directed at Objects .................................................................................4-12 4.5.1