BEHAVIORAL AND BRAIN SCIENCES (2001) 24, 939–1031 Printed in the United States of America A sensorimotor account of vision and visual consciousness J. Kevin O’Regan Laboratoire de Psychologie Expérimentale, Centre National de Recherche Scientifique, Université René Descartes, 92774 Boulogne Billancourt, France [email protected] http://nivea.psycho.univ-paris5.fr Alva Noë Department of Philosophy, University of California at Santa Cruz, Santa Cruz, CA 95064 [email protected] http://www2.ucsc.edu/people/anoe/ Abstract: Many current neurophysiological, psychophysical, and psychological approaches to vision rest on the idea that when we see, the brain produces an internal representation of the world. The activation of this internal representation is assumed to give rise to the experience of seeing. The problem with this kind of approach is that it leaves unexplained how the existence of such a detailed internal representation might produce visual consciousness. An alternative proposal is made here. We propose that seeing is a way of acting. It is a particular way of exploring the environment. Activity in internal representations does not generate the experience of seeing. The out- side world serves as its own, external, representation. The experience of seeing occurs when the organism masters what we call the gov- erning laws of sensorimotor contingency. The advantage of this approach is that it provides a natural and principled way of accounting for visual consciousness, and for the differences in the perceived quality of sensory experience in the different sensory modalities. Sev- eral lines of empirical evidence are brought forward in support of the theory, in particular: evidence from experiments in sensorimotor adaptation, visual “filling in,” visual stability despite eye movements, change blindness, sensory substitution, and color perception. Keywords: action; change blindness; consciousness; experience; perception; qualia; sensation; sensorimotor 1. Introduction 1.1. The puzzle of visual experience What is visual experience and where does it occur? Kevin O’Regan moved to Paris in 1975 after studying theoretical physics in England, to work in experimental It is generally thought that somewhere in the brain an in- psychology at the Centre National de Recherche Scien- ternal representation of the outside world must be set up tifique. After his Ph.D. on eye movements in reading he which, when it is activated, gives us the experience that we showed the existence of an optimal position for the eye all share of the rich, three-dimensional, colorful world. Cor- to fixate in words. His interest in the problem of the per- tical maps – those cortical areas where visual information ceived stability of the visual world led him to question seems to be retinotopically organized – might appear to be established notions of the nature of visual perception, good candidates for the locus of perception. and to recently discover, with collaborators, the phe- Cortical maps undoubtedly exist, and they contain infor- nomenon of “change blindness.” His current work in- mation about the visual world. But the presence of these volves exploring the empirical consequences of the new maps and the retinotopic nature of their organization can- approach to vision. not in itself explain the metric quality of visual phenome- Alva Noë is a philosopher at the University of Califor- nology. Nor can it explain why activation of cortical maps nia, Santa Cruz. He received a Ph.D. in philosophy from should produce visual experience. Something extra would Harvard University and a B. Phil. from Oxford Univer- sity, and he has been a Research Associate of the Cen- appear to be needed in order to make excitation in cortical ter for Cognitive Studies at Tufts University. He has maps provide, in addition, the subjective impression of see- published articles on topics in the philosophy of per- ing. ception, philosophy of mind, and other areas, including A number of proposals have come forth in recent years a previous Behavioral and Brain Sciences target article to suggest how this might come about. For example, it has on perceptual completion. He is currently at work on a been suggested, from work with blindsight patients, that book on the relation between perception and action, consciousness in vision may derive from a “commentary” and he is a co-editor of Vision and Mind: Selected Read- system situated somewhere in the fronto-limbic complex ings in the Philosophy of Perception (MIT Press, forth- (taken to include the prefrontal cortex, insula and claus- coming). trum; cf. Weiskrantz 1997, p. 226). Crick and Koch (1990), © 2001 Cambridge University Press 0140-525X/01 $12.50 939 O’Regan & Noë: A sensorimotor account of vision and visual consciousness Llinas and Ribary (1993), Singer (1993), and Singer and certainly true that retinal influx comes together in relatively Gray (1995) suggest that consciousness might be correlated circumscribed areas of the brain, and that this may provide with particular states of the brain involving coherent oscil- an architectural advantage in the neural implementation of lations in the 40–70 Hz range, which would serve to bind the calculations necessary to generate visual-type sensa- together the percepts pertaining to a particular conscious tions. But what is it about these pathways that generates the moment.1 Penrose (1994) and Hameroff (1994) suggest different sensations? Surely the choice of a particular sub- that the locus of consciousness might be a quantum process set of neurons or particular cortical regions cannot, in itself, in neurons’ microtubules. Edelman (1989) holds that re- explain why we attribute visual rather than auditory qua- entrant signaling between cortical maps might give rise to lities to this influx. We could suppose that the neurons consciousness. A variety of other possibilities that might involved are of a different kind, with, say, different neuro- constitute the “neural correlate of consciousness” has been transmitters, but then why and how do different neuro- compiled by Chalmers (1996b). transmitters give rise to different experiences? We could A problem with proposals of this kind is that they do lit- say that the type of calculation done in the different corti- tle to elucidate the mystery of visual consciousness (as cal areas is different, but then we must ask, how could cal- pointed out by, for example, Chalmers 1996b). For even if culations ever give rise to experience? The hard work is left one particular mechanism – for example, coherent oscilla- undone. Much still needs to be explained. tions in a particular brain area – were proven to correlate perfectly with behavioral measures of consciousness, the problem of consciousness would simply be pushed back 1.3. An alternative approach: The sensorimotor into a deeper hiding place: the question would now be- contingency theory come, why and how should coherent oscillations ever gen- The present paper seeks to overcome the difficulties de- erate consciousness? After all, coherent oscillations are ob- scribed above by adopting a different approach to the prob- served in many other branches of science, where they do lem of visual experience. Instead of assuming that vision not generate consciousness. And even if consciousness is as- consists in the creation of an internal representation of the sumed to arise from some new, previously unknown mech- outside world whose activation somehow generates visual anism, such as quantum-gravity processes in tubules, the experience, we propose to treat vision as an exploratory ac- puzzle still remains as to what exactly it is about tubules that tivity. We then examine what this activity actually consists allows them to generate consciousness, when other physi- in. The central idea of our new approach is that vision is a cal mechanisms do not. mode of exploration of the world that is mediated by knowl- edge of what we call sensorimotor contingencies. We show 1.2. What are sensory modalities? that problems about the nature of visual consciousness, the qualitative character of visual experience, and the differ- In addition to the problem of the origin of experience dis- ence between vision and other sensory modalities, can now, cussed in the preceding paragraphs, there is the problem of from the new standpoint, all be approached in a natural differences in the felt quality of visual experience. Why is way, without appealing to mysterious or arcane explanatory the experience of red more like the experience of pink than devices. it is like that of black? And, more generally, why is seeing red very different from hearing a sound or smelling a smell? It is tempting to think that seeing red is like seeing pink 2. The structure of vision because the neural stimulation going on when we see some- thing red is similar to that underlying our perception of pink: As stated above, we propose that vision is a mode of explo- almost the same ratios of long, medium and short wave- ration of the world that is mediated by knowledge, on the length photoreceptors will be stimulated by red and pink. part of the perceiver, of what we call sensorimotor contin- But note that though this seems reasonable, it does not suf- gencies. We now explore this claim in detail. fice: there is no a priori reason why similar neural processes should generate similar percepts.2 If neural activity is just 2.1. Sensorimotor contingencies induced an arbitrary code, then an explanation is needed for the par- by the visual apparatus ticular sensory experience that will be associated with each element of the code. Why, for example, should more in- Imagine a team of engineers operating a remote-controlled tense neural activity provoke more intense experiences? underwater vessel exploring the remains of the Titanic, and And what exactly is the mapping function: is it linear, loga- imagine a villainous aquatic monster that has interfered rithmic, or a power function? And why is it one of these with the control cable by mixing up the connections to and rather than another? Even these questions leave open the from the underwater cameras, sonar equipment, robot arms, more fundamental question of how a neural code could actuators, and sensors.