A Framework for Consciousness Francis Crick and Christof Koch

commentary

A framework for consciousness Francis Crick and Christof Koch

Here we summarize our present approach to the problem of consciousness. After an introduction outlining our general strategy, we describe what is meant by the term ‘framework’ and set it out under ten headings. This framework offers a coherent scheme for explaining the neural correlates of (visual) consciousness in terms of competing cellular assemblies. Most of the ideas we favor have been suggested before, but their combination is original. We also outline some general experimental approaches to the problem and, finally, acknowledge some relevant aspects of the brain that have been left out of the proposed framework.

General strategy framework may well apply, however, to see, for example, either introns or RNA The most difficult aspect of consciousness is other sensory modalities. We have been editing. And who would have guessed the so-called ‘hard problem’ of qualia1,2— especially interested in the alert macaque that DNA replication usually starts with the redness of red, the painfulness of pain, monkey because to find the NCC, not the synthesis of a short stretch of RNA, and so on. No one has produced any plau- only widespread neural activities but also which is then removed and replaced by sible explanation as to how the experience the detailed behavior of single neurons (or DNA? The broad framework acted as a of the redness of red could arise from the small groups of neurons) must be inves- guide, but careful experimentation was http://www.nature.com/natureneuroscience actions of the brain. It appears fruitless to tigated on very fast time scales. This is dif- needed for the true details to be discov- approach this problem head-on. Instead, ficult to do systematically in humans. ered. This lesson is broadly applicable we are attempting to find the neural corre- Methods such as fMRI are too coarse in throughout biology. late(s) of consciousness (NCC), in the hope both space and time to be of much use for that when we can explain the NCC in causal our problem. On the other hand, experi- A preamble on the cerebral cortex terms, this will make the problem of qualia ments in visual psychology are much eas- By the ‘cortical system’, we mean the cere- clearer3. In round terms, the NCC is the ier to do on humans than on monkeys. bral cortex plus other regions closely asso- minimal set of neuronal events that gives Moreover, humans can report what they ciated with it, such as the thalamus and rise to a specific aspect of a conscious per- are conscious of and thus convey the ‘con- the claustrum, and probably the basal cept. We discuss elsewhere4 why we think tents’ of their consciousness. For these ganglia, the cerebellum and the many consciousness has to be largely private. By reasons, experiments on monkeys and widespread brainstem projection systems. ‘private’, we mean that it is accessible exclu- humans should be pursued in parallel. We shall refer to the ‘front’ of the cor- sively to the owner of the brain; it is impos- tex and the ‘back’ of the cortex because

© Group 2003 Nature Publishing sible for me to convey to you the exact Framework the terms ‘frontal’ and ‘prefrontal’ can be nature of my conscious percept of the color A framework is not a detailed hypothesis ambiguous. For example, is the anterior red, though I can convey information about or set of hypotheses; rather, it is a sug- cingulate prefrontal? it, such as whether two shades of red appear gested point of view for an attack on a sci- The dividing line between front and to me to be the same or different. entific problem, often suggesting testable back is somewhat arbitrary. It roughly Our main interest is not the enabling hypotheses. Biological frameworks differ coincides with the central sulcus. It may factors needed for all forms of conscious- from frameworks in physics and chem- turn out that a good operational defini- ness, such as the activity of the ascending istry because of the nature of evolution. tion is that the front is all those parts that reticular systems in the brainstem. Rather, Biological systems do not have rigid laws, receive a significant input, via the thala- we are interested in the general nature of as physics has. Evolution produces mech- mus, from the basal ganglia. (This simple the neural activities that produce each anisms, and often sub-mechanisms, so division is probably not useful for olfac- particular aspect of consciousness, such that there are few ‘rules’ in biology which tion, however.) as perceiving the specific color, shape or do not have occasional exceptions. There is an absolutely astonishing vari- movement of an object. A good framework is one that sounds ety and specificity of actions performed As a matter of tactics, we have con- reasonably plausible relative to available by the cortical system. The visual system centrated on the visual system of primates, scientific data and that turns out to be of the higher mammals handles an almost leaving on one side some of the more dif- largely correct. It is unlikely to be correct infinite variety of visual inputs and reacts ficult aspects of consciousness, such as in all the details. A framework often con- to them in detail and with remarkable emotion and self-consciousness. Our tains unstated (and often unrecognized) accuracy. It is clear that the system is high- assumptions, but this is unavoidable. ly evolved, is likely to be specified epige- Francis Crick is at the Salk Institute for An example from molecular biology netically in considerable detail and can Biological Studies, 10010 N. Torrey Pines Road, might be helpful. The double-helical learn a large amount from experience. La Jolla, California 92037, USA. structure of DNA immediately suggest- The main function of the sensory cor- Christof Koch is at the California Institute of ed, in a novel way, the general nature of tex is to construct and use highly specific Technology, 1200 East California Boulevard, gene composition, gene replication and feature detectors, such as those for orien- Pasadena, California 91125, USA. gene action. This framework turned out tation, motion or faces. The features to e-mail: [email protected] to be broadly correct, but it did not fore- which any cortical neuron responds

nature neuroscience • volume 6 no 2 • february 2003 119 commentary

are usually highly specific but multi- rather than following just one method (Further discussion of this idea comes at dimensional. That is, one neuron does not very accurately. This appears to be how, the end of subhead #3 below.) respond to a single feature but to a family for example, people see in depth. We have discussed elsewhere17 whether of related features. Such features are some- Incoming visual information is often the neural activity in the front of the brain times called the ‘receptive field’ of that incomplete or ambiguous. If two similar is largely unconscious. One proposal18,19, neuron—the ‘non-classical receptive stimuli are presented in rapid succession, for example, is that humans are not direct- field’5 expresses the relevant context of the the brain blends them together into one ly conscious of their thoughts, but only of ‘classical receptive field’. The visual fields percept. If they are different but in con- sensory representations of them in their of neurons higher in the visual hierarchy tradiction, such as a face and a house, the imagination. At the moment, there is no are larger and respond to more complex brain selects one at a time (as in binocular consensus about this20. features than those that are lower down. rivalry) instead of blending them togeth- The hypothesis of the homunculus An important but neglected aspect of er. In cases where there is not enough is very much out of fashion these days, the firing of a neuron (or a small group of information to lead to an unambiguous but this is, after all, how everyone associated neurons) is its ‘projective interpretation of one’s environment9, the thinks of themselves. It would be sur- field’6. This term describes the perceptual cortical networks ‘fill in’—that is, they prising if this overwhelming illusion and behavioral consequences of stimulat- make their best guess, given the incom- did not reflect in some way the general ing such a neuron in an appropriate man- plete information. Such filling-in is like- organization of the brain. ner (for further discussion of motor and ly to happen in many places in the brain. premotor cortex, see ref. 7). Both the This general principle is an important 2. Zombie modes and consciousness receptive field and the projective field are guide to much of human behavior (as in Many actions in response to sensory dynamic, not merely static, and both can ‘jumping to conclusions’). inputs are rapid, transient, stereotyped be modified by experience. and unconscious14,21. They could be How are feature detectors formed? A Our present framework thought of as cortical reflexes. Con- http://www.nature.com/natureneuroscience broad answer is that neurons do this by Having outlined a few general points sciousness deals more slowly with broad- detecting common and significant corre- about the cortical system, let us now con- er, less stereotyped aspects of the sensory lations in their inputs and by altering their sider specifically the NCC and its atten- inputs (or a reflection of these in imagery) synapses (and perhaps other properties) dant properties. We are mainly interested and takes time to decide on appropriate so that they can more easily respond to in time periods on the order of a few hun- thoughts and responses. It is needed such inputs. In other words, the brain is dred milliseconds, or at the most, several because otherwise, a vast number of dif- very good at detecting apparent causation. seconds, so we can now leave on one side ferent zombie modes would be required. Exactly how it does this is more contro- processes that take more time, such as The conscious system may interfere some- versial. The main mechanism is probably permanently laying down a new memo- what with the concurrent zombie system. Hebbian, but Hebb’s seminal suggestion ry. We have listed the main ingredients of It seems to be a great evolutionary advan- needs to be expanded. our framework under ten headings. We tage to have zombie modes that respond All the above might suggest that cor- have not previously discussed in print rapidly, in a stereotyped manner, togeth- tical action is highly local. Nothing could items 3, 5, 7 and 10, though we have men- er with a slightly slower system that allows

© Group 2003 Nature Publishing be further from the truth. In the cortex, tioned the first three in a book chapter time for thinking and planning more there is continual and extensive interac- that is still in press10. Many of our basic complex behavior. tion, both among neighboring cells and ideas on consciousness, such as the Visual zombie modes in the cortex also very widely, thanks to the many long importance of attention and correlated probably use the dorsal stream in the pari- cortico-cortical and cortico-thalamo-cor- firing, were outlined in our 1990 paper11, etal region14. Some parietal activity, how- tical routes. This is much less true of the and in 1995, we suggested a plausible ever, also affects consciousness by thalamus itself. function for consciousness12 and later producing attentional effects on the ven- The sensory cortex is arranged in a updated our ideas in a 1998 review3. Pre- tral stream, at least under some circum- semi-hierarchical manner. (This is cer- viously we proposed that people are not stances. The conscious mode for vision tainly true of the visual cortex8, but is less directly conscious of the neural activity in depends largely on the early visual areas clear in the front of the brain.) That is, primary visual cortex (V1) and that the (beyond V1) and especially on the ventral most cortical areas do not detect simple (visual) cortex appears hierarchical stream. There are no recorded cases of correlations in the sensory input, but because it contains no strong loops12,13. purely parietal damage which led to a detect correlations among correlations More recently, we have supported the sug- complete loss of conscious vision. being expressed by other cortical areas. gestion14 that in addition to a slower, all- In a zombie mode, the main flow of This remarkable feature of the cortex is purpose conscious mode, the brain has information is probably feed-forward. It seldom emphasized. many ‘zombie modes’15, which are char- could be considered a forward-traveling There has been a great selective advan- acterized by rapid and somewhat stereo- net-wave. A net-wave is a propagating tage in reacting very rapidly, for both typed responses. wave of neural activity, but it is not the predators and prey. For this reason, the same as a wave in a continuous medium. best is the enemy of the good. Typically, it 1. The (unconscious?) homunculus Neural networks in a cortex have both is better to achieve a rapid but occasion- A good way to begin to consider the overall short and long connections, so a net-wave ally imperfect performance instead of a behavior of the cerebral cortex is to imag- may, in some cases, jump over interven- more prolonged one that always produces ine that that the front of the brain is ‘look- ing regions. In the conscious mode, it a perfect result. Another general principle ing at’ the sensory systems, most of which seems likely that the flow is in both direc- may be to use several rough and ready are at the back of the brain. This division of tions (see #5 below) so that it resembles methods in parallel to reach a conclusion, labor does not lead to an infinite regress16. more of a standing net-wave10.

120 nature neuroscience • volume 6 no 2 • february 2003 commentary

3. Coalitions of neurons If there are coalitions in the front of conscious unless there is an essential node The cortex is a very highly and specifically the cortex, they may have a somewhat dif- for it. For consciousness, there may be interconnected neural network. It has many ferent character from those formed at the other necessary conditions, such as pro- types of excitatory and inhibitory interneu- back of the cortex. There may be more jecting to the front of the brain12. rons and acts by forming transient coali- than one coalition that achieves winning A node, all by itself, cannot produce tions of neurons, the members of which status and hence produces conscious consciousness. Even if the neurons in that support one another. ‘Coalitions’ implies experience. The coalitions in the front node were firing appropriately, this would ‘assemblies’—an idea which goes back at may reflect feelings such as happiness and, produce little effect if their output synaps- least to Hebb22—plus competition among perhaps, the feeling of ‘authorship’, which es were inactivated. A node is a node, not them (see also ref. 23). On the basis of is related to free will25. Such feelings may a network. Thus a particular coalition is experimental results in the macaque, some be more diffuse and persist for a longer an active network, consisting of the rele- researchers suggest that selective attention time than coalitions in the back of cortex. vant set of interacting nodes that tem- biases the competition among rivalrous cell The terms ‘affect’ and ‘valuations’ are now porarily sustains itself. assemblies, but they do not explicitly relate being used for what we have traditional- Much useful information can be this idea to consciousness24. ly called ‘feelings’18,19. Our first working obtained from lesions. In humans, the The various neurons in a coalition in assumption (the homunculus) implies damaged area is usually fairly large. It is some sense support one another, either that it is better not to regard the back plus not clear what effects a very small, (possi- directly or indirectly, by increasing the the front as one single coalition, but rather bly bilateral) reversible lesion would have activity of their fellow members. The as two or more separate coalitions that in the macaque, as it is difficult to discov- dynamics of coalitions are not simple. In interact massively, but not in an exactly er exactly what a monkey is conscious of. general, at any moment the winning coali- reciprocal manner. The smallest useful node may be a corti- tion is somewhat sustained, and embodies cal column29 or, perhaps, a portion of a what we are conscious of. 4. Explicit representations cortical column. The feature which that http://www.nature.com/natureneuroscience It may help to make a crude political An explicit representation of a particular node represents is (broadly) its columnar analogy. The primaries and the early aspect of the visual scene implies that a property. This is because although a sin- events in an election would correspond small set of neurons exists that responds gle type of pyramidal cell usually sends its roughly to the preliminary unconscious as a detector for that feature, without fur- information to only one or two cortical processing. The winning coalition associ- ther complex neural processing. A possi- areas, the pyramidal cells in a column pro- ated with an object or event would corre- ble probe, or an operational test, for an ject the columnar property collectively to spond to the winning party, which would explicit representation might be whether a many cortical areas, and can thus strength- remain in power for some time and would single layer of ‘neurons’ could deliver the en any coalition that is forming. attempt to influence and control future correct answer. For example, if a single- events. ‘Attention’ would correspond to the layered neural network were fed the activ- 5. The higher levels first efforts of journalists, pollsters and others ity of retinal neurons, it would not be able For a new visual input, the neural activity to focus on certain issues rather than oth- to recognize a face. Fed from the relevant first travels rapidly and unconsciously up ers, and thus attempt to bias the electorate parts of inferior temporal cortex, howev- the visual hierarchy to a high level, possi-

© Group 2003 Nature Publishing in their favor. Perhaps those large pyrami- er, it could reliably signal ‘face’ or ‘no face’. bly in the front of the brain (this might dal cells in cortical layer 5 that project to There is much evidence from both instantiate a zombie mode). Signals then the superior colliculus and the thalamus humans and monkeys that if there are no start to move backward down the hierar- (both involved in attention) would corre- such explicit neurons, or if they are all lost chy so that the first stages to reach con- spond to electoral polls. These progress by brain damage, then the subject is sciousness are at the higher levels from early, tentative polls to later, rather unable to consciously perceive that aspect (showing the gist of the scene30,31; see also more accurate ones as the election directly. Well-known clinical examples are ref. 32), which send these ‘conscious’ sig- approaches. It is unlikely that all this hap- achromatopsia (loss of color perception), nals again to prefrontal cortex, followed pens in the brain in a fixed time sequence. prosopagnosia (loss of face recognition) by corresponding activity at successive The brain may resemble more the British or akinetopsia (loss of motion percep- lower levels (to provide the visual details). system, in which the time between one tion). In all cases, one or a few attributes This is an oversimplified description. election and the next can be irregular. Such of conscious experience have been lost, There are also many horizontal connec- an analogy should not be pressed too far. while most other aspects remain intact. In tions in the hierarchy. Like all analogies, it should be regarded as the macaque, a small, irreversible lesion How far up the hierarchy the initial a possible source of ideas, which of course of the motion area MT/V5 leads to a tem- net-wave travels may depend upon will have to be confirmed by experiment. poral deficit in motion perception that whether attention is diffused or focused Coalitions can vary both in size and in recovers within days. Larger lesions cause at some particular level. character. For example, a coalition pro- a more permanent loss26. duced by visual imagination (with one’s Note that an explicit representation is 6. Driving and modulating connections eyes closed) may be less widespread than a a necessary but not sufficient condition In considering the physiology of coali- coalition produced by a vivid and sus- for the NCC to occur. tions, it is especially important to under- tained visual input from the environment. One can describe this in terms of stand the nature of neural connections. In particular, representations of imagined ‘essential nodes’27,28. The cortical neural The classification of neuronal inputs is visions may fail to reach down to the networks (at least for perception) can be still in a primitive state. It is a mistake to lower echelons of the visual hierarchy. thought of as having nodes. Each node is think of all excitatory neural connections Coalitions in dreams may be somewhat needed to express one aspect of one per- as the same type. First, connections to a different from waking ones. cept or another. An aspect cannot become cortical neuron fall roughly into two

nature neuroscience • volume 6 no 2 • february 2003 121 commentary

Fig. 1. The snapshot hypothesis proposes that the conscious perception of motion is not rep- resented by the change of firing rate of the rel- evant neurons, but by the (near) constant firing of certain neurons that represent the motion. The figure is an analogy. It shows how a static picture can suggest motion.

should be the underlying NCC (for exam- ple, firing at either a low or high level). This activity may also show hysteresis; that is, it may stay there longer than its sup- port warrants. What could be special about this activ- ity that reaches above the consciousness threshold? It might be some particular way of firing, such as a sustained high rate, some sort of synchronized firing or firing in bursts. Or it might be the firing of special types of neurons, such as those pyramidal cells that project to the front of the brain39 (Fig. 2). This may seem http://www.nature.com/natureneuroscience unlikely but, if true, would greatly sim- plify the problem, both experimentally broad classes: driving and modulating the mechanisms for position-estimation and theoretically. inputs13. For cortical pyramidal cells, dri- and for detecting motion are largely sepa- What is required to maintain this spe- ving inputs may largely contact the basal rate. (Recall the motion after-effect.) Thus, cial activity above the threshold? It might dendrites, whereas modulatory inputs a particular motion can be represented by be something special about the internal include back-projections (largely to the a constant rate of firing of the relevant neu- dynamics of the neuron, perhaps involv- apical dendrites) or diffuse projections, rons. It is not surprising, then, that if the ing the accumulation of chemicals such as especially those from the intralaminar eyes do not move in smooth pursuit, the Ca2+, either in the neuron itself or in one nuclei of the thalamus. brain is very poor at recognizing accelera- of its associated inhibitory neurons. It This classification may be too simple. tion even though it is good at distinguishing might also be re-entrant circuits in the In some cases, a single type of input to a movements36. (As perceived motion is con- cortical system40. Positive feedback loops neuron may be driving, such as the input stant during a snapshot, it can only change could, by iteratively exciting the neuron,

© Group 2003 Nature Publishing from the lateral geniculate nucleus (LGN) between snapshots, which suggests that push its activity increasingly upward so to V1. In other cases, several types of ‘dri- there is little or no explicit representation that the activity not only reaches above ving’ inputs may be needed to make that for such a change. Hence, acceleration is some critical threshold, but is maintained neuron fire at a significant rate. It is pos- not easily seen). All the other conscious there for some time. sible that the connections from the back attributes of the percept at that moment are There are a few complications: the of the brain to the front are largely driving, part of the snapshot. threshold level may depend on the rate whereas the reverse pathways are largely The durations of successive snapshots of approach to the threshold or on how modulatory, but this is not experimental- are unlikely to be constant. (They are dif- long the input is sustained, or both of ly established. This general pattern would ficult to measure directly.) Moreover, the these. At the beginning of each new not hold for cross-modal connections. time of a snapshot for shape, say, may not snapshot, there may be some hold-over The above tentative classification is exactly coincide with that for, say, color. from the previous one. largely for excitatory cells. Strong loops It is possible that these durations may be Put another way, these are partial of driving connections probably do not related to the α rhythm37 or even the δ descriptions of conscious coalitions form- occur under normal conditions13. It rhythm. The theory of the ‘perceptual ing, growing or disappearing. seems likely that cortical layer 5 cells moment’ was suggested as early as 1955, There is no evidence for a regular which project to the thalamus are dri- when it was not known how motion is clock in the brain on the second or frac- ving, and that those from layer 6 are represented in the brain38, but in recent tion-of-a-second time scale. The dura- modulating13,33. years, it has been largely forgotten. tion of any snapshot (or fragment of a To reach consciousness, some (unspec- snapshot) is likely to vary somewhat, 7. Snapshots ified) neural activity for that feature has depending on such factors as sudden on- Has a successful coalition any special char- to cross a threshold. It is unlikely to do so signals, off-signals, competition, habit- acteristics? unless it is, or is becoming, the member uation and so on. Several psychological We propose that conscious awareness of a successful coalition. It is held above effects have been described41, such as an (for vision) is a series of static snapshots, threshold, possibly as a constant value of illusion under constant illumination with motion ‘painted’ on them34,35 (Fig. 1). the activity, for a certain time (the time of similar to the wagon-wheel effect, which By this we mean that perception occurs in that snapshot). As specific attributes of hint that there are some irregular batch- discrete epochs. It is well established that conscious perception are all-or-none, so like effects in vision.

122 nature neuroscience • volume 6 no 2 • february 2003 commentary

Fig. 2. The dendritic arborization of the dif- ferent types of neurons in the inferior tempo- ral gyrus of the macaque monkey that project to the prefrontal cortex near the principal sul- cus (top, shaded gray). The neurons were recovered from four slices (A–D) as indicated. There are other types of neurons in this area that project to other places. Note that only one type of cell has apical dendrites that reach to layer 1. Drawing from de Lima, A.D., Voigt, T. and Morrison, J.H. (1990)39, reprinted with permission of Wiley-Liss, Inc., a subsidiary of John Wiley & Sons, Inc.

8. Attention and binding Attention can usefully be divided into two forms: either rapid, saliency-driven and bottom-up or slower, volitionally con- trolled and top-down. Each form of atten- tion can also be more diffuse or more focused. Attention probably acts by bias- ing the competition among rival coali- tions, especially during their formation24. http://www.nature.com/natureneuroscience Bottom-up attention may often start from certain layer 5 neurons that project to parts of the thalamus and the superior collicu- lus. Top-down attention from the front of the brain may go by somewhat diffuse back-projections to apical dendrites in lay- ers I, II and III, and perhaps also via the attentive versus attentive’ processing 9. Styles of firing intralaminar nuclei of the thalamus describe two independent dimensions Synchronized firing (including various (because these have inputs from the front rather than variations along a single oscillations) may increase the effectiveness of the brain). Although such projections dimension (Reddy, L., VanRullen, R. & of a neuron, while not necessarily altering are widespread, it does not follow that they C.K., Vision Sci. Soc. 2nd Annu. Mtg., abstr. its average firing rate43. The extent and are nonspecific. To attend to ‘red’ involves 443, 2002). Their results can all be significance of synchronized firing in cor- specific connections to many places in cor- expressed in terms of the relevant neural tex remains controversial44. Computations 45 © Group 2003 Nature Publishing tex. An attractive hypothesis is that the networks. Several objects/events can be show that this effectiveness is likely to thalamus is largely the organ of attention. handled simultaneously—more than one depend on how the correlated input influ- The reticular nucleus of the thalamus may object/event can be attended to at the ences the excitatory and inhibitory neu- help select among attentional signals on a same time—if there is no significant over- rons in the recipient region to which the broad scale. Although attention can pro- lap in any cortical neural network. That synchronized neurons project. duce consciousness of a particular object is, if two or more objects/events do not We no longer think11 that synchro- or event by biasing competition among have any very active essential nodes in nized firing, such as the so-called 40 Hz coalitions, activities associated with non- common, they can be consciously per- oscillations, is a sufficient condition for attended objects are quite transient, giv- ceived. Under such conditions, several the NCC. One likely purpose of synchro- ing rise to fleeting consciousness (such as largely separate (sensory) coalitions may nized firing is to assist a nascent coalition the proto-objects suggested in ref. 42). exist. If there is necessarily such an over- in its competition with other (nascent) What is binding? (For reviews that lap, then (top-down) attention is needed coalitions. If the visual input is simple, address the binding problem, see Neuron to select one of them by biasing the com- such as a single bar in an otherwise empty 24, 1999.) This is the term used for the petition among them. field, there might not be any significant process that brings together rather differ- This approach largely solves the clas- competition, and synchronized firing may ent aspects of an object/event, such as its sical binding problem, which was main- not occur. Similarly, such firing may not shape, color, movement and so on. Bind- ly concerned with how two different be needed once a successful coalition has ing can be of several types11. If it has been objects/events could be ‘bound’ simul- reached consciousness, when it may be laid down epigenetically, or has been learnt taneously. On this view, the ‘binding’ of able to maintain itself without the assis- by experience, it is already embodied in the features of a single object/event is tance of synchrony, at least for a time46. one or more essential nodes so that no spe- simply the membership in a particular An analogy: after obtaining tenure, you cial binding mechanism is needed. If the coalition. There is no single cortical area can relax a little. binding required is (relatively) novel, then where it all comes together. The effects At any essential node, the earliest spike in some way the activities of separate essen- of that coalition are widely distributed to arrive may sometimes have the advan- tial nodes must be made to act together. over both the back and the front of the tage over spikes arriving shortly there- Recent psychophysics suggests that brain. Thus, effectively, they bind by after47. In other words, the exact timing ‘parallel versus serial’ search and ‘pre- interacting in a diffuse manner. of a spike may influence the competition.

nature neuroscience • volume 6 no 2 • february 2003 123 commentary

10. Penumbra and meaning pathways, but there is little reference to The idea of snapshots is a guess at the Consider a small set of neurons that fires consciousness. dynamic properties of the parts of a suc- to, say, some aspect of a face. The experi- There are two fairly recent books cessful coalition, as coalitions are not sta- menter can discover what visual features expressing ideas that overlap considerably tic, but constantly changing. The interest such a set of neurons, but how with ours. The first of these is by Edelman penumbra, on the other hand, is all the does the brain know what that firing rep- and Tononi23. Their “dynamic core” is neural activity produced by the current resents? This is the problem of ‘meaning’ very similar to our coalitions. They also NCC, yet not strictly part of it. in its broadest sense. divide consciousness into primary con- We also speculate that the actual NCC The NCC at any one time will only sciousness (which is what we are mainly may be expressed by only a small set of directly involve a fraction of all pyrami- concerned with) and higher-order con- neurons, in particular those that project dal cells, but this firing will influence sciousness (which we have, for the from the back of cortex to those parts of many neurons that are not part of the moment, put on one side). They state the front of cortex that are not purely NCC. These we call the ‘penumbra’. The strongly, however, that they don’t think motor and that receive feedback from penumbra consists of both synaptic effects there is a special subset of neurons that there. However, there is much neural and also firing rates. The penumbra is not alone expresses the NCC. activity leading up to and supporting the the result of just the sum of the effects of The second book54 is by Bachmann, NCC, so it is important to study this as each essential node separately, but the who for many years has used the term well as the NCC proper. Moreover, dis- effects of that NCC as a whole. This “microgenesis” to mean what is happen- covering the temporal sequence of such penumbra includes past associations of ing in the 100–200 ms leading up to con- activities (A precedes B) will help us to NCC neurons, the expected consequences sciousness, which is our own main concern move from correlation to causation. of the NCC, movements (or at least pos- as well. He considers carefully the relevant The explanation here of this inter- sible plans for movement) associated with psychological phenomena, such as the dif- locking set of ideas is necessarily abbrevi- NCC neurons, and so on. For example, a ferent types of masking, but has fewer ideas ated. We have outlined some of this in http://www.nature.com/natureneuroscience hammer represented in the NCC is likely about the detailed behavior of neurons. more detail elsewhere10. A more extend- to influence plans for hammering. A framework somewhat similar to ed account, together with descriptions of The penumbra, by definition, is not ours has recently been described by the key experimental data, will be pub- itself conscious, although part of it may Dehaene and Naccache55, though they do lished in a forthcoming book by C.K.56. become part of the NCC as the NCC not elaborate on the snapshot hypothesis The above framework is a guide to con- shifts. Some of the penumbra neurons or on the neural basis of essential nodes. structing more detailed hypotheses so that may project back to parts of the NCC, or they can be tested against already-existing its support, and thus help to support the General remarks experimental evidence and, above all, to NCC. The penumbra neurons may be the Almost all of the above ideas have been suggest new experiments. The aim is to site of unconscious priming48. mentioned previously, either by us or by couch all such explanations in terms of the others. We have given references to some behavior of identified neurons and in the Related ideas of these. We believe that the framework dynamics of very large neural assemblies. In the last 15–20 years, there has been an we have proposed knits all these ideas

© Group 2003 Nature Publishing immense flood of books and papers together, so that for the first time we have Future experiments about consciousness. For an extensive a coherent scheme for the NCC in philo- These fall under several headings. Much bibliography, see T. Metzinger’s home- sophical, psychological and neural terms. further experimental work on small page: www.philosophie.uni-mainz.de/ What ties all these suggestions togeth- groups of neurons is required for cases in metzinger. Many people have said that er is the idea of competing coalitions. The which the percept differs significantly from consciousness is ‘global’ or has a ‘unity’ illusion of a homunculus inside the head the sensory input, such as in binocular (whatever that is), but have provided few looking at the sensory activities of the rivalry57 and in the many visual illusions58. details about such unity49. For many brain suggests that the coalition(s) at the Knowledge of the detailed neu- years, Baars50 has argued that conscious- back are in some way distinct from the roanatomy of the cerebral cortex needs to ness must be widely distributed. coalition(s) at the front. The two types of be greatly expanded, in particular to char- We are not receptive to physicists try- coalitions interact extensively, but not acterize the many different types of pyra- ing to apply exotic physics to the brain, exactly reciprocally. midal cells in any particular cortical area. about which they seem to know very lit- Zombie modes show that not all What do they look like, where do they pro- tle, and even less about consciousness. motor outputs from the cortex are carried ject and, eventually, does each have a set Dennett51,52 has written at length out consciously. Consciousness depends of characteristic genetic markers? Are there about his ideas of “multiple drafts”, often on certain coalitions that rest on the prop- types of pyramidal cells that do not occur using elaborate analogies, but he seems not erties of very elaborate neural networks. in all cortical areas? When recording spik- to believe in the existence of consciousness We consider attention to consist of mech- ing activity from a neuron, it would be in the same way as we do. Dennett does anisms that bias the competition among very desirable to know what type it is, and consider a limited number of psychologi- these nascent coalitions. to where this particular cell projects. cal experiments, but neurons, he tells us, We suggest that each node in these The anatomical methods to character- “are not my department” (pers. comm.). networks has a characteristic behavior. ize types of neurons for the macaque Grossberg53 has written for many years We speculate that the smallest group of monkey have been available for some about his “adaptive resonance theory” neurons to be worth considering as a years39 (Fig. 2). In the last two decades, (ART). This he has developed using very node is a cortical column, with its own very little work has been carried out on simple neural models. ART involves inter- characteristic behavior (its receptive and cell types and their connectivity, mainly actions between the forward and the back projective fields). for lack of funds, since such work is not

124 nature neuroscience • volume 6 no 2 • february 2003 commentary

‘hypothesis-driven’. However, because Acknowledgments homunculus. in The Neural Correlates of structure is often a clue to function, We thank P.S. Churchland, D. Eagleman, Consciousness (ed. Metzinger, T.) 103–110 (MIT Press, Cambridge, Massachusetts, detailed neuroanatomy is essential back- G. Kreiman, N. Logothetis, G. Mitchison, T. Poggio, 2000). ground knowledge (of the type the V. Ramachandran, A. Revonsuo and J. Reynolds for 18. Jackendoff, R. Consciousness and the human genome project provides for mol- thoughtful comments, O. Crick for the drawing and Computational Mind (MIT Press, Cambridge, ecular biology). the J.W. Kieckhefer Foundation, the W.M. Keck Massachusetts, 1987). On occasion, multi-unit electrodes are Foundation Fund for Discovery in Basic Medical 19. Jackendoff, R. How language helps us think. chronically implanted into alert patients Research at Caltech, the National Institutes of Pragmat. Cogn. 4, 1–34 (1996). (for example, to localize seizure onset Health, the National Institute of Mental Health 20. Crick, F. & Koch, C. The unconscious areas in epileptic patients). With their and the National Science Foundation for homunculus. Neuro-psychoanalysis 2, 3–11 and subsequent pages for multi-authored consent, this can provide sparse but criti- financial support. discussion (2000). cal data about the behavior of neurons 21. Rossetti, Y. Implicit short-lived motor during conscious perception or imagery59. RECEIVED 12 SEPTEMBER; ACCEPTED representations of space in brain damaged and It would be very valuable if cortical tissue 9 DECEMBER 2002 healthy subjects. Conscious. Cogn. 7, 520–558 could be stimulated appropriately with (1998). 22. Hebb, D. The Organization of Behavior: a such electrodes to generate specific per- 1. Chalmers, D.J. The Conscious Mind: in Search 60 Neuropsychological Theory (John Wiley, New cepts, thoughts or actions . of a Fundamental Theory (Oxford Univ. Press, York, 1949). New York, 1995). To study the dynamics of coalitions 23. Edelman, G.M. & Tononi, G. A Universe of requires simultaneous recordings on a fast 2. Shear, J. Explaining Consciousness: the Hard Consciousness (Basic Books, New York, 2000). Problem (MIT Press, Cambridge, time scale, from small groups of neurons Massachusetts, 1997). 24. Desimone, R. & Duncan, J. Neural in many places in the brain. This might mechanisms of selective visual attention. 3. Crick, F.C. & Koch, C. Consciousness and Annu. Rev. Neurosci. 18, 193–222 (1995). be done on a primate with a relatively neuroscience. Cereb. Cortex 8, 97–107 (1998). smooth cortex, such as the owl monkey. 25. Wegner, D. The Illusion of Conscious Will (MIT http://www.nature.com/natureneuroscience 4. Crick, F.C. & Koch, C. Why neuroscience may Press, Cambridge, Massachusetts, 2002). It would require recording simultaneous- be able to explain consciousness. Sci. Am. 273, ly from probably a thousand or more elec- 84–85 (1995). 26. Newsome, W.T. & Pare, E.B. A selective impairment of motion perception following trodes, spaced about 1 mm apart, each 5. Allman, J., Miezin, F. & McGuinness, E. lesions of the middle temporal visual area capable of picking up spikes from single Stimulus specific responses from beyond the (MT). J. Neurosci. 8, 2201–2211 (1988). classical receptive field: neurophysiological cells as well as the local field potential. mechanisms for local–global comparisons in 27. Zeki, S.M. Parallel processing, asynchronous The immense amount of data this visual neurons. Annu. Rev. Neurosci. 8, perception, and a distributed system of 407–430 (1985). consciousness in vision. Neuroscientist 4, would produce could be displayed visually 365–372 (1998). on a two-dimensional map of the cortical 6. Lehky, S.R. & Sejnowski, T.J. Network model 28. Zeki, S. & Bartels, A. Toward a theory of visual surface, either speeded up or slowed down, of shape-from-shading: neural function arises from both receptive and projective fields. consciousness. Conscious. Cogn. 8, 225–259 so that the eye could grasp the nature of Nature 333, 452–454 (1988). (1999). the traveling net-waves as a preliminary to 7. Graziano, M.S., Taylor, C.S. & Moore, T. 29. Mountcastle, V.B. Perceptual Neuroscience a more detailed study of them. The tech- Complex movements evoked by (Harvard Univ. Press, Cambridge, nical difficulties in recording from so many microstimulation of precentral cortex. Neuron Massachusetts, 1998). 34, 841–851 (2002). 30. Biederman, I. Perceiving real-world scenes. © Group 2003 Nature Publishing neurons at once in an alert animal are for- midable but not insuperable. To develop 8. Felleman, D.J. & Van Essen, D.C. Distributed Science 177, 77–80 (1972). hierarchical processing in the primate cerebral the method, one could start with a smaller 31. Wolfe, J.M. & Bennett, S.C. Preattentive object cortex. Cereb. Cortex 1, 1–47 (1991). files: shapeless bundles of basic features. number of electrodes, more widely spaced, 9. Poggio, T., Torre, V. & Koch, C. Vision Res. 37, 25–43 (1997). and work on one side of the brain of an Computational vision and regularization 32. Hochstein, S. & Ahissar, M. View from the animal with the corpus callosum cut. theory. Nature 317, 314–319 (1985). top: hierarchies and reverse hierarchies in the 10. Crick, F.C. & Koch, C. What are the neural visual system. Neuron 36, 791–804 (2002). Omissions correlates of consciousness? in Problems in 33. Sherman, S.M. & Guillery, R. Exploring the Systems Neuroscience (eds. van Hemmen, L. & Thalamus (Academic Press, San Diego, 2001). Some major omissions from this discus- Sejnowski, T.J.) (Oxford Univ. Press, New sion are (i) a more detailed consideration York, 2003). 34. Zihl, J., Von Cramon, D. & Mai, N. Selective of the role of the thalamus (and especial- disturbance of movement vision after bilateral 11. Crick, F.C. & Koch, C. Towards a brain damage. Brain 106, 313–340 (1983). ly of the intralaminar nuclei), (ii) the neurobiological theory of consciousness. Sem. Neurosci. 2, 263–275 (1990). 35. Hess, R.H., Baker, C.L. Jr. & Zihl, J. The actions of the basal ganglia and of the dif- ‘motion-blind’ patient: low-level spatial and fuse inputs from the brain stem and (iii) 12. Crick, F. & Koch, C. Are we aware of neural temporal filters. J. Neurosci. 9, 1628–1640 activity in primary visual cortex? Nature 375, (1989). a more detailed scheme for the overall 121–123 (1995). 36. Simpson, W.A. Temporal summation of visual organization of the front of the cortex and 13. Crick, F. & Koch, C. Constraints on cortical motion. Vision Res. 34, 2547–2559 (1994). for motor outputs. For example, is there and thalamic projections: the no-strong-loops some sort of hierarchy in the front? Are hypothesis. Nature 391, 245–250 (1998). 37. Varela, F.J., Toro, A., John, E.R. & Schwartz, E.L. Perceptual framing and cortical alpha 14. Milner, D.A. & Goodale, M.A. The Visual there separate streams of information, as rhythm. Neuropsychologia 19, 675–686 (1981). there are in the visual system? Do the neu- Brain in Action (Oxford Univ. Press, Oxford, UK, 1995). 38. Stroud, J.M. The fine structure of rons in the front of cortex show columnar psychological time. in Information Theory in 15. Koch, C. & Crick, F.C. On the zombie within. behavior and if so, what for? Psychology (ed. Quasten, H.) 174–207 (Free Nature 411, 893 (2001). On the other hand, our concentration Press, Glencoe, Illinois, 1955). 16. Attneave, F. In defense of homunculi. in 39. de Lima, A.D., Voigt, T. & Morrison, J.H. on the NCC, and the postponement of the Sensory Communication (ed. Rosenblith, Morphology of the cells within the inferior so-called hard problem of qualia, is part W.A.) 777–782 (MIT Press and John Wiley, temporal gyrus that project to the prefrontal New York, 1961). of our strategic approach to the overall cortex in the macaque monkey. J. Comp. problem of consciousness. 17. Crick, F.C. & Koch, C. The unconscious Neurol. 296, 159–172 (1990).

nature neuroscience • volume 6 no 2 • february 2003 125 commentary

40. Edelman, G.M. The Remembered Present: a & Lehto, M. The neural generation of a 54. Bachmann, T. Microgenetic Approach to the Biological Theory of Consciousness (Basic unified illusion in human vision. Neuroreport Conscious Mind (John Benjamins, Amsterdam Books, New York, 1989). 8, 3867–3870 (1997). and Philadelphia, 2000). 41. Purves, D., Paydartar, J.A. & Andrews, T.J. The 47. Van Rullen, R. & Thorpe, S.J. The time course 55. Dehaene, S. & Naccache, L. Towards a wagon wheel illusion in movies and reality. of visual processing: from early perception to cognitive neuroscience of consciousness: basic Proc. Natl. Acad. Sci. USA 93, 3693–3697 decision-making. J. Cogn. Neurosci. 13, evidence and a workspace framework. (1996). 454–461 (2001). Cognition 79, 1–37 (2001). 42. Rensink, R.A. Seeing, sensing and 48. Schacter, D.L. Priming and multiple memory 56. Koch, C. The Quest for Consciousness: a scrutinizing. Vision Res. 40, 1469–1487 systems: perceptual mechanisms of implicit Neurobiological Approach (Roberts and (2000). memory. J. Cogn. Neurosci. 4, 255–256 (1992). Company Publishers, California, in press). 43. Singer, W. & Gray, C.M. Visual feature 49. Bayne, T. & Chalmers, D.J. What is the unity of 57. Leopold, D.A. & Logothetis, N.K. Multistable integration and the temporal correlation consciousness? in The Unity of Consciousness: phenomena: changing views in perception. hypothesis. Annu. Rev. Neurosci. 18, 555–586 Binding, Integration, Dissociation (ed. Trends Cogn. Sci. 3, 254–264 (1999). (1995). Cleeremans, A.) (Oxford Univ. Press, Oxford, UK, in press). 58. Eagleman, D.M. Visual illusions and 44. Shadlen, M.N. & Movshon, J.A. Synchrony neurobiology. Nat. Rev. Neurosci. 2, 920–926 unbound: a critical evaluation of the temporal 50. Baars, B.J. In the Theater of Consciousness (2001). (Oxford Univ. Press, New York, 1997). binding hypothesis. Neuron 24, 67–77, 59. Kreiman, G., Fried, I. & Koch, C. Single- 111–125 (1999). 51. Dennett, D.C. Consciousness Explained (Little, neuron correlates of subjective vision in the 45. Salinas, E. & Sejnowski, T.J. Correlated Brown & Co., Boston, Massachusetts, 1991). human medial temporal lobe. Proc. Natl. neuronal activity and the flow of neural 52. Dennett, D. Are we explaining consciousness Acad. Sci. USA 99, 8378–8383 (2002). information. Nat. Rev. Neurosci. 2, 539–550 yet? Cognition 79, 221–237 (2001). 60. Fried, I., Wilson, C.L., MacDonald, K.A. & (2001). 53. Grossberg, S. The attentive brain. Am. Sci. 83, Behnke, E.J. Electric current stimulates 46. Revonsuo, A., Wilenius-Emet, M., Kuusela, J. 438–449 (1995). laughter. Nature 391, 650 (1998). http://www.nature.com/natureneuroscience © Group 2003 Nature Publishing

126 nature neuroscience • volume 6 no 2 • february 2003