Time Course of Brain Activity during Change Blindness and Change Awareness: Performance is Predicted by Neural Events before Change Onset

Gilles Pourtois1,2, Michael De Pretto1,3, Claude-Alain Hauert3, Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/18/12/2108/1756025/jocn.2006.18.12.2108.pdf by guest on 18 May 2021 and Patrik Vuilleumier1,2,3

Abstract & People often remain ‘‘blind’’ to visual changes occurring dur- potential) after the first display, before the change itself. The ing a brief interruption of the display. The processing stages amplitude of the N170 and P3 responses after the second visual responsible for such failure remain unresolved. We used event- display were also modulated by awareness of the face change. related potentials to determine the time course of brain activity Furthermore, a unique topography of event-related potential during conscious change detection versus change blindness. activity was observed during correct change and correct no- Participants saw two successive visual displays, each with two change reports, but not during blindness, with a recurrent time faces, and reported whether one of the faces changed between course in the stimulus sequence and simultaneous sources in the first and second displays. Relative to blindness, change the parietal and temporo-occipital cortex. These results indicate detection was associated with a distinct pattern of neural activity that awareness of visual changes may depend on the attentional at several successive processing stages, including an enhanced state subserved by coordinated neural activity in a distributed occipital P1 response and a sustained frontal activity (CNV-like network, before the onset of the change itself. &

INTRODUCTION Several possible mechanisms may be responsible for The ability to detect changes in the environment is a change blindness. A key aspect is thought to involve a crucial aspect of perception and is undeniably important limited processing capacity of the brain, resulting in the in our everyday life. However, change detection is not a inability to retain an exhaustive representation of sen- simple and straightforward process as it might first sory information over time and/or across brief interrup- appear. Behavioral studies show that large visual tions in inputs (e.g., Simons & Rensink, 2005; Noe¨, changes may occur directly in our field of view but fail Pessoa, & Thompson, 2000). However, different mech- to enter awareness, even when such changes are signif- anisms might potentially operate at distinct stages of icant, repeatedly made, and anticipated (Rensink, 2002). processing and induce change blindness in different This phenomenon of ‘‘change blindness’’ is not caused ways. According to Simons (2000), an inability to detect by intrinsic properties of visual stimuli making changes changes between two successive stimuli might arise difficult to notice, because the same changes are readily because of a failure in processing the first stimulus, detected once attention is drawn to them (Simons & the second stimulus, both, or some links between these Rensink, 2005). Change blindness may arise for simple two stages. For instance, a new stimulus might replace geometric shapes or colors (Simons, 2000), complex and erase the trace of the previous stimulus, precluding scenes (Grimes, 1996), or familiar objects such as faces the detection of any change unless attention is directed (Ro, Russell, & Lavie, 2001). Such apparent blindness for to its location at the time when it occurs (Shapiro, otherwise salient visual stimuli can serve as the flip side Arnell, & Raymond, 1997). Alternatively, only the first of processes involved in conscious change detection and stimulus might be fully retained, and no information thus provide important insights on the functional con- might be extracted from a second stimulus in the ab- straints of perceptual awareness. sence of indirect cues signaling the possibility of change (e.g., apparent motion; see Simons, 2000). Another hypothesis is that sensory information from each stim- 1Neurology & Imaging of Cognition Laboratory, Department ulus is fully processed, but none of them is stored of Neuroscience & Neurology Clinic, 2Swiss Center for Affective beyond its presentation (Noe¨ et al., 2000). Still, other Sciences, 3University of Geneva possibilities are that each successive stimulus can be

D 2006 Massachusetts Institute of Technology Journal of Cognitive Neuroscience 18:12, pp. 2108–2129 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2006.18.12.2108 by guest on 26 September 2021 perceptually processed and stored while only their com- Lamme, 2003). Thus, it is possible that some critical parison fails (Mitroff, Simons, & Levin, 2004; Scott- differences in neural activity during change awareness Brown, Baker, & Orbach, 2000) because of some insuf- relative to change blindness might already arise before ficiency in internal representation or short-term memory the activity elicited by the visual change itself. Because of capacity or that information from two successive stimuli this slow temporal resolution, previous fMRI studies is merged into a single representation, perhaps because could not answer the question of whether change conscious scene perception builds slowly over time blindness (e.g., in a flicker paradigm) primarily results (Simons, 2000). Behavioral studies have clearly shown from a nonexhaustive processing of the initial stimulus,

that focused attention is necessary to detect changes, of the subsequent changing stimulus, of both and/or Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/18/12/2108/1756025/jocn.2006.18.12.2108.pdf by guest on 18 May 2021 because attended and task-relevant objects in a scene from a deficiency in the link between stimuli allowing are more likely to be encoded and compared (Rensink, their comparison (see Simons, 2000). O’Regan, & Clark, 1997; Shapiro et al., 1997), but atten- Here we used ERPs in a simplified flicker paradigm tion at the time of change onset might not be sufficient adapted from Beck et al. (2001), allowing to track brain (Simons & Rensink, 2005) because changes to attended activity on a millisecond basis evoked by each successive objects sometimes go unnoticed when these changes stimulus in conditions of either change blindness or are unexpected (Triesch, Ballard, Hayhoe, & Sullivan, change awareness. We used faces as visual stimuli be- 2003; Williams & Simons, 2000). A useful approach to cause they evoke a specific component in ERPs (N170; investigate some of these different possible mechanisms Bentin, Allison, Puce, Perez, & McCarthy, 1996) in addi- of change blindness, each potentially involving different tion to other well-documented exogenous responses. stages of processing during scene perception, can be Our main goal was to determine whether any differen- provided by event-related potentials (ERPs). By indexing tial activation associated with change blindness versus the time course of brain activity both before and after change awareness might arise during responses to the change onset, ERPs might help determine whether changing stimulus and/or already before the change itself attention is critically required at the time of stimulus (see Vogel & Machizawa, 2004, for similar approach in a change only, or whether other processing stages are working-memory task). We could also test whether any differentially modulated. difference between change blindness and awareness Few studies have investigated the neural correlates of might concern the early stages of visual processing (e.g., change blindness and change awareness, despite much P1–N1), stimulus categorization stages (e.g., N170), and/ recent interest in the mechanisms of conscious vision or later comparison and decision stages (e.g., P3). (Koch, 2004) and important theoretical implications Only a few previous studies have used electroenceph- for understanding visual perception (Block, 2005). In a alogram (EEG) to investigate change blindness, but all pioneer study (Beck, Rees, Frith, & Lavie, 2001), func- concentrated on responses consecutive to the change, tional magnetic resonance imaging (fMRI) was used using either flicker paradigms with successive geo- during a simplified ‘‘flicker’’ paradigm, in which a metric shapes (Koivisto & Revonsuo, 2003; Niedeggen, change between two successive visual displays could in- Wichmann, & Stoerig, 2001) or a cyclic presentation of volve either a face or a house (or none). Correct detec- alternating stimuli (Fernandez-Duque, Grossi, Thornton, tion of changes (as compared to change blindness) was & Neville, 2003). Comparing awareness versus blindness associated with enhanced activation of category-specific for visual changes in these studies has shown differen- regions in the ventral temporal cortex (e.g., in fusiform tial activity in a posterior negative wave approximately gyrus for face changes), together with enhanced activa- 200 msec postchange (Koivisto & Revonsuo, 2003), as tion of more dorsal regions in the frontal and parietal well as later increases in the amodal P3 component cortex (see also Pessoa & Ungerleider, 2004). However, (Niedeggen et al., 2001), typically associated with con- fMRI relies on slow hemodynamic responses and cannot scious decision stages (Polich & Kok, 1995). Earlier reveal the precise temporal dynamics of activity associ- effects over the frontal and occipital regions were also ated with blindness or awareness of changes over the found to be approximately 100 msec after the onset of a successive stages of processing for each visual event. reported change during a cyclic alternation of stimuli In the study of Beck et al. (2001), the critical enhance- (Fernandez-Duque et al., 2003). These results suggest a ment of visual and fronto-parietal areas could potentially complex sequence of processing stages during con- follow, co-occur, or even precede the actual sensory scious change detection but do not address the question inputs into the visual system. Indeed, other neuroim- of whether any critical difference in brain activity before aging data (Corbetta, Kincade, Ollinger, McAvoy, & change onset might influence the ability to detect such Shulman, 2000; Kastner, Pinsk, De Weerd, Desimone, change at a later point in time, as specifically investi- & Ungerleider, 1999) suggest an important role of gated here. Furthermore, previous ERP studies of preparatory states and shifts of baseline activity during change blindness concentrated on waveform analysis selective attention, preceding any stimulus-driven re- (i.e., amplitude and latency of selected components; sponses (see Tallon-Baudry, Bertrand, Henaff, Isnard, see Picton et al., 2000), whereas we complemented this & Fischer, 2005; Super, van der Togt, Spekreijse, & approach with topographic and source localization

Pourtois et al. 2109 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2006.18.12.2108 by guest on 26 September 2021 methods that provide additional insights on the spatio- deficient initial encoding) of the first stimulus (S1), temporal dynamics of brain activity (Michel, Seeck, & conditions may mainly differ during the exogenous Landis, 1999; Lehmann & Skrandies, 1980). visual (e.g., P1–N1) and categorical (N170) responses In the present study, participants performed a change to S1 (i.e., before change itself ), whereas only ERPs to S2 detection task adapted from Beck et al. (2001), where might differ between conditions if processing of the each trial consisted of a sequence of two successive second stimulus is more determinant. Alternatively, early displays, each with a pair of two different faces pre- visual responses to both the first and second stimuli sented in the peripheral visual field (Figure 1). The first might be unaffected if change blindness results from

pair of faces (stimulus S1) was shown for 150 msec, one some failure at postperceptual comparison stages or in Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/18/12/2108/1756025/jocn.2006.18.12.2108.pdf by guest on 18 May 2021 in each hemifield, followed by an empty screen for working memory. In addition, topographic analyses and 150 msec, and then replaced by a second pair of faces source localization enabled us to determine whether (stimulus S2) for another 150 msec. Critically, in the awareness of changes might correlate with a unique second pair, one of the two faces could be different than configuration of brain generators at a particular latency in the first pair, either on the right or left side (a third of or rather with a prolongation or a higher strength of one trials each), or the two faces could be the same as in the (or more) configuration(s) similar to those present first pair (a third of trials). Participants reported whether during change blindness. they had detected a changing face or not (without any speed constraints). To ensure an adequate central fixa- tion, participants also had to monitor whether numbers METHODS or letters were presented at the center of the screen (see also Beck et al., 2001). Crucially, we could compare trials Participants with similar stimuli but two different perceptual out- Participants in the main EEG experiment were 12 right- comes, that is, when a face identity change was either handed naive students (6 women; mean age = 24 years, correctly perceived (change awareness) or missed SD = 3 years) from the University of Geneva. We also (change blindness) and trials with different stimuli but recruited a different group of 18 other students, 10 of similar perceptual outcome (i.e., when a face identity whom (6 women; mean age = 25 years, SD = 3 years) change was missed, relative to an absence of change participated in a behavioral pilot test to define appro- correctly reported). priate stimulus parameters to obtain an equal pro- By recording EEG over the two successive stimuli in portion of correct detections and misses and 8 of this simplified flicker paradigm and combining both whom (4 women; mean age = 26 years, SD = 3 years) waveform and topographical analyses of ERPs, we might participated in another control behavioral test to estab- directly find evidence for the different theoretical lish that changes in a single visual stimulus could be hypotheses put forward by Simons (2000). If change correctly detected when presented alone and fully at- blindness results from a rapid fading of the trace (or tended (see below). All participants had normal or

Figure 1. Sequence of events within a trial during our EEG experiment. Two visual displays (each consisting of a small alphanumerical symbol presented centrally, plus two different faces in the periphery) were briefly presented for 150 msec each, separated by a 150-msec empty screen (here with a left face change). Subjects were asked to report whether a number (1–9) appeared at the central location task (to ensure adequate central fixation) and then whether one of the face identities has changed (using two successive response screens, not shown here).

2110 Journal of Cognitive Neuroscience Volume 18, Number 12 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2006.18.12.2108 by guest on 26 September 2021 corrected-to-normal vision and no history of neurologi- that fixation remained central across all face conditions. cal or psychiatric illness. Note that small changes in the central alphanumerical symbol between S1 and S2 were constant across the different experimental conditions and therefore can- Visual Stimuli celed out any potential contamination of the ERPs to peripheral face stimuli. During the critical EEG experiment, each visual display Participants had to perform a dual task: (1) to monitor consisted of a character presented at the center of the the occurrence of a number at the center and (2) to screen (either a letter [from a choice of nine: A, E, H, K, report any change in the identity of one of the two Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/18/12/2108/1756025/jocn.2006.18.12.2108.pdf by guest on 18 May 2021 N, R, T, U, X] or an arabic number [from 1 to 9]), peripheral faces. Responses were always required ac- together with two flanking faces (each 78 88, with cording to this task order (Figure 1): 600 msec after their center 5.78 to the right or left of the central onset of S2, a response screen prompted participants to character), all on a black background (Figure 1). The report whether a number had been presented or not (by central stimuli, as well as the size and position of pressing one of two buttons, within a 2-second limit); peripheral faces, were preselected to adjust task difficul- then a second response screen prompted them to ty during a behavioral pretest. Face images were gray- report whether one of the faces had changed or not scale photographs taken from a standardized set (KDEF; (again using two buttons, within a 2-second limit). D. Lundqvist, A. Flykt, and A. O¨hman, Department of Instructions emphasized accuracy in the primary central Neurosciences, Karolinska Hospital, Stockholm, Sweden, task and also urged participants against guessing by 1998), all with a neutral expression (50 different identi- asking them to adopt a strict criterion for their ‘‘change’’ ties, half of each sex). The two faces shown on any given responses (i.e., report a face change ‘‘only when seen trial were always from two different person identities, with confidence,’’ otherwise report no change). but with the same sex. Each individual face was com- All stimuli were presented on a 17-in. computer bined with one out of two other faces, with similar screen with a PC Pentium II running STIM software. external hair shape to minimize any strategy based on Participants were seated in a shielded room in front of low-level visual cues, yielding 100 different pairs in total the screen (viewing distance = 50 cm). Behavior was (with each AB combination mirrored by a symmetric monitored by closed-circuit TV. Subjects received one BA pairing). practice block of 30 trials, followed by 10 runs of 60 trials each (600 trials total), with a different random order of the different experimental conditions (20 trials with no Procedure in the Main face change, 20 with a left face change, and 20 with a Electroencephalogram Experiment right face change). Each face pair (n = 100) appeared On each trial, the two displays (S1 and S2) were four times in each condition. Thus, in total, each face presented in rapid succession after a central fixation identity (n = 50) was presented 48 times, with equal cross (250 msec) and a fixed empty delay (250 msec) probability in the left or right visual field (LVF and RVF, (Figure 1). Each display was presented for 150 msec, respectively). Each identity appeared equally often in separated by an empty screen for 150 msec. In S2, the the first than in the second display on trials with a two faces could be identical to those in S1 (one third of change and also equally often on trials with no change. trials, no-change condition) or one of the two faces (either on the left or right side) was replaced by a new Behavioral Pretest identity, always with the same sex (two thirds of trials, half on the left and half on the right). The short stimulus Before the EEG experiment, we performed a pilot duration (150 msec) plus the central task ensured that behavioral study in a different group of participants peripheral change detection was performed without (n = 10) to determine the stimulus parameters and saccades (similar to Beck et al., 2001). Orthogonally to procedure that fulfilled the following criteria: (a) a the peripheral face conditions, the central character was balanced distribution between the rate of change blind- also systematically manipulated between S1 and S2, such ness and change awareness trials (approximately half that either two different letters were successively pre- each), allowing an adequate comparison of each condi- sented (half of trials) or a number was presented in one tion; (b) a low rate of false detection on trials with no the two displays (first or second, a quarter of trials each). change (20% or lower), to ensure a clear difference in Correct performance in the central task was likely to perceptual experience during change awareness versus require adequate fixation, because the characters were blindness; (c) a high accuracy in the central task, but too small and similar to be easily discriminated in without ceiling performance and without any tradeoff peripheral vision, as confirmed during pilot testing and with peripheral face detection. Using the same stimulus by monitoring eye position by a closed TV circuit during parameters as above, this behavioral pretest showed an EEG recordings. Our procedure thus ensured that the equal numbers of misses (mean 49%) and hits (mean occurrence of a central number was unpredictable and 51%) on trials with a peripheral face change, whereas

Pourtois et al. 2111 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2006.18.12.2108 by guest on 26 September 2021 there was a low rate of false alarms (17%) on trials with analyses and source reconstructions, averaged ERPs no face change, and also a good performance in the were re-referenced using a common average reference. central number task (2.9% of errors).

Data Analysis Control Behavioral Experiment Global Waveform Modulations In addition, we tested another group of participants To determine the timing of differences between con- (n = 8) on a much more simple task, in which faces ditions over the whole stimulus sequence (e.g., change Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/18/12/2108/1756025/jocn.2006.18.12.2108.pdf by guest on 18 May 2021 were presented unilaterally in the periphery, always in detection vs. change blindness), we calculated pointwise LVF or always in RVF (for separate blocks of 100 paired t tests between ERPs in each condition, using the consecutive trials on each side), while subjects main- mean raw amplitude recorded every 2 msec in each tained fixation at the screen center. Like in the main subject and the variance across subjects. The first time experiment, the first face (150 msec) was followed by a point where t test values exceeded the .05 alpha prob- brief blank interval (150 msec) and then a second face ability criterion for at least 10 consecutive data points (150 msec) that could be either the same or different (>20 msec at 500 Hz) over at least two adjacent elec- (half of trials each). But here, changes occurred at a fully trodes was considered as the onset of a significant predictable and attended position in the peripheral difference (see Guthrie & Buchwald, 1991). Although visual field, without any other concurrent task. In these this approach does not entirely protect against spurious conditions, face changes were correctly detected in a significant effects because consecutive EEG samples are large majority of trials (90% hits) in both visual fields not independent, this combined spatio-temporal crite- (but with a significant advantage for the LVF relative to rion provides a reliable global estimate of the onset and the RVF: 91% vs. 87% correct, respectively, t(7) = 2.3, offset of stable ERP effects (in time and space), without p = .055, consistent with a right hemispheric dominance any a priori selection of restricted time points or limited in face processing; see also below). False alarm rates electrodes (see Murray et al., 2004; Guthrie & Buchwald, were low (6% in LVF and 9% in RVF, t(7) = 1.24, ns). 1991, for similar methods). This control experiment shows that face identity changes could be reliably perceived when the location of changes could be predicted and attended, indicating Component Analyses that failures in the main EEG experiment were not Amplitudes of prominent ERP components were quanti- because of the perceptual difficulty of face identification, fied in terms of mean voltage within a specified latency but to higher attentional demands of the task, and thus window (centered on the component’s peak), with reflected a genuine ‘‘change blindness’’ phenomenon respect to a 200-msec prestimulus baseline (Picton (Simons & Rensink, 2005). et al., 2000). Latency analyses of the components were also performed but not reported in details because they did not vary significantly as a function of our experimen- Data Acquisition tal factors (see Table 1). Relevant sites for each compo- Visual ERPs were recorded and processed using a NEU- nent were selected based on previous studies (e.g., ROSCAN 64 channels (Synamps, El Paso, TX). Horizontal Fernandez-Duque et al., 2003; Koivisto & Revonsuo, and vertical electrooculograms (EOGs) were monitored 2003) and topographic properties of the current data using four facial bipolar electrodes placed on the outer set. After the onset of S1, three conspicuous compo- canthi of each eye and in the inferior and superior nents were identified (Table 1 and Figure 2A): a lateral areas of the left orbit. The 62 Ag/AgCl electrodes were occipital P1 (Luck, Heinze, Mangun, & Hillyard, 1990), a mounted on a quickcap, according to the extended posterior temporal N170 (Itier & Taylor, 2004a; Bentin 10-20 system, with a linked-mastoid reference, amplified et al., 1996), and an occipital positivity corresponding with a gain of 30 K and bandpass filtered at 0.01–100 Hz to P2 (Noesselt et al., 2002; Johannes, Munte, Heinze, with a 50-Hz notch filter. Impedance was kept below & Mangun, 1995). P1 (80–140 msec) was maximal at lat- 5k . EEG and EOG were continuously acquired at a rate eral and superior occipital electrodes O1/PO7/PO5/PO3 of 500 Hz and stored for off-line averaging. EEG was (left hemisphere) and O2/PO8/PO6/PO4 (right), with a corrected for eye blinks (Gratton, Coles, & Donchin, typical dipolar topography (Luck et al., 1990). The N170 1983). After removal of artifacts (epochs with EEG or (130–200 msec) had a more anterior occipito-temporal residual EOG exceeding ±75 AV), epoching was made topography (PO7/PO5/P7/P5 over the left, PO8/PO6/P8/ from 200 msec before the onset of S1 until 800 msec P6 over the right), with inverted polarity relative to P1 after onset, covering the complete stimulus sequence. (Bentin et al., 1996; George, Evans, Fiori, Davidoff, & EEG data were first baseline-corrected on the prestimu- Renault, 1996; Jeffreys & Tukmachi, 1992). The P2 (210– lus interval (200/0 msec), averaged to individual ERPs, 310 msec) was maximal at posterior medial electrodes and finally low-pass filtered at 30 Hz. For topographic (PO3/POz/PO4). A negative activity was also present

2112 Journal of Cognitive Neuroscience Volume 18, Number 12 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2006.18.12.2108 by guest on 26 September 2021 Table 1. Mean Amplitude (in AV) and Mean Latency (in msec) of the Different ERP Components

Left Face Change p Value Detected vs. p Value Detected vs. Visual Display Component Detected Undetected Undetected (Amplitude) No Change No Change (Amplitude)

First (S1) P1 7.3 (110.0) 6.6 (110.4) .04 6.8 (109.0) .07 N170 5.9 (168.7) 6.1 (169.2) .75 6.2 (168.5) .45 P2 4.3 (240.3) 3.9 (237.9) .52 3.7 (240.4) .22 Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/18/12/2108/1756025/jocn.2006.18.12.2108.pdf by guest on 18 May 2021 Frontal 0.7 (289.6) 1.9 (293.7) .01 1.1 (295.0) .41 Second (S2) P1 4.4 (407.9) 3.9 (408.4) .49 3.9 (407.2) .36 N170 2.8 (454.5) 4.1 (454.7) .04 3.4 (454.3) .18 P2 5.3 (501.0) 5.2 (504.3) .85 5.1 (504.3) .57 P300 9.4 (708.8) 7.1 (697.1) .002 7.0 (697.1) .001

250 msec after S1 (50 msec before the onset of the sec- over time, across all conditions. Because only topo- ond visual display), with a fronto-central topography and graphic landscape differences are of interest, scalp amplitude variation at electrodes AF4/F8/F6 (right) and maps are first scaled to unitary strength by dividing AF3/F7/F5 (left) sharing some electrophysiological simi- the voltage at each electrode by the global field power larities with the early phase of a CNV potential (Walter, (GFP). The optimal number of topographic maps ex- Cooper, Aldridge, McCallum, & Winter, 1964). plaining the whole data set is determined objectively by For ERPs after S2, the same three visual peaks were a cross-validation criterion (Pascual-Marqui et al., 1995). identified, including a P1 (370–440 msec postonset of S1), The dominant topographies (identified in the group- N170 (430–500 msec), and P2 (450–530 msec). The first averaged data) are then fitted to ERPs of each individual two components had a similar topography during S2 subject using a spatial fitting procedure, providing a than during S1 (Figure 7), but P2 had a slightly different quantitative estimates of their relative expression (in topography, now maximal at centro-parietal (C1/Cz/C2, strength and duration) across subjects and conditions CP1/CPz/CP2, and P1/Pz/P2) rather than occipito-parietal (Michel et al., 1999). These analyses were carried out electrodes. In addition, at a later latency (640–750 msec using CARTOOL software (version 3.0; developed by D. poststimulus), a large and sustained positivity was ap- Brunet, Functional Brain Mapping Laboratory, Geneva, parent, with a centro-parietal topography and maximal Switzerland). amplitude at C1/Cz/C2, CP1/CPz/CP2, and P1/Pz/P2, com- patible with a P3 or P300 (Polich & Kok, 1995). Distributed Source Localization Differences between conditions were analyzed by com- paring the mean amplitude of each component at adja- To estimate the likely neural sources underlying the cent electrode positions over each hemisphere (Picton electric field configurations identified by the segmenta- et al., 2000). Repeated measures analyses of variance tion analysis, we used a distributed linear inverse solu- (ANOVAs) were performed using a Greenhouse–Geisser tion, based on a local auto-regressive average (LAURA) correction for nonsphericity when required, with three model for the unknown current density, derived from main experimental conditions: face-change correctly re- biophysical laws describing electric fields in the brain ported (change awareness), face-change missed (change (Grave de Peralta Menendez, Murray, Michel, Martuzzi, blindness), and no-change correctly reported. & Gonzalez Andino, 2004). This source localization technique provides a linear distributed inverse solu- tion, using a realistic head model with 4024 lead field Topographic Analyses nodes, selected from a 6 6 6-mm grid equally To identify ERP differences due to variations in global distributed within the gray matter of the Montreal topography, rather than just local amplitude effects at Neurological Institute template brain. This method em- selected electrodes, we used a temporal segmentation ulates the properties of brain activity by computing algorithm derived from standard spatial cluster analysis multiple simultaneously active sources based on bio- (Pascual-Marqui, Michel, & Lehmann, 1995), as already physically driven inverse solutions without a priori as- applied in other cognitive domains (Itier & Taylor, sumption on the number and position of the possible 2004a; Leonards, Palix, Michel, & Ibanez, 2003; Michel generators. The procedure was implemented using et al., 1999). This procedure determines the dominant CARTOOL software. Additional analyses were also per- topographies appearing in the group-averaged ERPs formed using a similar source estimation procedure by

Pourtois et al. 2113 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2006.18.12.2108 by guest on 26 September 2021 Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/18/12/2108/1756025/jocn.2006.18.12.2108.pdf by guest on 18 May 2021

Figure 2. ERP data for the whole stimulus sequence. (A) Group-averaged (n = 11) waveforms (in the left face change detection condition) are shown superimposed across all electrodes (n = 62), revealing a series of conspicuous electric deflections (P1, N170, P2, sustained negative activity, and P3) after the first visual display (S1) and the second visual display (S2). (B) Modulations of ERP waveform during change detection versus change blindness were assessed by a running paired t test for each electrode ( y axis) and each time point (x axis) using the variance across subjects. Statistical p values are coded on a three-level grayscale (see Methods for details). This global comparison indicates four distinct periods where the two conditions differed (for >20 msec), each highlighted by dotted outline boxes. The earliest effect occurred at approximately 50 to 120 msec (1), with subsequent effects at 290 to 375 msec (2), 400 to 450 msec (3), and 600 to 800 msec (4).

low-resolution electromagnetic tomography (LORETA, These behavioral data ensure that participants correctly version 3; Pascual-Marqui, Michel, & Lehmann, 1994). maintained fixation at the center. Importantly, error rates in this central task did not vary (all t <1.3, p > .20) as a function of whether subjects detected RESULTS the peripheral face change (24.4% of trials with central errors), failed to detect this change (38.9%), or correctly Behavioral Results for the Central Fixation Task reported the absence of a change (30.0%). During EEG recordings, all 12 participants performed These behavioral results were corroborated by eye accurately on the central number task (mean error movements recorded during EEG. For each condition rate = 2.3%), except for one subject who was clearly (detected, missed, or no change), we calculated the deviant (12% errors) relative to the group mean ± 2 SD. average eye position measured by the horizontal EOG This subject was excluded from subsequent analyses. For during three successive bins of 150 msec, reflecting the remaining participants, the mean error rate in the the occurrence of any saccade during S1 (150 msec), central task dropped to 1.4% (min = 0, max = 3.3%). S2 (150 msec), or the blank interval (150 msec). A

2114 Journal of Cognitive Neuroscience Volume 18, Number 12 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2006.18.12.2108 by guest on 26 September 2021 3 (conditions) 3 (bins) ANOVA on these data did not condition (see Methods). Global waveform modulations reveal any significant effect or interaction (all F <1, (based on a running paired t test; see Methods) and p > .34). Altogether, these data indicate that the pe- topographic segmentations (based on a spatial cluster ripheral change detection task was performed without analysis) showed significant effects only for changes in any systematic horizontal eye movement or any tradeoff the LVF (see below), but not in the RVF, probably with the central task performance. because of the fact that we obtained a too small number of correct change detection in the RVF (25%) in these conditions, leading to insufficient signal-to-noise ratio to Behavioral Results for the Peripheral Face Task compute reliable ERPs. However, visual inspection of Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/18/12/2108/1756025/jocn.2006.18.12.2108.pdf by guest on 18 May 2021 Overall, our paradigm was successful to achieve a bal- ERPs generally showed a comparable pattern for anced proportion of trials where a face change was changes in both visual fields. either correctly detected or missed (mean 43% vs. Our statistical analyses included four steps. We exam- 57%, respectively), whereas false alarms remained rela- ined (a) point-by-point differences in the time course of tively rare when no change occurred (9.4%). Thus, ERP amplitude between conditions; (b) changes in am- participants reported a face change only when it was plitude and latency of classic components peaks; (c) var- perceived with sufficient confidence, ensuring that these iations in topographic distribution of electric activity trials were associated with robust awareness of the new over time; (d) possible neural sources for these differ- face as compared with change blindness and no change. ent topographies. These results are described in the Moreover, we observed a strong asymmetry between following sections. the two hemifields. Participants correctly reported 61% of changes in LVF (range = 21%–89%) but only 25% of Global Waveform Modulations changes in RVF (range = 7%–44%). This asymmetry, observed for 10 out of 11 subjects, was highly significant To determine global differences without a priori selec- [t(10) = 5.6, p < .001] and reflected a true difference in tion of specific time points or electrodes, a running perceptual sensitivity, as shown by a higher discrimina- pointwise paired t test was performed between change tion measure (d0) for changes in left than right faces blindness and change awareness from 200 to 800 msec [mean = 2.17 vs. 1.14; t(10) = 5.18, p < .001]. This is post-S1 onset. This revealed four distinct periods of consistent with neuropsychological data showing a dom- differential activity with a stability of >10 consecutive inance of the right hemisphere/LVF for face processing time points (i.e., >20 msec; see Guthrie & Buchwald, (Grusser & Landis, 1991). An item-based analysis indi- 1991). The earliest difference (Figure 2B) started at cated that hits (correct change detection) were random- approximately 50–120 msec (i.e., during S1). Other dif- ly distributed across the 100 different face pairs and did ferences occurred around approximately 290–375 msec not concern a specific subset of stimuli. and approximately 400–450 msec, as well as later, ap- proximately 600–800 msec post-S1 onset. These ERP modulations suggest that conscious awareness of face Event-related Potential Results changes corresponds to a distinct pattern of brain re- EEG recorded from the onset of S1 to S2 allowed us to sponses at several successive stages of processing. More determine the processing stage(s) when neural activity critically, brain activity corresponding to successful differed among three conditions: (a) face-change cor- change detection already differed from brain activity rectly reported, (2) face-change missed, and (3) no- corresponding to change blindness performance during change correctly reported. Thus, the critical comparison two distinct time periods before the occurrence of the between awareness versus blindness [condition (a) vs. change itself, that is, during S1 and during the interval (b)] concerned ERPs for the same stimulus condition just preceding S2 (Figure 2B). However, these data alone but with different percepts. ERPs were computed using do not indicate whether the differences between con- only those trials (97.7%) where performance on the ditions concerned the timing, amplitude, and/or topog- central task was correct and blocks when performance raphy of activity (see below). on the peripheral face task was associated with a low Using a similar pointwise t test, we also compared rate of false alarms (<20%). After this strict procedure, trials with correct change detection versus no change we excluded 17 out of 110 possible blocks (11 subjects and trials with change blindness versus no change. In 10 runs). For the remaining blocks, false alarms were the former comparison, two ERP modulations similar very rare (mean = 4%, range = 0%–11%). Furthermore, to the above were identified (around 400–450 and 600– because of the hemifield asymmetry in change detection 800 msec), but the latter comparison did not show any rates (i.e., better on the left than right side, see above), reliable effect across time points and electrodes. This ERPs were computed and subsequently analyzed using negative result indicated that despite the different visual only trials when face changes occurred in the LVF, for input in these two conditions (change blindness vs. no which we had a comparable rate of hits and misses change), both subjects’ awareness and brain activity (60% vs. 40%) and hence sufficient ERPs in each similarly categorized the stimuli as an absence of change.

Pourtois et al. 2115 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2006.18.12.2108 by guest on 26 September 2021 These data therefore reveal distinctive ERPs for con- and the relevant Electrode Sites (e.g., anterior vs. pos- sciously perceived changes of faces, but provide no terior position, lateral vs. central line, where appropri- evidence for an effect of missed changes. ate; see Methods). Only results for amplitude differences are reported, because the same analyses on latencies did not show any significant effect (all p > .10; see Table 1). Classical Component Analyses These analyses confirmed that brain activity corre- A sequence of well-known exogenous responses was sponding to change awareness differed from brain activity identified, including the occipital P1 (Luck et al., 1990), corresponding to change blindness already approxi-

posterior temporal N170 (Bentin et al., 1996), and parieto- mately 200 msec before the change actually occurred. Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/18/12/2108/1756025/jocn.2006.18.12.2108.pdf by guest on 18 May 2021 occipital P2 (Noesselt et al., 2002; Johannes et al., 1995), The amplitude of P1 evoked by S1 was enhanced when a for each of the two successive face displays (S1 and S2). subsequent face change was detected versus missed (Fig- These visual responses were followed by a late P3 ure 3 and Table 1). A 3 (condition) 2(hemisphere) component, arising after S2, before the participants’ re- 4 (electrode) ANOVA on mean amplitude of this com- sponse (see Figure 2A). The amplitude and latency of ponent revealed a significant main effect of condition each peak (Picton et al., 2000) were examined by re- [F(2,20) = 3.66, p = .046]. This first P1 was larger for peated measures ANOVAs, using the factors of Percep- change detection (mean = 7.31 AV) than change blind- tual Condition (change awareness, change blindness, ness (mean = 6.61 AV) [t(10) = 2.39, p =.038],whereas and no-change correct), Hemisphere (right and left), there was only a trend between change detection and no

Figure 3. Occipital P1 effect, 110 msec following the first visual display. (A) Spatial layout of the 62 electrode sites used for EEG recording. Electrodes (n = 8) selected for P1 analyses are highlighted by a shaded area on each side. A group-averaged ERP waveform (at electrode PO6) is displayed from 200 msec until +800 msec following stimulus onset, with a shaded area highlighting the time course of P1 and N170 for the first visual display (S1). (B) Axial view of P1 topography (voltage map) on the scalp (here during change detection). (C) Details of the waveform highlighted in (A), showing ERP differences (at PO6) between change detection (in gray) and change blindness (in black). The P1 was larger in the former than the latter condition, but there was no effect for the N170. (D) Mean amplitude of P1 (in OV, collapsed across the 8 selected electrodes) and standard errors (SEM) as a function of trial type, showing enhanced P1 for correct change detection.

2116 Journal of Cognitive Neuroscience Volume 18, Number 12 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2006.18.12.2108 by guest on 26 September 2021 change (mean = 6.77 AV) [t(10) = 2.01, p =.072].A wise comparisons performed on the mean amplitude of direct comparison between change blindness and no the P2 component evoked by S1 did not reveal any change did not reveal any significant effect or interaction significant effect or interaction (all p >.22). (all p > .09). This result suggests an enhanced processing At a later latency (during the interval between S1 and of S1 during change awareness and correct no-change S2), a CNV-like activity was observed over fronto-central trials, relative to change blindness, arising when the sites (Figure 4), clearly distinct from the preceding P2 change has not occurred yet. For the N170 component and peaking approximately 290 msec poststimulus onset evoked by S1, a 3 (condition) 2(hemisphere) 2 (see also Figure 2A). A 3 (condition) 2(hemisphere)

(line) 2 (position) ANOVA on mean amplitudes did not 3 (position) ANOVA on the mean amplitude of this Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/18/12/2108/1756025/jocn.2006.18.12.2108.pdf by guest on 18 May 2021 reveal any significant effect of condition (Table 1). Thus, frontal activity revealed a highly significant main effect this face-selective component did not differ (all p >.16) of condition [F(2,20) = 4.89, p = .019]. This sustained between change awareness (mean = 5.9 AV), change frontal activity was systematically more negative (mean = blindness (mean = 6.1 AV),ornochange(mean= 1.91 AV) during change blindness as compared with 6.2 AV), as illustrated in Figure 3C. Similarly, a 3 change detection (mean = 0.72 AV) [F(1,10) = 9.16, (condition) 3 (position) ANOVA, plus follow-up pair- p =.013]andnochange(mean=1.05 AV) [F(1,10) =

Figure 4. Frontal effect, 250 msec following the first visual display. (A) Spatial layout of the 62 electrodes, with the lateral sites (n = 6) selected to analyze this frontal activity highlighted by a shaded area on each side. A group-averaged ERP waveform (electrode F7) is displayed from 200 msec until +800 msec following stimulus onset, with a shaded area highlighting the time period when a sustained frontal activity was found during change detection. (B) Axial topography (voltage map) showing an anterior negative activity during change detection, not seen during change blindness. (C) Details of the waveform highlighted in (A), showing ERP differences (at F7) between change detection (in gray) and change blindness (in black). A sustained negative shift in lateral frontal activity was systematically larger in the latter than the former condition. (D) Mean amplitude of this frontal activity (in AV, collapsed across 6 electrodes) and standard errors (SEM) as a function of trial type, showing an enhanced negative activity for change blindness relative to both correct change detection and correct no-change report.

Pourtois et al. 2117 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2006.18.12.2108 by guest on 26 September 2021 4.70, p =.055].However,therewasnodifference formed on the mean amplitude of the P1 evoked by S2 between the two latter conditions [F(1,10) = 0.74, did not reveal any significant effect of condition or p = .41]. This frontal effect started just before the onset interaction with this factor (all p > .16). By contrast, of S2 (Figure 4), and was therefore not caused by any the 3 (condition) 2 (hemisphere) 2 (line) 2 effect time-locked to this second display. (position) ANOVA on the mean amplitude of the N170 Visual responses evoked by S2 showed a similar P1– evoked by S2 showed a significant effect of condition N170–P2 sequence (Table 1), but with a marked atten- [F(2,20) = 3.61, p = .046; see Figure 5]. N170 responses uation of the overall amplitude of P1 (mean = 4.05 AV) to S2 were smaller when a change of face was detected

and N170 (mean = 3.5 AV) as compared with the same (mean = 2.8 AV) as compared with change blindness Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/18/12/2108/1756025/jocn.2006.18.12.2108.pdf by guest on 18 May 2021 components evoked by S1 (P1, mean = 6.89 AV; N170, (mean = 4.1 AV) [F(1,10) = 5.83, p = .036]. In the no- mean = 6.1 AV). This reduction across the two con- change condition, N170 showed an intermediate magni- secutive displays was highly significant [P1, F(1,10) = tude (mean = 3.4 AV; see Figure 5), not statistically 12.83, p = .005; N170, F(1,10) = 7.82, p = .019] and different from either change detection [F(1,10) = 2.10, consistent with the well-known attenuation (‘‘gating’’) p = .19] or change blindness [F(1,10) = 2.01, p = .19]. of exogenous sensory responses to stimuli presented in Subsequent topographic analyses indicated that this am- rapid succession (Graham, 1975). However, a 3 (condi- plitude difference for N170 corresponded to a distinct tion) 2 (hemisphere) 4 (electrode) ANOVA per- topography between change detection versus blindness

Figure 5. N170 effect, 150 msec following the second visual display. (A) Spatial layout of the 62 electrodes, with sites (n = 8) selected for the N170 highlighted by a shaded area on each side. A group-averaged ERP waveform (electrode PO8) is displayed from 200 msec until +800 msec following stimulus onset, with a shaded area highlighting the time course of P1 and N170 for the second visual display. (B) Axial view of N170 topography (voltage map) in the change detection condition. (C) Details of the waveform highlighted in (A), showing ERP differences (at PO8) between change detection (in gray) and change blindness (in black). The N170 was significantly smaller in the former than the latter, without any difference in P1. (D) Mean amplitude of N170 (in AV, collapsed across 8 electrodes) and standard errors (SEM) as a function of trial type, showing a reduction of N170 during change detection as compared with change blindness, with an intermediate amplitude during no change.

2118 Journal of Cognitive Neuroscience Volume 18, Number 12 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2006.18.12.2108 by guest on 26 September 2021 during the time range of the N170 (see below). This .002] and no change (mean = 7.0 AV) [t(10) = 4.41, result suggests a modification in the visual encoding of p = .001]. P3 amplitude was not different between the second face when a change is consciously perceived, the two latter conditions [t(10) = 0.26, p = .80]. This as compared with blindness for the same physical modulation by the detection of changes is consistent change. Finally, the 3 (condition) 3 (line) 3 (posi- with the postperceptual P3 activity typically associated tion) ANOVA performed on the mean amplitude of P2 with categorization and decision processes (Polich & responses evoked by S2 did not reveal any significant Kok, 1995). effect of condition or interaction with this factor (all

p > .16). Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/18/12/2108/1756025/jocn.2006.18.12.2108.pdf by guest on 18 May 2021 Topographic Modulations We also analyzed the differences in a P3 compo- nent after exogenous visual responses to S2, approx- Changes in the topography of electric field across dif- imately 650–750 msec posttrial onset (Table 1 and ferent conditions may arise independently of differ- Figure 6). This revealed a highly significant effect of ences in strength (i.e., amplitude of the component condition [F(2,20) = 13.71, p < .001] due to a larger waveforms or GFP), reflecting an activation of distinct P3 for change detection (mean = 9.4 AV) relative to neural generators rather than a modulation in the change blindness (mean = 7.1 AV) [t(10) = 4.16, p = magnitude or latency of responses seen at discrete

Figure 6. P3 effect, 350 msec following the second visual display. (A) Spatial layout of the 62 electrodes, with sites (n = 9) selected for the P3 highlighted by a shaded area on each side. A group-averaged ERP waveform (at electrode Cz) is displayed from 200 msec until +800 msec following stimulus onset, with a shaded area highlighting the time course of P3. (B) Axial view of P3 topography (voltage map) in the change detection condition. (C) Details of the waveform highlighted in (A), showing ERP differences (at Cz) between change detection (in gray) and change blindness (in black). The P3 was consistently larger in the former than the latter. (D) Mean amplitude of P3 (in AV, collapsed across 9 electrodes) and standard errors (SEM) as a function of trial type, showing an increased P3 during change detection condition relative to other conditions.

Pourtois et al. 2119 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2006.18.12.2108 by guest on 26 September 2021 electrode positions (Lehmann & Skrandies, 1980); al- and the expression of Map 4 (see Figure 7). This dissoci- though strength changes may also modify topography ation emphasizes the complementary value of compo- when using single dipole fitting (Urbach & Kutas, 2002) nent and topographic analyses. unlike here. Such changes in the global configuration of The distinctiveness of Map 4 across conditions was electrical activity over time can provide important addi- further tested by calculating its degree of expression (in tional information about the spatio-temporal dynamics time frames) in each individual and each condition, of visual processing, not always available in conventional using a spatial fitting procedure (Michel et al., 1999), waveform measures (Pourtois, Dan, Grandjean, Sander, and then performing a pairwise statistical comparison

& Vuilleumier, 2005; Michel et al., 1999). To determine between the fitting values of different conditions. This Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/18/12/2108/1756025/jocn.2006.18.12.2108.pdf by guest on 18 May 2021 whether awareness versus blindness of face changes was analysis confirmed that Map 4 was generally present associated with distinct topographies of neural activity for a shorter total duration within the 0- to 600-msec (in addition to the amplitude effects described above), window postonset for change blindness (mean = 12.6 we applied a standardized spatial cluster analysis (or tem- time frames), as compared with change detection poral segmentation method, Pascual-Marqui et al., 1995) (mean = 22.3 time frames) [t(10) = 1.76, p = .05, one- that can define the dominant electric configuration tailed] or with the no-change condition (mean dura- appearing over time, across the different conditions, tion = 19.8 time frames) [t(10) = 1.82, p = .05]. There irrespective of changes in amplitude (see Methods). was no difference between no-change and change de- This temporal segmentation analysis identifies a se- tection [t(10) = 0.54, p = .30]. In addition, to test for quence of statistically distinct topographic maps in the the recurring property of this map (see Figure 7A and time course of EEG responses, each being presumably C), we also computed the mean number of separate associated with different functional stages of informa- occurrences (>10 consecutive time frames) during the tion processing (i.e., microstates; see Michel et al., 1999; 0- to 600-msec temporal window for each subject and Lehmann & Skrandies, 1980). This approach could be each condition. This confirmed that separate repeti- particularly useful here to determine whether awareness tions of this map were more frequent during change of face changes is associated with (a) a unique functional detection (mean = 2.1) than during change blindness topographic map at a specific latency (not present (mean = 1.2) [t(10) = 1.76, p = .05, one-tailed]. Note during blindness), (b) a prolongation of a specific map that the number of separate repetitions in individual (also present but shorter during blindness), or (c) a subjects does not necessarily correspond to the num- higher strength for one or more of the maps seen in the ber of segments in the group-average results, due to other conditions. the differences in duration for each occurrence in in- A temporal segmentation analysis was performed dividual subjects (see Michel et al., 1999). across all experimental conditions (Michel et al., 1999), Finally, this spatial cluster analysis also revealed that using a temporal window from S1 onset until 600 msec the scalp topography corresponding to the face-selective postonset (i.e., 300 time frames including all prominent N170 component (Map 3, Figure 7D) was different ERPs). Results revealed that the group-averaged data between S1 and S2 during change detection (Figure 7A) could be segmented by a solution of 10 different maps, but partly identical across these two successive displays explaining 94.5% of the total variance. As can be seen during change blindness (Figure 7B) and no-change in Figure 7, the temporal organization of these maps conditions (Figure 7C) [with a global duration of this was almost identical across the three conditions, with topography in the 400- to 500-msec window being the exception of the distinctive topography of Map 4 shorter for change detection than change blindness; (Figure 7D). Map 4 was not reliably elicited during change t(10) = 1.81, p = .05, one-tailed]. This result adds to blindness (Figure 7B) relative to correct change detection our finding of a significant modulation of N170 ampli- (Figure 7A) and correct no-change (Figure 7C) condi- tude at posterior electrodes, as a function of awareness tions. It first appeared 220 msec after onset of S1 (i.e., versus blindness. In other words, when a face change 80 msec before onset of S2) and was then reproduced at occurred but was missed, the topographic map cor- several moments in both change detection and correct responding to the face-selective N170 evoked by S2 no-change conditions (at 350, 460, 520, and 580 msec for partly replicated the topography evoked during S1 (Fig- change detection; at 220, 350, 520 and 580 msec for ure 7B), unlike when the face change was detected correct no-change conditions, in the average group (Figure 7A). These results indicate that the local modu- data). A careful inspection of the topographic data (see lation of N170 at posterior electrodes was in fact associ- Figure 7) indicated that Map 4 associated with awareness ated with a concurrent change in the underlying neural of face change was actually elicited during time periods generators (Lehmann & Skrandies, 1980). corresponding to low-amplitude neural events that did Some differences were also found in Map 7, subsequent not overlap with the expression of ERP components (as to the N170 (Map 3) evoked by S2 (see Figure 7D) and usually revealed by higher GFP in these analyses). Thus, concomitant with the P2 component. However, this there was no clear relation between the P1 effect (Map 2, global change in P2 topography between S1 (Map 5) Figure 7D) or sustained frontal effect (Map 6, Figure 7D) and S2 (Map 7) was identical across all three conditions

2120 Journal of Cognitive Neuroscience Volume 18, Number 12 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2006.18.12.2108 by guest on 26 September 2021 Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/18/12/2108/1756025/jocn.2006.18.12.2108.pdf by guest on 18 May 2021

Figure 7. Temporal segmentation of ERP topography. The spatial cluster analysis (from stimulus onset until 600 msec) showed a segmentation solution with 10 different maps. The succession of these maps is shown for (A) correct change detection, (B) change blindness, and (C) correct no-change reports. Time is represented along the x axis and GFP along the y axis. Each distinct topography is labeled with a number (1–10). Black and white segments indicate topographies occurring across all experimental conditions, whereas red and blue segments indicate topographies that differed between conditions. (D) Axial views of the electric field configuration (scaled to unitary strength by dividing the voltage at each electrode by the GFP) corresponding to each of the numbered map segment. A distinctive map (Map 4) is observed only during the change detection and no-change conditions, arising at several successive moments in time during the whole trial sequence (indicated by arrows). This map was not significantly expressed in the change blindness condition. Another map (Map 3) corresponding to the N170 was repeated during the second visual display (with a briefer duration than during the first display) only when no face change was perceived (i.e., during change blindness and no change), whereas it was replaced by Map 4 during change detection.

(Figure 7A–C), unlike the N170 topography that was gyrus (Brodmann’s area 18/19). This pattern is consist- significantly modulated by detection of changes. ent with previous studies on this early occipital activity (Pourtois, Grandjean, Sander, & Vuilleumier, 2004). For the map corresponding to N170 (Map 3 in Figure 7D), Source Localization LAURA identified several sources in lateral occipital and Lastly, we used source localization by LAURA (Grave de lateral temporal cortex (Brodmann’s area 19, 37), in- Peralta Menendez et al., 2004) to estimate the likely cluding superior temporal sulcus, compatible with face- neural generators accounting for the different topogra- selective activations in similar regions in previous brain phies measured over the scalp, particularly those asso- imaging (e.g., Haxby, Hoffman, & Gobbini, 2000) and ciated with the main peaks of ERPs (i.e., P1 and N170, ERP studies (Itier & Taylor, 2004b). More importantly, respectively, Maps 2 and 3 in Figure 7D) and those that for the critical recurrent map associated with change distinguished change detection from change blindness awareness (Map 4 in Figure 7D), LAURA identified a (Map 4 in Figure 7D). For the topography corre- distributed network of bilateral brain sources, includ- sponding to P1 (Map 2 in Figure 7D), LAURA found ing the posterior parietal cortex, medial occipital lobe, bilateral sources in extrastriate visual cortex (Figure 8A), and lateral inferotemporal cortex (Figure 8B). The peak located medially in the inferior and superior occipital of these sources was further estimated in Talairach

Pourtois et al. 2121 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2006.18.12.2108 by guest on 26 September 2021 Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/18/12/2108/1756025/jocn.2006.18.12.2108.pdf by guest on 18 May 2021

Figure 8. Source localization results by LAURA. (A) Estimates of neural generators for the P1 topography (Map 2 in Figure 7D) were located in the bilateral occipital cortex, including the ventral and dorsal extrastriate areas. (B) Sources for the distinct topography associated with change awareness (Map 4 in Figure 7D) were found in a distributed network of regions in the posterior parietal and lateral occipital cortex (on both sides).

coordinates (Pascual-Marqui et al., 1994) and found to conditions. Critically, recording brain activity over the involve Brodmann’s area 7 in superior parietal lobule whole stimulus sequence, both before and subsequent (±24x, 60y,64z), as well as area 37 in posterior lateral to the change, allowed us to probe several stages in fusiform gyrus (left: 52x, 60y, 13z; right: 46x, 67y, visual processing where change blindness might arise 6z) and area 18/19 in middle occipital gyrus (±31x, (Simons, 2000). Behaviorally, our participants showed 88y,8z). Taken together, these results indicate that a few false alarms on no-change trials (9.4%), suggesting a concomitant activation of parietal and more ventral relatively strict response criterion and, hence, a reliable temporo-occipital regions was specifically and recur- perceptual awareness of visual change during correct rently present during trials with change awareness, as reports. In addition, accuracy in the detection of periph- opposed to change blindness. eral changes was not associated with any systematic difference in performance for the central task or in horizontal eye movements, suggesting that blindness or awareness could arise despite equally good fixation DISCUSSION in each condition. By using EEG in a simplified flicker paradigm, our study Critically, our EEG results revealed that the neural could uncover the distinctive time course of brain correlates of conscious change detection did not con- activity during change blindness and change awareness, cern a single processing stage, but were distributed over whereas visual stimuli remained identical in the two a large temporal window and involved several successive

2122 Journal of Cognitive Neuroscience Volume 18, Number 12 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2006.18.12.2108 by guest on 26 September 2021 events. Strikingly, the earliest correlate of successful 2001), we show that conscious detection of a visual change detection, as compared with change blindness, change does not depend on a single, discrete neural arose shortly after the onset of the first visual stimulus event, nor does it depend solely on processing new (S1), that is, before the change itself (S2). Overall, we visual information about the change; but activity encod- found a sequence of five major effects in EEG that ing the visual scene before the change may also play a distinguished brain activity during change awareness crucial role (Vogel & Machizawa, 2004). Our findings and change blindness. that ERPs in the change detection condition differed (a) During S1, correct change detection trials already from those in the change blindness condition during

differed from change blindness trials for the visual P1 four nonoverlapping time periods (two of them before Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/18/12/2108/1756025/jocn.2006.18.12.2108.pdf by guest on 18 May 2021 (130 msec postonset), whose amplitude was larger in the onset of the face change) suggest that several the former than in the latter case, although visual possible mechanisms may produce change blindness. stimulation was identical and expectation of a change First, it is possible that change blindness could equally equally likely in these two conditions. The latency, scalp result from a failure in any of these four stages (P1 for topography, and extrastriate sources of this early com- S1, CNV before S2, N170 after S2, or P3 after S2), each ponent were consistent with previous studies (Martinez having an independent, but fundamental weight during et al., 1999). (b) During the interval between S1 and S2, conscious detection of change. Alternatively, our results another difference arose between change detection and are also compatible with the view that change blindness blindness in a sustained negative activity over frontal might result from a complex ‘‘syndrome’’ due to simul- regions (CNV-like potential), starting approximately taneous failures of these four stages, having all a unique 250 msec postonset, again preceding the change that but complementary contribution. Finally, change blind- occurred 50 msec later. (c) During S2, after a change of ness might be produced primarily from a defect in face identity, correct detections were associated with allocating sufficient attention resources toward the first reduced amplitude of the N170, typically evoked by stimulus (S1), resulting in a cascade of subsequent faces (Bentin et al., 1996). Furthermore, this modulation deficits (at the level of the subsequent CNV, N170, and of N170 amplitude was concomitant with a modification P3 respectively), in agreement with previous psycho- of the electric field topography, suggesting some differ- physical work (Rensink, 2002). More experimental work ence in the generators of N170 during awareness versus is needed to tease apart these different accounts of blindness of a changing face. (d) At a later postpercep- change blindness. Below, we will discuss the implica- tual stage, 350 msec after onset of S2, conscious tions of each of these new ERP findings. change detection enhanced the amplitude of the P3, in accord with an involvement in target detection and/or response selection mechanisms (Polich & Kok, 1995). P1 and Visual Attention (e) Finally, using a spatial cluster analysis of ERP topog- raphy over time (Pascual-Marqui et al., 1995), we could P1 is an early exogenous visual response generated in ex- identify a specific configuration of activity (EEG Map 4; trastriate cortex whose amplitude is typically enhanced see Figure 7D) that arose during correct change detec- by selective spatial attention through sensory gain mech- tion, but not change blindness, and exhibited a recur- anisms (Hillyard & Anllo-Vento, 1998). A larger P1 for the rent time course during the stimulus sequence (during first visual display during change detection versus both S1 and S2), rather than a single transient occur- change blindness indicates that sensory processing of rence. The neural sources for this unique map were S1 was better in the former than the latter condition, estimated to involve a simultaneous activity in dorsal presumably allowing then a more efficient comparison parietal and ventral occipito-temporal regions. of the first pair of faces with the subsequent pair. This Our results demonstrate for the first time that change enhanced P1 might reflect a greater engagement of blindness (relative to correct detection) may correspond attentional resources on the initial display. However, to some failures arising during the processing of both attention did not appear to be selectively focused on the the first and the second visual display, rather than side of the upcoming change during successful detec- during one of them only or than during a later compar- tion (and vice versa, directed to the opposite side during ison process only (see Simons, 2000). These findings blindness), as we did not observe a larger modulation add to previous EEG studies of change blindness that of P1 over the hemisphere contralateral to detected focused on correlates of change blindness and change changes (i.e., right occipital P1 for LVF). Instead, P1 awareness consecutive to the change (e.g., see Niedeggen was bilaterally enhanced, suggesting a better encoding et al., 2001, for a P3 effect; Koivisto & Revonsuo, 2003 for of the entire visual display, perhaps reflecting more a N1 effect). Although our data are broadly consistent distributed attention or greater alertness (Fernandez- with these previous results and confirm that midlatency Duque & Posner, 1997). We surmise that this early P1 ERP components (200–300 msec) are modulated by effect indicates dynamic moment-by-moment changes changes in perceptual awareness (Fernandez-Duque in attention, rather than more global fluctuations in et al., 2003; Dehaene et al., 2001; Niedeggen et al., vigilance state, because this difference was seen for P1

Pourtois et al. 2123 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2006.18.12.2108 by guest on 26 September 2021 during S1 only, but not for other attention-sensitive did not correspond to a different content of awareness components (e.g., N1) or for P1 during S2 (i.e., a few per se (Pins & Ffytche, 2003). hundreds milliseconds later). Furthermore, we analyzed only trials where the central task was correctly per- formed, ruling out that reduced P1 to S1 during change Sustained Frontal Activity blindness reflected transient lapses in vigilance level, as vigilance was in fact sufficient to perform the central A second neural correlate of correct change detection task during the very same trials. Moreover, correct was found before change onset, with a slow negative

detections were regularly interspersed among trials activity over lateral frontal electrodes that was larger Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/18/12/2108/1756025/jocn.2006.18.12.2108.pdf by guest on 18 May 2021 where changes were missed over the experiment time during change blindness than awareness, starting ap- course, ruling out more general fatigue effects. There- proximately 250 msec after S1 but approximately fore, our results corroborate behavioral studies suggest- 50 msec before S2. This CNV-like activity did not differ ing that attention may play a major role in the ability to between correct change and correct no-change reports. detect visual changes (Rensink, 2002; Shapiro et al., Therefore, as for P1, this difference cannot result from 1997), with blindness being much less likely for changes systematic anticipation or response biases in reporting in an attended object or location than for the same changes, and it is unlikely that any anticipatory effects changes occurring outside attention (see Fernandez- couldhavearisenmoreoftenwhenachangewas Duque et al., 2003). But here we show that attention actually going to occur and appear on the left, than in effects (selectively impacting on P1 activity) can arise the other experimental conditions. Moreover, the size of before attention being directed to the change itself, this frontal effect was visibly as large or even larger than suggesting that successful change detection critically the P1 effect (compare Figures 4D and 3D, respectively). depends upon a correct encoding of the initial display A similar CNV activity is often recorded during an (Simons, 2000). interval (of a few seconds) between a warning or pre- We found no significant modulation of the P1 re- paratory stimulus (S1) and an imperative stimulus (S2) sponse to the second visual display (S2) associated with that requires a speeded response (Walter et al., 1964). awareness of the face change, although it might be Although here the second visual display appeared rap- expected that correct detection of changes should also idly (300 msec) after the first visual display and did not require selective attention at the time of S2. However, if strictly correspond to an imperative stimulus (the re- we consider that attention may vary independently for sponse probe screen was presented at a later point in S1 and S2 on a trial-by-trial basis, then it is likely that time; see Figure 1), the present frontal effect shared sensory processing and P1 activity for S2 could not only many topographical similarities with the initial portion be enhanced on those trials when changes were de- of CNV (Brunia & van Boxtel, 2001). This could index a tected (i.e., when sensory processing and P1 were also more efficient preparation for the detection of upcom- enhanced for the first stimulus), but could equally be ing changes (with better recruitment or focusing of enhanced on other trials when change blindness even- attention resources) or a more efficient encoding of S1 tually ensued because processing of the first display was into short-term memory. Both hypotheses might ac- insufficient (and/or its maintenance during the interval). count for a more positive frontal activity during both In other words, an amplification of P1 to the second change detection and correct no-change reports, relative display might be missed or underestimated because of to change blindness. the occurrence of such amplification in many trials Alternatively, it is possible that this frontal activity where P1 was reduced for the first display and thus might relate only to the maintenance of a short-term resulted in change blindness. memory trace, between the disappearance of S1 and More generally, these results support the notion that the appearance of S2, allowing a more efficient compar- perceptual awareness is closely connected to attentive ison of the faces in each display. The lateral frontal processes (Lamme, 2003; Driver & Vuilleumier, 2001). topography of this activity would be consistent with a Furthermore, preparatory states in attention can modu- major role of frontal cortex in working-memory pro- late neural activity in visual cortex even before stimulus cesses (e.g., D’Esposito, Postle, & Rypma, 2000) and onset (Kastner et al., 1999), involving changes in base- converges with previous suggestions that visual atten- line firing rates or oscillations (Tallon-Baudry et al., tion alone is not sufficient to perceive changes in a scene 2005), and in some situations, this prestimulus activity (Simons & Rensink, 2005; Simons, 2000; Rensink et al., can determine or predict the subsequent perceptual 1997). This significant frontal difference during the in- processing (Tallon-Baudry et al., 2005; Austen & Enns, terval before the change indicates that successful de- 2003; Super et al., 2003). In our study, the early differ- tection may not only require the formation of a robust ence in P1 for detected versus missed changes, preced- representation for the first display (as indexed by occip- ing change onset, clearly demonstrates a crucial role for ital P1 effects) but also the maintenance of this repre- attentional mechanisms that can influence subsequent sentation across time delays (presumably indexed by awareness of the changes (Rensink, 2002), although it frontal effects).

2124 Journal of Cognitive Neuroscience Volume 18, Number 12 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2006.18.12.2108 by guest on 26 September 2021 Future ERP studies should determine which stimuli engage a partly different network, producing a different and tasks may facilitate the occurrence of these antici- topography distribution. Taken together, these results patory effects (P1 and CNV) during change detection, suggest that N170 activity may reflect high-level process- and whether similar effects are also observed during es involved in face identity recognition (Itier & Taylor, other tasks. For example, other experimental conditions 2004a; George et al., 1997), rather than structural encod- (tapping either into mechanism of perception, memory, ing or visual categorization only (Eimer, 2000). This or attention) where task difficulty is systematically ma- topography-specific N170 effect might correspond to nipulated might also produce partly similar patterns of the activation of face processing systems critically impli-

ERP effects, with a differential brain activity arising cated in the subjective perceptual experience of a face Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/18/12/2108/1756025/jocn.2006.18.12.2108.pdf by guest on 18 May 2021 before the target onset but predicting performance change during correct detection trials. success at a later point in time (see Tallon-Baudry By providing a rapid estimate for the time when et al., 2005; Vogel & Machizawa, 2004). face changes are detected, these findings also con- verge with Koivisto and Revonsuo (2003) in that dis- tinct neural activity around 200-msec postchange onset N170 and Awareness of Face Changes might correlate with differences in the content of visual After the onset of S2, the N170 had a smaller amplitude awareness (see also Vuilleumier et al., 2001; Corthout, when a change of face was detected than when it was Uttl, Ziemann, Cowey, & Hallett, 1999, for a modulation missed or when no change occurred. This effect arose of visual ERPs at similar latencies for seen vs. unseen without any difference in earlier visual responses to the stimuli). second display, for example, in the just preceding P1. These findings suggest that neural activity in the time P300 and Change Detection range of N170, typically related to structural face pro- cessing (Bentin et al., 1996), may be sensitive to the After the N170, awareness of face changes produced a perceived identity of faces (repeated vs. changed), but large and sustained positive activity with a centro-parietal does not simply respond to the physical visual features topography, compatible with a classic P3 component in repeated or novel faces (Campanella et al., 2000). (Polich & Kok, 1995). Similar P3 responses have already When missed, the same change of faces did not affect been observed during conscious detection of visual the N170, ruling out that these differences resulted from changes (Fernandez-Duque et al., 2003; Koivisto & a simple visual repetition or ‘‘priming’’ effect. Moreover, Revonsuo, 2003; Niedeggen et al., 2001) or detection of the fact that the N170 did not differ between change targets with low probability (Polich & Kok, 1995; Johnson, detection and change blindness for S1, but only for S2, is 1986). The higher amplitude of P3 for change detection consistent with an involvement of this face-specific relative to the other conditions (and the lack of difference component in some higher level mechanism of face between no-change and change blindness) is consistent recognition and confirms that this ERP response may with a postperceptual effect, probably not reflecting differentiate changes in face identity (Campanella et al., changes in the content of visual awareness per se, but 2000; George, Jemel, Fiori, & Renault, 1997). other aspects in the subjective experience of change, in- In keeping with this, our temporal segmentation cluding subsequent decision and response selection analysis revealed that the N170 modulation for S2 did stages or some form of memory updating (Polich & Kok, not only affect its amplitude, but also evoked a distinct 1995). This late P3 effect is therefore likely to be less topography, which was selectively modified during the dependent upon specific stimulus content than the pre- correct detection of face changes relative to misses and ceding N170 effect, as might be demonstrated in future no-change trials (i.e., with Map 4 replacing Map 3, experiments using different categories of visual objects. respectively, from 150 to 170 msec after S2 onset). Thus, for both change blindness and no-change condi- Recurrent Activity in a Distributed Network tions, the initial N170 topography (Map 3) was repeated of Dorsal and Ventral Brain Areas in part with the same configuration during the first and the second face displays (150–210 msec after S1, and A major novel finding in our study was provided by the 150–170 msec after S2), whereas another distinct temporal segmentation analysis of ERP topography, topography followed in all conditions (Map 7, from revealing that awareness of changes was associated with 170 to 210 msec after S2 onset). This repetition of a recurrent neural process (corresponding to Map 4, the same N170 activity between the first and second face Figure 7D) arising at several successive stages during displays when no change was perceived is consistent the stimulus sequence, both before and after change with a lack of comparison mismatch in these two con- onset (i.e., during both S1 and S2). This configuration of ditions and suggests that such comparison (Mitroff et al., EEG activity was seen selectively during correct change 2004; Scott-Brown et al., 2000) may partly be indexed by detection, as well as during the correct no-change N170 activity (George et al., 1997). In contrast, the N170 condition, although it was not reliably expressed during elicited by the detection of a new face appeared to change blindness. This recurrent configuration of neural

Pourtois et al. 2125 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2006.18.12.2108 by guest on 26 September 2021 activity therefore reflected the higher success of visual establish a robust representation of visual stimuli for awareness during correct change and correct no-change accurate conscious report (Di Lollo et al., 2000). Future reports, in contrast to change blindness. Importantly, studies are needed to determine whether similar recur- source estimation using LAURA (Grave de Peralta rent EEG states correlate with visual awareness in other Menendez et al., 2004) suggested that this activity attentional or masking paradigms. (Map 4) was generated by a concurrent activation of bilateral regions in superior parietal cortex and inferior No Evidence for Implicit Processing temporo-occipital cortex (Figure 8B), close to the pos-

terior and lateral fusiform regions implicated in face and We found no ERP effect for undetected changes relative Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/18/12/2108/1756025/jocn.2006.18.12.2108.pdf by guest on 18 May 2021 object recognition (Haxby et al., 2000). to the no-change condition, suggesting that ERPs were These results are highly consistent with the fMRI not sensitive to ‘‘implicit’’ processing potentially arising findings of Beck et al. (2001) in a similar change during change blindness (Mitroff, Simons, & Franconieri, blindness paradigm, where correct detection of face 2002; Smilek, Eastwood, & Merikle, 2000). This is un- changes correlated with increased activation in fusiform likely to result from our specific task (see Beck et al., and in posterior parietal areas, known to be involved in 2001) or instructions emphasizing strict confidence for attentional processes (Corbetta et al., 2000). A similar reporting changes (note that our subjects still made false pattern of ventral–temporal and dorsal–parietal activa- alarms on 10% of no-change trials and less than 2% in tion for seen relative to unseen stimuli was found in the fMRI study of Beck et al., 2001). A conservative fMRI studies of patients with spatial neglect (Vuilleumier criterion was encouraged to ensure a reliably distinct et al., 2001, 2002), as well as during visual masking in percept during change awareness but could have con- healthy subjects (Dehaene et al., 2001). Our source tributed to include trials with a ‘‘weak’’ or ‘‘vague’’ localization results also converge with recent findings sense of change in the change blindness condition (Beck, Muggleton, Walsh, & Lavie, 2006) showing that (Rensink, 2004). Indeed, a consequence of a conserva- repetitive transcranial magnetic stimulation over poste- tive criterion is that some proportion of the missed rior parietal cortex can interfere with change detection changes might reflect low-confidence detection. Despite during a flicker paradigm similar to the current task. this, we found no evidence that unreported changes Altogether, these data support the idea that functional modulated ERPs in terms of either component or to- coupling between distant brain areas in ventral and pography (relative to no change). This again under- dorsal visual stream might play a critical role in visual scores that the four successive events in ERPs during awareness (Driver & Vuilleumier, 2001; Kanwisher, change detection versus blindness were specifically re- 2001). However, our new findings suggest that such lated to differences in awareness, rather than physical functional coupling might not arise at a single moment inputs. However, future ERP studies should also collect in time, but involves a recurrent dynamic process, taking confidence ratings to test if some effects might correlate place during several periods over the course of stimulus with subjective certainty or qualitative aspects rather processing. Such recurrent activity could not be identi- than with all-or-none detection judgments (see Sergent, fied in previous fMRI studies because of their poor Baillet, & Dehaene, 2005; Sergent & Dehaene, 2004). temporal resolution as compared with EEG. Our new The lack of implicit effect in our study contrasts with findings further demonstrate that a concomitant activa- behavioral evidence for implicit representation of missed tion of parietal and visual areas may not only arise from changes (e.g., Fernandez-Duque & Thornton, 2000) but neural events triggered by the perceived changes but agrees with previous ERP studies of change blindness may also precede the onset of changes that will be (Henderson, Baker, & Orbach, 2006; Koivisto & Revonsuo, subsequently detected. 2003; Orbach & Henderson, 2003; Niedeggen et al., Elucidating the temporal dynamics and interactions 2001). A late anterior effect (240–300 msec) was observed between parietal and inferotemporal areas is a central in one study (Fernandez-Duque et al., 2003) that com- issue in current research on perceptual awareness (Block, pared ERPs to complex pictures during a continuous 2005). The present results seem to challenge a simple flicker presentation when subjects failed to see a change view of awareness arising from the access of sensory and when no change was present. This effect may have information to higher level or distributed networks in been amplified by the continuous (or cyclic) repetition of a single ‘‘winner-take-all’’ step. Instead, our data sup- stimuli in this flicker paradigm. Moreover, unconscious port the view that awareness may result from a sate- processing might yield less synchronous electrical re- dependent mechanism (Austen & Enns, 2003), with sponses in visual pathways, precluding the recording of recurrent activity and cross-talk between distant areas reliable ERPs (Driver & Vuilleumier, 2001). operating in a sustained manner, through recursive loops and feedback interactions within distributed networks Conclusions (Block, 2005; Di Lollo, Enns, & Rensink, 2000). Such recursive and coordinated activity might be crucial to Our study provides several new findings on the mech- accrue reliable information from sensory inputs and to anisms of change detection and change blindness. We

2126 Journal of Cognitive Neuroscience Volume 18, Number 12 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2006.18.12.2108 by guest on 26 September 2021 show that awareness of changes may depend on distinct Di Lollo, V., Enns, J. T., & Rensink, R. A. (2000). Competition patterns of brain activity, implicating several successive for consciousness among visual events: The psychophysics of reentrant visual processes. Journal of Experimental neural events over a few hundreds milliseconds, starting Psychology: General, 129, 481–507. already before change onset. Further, by combining Driver, J., & Vuilleumier, P. (2001). Perceptual awareness classical ERPs with topographic analyses, we show that and its loss in unilateral neglect and extinction. Cognition, a distinct configuration of activity is associated with 79, 39–88. change awareness, characterized by concurrent activa- Eimer, M. (2000). The face-specific N170 component reflects late stages in the structural encoding of faces. tion of parietal and occipito-temporal areas at several NeuroReport, 11, 2319–2324. successive latencies. Our data provide new insights Fernandez-Duque, D., Grossi, G., Thornton, I. M., & Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/18/12/2108/1756025/jocn.2006.18.12.2108.pdf by guest on 18 May 2021 on the dynamic neural underpinnings of attention and Neville, H. J. (2003). Representation of change: Separate awareness. electrophysiological markers of attention, awareness, and implicit processing. Journal of Cognitive Neuroscience, 15, 491–507. Fernandez-Duque, D., & Posner, M. I. (1997). Relating the Acknowledgments mechanisms of orienting and alerting. Neuropsychologia, This work is supported by a grant from the Swiss National 35, 477–486. Science Foundation to P. V. (632.065935). Fernandez-Duque, D., & Thornton, I. M. (2000). Change detection without awareness: Do explicit reports Reprint requests should be sent to Gilles Pourtois, Neurology underestimate the representation of change in the & Imaging of Cognition, Clinic of Neurology and Depart- visual system? Visual Cognition, 7, 324–344. ment of Neuroscience, University Medical Centre (CMU), 1 rue George, N., Evans, J., Fiori, N., Davidoff, J., & Renault, B. Michel-Servet, CH-1211 Geneva, Switzerland, or via e-mail: gilles. (1996). Brain events related to normal and moderately [email protected] or patrick.vuilleumier@medicine. scrambled faces. Brain Research, Cognitive Brain unige.ch. Research, 4, 65–76. George, N., Jemel, B., Fiori, N., & Renault, B. (1997). Face and shape repetition effects in humans: A spatio-temporal ERP study. NeuroReport, 8, 1417–1423. REFERENCES Graham, F. K. (1975). The more or less startling effects of weak prestimulation. Psychophysiology, 12, 238–248. Austen, E. L., & Enns, J. T. (2003). Change detection in an Gratton, G., Coles, M. G. H., & Donchin, E. (1983). attended face depends on the expectation of the observer. A new method for off-line removal of ocular artifact. Journal of Vision, 3, 64–74. Electroencephalography and Clinical Neurophysiology, Beck, D. M., Muggleton, N., Walsh, V., & Lavie, N. (2005). 55, 468–484. Right parietal cortex plays a critical role in change Grave de Peralta Menendez, R., Murray, M. M., Michel, C. M., blindness. Cerebral Cortex, 16, 712–717. Martuzzi, R., & Gonzalez Andino, S. L. (2004). Electrical Beck, D. M., Rees, G., Frith, C. D., & Lavie, N. (2001). Neural neuroimaging based on biophysical constraints. correlates of change detection and change blindness. Neuroimage, 21, 527–539. Nature Neuroscience, 4, 645–650. Grimes, J. (1996). On the failure to detect changes in scenes Bentin, S., Allison, T., Puce, A., Perez, E., & McCarthy, G. across saccades. In K. Akins (Ed.), Vancouver studies (1996). Electrophysiological studies of face perception in in cognitive science: Vol. 2: Perception (pp. 89–110). humans. Journal of Cognitive Neuroscience, 8, 551–565. New York: Oxford University Press. Block, N. (2005). Two neural correlates of consciousness. Grusser, O. J., & Landis, T. (1991). Visual agnosias and other Trends in Cognitive Science, 9, 46–52. disturbances of visual perception and cognition. Vision Brunia, C. H., & van Boxtel, G. J. (2001). Wait and see. and visual dysfunction (vol. 12). Basingstoke: Macmillan. International Journal of Psychophysiology, 43, 59–75. Guthrie, D., & Buchwald, J. S. (1991). Significance testing Campanella, S., Hanoteau, C., Depy, D., Rossion, B., of difference potentials. Psychophysiology, 28, 240–244. Bruyer, R., Crommelinck, M., et al. (2000). Right N170 Haxby, J. V., Hoffman, E. A., & Gobbini, M. I. (2000). The modulation in a face discrimination task: An account for distributed human neural system for face perception. categorical perception of familiar faces. Psychophysiology, Trends in Cognitive Science, 4, 223–233. 37, 796–806. Henderson, R. M., Baker, M. R., & Orbach, H. S. (2006). Corbetta, M., Kincade, J. M., Ollinger, J. M., McAvoy, Is there a mismatch negativity during change blindness? M. P., & Shulman, G. L. (2000). Voluntary orienting is NeuroReport, 17, 1011–1015. dissociated from target detection in human posterior Hillyard, S. A., & Anllo-Vento, L. (1998). Event-related brain parietal cortex. Nature Neuroscience, 3, 292–297. potentials in the study of visual selective attention. Corthout, E., Uttl, B., Ziemann, U., Cowey, A., & Hallett, M. Proceedings of the National Academy of Sciences, U.S.A., (1999). Two periods of processing in the (circum)striate 95, 781–787. visual cortex as revealed by transcranial magnetic Itier, R. J., & Taylor, M. J. (2004a). N170 or N1? Spatiotemporal stimulation. Neuropsychologia, 37, 137–145. differences between object and face processing using Dehaene, S., Naccache, L., Cohen, L., Bihan, D. L., Mangin, ERPs. Cerebral Cortex, 14, 132–142. J. F., Poline, J. B., et al. (2001). Cerebral mechanisms Itier, R. J., & Taylor, M. J. (2004b). Source analysis of the of word masking and unconscious repetition priming. N170 to faces and objects. NeuroReport, 15, 1261–1265. Nature Neuroscience, 4, 752–758. Jeffreys, D. A., & Tukmachi, E. S. A. (1992). The vertex-positive D’Esposito, M., Postle, B. R., & Rypma, B. (2000). Prefrontal scalp potential evoked by faces and by objects. cortical contributions to working memory: Evidence Experimental Brain Research, 91, 340–350. from event-related fMRI studies. Experimental Brain Johannes, S., Munte, T. F., Heinze, H. J., & Mangun, G. R. Research, 133, 3–11. (1995). Luminance and spatial attention effects on early

Pourtois et al. 2127 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2006.18.12.2108 by guest on 26 September 2021 visual processing. Brain Research, Cognitive Brain Pascual-Marqui, R. D., Michel, C. M., & Lehmann, D. (1995). Research, 2, 189–205. Segmentation of brain electrical activity into microstates: Johnson, R., Jr. (1986). A triarchic model of P300 amplitude. Model estimation and validation. IEEE Transactions on Psychophysiology, 23, 367–384. Biomedical Engineering, 42, 658–665. Kanwisher, N. (2001). Neural events and perceptual Pessoa, L., & Ungerleider, L. G. (2004). Neural correlates awareness. Cognition, 79, 89–113. of change detection and change blindness in a working Kastner, S., Pinsk, M. A., De Weerd, P., Desimone, R., & memory task. Cerebral Cortex, 4, 511–520. Ungerleider, L. G. (1999). Increased activity in human Picton, T. W., Bentin, S., Berg, P., Donchin, E., Hillyard, visual cortex during directed attention in the absence of S. A., Johnson, R., et al. (2000). Guidelines for using visual stimulation. Neuron, 22, 751–761. human event-related potentials to study cognition: Koch, C. (2004). The quest for consciousness: A neuroscientific Recording standards and publication criteria. Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/18/12/2108/1756025/jocn.2006.18.12.2108.pdf by guest on 18 May 2021 approach. Englewood, CO: Roberts & Company Psychophysiology, 37, 127–152. Publishers. Pins, D., & Ffytche, D. (2003). The neural correlates of Koivisto, M., & Revonsuo, A. (2003). An ERP study of conscious vision. Cerebral Cortex, 13, 461–474. change detection, change blindness, and visual awareness. Polich, J., & Kok, A. (1995). Cognitive and biological Psychophysiology, 40, 423–429. determinants of P300: An integrative review. Biological Lamme, V. A. (2003). Why visual attention and awareness Psychology, 41, 103–146. are different. Trends in Cognitive Science, 7, 12–18. Pourtois, G., Dan, E. S., Grandjean, D., Sander, D., & Lehmann, D., & Skrandies, W. (1980). Reference-free Vuilleumier, P. (2005). Enhanced extrastriate visual identification of components of checkerboard-evoked response to bandpass spatial frequency filtered fearful multichannel potential fields. Electroencephalography faces: Time course and topographic evoked-potentials and Clinical Neurophysiology, 48, 609–621. mapping. Human Brain Mapping, 26, 65–79. Leonards, U., Palix, J., Michel, C., & Ibanez, V. (2003). Pourtois, G., Grandjean, D., Sander, D., & Vuilleumier, P. Comparison of early cortical networks in efficient and (2004). Electrophysiological correlates of rapid spatial inefficient visual search: An event-related potential study. orienting towards fearful faces. Cerebral Cortex, 14, Journal of Cognitive Neuroscience, 15, 1039–1051. 619–633. Luck, S. J., Heinze, H. J., Mangun, G. R., & Hillyard, S. A. (1990). Rensink, R. A. (2002). Change detection. Annual Review Visual event-related potentials index focused attention of Psychology, 53, 245–277. within bilateral stimulus arrays. 2. Functional dissociation Rensink, R. A. (2004). Visual sensing without seeing. of P1 and N1 components. Electroencephalography and Psychological Science, 15, 27–32. Clinical Neurophysiology, 75, 528–542. Rensink, R. A., O’Regan, J. K., & Clark, J. J. (1997). To see Martinez, A., Anllo-Vento, L., Sereno, M. I., Frank, L. R., or not to see: The need for attention to perceive changes Buxton, R. B., Dubowitz, D. J., et al. (1999). Involvement in scenes. Psychological Science, 8, 368–373. of striate and extrastriate visual cortical areas in spatial Ro, T., Russell, C., & Lavie, N. (2001). Changing faces: attention. Nature Neuroscience, 2, 364–369. A detection advantage in the flicker paradigm. Psychological Michel, C. M., Seeck, M., & Landis, T. (1999). Spatiotemporal Science, 12, 94–99. dynamics of human cognition. News in Physiological Scott-Brown, K. C., Baker, M. R., & Orbach, H. S. (2000). Sciences, 14, 206–214. Comparison blindness. Visual Cognition, 7, 253–267. Mitroff, S. R., Simons, D. J., & Franconieri, S. L. (2002). Sergent, C., Baillet, S., & Dehaene, S. (2005). Timing of the The siren song of implicit change detection. Journal brain events underlying access to consciousness during of Experimental Psychology: Human Perception and the attentional blink. Nature Neuroscience, 8, 1391–1400. Performance, 28, 798–815. Sergent, C., & Dehaene, S. (2004). Is consciousness a Mitroff, S. R., Simons, D. J., & Levin, D. T. (2004). Nothing gradual phenomenon? Evidence for an all-or-none compares 2 views: Change blindness can occur despite bifurcation during the attentional blink. Psychological preserved access to the changed information. Perception Science, 15, 720–728. & Psychophysics, 66, 1268–1281. Shapiro, K. L., Arnell, K. A., & Raymond, J. E. (1997). The Murray, M. M., Michel, C. M., Grave de Peralta, R., Ortigue, S., attentional blink: A view on attention and a glimpse on Brunet, D., Gonzalez Andino, S., et al. (2004). Rapid consciousness. Trends in Cognitive Sciences, 1, 291–296. discrimination of visual and multisensory memories Simons, D. J. (2000). Current approaches to change revealed by electrical neuroimaging. Neuroimage, 21, blindness. Visual Cognition, 7, 1–15. 125–135. Simons, D. J., & Rensink, R. A. (2005). Change blindness: Past, Niedeggen, M., Wichmann, P., & Stoerig, P. (2001). Change present, and future. Trends in Cognitive Science, 9, 16–20. blindness and time to consciousness. European Journal Smilek, D., Eastwood, J. D., & Merikle, P. M. (2000). Does of Neuroscience, 14, 1719–1726. unattended information facilitate change detection? Noe¨, A., Pessoa, L., & Thompson, E. (2000). Beyond the Journal of Experimental Psychology: Human Perception grand illusion: What change blindness really teaches us and Performance, 26, 480–487. about vision. Visual Cognition, 7, 93–106. Super, H., van der Togt, C., Spekreijse, H., & Lamme, V. A. Noesselt, T., Hillyard, S. A., Woldorff, M. G., Schoenfeld, A., (2003). Internal state of monkey primary visual cortex Hagner, T., Jancke, L., et al. (2002). Delayed striate cortical (V1) predicts figure-ground perception. Journal of activation during spatial attention. Neuron, 35, 575–587. Neuroscience, 23, 3407–3414. Orbach, H. S., & Henderson, R. M. (2003). Are there Tallon-Baudry, C., Bertrand, O., Henaff, M. A., Isnard, J., event-related potential (ERP) correlates of implicit & Fischer, C. (2005). Attention modulates gamma-band change detection? A miscuing paradigm. Journal of oscillations differently in the human lateral occipital Vision, 3, 588a. cortex and fusiform gyrus. Cerebral Cortex, 15, 654–662. Pascual-Marqui, R. D., Michel, C. M., & Lehmann, D. (1994). Triesch, J., Ballard, D. H., Hayhoe, M. M., & Sullivan, B. T. Low resolution electromagnetic tomography: A new (2003). What you see is what you need. Journal of Vision, method for localizing electrical activity in the brain. 3, 86–94. International Journal of Psychophysiology, 18, 49–65. Urbach, T. P., & Kutas, M. (2002). The intractability of

2128 Journal of Cognitive Neuroscience Volume 18, Number 12 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2006.18.12.2108 by guest on 26 September 2021 scaling scalp distributions to infer neuroelectric sources. and unseen faces in visuospatial neglect: A combined Psychophysiology, 39, 791–808. event-related functional MRI and event-related potential Vogel, E. K., & Machizawa, M. G. (2004). Neural activity study. Proceedings of the National Academy of Sciences, predicts individual differences in visual working memory U.S.A., 98, 3495–3500. capacity. Nature, 428, 748–751. Walter, W. G., Cooper, R., Aldridge, V. J., McCallum, Vuilleumier, P., Armony, J. L., Clarke, K., Husain, M., W. C., & Winter, A. L. (1964). Contingent negative Driver, J., & Dolan, R. J. (2002). Neural response variation: An electric sign of sensorimotor association to emotional faces with and without awareness: Event- and expectancy in the human brain. Nature, 203, related fMRI in a parietal patient with visual extinction 380–384. and spatial neglect. Neuropsychologia, 40, 2156–2166. Williams, P., & Simons, D. J. (2000). Detecting changes Vuilleumier, P., Sagiv, N., Hazeltine, E., Poldrack, R. A., in novel, complex three-dimensional objects. Visual Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/18/12/2108/1756025/jocn.2006.18.12.2108.pdf by guest on 18 May 2021 Swick, D., Rafal, R. D., et al. (2001). Neural fate of seen Cognition, 7, 297–322.

Pourtois et al. 2129 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2006.18.12.2108 by guest on 26 September 2021