Modulates Maintenance of Representations in Visual Short-term

Bo-Cheng Kuo, Mark G. Stokes, and Anna Christina Nobre Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/24/1/51/1780230/jocn_a_00087.pdf by MIT Libraries user on 17 May 2021

Abstract ■ Recent studies have shown that selective attention is of consid- retrospective cues during the maintenance period could predict a erable importance for encoding task-relevant items into visual specific item (spatial retrocue) or multiple items (neutral retrocue)

short-term memory (VSTM) according to our behavioral goals. thatwouldbeprobedattheendofthememorydelay.Ourresults Downloaded from http://direct.mit.edu/jocn/article-pdf/24/1/51/1943573/jocn_a_00087.pdf by guest on 03 October 2021 However, it is not known whether top–down attentional biases revealed that VSTM performance is significantly improved by ori- can continue to operate during the maintenance period of VSTM. enting attention to the location of a task-relevant item. The behav- We used ERPs to investigate this question across two experiments. ioral benefit was accompanied by modulation of neural activity Specifically, we tested whether orienting attention to a given spa- involved in VSTM maintenance. Spatial retrocues reduced the tial location within a VSTM representation resulted in modulation magnitude of the CDA, consistent with a reduction in memory of the contralateral delay activity (CDA), a lateralized ERP marker load. Our results provide direct evidence that top–down control of VSTM maintenance generated when participants selectively modulates neural activity associated with maintenance in VSTM, encode memory items from one hemifield. In both experiments, biasing competition in favor of the task-relevant information. ■

INTRODUCTION teristics of the mnemonic information. Activation in these Visual short-term memory (VSTM) is the function by posterior visual areas is to be maintained and coor- which potentially relevant visual information is main- dinated by top–down signals from multisensory executive tained in the form of internal representations that persist regions, such as pFC (Curtis & DʼEsposito, 2003; Awh beyond the original sensory input to guide subsequent & Jonides, 2001; Miller, Li, & Desimone, 1993). Single-unit behavior. The capacity of VSTM is limited: Only a small recordings in nonhuman primates demonstrate sustained number of items can be retained in VSTM at any given firing for neurons that are selective for the specific memo- time. Recent studies have shown that selective attention randa throughout the maintenance period in a delayed- is of considerable importance for gating the encoding of match-to-sample task (Super, Spekreijse, & Lamme, 2001; task-relevant items into VSTM according to behavioral Chelazzi, Duncan, Miller, & Desimone, 1998; Miller et al., goals and expectations (Gazzaley, 2011; Murray, Nobre, & 1993). Electrophysiological studies in humans reveal an Stokes, 2011; Rutman, Clapp, Chadick, & Gazzaley, 2010; ERP marker of sustained mnemonic activity, known as Zanto & Gazzaley, 2009; Schmidt, Vogel, Woodman, & “contralateral delay activity” (CDA) or “sustained poste- Luck, 2002). In this study, we investigate whether and rior contralateral negativity” (Eimer & Kiss, 2010; Ikkai, how attention can also shape VSTM during maintenance McCollough, & Vogel, 2010; Fukuda & Vogel, 2009; Drew according to changing task-goals. & Vogel, 2008; Jolicœur, Brisson, & Robitaille, 2008; Accumulating neural evidence has revealed that the main- McCollough, Machizawa, & Vogel, 2007; Robitaille & tenance of information in VSTM may involve stimulus- Jolicoeur, 2006; Vogel, McCollough, & Machizawa, 2005; related perceptual codes in sensory brain areas (Munneke, Vogel & Machizawa, 2004). This is a sustained lateralized Heslenfeld, & Theeuwes, 2010; Ester, Serences, & Awh, negativity over posterior electrodes, which persists through- 2009; Harrison & Tong, 2009; Serences, Ester, Vogel, & out the mnemonic delay period and scales with the number Awh, 2009; Stokes, Thompson, Cusack, & Duncan, 2009; of items being maintained (Anderson, Vogel, & Awh, 2011; Pasternak & Greenlee, 2005; Druzgal & DʼEsposito, 2001, Bor & Owen, 2006; Vogel et al., 2005; Vogel & Machizawa, 2003; Postle, Druzgal, & DʼEsposito,2003;Chelazzi,Miller, 2004). It is thought to reflect delay activity underlying VSTM Duncan,&Desimone,1993).Accordingtothisview,VSTM maintenance in visual extrastriate and possibly parietal maintenance is at least partly mediated by persistent activity areas, although the neural sources of the effect still require of neural populations that represent the perceptual charac- further substantiation. Importantly, mnemonic delay ac- tivity reflects maintenance for task-relevant items. Before the presentation of the memory array, participants are University of Oxford cued to encode stimuli from only one visual hemifield.

© 2011 Massachusetts Institute of Technology Journal of Cognitive 24:1, pp. 51–60 During the subsequent maintenance interval, neural activ- (spatial) or noninformative (neutral) cues after the mem- ity is greater in the posterior region that is contralateral ory array, yet before the test probe. These retrocues in- to the location of items in VSTM, relative to the ipsilateral dicated the location of the single item that would be region, reflecting selectivity for the task-relevant items required to perform the subsequent comparison with held in VSTM. the probe stimulus, thus effectively reducing the task- If task-goals change dynamically after the initial encod- relevant load from multiple items to just one item. Impor- ing of the events, stimuli maintained in VSTM may gain or tantly, because this predictive information is presented lose relevance; therefore, a mechanism for dynamic mod- long after the offset of the memory array, retrocues can- ulation of specific memoranda would be highly advanta- not be used to bias initial processing of the visual stimulus Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/24/1/51/1780230/jocn_a_00087.pdf by MIT Libraries user on 17 May 2021 geous in optimizing and controlling the limited contents for selective access to VSTM but may influence informa- of VSTM. Recent behavioral evidence demonstrates that tion that is already being maintained in VSTM. attention can be directed to specific items held in VSTM Consistent with previous studies (Nobre, Griffin, & Rao, (Vandenbroucke, Sligte, & Lamme, 2011; Makovski, 2008; Griffin & Nobre, 2003), we found that recall was faster Sussman, & Jiang, 2008; Lepsien & Nobre, 2007; Makovski and more accurate for cued items, relative to uncued items.

& Jiang, 2007; Matsukura, Luck, & Vecera, 2007; Yeh, Kuo, More importantly, we found that load-dependent neural Downloaded from http://direct.mit.edu/jocn/article-pdf/24/1/51/1943573/jocn_a_00087.pdf by guest on 03 October 2021 & Liu, 2007; Makovski, Shim, & Jiang, 2006; Griffin & Nobre, activity reflecting VSTM maintenance could be modulated 2003; Landman, Spekreijse, & Lamme, 2003). Presenting a by spatial cues that effectively reduced the task-relevant spatial cue during the delay period to indicate which item memoranda to one item across two experiments. Our in VSTM is likely to be probed dramatically improves VSTM results provide direct evidence that top–down control di- performance for cued relative to uncued items. Critically, rectly modulates neural activity associated with mainte- these retroactive cues (retrocues) are presented long after nance in VSTM. iconic memory decay and, therefore, do not influence ini- tial encoding into VSTM. However, the mechanisms by which selective attention can bias maintenance in VSTM METHODS remain largely unknown. Experiment 1 As mentioned above, current evidence suggests that VSTM is mediated by sustained activation of memoranda- Participants specific perceptual representations in posterior brain areas. All participants in this study were right-handed according A perceptual basis for VSTM could provide a natural mech- to the Edinburgh handedness inventory (Oldfield, 1971). anism for dynamic biasing even after initial encoding into Twenty-five participants were recruited. They had normal VSTM. In particular, similar attentional mechanisms that or corrected-to-normal visual acuity, provided informed bias competitive processing during may oper- written consent, and were financially reimbursed for their ate upon activity in these posterior areas when they are time. Data from seven participants were excluded because supporting VSTM representations (DellʼAcqua, Sessa, of poor performance on the task (<60% correct trials) or too Toffanin, Luria, & Jolicoeur, 2010; Eimer & Kiss, 2010; few trials remaining after EEG artifact rejection (<25 trials). Kuo, Rao, Lepsien, & Nobre, 2009). For example, shifts of The behavioral and ERP analyses were performed on the attention during VSTM maintenance could filter out items remaining 18 participants (nine women, age range = 21– as they become task irrelevant, thereby reducing memory 33 years, mean age = 27 years). All experimental methods load and inter-item competition (Edin et al., 2009) and con- had ethical approval from the Central University Research sequently increasing the probability of recall for the rele- Ethics Committee of the University of Oxford. vant cued items. In this study, we tested whether top–down attentional Behavioral Task signals can continue to modulate persistent delay activity underlying VSTM across two experiments. We exploited The task of the first experiment is illustrated in Figure 1A. the CDA as an index of VSTM maintenance to test whether Retrocue type (spatial, neutral) and VSTM load (two-item, the load of items being actively maintained can be ad- four-item) were manipulated within participants, in a two- justed dynamically as new information becomes available, way repeated measures factorial design. signaling which memoranda are most relevant for task Stimuli were presented on a CRT screen using Presenta- performance. As common practice in experiments using tion software (Neurobehavioral Systems, Inc., Albany, CA). the CDA measure, at the beginning of each trial partici- Each trial began with the onset of a centrally displayed pants viewed an arrow cue, instructing them to only en- asterisk (500–1000 msec duration, randomized), which sig- code items in either the left or right visual hemifield. Next, naled the onset of the trial. Next, an arrow pointing to a memory array of multiple colored squares was pre- either the left or right visual hemifield was presented at sented within each visual hemifield. After a retention in- the center of the screen for 200 msec, followed, at a vari- terval, a test stimulus was presented to probe the contents able interstimulus interval (500–1000 msec duration), by the of VSTM. During this retention interval, attentional orient- memory array consisting of two or four colored squares in ing was manipulated by providing spatially informative each hemifield. Participants were instructed to remember

52 Journal of Volume 24, Number 1 Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/24/1/51/1780230/jocn_a_00087.pdf by MIT Libraries user on 17 May 2021 Downloaded from http://direct.mit.edu/jocn/article-pdf/24/1/51/1943573/jocn_a_00087.pdf by guest on 03 October 2021

0 Figure 1. Schematics of experimental trials: VSTM task (top of each panel) used in (A) Experiment 1 and (B) Experiment 2. Sensitivity measures (d ), Pashler–Cowan K measures (K ), and mean response times (RT, msec of each panel) were estimated (bottom of each panel). Error bars represent standard errors of the means. as many items as possible that were presented within the between trials, which included a 1000-msec response cued hemifield. There were eight positions for the presen- period, varied randomly between 2000 and 2500 msec. tation of a memory array (four positions in each hemifield), All trial types were equiprobable and randomized within each arranged according to two imaginary concentric circles 16 blocks of 32 trials, yielding 512 trials in total (64 target- with radii of approximately 3.06° and 5.44° of visual angle. present and 64 target-absent trials in each retrocue type and When 800 msec of the delay period had elapsed, a ret- VSTM load condition). Participants first completed one rocue appeared for 200 msec at the center of the screen practice block of 32 trials, and they were encouraged to rest [spatial retrocue (50%), neutral retrocue (50%)]. In spa- between blocks during the main experiment. Participants tial cue trials, the cue was presented by highlighting two were comfortably seated in a dimly illuminated, electrically adjacent sides of the central cross, thereby pointing to one shielded room, facing a computer monitor placed 100 cm of four quadrants within the relevant hemifield, thereby in- in front of them. They were instructed to maintain fixation dicating one specific location within the array being main- on a small fixation marker at the center of the monitor tained. In neutral trials, the cue consisted of a uniform during the experimental trials and to respond as quickly brightening of the cross, therefore providing no specific and accurately as possible. Participants responded using spatial information. After a randomized variable interval fol- their right hand by pressing mouse button. Participants lowing the retrocue (600–1000 msec duration for postcue were also asked to minimize blinking and moving their eyes interval), a colored square was presented at central fixation while performing each trial throughout the experiment. for 200 msec. Participants decided whether the test probe All visual stimuli were presented against a black back- was the same as the cued item from the previous memory ground, and a central fixation point was present through- array, responding match (50%) or nonmatch (50%) with out the experiment. The color of each memory item was the left or right mouse button, respectively. The interval randomly selected from a set of eight possible colors (red,

Kuo, Stokes, and Nobre 53 green, yellow, blue, gray, pink, purple, and brown), and were derived by computing the difference between the the size and position of each item was computed accord- voltages at electrodes placed to the side of each eye ing to the human cortical magnification factor (Cowey & (HEOG) and above and below the left eye (VEOG). The Rolls, 1974). Memory items presented at 3.06° and 5.44° of re-referenced and transformed continuous data were visual angle from central fixation subtended 0.77° and then low-pass filtered (40 Hz) to exclude high-frequency 1.36° of visual angle, respectively. The test item presented noise. at fixation subtended 0.5° of visual angle in size. The continuous EEG was then segmented into epochs, time-locked at the onset of the memory array. Epochs were 2000-msec long, beginning 200 msec before the on- Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/24/1/51/1780230/jocn_a_00087.pdf by MIT Libraries user on 17 May 2021 Behavioral Analyses set of the memory array and ending after 1800 msec, which Behavioral measures, including sensitivity score for corresponded to the minimal time point before the presen- 0 match/no-match discrimination (d score; Green & Swets, tation of the probe stimulus. The prestimulus baseline of 1966), memory capacity measurement (Pashler–Cowan K 200 msec was used for all analyses. Data quality was initially measure; Cowan, 2001; Pashler, 1988), and RTs were each inspected using automated algorithms. Epochs containing μ analyzed by a repeated measures ANOVA with two factors: excessive noise or drift (±100 V) at any electrode were Downloaded from http://direct.mit.edu/jocn/article-pdf/24/1/51/1943573/jocn_a_00087.pdf by guest on 03 October 2021 Retrocue Type (spatial and neutral) and VSTM Load (two- excluded, and epochs with eye movement artifacts (blinks 0 item and four-item). The d score was calculated using the or saccades) were also rejected. Blinks were identified as 0 following equation: d = Z (hit rate) − Z (false alarm rate). large deflections (±50 μV) in the HEOG or VEOG electro- The K measure was calculated using the following equa- des. Visual inspection was also carried out to confirm ap- tion: K = S (set size of the initial array) × (hit rate − false propriate removal of artifacts and to identify residual 0 alarm rate). For d score and K measure, hit rate was de- saccades in the individual HEOG traces. Trials with incor- fined as the conditional probability that the participants rect behavioral responses were also discarded from the responded “target-present” given that the target was final analyses. presented and the false-alarm rate was defined as the Epochs were then averaged according to retrocue type, conditional probability that the participants responded VSTM load, and visual hemifield. ERPs were derived from “target-present” when the target was absent. Only cor- both target-present and target-absent trials. ERPs from rect responses were included for RT analyses. trials containing targets located in the right and left hemi- field were combined by an averaging procedure that pre- serves the electrode location relative to the target side EEG Recording (contralateral or ipsilateral). To maintain an acceptable The EEG was recorded continuously using NuAmp ampli- signal-to-noise ratio, a lower limit of 25 artifact-free trials fiers (Neuroscan, Inc., Charlotte, NC) from 34 silver/silver per subject per condition was set. chloride electrodes placed on the scalp with an elasticated cap, positioned according to the 10–20 international sys- EEG Analysis tem (AEEGS, 1991). The montage included six midline sites (FZ, FCZ, CZ, CPZ, PZ, and OZ) and 14 sites over The aim of this study was to test whether the information each hemisphere (FP1/FP2, F3/F4, F7/F8, FC3/FC4, FT7/ carried by spatial retrocues modulates persistent delay FT8, C3/C4, T7/T8, CP3/CP4, TP7/TP8, P3/P4, P7/P8, PO3/ activity, namely, CDA. The mean amplitudes of the delay PO4, PO7/PO8, and O1/O2). Vertical eye movements activity were computed between 500 and 800 msec dur- were recorded by electrodes placed on the supraorbital ing precue interval (before appearance of retrocue) and and infraorbital ridges of the right eye (vertical electro- between 1500 and 1800 msec during postcue interval oculogram [VEOG]), and horizontal eye movements by (500–800 msec after appearance of retrocue) at the parietal– electrodes placed on the outer canthi of the right and left occipital electrodes (PO7/PO8) contralateral and ipsilateral eyes (horizontal electrooculogram [HEOG]). Additional to the side of the target (McCollough et al., 2007; Vogel electrodes were used as ground and reference sites. Elec- et al., 2005; Vogel & Machizawa, 2004). A three-way re- trodes were referenced to the right mastoid site (A2) dur- peated measures ANOVA was computed on the mean am- ing recording. The electrode between FPZ and FZ (AFZ) on plitudes of the delay activity, testing the effects of retrocue the midline served as the ground electrode. Electrode im- type (spatial and neutral), VSTM load (two-item and four- pedances were kept below 5 kΩ. The ongoing brain activity item), and visual hemifield (contralateral and ipsilateral to at each electrode site was sampled every 1 msec (1000-Hz target). Of main interest was the three-way interaction analogue-to-digital sampling rate). Activity was filtered with among retrocue type, VSTM load, and visual hemifield, a low-pass filter of 300 Hz, and no high-pass filter was used. which would indicate a change in the size of the lateralized CDA after a spatial retrocue. The Greenhouse–Geisser epsilon correction for nonsphericity was applied to all EEG Processing ERP analyses where appropriate (Jennings & Wood, 1976), The EEG was re-referenced off-line to the algebraic aver- and only corrected probability values and degrees of free- age of the right and left mastoids. Bipolar EOG signals dom were reported.

54 Journal of Cognitive Neuroscience Volume 24, Number 1 Results effect of Visual Hemifield with a greater mean negative amplitude on the contralateral side relative to ipsilateral Behavioral results. The behavioral results are sum- sides (F(1, 17) = 37.73, p < .005). As predicted, we found marized in Figure 1 and Table 1. A significant retrocueing the CDA to be load-dependent. We observed a significant effect was observed across all measures, with higher 0 interaction of VSTM Load and Visual Hemifield [F(1, 17) = mean d scores [F(1, 17) = 58.70, p < .005], higher mean 20.56, p < .005], owing to a greater increase in voltage K measures [F(1, 17) = 76.72, p < .005], and faster RT negativity over posterior electrodes that were contralateral [F(1, 17) = 252.93, p < .005] in spatial trials (2.34 ± 0.71 0 to the memorized hemifield relative to the ipsilateral hemi- d score, 2.05 ± 0.54 K value, 528.46 ± 87.71 msec in RT) Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/24/1/51/1780230/jocn_a_00087.pdf by MIT Libraries user on 17 May 2021 0 field within increasing VSTM load (two-item trial: 0.97 ± relative to neutral trials (1.83 ± 0.78 d score, 1.59 ± 0.42 0.97 μV, four-item trial: 1.70 ± 0.99 μV) [t(17) = 4.54, p < K value, 648.20 ± 98.02 msec in RT). We also observed a .005]. There was also a significant main effect of VSTM significant main effect of VSTM Load across all measures, 0 Load, attributable to a greater negative amplitude for four- showing higher mean d scores [F(1, 17) = 113.16, p < item (−2.46 ± 2.58 μV) relative to two-item trials (−1.45 ± .005], lower K measures [F(1, 17) = 21.91, p < .005], 2.07 μV) [F(1, 17) = 15.13, p < .005]. and faster RT [F(1, 17) = 98.85, p < .005], in two-item

0 The main hypothesis of interest was whether the load- Downloaded from http://direct.mit.edu/jocn/article-pdf/24/1/51/1943573/jocn_a_00087.pdf by guest on 03 October 2021 trials (2.66 ± 0.52 d score, 1.59 ± 0.17 K value, 551.41 ± dependent neural activity reflecting VSTM maintenance 90.89 msec in RT) relative to four-item trials (1.52 ± 0.56 0 (i.e., CDA) could be modulated dynamically by spatial d score, 2.06 ± 0.66 K value, 625.25 ± 116.12 msec in RT). cues that effectively reduced the task-relevant memo- Finally, the interaction between Retrocue Type and VSTM 0 randa to one item. Accordingly, we tested whether the Load was significant in all measures [d score: F(1, 17) = amplitude of CDA was attenuated after a spatial cue rela- 6.12, p <.05;K measure: F(1, 17) = 40.91, p < .005; RT: tive to the neutral cue during the postcue interval (1500– F(1, 17) = 9.16, p < .05], owing to a reduction of the load 0 1800 msec). Direct comparison of the voltage difference effect associated with spatial retrocues [increased d score: between ipsilateral versus contralateral side indicated a t(17) = 2.46, p < .05; enhanced K measure: t(17) = 6.44, significant main effect of Load [F(1, 17) = 8.17, p < p < .005; faster RT: t(17) = 3.03, p < .05]. These behav- .05], showing greater mean amplitude for four-item ioral results confirmed that participants were able to orient (0.28 ± 1.09 μV) relative to two-item trials (−0.22 ± their attention toward the item held in VSTM, resulting in 0.90 μV) [t(17) = 2.86, p < .05]. More importantly, we benefits of VSTM performance and consequently attenuat- observed a significant three-way interaction among Retro- ing the load effect. cue Type, VSTM Load, and Visual Hemifield [F(1, 17) = 4.35, p = .05]. Follow-up analyses were conducted to clar- ERP results. The ERP results are plotted in Figure 2. ify the pattern of interaction effect. Analysis of neutral First, we tested for the presence of CDA during the pre- trials still revealed a significant load effect [t(17) = cue interval (500–800 msec) and found a significant main 3.42, p < .005] (four-item trial: 0.67 ± 1.20 μV; two-item trial: −0.32 ± 1.45 μV). In contrast, no difference was found in amplitude of CDA between four-item (−0.11 ± Table 1. Mean d0 Scores, K Measures, and RTs (in msec) and 1.23 μV) and two-item (−0.12 ± 1.04 μV) spatial cue trials Standard Deviations of the Means of Correct Responses in Each ( p > .1). Overall, the results highlighted that the difference Condition in the CDA between two and four items was reduced after a μ Experiment 1 spatial cue (0.01 ± 1.26 V) in contrast to a neutral cue (1.0 ± 1.23 μV) [t(17) = 2.09, p = .05], revealing a greater reduc- Neutral Cue Spatial Cue tion in CDA after a spatial cue than after a neutral cue. 0 d Score 2-item 2.48 ± 0.44 2.84 ± 0.56 To test for a functional link between the neural activity reflected in the CDA and VSTM performance in our task, 4-item 1.19 ± 0.46 1.85 ± 0.47 we also examined the correlation between the increase in K measure 2-item 1.53 ± 0.16 1.65 ± 0.16 amplitude of CDA between two and four items and the increase in K measured on neutral trials between two 4-item 1.66 ± 0.58 2.45 ± 0.51 and four items across participants (Vogel & Machizawa, RT (msec) 2-item 603.38 ± 84.22 499.44 ± 68.40 2004). During the precue interval, there was a significant 4-item 693.01 ± 94.86 557.48 ± 99.15 relationship between the set size effect on CDA amplitudes and K [Pearson correlation: r(17) = 0.63, p < .05]. This Experiment 2 finding showed that individual differences in VSTM capac- No Cue Neutral Cue Spatial Cue ity can be reflected in the measures of delay activity, repli- 0 cating previous reports of correlations between CDA and d Score 1.11 ± 0.43 1.22 ± 0.31 1.84 ± 0.57 VSTM capacity (Vogel & Machizawa, 2004). We also ob- K measure 1.59 ± 0.45 1.74 ± 0.38 2.45 ± 0.64 served a significant correlation on neutral trials in the inter- val after the cue was presented [r(17) = 0.43, p < .05], RT (msec) 745.63 ± 64.11 695.27 ± 71.14 538.86 ± 71.83 further demonstrating that the neutral cue did not alter

Kuo, Stokes, and Nobre 55 Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/24/1/51/1780230/jocn_a_00087.pdf by MIT Libraries user on 17 May 2021 Downloaded from http://direct.mit.edu/jocn/article-pdf/24/1/51/1943573/jocn_a_00087.pdf by guest on 03 October 2021

Figure 2. The ERP results of Experiment 1. (A) ERP waveforms averaged across all participants are shown for the four items for neutral (left ) and spatial retrocue trials (right) over contralateral (blue lines) and ipsilateral (red lines) posterior parietal–occipital electrode pair: PO7/PO8. Our result shows equivalent CDA for neutral and spatial trials during the precue interval (500–800 msec). The amplitude of CDA is attenuated after a spatial cue, relative to the neutral cue, during the postcue interval (1500–1800 msec). (B) The voltage difference between contralateral versus ipsilateral side is also shown for two items (light green lines) and four items (dark green lines) in the neutral and spatial retrocue conditions, respectively. CDA is load dependent, showing greater CDA for four items than two items during the precue interval (500–800 msec). Spatial retrocues reduced the magnitude of the CDA during the postcue interval (1500–1800 msec). The temporal windows for CDA analyses are indicated by gray squares.

the relationship between the maintenance-related delay ac- To focus on the modulatory effect of spatial and neu- tivity and capacity estimates. tral retrocues in the postcue interval more clearly, the spatial or neutral retrocue was presented at the time we found the CDA to be around its maximal amplitude Experiment 2 in Experiment 1: 600–800 msec after the memory array. The earlier presentation of the retrocue enabled us to test The goals of Experiment 2 were to replicate our finding the modulatory effect over an extended period and to ob- that VSTM maintenance could be dynamically modulated tain a more stable measure of the CDA during the postcue by spatial attention and to ensure that these effects did interval (1100–1800 msec duration; see Figure 1B for an ex- not simply result from a potential disruption of VSTM ample). The sequence of events in each trial and all other maintenance by the presentation of a neutral cue. To rule procedures for EEG recording, processing, and analyses out any effect related to the mere presentation of a visual were the same as the previous experiment. stimulus as a cue, we included an additional no-cue con- dition. In Experiment 2, the memory array always con- sisted of four colored squares in each hemifield. In the Participants no-cue control condition and in the neutral retrocue con- dition, stimuli maintained in VSTM did not gain or lose All participants in this study were right-handed according relevance. As in Experiment 1, retrocue type was manipu- to the Edinburgh handedness inventory (Oldfield, 1971). lated within participants in a one-way repeated measures Sixteen participants were recruited. They had normal or design. All trial types were equiprobable and randomized, corrected-to-normal visual acuity, provided informed and presentation order was randomized within 16 blocks written consent, and were financially reimbursed for their of 24 trials, yielding 384 trials in total (64 target-present time. Data from one participant was excluded, owing to too and 64 target-absent trials in each retrocue type). few trials remaining after EEG artifact rejection (<25 trials).

56 Journal of Cognitive Neuroscience Volume 24, Number 1 The behavioral and ERP analyses were performed on the ter performance in spatial trials compared with neutral remaining 15 participants (eight women and seven men, and no-cue trials. We observed a significant main effect of 0 age range = 21–33 years, mean age = 26.5 years). All experi- Retrocue Types across all measures [d scores: F(2, 14) = mental methods had ethical approval from the Central Univer- 19.45, p < .005; K measures: F(2, 14) = 19.70, p < .005; RT: sity Research Ethics Committee of the University of Oxford. F(2, 14) = 203.71, p < .005]. Follow-up comparisons 0 showed higher mean d scores, higher K measure, and faster 0 Behavioral Analysis RT in spatial trials than in both neutral [d score: t(14) = 0 5.11, p < . 005; K value: t(14) = 4.89, p < .005, RT: t(14) = Each behavioral measure (d score, K measure, and RT) 0 Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/24/1/51/1780230/jocn_a_00087.pdf by MIT Libraries user on 17 May 2021 12.00, p < .005] and no-cue trials [d score: t(14) = 4.80, p < was analyzed using a one-way repeated measures ANOVA .005, K value: t(14) = 5.12, p < .005, RT: t(14) = 18.00, p < (Retrocue Type: spatial, neutral, and no cue). .005]. Faster RT was also observed in neutral trials than in no-cue trials [t(14) = 7.92, p < .005]. No other significant EEG Analysis 0 effect was observed for d scores or K values ( ps>.1).As The mean amplitudes of the delay activity were computed in Experiment 1, participants were capable of orienting – between 1100 and 1800 msec (500 1200 msec after appear- their attention toward the item held in VSTM, resulting in Downloaded from http://direct.mit.edu/jocn/article-pdf/24/1/51/1943573/jocn_a_00087.pdf by guest on 03 October 2021 ance of retrocue) during postcue interval at PO7/PO8 con- benefits of VSTM performance. tralateral and ipsilateral to the side of the target. A two-way repeated measures ANOVA was computed on the mean ERP results. We tested whether the magnitude of CDA amplitudes of the delay activity, testing the effects of Retro- could be influenced by a neutral cue as well as a spatial cue, cue Type (spatial, neutral, and no cue) and Visual Hemifield relative to the no-cue condition. The ERP results are (contralateral and ipsilateral to target). Of main interest plotted in Figure 3. We first demonstrated a significant was the interaction between Retrocue Type and Visual main effect of Visual Hemifield [F(1, 14) = 40.91, p < Hemifield. .005] and, more importantly, a significant interaction be- tween Retrocue Type and Visual Hemifield [F(1, 14) = 16.45, p < .005]. Follow-up analyses showed that the differ- Results ence in voltage for the ipsilateral side in contrast to contra- Behavioral results. The behavioral results are summar- lateral side was significantly larger in both neutral [t(14) = ized in Figure 1 and Table 1. Overall, participants had bet- 3.32, p = .005] (0.75 ± 0.48 μV) and no-cue trials (1.46 ±

Figure 3. The ERP results of Experiment 2. ERP waveforms averaged across all participants are shown for no-cue (left), neutral (middle), and spatial (right) cue conditions over contralateral (waveforms: blue lines; topographies: right side) and ipsilateral (waveforms: red lines; topographies: left side) posterior parietal–occipital electrode pair: PO7/PO8. The amplitude of CDA is attenuated after a spatial cue, relative to the neutral cue and no-cue condition, during the postcue interval (1100–1800 msec). The voltage difference between contralateral versus ipsilateral side is also shown (waveforms: green lines). The temporal windows for CDA analyses are indicated by gray squares. The topographic maps isolated the lateralized differences in voltage between contralateral (CO) and ipsilateral (IP) sites. These maps show the symmetrical relative differences in voltage, which is more negative over the contralateral scalp and more positive over the ipsilateral scalp. The voltage distributions are shown from posterior perspective. The color scale shows the range of possible voltage values. Blue indicates negative voltage, and red indicates positive voltage.

Kuo, Stokes, and Nobre 57 0.90 μV) [t(14) = 4.67, p < .005] than in the spatial trials More importantly, our ERP data revealed that the mag- (−0.42 ± 0.99 μV). We also observed larger CDA for no- nitude of persistent activity was sharply attenuated by cue trials in contrast to neutral cue trials [t(14) = 3.43, appearance of spatial retrocues. This finding supports p < .005], although both were significantly greater than the hypothesis that top–down attention in VSTM may share zero [neutral: t(14) = 6.04, p < .005; no cue: t(14) = properties with attentional mechanisms that modulate 6.27, p <.005]. perceptual analysis to bias competition in favor of the task-relevant information. Previously, Lepsien and Nobre (2007) used fMRI to test for modulation of maintenance- related activity in an object-based VSTM task. They showed Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/24/1/51/1780230/jocn_a_00087.pdf by MIT Libraries user on 17 May 2021 DISCUSSION that retrocues signaling the relevance of the face or scene It is now well established that maintenance of information in the previous two-item array for performance of a sub- in VSTM is associated with persistent activity in neural sequent probe-match comparison modulated activity in ensembles in posterior brain regions that represent the perceptual areas preferentially processing faces (posterior perceptual characteristics of mnemonic information (see fusiform gyrus) or scenes (parahippocampal gyrus). How-

Pasternak & Greenlee, 2005, for a review). In this study, ever, these results were ambiguous. Their task design Downloaded from http://direct.mit.edu/jocn/article-pdf/24/1/51/1943573/jocn_a_00087.pdf by guest on 03 October 2021 we tested whether top–down attention can directly modu- made it unclear whether the cue-related modulations with- late the maintenance of VSTM representations across two in these areas reflected changes to maintenance-related ERP experiments. By manipulating the spatially predictive activity or anticipation of a specific category of probe stim- information of an attentional retrocue, we showed that ulus (face or scene). Indeed, in a recent follow-up study, maintenance-related activity is modulated by changes in Lepsien, Thornton, and Nobre (2011) provides further evi- the task relevance of particular items in VSTM, as indicated dence that anticipatory attention to relevant probe items by spatially informative retrocues. Specifically, spatial retro- can influence activity in visual areas. We were careful, cues reduced the magnitude of persistent delay activity, therefore, to design our task in a way that would preclude consistent with a reduction in the effective memory load. any effect of anticipatory spatial attention to the cue or probe This neural modulation was also consistent with behavioral arrays. benefits of spatial retrocueing. Presumably, reducing the In our task, the changes in the magnitude of lateralized number of task-irrelevant items stored in VSTM increases neural activity in this task can only reflect changes in spa- the probability of recall for the cued item. tiotopic VSTM maintenance during the retention period. The pattern of behavioral results replicates and extends Spatial and neutral retrocues were presented centrally previous studies, confirming that VSTM performance can and, therefore, should not result in changes in the laterali- be regulated by orienting attention to the internal repre- zation of neural activity. The continued correlation between sentation of the task-relevant item (Griffin & Nobre, 2003; CDA amplitude and capacity measures after neutral cues Landman et al., 2003). Spatial retrocues were accompa- confirms this to be the case. Probe stimuli also appeared 0 nied by faster RT, higher d scores, and K measures relative centrally and did not differ across the conditions of interest. to neutral retrocue and no-cue conditions. This behavioral A single color probe stimulus was presented at fixation, and facilitation associated with spatial retrocues was more the participant had to decide whether it was one of the items pronounced as VSTM load increased (Nobre et al., 2008; in the initial memory array. There was no basis, therefore, Lepsien & Nobre, 2006), consistent with the hypothesis for the formation of any anticipatory spatial bias that could that attentional selection during VSTM maintenance effec- interfere with our CDA measure. Although lateralized mark- tively reduces the number of items that need to be re- ers of VSTM markers and spatial attention can often co- tained. These behavioral results are in accordance with occur in many experimental designs (for a discussion, see previous studies that have demonstrated that attentional Stokes, 2011), these were de-coupled within the present retrocues presented beyond the phase of VSTM encoding task design. can still shape internal representations (Nobre et al., 2008), Our results also revealed some attenuation of the CDA in providing further evidence that VSTM representations neutral retrocue trials, relative to no-cue trials in Experi- can be modulated dynamically to accommodate changing ment 2. We speculate that the CDA may reflect neural ac- task-goals. tivity correlated to tonic firing in extrastriate visual areas Our result is also in line with previous evidence that during the delay, which is known to be sensitive to percep- CDA reflects VSTM maintenance and its magnitude corre- tual interference (Miller et al., 1993). Interestingly, this lates with VSTM capacity (Vogel & Machizawa, 2004). The CDA decrement was not accompanied by a change in accu- strength of persistent activity varied as a function of VSTM racy or sensitivity. load before appearance of retrocues. Moreover, the corre- The nature of VSTM is still not fully understood. How- lation between the CDA and capacity estimates was still ever, recent studies have suggested that VSTM may involve evident after appearance of the neutral cue, demonstrat- similar neural codes to those that mediate perceptual infor- ing that the CDA continuous to provide a valid index of mation (DellʼAcqua et al., 2010; Astle, Scerif, Kuo, & Nobre, VSTM maintenance, even after the presentation of a po- 2009; Kuo et al., 2009; Jiang, Olson, & Chun, 2000; Gratton, tentially distracting visual stimulus. 1998). We suggest that the shared neural organization for

58 Journal of Cognitive Neuroscience Volume 24, Number 1 perceptual and VSTM representations provides a common memory-guided visual search. Journal of , – framework for top–down attentional modulations that 80, 2918 2940. Chelazzi, L., Miller, E. K., Duncan, J., & Desimone, R. (1993). A optimize task-relevant processing during multiple domains neural basis for visual search in inferior temporal cortex. of processing—perception, VSTM, and possibly beyond. Nature, 363, 345–347. Top–down attentional signals during VSTM maintenance Cowan, N. (2001). The magical number 4 in short-term can bias internal representations on the basis of the original memory: A reconsideration of mental storage capacity. – spatial configuration of the perceptual inputs. Behavioral and Brain Sciences, 24, 87 114. Cowey, A., & Rolls, E. T. (1974). Human cortical magnification In conclusion, evidence that retrocues can modulate

factor and its relation to visual acuity. Experimental Brain Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/24/1/51/1780230/jocn_a_00087.pdf by MIT Libraries user on 17 May 2021 VSTM maintenance through top–down attentional orient- Research, 21, 447–454. ing has important implications for views about the nature Curtis, C. E., & DʼEsposito, M. (2003). Persistent activity in of VSTM. In particular, our findings provide further sup- the prefrontal cortex during working memory. Trends in – port for the view that VSTM representations are flexible Cognitive Sciences, 7, 415 423. DellʼAcqua, R., Sessa, P., Toffanin, P., Luria, R., & Jolicoeur, P. and can be modulated dynamically according to changing (2010). Orienting attention to objects in visual short-term goals and expectations (Kuo, Yeh, Chen, & DʼEsposito, memory. Neuropsychologia, 48, 419–428.

2011; Kuo et al., 2009; Lepsien & Nobre, 2006, 2007). Drew, T., & Vogel, E. K. (2008). Neural measures of individual Downloaded from http://direct.mit.edu/jocn/article-pdf/24/1/51/1943573/jocn_a_00087.pdf by guest on 03 October 2021 Top–down attentional orienting can modulate the main- differences in selecting and tracking multiple moving – tenance of the short-lived representations within VSTM objects. Journal of Neuroscience, 28, 4183 4191. Druzgal, T. J., & DʼEsposito, M. (2001). Activity in fusiform and bias competition in a favor of the most task-relevant face area modulated as a function of working memory load. information. We suggest that dynamic modulation of Cognitive Brain Research, 10, 355–364. maintenance-related activity is likely to operate in conjunc- Druzgal, T. J., & DʼEsposito, M. (2003). Dissecting contributions tion with other optimizing mechanisms, such as attention- of prefrontal cortex and fusiform face area to face working – dependent encoding into VSTM (Murray et al., 2011; memory. Journal of Cognitive Neuroscience, 15, 771 784. Edin, F., Klingberg, T., Johansson, P., McNab, F., Tegnér, J., & Schmidt et al., 2002) and selective biasing of search/retrieval Compte, A. (2009). Mechanism for top–down control of processes (Nobre et al., 2008). As in perception, we argue working memory capacity. Proceedings of the National that goal-dependent biases do not operate at a single Academy of Sciences, U.S.A., 106, 6802–6807. bottleneck but at multiple stages between stimulus and Eimer, M., & Kiss, M. (2010). An electrophysiological measure response, depending on how task-goals and expectations of access to representations in visual working memory. Psychophysiology, 47, 197–200. are determined and how they unfold over time. Ester, E. F., Serences, J. T., & Awh, E. (2009). Spatially global representations in human primary visual cortex during working memory maintenance. Journal of Neuroscience, Acknowledgments 29, 15258–15265. This study was supported by a project grant from the Wellcome Fukuda, K., & Vogel, E. K. (2009). Human variation in ʼ overriding attentional capture. Journal of Neuroscience, Trust (082791/Z/07/Z) to A. C. N.; M. G. S. was supported by St. John s – College, Oxford. We thank Alexandra Murray for her comments 29, 8726 8733. and suggestions on an early version of the manuscript. Gazzaley, A. (2011). Influence of early attentional modulation on working memory. Neuropsychologia, 49, 1410–1424. Reprint requests should be sent to Bo-Cheng Kuo, Department Gratton, G. (1998). The contralateral organization of visual of Experimental Psychology, University of Oxford, South Parks memory: A theoretical concept and a research tool. Road, Oxford OX1 3UD, UK, or via e-mail: bo-cheng.kuo@psy. Psychophysiology, 35, 638–647. ox.ac.uk. Green, D. M., & Swets, J. A. (1966). Signal detection theory and psychophysics. Huntington, NY: Robert E. Krieger Publishing Company. REFERENCES Griffin, I. C., & Nobre, A. C. (2003). Orienting attention to locations in internal representations. Journal of Cognitive AEEGS. (1991). American Electroencephalographic Society Neuroscience, 15, 1176–1194. guidelines for standard electrode position nomenclature. Harrison, S. A., & Tong, F. (2009). Decoding reveals the Journal of Clinical Neurophysiology, 8, 200–202. contents of visual working memory in early visual areas. Anderson, D. E., Vogel, E. K., & Awh, E. (2011). Precision in Nature, 458, 632–635. visual working memory reaches a stable plateau when Ikkai, A., McCollough, A. W., & Vogel, E. K. (2010). individual item limits are exceeded. Journal of Neuroscience, Contralateral delay activity provides a neural measure 31, 1128–1138. of the number of representations in visual working Astle, D. E., Scerif, G., Kuo, B.-C., & Nobre, A. C. (2009). memory. Journal of Neurophysiology, 103, 1963–1968. Spatial selection of features within perceived and Jennings, J. R., & Wood, C. C. (1976). Letter: The epsilon- remembered objects. Frontiers in Human Neuroscience, adjustment procedure for repeated-measures analyses of 3, 1–9. variance. Psychophysiology, 13, 277–278. Awh, E., & Jonides, J. (2001). Overlapping mechanisms of Jiang, Y. V., Olson, I. R., & Chun, M. M. (2000). Organization attention and spatial working memory. Trends in Cognitive of visual-short term memory. Journal of Experimental Sciences, 5, 119–126. Psychology: , Memory, & Cognition, 26, 683–702. Bor, D., & Owen, A. M. (2006). Working memory: Linking Jolicœur, P., Brisson, B., & Robitaille, N. (2008). Dissociation capacity with selectivity. Current Biology, 16, R136–R138. of the N2pc and sustained posterior contralateral negativity Chelazzi, L., Duncan, J., Miller, E. K., & Desimone, R. (1998). in a choice response task. Brain Research, 1215, Responses of neurons in inferior temporal cortex during 160–172.

Kuo, Stokes, and Nobre 59 Kuo, B.-C., Rao, A., Lepsien, J., & Nobre, A. C. (2009). Searching handedness: The Edinburgh inventory. Neuropsychologia, for targets within the spatial layout of visual short-term 9, 97–113. memory. Journal of Neuroscience, 29, 8032–8038. Pashler, H. (1988). Familiarity and visual change detection. Kuo, B.-C., Yeh, Y.-Y., Chen, A. J. W., & DʼEsposito, M. Perception & Psychophysics, 44, 369–378. (2011). Functional connectivity during top–down modulation Pasternak, T., & Greenlee, M. W. (2005). Working memory in of visual short-term memory representations. primate sensory systems. Nature Reviews Neuroscience, 6, Neuropsychologia, 49, 1589–1596. 97–107. Landman, R., Spekreijse, H., & Lamme, V. A. F. (2003). Large Postle, B. R., Druzgal, T. J., & DʼEsposito, M. (2003). Seeking capacity storage of integrated objects before change the neural substrates of visual working memory storage. – – blindness. Vision Research, 43, 149 164. Cortex, 39, 927 946. Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/24/1/51/1780230/jocn_a_00087.pdf by MIT Libraries user on 17 May 2021 Lepsien, J., & Nobre, A. C. (2006). Cognitive control of Robitaille, N., & Jolicoeur, P. (2006). Fundamental properties attention in the : Insights from orienting of the N2pc as an index of spatial attention: Effects of attention to mental representations. Brain Research, masking. Canadian Journal of Experimental Psychology, 1105, 20–31. 60, 101–111. Lepsien, J., & Nobre, A. C. (2007). Attentional modulation Rutman, A. M., Clapp, W. C., Chadick, J. Z., & Gazzaley, A. of object representations in working memory. Cerebral (2010). Early top–down control of visual processing Cortex, 17, 2072–2083. predicts working memory performance. Journal of – Lepsien, J., Thornton, I., & Nobre, A. C. (2011). Cognitive Neuroscience, 22, 1224 1234. Downloaded from http://direct.mit.edu/jocn/article-pdf/24/1/51/1943573/jocn_a_00087.pdf by guest on 03 October 2021 Modulation of working-memory maintenance by Schmidt, B. K., Vogel, E. K., Woodman, G. F., & Luck, S. J. (2002). directed attention. Neuropsychologia, 49, 1569–1577. Voluntary and automatic attentional control of visual working Makovski, T., & Jiang, Y. V. (2007). Distributing versus memory. Perception & Psychophysics, 64, 754–763. focusing attention in visual short-term memory. Serences, J. T., Ester, E. F., Vogel, E. K., & Awh, E. (2009). Psychonomic Bulletin & Review, 14, 1072–1078. Stimulus-specific delay activity in human primary visual Makovski, T., Shim, W. M., & Jiang, Y. V. (2006). cortex. Psychological Science, 20, 207–214. Interference from filled delays on visual change detection. Stokes, M. G. (2011). Top–down visual activity underlying Journal of Vision, 6, 1459–1470. VSTM and preparatory attention. Neuropsychologia, 49, Makovski, T., Sussman, R., & Jiang, Y. V. (2008). Orienting 1425–1427. attention in visual working memory reduces interference Stokes, M. G., Thompson, R., Cusack, R., & Duncan, J. (2009). from memory probes. Journal of Experimental Psychology: Top–down activation of shape-specific population codes Learning, Memory, and Cognition, 34, 369–380. in visual cortex during mental imagery. Journal of Matsukura, M., Luck, S. J., & Vecera, S. P. (2007). Attention Neuroscience, 29, 1565–1572. effects during visual short-term memory maintenance: Super, H., Spekreijse, H., & Lamme, V. A. F. (2001). A neural Protection or prioritization? Perception & Psychophysics, 69, correlate of working memory in the monkey primary 1422–1434. visual cortex. Science, 293, 120–124. McCollough, A. W., Machizawa, M. G., & Vogel, E. K. (2007). Vandenbroucke, A. R. E., Sligte, I. G., & Lamme, V. A. F. Electrophysiological measures of maintaining representations (2011). Manipulations of attention dissociate fragile in visual working memory. Cortex, 43, 77–94. visual short-term memory from visual working memory. Miller, E. K., Li, L., & Desimone, R. (1993). Activity of Neuropsychologia. 49, 1559–1568. neurons in anterior inferior temporal cortex during a Vogel, E. K., & Machizawa, M. G. (2004). Neural activity predicts short-term memory task. Journal of Neuroscience, 13, individual differences in visual working memory capacity. 1460–1478. Nature, 428, 748–751. Munneke, J., Heslenfeld, D. J., & Theeuwes, J. (2010). Vogel, E. K., McCollough, A. W., & Machizawa, M. G. (2005). Spatial working memory effects in early visual cortex. Neural measures reveal individual differences in controlling Brain and Cognition, 72, 368–377. access to working memory. Nature, 438, 500–503. Murray, A. M., Nobre, A. C., & Stokes, M. G. (2011). Markers of Yeh, Y.-Y., Kuo, B.-C., & Liu, H. L. (2007). The neural correlates preparatory attention predict visual short term memory of attention orienting in visuospatial working memory for performance. Neuropsychologia, 49, 1458–1465. detecting feature and conjunction changes. Brain Research, Nobre, A. C., Griffin, I. C., & Rao, A. (2008). Spatial attention 1130, 146–157. can bias search in visual short-term memory. Frontiers Zanto, T. P., & Gazzaley, A. (2009). Neural suppression of in Human Neuroscience, 1, 1–9. irrelevant information underlies optimal working memory Oldfield, R. C. (1971). The assessment and analysis of performance. Journal of Neuroscience, 29, 3059–3066.

60 Journal of Cognitive Neuroscience Volume 24, Number 1