Proc. Natl. Acad. Sci. USA Vol. 90, pp. 8722-8726, September 1993 Neurobiology Modulation of early sensory processing in human during auditory selective (event-related el*d/magnecephalorphy/event-related potential/magnetic resonance lai/Hesch's gyrus) MARTY G. WOLDORFF*t, CHRISTOPHER C. GALLEN*, ScoTT A. HAMPSON0§, STEVEN A. HILLYARD*, CHRISTO PANTEV*§, DAVID SOBEL¶, AND FLOYD E. BLOOMS *De~partment of , 0608, University of California at San Diego, 9500 Gilman Drive, La Jolla, CA 92093-0608; and Departments of *Neuropharniacology and lRadiology, Research Institutes of Scripps Clinic, La Jolla, CA 92037 Contributed by Floyd E. Bloom, June 4, 1993

ABSTRACT Neuromagnetic fields were recorded from in the interval 20-50 msec (the P20-50) was also found to be human subject as they listened selectively to sequences of enlarged with attention (6, 7). rapidly presented tones in one ear while ignoring tones of a These ERP results support the view that the flow of diferent pitch In the oppoite ear. Tones in the attended ear auditory sensory information can be altered by attention at a evoked larger magnetic brain responses than did unattended relatively early stage of processing. They do not specify, tones in the latency ranges 20-50 msec and 80-130 msec however, the brain structures in which this selection posttimulus. Source loction echniques in coijunction takes place. One approach to address this question is to study With metic resonance imagin placed the neural generators neuromagnetic recordings (ERFs), which offer an advantage of these early attention-sensitive brain responses in auditory over ERPs for localization of the anatomical sources of cortex on the supratemporal plane. These data demonstrate evoked brain activity in cortical sulci (e.g., auditory cortex that focused auditory attention in humans can selectively on the supratemporal plane). This advantage is due to ERF modulate sensory processing in auditory cortex beginning as recordings being selectively sensitive to activity from such early as 20 msec p mulu, thereby providing strong evi- sources and due to magnetic fields being less distorted by the dence for an "early section" mechanism ofauditory attention skull (8, 9). that can regulate auditory input at or before the initial sages Several studies have applied source localization tech- of coricl analyis. niques to ERF recordings and concluded that at least part of the enhanced activity elicited by attended sounds in the N100 Selective attention improves the perception of high-priority latency range arises from the vicinity of auditory cortex (10, stimuli in the environment at the expense of other, less 11). However, the precise anatomical source(s) ofthis atten- relevant, stimuli. In the auditory modality, for example, a tion effect have yet to be verified by superposition of calcu- lated source coordinates onto magnetic resonance (MR) person can attend selectively to a particular speaker's voice images of the subjects' brains.11 Even less information is while tuning out other, simultaneous, conversations (the available regarding the neuralgenerator(s) ofthe P20-50 ERP so-called "cocktail party" phenomenon). Despite extensive attention effect, as no magnetic counterpart ofthis very early research (1-3), many gaps remain in our understanding ofthe ERP modulation has yet been reported. neural and psychological mechanisms that underlie auditory In the current study, neuromagnetic and MR imaging selective attention. techniques were combined to localize the neuroanatomical The neural mechanisms of selective attention can be in- origins of the early effects of attention on tone-evoked brain vestigated noninvasively in humans by recording the brain's activity. The results provide evidence that focused auditory event-related electrical potentials (ERPs) and/or event- attention exerts selective control over early sensory process- related magnetic fields (ERFs). ERPs and ERFs are extracted ing in the auditory cortical areas on the supratemporal plane from the scalp-recorded electroencephalogram (EEG) or its beginning at 20 msec poststimulus. magnetic counterpart, the magnetoencephalogram (MEG), by computer averaging the time-locked brain activity elicited by repeated stimulus occurrences. The resultant ERP and METHODS ERF waveforms reflect with high temporal resolution the Selective auditory attention was studied using the same patterns of neuronal activity that are evoked by a stimulus. fast-rate dichotic listening paradigm that we have used pre- By analyzing changes in the ERPs and ERFs as a function of viously in ERP studies (6, 7), with the ERP recordings now the direction of attention, inferences can be made about the complemented by recordings ofneuromagnetic fields (ERFs) timing, level of processing, and anatomical location of stim- ulus selection processes in the brain. Abbreviations: ERF, event-related magnetic field; ERP, event- Previous studies have recorded ERPs in an experimental related electrical potential; MEG, (-gram); analog ofthe cocktail party situation, with sequences oftone ECD, equivalent current dipole; EEG, electroencephalogram. randomly presented to the left and right ears while subjects tTo whom reprint requests should be sent at present address: selectively attended to the tones in one ear and ignored tones Research Imaging Center, University of Texas Health Science Center at San Antonio, 7703 Floyd Curl Drive, San Antonio, TX in the opposite ear (4-7). Under conditions of high auditory 78284-6240. sensory load, attended-ear tones were found to elicit an §Present address: Institute ofExperimental Audiology, University of enlarged negative ERP wave that peaked at around 100 msec Munster, Kardinal-von-Galen-Ring 10, D4400 Munster, Germany. poststimulus and overlapped closely in time the major sen- IThere is considerable evidence that at least some of the "exoge- sory-evoked negative wave, N100. An earlier positive wave nous" tone-evoked activity (i.e., non-attention-related) in the 30- to 150-msec latency range arises from neural generators in the vicinity of the auditory cortex on the supratemporal plane (12-15). Recent The publication costs ofthis article were defrayed in part by page charge neuromagnetic studies have localized portions ofthis early sensory- payment. This article must therefore be hereby marked "advertisement" evoked activity to the supratemporal-plane auditory cortex as in accordance with 18 U.S.C. §1734 solely to indicate this fact. visualized on subjects' MR scans (16-18). 8722 Downloaded by guest on September 28, 2021 Neurobiology: Woldorff et al. Proc. Natl. Acad. Sci. USA 90 (1993) 8723 from over the left hemisphere using a 37-channel magnetom- from the International 10-20 System sites Cz, C3, and T3, eter. Seven normal volunteer subjects (ages 22-36) were referred to the left earlobe. The recording bandpass was studied in a magnetically shielded chamber. Tone sequences 0.1-200 Hz for the MEG and 0.05-250 Hz for the EEG. For were delivered through a sound-tube system and consisted of each subject, averaged ERF and ERP waveforms were 1000-Hz tone pips to the left ear and 3150-Hz tone pips to the obtained separately for the attended and unattended stan- right ear, all of 14-msec duration with 5-msec rise and fall dards and targets. Trials with artifacts (e.g., eye blinks) were times. The left- and right-ear tones were presented in random rejected from the averages. One magnetic sensor with highly order with interstimulus intervals ranging randomly between elevated noise levels was excluded from analysis. 125 and 325 msec. The subjects' task was to listen selectively To quantify the effects of attention on the M100, mean to the tones in one ear and detect occasional (91% per ear), amplitudes were measured separately for attended and un- difficult-to-detect, "target" tones that were 12 decibels (dB) attended ERF waveforms for each subject across a latency fainter than the 55-dBSL (sensation level) "standard" tones window (80-130 msec) centered over the grand-average in that ear. All tones in the other ear were to be ignored. Ten M100 peak. These measured values were then entered into an runs each of attend-left and attend-right conditions, each 2 analysis of variance (ANOVA). To quantify activity in the min long, were presented in counterbalanced order. Only earlier middle-latency range, statistical analyses were also responses evoked by the right-ear standard tones (i.e., con- performed on the magnetic M50 wave. The analyses on this tralateral to the recording sites) are presented in this report. wave, which in this experiment began at about 20 msec and The recording probe apparatus (Biomagnetic Technolo- lasted until 55 msec, were analogous to those for the M100, gies, San Diego) contained 37 magnetic sensors that utilized but with the measurement window set at 28-46 msec. superconducting quantum interference devices (SQUIDs). Topographic maps of the ERF distributions were calcu- This sensor array, spanning a circular area of 125-mm diam- lated for the M50 and M100 peaks under the different eter, was placed over the scalp overlying the left auditory attention conditions, both for the individual subjects' ERFs cortex. A transceiver-based system was used to localize the and for the ERFs grand-averaged over all the subjects. For position ofthe magnetic sensor array relative to the head and each of these surface field distributions, a best-fitting equiv- to construct a spatial reference frame for the MEG measure- alent current dipole (ECD) was calculated in the MEG ments with respect to a set offiducial skull landmarks. Three reference frame (i.e., with respect to the fiducial skull land- channels ofEEG were also recorded over the left hemisphere marks), using an algorithm based on least-squares approxi-

FiG. 1. (A) Grand-average waveforms (i.e., averaged across all seven subjects) of the event-related magnetic activity elicited by right-ear standard tones when they were attended versus when they were unattended, displayed at approximate locations of the magnetic sensors over the left hemisphere. At the upper right are the simultaneously recorded ERPs from the C3 site. Positive (upward) values forthe magnetic activity indicate that the fields are directed out ofthe head, and negative values indicate inward-directed fields [calibration bars = +20 femtotesla (MT)]. ERP scalp negativity is plotted upward [calibration bars = _1 microvolt (.tV)]. Large arrows mark the polarity-inverting M100 at sites 25 and 33; small arrows denote the polarity-inverting MS0. (B) Grand-average attentional-difference waveforms (attended minus unattended ERFs) derived from the data in A for four sites (denoted with asterisks in A) in the anterior-to-posterior line across the array. Large and small arrows mark the polarity-inverting attention effects for M100 and M50, respectively. Downloaded by guest on September 28, 2021 8724 Neurobiology: Woldorff et al. Proc. Natl. Acad Sci. USA 90 (1993) Attended Unattended Attention Effect

z z z

A

Single Subject MIQO

B

Grand Average MioO

Grand Average M50

FIG. 2. Topographic plots (isocontour lines) showing magnetic field distributions for the M100 and the M50 (each individually scaled to emphasize distribution rather than absolute magnitude). (A) Field distributions at the peak of the M100 from a single subject for the attended response, the unattended response, and the attentional-difference wave (i.e., the subtracted difference between the attended and unattended responses). Note the dipolar field distribution, with a maximum (shaded dark) where the magnetic field lines are directed out of the head and a minimum (shaded light) where the magnetic field lines are directed into the head. The arrow indicates the direction of the single ECD source that would produce a set offields that would best fit this distribution. Isocontour scales (differences between adjacent isocontour lines) are 12.4, 8.5, and 3.9 ff for the attended, unattended, and attentional-difference waves, respectively. (B) Corresponding field distributions for the M100 from the grand-averaged waveforms. Isocontour scales are 6.4, 3.7, and 2.9 ff for the attended, unattended, and attentional-difference waves, respectively. (C) Same as B for the M50. Isocontour scales are 4.8, 3.8, and 1.1 ff for the attended, unattended, and attentional-difference waves, respectively. mation (19).** MR images were obtained for four of the negativity that closely overlapped the sensory-evoked N100 subjects, and, using the skull landmarks, the neuromagnetic wave. A corresponding effect can also be seen on the reference frames for these subjects were co-registered with magnetic counterpart to the N100, known as the M100 (e.g., their MR reference frames. This allowed these subjects' ECD see posterior channel 25). Moreover, both the M100 and its localization coordinates to be transposed onto their MR enhancement with attention exhibited a clear dipolar distri- images. bution-that is, the stimulus-evoked magnetic fields at 100 msec are directed out of the head posteriorly (e.g., channel RESULTS AND DISCUSSION 25, thick arrow) and into the head anteriorly (e.g., channel 33, thick arrow), with both of these extrema increasing in mag- A key feature of the paradigm used in this study is that it nitude with attention. allows a comparison between the evoked responses to the The effect of attention on the amplitude of the M100 was same physical stimuli (e.g., right-ear standard tones) under reflected statistically in a significant ANOVA interaction of different selective attention conditions (attend right ear ver- attention X site (P < 0.008, after applying the Huynh-Feldt sus attend left ear). Thus, any ERP or ERF differences found correction fornon-sphericity). This interaction was due to the can be attributed to the internal focusing of attention either M100 measures for the attended waveforms yielding larger toward or away from those stimuli. positive values at posterior sites and larger negative values at Fig. 1A shows the ERF and ERP waveforms for right-ear anterior sites than for the corresponding unattended wave- standard tones when they were attended versus unattended, forms. In addition, specific comparisons at the extremum grand-averaged across the seven subjects. As in previous sites showed significantly larger magnitudes for the attended studies (6, 7), attended-ear tones elicited an enhanced ERP waveforms (P <0.006-0.05). The P20-50 ERP attention effect also was found to have a **The ECD is the single equivalent dipole source that would produce we a distribution offields that would best fit an observed distribution. dipolar magnetic counterpart that have termed the "M20- For the dipolar distributions analyzed in this report, the ECD 50" (Fig. 1A, channels 33 and 25, thin arrows). The M20-50 location can be considered as representing the estimated centroid was evident as an augmentation of the magnetic M50 wave of the region of active tissue. with attention. This effect was reflected statistically in a Downloaded by guest on September 28, 2021 NeUrObiOlOgY: Woldorf et al. Proc. Natl. Acad. Sci. USA 90 (1993) 8725 A

w ~~~~~~~~~~~~~~~ l ..~~~~~~~~~

I FIG. 3. (A) Coronal and sagittal MR images showing the estimated locations and angles ofthe ECD sources for the M100 and M100 attention effect for the subject whose ERF distributions are shown in Fig. 2A. The white square is for the attended M100, the hatched square is for the unattended M100, and the white circle is for the M100-latency attention effect. Note these estimated source locations are within millimeters of each other in auditory cortex on the supratemporal plane, localizingjust lateral to Heschl's gyrus. The small black lines attached to each symbol indicate the direction of the estimated current dipole. The long vertical white line in each image indicates the plane through which the other image is taken. (B) Same as A for a second subject. significant attention X site interaction in the M50 analysis (P the subjects. Displayed are the ERF distributions ofthe M100 < 0.04, after applying the Huynh-Feldt correction), which to attended tones, the M100 to unattended tones, and the resulted from the ERF amplitude measures for the attended attended-minus-unattended M100 attention effect. Each of waveforms having larger positive values anteriorly and larger these contour plots shows a highly dipolar distribution, with negative values posteriorly than for the unattended wave- a single posterior maximum and single anterior minimum. forms. Specific comparisons at the extremum sites also This suggests that the activity in each case can be well showed significantly larger magnitudes for the attended modeled as arising predominantly from a focal dipolar source waveform M5Os (P <0.025-0.05). located a few centimeters deep (i.e., in the region ofauditory The attention effects are highlighted in the "attentional- cortex) and oriented parallel to the displayed arrow (i.e., difference waves" formed by subtracting the unattended perpendicular to the Sylvian fissure). The high degree of response waveform from its attended counterpart at each similarity among these three distributions suggests that their recording site. Fig. 1B shows these difference waves for four sources have similar locations and orientations. sites along an anterior-to-posterior line across the recording Foreach subject, the best-fitting ECD was calculated in the array through the M50 and M100 extrema. It can be seen that MEG reference frame for each ofthese M100-latency surface both the M20-50 attention effect (peaking at about 40 msec) field distributions. For most of the subjects, the observed and the M100-latency attention effect (peaking at 100 msec) field distributions for the M100 and for the M100-latency invert at anterior sites relative to posterior sites in a manner attention effect were each well accounted forby single ECDs, similar to the M50 and M100 components themselves. with correlations between the dipole model fields and the Topographic plots of the evoked magnetic field distribu- recorded fields ranging from 0.96 to 0.99. MR images were tions at the peak ofthe M100 are shown in Fig. 2A for one of obtained for the four subjects with the best-fitting ECDs and Downloaded by guest on September 28, 2021 8726 Neurobiology: Woldorff et al. Proc. Natl. Acad Sci. USA 90 (1993) these subjects' ECD coordinates were transposed onto their the sensory-evoked cortical components M50 and M100. This MR images. result, coupled with the finding that the sources of these Fig. 3A shows a coronal and a sagittal MR section from one attention effects were co-localized with the sources of the of these subjects and indicates the estimated locations and corresponding sensory-evoked components, strongly sug- angles ofthe ECD sources for the attended M100, unattended gests the existence of a mechanism of sensory gain control M100, and M100-latency attention effect. These estimated within the auditory input channels at or before the initial source locations are indeed within millimeters of each other stages ofcortical processing. Thus, these results not only add in auditory cortex on the supratemporal plane, localizingjust strong support for psychological theories of attention that lateral to Heschl's gyrus. In addition, the estimated direction posit an early, pre-perceptual selection of stimulus input, of current flow in each case is oriented approximately per- they also provide specific information concerning some ofthe pendicular to the cortical surface ofthe supratemporal plane, neuroanatomical structures and physiological mechanisms as predicted by current dipole theory (8, 9). Estimated source by which such selection is accomplished. locations for a second subject are shown in Fig. 3B, and again all three ECDs lie very near to Heschl's gyrus. The remaining We thank Jonathan C. Hansen and Paul Krewski for important two subjects for whom MR images were obtained showed technical assistance; Biomagnetic Technologies, Inc., for providing similar patterns of ECD localization for the M100 and M100- access to the magnetometer and associated facilities; and Steve latency attention effect. These MR localization results pro- Luck, Lourdes Anllo-Vento, Vince Clark, William Rogasner, Robert vide the strongest evidence to date that the magnetic atten- Hannah, and Helen Mayberg foruseful comments on earlier versions tion effect at 100 msec arises predominantly from enhanced ofthis manuscript. This work was supported by National Institute of in cortex and that this Mental Health Grant MH-25594, Office of Naval Research Contract activity supratemporal-plane auditory N00014-89-J-1743, National Institute of Neurological Disorders and enhanced activity consists primarily of an attention-related Stroke Program Project Grant NS-17778, and the San Diego Mc- modulation of the M100 neural generator. Donnell-Pew Center for Cognitive . This is paper The M20-50 attention effect had an insufficient signal/ NP8170 from The Research Institutes of Scripps Clinic. noise ratio in individual subjects to be successfully fit using the ECD source localization techniques. However, the con- 1. Kahnemann, D. & Treisman, A. (1984) in Varieties ofAtten- sistency of the probe placements and magnetic responses tion, eds., Parasuraman, R. & Davies, R. (Academic, London), across subjects allowed visualization of this effect in the pp. 29-61. grand average (Fig. 1). Fig. 2 B and C show the field 2. Hillyard, S. A. & Picton, T. W. (1987) in Handbook ofPhys- distributions of the M100 and M50 derived from the grand- iology, Section 1, Vol. 5, Part 2, ed. Mountcastle, V. B. (Am. and attentional-difference Physiol. Soc., Baltimore), pp. 519-584. averaged attended, unattended, 3. Posner, M. & Petersen, S. (1990) Annu. Rev. Neurosci. 13, waveforms. All six ofthese distributions were highly dipolar, 25-42. suggesting that the relative location of the M20-50 source 4. Hillyard, S. A., Hink, R. F., Schwent, V. L. & Picton, T. W. could be estimated by applying the ECD source localization (1973) Science 182, 177-180. approach to the grand-average data. Thus, best-fit ECDs 5. Naatanen, R. (1990) Behav. Brain Sci. 13, 201-288. were calculated for the grand-averaged distributions using 6. Woldorff, M. G., Hansen, J. C. & Hillyard, S. A. (1987) in averaged head-shape and probe-placement information. Current Trends in Event-Related Potential Research, eds. These calculations yielded excellent-fitting ECDs (correla- Johnson, R., Rohrbaugh, J. W. & Parasuraman, R. (Elsevier, tions of 0.98-0.995) for each of the six grand-average M100 London), pp. 146-154. and M50 distributions shown in Fig. 2 B and C. Although 7. Woldoriff, M. G. & Hillyard, S. A. (1991) Electroencephalogr. Clin. Neurophysiol. 79, 170-191. these source estimates could not be localized directly on any 8. Okada, Y., Lauritzen, M. & Nicholson, C. (1987) Phys. Med. individual subject's MR scan, theirrelative x-y-z coordinates Biol. 32, 43-51. in the MEG reference frame could be evaluated. In this 9. Williamson, S. & Kaufman, L. (1987) in Handbook ofElectro- frame, the grand-average M50 (to both attended and unat- encephalography and Clinical Neurophysiology: Methods of tended tones) and the corresponding attention effect (i.e., the Analysis ofBrain Electrical andMagnetic Signals, eds. Gevins, M20-50) were localized quite near (2-14 mm more medial) A. & Remond, A. (Elsevier, New York), pp. 405-448. the grand-average MlOOs and the individual subjects' MlOOs. 10. Arthur, D., Hillyard, S. A., Flynn, E. & Schmidt, A. (1989) in Because these M100 sources were localized in individual Advances in Biomagnetism, eds. Williamson, S. J., Hoke, M., to on the in or Stroink, G. & Kotani, M. (Plenum, New York), pp. 113-116. subjects regions supratemporal plane slightly 11. Rif, J., Hari, R., Hamalainen, M. & Sams, M. (1991) Electro- lateral to Heschl's gyrus, it follows that the earlier M20-50 encephalogr. Clin. Neurophysiol. 79, 464-472. attention effect also derives from this area. 12. Romani, G. L., Williamson, S. J., Kaufman, L. & Brenner, D. The present study combined neuromagnetic and MR im- (1982) Exp. Brain Res. 47, 381-393. aging techniques to demonstrate that focused auditory atten- 13. Pelizzone, M., Hari, R., Makela, J., Huttunen, J., Alfors, S. & tion exerts a selective control over early sensory processing Hamalainen, M. S. (1987) Neurosci. Lett. 82, 303-307. in the auditory cortex of the supratemporal plane. This 14. Hari, R. & Lounasmaa, 0. V. (1989) Science 244, 432-436. cortical localization was verified by directly plotting the 15. Scherg, M. & Von Cramon, D. (1986) Electroencephalogr. source coordinates for the attentional enhancement of the Clin. Neurophysiol. 65, 344-360. M100 onto MR scans of the individual 16. Yamamoto, T., Williamson, S. J., Kaufman, L., Nicholson, C. component subjects' & Llinas, R. (1988) Proc. Natl. Acad. Sci. USA 85, 8732-8736. brains. In addition, the discovery of a still earlier magnetic 17. Pantev, C., Hoke, M., Lehnertz, K., Lutkenhoner, B., Fahr- attention effect (the M20-50) that arises from the same endorf, G. & Stober, U. (1990) Electroencephalogr. Clin. cortical area indicates that this selective control over sensory Neurophysiol. 75, 173-184. transmission can begin as early as 20 msec after stimulus 18. Arthur, D. L., Lewis, P. S., Medvick, P. A. & Flynn, E. R. onset. Furthermore, the waveshapes of these early attention (1991) Electroencephalogr. Clin. Neurophysiol. 78, 348-359. effects took precisely the form ofan amplitude modulation of 19. Marquardt, D. (1%3) SIAM J. Appl. Math. 11, 431-441. Downloaded by guest on September 28, 2021