ORIGINAL RESEARCH ARTICLE published: 04 April 2013 HUMAN NEUROSCIENCE doi: 10.3389/fnhum.2013.00115 Quantifying attentional modulation of auditory-evoked cortical responses from single-trial electroencephalography Inyong Choi 1, Siddharth Rajaram 1, Lenny A. Varghese 1,2 and Barbara G. Shinn-Cunningham 1,2* 1 Center for Computational Neuroscience and Neural Technology, Boston University, Boston, MA, USA 2 Department of Biomedical Engineering, Boston University, Boston, MA, USA Edited by: Selective auditory attention is essential for human listeners to be able to communicate John J. Foxe, Albert Einstein College in multi-source environments. Selective attention is known to modulate the neural of Medicine, USA representation of the auditory scene, boosting the representation of a target sound Reviewed by: relative to the background, but the strength of this modulation, and the mechanisms Kimmo Alho, University of Helsinki, Finland contributing to it, are not well understood. Here, listeners performed a behavioral Sarah E. Donohue, Duke University, experiment demanding sustained, focused spatial auditory attention while we measured USA cortical responses using electroencephalography (EEG). We presented three concurrent *Correspondence: melodic streams; listeners were asked to attend and analyze the melodic contour of one Barbara G. Shinn-Cunningham, of the streams, randomly selected from trial to trial. In a control task, listeners heard the Auditory Neuroscience Laboratory, Center for Computational same sound mixtures, but performed the contour judgment task on a series of visual Neuroscience and Neural arrows, ignoring all auditory streams. We found that the cortical responses could be Technology, Boston University, fit as weighted sum of event-related potentials evoked by the stimulus onsets in the 677 Beacon St., Boston, competing streams. The weighting to a given stream was roughly 10 dB higher when MA 02421, USA. e-mail: [email protected] it was attended compared to when another auditory stream was attended; during the visual task, the auditory gains were intermediate. We then used a template-matching classification scheme to classify single-trial EEG results. We found that in all subjects, we could determine which stream the subject was attending significantly better than by chance. By directly quantifying the effect of selective attention on auditory cortical responses, these results reveal that focused auditory attention both suppresses the response to an unattended stream and enhances the response to an attended stream. The single-trial classification results add to the growing body of literature suggesting that auditory attentional modulation is sufficiently robust that it could be used as a control mechanism in brain–computer interfaces (BCIs). Keywords: auditory attention, spatial attention, auditory event-related potentials, single-trial classification, brain−computer interfaces INTRODUCTION 1973), which, given its 100 ms latency (relative to stimulus onset) Most human listeners are able to selectively attend to a target suggests it is generated in early auditory sensory cortex (Scherg sound in a complex scene with relative ease. This ability depends et al., 1989). The idea that selective auditory attention strongly on both sensory and cognitive processes, which interact to enable modulates the neural representation of sound in sensory auditory us to segregate competing streams, focus selective attention on an cortex is also supported by MEG studies (Woldorff et al., 1993; important target source, and recognize the target sound’s content Alho et al., 2012; Ding and Simon, 2012)andfMRIdata(Grady (Bregman, 1990; Wrigley and Brown, 2004; Shinn-Cunningham et al., 1997; Jäncke et al., 1999; Janata et al., 2002). and Best, 2008; Lee et al., 2013). Though the specific mecha- The current study explores how selective attention modulates nisms supporting these processes are not well understood, gross ERPs evoked by competing musical streams. Listeners performed changes in neural activity due to attention can be observed in a “contour judgment” task that required them to focus atten- auditory-evoked event related potentials (ERPs) measured using tion on one of three simultaneous melodic contours and make electroencephalography (EEG; e.g., Hillyard et al., 1973; Hansen judgments about the shape of the attended contour. This task and Hillyard, 1980; Woldorff et al., 1987). Such studies find mimics a real-world listening situation by requiring listeners to changes in the amplitude and shape of ERPs, suggesting that selec- focus and sustain attention on a stream in order to analyze its tive attention acts as a gain on neural activity, causing a relative content. We fit the EEG data as a scaled sum of the neural enhancement of the representation of attended sensory inputs responses elicited by the individual streams played in isolation, and a relative decrease in the representation of unattended or allowing the scaling to depend on how a listener focuses atten- ignored inputs (Hillyard et al., 1998). A particularly salient effect tion. By finding the best scaling factors, or “attentional gains,” we of selective auditory attention is the enhancement of the N1 ERP quantified the amount of attentional modulation of the cortical component evoked by an attended sound (e.g., Hillyard et al., response. Frontiers in Human Neuroscience www.frontiersin.org April2013|Volume7|Article115| 1 Choi et al. Attentional modulation of auditory cortex A number of existing brain–computer interfaces (BCIs) track zigzagged. Subjects were instructed to ignore the auditory streams changes in EEG signals corresponding to changes in how a user during visual-trial presentations. directs attention to visual objects (Kelly et al., 2005; Allison et al., The acoustic streams were statistically identical in auditory- 2010). Traditional ERP studies of auditory attention demonstrate attention and in visual trials; only the task of the subject differed task-related changes in the morphology of ERPs averaged over across trial types, and only the direction of the attended audi- many trials (Hill et al., 2005). While such studies show that atten- tory stream differed between attend-left and attend-right auditory tion modulates the neural response, they do not test whether trials. In both auditory-attention and visual trials, subjects iden- effects are strong enough or consistent enough that single-trial tified which kind of sequence was present by pressing one of three evoked responses can be used to deduce how attention is directed. buttons (thus equating the motor planning and execution in the To the degree that single-trial EEG classification is possible, it responses to the two trial types). suggests that a BCI could be constructed that determines how a user (such as a locked-in patient) is focusing attention and then STIMULI uses this information to navigate a command menu or control a All auditory stimuli were generated and processed using Matlab device. A few recent studies suggest that auditory attention can (Mathworks, Natick, MA). The auditory stimuli consisted of three modulate EEG responses sufficiently to be used in such a manner concurrent melodic streams, each of which was comprised of (e.g., Kerlin et al., 2010; Hill and Schölkopf, 2012; Lopez-Gordo multiple complex tones (henceforth referred to as “notes”). On et al., 2012). Because the modulation of attentional gain was pro- each trial, each of the three streams had a distinct isochronous nounced in our experimental paradigm, we tested whether our rhythm (three, four, or five notes), a distinct timbre (cello, clar- single-trial EEG results could be used to classify the direction to inet, or oboe), a distinct pitch range that did not overlap with which a listener was attending. We used a template-matching clas- that of the other streams (bottom, middle, top), and a distinct sification approach (Woody, 1967; Kerlin et al., 2010)toestimate lateral location (left, center, or right; see an example in Figure 1). from single-trial epochs which source the listener had attended This redundant set of cues ensured that the competing streams on each trial. Classification rates were significantly above chance were easily segregated, perceptually, so that listeners could focus for all subjects. Given this success using single-trial non-invasive attention on whichever stream was important on a given trial. EEG, our results add to the growing body of literature demon- The center stream, which was never the focus of attention, strating the potential for auditory selective attention to be used always consisted of three notes, each 1 s in duration; the left in EEG-based BCIs. Our approach models ERP waveforms as stream always contained four notes, each 750 ms in duration; templates and uses a cross-subject validation to test classification and the right stream had five notes, each 600 ms in duration. performance; thus, our success suggests that an auditory atten- All streams started and ended together and had the same total tion BCI even could even be used successfully “out of the box,” duration of 3 s. By design, although all three streams turned on without user-specific training of the EEG classifier. together, each of the subsequent note onsets in each of the streams was distinct from the onsets in the other streams (see Figure 1). MATERIALS AND METHODS To achieve a natural, gradual time course, cosine squared onset SUBJECTS Ten volunteers (two female, aged 21–34 years) participated in the experiments. All were right handed and had normal