Differential Influences of Emotion, Task, and Novelty on Brain Regions Underlying the Processing of Speech Melody

Thomas Ethofer1,2, Benjamin Kreifelts1, Sarah Wiethoff1, 1 1 2

Jonathan Wolf , Wolfgang Grodd , Patrik Vuilleumier , Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/21/7/1255/1760188/jocn.2009.21099.pdf by guest on 18 May 2021 and Dirk Wildgruber1

Abstract & We investigated the functional characteristics of brain re- emotion than word classification, but were also sensitive to gions implicated in processing of speech melody by present- anger expressed by the voices, suggesting that some percep- ing words spoken in either neutral or angry prosody during tual aspects of prosody are also encoded within these regions a functional magnetic resonance imaging experiment using subserving explicit processing of vocal emotion. The bilateral a factorial habituation design. Subjects judged either affective OFC showed a selective modulation by emotion and repeti- prosody or word class for these vocal stimuli, which could be tion, with particularly pronounced responses to angry prosody heard for either the first, second, or third time. Voice-sensitive during the first presentation only, indicating a critical role of temporal cortices, as well as the amygdala, insula, and medio- the OFC in detection of vocal information that is both novel dorsal thalami, reacted stronger to angry than to neutral pros- and behaviorally relevant. These results converge with previ- ody. These stimulus-driven effects were not influenced by the ous findings obtained for angry faces and suggest a general task, suggesting that these brain structures are automatically involvement of the OFC for recognition of anger irrespective engaged during processing of emotional information in the of the sensory modality. Taken together, our study reveals that voice and operate relatively independent of cognitive demands. different aspects of voice stimuli and perceptual demands By contrast, the right middle temporal gyrus and the bilateral modulate distinct areas involved in the processing of emotion- orbito-frontal cortices (OFC) responded stronger during al prosody. &

INTRODUCTION et al., 1998), decreased (Morris et al., 1999), or no dif- Modulation of speech melody (prosody) provides a ferential activity (Wildgruber et al., 2005) in amygdala powerful cue to express one’s affective state. Previous responses to affective prosody or nonlinguistic vocaliza- neuroimaging studies demonstrated that voice-sensitive tions. In addition, it has been demonstrated that the regions (Belin, Zatorre, Lafaille, Ahad, & Pike, 2000) in orbito-frontal cortex (OFC) displays a particular sensitivity the middle part of the right superior temporal gyrus to angry expressions in faces (Blair, Morris, Frith, Perrett, (STG) exhibit stronger responses to a variety of vocally ex- & Dolan, 1999) but probably also in voices (Hornak et al., pressed emotions than to neutral prosody (Ethofer et al., 2003). So far, however, the specific role of these different 2007; Ethofer, Anders, Wiethoff, et al., 2006; Grandjean regions remains unclear, and the exact aspects of affective et al., 2005). Another brain structure where differential vocal information encoded in the amygdala, the OFC, responses to vocally expressed emotions were often and other related brain regions are still largely unknown. reported is the amygdala (Quadflieg, Mohr, Mentzel, Explicit evaluation of affective prosody has been Miltner, & Straube, 2008; Fecteau, Belin, Joanette, & linked to both right posterior middle temporal gyrus/ Armony, 2007; Ethofer, Anders, Erb, Droll, et al., 2006; superior temporal sulcus (MTG/STS; Ethofer, Anders, Ethofer, Pourtois, & Wildgruber, 2006; Sander et al., 2005; Erb, Herbert, et al., 2006; Wildgruber et al., 2005; Mitchell, Morris, Scott, & Dolan, 1999; Phillips et al., 1998), although Elliott, Barry, Cruttenden, & Woodruff, 2003) and the its role in processing of vocal emotions is still controver- bilateral inferior frontal cortex/orbito-frontal cortex sial, as different studies found either increased (Quadflieg (IFG/OFC; Ethofer, Anders, Erb, Herbert, et al., 2006; et al., 2008; Fecteau et al., 2007; Sander et al., 2005; Phillips Wildgruber et al., 2004; Wildgruber, Pihan, Ackermann, Erb, & Grodd, 2002; Imaizumi et al., 1997), as these brain 1University of Tu¨bingen, Tu¨bingen, Germany, 2University Medi- regions show task-related brain activity with stronger cal Center of Geneva, Geneva, Switzerland responses during judgment of affective prosody than a

D 2008 Massachusetts Institute of Technology Journal of Cognitive Neuroscience 21:7, pp. 1255–1268 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2009.21099 by guest on 26 September 2021 variety of control tasks including judgment of emotional sentative and well-discriminated category of emotion ex- word content (Ethofer, Anders, Erb, Herbert, et al., 2006; pressed by prosody with strong social relevance. Based Mitchell et al., 2003), phonetic identification (Wildgruber on previous neuroimaging studies, we hypothesized that et al., 2005), evaluation of linguistic prosody (Wildgruber the bilateral STG (Ethofer et al., 2007; Ethofer, Anders, et al., 2004), and speaker identification (Imaizumi et al., Wiethoff, et al., 2006; Grandjean et al., 2005), the amyg- 1997). The suggestion that these brain regions cooper- dala (Quadflieg et al., 2008; Sander et al., 2005; Morris ate during voice processing is supported by functional et al., 1999; Phillips et al., 1998), and the OFC (Blair et al., magnetic resonance imaging (fMRI) data testing for 1999) should exhibit sensitivity to anger as expressed by their functional (Obleser, Wise, Alex Dresner, & Scott, prosody, whereas a distinct set of cortical regions com- 2007) and effective (Ethofer, Anders, Erb, Herbert, et al., prising the right posterior MTG/STS (Ethofer, Anders, Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/21/7/1255/1760188/jocn.2009.21099.pdf by guest on 18 May 2021 2006) connectivity, as well as by results from diffusion Erb, Herbert, et al., 2006; Wildgruber et al., 2005; tensor imaging demonstrating structural connectivity Mitchell et al., 2003) and the bilateral IFG/OFC (Ethofer, between these areas (Glasser & Rilling, 2008). Anders, Erb, Herbert, et al., 2006; Wildgruber et al., 2002, Besides the emotional information, novelty consti- 2004; Imaizumi et al., 1997) was expected to show task- tutes an important factor that determines the behavioral related brain activity. relevance of a stimulus as automatic attention to novel Furthermore, we wanted to clarify whether task- and stimuli is crucial to detect possible significant changes in stimulus-driven components of the brain structures sub- the environment (Sokolov, 1963). Repeated exposure to serving prosody recognition are influenced by the nov- a certain stimulus leads to a faster and more efficient elty of the stimuli and whether habituation in these processing of that stimulus. This phenomenon is called regions is modulated by task instructions or expressed perceptual priming and reflects an implicit form of emotion. To this end, we presented adjectives and nouns memory (Tulving & Schacter, 1990). At the neural level, spoken in angry or neutral prosody during an event- stimulus repetition leads to a decrease in the neural related fMRI experiment and instructed our participants responses, an effect that is termed ‘‘repetition suppres- either to judge the emotion expressed in the voice or sion’’ (for a review, see Grill-Spector, Henson, & Martin, to classify the spoken word as adjective or noun. Each 2006). This neural adaptation to repeated stimulus stimulus was presented three times. Thus, the main fMRI exposure is a robust effect and has been investigated experiment conformed to a 3 Â 2 Â 2 factorial adaptation both at the neural level in monkeys, using single-cell design with presentation (first, second, third), emotion recordings (e.g., Sobotka & Ringo, 1996; Li, Miller, & (angry, neutral), and task (emotion classification, word Desimone, 1993), and at the hemodynamic level in classification) as separate factors. In addition, we also per- humans, using fMRI (e.g., Grill-Spector et al., 1999; formed an fMRI localizer scan to define voice-sensitive Buckner et al., 1995). Across various studies, this effect brain regions (Belin et al., 2000) and to verify that stimulus- has been exploited to investigate different aspects of driven effects due to the expressed affective prosody oc- face processing (Henson & Mouchlianitis, 2007; Bunzeck, curred within these regions. Schutze, & Duzel, 2006; Hasson, Nusbaum, & Small, 2006; Yi, Kelley, Marois, & Chun, 2006; Soon, Venkatraman, & Chee, 2003; Henson, Shallice, Gorno-Tempini, & Dolan, METHODS 2002; Henson, 2000) and voice processing (Dehaene- Lambertz et al., 2006; Hasson et al., 2006; Orfanidou, Subjects Marslen-Wilson, & Davis, 2006; Cohen, Jobert, Le Bihan, Twelve subjects (6 women, 6 men, mean age = 23.2 & Dehaene, 2004; Belin & Zatorre, 2003). Concerning years) took part in the behavioral study and 24 subjects emotional communication, Ishai, Pessoa, Bikle, and (12 women, 12 men, mean age = 26.3 years) were in- Ungerleider (2004) used an adaptation paradigm to ex- cluded in the fMRI study. All participants were right- amine whether habituation in face-sensitive regions handed native speakers of German and had no history of (Haxby et al., 2001; Kanwisher, McDermott, & Chun, neurological or psychiatric diseases. Right-handedness 1997) is modulated by emotion displayed in facial ex- was assessed using the Edinburgh Inventory (Oldfield, pressions. They found that face-sensitive regions exhibit 1971). The study was approved by the Ethical Com- a stronger habituation for emotional than for neutral mittee of the University of Tu¨bingen and conducted ac- faces. So far, similar adaptation paradigms have not been cording to the Declaration of Helsinki. used to investigate processing of affective prosody. In the present study, we investigated whether brain Stimulus Material regions showing differential activity to distinctive stimulus features (i.e., angry vs. neutral prosody) are addition- The stimulus set used in the present study comprised ally modulated by task demands (i.e., when prosody is 16 adjectives and 16 nouns which were spoken by six pro- relevant vs. incidental); and vice versa, whether regions fessional actors (3 men, 3 women) in neutral or angry displaying task-dependent activity are also modulated prosody (8 adjectives and 8 nouns, respectively). Stimulus by stimulus-driven factors. We chose anger as a repre- examples are provided as supplemental material. These

1256 Journal of Cognitive Neuroscience Volume 21, Number 7 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2009.21099 by guest on 26 September 2021 stimuli were selected from a larger corpus of stimuli and maximize behavioral priming and repetition suppression rated in several prestudies (see Ethofer et al., 2007) to effects, we limited the maximal time interval between ensure that they are emotionally neutral and low arousing the repetitions by clustering the stimuli in eight seg- with respect to the word content, that they represent the ments consisting of 12 trials. Each segment contained emotional category intended by the actors, and to evaluate two adjectives and two nouns (one spoken in angry and them with respect to emotional valence and arousal as one in neutral prosody, respectively), which were pre- expressed by prosody. The gender of the speakers was sented three times. Stimulus presentation of the 12 trials balanced over prosodic categories (angry, neutral) and within these segments and the order of the eight seg- word classes (adjectives, nouns). For each stimulus, the ments were fully randomized. Each trial started with a mean intensity (I) and variability (standard deviation [SD] visual display of the judgment scale that informed the Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/21/7/1255/1760188/jocn.2009.21099.pdf by guest on 18 May 2021 of I, normalized to mean I), mean fundamental frequency subject whether the next stimulus should be classified (F0), and variability of F0 (SD of F0, normalized to the mean according to the affective prosody (negative, neutral) or F0)werecalculatedfromI and F0 contours determined for word class (adjective, noun). This display was presented the whole stimulus duration using Praat software (version: for 6 sec and the voice stimuli were presented 2 sec after 4.3.20, www.praat.org; Boersma, 2001). Since a previous onset of the scale. Within each segment, subjects had fMRI study of our group revealed that mean I,numberof to classify half of the stimuli with respect to affective syllables, and stimulus duration can influence activation prosody and the other half according to the word class. within voice-sensitive regions (Wiethoff et al., 2008), stim- The task did not change for the three presentations of ulus selection aimed at minimizing differences in these each stimulus. Word classification was chosen as non- parameters over prosodic categories (angry, neutral) and emotional control task because its task difficulty is close word classes (adjectives, nouns). Valence (ranging from to that of emotion classification (see below results of 1 = highly negative to 9 = highly positive) and arousal the behavioral prestudy), whereas gender classification (ranging from 1 = very calming to 9 = highly arousing) is usually much easier to perform and can result in bi- expressed by word content and affective prosody, number ases due to different levels of cognitive effort (e.g., Bach of syllables, and acoustic parameters of the stimuli are et al., 2008). For the sake of statistical power and to have presented in Table 1. To verify that repeated presentation an equal number of response alternatives in the emotion of these stimuli results in priming with shortening of reac- classification and the word classification task, we restrict- tion times and that the tasks to be used in the main fMRI ed the present study to angry and neutral prosody. To experiment are comparable with respect to task difficulty, reduce synchronization of scanner noise and acoustic we evaluated our stimuli in a behavioral prestudy. stimulation, trial onsets were jittered relative to scan onset in steps of 500 msec and the intertrial interval ranged from 8.85 to 12.85 sec. Subjects were instructed Experimental Design that they should classify the stimuli as accurately and In both the behavioral and the main fMRI experiment, quickly as possible, and that they could report their de- stimuli were presented in an event-related design. To cision during stimulus presentation.

Table 1. Valence and Arousal as Expressed by Word Content and Affective Prosody, Number of Syllables, and Acoustic Parameters of the Stimuli

Valence Arousal (Word Content) (Word Content) Valence (Prosody) Arousal (Prosody) Number of Syllables Adjectives (AP) 4.9 ± 0.5 3.2 ± 0.4 2.9 ± 0.5 5.9 ± 0.5 2.8 ± 0.5 Adjectives (NP) 5.1 ± 0.5 3.3 ± 0.3 4.8 ± 0.2 2.7 ± 0.4 2.5 ± 1.2 Nouns (AP) 5.1 ± 0.4 2.0 ± 0.5 3.0 ± 0.6 6.0 ± 1.2 3.1 ± 1.0 Nouns (NP) 5.2 ± 0.4 1.8 ± 0.5 5.1 ± 0.3 3.0 ± 0.4 3.0 ± 1.1

Mean I [a.u.] SD of I [%] Mean F0 [Hz] %SD of F0 [%] Duration [sec] Adjectives (AP) 78.6 ± 0.3 10.9 ± 3.3 154.7 ± 39.7 15.3 ± 6.9 0.84 ± 0.35 Adjectives (NP) 78.6 ± 0.3 12.7 ± 2.1 143.0 ± 42.6 8.9 ± 4.0 0.82 ± 0.25 Nouns (AP) 78.5 ± 0.3 10.4 ± 2.3 164.8 ± 47.8 18.3 ± 12.2 0.86 ± 0.31 Nouns (NP) 78.6 ± 0.3 10.9 ± 3.0 150.1 ± 46.6 12.8 ± 6.3 0.81 ± 0.32

All values represent mean ± standard deviation. AP = angry prosody; NP = neutral prosody; a.u. = arbitrary units; Hz = Hertz.

Ethofer et al. 1257 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2009.21099 by guest on 26 September 2021 In the fMRI localizer scan, a block design with 32 au- to the first volume of the time series, unwarping by use ditory stimulation blocks (8 sec duration) and 16 silent of a static field map (Andersson, Hutton, Ashburner, blocks (8 sec duration) was employed. The distribution Turner, & Friston, 2001), normalization into MNI space of the silent blocks was pseudorandomized and presen- (Montreal Neurological Institute, Collins, Neelin, Peters, tation of auditory stimulation blocks was fully random- & Evans, 1994, resampled voxel size: 3 Â 3 Â 3mm3) and ized. Auditory stimulation blocks included 16 blocks spatial smoothing with an isotropic Gaussian filter (10 mm with human voices (e.g., speech, sighs, laughs), 8 blocks full width at half maximum). Statistical analysis relied on a with animal sounds (cries of various animals), and 8 general linear model (Friston et al., 1994). For the main blocks with modern human environmental sounds (e.g., experiment, separate regressors were defined for each of doors, bells, telephones, cars). (For a detailed descrip- the 12 conditions of the 3 Â 2 Â 2 factorial design using Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/21/7/1255/1760188/jocn.2009.21099.pdf by guest on 18 May 2021 tion of the stimulus material, see Belin et al., 2000.) Sub- a stick function convolved with the hemodynamic re- jects were instructed to close their eyes and to listen to sponse function. Events were time-locked to stimulus the stimuli. In both fMRI sessions, stimuli were present- onset. For the localizer scan, three regressors were de- ed binaurally via magnetic resonance-compatible head- fined for presentation of human voices, animal voices, phones with piezoelectric signal transmission (Jancke, and modern human environmental sounds using a box- Wustenberg, Scheich, & Heinze, 2002). car function with a length of 8 sec that were convolved with the hemodynamic response function. To remove low-frequency components, a high-pass filter with a cut- Analysis of Behavioral Data off frequency of 1/128 Hz was used. Serial autocorrela- Accuracy rates during the two tasks were compared using tions were accounted for by modeling the error term as two-tailed paired t tests. Reaction times were submitted a first-order autoregressive process with a coefficient of to a three-factorial ANOVA with repetition (first, second, 0.2 (Friston et al., 2002) plus a white noise component third), emotion (angry, prosody), and task (emotion vs. (Purdon & Weisskoff, 1998). word classification) as within-subject factors. All values The following contrast images were calculated for the are given in mean ± standard error of the mean. main experiment:

(1) To examine brain regions showing stimulus-driven Image Acquisition effects, we compared activation to angry prosody Structural and functional imaging data were acquired relative to neutral prosody (anger > neutral). using a 1.5-T whole-body scanner (Siemens AVANTO, (2) To investigate brain regions showing task-driven ef- Erlangen, Germany). A magnetization prepared rapid ac- fects, we compared activation during judgment of af- quisition gradient-echo sequence was employed to ac- fective prosody relative to judgment of word class quire high-resolution (1 Â 1 Â 1mm3)T1-weighted (emotion > word class). To exclude that task-related structural images (TR = 1980 msec, TE = 3.09 msec, differences in brain activation can be solely explained TI = 1100 msec). Functional images were obtained us- by differences in task difficulty, parameter estimates ing a multislice echo-planar imaging (EPI) sequence were extracted from the most significantly activated (26 axial slices acquired in descending order, slice thick- voxel of every brain region with stronger responses ness 4 mm + 1 mm gap, TR = 2.17 sec, TE = 40 msec, during emotion- versus word-classification task, and field of view [FOV] = 192 Â 192 mm2,64Â 64 matrix, then submitted to a regression analysis with reaction flip angle = 908, bandwidth 1532 Hz/Px). EPI time series time as independent variable. The regression residu- consisted of 490 images for the main experiment and als obtained from these analyses were subsequently 255 images for the localizer scan. For off-line correction of tested in separate paired t tests to investigate whether EPI image distortions, a static fieldmap (36 slices acquired task-related brain activity was still significant after neu- in descending order, slice thickness = 3 mm + 1 mm gap, tralization of possible contributions of task difficulty. TR = 487 msec, TE (1) = 5.28 msec, TE (2) = 10.04 msec, (3) To identify brain regions showing repetition sup- FOV = 192 Â 192 mm2,64Â 64 matrix) was acquired pression effects, we compared activation to the first prior to the functional measurements. versus second presentation and to the second versus the third presentation of voice stimuli (first > second and second > third, respectively). Data Analysis (4) To identify brain regions where repetition suppres- For both the main experiment and the localizer scan, the sion is modulated by affective prosody, we computed first five fMRI volumes were discarded from further anal- the interaction between repetition and emotion. To ysis to exclude measurements that preceded T1 equilib- maximize sensitivity, we used the respective responses rium. Functional images were analyzed using statistical of the first and third presentation for this analysis parametric mapping software (SPM5, Wellcome Depart- ([angry prosody, first presentation > angry prosody, ment of Imaging Neuroscience, London, UK). For both third presentation] > [neutral prosody, first presenta- sessions, preprocessing steps comprised realignment tion > neutral prosody, third presentation]).

1258 Journal of Cognitive Neuroscience Volume 21, Number 7 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2009.21099 by guest on 26 September 2021 (5) To investigate brain regions where repetition suppres- RESULTS sion is modulated by task demands, we computed the Behavioral Data interaction between repetition and task. Again, to maximize sensitivity, we used the first and third In the behavioral prestudy, participants identified stim- presentation for this analysis ([judgment of emotion, uli during the word classification task with a slightly first presentation > judgment of emotion, third pre- higher accuracy (92.4 ± 1.8%) than during the emotion sentation] > [judgment of word class, first presenta- classification task (90.1 ± 2.1%). This difference in action > judgment of word class, third presentation]). curacy was not significant. A three-factorial ANOVA with repetition (first, second, third), emotion (angry, neutral),

and task (emotional classification, word classification) Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/21/7/1255/1760188/jocn.2009.21099.pdf by guest on 18 May 2021 Statistical inference was based on second-level random as within-subject factors and reaction time as depen- effects analyses. Activations are reported descriptively at dent variable yielded a significant main effect of repe- a height threshold of p < .005 and an extent threshold tition [F(2, 10) = 29.6, p < .001]. This was due to an of k > 8 voxels. Significance was assessed at the clus- acceleration of reaction times between the first (1.30 ± ter level with an extent threshold of p < .05 (corre- 0.11 sec) and the second presentations [1.03 ± 0.09 sec, sponding to a minimal cluster size of 180 voxels), paired t(11) = 7.8, p < .001, one-tailed; see Figure 1A]. corrected for multiple comparisons across the whole No such acceleration occurred between the second brain. For the activations within the amygdala and the and the third presentations (1.03 ± 0.10 sec). No main OFC, the search volume was restricted to these brain re- effect of task or emotion was found. A significant inter- gions as defined by the automatic anatomical labeling action between emotion and task [F(1, 11) = 17.3, p < toolbox (Tzourio-Mazoyer et al., 2002) and correction .01] was due to faster reaction times for angrily than for multiple comparisons was carried out using small for neutrally spoken words during the emotion classi- volume correction (Worsley et al., 1996). fication task, whereas the reverse pattern with longer To examine whether brain regions showing task-driven reaction times for angry than for neutral prosody was or stimulus-driven effects are modulated by repeated observed during the word classification task. No other presentation, parameter estimates from these regions first- or second-order interaction was significant. were submitted to repeated measures three-factorial In the fMRI study, participants judged the stimuli with ANOVAs with repetition (first, second, third), emotion similar accuracy rates as those in the behavioral pre- (angry, prosody), and task (judgment of emotion > study. However, the difference in accuracy rates be- judgment of word class) as within-subject factors. tween the word classification task (92.8 ± 0.7%) and To investigate whether stimulus-driven effects due to the emotional classification task (89.0 ± 1.6%) was now affective prosody are located within voice-sensitive re- slightly more pronounced and reached significance gions, brain activation maps during perception of hu- [paired t(23) = 2.6, p < .05, two-tailed]. A similar pat- man voices were contrasted with brain activation maps tern of reaction times was found in the fMRI study as during perception of animal sounds and environmental in the behavioral prestudy (see Figure 1B). The three- sounds. Statistical parametric t maps obtained at the factorial ANOVA on reaction times yielded a significant single subject level were obtained at a height threshold main effect of repetition [F(2, 22) = 105.6, p < .001]: of p < .005 (uncorrected) and probability maps showing Again, there was an acceleration between the first (1.51 ± a differential activation at this threshold for (i) at least 0.07 sec) and the second presentations [1.19 ± 0.06 sec, 50% and (ii) at least 75% of the subjects were calculated paired t(23) = 12.0, p < .001, one-tailed], but also across the whole brain. between the second and third presentations [1.13 ±

Figure 1. Reaction times during the behavioral prestudy (A) and the fMRI experiment (B).

Ethofer et al. 1259 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2009.21099 by guest on 26 September 2021 0.06 sec, paired t(23) = 3.3, p < .05, one-tailed]. Fur- icant main effect of repetition for the bilateral IFG/OFC thermore, the main effect of task was also significant [F(2, 22) = 24.3, p < .001 and F(2, 22) = 8.5, p < .01 for during the fMRI study [F(1, 23) = 4.5, p < .05] with the right and left side, respectively] and for the right pos- slightly faster reaction times for word classification terior MTG/STS [F(2, 22) = 20.9, p < .001]. Moreover, a (1.23 ± 0.05 sec) than emotion classification [1.32 ± significant main effect of emotion was also found for 0.07 sec, paired t(23) = 2.2, p < .05, two-tailed]. There the bilateral IFG/OFC [F(1, 23) = 18.8, p < .001 and F(1, was no main effect of emotion, and no first- or second- 23) = 8.6, p < .01 for the right and left side, respectively] order interaction was significant in the fMRI experiment. and the right posterior MTG [F(1, 23) = 23.2, p < .001]. The interaction between emotion and repetition was marginally significant for the bilateral IFG/OFC [F(2, 22) = Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/21/7/1255/1760188/jocn.2009.21099.pdf by guest on 18 May 2021 fMRI Results: Functional Voice Localizer 3.4, p =.051andF(2, 22) = 3.3, p = .06 for the right and In the localizer scan, a consistent pattern with stronger left side, respectively]. No other first- or second-order in- responses to human voices than to animal voices or en- teraction was found to be significant in brain regions vironmental sounds was found in the bilateral STG ex- showing task-related differences in activation. clusively (see Figure 2B). This selective activation in Comparison of event-related responses during first the bilateral STG is consistent with previous studies on and the second presentations revealed widespread acti- voice-specific areas in the temporal cortex (Ethofer et al., vations in cortical areas including the bilateral STG, the 2007; Ethofer, Anders, Wiethoff, et al., 2006; Grandjean middle and inferior frontal cortex, the parietal and oc- et al., 2005; Belin et al., 2000). cipital cortex, as well as subcortical structures, such as the thalamus and the caudate nucleus (see Figure 4A, brain areas marked in red and yellow). Regions that showed fMRI Results: Main Experiment additionally significant habituation of their hemody- In the main experiment on task- and stimulus-dependent namic responses between the second and third presen- effects, we first compared the brain responses to angry tations were located in the bilateral inferior frontal and and neutral prosody. This contrast revealed strong ac- superior temporal cortex (see Figure 4A, brain areas tivations within the bilateral STG, IFG/OFC, insula, amyg- marked in blue and light blue). dala, and mediodorsal thalami (see Figure 2A and A whole-brain analysis revealed that the right and left Table 2). The three-factorial repeated measures ANOVA OFC were the only brain areas showing an interaction on activation parameters (betas) extracted from these between emotion and repetition. This interaction was regions revealed a significant main effect of repetition due to stronger repetition decreases for angry than for for all these brain regions [all F(1, 22) > 7, p < .01]. In neutral prosody (see Figure 4B and Table 2). A similar addition, the bilateral IFG/OFC showed a main effect whole-brain analysis on interactions between task and of task [F(1, 23) = 6.1, p < .05 and F(1, 23) = 32.3, repetition did not yield any suprathreshold clusters. p < .001 for the right and left side, respectively], as well as an interaction between emotion and repetition DISCUSSION [F(2, 22) = 7.7, p < .01 and F(2, 22) = 3.5, p < .05 for the right and left side, respectively]. The activation clusters The present neuroimaging study allowed us to disentan- in the bilateral STG showing stronger responses to angle the differential contributions of emotion, task instruc- gry than neutral prosody overlapped precisely with the tion, and stimulus repetition (or novelty) to cerebral region previously defined by the voice localizer experi- responses evoked by affective prosody. Brain imaging re- ment (compare Figure 2B and Figure 2C). sults converged with behavioral data to indicate diferen- We then examined task-related effects by comparing tial effects of stimulus and task factors on voice processing. activations during the emotion classification relative to the word classification task. This contrast showed in- Behavioral Results creases in the bilateral IFG/OFC, the right dorsomedial prefrontal cortex, and the right posterior MTG/STS (see We found reliable evidence for priming with acceleration Figure 3 and Table 2). These effects were not simply of reaction times between the first and the second pre- due to task difficulty because we found that residuals sentations during the behavioral prestudy. This accel- obtained from regression analyses using reaction time eration of the subjects’ responses did not interact with as an independent factor, and activation parameters task instructions or emotion expressed by prosody, in (betas) from each of these regions as dependent vari- agreement with previous behavioral findings for facial able, were still significantly larger for the emotion clas- expressions (Ishai et al., 2004). Moreover, similar hit rates sification than for the word classification task [paired and reaction times for the two tasks indicated a similar t(23) > 2.2 for all four regions, p < .05, one-tailed]. No degree of difficulty. However, there was an interaction significant activations were found in the reverse con- between emotion and task, reflecting the fact that trast. In addition, the three-factorial repeated measures subjects responded faster to words spoken in angry than ANOVA on responses in these regions revealed a signif- in neutral prosody during emotion classification, whereas

1260 Journal of Cognitive Neuroscience Volume 21, Number 7 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2009.21099 by guest on 26 September 2021 Figure 2. (A) Brain regions showing stimulus-driven effects (angry prosody > neutral prosody) rendered on the surface of a brain template and a transversal slice obtained at z = À15 with a threshold of p < .005 (uncorrected). Parameter estimates are presented for the right STG (upper left panel), the left STG (upper right panel), the right Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/21/7/1255/1760188/jocn.2009.21099.pdf by guest on 18 May 2021 IFG/OFC (lower left panel), and the left IFG/OFC lower right panel). (B) Voice-sensitive regions obtained in the localizer scan, showing consistent activation at a height threshold of p < .005 (uncorrected) in at least 50% (blue) or 75% (red) of the subjects rendered on a transversal slice at z = À3. (C) For comparison, stimulus-driven effects (angry prosody > neutral prosody) in the main fMRI experiment are illustrated at a threshold of p < .005 (uncorrected) on the same transversal slice at z = À3.

angry prosody resulted in longer reaction times during any emotionality. On the other hand, slower reaction word classification. A possible explanation for this pat- times for angry than for neutral prosody during word tern might be that, in some cases, one can already infer classification might reflect an involuntary shift of the the stimulus’ emotional intention from the first few syl- subjects’ attention to the emotional information in the lables, whereas the decision that a stimulus is neutral re- voice (Grandjean, Sander, Lucas, Scherer, & Vuilleumier, quires listening to the whole word before excluding 2007; Wambacq, Shea-Miller, & Abubakr, 2004).

Ethofer et al. 1261 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2009.21099 by guest on 26 September 2021 Table 2. Brain Regions Showing a Main Effect of Emotion, mostly due to a lack of prolongation of reaction times Main Effect of Task, or an Interaction between Emotion to angrily spoken words during word classification. Im- and Repetition portantly, however, hit rates for emotion discrimination Anatomical MNI Cluster were similar for the behavioral and the fMRI study Definition Coordinates Z Score Size (90.1% and 89.0%, respectively), indicating that this was not due to a general incomprehensibility of affective Main Effect of Emotion (Angry > Neutral) prosody in the presence of scanner noise. We speculate Right STG 57 30 À3 5.53 593* that the higher level of attention the subjects had to maintain during scanning to successfully complete the

Left STG À57 À15 À3 4.74 387* Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/21/7/1255/1760188/jocn.2009.21099.pdf by guest on 18 May 2021 task made them less susceptible to involuntary shifts of Right IFG/OFC 48 30 À9 3.82 310* attention induced by affective prosody. Left IFG/OFC À48 27 À9 4.83 267* Left Insula À24 15 À18 3.85 107 Brain Areas Showing Stimulus-driven Activity Right Insula 30 6 À18 3.72 70 Mediodorsal Thalamus 6 3 6 3.46 74 Brain regions responding stronger to angry than to neutral prosody were situated bilaterally in the middle part Left amygdala À18 À9 À15 3.87 43** of the STG, IFG/OFC, amygdala, insula, and mediodorsal Right amygdala 18 À9 À18 2.90 9** thalamus. Stimulus-driven effects in the bilateral mid-STG strongly overlapped with sensitivity to human voices rel- Main Effect of Task (Emotional Classification > Word ative to animal voices or modern environmental sounds. Classification) These combined results confirm that a selective enhancement of neural responses by affective prosody occurs Left IFG/OFC À45 36 À9 5.66 357* within voice-sensitive regions of the associative auditory Right IFG/OFC 45 39 À6 4.20 275* cortex (Belin et al., 2000). The mid-STG has previously been reported to show stronger responses to a variety Right DMPFC 12 27 48 4.53 173 of emotional prosodic categories (Ethofer et al., 2007; Right posterior MTG/STS 51 À33 3 3.09 17 Ethofer, Anders, Wiethoff, et al., 2006; Grandjean et al., 2005). As in previous studies, this enhancement was Interaction between Emotion and Repetition ((Angry First > particularly prominent in the right hemisphere. However, Angry Third) > (Neutral First > Neutral Third)) in the present study, the left mid-STG also showed sig- Right OFC 54 30 À9 3.93 69*** nificant stimulus-driven effects that survived correction for multiple comparisons across the whole brain, sug- Left OFC À42 30 À15 3.04 31*** gesting that associative auditory cortices of both hemi- Height threshold: p < .005 (uncorrected), extent threshold k >8 spheres are involved in decoding affective information voxels. from the voice, with the right-hemispheric areas show- IFG = inferior frontal gyrus; OFC = orbito-frontal cortex; DMPFC = ing only a relative superiority. This interpretation is sup- dorsomedial prefrontal cortex; STG = superior temporal gyrus; MTG = ported by results from studies of patients with unilateral middle temporal gyrus. brain lesions demonstrating that left-hemispheric brain *p < .05 (corrected at cluster level for multiple comparisons across the lesions produce impairments midway between healthy whole brain). controls and patients with right-hemispheric lesions **p < .05 (corrected at cluster level for multiple comparisons within the amygdala). during comprehension of affective prosody (Kucharska- Pietura, Phillips, Gernand, & David, 2003). ***p < .05 (corrected at cluster level for multiple comparisons with the OFC). Remarkably, our results reveal that brain activity in the bilateral mid-STG was not influenced by task demands. In keeping with previous results (Ethofer, Anders, In general, the same pattern of behavioral responses Wiethoff, et al., 2006), this insensitivity to task instruc- and reliable priming effects was observed during fMRI tions accords with the suggestion that the mid-STG con- scanning. Again, the facilitation of responses with repe- stitute a relatively early stage for the perceptual analysis tition did not interact with task instructions or vocally of voices, whose activity is mainly driven by stimulus expressed emotion. However, the difference in hit rates features irrespective of whether the subjects’ attention and reaction times between the two tasks was slightly is focused on affective prosody or some other aspect of more pronounced in the fMRI study, suggesting that the stimuli (Wildgruber, Ackermann, Kreifelts, & Ethofer, the scanner noise might have impeded judgment of af- 2006). Brain activity in the bilateral mid-STG showed fective prosody more than judgment of the word class. clear repetition suppression effects, however, there was Furthermore, the interaction between emotion and task no effect of task instructions or emotion on habituation failed to reach significance in the fMRI study, which was in this brain region.

1262 Journal of Cognitive Neuroscience Volume 21, Number 7 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2009.21099 by guest on 26 September 2021 Figure 3. Brain regions showing task-driven effects (emotion classification > word classification) rendered on the surface of a brain template and a sagittal slice obtained at x = 6 with a threshold of p < .005 (uncorrected). Parameter estimates are presented for the right MTG/STS (upper left panel), the right dorsomedial Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/21/7/1255/1760188/jocn.2009.21099.pdf by guest on 18 May 2021 prefrontal cortex (DMPFC; upper right panel), the right IFG/OFC (lower left panel), and the left IFG/OFC (lower right panel).

Other brain regions that showed a similar response that some enhancement of neural activity might already pattern, with clear sensitivity to stimulus-driven factors occur at early processing levels downstream to the audi- but insensitivity to task demands, and repetition sup- tory cortex. pression effects unaffected by either task or emotion, The limited temporal resolution of fMRI impedes di- included the bilateral amygdala, insula, and mediodorsal rect inference on the precise temporal distribution of thalami. Neuroimaging evidence for an emotional mod- brain activity. However, based on their shared functional ulation of voice processing in these brain regions is less properties including stimulus-driven brain activity, insen- established than for the mid-STG. In particular, for the sitivity to task demands, and marked repetition suppres- amygdala, conflicting results have been reported with sion without influences of task or emotion, we speculate either increased (Phillips et al., 1998) or decreased that the mediodorsal thalamus, the amygdala, the insula, (Morris et al., 1999) responses to fearful vocalizations and the mid-STG constitute an early processing stage (e.g., screams). However, the present study is in line during recognition of vocal emotions, which responds to with previous studies reporting stronger amygdala re- anger prosody in a largely automatic manner. sponses to angry relative to neutral prosody in the amygdala (Quadflieg et al., 2008; Sander et al., 2005) and the Brain Areas Showing Task-related Activity anterior insula (Quadflieg et al., 2008). Furthermore, increased activity to affective prosody within the medio- Brain regions showing stronger responses during the dorsal thalamus for a variety of emotional categories emotional classification than during the word classifica- including anger has been recently reported (Kreifelts, tion task were located in the right posterior MTG, in the Ethofer, Grodd, Erb, & Wildgruber, 2007), suggesting bilateral IFG/OFC, and in the right medial prefrontal

Ethofer et al. 1263 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2009.21099 by guest on 26 September 2021 Figure 4. (A) Brain regions showing habituation between the first and second presentations (yellow/red) and additionally between the second and the third presentations (light blue/blue) rendered on the surface of a brain template and a transversal slice obtained at z = 3 with a threshold of p < .005 (uncorrected). Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/21/7/1255/1760188/jocn.2009.21099.pdf by guest on 18 May 2021 (B) Brain regions showing an interaction between stimulus-driven effects and repetition ([angry prosody, first presentation > angry prosody, third presentation] > [neutral prosody, first presentation > neutral prosody, third presentation]) rendered on the surface of a brain template and a transversal slice obtained at z = À12 with a threshold of p < .005 (uncorrected). Parameter estimates are presented for the right OFC (lower left panel) and the left OFC (lower right panel).

cortex. Task-related activation within the right posterior prefrontal cortex increases with working memory load MTG is consistent with results from previous fMRI stud- (Mitchell, 2007). However, this result was not specific for ies, in which participants made explicit judgments on emotional prosody because it occurred in a similar man- the emotional category (Wildgruber et al., 2005; Mitchell ner during lexico-semantic processing. A recent fMRI et al., 2003) or emotional valence (Ethofer, Anders, Erb, study demonstrated activation of an auditory–motor Herbert, et al., 2006) of auditory stimuli. It has been mirror system in the premotor cortex during passive suggested (Wildgruber et al., 2006) that activation in listening to various affective vocalizations (Warren et al., the right posterior MTG might reflect effortful extraction 2006). No differential task- or stimulus-dependent re- of suprasegmental features (Poeppel et al., 2004; Meyer, sponses were found in this brain region in the present Alter, Friederici, Lohmann, & von Cramon, 2002; Zatorre, study. Suppression of spontaneous orofacial gestures 2001) that are characteristic for emotional information in due to the explicit categorization task may offer a pos- the voice. sible explanation for this discrepancy. Enhanced responses within the bilateral IFG/OFC dur- To our knowledge, no imaging study so far has indi- ing explicit categorization of affective prosody is also cated that the right medial prefrontal cortex is implicated consistent with previous neuroimaging results (Ethofer, in active judgment of affective prosody. Although this ef- Anders, Erb, Herbert, et al., 2006; Wildgruber et al., 2004; fect is therefore only reported descriptively here, it con- Imaizumi et al., 1997), and with lesion studies demon- verges with the view that medial prefrontal regions are strating that damage to either the right or left IFG/OFC critically involved in more subjective appraisal of emo- can impair comprehension of affective prosody (Hornak tional significance of sensory events, as already found et al., 2003). It has been suggested that that the in- in the visual domain (Posner et al., 2008; Vuilleumier, volvement of the prefrontal cortex during processing of Armony, & Dolan, 2003; Lane, Fink, Chau, & Dolan, 1997). emotional prosody might reflect engagement of work- Reaction times obtained for the emotion classifica- ing memory components because activity in the lateral tion task were slightly longer than those for the word

1264 Journal of Cognitive Neuroscience Volume 21, Number 7 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2009.21099 by guest on 26 September 2021 classification task, raising the question of whether task- sensory areas, but presumably occurs for many brain re- related effects might potentially be biased by differences gions engaged by a certain stimulus in the context of a in task difficulty. However, by using additional regres- given task. sion analyses, we were able to remove all the variance To disentangle habituation effects that are specific for correlating with reaction time, and thus, to demonstrate processing of angry prosody, we presented our stimuli that, for all regions showing these differences, the task- within a factorial design. Such factorial habituation de- related activation was still significantly larger for emo- signs represent an effective way of isolating habituation than for word classification. Therefore, it is unlikely tion effects attributable to certain cognitive factors from that different levels of difficulty can account for the task- other confounding components which might also show related activations found in our study. habituation (such as phonological, lexical, semantic, Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/21/7/1255/1760188/jocn.2009.21099.pdf by guest on 18 May 2021 In addition to task-related differences, the right pos- working memory, or motor components) as these con- terior MTG/STS and the bilateral IFC/OFC also exhibited founding factors are eliminated in the interaction be- stimulus-driven effects, with stronger responses to angry tween repetition and the cognitive factor of interest. than to neutral prosody. Thus, these results lend support The only brain region which showed a specific habitua- to the notion that neural activity within these cortical tion with an interaction between repetition and emo- areas is not exclusively driven by task demands, but is tion as expressed by prosody was found in the bilateral also modulated by stimulus properties. Furthermore, OFC. This interaction was due to the fact that hemo- these findings are in agreement with previous reports dynamic responses in the OFC strongly habituated for on the right MTG/STS and the IFC/OFC, showing differ- stimuli spoken in angry prosody, whereas habituation ential responses to vocally expressed happiness and for neutral trials was much less pronounced. Thus, the anger (Johnstone, van Reekum, Oakes, & Davidson, OFC showed particularly strong responses to stimuli that 2006). The presence of stimulus-driven effects within were both novel and emotional, suggesting that this task-dependent brain areas, albeit at a lower level of sig- region may be essential for prompting the organism to nificance than in the mid-STG, suggests that enhanced evaluate and respond to stimuli with affective value when neural responses to affective prosody spread through these have not been encountered before. the neural system beyond early cortical areas engaged This functional property makes the OFC an ideal site by voice analysis, and that emotional features may also for detection of socially relevant stimuli and adds to le- boost activity in higher-order cortical areas underlying sion studies suggesting that the OFC is critical for nor- explicit judgment of prosody. mal social functioning (Beer, John, Scabini, & Knight, 2006), especially for correct recognition and appropri- ate reaction to the emotion of anger (Blair & Cipolotti, General Effects of Stimulus Repetition and 2000). Furthermore, these novel results on vocally ex- Specific Interactions between Repetition pressed anger are in line with a previous neuroimaging and Affective Prosody study (Blair et al., 1999) showing enhanced responsive- The most significant decrease of hemodynamic re- ness of the OFC to angry relative to neutral faces. Taken sponses across the three stimulus presentations was together, these data point to a more general role of the found within the left inferior frontal cortex, a region OFC for the detection of negative emotional information that has been reported to show repetition suppression irrespective of modality—a hypothesis that is anatomi- effects for various types of stimuli including faces/scenes cally well supported by the fact that it receives highly (Bunzeck et al., 2006), objects (Vuilleumier, Schwartz, processed afferents from all sensory cortices (for a re- Duhoux, Dolan, & Driver, 2005), visually presented view, see Rolls, 2004) and received recent support from words (Meister, Buelte, Sparing, & Boroojerdi, 2007), an fMRI study evidencing differential responses of this spoken words/pseudowords (Orfanidou et al., 2006), and region in response to valence information expressed by spoken sentences (Dehaene-Lambertz et al., 2006; word content (Alia-Klein et al., 2007). Hasson et al., 2006). A significant habituation of brain In summary, our results shed new lights on several responses across the three presentations was also found functional properties of the brain regions implicated within bilateral superior temporal cortices in keeping in the processing of affective prosody. The bilateral with studies reporting repetition suppression effects in amygdala, insula, mediodorsal thalami, as well as voice- the auditory cortex for repeated exposure to acoustic sensitive regions in the mid-STG, demonstrated stimulus- stimuli (Dehaene-Lambertz et al., 2006; Hasson et al., driven effects with stronger responses to angry than 2006; Saad et al., 2006). However, comparison of brain to neutral prosody. These stimulus-driven effects were responses to the first and second stimulus presentations not influenced by task demands, suggesting that these revealed widespread activations in cortical areas includ- brain regions represent a processing stage for iden- ing bilateral sensory cortices in the temporal and occipital tification of emotional cues in the voice that operates lobes, prefrontal and motor cortices, as well as subcortical relatively independent of task-related control. In con- structures. These findings strongly suggest that the phe- trast to this, the right MTG/STS and the bilateral IFG/ nomenon of repetition suppression is not restricted to OFC showed significant task-related increases, but also

Ethofer et al. 1265 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2009.21099 by guest on 26 September 2021 responded stronger to angry than to neutral prosody, Blair, R. J., Morris, J. S., Frith, C. D., Perrett, D. I., & indicating that some features from early processing Dolan, R. J. (1999). Dissociable neural responses to facial expressions of sadness and anger. Brain, 122, levels are also encoded in higher-order cortices that 883–893. subserve explicit recognition of affective prosody. Habit- Boersma, P. (2001). Praat, a system for doing phonetics by uation of responses due to repeated stimulus presenta- computer. Glot International, 5, 341–345. tion was observed in widespread cortical and subcortical Buckner, R. L., Petersen, S. E., Ojemann, J. G., Miezin, F. M., regions, demonstrating that repetition suppression is Squire, L. R., & Raichle, M. E. (1995). Functional anatomical studies of explicit and implicit memory retrieval tasks. not limited to perceptual stages but involves a distrib- Journal of Neuroscience, 15, 12–29. uted system recruited by processing of emotional voices. Bunzeck, N., Schutze, H., & Duzel, E. (2006). Category-specific Importantly, the OFC was the only area that showed an organization of prefrontal response-facilitation during Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/21/7/1255/1760188/jocn.2009.21099.pdf by guest on 18 May 2021 emotion-specific habituation with enhanced responses priming. Neuropsychologia, 44, 1765–1776. during initial presentations of angry prosody that strong- Cohen, L., Jobert, A., Le Bihan, D., & Dehaene, S. (2004). Distinct unimodal and multimodal regions for word ly decreased across repetitions. This unique pattern with processing in the left temporal cortex. Neuroimage, 23, pronounced responses to stimuli that were both novel 1256–1270. and emotional points to this region as a key structure for Collins, D. L., Neelin, P., Peters, T. M., & Evans, A. C. (1994). appraisal of socially relevant information and, together Automatic 3D intersubject registration of MR volumetric with previous findings for angry facial expressions, sug- data in standardized Talairach space. Journal of Computer Assisted Tomography, 18, 192–205. gests a major role for the OFC in processing of anger Dehaene-Lambertz, G., Dehaene, S., Anton, J. L., Campagne, A., across different sensory modalities. Increased OFC ac- Ciuciu, P., Dehaene, G. P., et al. (2006). Functional tivations to vocal anger in patients with social phobia segregation of cortical language areas by sentence (Quadflieg et al., 2008) indicate that this might be of repetition. Human Brain Mapping, 27, 360–371. clinical relevance for diagnosis and, possibly, also mon- Ethofer, T., Anders, S., Erb, M., Droll, C., Royen, L., Saur, R., et al. (2006). Impact of voice on emotional judgment of itoring of treatment of patients with social disorders. faces: An event-related fMRI study. Human Brain Mapping, 27, 707–714. Ethofer, T., Anders, S., Erb, M., Herbert, C., Wiethoff, S., Acknowledgments Kissler, J., et al. (2006). Cerebral pathways in processing of affective prosody: A dynamic causal modeling study. This study was supported by the Deutsche Forschungsgemein- Neuroimage, 30, 580–587. schaft (Sonderforschungsbereich 550-B10). Ethofer, T., Anders, S., Wiethoff, S., Erb, M., Herbert, C., Saur, R., et al. (2006). Effects of prosodic emotional Reprint requests should be sent to Thomas Ethofer, Laboratory intensity on activation of associative auditory cortex. for Behavioral Neurology & Imaging of Cognition, Department NeuroReport, 17, 249–253. of Neurosciences and Clinic of Neurology, University Medical Ethofer, T., Pourtois, G., & Wildgruber, D. (2006). Center of Geneva, 1 Rue Michel-Servet, 1211 Geneva, Switzerland, Investigating audiovisual integration of emotional or via e-mail: [email protected]. signals in the human brain. Progress in Brain Research, 156, 345–361. Ethofer, T., Wiethoff, S., Anders, S., Kreifelts, B., Grodd, W., REFERENCES & Wildgruber, D. (2007). The voices of seduction: Cross-gender effects in processing of erotic prosody. Alia-Klein, N., Goldstein, R. Z., Tomasi, D., Zhang, L., Social Cognitive and Affective Neuroscience, 2, Fagin-Jones, S., Telang, F., et al. (2007). What is in 334–337. a word? No versus Yes differentially engage the lateral Fecteau, S., Belin, P., Joanette, Y., & Armony, J. L. (2007). orbitofrontal cortex. Emotion, 7, 649–659. Amygdala responses to nonlinguistic emotional Andersson, J. L., Hutton, C., Ashburner, J., Turner, R., vocalizations. Neuroimage, 36, 480–487. & Friston, K. J. (2001). Modeling geometric deformations Friston, K. J., Glaser, D. E., Henson, R. N., Kiebel, S., Phillips, in EPI time series. Neuroimage, 13, 903–919. C., & Ashburner, J. (2002). Classical and Bayesian Bach, D., Grandjean, D., Sander, D., Herdener, M., Strik, W. K., inference in neuroimaging: Applications. Neuroimage, & Seifritz, E. (2008). The effect of appraisal level on 16, 484–512. processing of emotional prosody in meaningless speech. Friston, K. J., Holmes, A. P., Worsley, K. J., Poline, J. P., Frith, Neuroimage, 42, 919–927. C. D., & Frackowiak, R. S. J. (1994). Statistical parametric Beer, J. S., John, O. P., Scabini, D., & Knight, R. T. (2006). maps in neuroimaging: A general linear approach. Human Orbitofrontal cortex and social behavior: Integrating Brain Mapping, 2, 189–210. self-monitoring and emotion–cognition interactions. Glasser, M. F., & Rilling, J. K. (2008). DTI tractography of the Journal of Cognitive Neuroscience, 18, 871–879. human brain’s language pathways. Cerebral Cortex. Epub Belin, P., & Zatorre, R. J. (2003). Adaptation to speaker’s ahead of print. voice in right anterior temporal lobe. NeuroReport, Grandjean, D., Sander, D., Lucas, N., Scherer, K. R., & 14, 2105–2109. Vuilleumier, P. (2007). Effects of emotional prosody on Belin, P., Zatorre, R. J., Lafaille, P., Ahad, P., & Pike, B. auditory extinction for voices in patients with spatial neglect. (2000). Voice-selective areas in human auditory cortex. Neuropsychologia, 46, 487–496. Nature, 403, 309–312. Grandjean, D., Sander, D., Pourtois, G., Schwartz, S., Blair, R. J., & Cipolotti, L. (2000). Impaired social response Seghier, M. L., Scherer, K. R., et al. (2005). The voices of reversal. A case of ‘‘acquired sociopathy’’. Brain, wrath: Brain responses to angry prosody in meaningless 123, 1122–1141. speech. Nature Neuroscience, 8, 145–146.

1266 Journal of Cognitive Neuroscience Volume 21, Number 7 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2009.21099 by guest on 26 September 2021 Grill-Spector, K., Henson, R., & Martin, A. (2006). Repetition mediating slow prosodic modulations in spoken sentences. and the brain: Neural models of stimulus-specific effects. Human Brain Mapping, 17, 73–88. Trends in Cognitive Sciences, 10, 14–23. Mitchell, R. L. (2007). fMRI delineation of working memory for Grill-Spector, K., Kushnir, T., Edelman, S., Avidan, G., emotional prosody in the brain: Commonalities with the Itzchak, Y., & Malach, R. (1999). Differential processing lexico-semantic emotion network. Neuroimage, 36, of objects under various viewing conditions in the 1015–1025. human lateral occipital complex. Neuron, 24, 187–203. Mitchell, R. L., Elliott, R., Barry, M., Cruttenden, A., Hasson, U., Nusbaum, H. C., & Small, S. L. (2006). Repetition & Woodruff, P. W. (2003). The neural response to emotional suppression for spoken sentences and the effect of task prosody, as revealed by functional magnetic resonance demands. Journal of Cognitive Neuroscience, 18, 2013–2029. imaging. Neuropsychologia, 41, 1410–1421. Haxby, J. V., Gobbini, M. I., Furey, M. L., Ishai, A., Morris, J. S., Scott, S. K., & Dolan, R. J. (1999). Saying it with Schouten, J. L., & Pietrini, P. (2001). Distributed and feeling: Neural responses to emotional vocalizations. Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/21/7/1255/1760188/jocn.2009.21099.pdf by guest on 18 May 2021 overlapping representations of faces and objects in ventral Neuropsychologia, 37, 1155–1163. temporal cortex. Science, 293, 2425–2430. Obleser, J., Wise, R. J., Alex Dresner, M., & Scott, S. K. (2007). Henson, R. (2000). Neuroimaging evidence for dissociable Functional integration across brain regions improves forms of repetition priming. Science, 287, 1269–1272. speech perception under adverse listening conditions. Henson, R. N., & Mouchlianitis, E. (2007). Effect of spatial Journal of Neuroscience, 27, 2283–2289. attention on stimulus-specific haemodynamic repetition Oldfield, R. C. (1971). The assessment and analysis of effects. Neuroimage, 35, 1317–1329. handedness: The Edinburgh inventory. Neuropsychologia, Henson, R. N., Shallice, T., Gorno-Tempini, M. L., & Dolan, R. J. 9, 97–113. (2002). Face repetition effects in implicit and explicit Orfanidou, E., Marslen-Wilson, W. D., & Davis, M. H. (2006). memory tests as measured by fMRI. Cerebral Cortex, 12, Neural response suppression predicts repetition priming 178–186. of spoken words and pseudowords. Journal of Cognitive Hornak, J., Bramham, J., Rolls, E. T., Morris, R. G., Neuroscience, 18, 1237–1252. O’Doherty, J., Bullock, P. R., et al. (2003). Changes in Phillips, M. L., Young, A. W., Scott, S. K., Calder, A. J., emotion after circumscribed surgical lesions of the Andrew, C., Giampietro, V., et al. (1998). Neural responses orbitofrontal and cingulate cortices. Brain, to facial and vocal expressions of fear and disgust. 126, 1691–1712. Proceedings of the Royal Society of London, Series B, Imaizumi, S., Mori, K., Kiritani, S., Kawashima, R., Sugiura, M., Biological Sciences, 265, 1809–1817. Fukuda, H., et al. (1997). Vocal identification of speaker Poeppel, D., Guillemin, A., Thompson, J., Fritz, J., Bavelier, D., and emotion activates different brain regions. NeuroReport, & Braun, A. R. (2004). Auditory lexical decision, categorical 8, 2809–2812. perception, and FM direction discrimination differentially Ishai, A., Pessoa, L., Bikle, P. C., & Ungerleider, L. G. (2004). engage left and right auditory cortex. Neuropsychologia, Repetition suppression of faces is modulated by emotion. 42, 183–200. Proceedings of the National Academy of Sciences, U.S.A., Posner, J., Russell, J. A., Gerber, A., Gorman, D., Colibazzi, T., 101, 9827–9832. Yu, S., et al. (2008). The neurophysiological bases of Jancke, L., Wustenberg, T., Scheich, H., & Heinze, H. J. (2002). emotion: An fMRI study of the affective circumplex using Phonetic perception and the temporal cortex. Neuroimage, emotion-denoting words. Human Brain Mapping. Epub 15, 733–746. ahead of print. Johnstone, T., van Reekum, C. M., Oakes, T. R., & Davidson, Purdon, P. L., & Weisskoff, R. M. (1998). Effect of temporal R. J. (2006). The voice of emotion: An FMRI study autocorrelation due to physiological noise and stimulus of neural responses to angry and happy vocal paradigm on voxel-level false-positive rates in fMRI. expressions. Social Cognitive and Affective Neuroscience, 1, Human Brain Mapping, 6, 239–249. 242–249. Quadflieg, S., Mohr, A., Mentzel, H. J., Miltner, W. H., Kanwisher, N., McDermott, J., & Chun, M. M. (1997). & Straube, T. (2008). Modulation of the neural network The fusiform face area: A module in human extrastriate involved in the processing of anger prosody: The role of cortex specialized for face perception. Journal of task-relevance and social phobia. Biological Psychology, Neuroscience, 17, 4302–4311. 78, 129–137. Kreifelts, B., Ethofer, T., Grodd, W., Erb, M., & Wildgruber, D. Rolls, E. T. (2004). The functions of the orbitofrontal cortex. (2007). Audiovisual integration of emotional signals in Brain and Cognition, 55, 11–29. voice and face: An event-related fMRI study. Neuroimage, Saad, Z. S., Chen, G., Reynolds, R. C., Christidis, P. P., 37, 1445–1456. Hammett, K. R., Bellgowan, P. S., et al. (2006). Functional Kucharska-Pietura, K., Phillips, M. L., Gernand, W., & David, imaging analysis contest (FIAC) analysis according to A. S. (2003). Perception of emotions from faces and voices AFNI and SUMA. Human Brain Mapping, 27, 417–424. following unilateral brain damage. Neuropsychologia, 41, Sander, D., Grandjean, D., Pourtois, G., Schwartz, S., 1082–1090. Seghier, M. L., Scherer, K. R., et al. (2005). Emotion and Lane, R. D., Fink, G. R., Chau, P. M., & Dolan, R. J. (1997). attention interactions in social cognition: Brain regions Neural activation during selective attention to subjective involved in processing anger prosody. Neuroimage, emotional responses. NeuroReport, 8, 3969–3972. 28, 848–858. Li, L., Miller, E. K., & Desimone, R. (1993). The representation Sobotka, S., & Ringo, J. L. (1996). Mnemonic responses of of stimulus familiarity in anterior inferior temporal cortex. single units recorded from monkey inferotemporal cortex, Journal of Neurophysiology, 69, 1918–1929. accessed via transcommissural versus direct pathways: A Meister, I. G., Buelte, D., Sparing, R., & Boroojerdi, B. (2007). dissociation between unit activity and behavior. Journal A repetition suppression effect lasting several days within of Neuroscience, 16, 4222–4230. the semantic network. Experimental Brain Research, Sokolov, E. N. (1963). Higher nervous functions; the orienting 183, 371–376. reflex. Annual Review of Physiology, 25, 545–580. Meyer, M., Alter, K., Friederici, A. D., Lohmann, G., Soon, C. S., Venkatraman, V., & Chee, M. W. (2003). & von Cramon, D. Y. (2002). fMRI reveals brain regions Stimulus repetition and hemodynamic response

Ethofer et al. 1267 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2009.21099 by guest on 26 September 2021 refractoriness in event-related fMRI. Human Brain of emotional prosody—Influence of acoustic parameters Mapping, 20, 1–12. and arousal. Neuroimage, 39, 885–893. Tulving, E., & Schacter, D. L. (1990). Priming and human Wildgruber, D., Ackermann, H., Kreifelts, B., & Ethofer, T. memory systems. Science, 247, 301–306. (2006). Cerebral processing of linguistic and emotional Tzourio-Mazoyer, N., Landeau, B., Papathanassiou, D., prosody: fMRI studies. Progress in Brain Research, Crivello, F., Etard, O., Delcroix, N., et al. (2002). 156, 249–268. Automated anatomical labeling of activations in SPM using Wildgruber, D., Hertrich, I., Riecker, A., Erb, M., Anders, S., a macroscopic anatomical parcellation of the MNI MRI Grodd, W., et al. (2004). Distinct frontal regions subserve single-subject brain. Neuroimage, 15, 273–289. evaluation of linguistic and emotional aspects of speech Vuilleumier, P., Armony, J. L., & Dolan, R. J. (2003). intonation. Cerebral Cortex, 14, 1384–1389. Reciprocal links between emotion and attention. In R. S. J. Wildgruber, D., Pihan, H., Ackermann, H., Erb, M., & Grodd, W. Frackowiak, J. Ashburner, W. Penny, S. Zeki, K. J. Friston, (2002). Dynamic brain activation during processing of Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/21/7/1255/1760188/jocn.2009.21099.pdf by guest on 18 May 2021 C. D. Frith, et al. (Eds.), Human brain function emotional intonation: Influence of acoustic parameters, (2nd ed., pp. 419–444). San Diego: Academic Press. emotional valence, and sex. Neuroimage, 15, 856–869. Vuilleumier, P., Schwartz, S., Duhoux, S., Dolan, R. J., & Wildgruber, D., Riecker, A., Hertrich, I., Erb, M., Grodd, W., Driver, J. (2005). Selective attention modulates neural Ethofer, T., et al. (2005). Identification of emotional substrates of repetition priming and ‘‘implicit’’ visual intonation evaluated by fMRI. Neuroimage, 24, memory: Suppressions and enhancements revealed by 1233–1241. fMRI. Journal of Cognitive Neuroscience, 17, 1245–1260. Worsley, K. J., Marrett, S., Neelin, P., Vandal, A. C., Friston, K. J., Wambacq, I. J., Shea-Miller, K. J., & Abubakr, A. (2004). & Evans, A. C. (1996). A unified statistical approach for Non-voluntary and voluntary processing of emotional determining significant signals in images of cerebral prosody: An event-related potentials study. NeuroReport, activation. Human Brain Mapping, 4, 74–90. 15, 555–559. Yi, D. J., Kelley, T. A., Marois, R., & Chun, M. M. (2006). Warren, J. E., Sauter, D. A., Eisner, F., Wiland, J., Dresner, M. A., Attentional modulation of repetition attenuation is Wise, R. J., et al. (2006). Positive emotions preferentially anatomically dissociable for scenes and faces. Brain engage an auditory–motor ‘‘mirror’’ system. Journal of Research, 1080, 53–62. Neuroscience, 26, 13067–13075. Zatorre, R. J. (2001). Neural specializations for tonal Wiethoff, S., Wildgruber, D., Kreifelts, B., Becker, H., processing. Annals of the New York Academy of Sciences, Herbert, C., Grodd, W., et al. (2008). Cerebral processing 930, 193–210.

1268 Journal of Cognitive Neuroscience Volume 21, Number 7 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2009.21099 by guest on 26 September 2021