Probabilistic models of expectation violation predict psychophysiological emotional responses to live concert music Hauke Egermann, Marcus T. Pearce, Geraint A. Wiggins & Stephen McAdams Cognitive, Affective, & Behavioral Neuroscience ISSN 1530-7026 Cogn Affect Behav Neurosci DOI 10.3758/s13415-013-0161-y 1 23 Your article is protected by copyright and all rights are held exclusively by Psychonomic Society, Inc.. This e-offprint is for personal use only and shall not be self-archived in electronic repositories. If you wish to self-archive your article, please use the accepted manuscript version for posting on your own website. You may further deposit the accepted manuscript version in any repository, provided it is only made publicly available 12 months after official publication or later and provided acknowledgement is given to the original source of publication and a link is inserted to the published article on Springer's website. The link must be accompanied by the following text: "The final publication is available at link.springer.com”. 1 23 Author's personal copy Cogn Affect Behav Neurosci DOI 10.3758/s13415-013-0161-y Probabilistic models of expectation violation predict psychophysiological emotional responses to live concert music Hauke Egermann & Marcus T. Pearce & Geraint A. Wiggins & Stephen McAdams # Psychonomic Society, Inc. 2013 Abstract We present the results of a study testing the often- emotion induction, leading to a further understanding of the theorized role of musical expectations in inducing listeners’ frequently experienced emotional effects of music. emotions in a live flute concert experiment with 50 participants. Using an audience response system developed for this purpose, Keywords Emotion . Music . Expectation . Statistical we measured subjective experience and peripheral psychophys- learning . Computational modeling . Psychophysiology iological changes continuously. To confirm the existence of the link between expectation and emotion, we used a threefold approach. (1) On the basis of an information-theoretic cognitive Music has been shown to induce emotional reactions that are model, melodic pitch expectations were predicted by analyzing accompanied by activations in several reaction components: the musical stimuli used (six pieces of solo flute music). (2) A subjective feelings, psychophysiological activations, and ex- continuous rating scale was used by half of the audience to pressive behavior (Juslin & Västfjäll, 2008). However, most measure their experience of unexpectedness toward the music previous experimental research has been rather exploratory, heard. (3) Emotional reactions were measured using a showing that music induces emotion, but not providing theo- multicomponent approach: subjective feeling (valence and retically founded explanations for the phenomena observed. arousal rated continuously by the other half of the audience More than a decade ago, Scherer and Zentner (2001)noted: members), expressive behavior (facial EMG), and peripheral “This is a bad omen for future research, since it is to be feared arousal (the latter two being measured in all 50 participants). that additional, isolated research efforts with little or no theo- Results confirmed the predicted relationship between high- retical underpinnings are more likely to add to the current information-content musical events, the violation of musical confusion than to the insight to which the researchers aspire” expectations (in corresponding ratings), and emotional reac- (p. 382). However, beginning with Scherer and Zenter’s paper, tions (psychologically and physiologically). Musical structures several theoretical attempts have been made to explain the leading to expectation reactions were manifested in emotional underlying mental processes that are involved in creating reactions at different emotion component levels (increases in emotional responses to music. Scherer and Zentner formulated subjective arousal and autonomic nervous system activations). “production rules” describing in detail several mental mecha- These results emphasize the role of musical structure in nisms that could be used to explain emotional responses to music. A few years later, Juslin and Västfjäll continued this idea and presented a seminal review paper, positing seven : H. Egermann S. McAdams possible ways to explain the observed effects of music. McGill University, Montreal, Canada Here, they summarized previous ideas about emotion induc- tion mechanisms in general and those specific to music: H. Egermann (*) Audio Communication Group, Technische Universität Berlin, cognitive appraisal of music and the listening situation, brain Sekr. EN-8, Einsteinufer 17c, stem reflexes to acoustic characteristics, visual imagery in- 10587 Berlin, Germany duced through sound, evaluative conditioning from pairing e-mail: [email protected] music with another emotion-inducing stimulus, emotional M. T. Pearce : G. A. Wiggins episodic memory associated with the music, emotional conta- Queen Mary University of London, London, UK gion through emotional expressions in the music, and musical Author's personal copy Cogn Affect Behav Neurosci expectation. The last mechanism will be the focus of the study expectations have been investigated by comparing responses presented here. There have been many theoretical and empir- to chords varying in harmonic relatedness to the context with ical attempts to link musical structures and expectations, but more distantly related chords (assumed to be less expected) empirical evidence explicitly investigating the connection leading to longer reaction times in priming tasks between expectation and emotion in music is limited. (Bharucha & Stoeckig, 1986, with chord distance quan- Therefore, we conducted an experiment in which we tested tified by the number of shared parent keys), delayed whether statistical properties of composed musical structures and lower completion/expectation ratings (Bigand & violate or confirm subjective expectations, and also whether Pineau, 1997; Schmuckler & Bolz, 1994), and several specific they lead to emotional reactions in subjective, expressive, and event-related brain potentials, such as the P300 (Carrión & Bly, physiological response components. In order to maintain a 2008;Janata,1995). Concerning melodic expectations, several naturalistic research paradigm, we conducted the experiment theoretical models making expectation-related predictions in a live concert setting, using an audience response system have been suggested (Larson, 2004; Margulis, 2005; developed in-house that measured participants’ reactions in Ockelford, 2006) with partial empirical support. Music theorist real time. Advancing the theory on the underlying mecha- Eugene Narmour (1990, 1992) was among the most popular, nisms of musical expectation, we furthermore used a compu- proposing several melodic principles in his implication- tational machine-learning algorithm to analyze the music realization theory, which are intended to describe the expected presented and predict human expectations and emotions. melodic continuation of implicative intervals. Some of those principles, such as the principle of pitch proximity, have been confirmed in experimental testing (Cuddy & Lunney, 1995; Musical expectations Schellenberg, 1996, Thompson & Stainton, 1998). Unlike Meyer (1956), Narmour conceived some of his melodic orga- As early as the 1950s, Leonard B. Meyer (1956, 1957)began nization principles as universal, innate, and bottom-up process- to theorize about the relationships between musical structures es, similar to Gestalt principles of perception. However, recent and listener’s expectations (which may be confirmed or vio- theories of auditory statistical learning, also supported by lated). Emphasizing the role of cultural learning through ex- evidence reported by Huron (2006) and Pearce and Wiggins posure to the syntactical properties of music, he engendered a (2006), propose that melodic expectations do not rely on great deal of scholarship and empirical research, describing underlying patterns of universal bottom-up principles but have and testing which musical properties create which expecta- merely been formed through exposure to syntactic relation- tions. Reviewing this work, Huron (2006) suggested that there ships within musical structures of a given culture (Abdallah & are four different types of expectation associated with music Plumbley, 2009; Pearce & Wiggins, 2006). and created by different auditory memory modules. Veridical Furthermore, computational simulations of this learning expectations are derived from episodic memory and contain process have yielded robust predictions of perceptual ex- knowledge of the progression of a specific piece. Schematic pectations, outperforming other rule-based models like expectations arise from being exposed to certain musical Narmour's (1990, 1992). For that reason, our experiment styles and contain information about general event patterns uses a computational model of auditory expectation to spec- of different musical styles and music in general (based on ify precise, quantitative measures of structural predictability semantic memory). Dynamic expectations are built up through for each note in a melody (the information dynamics of knowledge stored in short-term memory about a specific piece music model [IDyOM]). The model itself has been presented that one is currently listening to and are updated in real time (Pearce 2005; Pearce, Conklin, & Wiggins, 2005; Pearce & through listening.