Expectancy and Musical Emotion Effects of Pitch and Timing
Total Page:16
File Type:pdf, Size:1020Kb
Manuscript Expectancy and musical emotion Effects of pitch and timing expectancy on musical emotion Sauvé, S. A.1, Sayed, A.1, Dean, R. T.2, Pearce, M. T.1 1Queen Mary, University of London 2Western Sydney University Author Note Correspondence can be addressed to Sarah Sauvé at [email protected] School of Electronic Engineering and Computer Science Queen Mary University of London, Mile End Road London E1 4NS United Kingdom +447733661107 Biographies S Sauve: Originally a pianist, Sarah is now a PhD candidate in the Electronic Engineering and Computer Science department at Queen Mary University of London studying expectancy and stream segregation, supported by a college studentship. EXPECTANCY AND MUSICAL EMOTION 2 A Sayed: Aminah completed her MSc in Computer Science at Queen Mary University of London, specializing in multimedia. R.T. Dean: Roger is a composer/improviser and researcher at the MARCS Institute for Brain, Behaviour and Development. His research focuses on music cognition and music computation, both analytic and generative. M.T. Pearce: Marcus is Senior Lecturer at Queen Mary University of London, director of the Music Cognition and EEG Labs and co-director of the Centre for Mind in Society. His research interests cover computational, psychological and neuroscientific aspects of music cognition, with a particular focus on dynamic, predictive processing of melodic, rhythmic and harmonic structure, and its impact on emotional and aesthetic experience. He is the author of the IDyOM model of auditory expectation based on statistical learning and probabilistic prediction. EXPECTANCY AND MUSICAL EMOTION 3 Abstract Pitch and timing information work hand in hand to create a coherent piece of music; but what happens when this information goes against the norm? Relationships between musical expectancy and emotional responses were investigated in a study conducted with 40 participants: 20 musicians and 20 non-musicians. Participants took part in one of two behavioural paradigms measuring continuous expectancy or emotional responses (arousal and valence) while listening to folk melodies that exhibited either high or low pitch predictability and high or low onset predictability. The causal influence of pitch predictability was investigated in an additional condition where pitch was artificially manipulated and a comparison conducted between original and manipulated forms; the dynamic correlative influence of pitch and timing information and its perception on emotional change during listening was evaluated using cross-sectional time series analysis. The results indicate that pitch and onset predictability are consistent predictors of perceived expectancy and emotional response, with onset carrying more weight than pitch. In addition, musicians and non-musicians do not differ in their responses, possibly due to shared cultural background and knowledge. The results demonstrate in a controlled lab-based setting a precise, quantitative relationship between the predictability of musical structure, expectation and emotional response. Keywords: emotion, expectation, time series, information content, IDyOM EXPECTANCY AND MUSICAL EMOTION 4 Effects of pitch and timing expectancy on musical emotion Music is capable of inducing powerful physiological and psychological emotional states (Bittman et al., 2013; Castillo-Pérez, Gómez-Pérez, Velasco, Pérez-Campos, & Mayoral, 2010). For example, the practice of music therapy stemmed from the observation that music can have a positive emotional impact (Khalfa, Bella, Roy, Peretz, & Lupien, 2003; Pelletier, 2004). However, many studies of emotion induction by music have simply investigated which emotions are induced rather the psychological mechanisms that account for why these emotions occur (Juslin & Västfjäll, 2008). The present research aims to address this omission by examining one theorized psychological mechanism of musical emotion induction in isolation. Although factors such as personality, age and gender have an influence (Rentfrow & Gosling, 2003), we focus here on the properties of music that are involved in emotion induction. While there is general consensus that music can elicit emotional responses (see Juslin & Sloboda, 2001, for an extensive review), why and how it does so is less clear. Juslin & Vastfjall (2008) describe six potential psychological mechanisms that might explain how emotions are induced through music: (1) brain stem reflexes, (2) evaluative conditioning, (3) emotional contagion, (4) visual imagery, (5) episodic memory, and (6) musical expectancy. Hearing a sudden loud or dissonant event causes a change in arousal (brain stem reflex) whereas a piece repetitively paired with a positive, or negative, situation will create a positive, or negative, emotional reaction (evaluative conditioning). Emotional contagion is the induction of emotion through mimicry of behavioural or vocal expression of emotion; for example shorter durations and ascending pitch contours tend to reflect happiness while longer durations and descending pitch contours communicate sadness. Visual imagery refers to the mental imagery evoked by the music, which can have positive or negative affect. Finally, pairing between a sound and a past EXPECTANCY AND MUSICAL EMOTION 5 event can trigger the emotion related to that event when the sound is subsequently heard (episodic memory). The present study focuses on musical expectancy, while controlling for all other potential mechanisms proposed above. Meyer (1956) argues that emotion is generated through musical listening because listeners actively generate predictions reflecting what they expect to hear next (see also Huron, 2006). Unexpected events are surprising and associated with an increase in tension while expected events are associated with resolution of tension (e.g. Gingras et al., 2016). According to this account, surprising events generally evoke high arousal and low valence (Egermann et al., 2013; Koelsch, Fritz, & Schlaug, 2008; Steinbeis, Koelsch, & Sloboda, 2006). However, listeners familiar with a piece of music can come to appreciate an event that has low expectancy through an appraisal mechanism, resulting in a high valence response (Huron, 2006). This apparent contradiction highlights the importance of isolating the psychological mechanisms behind musical emotional induction. There are also different influences on musical expectation (Huron, 2006). Schematic influences reflect general stylistic patterns acquired through extensive musical listening to many piece of music while veridical influences reflect specific knowledge of a familiar piece of music. Dynamic influences reflect dynamic learning of structure within an unfamiliar piece of music (e.g. recognizing a repeated motif). Listening to new, unfamiliar music in a familiar style engages schematic and dynamic mechanisms, the former reflecting long-term learning over years of musical exposure and the latter short-term learning within an individual piece of music. Both these long- and short-term mechanisms can be simulated as a process of statistical learning and probabilistic generation of expectations (Pearce, 2005). Furthermore, these may be different for musicians and non- musicians due to extensive exposure and training in a particular style (Juslin & Vastfjall, 2008). EXPECTANCY AND MUSICAL EMOTION 6 We now consider the properties of musical events for which expectations are generated. Prominent among such properties are the pitch and timing of notes and we consider each in turn. Music theorists have described musical styles as structurally organised, reflecting well formalised rules that constitute a kind of grammatical syntax (Lerdahl & Jackendoff, 1983). In the tradition of Western tonal music, compositions usually follow these rules by adhering to a tonal structure and enculturated listeners are able to identify when a piece of music infringes tonal rules based on their exposure in everyday music listening (Carlsen, 1981; Krumhansl, Louhivuori, Toiviainen, Järvinen, & Eerola, 1999; Trainor & Trehub, 1992). Two kinds of models have been developed to explain listeners’ pitch expectations: first, models that include static rules; and second, models that focus on learning. An influential example of a rule-based model is the Implication-Realization (IR) model, developed by Eugene Narmour (1991) which includes rules defining the expectedness of the final note in a sequence of three notes, where the first pair of notes forms the implicative interval and the second pair of notes the realized interval. The size and direction of the implicative interval sets up expectations of varying strengths for the realized interval. While the original IR model contained five bottom-up rules of melodic implication, Schellenberg (1997) reduced the five bottom-up rules of the IR model to two: pitch proximity and pitch reversal. For example, according to the rules of pitch reversal, a small interval implies another small interval in the same direction while a large interval implies a subsequent small interval in the opposite direction. Such patterns reflect actual patterns in existing Western music (Huron, 2006; Thompson & Stainton, 1996) suggesting the possibility that listeners might learn these patterns through experience. Statistical learning is a powerful tool for explaining the acquisition of pitch expectations in music, where common sequential patterns are learned through incidental exposure (Huron, EXPECTANCY AND MUSICAL EMOTION 7 2006; Pearce, 2005; Saffran, Johnson, Aslin, & Newport,