ORIGINAL RESEARCH ARTICLE published: 03 January 2014 doi: 10.3389/fpsyg.2013.01008 Action-based effects on music perception

Pieter-Jan Maes 1*, Marc Leman 2, Caroline Palmer 3 and Marcelo M. Wanderley 1

1 Department of Music Research, McGill University, Montreal, QC, Canada 2 Department of Musicology, Ghent University, Ghent, Belgium 3 Department of Psychology, McGill University, Montreal, QC, Canada

Edited by: The classical, disembodied approach to music cognition conceptualizes action and Adam M. Croom, University of perception as separate, peripheral processes. In contrast, embodied accounts of music Pennsylvania, USA cognition emphasize the central role of the close coupling of action and perception. It Reviewed by: is a commonly established fact that perception spurs action tendencies. We present Adam M. Croom, University of Pennsylvania, USA a theoretical framework that captures the ways in which the human motor system Martin Lotze, University of and its actions can reciprocally influence the perception of music. The cornerstone of Greifswald, Germany this framework is the common coding theory, postulating a representational overlap Michael Hove, Harvard Medical in the brain between the planning, the execution, and the perception of movement. School, USA The integration of action and perception in so-called internal models is explained as a *Correspondence: Pieter-Jan Maes, Department of result of associative learning processes. Characteristic of internal models is that they Music Research, McGill University, allow intended or perceived sensory states to be transferred into corresponding motor 527 Sherbrooke St. West, Montreal, commands (inverse modeling), and vice versa, to predict the sensory outcomes of planned QC H3A 1E3, Canada actions (forward modeling). Embodied accounts typically refer to inverse modeling to e-mail: pieter-jan.maes@ mail.mcgill.ca explain action effects on music perception (Leman, 2007). We extend this account by pinpointing forward modeling as an alternative mechanism by which action can modulate perception. We provide an extensive overview of recent empirical evidence in support of this idea. Additionally, we demonstrate that motor dysfunctions can cause perceptual disabilities, supporting the main idea of the paper that the human motor system plays a functional role in auditory perception. The finding that music perception is shaped by the human motor system and its actions suggests that the musical mind is highly embodied. However, we advocate for a more radical approach to embodied (music) cognition in the sense that it needs to be considered as a dynamical process, in which aspects of action, perception, introspection, and social interaction are of crucial importance.

Keywords: embodied music cognition, common coding theory, sensory-motor association learning, dynamical systems, internal model

1. INTRODUCTION emerge, demonstrating how the musical mind can be shaped Music is known to be a powerful medium that evokes body move- by the human motor system and the movements it produces ments in listeners, ranging from tapping the feet, shaking the (Phillips-Silver and Trainor, 2005, 2007; Repp and Knoblich, head, swaying the arms and hips, to more sophisticated forms of 2009; Sedlmeier et al., 2011; Iordanescu et al., 2013; Loehr, 2013; free or stylized dance. Research has shown that these body move- Manning and Schutz, 2013; Maes and Leman, 2013; Timm et al., ments often reflect the performer’s movements from which the 2013). This line of research reflects an important paradigm shift music originated (Leman et al., 2009; Godøy and Leman, 2010), within cognitive science. The classical view, inspired by the devel- certain aspects of the melody, harmony, rhythm and timbre (Maes opments of computer science and artificial intelligence in the et al., 2010; Naveda and Leman, 2010; Toiviainen et al., 2010; 1950s–1960s, pertains to an “information processing” approach Burger et al., 2013; Leman et al., 2013), or even the listeners’ mood that considers a strictly unidirectional information flow from per- (Van Dyck et al., 2013). These and similar studies importantly ception (input) to cognition (central processing unit) to action indicate that the listeners’ musical mind (attention, intention, (output) (Neisser, 1967; Laske, 1974; Fodor, 1975; Pylyshyn and mood, feelings, etc.) can be accessed through body movement, Demopoulos, 1986; Massaro, 1990). Accordingly, sensory infor- without the need for symbolic representations like language or mation received from the external world is perceived, translated musical scores. However, despite the explicit focus on the human into a syntactic code of meaningful symbols, and processed body and body movements, these and similar studies do not con- according to a systematic set of rules. Then, body movements sider the musical mind as being fundamentally embodied. The and other sorts of behavior are considered as mere outcomes of findings do not exclude the possibility that movement responses these higher-level, formal symbol manipulations. Hence, in this to music are mere peripheral epiphenomena resulting from cen- classical view of cognition, perception and action are completely tral cognitive processes. Only recently have studies started to separated from each other, and are outside central cognition

www.frontiersin.org January 2014 | Volume 4 | Article 1008 | 1 Maes et al. Action effects on music perception

[what Hurley (2001) describes as the “sandwich model of cog- music. Recently, there has been a proliferation of studies address- nition”]. This classical model is obsolete, as research shows that ing the role of forward models in action-based effects on visual, perception and action are strongly intertwined and can mutually auditory, and somatosensory perception. In the domain of visual exertinfluenceoneachother.Inwhatbecametheembodied cog- perception, several papers review action-based effects on visual nition theory, the human body - with its perceptual and motor perception (Schütz-Bosbach and Prinz, 2007; Shin et al., 2010; systems - and its interaction with the outside world, became Witt, 2011; Halász and Cunnington, 2012). Currently, such a central to human cognition (Varela et al., 1991; Leman, 2007; review of studies investigating action-based effects on auditory Chemero, 2009; Krueger, 2009; Glenberg, 2010; Shapiro, 2010). perception does not exist. An important goal of the present paper Within this framework of embodied cognition, the common cod- is to provide such a review of studies in support of the proposed ing theory (Prinz, 1990, 1997; Hommel et al., 2001) has been an theories and principles. influential theory postulating a close coupling between percep- The paper is structured as follows. In section 2, we argue tion and action. Although the theory is not readily falsifiable, it that sensory-motor association learning can be considered a cen- provides a general framework for developing more detailed and tral mechanism underlying the development of internal models. testable explanatory models (cf. Hommel et al., 2001). In essence, Accordingly, we claim that the ability to predict the auditory con- the theory states that the planning or execution of an action, and sequences of one’s actions, which is one of the core mechanisms of themereperceptionofthe(multi-)sensoryconsequencesofthat action-based effects on perception, depends on previous acquired action, are similarly represented (coded) in the brain, thereby sensory-motor associations. Further in that section, we define recruiting both sensory and motor brain areas. Important in this the concepts of temporal contiguity and probabilistic contingency theory is that the integration of motor and sensory representa- as two main principles underlying associative learning processes. tions leads to internal models of the relationship between the Additionally, we discuss musical instrument playing as a special body and the external environment, which can contain inverse but highly illustrative case of sensory-motor association learning. and forward components (Wolpert et al., 1995). Inverse models In section 3, we provide extensive empirical evidence for the claim represent an information flow from perception to action, in the that the principle of motor resonance, inherent in inverse mod- sense that they allow the system to estimate from incoming sen- els (section 3.1), together with auditory predictions generated by sory information the corresponding motor commands required forward models (section 3.2), can modulate auditory perception. to generate that specific sensory state [cf. Rizzolatti et al. (2001): Also, we demonstrate that deficits in the motor system may have direct-matching hypothesis]. In contrast, forward models repre- impaired auditory perception as a consequence (section 3.3). To sent an information flow from action to perception, in the sense conclude, an extensive discussion is presented in which we advo- that they allow to predict the likely sensory outcome of a planned cate a radical approach to embodied music cognition based on or executed action (Davidson and Wolpert, 2005; Bubic et al., dynamical systems. Moreover, we pinpoint music as an ideal study 2010; Waszak et al., 2012). Currently, the idea is gaining consensus object to extend this approach based on dynamical systems to that the combination of inverse and forward modeling processes embodied cognition, as it incorporates expressivity, introspection guides people’s interaction with the external world, including (affect, motivation, intentions, metacognition, etc.), and social motor control and sensory processing. interaction as crucial components. In the present paper, we set the common coding theory, and the related theory of internal models, as a theoretical frame- 2. ASSOCIATIVE LEARNING work for understanding action-based effects on music perception. Above, we outlined the common coding of action and percep- We conjecture that a focus on both inverse and forward mod- tion as a core mechanism underlying people’s engagement with eling processes can provide a comprehensive view of how the music (motor control and sensory processing). However, this human motor system and its actions influence music perception. account does not address the question of how action and per- In the domain of embodied music cognition, one typically refers ception become integrated. We advocate that this integration is to inverse modeling processes to explain action-based effects on established, in large part, through associative learning processes. music perception. Music spurs body movements that amount to The study of these processes can be traced back to the philos- expressive qualities, intentions, inner feelings, etc. Many of the ophy of Aristotle who stated that things that occur near each musical elements that contribute to expressivity (e.g., dynam- other in time and/or space are readily associated (i.e., law of ics, articulation, touch, phrasing, vibrato, rubato, etc.) directly contiguity). During the Enlightenment, these ideas were further relate to physical aspects of movement and space. Inverse mod- developed by the Associationist School (e.g., David Hume, John eling processes enable us to render (or decode) perceived patterns Locke, John Stuart Mill, etc.). In the nineteenth century William of musical expressivity into corresponding body movements. This James stated, as an elementary law of association, that “when two corporeal mirroring process is responsible for listeners’ tendency elementary brain-processes have been active together or in imme- to ascribe intentions, inner feelings, etc. to music (Godøy, 2003; diate succession, one of them, on reoccurring, tends to propagate Leman, 2007; Cox, 2011). We want to extend this “traditional” its excitement into the other” (James, 1890, p.566). In the late embodied perspective to the role of the human body in music 1940s, this principle was paraphrased in Hebb’s law “neurons that cognition with a focus on forward modeling processes. From fire together wire together.” A more recent account is the the- this perspective, it is not about how the body resonates with ory of associative sequence learning (ASL) introduced by (Heyes the music, but rather about how predicted sensory outcomes of and Ray, 2000). The ASL theory suggests that imitation is medi- planned or performed actions can be projected onto the perceived ated by associative processes that establish links between sensory

Frontiers in Psychology | Theoretical and Philosophical Psychology January 2014 | Volume 4 | Article 1008 | 2 Maes et al. Action effects on music perception

and motor representations. This theory has been applied to the process of internally generating a model of the stimulus rhythm. human mirror neuron system (MNS) in an attempt to recon- In a study of Leaver et al. (2009), anticipatory/predictive imagery sider its origin and function. The classical view on the MNS—as of musical melodies was shown to be associated with activation in originated in the work of Gallese et al. (1996); Rizzolatti et al. a variety of cortical (frontal and parietal) and subcortical (basal (2001); Kohler et al. (2002)—is that it is an innate system, only ganglia and cerebellum) structures. Interestingly, different neu- marginally influenced by sensory-motor experience, and inher- ral substrates underlay different stages of development of learned ently codes the meaning of actions (e.g., goals, intentions, etc.). conditional associations between melodies (“moderately learned” This view was soon adopted to explain various important psy- vs. “well-learned”). Findings show that the supplementary motor chological and social functions, such as action understanding, area and the basal ganglia (putamen) are particularly important learning by imitation, empathy, and social interaction. However, in early and moderate stages of learning, while the frontal cortex critical voices have been raised in opposition to this classical view, seems to dominate end stages (cf. Pasupathy and Miller, 2005). in particular to the idea that mirror neurons are adapted by evo- These dynamics in neural activation involved in sensorimotor lution to directly and consistently encode action goals (Hickok, association learning characterizes motor skill learning in general. 2009; Heyes, 2010; Catmur, 2012). The alternative view—what Studies have demonstrated that the recruitment of distributed Heyes (2010)termedtheassociative hypothesis—states that the brain regions in the process of acquiring motor skills depends development of the MNS is promoted by sensory-motor asso- on the type of motor task (motor sequence learning vs. motor ciative learning. Empirical evidence is provided in the context of adaptation) and on the stage of learning (fast learning, slow learn- music and dance. Haslinger et al. (2005) compared expert pianists ing, consolidation, automatization, retention) (Ungerleider et al., with musically naive controls with fMRI while observing piano- 2002; Luft and Buitrago, 2005; Doyon et al., 2009). playing and non-piano-playing finger movements. The results showed that the expert pianists exhibited stronger activation in 2.1. CONTINUITY AND CONTINGENCY brain areas associated with the MNS (inferior fronto-parieto- Auditory-motor association learning—i.e., the acquisition of temporal region) compared to the control participants. Similarly, knowledge of sound-movement relationships—is modulated in the context of dance, Calvo-Merino et al. (2005) showed that by both temporal “contiguity” and probabilistic “contingency” activation in brain areas related with the MNS in expert dancers (Cooper et al., 2012). “Contiguity” refers to the proximity of two (classical ballet and capoeira) was higher when they observed events (e.g., movement and sound) in time and space. The con- a familiar dance style. In conclusion, the associative hypothesis cept originates in Aristotle’s law of contiguity, stating that things states that, through systematically repeated experiences, sensory that occur near each other in time and/or space are readily asso- events are associated with particular motor acts and excitatory ciated. It is not, however, the case that association learning occurs links between both are created, resulting in the development of every time two events are linked together in time or space. Instead, “internal models.” Accordingly, when a sensory representation is it is necessary that the relationship between the events is pre- activated, the corresponding motor representation is automati- dictable. “Contingency” refers to this degree of probability or the cally co-activated (inverse modeling), and vice versa: when an likelihood that two or more events belong together. In statistical action is merely planned or executed, the corresponding sensory terms, contingency is related to covariance, being a measure of representation is automatically co-activated (forward modeling). how much two random variables change together. As will be explained further in Section 3, both inverse and for- Elsner and Hommel (2004) present two experiments in which ward modeling processes can contribute to action-based effects the role of contiguity and contingency were investigated in on auditory perception. the development of sensory-motor associations. Each experi- An important challenge of future research is to further iden- ment consisted of a training phase followed by a test phase. tify the neural substrates underlying associative learning pro- In the training phase, participants learned action-effect asso- cesses. Studies pinpoint the cerebellum (Imamizu and Kawato, ciations by repeatedly pressing keys (action) triggering corre- 2009; Timmann et al., 2010), the striatum—an input nucleus of sponding tones (effect). In the subsequent test phase, tones the basal ganglia—(Pasupathy and Miller, 2005; Williams and were presented and participants were asked to make speeded Eskandar, 2006; Lalazar and Vaadia, 2008; Melcher et al., 2012), responses to these stimuli by pressing keys either in a consis- prefrontal areas (Deiber et al., 1997; Bangert and Altenmüller, tent fashion (i.e., action-effect mapping as in the training phase) 2003; Pasupathy and Miller, 2005), the supplementary motor area or inconsistent fashion (i.e., other action-effect mapping as in (Pasupathy and Miller, 2005), and the premotor cortex (Deiber the training phase). If an action-effect association was estab- et al., 1997; Schubotz, 2007; Chen et al., 2009; Imamizu and lished in the training phase, then participants were expected Kawato, 2009) as important neural structures underlying asso- to respond faster in an acquisition-consistent fashion than in ciation learning leading to the development of internal models an acquisition-inconsistent fashion. In the training phase of and predictive mechanisms. In the field of music research, evi- Experiment 1, the contiguity between action and effect was sys- dence suggests that the striatum is involved in prediction and tematically manipulated by adding an increasing delay between anticipation. Grahn and Rowe (2013) assessed the role of the the two (50, 1000, and 2000 ms). In the test phase, participants putamen—one of the two nuclei that make up the striatum—in responded faster in acquisition-consistent test blocks compared beat prediction. Their findings show that the putamen becomes to acquisition-inconsistent test blocks when action-effects train- active only after having established a predictable sense of the ing delays were 50 or 1000 ms. Accordingly, association learning beat. Accordingly, they conclude that putamen activity reflects the seemed to be successful only with action-effect delays of up to

www.frontiersin.org January 2014 | Volume 4 | Article 1008 | 3 Maes et al. Action effects on music perception

1000 ms, signaling an effect of contiguity in association learn- to play a musical instrument, auditory-motor linkages are devel- ing. In the training phase of Experiment 2, the contingency oped as a result of that training (Pascal-Leone, 2001; Bangert and between action and effect was systematically manipulated by Altenmüller, 2003; Lotze et al., 2003; Lahav et al., 2005; D’Ausilio varying the relative frequencies of the presence or absence of et al., 2006; Lahav et al., 2007; Hyde et al., 2009; Herholz and tones with corresponding keypresses. Again, it was shown that Zatorre, 2012). Also studies have shown that during passive music the acquisition-consistency effect in the test phase was affected listening, trained musicians exhibit stronger auditory-motor cou- by the contingency of action and effect in the training phase. plings compared to non-musicians (Haueisen and Knösche, 2001; Together, these findings show that both the contiguity and con- Gaser and Schlaug, 2003; Baumann et al., 2007). This supports tingency between actions (here, keypresses) and auditory events the idea that auditory-motor linkages are established by inten- (here, sinusoidal tones, MIDI marimba/flute tones) are important sive training which involves long-term skill acquisition and the in the process of acquiring sensory-motor associations. repetitive rehearsal of the same skills (Brown and Palmer, 2012, An interesting experimental paradigm in which contiguity 2013). and contingency could be further investigated is the counter- It is evident that sensory-motor association processes are mirror sensory-motor training paradigm (Cook et al., 2010). important for voluntary action control, as in musical instrument In this paradigm, previously established associations between performance (Hommel, 1997, 2003; Elsner and Hommel, 2001). motor and sensory events are manipulated by repeatedly pair- However, more important in the light of the present paper is ing the observation of an action with the execution of another the idea that sensory-motor relationships, and the integration of action. One typically finds (e.g., by measuring neural responses, these relationships into internal models, may influence perceptual or reaction times) that the original sensory-motor association processes and accordingly shape the musical mind. In the follow- gets weakened, depending on the principles of contingency and ing sections, we will discuss empirical evidence demonstrating contiguity. This paradigm has been applied to visual-motor learn- that sensory-motor association learning, with musical instrument ing processes, but not yet to auditory-motor learning processes. training as a special case, may lead to action-based effects on However, the paradigm offers unique possibilities to study for auditory perception. instance how counter-mirror training can alter auditory-motor links established in musical instrument playing. 3. EMPIRICAL EVIDENCE: A REVIEW 3.1. INVERSE MODEL: PERCEPTION → ACTION 2.2. MUSICAL INSTRUMENT LEARNING Inverse models enable us to predict the motor commands that Learning to play an instrument can be considered a special, highly are required to achieve a desired sensory state. It is obvious that illustrative case of sensory-motor association learning in which this is of utmost importance when playing a musical instrument. action and perception become intricately interwoven. The act But inverse models hold an important role in music perception of playing an instrument can be considered as a goal-directed, as well, as they allow to predict and simulate the physical aspects intentional act (Dalla Bella and Palmer, 2011). Ultimately, the of motion and space implied in the music. There is ample evi- goal of playing a musical instrument is to produce a certain dence that merely listening to sounds or music automatically sound. However, in order to reach that goal, one first needs triggers motor responses, as a function of their previously estab- to obtain knowledge about the relationship between the actions lished associations [motor resonance (Schütz-Bosbach and Prinz, afforded by the musical instrument, and the auditory conse- 2007), perceiving action (Hurley, 2008), etc.]. This has been quences of these actions. This knowledge is gradually acquired shown in neurophysiological studies (Haueisen and Knösche, by exploring and manipulating the possibilities afforded by the 2001; Bangert and Altenmüller, 2003; Gaser and Schlaug, 2003; instrument using (at first) arbitrary actions that lead to (at first) Lahav et al., 2005, 2007; D’Ausilio et al., 2006; Baumann et al., unexpected auditory events (Hommel, 2003). In that process of 2007; Chen et al., 2008). Additionally, results from behavioral exploration and interaction, one systematically and repeatedly studies show that motor responses to sounds are typically faster associates performed actions with heard sounds, and internal when the specific sounds and actions have been repeatedly and models are developed as a result, capturing the relationship consistently paired on previous occasions (Elsner and Hommel, between actions and sound. For example, in the case of the 2001; Rusconi et al., 2006; Lidji et al., 2007; Trimarchi and piano, one starts to understand that the key-to-pitch mapping Luzzatti, 2011; Stewart et al., 2013a,b). These findings provide is functionally organized (left-right motion corresponds to low- support for the idea that an action becomes automatically acti- high pitch), or that depressing the sustain pedal creates a legato vated (or, primed) as a result of the mere perception of the 1 effect. At that point, playing a musical instrument may become auditory consequences normally associated with that action. a goal-directed act, in the sense that performers have the abil- Other studies have focused on overt body movements that people ity to intentionally produce certain sounds by performing certain make in response to music for music presented in visual form, or actions. Additionally, it must be noted that the process of explo- via motion imagery (Eitan and Granot, 2006; Leman et al., 2009; ration in which action and perception mutually interact, is a Caramiaux et al., 2010; Godøy, 2010; Kozak et al., 2012; Bernardi continuous process throughout the life of a music performer. et al., 2013; Küssner, 2013; Lotze, 2013). These studies show that It incorporates aspects of creativity, intuition and surprise, and people can consistently translate acoustic properties of sound and can in itself be a “raison d’être” of playing an instrument (cf. music into body movements, although Küssner (2013)reports Sudnow, 1978). A large body of empirical studies exist that support these ideas. 1see Cox and Hasselman (2013) for some critical remarks on typical effect- For example, it has been shown that when people are trained priming studies.

Frontiers in Psychology | Theoretical and Philosophical Psychology January 2014 | Volume 4 | Article 1008 | 4 Maes et al. Action effects on music perception

that musicians are more consistent (i.e., less varying) in visual- accompanied with intense feelings of enjoyment and creativity izing sound and music by means of drawings. More important in (Csikszentmihalyi, 1988). This aspect of motor resonance is an the scope of the present article is the idea that the power of music essential component of musical aesthetic experiences and is fun- to induce body movements in listeners implies that merely listen- damental for shaping the “musical mind.” Additionally, it may be ing to music becomes a kinaesthetic experience. Musical groove is a factor that explains the ability of music to alter people’s experi- a relevant example of a musical quality that induces body move- ence of space and time (Schäfer et al., 2013),andtocontributeto ments in listeners (Janata et al., 2012; Stupacher et al., 2013). people’s general well-being (Croom, 2011). The notion of music-induced body movement may be related to two ideas showing how inverse models, and the related concept 3.2. FORWARD MODEL: ACTION → PERCEPTION of motor resonance (or, motor simulation), can shape people’s As explained, forward internal models represent an information engagement with music and by extension the “musical mind.” flow from action to perception, in the sense that they allow the First, the recruitment of the body into the process of music prediction of the likely sensory outcome of a planned or exe- listening causes a connection to be made between the music and cuted action [cf. “perceptual resonance” (Schütz-Bosbach and the expressive qualities inherent to the movements that the music Prinz, 2007), “active perception” (Hurley, 2008), etc.]. Research induces. The human body acts thereby as a mediator between has pinpointed the cerebellum as a crucial locus for internal physical phenomena (sensory and motor processes) and subjec- forward models (Wolpert et al., 1998; Blakemore et al., 2001; tive, mental states (Leman, 2007). An interesting model to capture Knolle et al., 2012a; Ebner, 2013), presumably in interaction with the subtle qualities of movement expressivity is the Effort/Shape other brain structures [e.g., prefrontal areas (Lappe et al., 2013)]. model that originated in the Laban Movement Analysis (LMA) In this context, it is important to note that different predictive method (Laban, 1947; Laban and Ullmann, 1966). This model mechanisms exist which are supported by different brain systems. is particularly appropriate, as it provides an integrated concep- O’Reilly et al. (2013) for example differentiate between statisti- tual system connecting a set of physical movement properties cal and dynamic predictive models. Statistical models capture the with expressive qualities (e.g., weight, flow, space, time, etc.). The stochastic probability that two or more events are associated— model has been used in research to show how music-induced for example an action event and a reward or sensory event—and body movements correlate with verbal descriptors used by people are developed over a history of discrete events. Alternatively, to describe their perception of the music (Maes et al., 2014). in dynamic forward models, the relation between two events is Second, it is interesting to note that music-induced body deterministic and predictions are computed via explicit reference movements may instigate a sense of imagined participation with to pre-learned environmental dynamics. the production of the sound. This idea of imagined participa- Studies have shown that predictive models are important for tion is addressed in a broad range of musicological studies with motor control (Wolpert et al., 1995; Hommel, 1997), as well as different terminology, such as imagined activity (Maus, 1988), for the processing of sensory information coming from the exter- kinaesthetic empathy (Mead, 1999), imaginary agency (Levinson, nal environment (Halász and Cunnington, 2012). In the present 2006), simulated control (Leman, 2007), and active perception study, we focus on the latter in the context of auditory perception. (Krueger, 2009). What these accounts have in common is their We will discuss how sensory predictions generated by forward reference to a direct, sensory-motor engagement with music, models may influence the perception of sound and music. It will to how music literally “moves” people, and to how people feel be shown that sensory predictions can either attenuate, facilitate, immersed in, and resonate with, the physical sound energy. In or disambiguate auditory perception (cf. Halász and Cunnington, that sense, motor resonance may create the illusion of taking 2012). part in the actual skillful production of the music, which would be impossible in real life. Musical motion, however, is not lim- 3.2.1. Attenuation ited to purely physical movements of the human body. Schubotz Performing an action for which one can predict the sensory (2007) provides an answer to the question of how people can sim- consequences attenuates the perception of the actual sensory out- ulate or anticipate events that could not be readily reproduced by come, as reflected in self-reports and neuronal responses. In the their own motor system (e.g., rhythm of ocean waves, the flight domain of auditory perception, this phenomenon was first stud- of a mosquito, or an unfolding sequence of abstract stimuli on a iedinspeechproduction(Houde et al., 2002; Heinks-Maldonado computer screen). Schubotz demonstrates and explains that even et al., 2006). Later on, studies appeared in which the phenomenon abstract events—including auditory events—recruit our motor of motor-induced suppression (MIS) was studied with tones gen- system (in particular the premotor cortex and its parietal pro- erated by keypresses. Despite the fact that the tones and the jection areas) in order to support simulation and prediction actions that produce them (i.e., action-to-pitch mapping) are processes (see also Southgate, 2013). Accordingly, the micro and highly simple, a parallel can be drawn with musical instrument macro dynamics and subtleties inherent in the musical textures playing, like playing the piano, trumpet, etc. and structures, as for instance in the “Clocks and Clouds” (1973) A study conducted by Aliu et al. (2009) demonstrates that of György Ligeti or in electronic music productions (e.g., Infected the auditory response to tones generated by self-produced key- Mushroom, Aphex Twin, etc.), can evoke a fascinating continuum presses is attenuated relative to the response following passive of spatial imagery and motion, with which the listener may float listening to the same tones. However, because self- and externally- along. Accordingly, motor resonance may generate an experience generated tones were presented in separate blocked conditions, of flow, being a state of heightened focus and immersion, typically it could not be ruled out that the observed attenuation effect

www.frontiersin.org January 2014 | Volume 4 | Article 1008 | 5 Maes et al. Action effects on music perception

was modulated by differences in contextual task demands (e.g., to unveil the neural substrates of motor-based sensory predic- allocation of attention, arousal, etc.). To clarify this matter, Baess tion (Nelson et al., 2013; Roussel et al., 2013). However, more et al. (2011) mixed self- and externally-generated tones within research is needed in order to obtain a full picture of the neural blocks. The results of this study yielded an even larger attenu- mechanisms underlying the action effect of auditory attenuation. ation effect for self-generated tones than that observed in the blocked condition. Also, Timm et al. (2013)conductedastudy 3.2.2. Facilitation to further investigate the relationship between attention and the Manning and Schutz (2013) examined to what extent “moving to effects of motor prediction in perceiving auditory stimuli. The the beat” objectively improves timing perception. They presented study adapted the mixed paradigm of Baess et al. (2011) and addi- participants with sequences of 16 isochronous tones divided into tionally incorporated different conditions in which attention was groups of four followed by a probe tone. In the last group, the allocated to either the sound, the motor act, or to visual stim- second, third, and fourth “tones” were silent (i.e., timekeeping uli. Findings of this study demonstrated that an attenuation effect segment). The probe tone was “on-time” (i.e., sounding after for self-generated sounds was independent from the allocation of the same inter-onset interval), slightly early, or slightly late. The attention. Other studies investigated whether the attenuation of task of the participant was to judge whether the final probe tone auditory action effects occurs when actions are merely observed, sounded “on-time.” In one condition, participants were asked to instead of being self-generated. Sato (2008) hypothesized that, if tap along with the beat, while they remained still in the other there is a human mirror neuron system that codes a bidirectional condition. The results show that late offsets were better detected association between action execution and action perception, then when participants could move during the timekeeping segment. the mere observation of (well-learned) actions leading to a certain Additionally, it was found that “better” tappers (i.e., less variabil- auditory event should bring about a similar auditory attenua- ity) performed better on the detection task overall. In general, tioneffectaswhentheactionisself-generated.Theresultsofthis these findings confirm that movement may improve time percep- study confirmed this hypothesis, as similar auditory attenuation tion. Iordanescu et al. (2013) obtained similar results using a stan- was observed for self-generated and merely observed sound- dard temporal-bisection paradigm. Participants were presented producing actions. However, this finding was later contradicted in with sequences of three brief clicks with the location of the sec- astudybyWeiss and Schütz-Bosbach (2012), using a comparable ond click randomly varied. Participants had to judge whether the experimental protocol as in Sato (2008). They compared auditory second click was temporally closer to the first or the third click. In action effects for self-generated actions, observed unanticipated the “active” condition, participants initiated each trial themselves actions, and observed anticipated actions. The results showed that by pressing the space bar, while trials were externally generated in the attenuation of a sound is significantly higher when the sound- the “passive” condition. Again, in line with the results of Manning producing action is self-generated compared to merely observed. and Schutz (2013), people in the active condition demonstrated a Moreover, this effect was shown to be independent of whether the higher auditory sensitivity to temporal intervals. Moreover, it was observed action could be anticipated or not. This finding raises shown that this effect was not attributable to the tactile sensa- questions about the role that forward internal models play in tion from a keypress. It is interesting to note that the finding that the prediction mechanisms underlying action effects on auditory body movement can enhance time perception has been picked up attenuation (cf. Sato, 2008). More research is needed to clarify this by research in the domain of human-computer interaction (HCI) point. In a last study we address here, Knolle et al. (2012b)exam- design. Maes et al. (2012, 2013) present a dance application and ined whether auditory attenuation is a function of the degree of a music conducting application aiming to enhance users’ under- predictability of the self-generated sound. The result of this study standing of temporal musical structures by teaching them how indicated a lowering of the attenuation effect when self-generated to articulate these temporal structures into corresponding body sounds deviate from the expected outcome. movements (dancing, conducting). Together, these and similar studies (Baess et al., 2008; Hughes In another study, Brown and Palmer (2012)investigatedhow et al., 2013a,b; Jones et al., 2013; Loehr, 2013; Sanmiguel et al., motor and auditory learning contribute to auditory memory 2013) provide strong evidence in support of the existence of an for music. Pianists were asked to learn melodies on a Musical internal, motor-based prediction mechanism that can modulate Instrument Digital Interface (MIDI) piano keyboard in each of auditory perception. Planning or executing an action causes a four conditions (auditory only, motor only, strongly coupled copy of the motor command to be made (i.e., “efferent copy,” auditory-motor [i.e., normal performance], or weakly coupled or “corollary discharge”), which enables a prediction of the audi- auditory-motor [i.e., performing along with auditory recordings tory outcome of that motor command. A comparison between (acoustically similar or varying) without hearing their own feed- the prediction and the actual auditory input (“reafference input”) back]). After learning, participants heard melodies (half target, leads to a small prediction error, and subsequently to a min- half foils) in a subsequent recognition test and were instructed imal response in the auditory cortex reflecting an attenuated to indicate which melodies they had encountered in the learn- perception (Aliu et al., 2009). This mechanism enables to dis- ing conditions. It was found that motor learning (combined with criminate between auditory inputs that are a consequence of strongly coupled auditory learning) enhanced auditory recogni- our own actions and those that reach us from the external tion beyond auditory learning alone. Results were explained by world. It is important to consider that this mechanism requires the ability of sensory-motor associations formed during learn- (learned) internal models about the relationship between sensory ing to provide additional retrieval cues and to shape auditory and motor representations. Only recently, studies have started perception through mental simulation of action plans.

Frontiers in Psychology | Theoretical and Philosophical Psychology January 2014 | Volume 4 | Article 1008 | 6 Maes et al. Action effects on music perception

3.2.3. Disambiguation a training phase and a subsequent test phase. In the train- Music may have a certain degree of ambiguity in terms of percep- ing phase, infants were passively bounced (Phillips-Silver and tual and/or affective content. As discussed below, studies indicate Trainor, 2005), or adults bounced actively by bending their knees that it is possible for a listener to disambiguate this content by (Phillips-Silver and Trainor, 2007) on every second (duple) vs. planning or executing body movements during listening. Forward every third (triple) beat of an ambiguous musical rhythm pat- models provide an appropriate explanation for this disambigua- tern. In the subsequent test phase, infants’ listening preferences tion effect (Halász and Cunnington, 2012). The planning or were tested for two auditory versions of the rhythm pattern (duple execution of body movements enables one to automatically pre- and triple form) (Phillips-Silver and Trainor, 2005). In Phillips- dict the sensory consequences of these actions. Consequently, Silver and Trainor (2007), the adults were asked to listen to two these predicted sensory states can be projected onto the audi- auditory rhythms (duple and triple rhythm) and to select the one tory or musical material, which may guide (i.e., disambiguate) they thought matched what they heard during the training phase. the corresponding perception. Some additional remarks need to The results showed that the preferences and interpretations were be made, however. First, planning body movements does not oriented toward the auditory stimulus that matched the metrical only generate predictions of sensory states, but equally of subjec- form of their movement training. tive mental states related to affect and expressivity (e.g., valence, In a study by Naveda and Leman (2009)itwasshown arousal, etc.). In that sense, it is equally possible that subjec- that Samba music has a polymetric ambiguity, whereas Samba tive states are attributed to the music (Thompson et al., 2005; dance patterns typically have binary tendencies. Accordingly, the Juchniewicz, 2008; Sedlmeier et al., 2011; Maes and Leman, 2013). authors suggest that “perception of samba may be movement- Second, auditory or musical material doesn’t necessarily need to based in the sense that through self-movement (of the dancer be ambiguous in order for body movements to guide our per- in response to music) musical patterns get rhythmically ception in a specific direction. Music presents the performer and disambiguated.” listener with a flood of different auditory cues and accents. Body In a study by Sedlmeier et al. (2011), it was shown that real or movements can help selectively direct attention to certain cues, imagined body movements during music listening may codeter- and accordingly to impose a certain structure onto the music. mine music preferences. The experimenters activated or inhibited According to Urista (2003);Pierce(2007), body movements can specific muscles of the participants whose innervations have been help to isolate and explore musical elements as melody, beat, and shown to be associated with positive and negative emotions. This structural levels. Hence, cue selection (and, cue identification) was realized by instructing the participants to perform three spe- facilitated by body movement may refine music listening in gen- cific kinds of body movements or actions (activating/inhibiting eral, and shape our perception and understanding of the music. “smiling muscles,” vertical/horizontal head movements, and flex- Third, studies show that merely observing body movements, ion/extension of the arms). Activation of the positively associated instead of actually planning or executing them, may equally influ- muscle groups during listening to music led to higher preference ence perceptual and aesthetic judgments of the produced music ratings for that music than activation of the negatively associated (Thompson et al., 2005; Schutz and Lipscomb, 2007; Juchniewicz, ones. This suggests that body movements, both real and imag- 2008). Fourth, it is possible that executed or observed body move- ined, may play an important role in the development of music ments modulate auditory perception instantaneously, i.e., at the preferences. moment one listens to the music (Thompson et al., 2005; Schutz Su and Pöppel (2012) tested the hypothesis that the use of body and Lipscomb, 2007; Juchniewicz, 2008; Repp and Knoblich, movement is not merely a reaction to hearing rhythmic input, 2009; Sedlmeier et al., 2011). Additionally, it is also possible that but could actively assist the processing of temporal structures in when one repeatedly pairs body movements to music, the result- auditory events. They suggest that a self-initiated movement fre- ing action-based effects on music perception may endure for a quency, which is not tuned-in at first, could be attracted to one longer period of time, in the sense that the specific way of per- of the underlying periodicities of the presented sequence. Doing ceiving music may retain when merely listening to the music so guides the listener to start “hearing” the pulse at that level, without the need to intentionally plan or execute the correspond- forming a positive audio-motor feedback loop. The authors show ing body movements (Phillips-Silver and Trainor, 2005, 2007; that in the absence of overt movement, by contrast, this tun- Maes and Leman, 2013). So by sensory-motor associative learning ing process must then rely on the internal motor entrainment processes, music may become integrated with actions and more and/or the ability to analyze the sequence. Unlike musicians, non- importantly, with the sensory and affective states inherent to these musicians seemed to lack an effective internal motor simulation actions. It is a form of “evaluative conditioning” leading to effects that entrained to the pulse when it was not regularly present at of disambiguation and cue selection (Juslin and Västfjäll, 2008; the rhythmic surface, nor did they possess additional musical Maes and Leman, 2013). Moreover, depending on the nature of knowledge as a compensatory strategy. the learning process (e.g., duration, continuity, contingency, etc.), AstudybyIversen et al. (2009) investigated how the percep- these effects can be retained for different amounts of time. tion of a simple rhythmically ambiguous phrase (i.e., a repeating In the following section, we discuss several studies that illus- series of two tones followed by a rest) depends upon its intrin- trate these effects of disambiguation and cue selection. Phillips- sic metrical interpretation. Participants were asked to mentally Silver and Trainor (2005, 2007) addressed the interaction between place the downbeat on either the first or the second tone of body movement and the perception of musical rhythm. The pro- the rhythmical phrase. Using magnetoencephalography (MEG) it cedures of the experiments conducted in these studies contained was shown that different metrical interpretations evoked different

www.frontiersin.org January 2014 | Volume 4 | Article 1008 | 7 Maes et al. Action effects on music perception

neural responses, specifically in the upper beta range (20–30 Hz). 3.3. MOTOR DISORDERS This led the authors—given the suggested role of beta in motor The previously discussed action-based effects on auditory percep- processing—to the hypothesis that the motor system influences tion were rooted in learned auditory-motor associations. Apart metrical interpretation of sound, even in the absence of overt from that, another category of action-based effects can be dis- movement. In another study, Maes and Leman (2013) addressed tinguished. Several studies have shown that motor dysfunction the question of whether expressive body movements can condi- leads to considerable changes in individuals’ perception and tion children’s perception of musical expressiveness. They trained recognition of auditory and musical features. Pazzaglia et al. children with a happy or a sad choreography in response to music (2008) claimed a causative link between auditory recognition and that had an expressively ambiguous character. Afterwards, the action execution. Working with apraxia patients (limb apraxia, children’s perception of musical expressiveness in terms of valence buccofacial apraxia, or both), they showed that deficits in per- and arousal was assessed. The results suggested that the expres- forming gestures are causally linked to the patients’ inability sive qualities of the movements they learned to associate with the to recognize these gestures by their mere sounds. In the study, music had a significant impact on how children perceived musical apraxia patients were asked to listen to a sound and then choose expressiveness. from among four pictures the one corresponding to the heard In a study by Repp and Knoblich (2009), participants were sound. Limb and buccofacial apraxia patients were impaired in asked to play pairs of octave-ambiguous (Shepard) tones which recognizing sounds linked to respectively limb and buccofacial were a tritone apart. Although each tone of a pair is character- human actions. The authors advocated that lesions in frontal and ized by a specific pitch class (e.g., C - F#), they are ambiguous in parietal brain areas, which are actively associated with deficits pitch height. Participants were asked to play the pairs of tones by in execution tasks, were responsible for the observed gesture- pressing corresponding piano keys or computer keyboard keys, comprehension deficits. Also, studies indicate that the perception either in left-to-right or right-to-left direction. Consecutively, of musical features is impaired by motor dysfunctions. Beste they had to judge whether each pitch interval was rising or et al. (2011) demonstrated effects of movement deterioration on falling. Results showed that the participants gave significantly rhythm processing in Huntington’s disease patients. While listen- more “rising” responses when the order of keypresses was left- ing to music, patients exhibited weaker activations overall in brain to-right than when it was right-to-left. Moreover, this effect areas involved in the assessment of musical rhythms (cerebellar was larger for pianists compared to non-pianist musicians, most structures). Also, a study of Parkinson’s disease patients by Grahn likely because the specific pitch-to-sound mapping is stronger in and Brett (2009) found that basal ganglia dysfunction results in pianists (Experiment 1). Additionally, the same effect was found an impairment of the processing of rhythms that have a beat. when pianists merely observed another person pressing keys on a However, as the authors discuss, it cannot be excluded that patho- piano keyboard (Experiment 2). logical factors other than movement deterioration may contribute Other studies have shown that merely observing musician’s to impaired rhythm processing. For instance dopamine depletion, body movements can alter perceptual and aesthetic judgments of typical for Parkinson’s disease, has been shown to affect emotional the produced music. Schutz and Lipscomb (2007) examined to processing (Lotze et al., 2009), which may further modulate the what extent visual information of a marimba player’s gestures can processing of rhythms. In another study by Lucas et al. (2013), influence the perception of the duration of the produced tone. For impaired temporal information processing in Parkinson’s disease the experiment, video recordings were made of a marimba player patients has been ascribed to a deficit in the process of sensori- performing a series of tones using two stroke types (“long” gesture motor integration. These and other studies (see e.g., Grahn, 2012 and “short” gesture). The tones that were produced by both stroke for a review) demonstrate that rhythm perception involves a close types were acoustically indistinguishable. The visual and auditory link between auditory and motor processes. The existence of such components were separated from each other and fully crossed links has been exploited for motor rehabilitation purposes in the in order to create realistic musical stimuli. Then, participants domain of Parkinson’s disease, Huntington’s disease, and stroke. were asked to indicate perceived tone duration by means of a In this context, musical activities involving movement (control) 101-point slider. In an audio-only condition, no significant differ- and rhythm (perception) have been shown to improve general ences occurred between the ratings. However, in the audio-visual motor performance in Parkinson’s disease patients (Nombela condition, participants rated the tones produced with “long” et al., 2013a,b) and stroke patients (Altenmüller et al., 2009). gestures as significantly longer than the tones produced with It would be interesting to investigate further to what extent “short” gestures. In another study Thompson et al. (2005)showed improvements in motor skill benefit performance on perceptual that facial expressivity and expressive hand gestures of music tasks. performers (i.e., vocal and guitar performance) can influence lis- teners’ auditory perception of musical dissonance, melodic inter- 4. DISCUSSION val size, and affective valence. Similar findings are provided by Traditionally, body movements—whether performed by a music Juchniewicz (2008), showing that the type of physical movement performer or by a listener—were considered as the mere out- exhibited by a piano player while performing a musical excerpt put of internal cognitive processes that involved a system of (i.e., “no movement,” “head and facial movement,” and “full symbolic representations. Only recently, empirical evidence has body movement”) alters listeners’ judgments of the piano perfor- begun to appear indicating that the human motor system and its mance in terms of phrasing, dynamics, rubato and overall musical actions may actually modulate people’s experience, perception, performance. and understanding of sound and music. The present article

Frontiers in Psychology | Theoretical and Philosophical Psychology January 2014 | Volume 4 | Article 1008 | 8 Maes et al. Action effects on music perception

was intended to provide a theoretical framework in which cognition (Port and Van Gelder, 1995; Van Gelder, 1998; Beer, action-based effects on auditory perception may be understood. 2000; Chemero, 2009; McClelland et al., 2010; Shapiro, 2013). Additionally, the article serves as a review in which we investigate Music seems especially relevant as many musical activities— how the theory applies to recent empirical findings. The presented e.g., music production, dance, music listening—provide an eco- theoretical framework is centered around the common coding logical setting in which the intrinsic dynamics of action and theory (Prinz, 1990; Hommel et al., 2001). The basic assertion of perception can be studied (Bader, 2013a,b). Moreover, it is inter- this theory is that the planning or execution of an action recruits esting to note that people’s engagement with music involves the same sensory-motor brain areas as the mere perception of the not only sensory and motor components but also other com- sensory consequences of that action. We have argued that asso- ponents, such as “introspection”—referring to internal states ciative learning, in which actions and sensory states are repeatedly that include affect, motivation, intentions, metacognition, etc. experienced together, are of crucial importance in order for action (Barsalou, 2009)—and “social interaction.” Currently, research and perception to become integrated, and to form so-called inter- on internal models focuses almost exclusively on sensory and nal models. These internal models contain inverse and forward motor processes. However, to explain people’s interaction with components. Inverse models allow incoming sensory informa- music, and by extension with the world in general, it is neces- tion to activate the motor codes associated with the production sary to include aspects of introspection and social interaction into of that sensory state (cf. direct-matching hypothesis Rizzolatti theories on internal models. The integration of these aspects into et al., 2001). In contrast, forward models allow the sensory out- the present theoretical framework can deepen our understanding comes to be predicted from planned actions (Waszak et al., 2012). of music, and of the musical mind as fundamentally embod- The combination of inverse and forward models regulate goal- ied. In the following paragraphs, we briefly discuss these two directed motor control (Wolpert et al., 1995; Hommel, 1997), components. as well as the processing of sensory information coming from the external environment (Halász and Cunnington, 2012). We 4.1. MUSICAL EXPRESSIVITY explained that both inverse and forward models contribute to An important aspect of people’s engagement with music— action-based effects on auditory perception. Inverse models allow whether in listening to music or the actual production of music— that mere listening to music results in the activation of motor is musical expressivity. The musical elements that are said to con- codes, which is often manifested in overt movement responses stitute musical expressivity are manifold: dynamics, articulation, (cf. motor simulation, motor resonance, action mirroring, etc.). touch, phrasing, vibrato, etc. In the case of music production, These body movements are experienced and understood as inten- musical expressivity is often—but not exclusively—related to the tionally, expressively, and semantically meaningful, and cause the contents of the composition, and the main task of the musician is music to be experienced and understood accordingly. Forward to render the composition into sound. Of course there is always models have an impact on music perception in a different way. a certain degree of interpretation and expressivity from the per- They allow us to make predictions about the auditory outcomes former’s side. Music performance however does not necessarily of planned or executed actions, which guide and shape the per- rely on a pre-composed score, as in the case of improvisation or ception of sound and music. Predictions may either attenuate, jam sessions, where music may be created for the sake of explor- facilitate, or disambiguate the perception of sound and music. ing different sounds, rhythms, dynamics, etc. Apart from whether Together, these findings show that the human motor system and music is the result of playing a composition or improvisation, its actions have an impact on music perception and cognition. It what is conspicuous about many of the various elements con- is tempting to conclude based on this evidence that the “musi- tributing to musical expressivity is that they directly relate to their cal mind” is fundamentally embodied. However, according to physical origin, namely the body movements that produced the Wilson and Golonka (2013), the assertion that (music) cogni- music (Repp, 1993; Shove and Repp, 1995; Johnson, 1997; Godøy, tion is embodied has more radical and far-ranging implications. 2003; Leman, 2007; Cox, 2011). Accordingly, musical expressivity They claim that “embodiment is not simply another factor act- can be said to appeal to, at least to some extent, kinaesthetic sensa- ing on otherwise disembodied cognitive processes.” This would tions related to the effort and shape of body movements (Laban, retain the traditional Cartesian view that the brain is in con- 1947; Laban and Ullmann, 1966). Further, this kinaesthetic sen- trol and, in the case of people’s engagement in musical activities, sitivity may be associated with subjective phenomena like feel- literally “runs the show.” Instead, “radical embodiment” encom- ing, emotion, intentionality, etc. (Leman, 2007; Cochrane, 2010; passes a perspective on the body, the mind, and the environ- Sievers et al., 2013). In that sense, the human body has been con- ment as substantial elements of a dynamical system (Chemero, sidered as a mediator between sensory and motor processes and 2009). In essence, the term “dynamical system” points to a sys- mental states (Leman, 2007). A similar role has been attributed to tem of elements which are coupled, mutually interactive, and the body in the context of music listening. A listener is assumed evolve over time (Thelen and Smith, 1998). An important fea- to be able to decode—i.e., identify, imagine, or even physically ture of dynamical systems is the ability to self-organize. Order render—the elements of musical expressivity that relate to physi- and coherence appear out of the mutual interactions of the ele- cal motion and space, based on their own action repertoire and ments of the system without the use of explicit instructions, notion of space. This kinaesthetic sensitivity may be related to representations, or symbols. The dynamical system approach can subjective mental aspects of feeling, emotion, intentionality, etc. be applied to motor control and development (Turvey, 1990; In the same way as planning or executing an action enables people Kelso, 1995; Thelen and Smith, 1998; Warren, 2006), as well as to to make predictions of the sensory consequences of that action, it

www.frontiersin.org January 2014 | Volume 4 | Article 1008 | 9 Maes et al. Action effects on music perception

is possible to make predictions of the consequences on a men- (predict) the sensory consequences of one’s own and other’s tal level (e.g., feeling, emotion, intentionality). Accordingly, it is actions. reasonable to assume that the predictions of mental states modu- Our discussion of the components of “introspection” and late the perception of musical expressivity. It is only recently that “social interaction” indicates that musical activities involve a empirical support for this idea emerged (Sedlmeier et al., 2011; high-dimensional dynamical system in which the body, the mind, Maes and Leman, 2013). Also, it has been shown that the visual and the external environment are continually and mutually inter- observation of performers’ body movements influences people’s acting. In the case of musical instrument playing, music can perception of musical expressivity (Davidson, 1993; Thompson be considered as the result of a dynamical interaction between et al., 2005; Juchniewicz, 2008). These findings provide support the musicians’ motor and sensory system, the constraints and for including expressivity in theories of forward modeling applied opportunities of the pre-composed musical notation, the musi- to music perception and cognition. According to current theories cal instruments and the social environment, and the musicians’ of internal models, we have reason to believe that the relationship intentions, personality, mental states, etc. The system in which between mental states and action works in the opposite direc- these components interact is an open system, in the sense that tion as well (cf. inverse models). In that sense, a subjective state no individual component has causal priority in generating the coupled to music is assumed to modulate motor responses to music (Thelen and Smith, 1998). It is possible, however, that music. Support for this idea is given in a study of Van Dyck et al. the weight of the individual components on the produced sound (2013). According to the current view, internal models guide goal- varies depending on the specific musical activity (e.g., musical directed behavior as well as sensory processing. In that sense, improvisation, historical informed music performance, jam ses- internal models are the basic constituents of people’s interaction sion with an emphasis on social interaction, etc.). Similarly, music with the outside world. We advocate that this view should be listening can be considered as a dynamical process, in which the broadened by integrating other aspects of introspection (affect, experience, the perception, and the understanding of music is motivation, intentions, metacognition, etc.). Musical behaviors guided and shaped by the intrinsic dynamics of the body, the provide opportunities to study interactions between sensory, mind, and the external environment. In conclusion, adopting a motor and introspective processes, and the way these components fundamental embodied approach to music cognition requires us become associated with each other. The current view of embodied to consider music performance—involving motor coordination, music cognition considers introspection as a result of motor sim- control, and development—and music cognition as dynamical ulation processes (Leman, 2007). In other words, music induces processes. The integration of theories on internal models and body movements, which consequently trigger subjective aspects theories on dynamical systems can thereby enhance our under- of feeling, emotion, intentionality, etc. We advocate that the rela- standing of how our body, mind, and the external environment tionship between body and mind may be bidirectional, as aspects interact in our engagement with the act of music. of introspection may also influence motor responses to music. ACKNOWLEDGMENTS 4.2. SOCIAL INTERACTION This work was funded by a Natural Sciences and Engineering In daily life, much of what we do and experience happens in a Research Council of Canada (NSERC) grant 298173 to C. Palmer social context. A paramount example is people’s engagement with and by the Methusalem project on ‘Embodied music cognition music, as in music ensemble playing (Bastien and Hostager, 1988; and mediation technologies for cultural and creative applications’ Seddon, 2005), or when people dance together in a club or fes- to M. Leman. tival. These activities can be considered as forms of joint action involving coordinated actions, shared intention, shared atten- tion, shared representations, etc. (Keller, 2008; Goebl and Palmer, REFERENCES Aliu, S. O., Houde, J. F., and Nagarajan, S. S. (2009). Motor-induced sup- 2009; Loehr and Palmer, 2011; Obhi and Sebanz, 2011; Pacherie, pression of the auditory cortex. J. Cogn. Neurosci. 21, 791–802. doi: 2012; Phillips-Silver and Keller, 2012). Joint action in the con- 10.1162/jocn.2009.21055 text of music playing and dance has been shown to promote Altenmüller, E., Marco-Pallares, J., Münte, T., and Schneider, S. (2009). Neural social behavior (Kirschner and Tomasello, 2010) and to establish reorganization underlies improvement in stroke-induced motor dysfunc- a heightened sense of agency and a sense of we-ness (Pacherie, tion by music-supported therapy. Ann. N.Y. Acad. Sci. 1169, 395–405. doi: 10.1111/j.1749-6632.2009.04580.x 2012). Also, studies show that the social context may modulate Bader, R. (2013a). Nonlinearities and Synchronization in Musical Acoustics people’s experience and perception of music (Egermann et al., and Music Psychology, Vol. 2 of Current Research in Systematic Musicology. 2011; Liljeström et al., 2012). Currently, a major line of research is Heidelberg: Springer. devoted to the study of joint action in order to unveil the under- Bader, R. (2013b). “Synchronization and self-organization as basis of musi- lying mechanisms. Accumulating evidence suggests that these cal performance, sound production, and perception,” in Sound-perception- performance, Vol. 1 of Current Research in Systematic Musicology,edR.Bader mechanisms are similar to the ones involved in individual vol- (Heidelberg: Springer), 3–42. untary motor control and information processing. Accordingly, Baess, P., Horváth, J., Jacobsen, T., and Schröger, E. (2011). Selective suppression of internal models containing an inverse and forward component self-initiated sounds in an auditory stream: an ERP study. Psychophysiology 48, may explain how people manage to dynamically adapt to changes 1276–1283. doi: 10.1111/j.1469-8986.2011.01196.x Baess, P., Jacobsen, T., and Schröger, E. (2008). Suppression of the audi- in each other’s behavior. Inverse models are important for render- tory N1 event-related potential component with unpredictable self-initiated ing desired joint action sensory outcomes into particular action tones: evidence for internal forward models with dynamic stimulation. Int. J. plans. Supplementary forward models facilitate to anticipate Psychophysiol. 70, 137–143. doi: 10.1016/j.ijpsycho.2008.06.005

Frontiers in Psychology | Theoretical and Philosophical Psychology January 2014 | Volume 4 | Article 1008 | 10 Maes et al. Action effects on music perception

Bangert, M., and Altenmüller, E. (2003). Mapping perception to action in piano Dalla Bella, S., and Palmer, C. (2011). Rate effects on timing, key velocity, and practice: a longitudinal DC-EEG study. BMC Neurosci. 4:26. doi: 10.1186/1471- finger kinematics in piano performance. PLoS ONE 6:e20518. doi: 10.1371/jour- 2202-4-26 nal.pone.0020518 Barsalou, L. W. (2009). Simulation, situated conceptualization, and prediction. D’Ausilio, A., Altenmüller, E., Olivetti Belardinelli, M., and Lotze, M. (2006). Cross- Philos. Trans. R. Soc. B Biol. Sci. 364, 1281–1289. doi: 10.1098/rstb.2008.0319 modal plasticity of the motor cortex while listening to a rehearsed musical piece. Bastien, D. T., and Hostager, T. J. (1988). Jazz as a process of organizational Eur. J. Neurosci. 24, 955–958. doi: 10.1111/j.1460-9568.2006.04960.x innovation. Commun. Res. 15, 582–602. doi: 10.1177/009365088015005005 Davidson, J. W. (1993). Visual perception of performance manner in Baumann, S., Koeneke, S., Schmidt, C. F., Meyer, M., Lutz, K., and Jancke, L. (2007). the movements of solo musicians. Psychol. Music 21, 103–113. A network for audio–motor coordination in skilled pianists and non-musicians. doi: 10.1177/030573569302100201 Brain Res. 1161, 65–78. doi: 10.1016/j.brainres.2007.05.045 Davidson, P. R., and Wolpert, D. M. (2005). Widespread access to predictive Beer, R. D. (2000). Dynamical approaches to cognitive science. Trends Cogn. Sci. 4, models in the motor system: a short review. J. Neural Eng. 2, S313–S319. 91–99. doi: 10.1016/S1364-6613(99)01440-0 doi: 10.1088/1741-2560/2/3/S11 Bernardi, N. F., De Buglio, M., Trimarchi, P. D., Chielli, A., and Bricolo, E. (2013). Deiber, M.-P., Wise, S., Honda, M., Catalan, M., Grafman, J., and Hallett, Mental practice promotes motor anticipation: evidence from skilled music M. (1997). Frontal and parietal networks for conditional motor performance. Front. Hum. Neurosci. 7:451. doi: 10.3389/fnhum.2013.00451 learning: a positron emission tomography study. J. Neurophysiol. 78, Beste, C., Schüttke, A., Pfleiderer, B., and Saft, C. (2011). Music perception and 977–991. movement deterioration in Huntington’s disease. PLoS Curr. 3:RRN1252. doi: Doyon, J., Bellec, P., Amsel, R., Penhune, V., Monchi, O., Carrier, J., Lehéricy, 10.1371/currents.RRN1252 S., and Benali, H. (2009). Contributions of the basal ganglia and function- Blakemore, S.-J., Frith, C. D., and Wolpert, D. M. (2001). The cerebellum is involved ally related brain structures to motor learning. Behav. Brain Res. 199, 61–75. in predicting the sensory consequences of action. Neuroreport 12, 1879–1884. doi: 10.1016/j.bbr.2008.11.012 doi: 10.1097/00001756-200107030-00023 Ebner, T. J. (2013). “Cerebellum and internal models,” in Handbook of the Brown, R. M., and Palmer, C. (2012). Auditory-motor learning influences auditory Cerebellum and Cerebellar Disorders, eds M. Manto, J. D. Schmahmann, F. memory for music. Mem. Cogn. 40, 567–578. doi: 10.3758/s13421-011-0177-x Rossi, D. L. Gruol, and N. Koibuchi (Heidelberg: Springer), 1279–1295. Brown, R. M., and Palmer, C. (2013). Auditory and motor imagery mod- doi: 10.1007/978-94-007-1333-8_56 ulate learning in music performance. Front. Hum. Neurosci. 7:320. doi: Egermann, H., Sutherland, M. E., Grewe, O., Nagel, F., Kopiez, R., and Altenmüller, 10.3389/fnhum.2013.00320 E. (2011). Does music listening in a social context alter experience? A physiolog- Bubic, A., Von Cramon, D. Y., and Schubotz, R. I. (2010). Prediction, cognition and ical and psychological perspective on emotion. Musicae Scientiae 15, 307–323. the brain. Front. Hum. Neurosci. 4:25. doi: 10.3389/fnhum.2010.00025 doi: 10.1177/1029864911399497 Burger, B., Thompson, M. R., Luck, G., Saarikallio, S., and Toiviainen, P. (2013). Eitan, Z., and Granot, R. Y. (2006). How music moves: musical param- Influences of rhythm-and timbre-related musical features on characteristics eters and listeners images of motion. Music Percept. 23, 221–248. of music-induced movement. Front. Psychol. 4:183. doi: 10.3389/fpsyg.2013. doi: 10.1525/mp.2006.23.3.221 00183 Elsner, B., and Hommel, B. (2001). Effect anticipation and action control. J. Exp. Calvo-Merino, B., Glaser, D. E., Grèzes, J., Passingham, R. E., and Haggard, Psychol. Hum. Percept. Perform. 27, 229–240. doi: 10.1037/0096-1523.27.1.229 P. (2005). Action observation and acquired motor skills: an fMRI study Elsner, B., and Hommel, B. (2004). Contiguity and contingency in action- with expert dancers. Cereb. Cortex 15, 1243–1249. doi: 10.1093/cercor/ effect learning. Psychol. Res. 68, 138–154. doi: 10.1007/s00426-003- bhi007 0151-8 Caramiaux, B., Bevilacqua, F., and Schnell, N. (2010). “Towards a gesture-sound Fodor, J. A. (1975). The Language of Thought. Cambridge, MA: Harvard University cross-modal analysis,” in Gesture in embodied communication and human- Press. computer interaction (LNCS), Vol. 5934, eds S. Kopp and I. Wachsmuth (Berlin: Gallese, V., Fadiga, L., Fogassi, L., and Rizzolatti, G. (1996). Action recognition in Springer-Verlag), 158–170. the premotor cortex. Brain 119, 593–609. doi: 10.1093/brain/119.2.593 Catmur, C. (2012). Sensorimotor learning and the ontogeny of the mirror neuron Gaser, C., and Schlaug, G. (2003). Brain structures differ between musicians system. Neurosci. Lett. 540, 21–27. doi: 10.1016/j.neulet.2012.10.001 and non-musicians. J. Neurosci. 23, 9240–9245. doi: 10.1016/S1053-8119(01) Chemero, A. (2009). Radical Embodied Cognitive Science.Cambridge,MA:MIT 92488-7 press. Glenberg, A. M. (2010). Embodiment as a unifying perspective for psychology. Chen, J. L., Penhune, V. B., and Zatorre, R. J. (2008). Listening to musical Wiley Interdiscipl. Rev. Cogn. Sci. 1, 586–596. doi: 10.1002/wcs.55 rhythmsrecruitsmotorregionsofthebrain.Cereb. Cortex 18, 2844–2854. Godøy, R. I. (2003). Motor-mimetic music cognition. Leonardo 36, 317–319. doi: doi: 10.1093/cercor/bhn042 10.1162/002409403322258781 Chen, J. L., Penhune, V. B., and Zatorre, R. J. (2009). The role of auditory and Godøy, R. I. (2010). “Gestural affordances of musical sound,” in Musical Gestures: premotor cortex in sensorimotor transformations. Ann. N.Y. Acad. Sci. 1169, Sound, Movement, and Meaning, eds R. I. Godøy and M. Leman (New York, NY: 15–34. doi: 10.1111/j.1749-6632.2009.04556.x Routledge), 103–125. Cochrane, T. (2010). A simulation theory of musical expressivity. Austr. J. Philos. Godøy, R. I., and Leman, M. (2010). Musical Gestures: Sound, Movement, and 88, 191–207. doi: 10.1080/00048400902941257 Meaning. New York, NY: Routledge. Cook, R., Press, C., Dickinson, A., and Heyes, C. (2010). Acquisition of automatic Goebl, W., and Palmer, C. (2009). Synchronization of timing and imitation is sensitive to sensorimotor contingency. J. Exp. Psychol. Hum. Percept. motion among performing musicians. Music Percept. 26, 427–438. doi: Perform. 36, 840–852. doi: 10.1037/a0019256 10.1525/mp.2009.26.5.427 Cooper, R. P., Cook, R., Dickinson, A., and Heyes, C. M. (2012). Associative (not Grahn, J. A. (2012). Neural mechanisms of rhythm perception: current find- Hebbian) learning and the mirror neuron system. Neurosci. Lett. 540, 28–36. ings and future perspectives. Top. Cogn. Sci. 4, 585–606. doi: 10.1111/j.1756- doi: 10.1016/j.neulet.2012.10.002 8765.2012.01213.x Cox, A. (2011). Embodying music: principles of the mimetic hypothesis. Music Grahn, J. A., and Brett, M. (2009). Impairment of beat-based rhythm discrimina- Theory Online 17, 1–24. tion in Parkinson’s disease. Cortex 45, 54–61. doi: 10.1016/j.cortex.2008.01.005 Cox, R. F., and Hasselman, F. (2013). The case of Watson vs. James: effect- Grahn, J. A., and Rowe, J. B. (2013). Finding and feeling the musical beat: striatal priming studies do not support ideomotor theory. PLoS ONE 8:e54094. dissociations between detection and prediction of regularity. Cereb. Cortex 23, doi: 10.1371/journal.pone.0054094 913–921. doi: 10.1093/cercor/bhs083 Croom, A. M. (2011). Music, neuroscience, and the psychology of well-being: a Halász, V.,and Cunnington, R. (2012). Unconscious effects of action on perception. précis. Front. Psychol. 2:393. doi: 10.3389/fpsyg.2011.00393 Brain Sci. 2, 130–146. doi: 10.3390/brainsci2020130 Csikszentmihalyi, M. (1988). “The flow experience and its significance for Haslinger, B., Erhard, P., Altenmüller, E., Schroeder, U., Boecker, H., and Ceballos- human psychology,” in Optimal Experience: Psychological Studies of Flow in Baumann, A. O. (2005). Transmodal sensorimotor networks during action Consciousness, eds M. Csikszentmihalyi and S. Csikszentmihalyi (Cambridge, observation in professional pianists. J. Cogn. Neurosci. 17, 282–293. doi: UK: Cambridge University Press), 15–35. 10.1162/0898929053124893

www.frontiersin.org January 2014 | Volume 4 | Article 1008 | 11 Maes et al. Action effects on music perception

Haueisen, J., and Knösche, T. R. (2001). Involuntary motor activity in eds F. Morganti, A. Carassa, and G. Riva (Amsterdam: IOS press), pianists evoked by Music Perception. J. Cogn. Neurosci. 13, 786–792. doi: 205–221. 10.1162/08989290152541449 Kelso, J. A. (1995). Dynamic Patterns: The Self Organization of Brain and Behaviour. Heinks-Maldonado, T. H., Nagarajan, S. S., and Houde, J. F. (2006). Cambridge, MA: MIT Press. Magnetoencephalographic evidence for a precise forward model in speech pro- Kirschner, S., and Tomasello, M. (2010). Joint music making promotes proso- duction. Neuroreport 17, 1375–1379. doi: 10.1097/01.wnr.0000233102.43526.e9 cial behavior in 4-year-old children. Evol. Hum. Behav. 31, 354–364. Herholz, S. C., and Zatorre, R. J. (2012). Musical training as a framework doi: 10.1016/j.evolhumbehav.2010.04.004 for brain plasticity: behavior, function, and structure. Neuron 76, 486–502. Knolle, F., Schröger, E., Baess, P., and Kotz, S. A. (2012a). The cerebellum gener- doi: 10.1016/j.neuron.2012.10.011 ates motor-to-auditory predictions: ERP lesion evidence. J. Cogn. Neurosci. 24, Heyes, C. M. (2010). Where do mirror neurons come from? Neurosci. Biobehav. Rev 698–706. doi: 10.1162/jocn_a_00167 34, 575–583. doi: 10.1016/j.neubiorev.2009.11.007 Knolle, F., Schröger, E., and Kotz, S. A. (2012b). Prediction errors in Heyes, C. M., and Ray, E. D. (2000). What is the significance of imitation in animals? self-and externally-generated deviants. Biol. Psychol. 92, 410–416. Adv. Study Behav. 29, 215–245. doi: 10.1016/S0065-3454(08)60106-0 doi: 10.1016/j.biopsycho.2012.11.017 Hickok, G. (2009). Eight problems for the mirror neuron theory of action Kohler, E., Keysers, C., Umilta, M. A., Fogassi, L., Gallese, V., and Rizzolatti, G. understanding in monkeys and humans. J. Cogn. Neurosci. 21, 1229–1243. (2002). Hearing sounds, understanding actions: action representation in mirror doi: 10.1162/jocn.2009.21189 neurons. Science 297, 846–848. doi: 10.1126/science.1070311 Hommel, B. (1997). Toward an action-concept model of stimulus-response com- Kozak, M., Nymoen, K., and Godøy, R. I. (2012). “Effects of spectral features of patibility. Adv. Psychol. 118, 281–320. doi: 10.1016/S0166-4115(97)80041-6 sound on gesture type and timing,” in Gesture and Sign Language in Human- Hommel, B. (2003). “Acquisition and control of voluntary action,” in Voluntary Computer Interaction and Embodied Communication (LNCS), Vol. 7206, eds E. Action: Brains, Minds, and Sociality, eds S. Maasen, W. Prinz, and G. Roth Efthimiou, G. Kouroupetroglou, and S.-E. Fotinea (Berlin: Springer-Verlag), (Oxford, UK: Oxford University Press), 34–48. 69–80. doi: 10.1007/978-3-642-34182-3_7 Hommel, B., Müsseler, J., Aschersleben, G., and Prinz, W. (2001). The theory of Krueger, J. (2009). Enacting musical experience. J. Conscious. Stud. 16, 2–3. event coding (TEC): a framework for perception and action planning. Behav. doi: 10.1093/acprof:oso/9780199654888.003.0014 Brain Sci. 24, 849–878. doi: 10.1017/S0140525X01000103 Küssner, M. B. (2013). Music and shape. Lit. Linguist. Comput. 28, 472–479. Houde, J. F., Nagarajan, S. S., Sekihara, K., and Merzenich, M. M. (2002). doi: 10.1093/llc/fqs071 Modulation of the auditory cortex during speech: an MEG study. J. Cogn. Laban, R. (1947). Effort. London: MacDonald & Evans. Neurosci. 14, 1125–1138. doi: 10.1162/089892902760807140 Laban, R., and Ullmann, L. (1966). Choreutics. Hampshire: Dance Books Ltd. Hughes, G., Desantis, A., and Waszak, F. (2013a). Attenuation of auditory N1 (Edition published in 2011). results from identity-specific action-effect prediction. Eur. J. Neurosci. 37, Lahav, A., Boulanger, A., Schlaug, G., and Saltzman, E. (2005). The power of lis- 1152–1158. doi: 10.1111/ejn.12120 tening: auditory-motor interactions in musical training. Ann. N.Y. Acad. Sci. Hughes, G., Desantis, A., and Waszak, F. (2013b). Mechanisms of intentional 1060, 189–194. doi: 10.1196/annals.1360.042 binding and sensory attenuation: the role of temporal prediction, temporal Lahav, A., Saltzman, E., and Schlaug, G. (2007). Action representation of sound: control, identity prediction, and motor prediction. Psychol. Bull. 139, 133–151. audiomotor recognition network while listening to newly acquired actions. J. doi: 10.1037/a0028566 Neurosci. 27, 308–314. doi: 10.1523/JNEUROSCI.4822-06.2007 Hurley, S. (2001). Perception and action: alternative views. Synthese 129, 3–40. Lalazar, H., and Vaadia, E. (2008). Neural basis of sensorimotor learn- doi: 10.1023/A:1012643006930 ing: modifying internal models. Curr. Opin. Neurobiol. 18, 573–581. Hurley, S. (2008). The shared circuits model (SCM): how control, mirroring, and doi: 10.1016/j.conb.2008.11.003 simulation can enable imitation, deliberation, and mindreading. Behav. Brain Lappe, C., Steinsträter, O., and Pantev, C. (2013). Rhythmic and melodic deviations Sci. 31, 1–21. doi: 10.1017/S0140525X07003123 in musical sequences recruit different cortical areas for mismatch detection. Hyde, K. L., Lerch, J., Norton, A., Forgeard, M., Winner, E., Evans, A. C., Front. Hum. Neurosci. 7:260. doi: 10.3389/fnhum.2013.00260 and Schlaug, G. (2009). The effects of musical training on structural Laske, O. E. (1974). The information-processing approach to musical cognition. J. brain development. Ann. N.Y. Acad. Sci. 1169, 182–186. doi: 10.1111/j.1749- New Music Res. 3, 109–136. 6632.2009.04852.x Leaver, A. M., Van Lare, J., Zielinski, B., Halpern, A. R., and Rauschecker, J. P. Imamizu, H., and Kawato, M. (2009). Brain mechanisms for predictive control by (2009). Brain activation during anticipation of sound sequences. J. Neurosci. switching internal models: implications for higher-order cognitive functions. 29, 2477–2485. doi: 10.1523/JNEUROSCI.4921-08.2009 Psychol. Res. 73, 527–544. doi: 10.1007/s00426-009-0235-1 Leman, M. (2007). Embodied music Cognition and Mediation Technology. Iordanescu, L., Grabowecky, M., and Suzuki, S. (2013). Action enhances audi- Cambridge, MA: MIT Press. tory but not visual temporal sensitivity. Psychon. Bull. Rev. 20, 108–114. Leman, M., Desmet, F., Styns, F., Van Noorden, L., and Moelants, D. (2009). Sharing doi: 10.3758/s13423-012-0330-y musical expression through embodied listening: a case study based on Chinese Iversen, J. R., Repp, B. H., and Patel, A. D. (2009). Top-down control of rhythm per- guqin music. Music Percept. 26, 263–278. doi: 10.1525/mp.2009.26.3.263 ception modulates early auditory responses. Ann. N.Y. Acad. Sci. 1169, 58–73. Leman, M., Moelants, D., Varewyck, M., Styns, F., van Noorden, L., and Martens, J.- doi: 10.1111/j.1749-6632.2009.04579.x P. (2013). Activating and relaxing music entrains the speed of beat synchronized James, W. (1890). The Principles of Psychology.Vol.1.NewYork,NY:HenryHolt. walking. PLoS ONE 8:e67932. doi: 10.1371/journal.pone.0067932 doi: 10.1037/11059-000 Levinson, J. (2006). Contemplating Art: Essays in Aesthetics. New York, NY: Oxford Janata, P., Tomic, S. T., and Haberman, J. M. (2012). Sensorimotor coupling in University Press. doi: 10.1093/acprof:oso/9780199206179.001.0001 music and the psychology of the groove. J. Exp. Psychol. Gen. 141, 54–75. Lidji, P., Kolinsky, R., Lochy, A., and Morais, J. (2007). Spatial associations for musi- doi: 10.1037/a0024208 cal stimuli: a piano in the head? J. Exp. Psychol. Hum. Percept. Perform. 33, Johnson, M. L. (1997). Embodied musical meaning. Theory Pract. 22, 95–102. 1189–1207. doi: 10.1037/0096-1523.33.5.1189 Jones, A., Hughes, G., and Waszak, F. (2013). The interaction between atten- Liljeström, S., Juslin, P. N., and Västfjäll, D. (2012). Experimental evidence of tion and motor prediction. An ERP study. Neuroimage 83, 533–541. doi: the roles of music choice, social context, and listener personality in emo- 10.1016/j.neuroimage.2013.07.004 tional reactions to music. Psychol. Music 41, 577–597. doi: 10.1177/0305735612 Juchniewicz, J. (2008). The influence of physical movement on the 440615 perception of musical performance. Psychol. Music 36, 417–427. Loehr, J. D. (2013). Sensory attenuation for jointly produced action effects. Front. doi: 10.1177/0305735607086046 Psychol. 4:172. doi: 10.3389/fpsyg.2013.00172 Juslin, P. N., and Västfjäll, D. (2008). Emotional responses to music: the Loehr, J. D., and Palmer, C. (2011). Temporal coordination between need to consider underlying mechanisms. Behav. Brain Sci. 31, 559–621. performing musicians. Q. J. Exp. Psychol. 64, 2153–2167. doi: 10.1017/S0140525X08005293 doi: 10.1080/17470218.2011.603427 Keller, P. E. (2008). “Joint action in music performance,” in Enacting Lotze, M. (2013). Kinesthetic imagery of musical performance. Front. Hum. Intersubjectivity: a Cognitive and Social Perspective on the Study of Interactions, Neurosci. 7:280. doi: 10.3389/fnhum.2013.00280

Frontiers in Psychology | Theoretical and Philosophical Psychology January 2014 | Volume 4 | Article 1008 | 12 Maes et al. Action effects on music perception

Lotze, M., Reimold, M., Heymans, U., Laihinen, A., Patt, M., and Halsband, U. O’Reilly, J. X., Jbabdi, S., Rushworth, M. F., and Behrens, T. E. (2013). Brain sys- (2009). Reduced ventrolateral fmri response during observation of emotional tems for probabilistic and dynamic prediction: computational specificity and gestures related to the degree of dopaminergic impairment in parkinson disease. integration. PLoS Biol. 11:e1001662. doi: 10.1371/journal.pbio.1001662 J. Cogn. Neurosci. 21, 1321–1331. doi: 10.1162/jocn.2009.21087 Pacherie, E. (2012). “The phenomenology of joint action: self-agency vs. joint- Lotze, M., Scheler, G., Tan, H.-R., Braun, C., and Birbaumer, N. (2003). agency,” in Joint Attention: New Developments,edA.Seemann(Cambridge,MA: The musician’s brain: functional imaging of amateurs and profession- MIT Press), 343–389. als during performance and imagery. Neuroimage 20, 1817–1829. doi: Pascal-Leone, A. (2001). The brain that plays music and is changed by it. Ann. N.Y. 10.1016/j.neuroimage.2003.07.018 Acad. Sci. 930, 315–329. doi: 10.1111/j.1749-6632.2001.tb05741.x Lucas, M., Chaves, F., Teixeira, S., Carvalho, D., Peressutti, C., Bittencourt, J., Pasupathy, A., and Miller, E. K. (2005). Different time courses of learning-related et al. (2013). Time perception impairs sensory-motor integration in Parkinson’s activity in the prefrontal cortex and striatum. Nature 433, 873–876. doi: disease. Int. Arch. Med. 6, 1–7. doi: 10.1186/1755-7682-6-39 10.1038/nature03287 Luft, A. R., and Buitrago, M. M. (2005). Stages of motor skill learning. Mol. Pazzaglia, M., Pizzamiglio, L., Pes, E., and Aglioti, S. M. (2008). The sound of Neurobiol. 32, 205–216. doi: 10.1385/MN:32:3:205 actions in apraxia. Curr. Biol. 18, 1766–1772. doi: 10.1016/j.cub.2008.09.061 Maes, P.-J., Amelynck, D., and Leman, M. (2012). Dance-the-Music: an educational Phillips-Silver, J., and Keller, P. E. (2012). Searching for roots of entrainment and platform for the modeling, recognition and audiovisual monitoring of dance joint action in early musical interactions. Front. Hum. Neurosci. 6:26. doi: steps using spatiotemporal motion templates. EURASIP J. Adv. Signal Process. 10.3389/fnhum.2012.00026 2012, 1–16. doi: 10.1186/1687-6180-2012-35 Phillips-Silver, J., and Trainor, L. J. (2005). Feeling the beat: movement influ- Maes, P.-J., Amelynck, D., Lesaffre, M., Leman, M., and Arvind, D. (2013). The ences infant rhythm perception. Science 308, 1430. doi: 10.1126/science. “Conducting Master”: an interactive, real-time gesture monitoring system 1110922 based on spatiotemporal motion templates. Int. J. Hum. Comput. Interact. 29, Phillips-Silver, J., and Trainor, L. J. (2007). Hearing what the body feels: 471–487. doi: 10.1080/10447318.2012.720197 auditory encoding and rhythmic movement. Cognition 105, 533–546. Maes, P.-J., and Leman, M. (2013). The influence of body movements on chil- doi: 10.1016/j.cognition.2006.11.006 dren’s perception of music with an ambiguous expressive character. PLoS ONE Pierce, A. (2007). Deepening Musical Performance through Movement: the Theory 8:e54682. doi: 10.1371/journal.pone.0054682 and Practice of Embodied Interpretation. Bloomington, IN: Indiana University Maes, P.-J., Leman, M., Lesaffre, M., Demey, M., and Moelants, D. (2010). From Press. expressive gesture to sound: the development of an embodied mapping tra- Port, R. F., and Van Gelder, T., editors (1995). Mind as Motion: Explorations in the jectory inside a musical interface. J. Multimodal User Interfaces 3, 67–78. doi: Dynamics of Cognition.Cambridge,MA:MITPress. 10.1007/s12193-009-0027-3 Prinz, W. (1990). “A common coding approach to perception and action,” Maes, P.-J., Van Dyck, E., Lesaffre, M., Leman, M., and M, K. P. (2014). The cou- in Relationships Between Perception and Action: Current Approaches,edsO. pling of action and perception in musical meaning formation. Music Percept. Neumann and W. Prinz (Heidelberg: Springer), 167–201. doi: 10.1007/978-3- (forthcoming). 642-75348-0_7 Manning, F., and Schutz, M. (2013). “Moving to the beat” improves timing Prinz, W. (1997). Perception and action planning. Eur. J. Cogn. Psychol. 9, 129–154. perception. Psychon. Bull. Rev. 20, 1133–1139. doi: 10.3758/s13423-013-0439-7 doi: 10.1080/713752551 Massaro, D. W. (1990). “An information-processing analysis of perception and Pylyshyn, Z. W., and Demopoulos, W., editors (1986). Meaning and Cognitive action,” in Relationships between Perception and Action: Current Approaches, Structure: Issues in the Computational Theory of Mind.Norwood,NJ:ABLEX eds O. Neumann and W. Prinz (Berlin: Springer-Verlag), 133–166. doi: Publishing Company. 10.1007/978-3-642-75348-0_6 Repp, B. H. (1993). “Musical motion: some historical and contemporary perspec- Maus, F. (1988). Music as drama. Music Theory Spectr. 10, 56–73. doi: tives,” in Proceedings of the Stockholm Music Acoustics Conference (SMAC),edsA. 10.2307/745792 Friberg, J. Iwarsson, E. Jansson, and J. Sundberg (Stockholm: Kgl. Musikaliska McClelland, J. L., Botvinick, M. M., Noelle, D. C., Plaut, D. C., Rogers, T. T., Akademin), 128–135. Seidenberg, M. S., et al. (2010). Letting structure emerge: connectionist and Repp, B. H., and Knoblich, G. (2009). Performed or observed keyboard actions dynamical systems approaches to cognition. Trends Cogn. Sci. 14, 348–356. doi: affect pianists’ judgements of relative pitch. Q. J. Exp. Psychol. 62, 2156–2170. 10.1016/j.tics.2010.06.002 doi: 10.1080/17470210902745009 Mead, A. (1999). Bodily hearing: physiological metaphors and musical understand- Rizzolatti, G., Fogassi, L., Gallese, V., et al. (2001). Neurophysiological mechanisms ing. J. Music Theory 43, 1–19. doi: 10.2307/3090688 underlying the understanding and imitation of action. Nat. Rev. Neurosci. 2, Melcher, T., Winter, D., Hommel, B., Pfister, R., Dechent, P., and Gruber, 661–669. doi: 10.1038/35090060 O. (2012). The neural substrate of the ideomotor principle revisited: evi- Roussel, C., Hughes, G., and Waszak, F. (2013). A preactivation dence for asymmetries in action-effect learning. Neuroscience 231, 13–27. doi: account of sensory attenuation. Neuropsychologia 51, 922–929. doi: 10.1016/j.neuroscience.2012.11.035 10.1016/j.neuropsychologia.2013.02.005 Naveda, L., and Leman, M. (2009). A cross-modal heuristic for periodic pat- Rusconi, E., Kwan, B., Giordano, B. L., Umilta, C., and Butterworth, B. (2006). tern analysis of samba music and dance. J. New Music Res. 38, 255–283. doi: Spatial representation of pitch height: the SMARC effect. Cognition 99, 113–129. 10.1080/09298210903105432 doi: 10.1016/j.cognition.2005.01.004 Naveda, L., and Leman, M. (2010). The spatiotemporal representation of dance Sanmiguel, I., Todd, J., and Schröger, E. (2013). Sensory suppression effects to self- and music gestures using topological gesture analysis (TGA). Music Percept. 28, initiated sounds reflect the attenuation of the unspecific N1 component of the 93–111. doi: 10.1525/mp.2010.28.1.93 auditory ERP. Psychophysiology 14, 1–11. doi: 10.1111/psyp.12024 Neisser, U. (1967). Cognitive Psychology. New York, NY: Appleton-Century-Crofts. Sato, A. (2008). Action observation modulates auditory perception of the Nelson, A., Schneider, D. M., Takatoh, J., Sakurai, K., Wang, F., and Mooney, R. consequence of others’ actions. Conscious. Cogn. 17, 1219–1227. doi: (2013). A circuit for motor cortical modulation of auditory cortical activity. J. 10.1016/j.concog.2008.01.003 Neurosci. 33, 14342–14353. doi: 10.1523/JNEUROSCI.2275-13.2013 Schäfer, T., Fachner, J., and Smukalla, M. (2013). Changes in the representa- Nombela, C., Hughes, L. E., Owen, A. M., and Grahn, J. A. (2013a). Into the tion of space and time while listening to music. Front. Psychol. 4:508. doi: groove: can rhythm influence parkinson’s disease? Neurosci. Biobehav. Rev. 37, 10.3389/fpsyg.2013.00508 2564–2570. doi: 10.1016/j.neubiorev.2013.08.003 Schubotz, R. I. (2007). Prediction of external events with our motor sys- Nombela, C., Rae, C., Grahn, J., Barker, R., Owen, A., Rowe, J., et al. (2013b). How tem: towards a new framework. Trends Cogn. Sci. 11, 211–218. doi: often does music and rhythm improve patients’ perception of motor symptoms 10.1016/j.tics.2007.02.006 in Parkinson’s disease? J. Neurol. 260, 1404–1405. doi: 10.1007/s00415-013- Schutz, M., and Lipscomb, S. (2007). Hearing gestures, seeing music: vision 6860-z influences perceived tone duration. Perception 36, 888–897. doi: 10.1068/p5635 Obhi, S. S., and Sebanz, N. (2011). Moving together: toward understanding the Schütz-Bosbach, S., and Prinz, W. (2007). Perceptual resonance: action- mechanisms of joint action. Exp. Brain Res. 211, 329–336. doi: 10.1007/s00221- induced modulation of perception. Trends Cogn. Sci. 11, 349–355. doi: 011-2721-0 10.1016/j.tics.2007.06.005

www.frontiersin.org January 2014 | Volume 4 | Article 1008 | 13 Maes et al. Action effects on music perception

Seddon, F. A. (2005). Modes of communication during jazz improvisation. Br. J. Turvey, M. T. (1990). Coordination. Am. Psychol. 45, 938–953. doi: 10.1037/0003- Music Educ. 22, 47–61. doi: 10.1017/S0265051704005984 066X.45.8.938 Sedlmeier, P., Weigelt, O., and Walther, E. (2011). Music is in the muscle: Ungerleider, L. G., Doyon, J., and Karni, A. (2002). Imaging brain plastic- how embodied cognition may influence music preferences. Music Percept. 28, ity during motor skill learning. Neurobiol. Learn. Mem. 78, 553–564. doi: 297–306. doi: 10.1525/mp.2011.28.3.297 10.1006/nlme.2002.4091 Shapiro, L. (2010). Embodied Cognition. Boca Raton, FL: Taylor & Francis. Urista, D. J. (2003). Beyond words: the moving body as a tool for musical Shapiro, L. (2013). Dynamics and cognition. Minds Mach. 23, 353–375. doi: understanding. Music Theory Online 9. 10.1007/s11023-012-9290-2 Van Dyck, E., Maes, P.-J., Hargreaves, J., Lesaffre, M., and Leman, M. (2013). Shin, Y. K., Proctor, R. W., and Capaldi, E. (2010). A review of contemporary Expressing induced emotions through free dance movement. J. Nonverbal ideomotor theory. Psychol. Bull. 136, 943–974. doi: 10.1037/a0020541 Behav. 37, 175–190. doi: 10.1007/s10919-013-0153-1 Shove, P., and Repp, B. H. (1995). “Musical motion and performance: theoret- Van Gelder, T. (1998). The dynamical hypothesis in cognitive science. Behav. Brain ical and empirical perspectives,” in The Practice of Performance,edJ.Rink Sci. 21, 615–628. (Cambridge, UK: Cambridge University Press), 55–83. Varela, F. J., Thompson, E. T., and Rosch, E. (1991). The Embodied Mind: Cognitive Sievers, B., Polansky, L., Casey, M., and Wheatley, T. (2013). Music and movement Science and Human Experience.Cambridge,MA:MITPress. share a dynamic structure that supports universal expressions of emotion. Proc. Warren, W. H. (2006). The dynamics of perception and action. Psychol. Rev. 113, Natl. Acad. Sci. U.S.A. 110, 70–75. doi: 10.1073/pnas.1209023110 358–389. doi: 10.1037/0033-295X.113.2.358 Southgate, V. (2013). Do infants provide evidence that the mirror system Waszak, F., Cardoso-Leite, P., and Hughes, G. (2012). Action effect anticipation: is involved in action understanding? Conscious. Cogn. 22, 1114–1121. doi: neurophysiological basis and functional consequences. Neurosci. Biobehav. Rev. 10.1016/j.concog.2013.04.008 36, 943–959. doi: 10.1016/j.neubiorev.2011.11.004 Stewart, L., Verdonschot, R. G., Kajihara, T., and Sparks, J. (2013a). Weiss, C., and Schütz-Bosbach, S. (2012). Vicarious action preparation does not Action-perception coupling in violinists. Front. Hum. Neurosci. 7:349. result in sensory attenuation of auditory action effects. Conscious. Cogn. 21, doi: 10.3389/fnhum.2013.00349 1654–1661. doi: 10.1016/j.concog.2012.08.010 Stewart, L., Verdonschot, R. G., Nasralla, P., and Lanipekun, J. (2013b). Action– Williams, Z. M., and Eskandar, E. N. (2006). Selective enhancement of associative perception coupling in pianists: learned mappings or spatial musical associ- learning by microstimulation of the anterior caudate. Nat. Neurosci. 9, 562–568. ation of response codes (SMARC) effect? Q. J. Exp. Psychol. 66, 37–50. doi: doi: 10.1038/nn1662 10.1080/17470218.2012.687385 Wilson, A. D., and Golonka, S. (2013). Embodied cognition is not what you think Stupacher, J., Hove, M. J., Novembre, G., Schütz-Bosbach, S., and Keller, it is. Front. Psychol. 4:58. doi: 10.3389/fpsyg.2013.00058 P. E. (2013). Musical groove modulates motor cortex excitability: a Witt, J. K. (2011). Action’s effect on perception. Curr.Dir.Psychol.Sci. 20, 201–206. TMS investigation. Brain Cogn. 82, 127–136. doi: 10.1016/j.bandc.2013. doi: 10.1177/0963721411408770 03.003 Wolpert, D. M., Ghahramani, Z., and Jordan, M. I. (1995). An internal model Su, Y.-H., and Pöppel, E. (2012). Body movement enhances the extraction of for sensorimotor integration. Science 269, 1880–1882. doi: 10.1126/science. temporal structures in auditory sequences. Psychol. Res. 76, 373–382. doi: 7569931 10.1007/s00426-011-0346-3 Wolpert, D. M., Miall, R. C., and Kawato, M. (1998). Internal models in the Sudnow, D. (1978). Ways of the Hand.Cambridge,MA:MITPress. cerebellum. Trends Cogn. Sci. 2, 338–347. doi: 10.1162/jocn.2009.21055 Thelen, E., and Smith, L. B. (1998). “Dynamic systems theories,” in Handbook of Child Psychology, Theoretical Models of Human Development, 6th Edn., eds W. Damon and R. M. Lerner (Chichester: Wiley Online Library), Conflict of Interest Statement: The authors declare that the research was con- 258–312. ducted in the absence of any commercial or financial relationships that could be Thompson, W. F., Graham, P., and Russo, F. A. (2005). Seeing music performance: construed as a potential conflict of interest. visual influences on perception and experience. Semiotica 156, 203–227. doi: 10.1515/semi.2005.2005.156.203 Timm, J., SanMiguel, I., Saupe, K., and Schröger, E. (2013). The N1-suppression Received: 07 October 2013; paper pending published: 31 October 2013; accepted: 17 effect for self-initiated sounds is independent of attention. BMC Neurosci. 14:2. December 2013; published online: 03 January 2014. doi: 10.1186/1471-2202-14-2 Citation: Maes P-J, Leman M, Palmer C and Wanderley MM (2014) Action-based Timmann, D., Drepper, J., Frings, M., Maschke, M., Richter, S., Gerwig, M., effects on music perception. Front. Psychol. 4:1008. doi: 10.3389/fpsyg.2013.01008 and Kolb, F. (2010). The human cerebellum contributes to motor, emo- This article was submitted to Theoretical and Philosophical Psychology, a section of the tional and cognitive associative learning: a review. Cortex 46, 845–857. doi: journal Frontiers in Psychology. 10.1016/j.cortex.2009.06.009 Copyright © 2014 Maes, Leman, Palmer and Wanderley. This is an open-access Toiviainen, P., Luck, G., and Thompson, M. R. (2010). Embodied meter: hierar- article distributed under the terms of the Creative Commons Attribution License chical eigenmodes in music-induced movement. Music Percept. 28, 59–70. doi: (CC BY). The use, distribution or reproduction in other forums is permitted, pro- 10.1525/mp.2010.28.1.59 vided the original author(s) or licensor are credited and that the original publi- Trimarchi, P. D., and Luzzatti, C. (2011). Implicit chord processing and motor cation in this journal is cited, in accordance with accepted academic practice. No representation in pianists. Psychol. Res. 75, 122–128. doi: 10.1007/s00426-010- use, distribution or reproduction is permitted which does not comply with these 0292-5 terms.

Frontiers in Psychology | Theoretical and Philosophical Psychology January 2014 | Volume 4 | Article 1008 | 14