487 26.Musical Musical Syntax II: Empirical Synt Perspectives a Marcus Pearce, Martin Rohrmeier

26.1 Computational Research...... 487 Efforts to develop a formal characterization of mu- 26.1.1 Foundations ...... 487 sical structure are often framed in syntactic terms, 26.1.2 Early Approaches: Pattern Processing... 488 sometimes but not always with direct inspiration 26.1.3 Markov Modeling...... 489 from research on language. In Chap. 25,wepresent 26.1.4 Beyond Simple Markov Models: Hidden syntactic approaches to characterizing musical Markov Models structure and survey a range of theoretical issues and Dynamic Bayesian Networks ...... 490 involved in developing formal syntactic theories 26.1.5 Hierarchical Models ...... 491 of sequential structure in music. Such theories 26.1.6 Neural Networks...... 493 are often computational in nature, lending them- 26.2 Psychological Research...... 494 selves to implementation and our first goal here 26.2.1 Perception of Local Dependencies ...... 494 is to review empirical research on computational 26.2.2 Perception of Nonlocal Dependencies.. 495 modeling of musical structure from a syntactic point of view. We ask about the motivations for 26.3 Neuroscientific Research...... 496 implementing a model and assess the range of 26.3.1 Introduction ...... 496 approaches that have been taken to date. It is im- 26.3.2 Neural Basis of Syntactic Processing portant to note that while a computational model in Music...... 496 may be capable of deriving an optimal structural 26.3.3 Syntax in Music and Language ...... 497 description of a piece of music, human cognitive 26.3.4 Grouping Structure ...... 497 processing may not achieve this optimal perfor- 26.4 Implications and Issues...... 498 mance, or may even process syntax in a different 26.4.1 Convergence Between Approaches ...... 498 way. Therefore we emphasize the difference be- 26.4.2 Syntax, Semantics and Emotion ...... 498 tween developing an optimal model of syntactic References...... 499 processing and developing a model that simu- lates human syntactic processing. Furthermore,

we argue that, while optimal models (e.g., op- musical syntax from the perspective of compu- 26.1 | C Part timal compression or prediction) can be useful tational modeling, experimental psychology and as a benchmark or yardstick for assessing human cognitive neuroscience. There exists a large num- performance, if we wish to understand human ber of computational models of musical syntax, but cognition then simulating human performance we limit ourselves to those that are explicitly cog- (including aspects that are nonoptimal or even nitively motivated, assessing them in the context erroneous) should be the priority. Following this of theoretical, psychological and neuroscientific principle, we survey research on processing of research.

26.1 Computational Research

26.1.1 Foundations of representing. Therefore, there is a direct link between the theoretical characterization of musical syntax (dis- Different approaches to building computer models of cussedinChap.25) and the implementation and testing musical structure can be characterized, and distin- of these theories as computational models of cogni- guished, in terms of how expressive they are in terms tion, discussed here. Implementing a theory of musical of the degree of structural complexity they are capable syntax and processing has several potential advantages,

© Springer-Verlag GmbH Germany 2018 R. Bader (Ed.), Springer Handbook of Systematic Musicology, https://doi.org/10.1007/978-3-662-55004-5_26 NEXT (when the subse- ], for example, be- 11 same [26. ] extended the model of 10 earch, we distinguish between [26. Sumner ]. These models attempt to define and 11 Feroe , 10 and Simon The following sections illustrate these points, using Deutsch ers form. Nonetheless,theoretical it bounds on isoptimal structural often model. Furthermore, complexity theoretical useful models using ofsical to an mu- structure identify can helpspace us that understand human theacquire learners hypothesis the are syntactic facedartificial structure with intelligence of res when athe they musical representation defining style. therithms In search for space traversing andlearning, the algo- we space. distinguish Similarly betweenand the in hypothesis learning machine space mechanismsthe for case traversing of the musicalto syntax, space. potential the In stylistic hypotheses grammars correspond generatingscriptions structural of de- music and wetraversing can the think of space the of learningthe possible parameters as distinguishing grammars, those grammars specifying along the way. different kinds of computational models thatproposed have for been understanding musical syntax. quent symbol is identical to the previous one)(the and subsequent symbol issymbol obtained in by the taking specified the alphabettimes). next a Sequences specified number of of symbolsmore compactly may as then a sequence be of these described operations. 26.1.2 Early Approaches: Pattern Processing We begin with aare review of of twofoundations historical early for symbolic approaches importance models that thatcame subsequently because be- influential. they Onethe laid assumption early that listeners the approach usecesses pattern was induction to pro- based develop predictionsmelodies on for [26. successive events in formal languages for describing theby patterns humans perceived in temporal sequencesuse (such them as to music) explain and howprediction. these patterns are applied for gin with ordered alphabetsof for possible representing values the for(e.g., range a note particular names, musicalner note dimension restrict their durations). attention to Simonharmony, the rhythm dimensions and of and melody, Sum- formtonic and notes, use triads, duration, alphabets stress and forThe formal dia- operations structure. they consider are Simon and Sumner in severaling ways, structures in as particular sequences defin- ofsequence elementary operators operators and suchsion as and alternation. prime, They apply retrograde,to their inver- pattern various language alphabets,pitches, corresponding such to as collections themelodic) of major minor scales, and major, minor (natural, and diminished harmonic tri- and ]. 4 , without 3 context-free ]. 3 – ]). 9 ] as, e.g., discussed in 8 average listener oned artificial intelligence, certain kinds of ]. Note that if we are interested in ] grammars for blues harmony). In 7 6 , 5 ’s [26. As mentioned above, we must also emphasize the It is important to distinguish between models whose (which must run, generatingand output parameters from supplied thetakes as data as input) little ensuressumptions as that are possible explicitly it stated for [26. 1 granted and any as- theory by comparison of its behavior with theexpected output from theoretical accounts orhuman directly behavior with given the same inputs [26. understanding all theenvironmental factors context, (e.g., degree musicalcould of influence training, attention the etc.) structural descriptions that that listen- distinction between findingscription of an a piece optimal of music structural andrepresentation modeling the de- cognitive oftener. that The piece formerlikely bounds in to the the be lattercognition mind but subject (e.g., listeners to of limitations are of constraintswhich a working would of memory prevent lis- perception them load), tural and reaching description. an Note optimalassume struc- also the that existence it of is an problematic to cognitive representations of music,tional issue there of the is extent to thequestion which are the addi- representations actually in learned orspecified) inherited by human (i. beings e., ([26. innately the poverty of the stimulus debate [26. the past, taskstions that have usually requiredrequire been context-free simpler representational hand relationships have representa- been codedable while to tasks benefit from that theHowever, methods flexibility have now of been machine developed in learning. compu- tational linguistics to learn representation [26. knowledge is provided tothe them by style a of human good, expert old-fashi (in GOFAI) and those that acquire knowledgecal about structure musi- by learning (eithervised) supervised from or experience of unsuper- music, given some predefined structural representation, thelearned parameters of (in which thea are tradition neural network, of aference. machine Markov The appropriate model learning), approach or depends be onof grammatical the the in- it goal modeling. However, inproaches are many complementary cases in the thating two successfully complex ap- learn- representations (e.g., context-freeextremely rules) challenging is but the alternative approachresult can in models thatmusical do domain not for generalize which farSteedman beyond they the were developed (e.g., following well-known examples in cognitive science: 1. Implementing a theory as a computer program 2. Experiments can be run to evaluate the implemented Music Psychology – Physiology

Part C

Part C | 26.1 488 Musical Syntax II: Empirical Perspectives 26.1 Computational Research 489 ads, and seventh chords. They argue that the pattern how unexpected the next note is, once it has arrived – language facilitates processing in four ways: given a local model [26.21]. Note that the entropy and information content of a musical event or sequence are 1. Reduced redundancy of representation not properties of the music per se, but properties of the 2. Distinct alphabets may be invoked at different levels music from the perspective of an underlying model. 3. Embedded sequence structures and their associated It is interesting that one of the first nonmilitary alphabets may be encoded as chunks applications of early computers was software to gen- 4. The chunking of structures allows for the represen- erate music incorporating grammatical representations tation of configurations that satisfy proximity and of musical styles corresponding to probabilistic Markov the differentiation of different members of the al- models [26.22]. Early work using these models tended phabet in terms of frequency. to focus on fixed, low-order models with simple rep- resentational building blocks (e.g., chromatic pitch) to Deutsch and Feroe propose that multiple representa- estimate and compare the average entropy of differ- tions may be formed by listeners who, according to the ent musical works and corpora [26.16, 23–26]rather model, will tend to choose the most parsimonious. The than dynamic prediction of ongoing sequential musical acquisition of a representation is an ongoing process of structure. generation and testing of multiple hypothesized struc- More recent research addressed these limitations tural representations [26.12, 13]. by using variable-order Markov models, where the order varies depending on the context to generate ac- 26.1.3 Markov Modeling curate predictions dynamically throughout pieces of music [26.27, 28]. Prediction performance may also be Another early approach to modeling musical structure improved by combining predictions from a long-term was based on statistical learning and information the- model (trained on a large corpus of music in the style) ory, in particular using information content and entropy and a short-term model (which starts with an empty to measure the complexity of a musical style. It is in- model and learns incrementally from the current musi- teresting to note that information theory was applied to cal work) [26.27, 28]. The long-term model represents music as early as 1955 [26.14–17] just a few years af- the effects of long-term schematic exposure to a musi- ter Shannon’s foundational work was published [26.18]. cal style while the short-term model reflects more local Typically, this approach involves representing a musical learning of repeated structure within a musical work. work as a sequence of symbols drawn from an alphabet The probability distribution generated by the two mod- (e.g., a melody might be represented as a sequence of els may be combined using arithmetic or geometric pitch symbols, harmonic movement as a sequence of averaging, weighted by the entropy of the distribu- chord symbols). The learning task is to estimate a con- tions [26.29]. ditional probability governing the next symbol in the Improved prediction performance can also be musical sequence, given the preceding symbols. Such achieved by allowing the model to estimate and com- 26.1 | C Part models (in various guises) have been highly influential bine probabilities based on multiple features of the in terms of understanding predictive processing in hu- musical surface. Multiple viewpoint frameworks were man cognitive processing of music [26.19, 20]. originally developed by Darrell Conklin [26.27, 30, 31] It is possible to vary the length of the context to allow the integration of information from models of used in estimating the probability, known as the or- different features (inspired in part by Ebcio˘glu [26.32]). der of the model (a zeroth-order model has no context, The framework assumes a symbolic representation in a first-order model a context of one symbol and so which music is represented as a sequence of discrete on). Usually the probabilities (i. e., the parameters of events composed of a finite number of attributes each the model) are estimated through statistical analysis of which may assume a value from a finite alphabet. For of a corpus of musical works. These models are also example, a melody is often represented as a sequence of known as Markov models because the probability of notes, each of which is composed of a pitch, onset time, an event is only dependent on the immediately previ- duration and loudness. ous context (the Markov property), or, in other words, A viewpoint is a mapping from a sequence of musi- the model does not take into account any nonlocal de- cal events to an element from the alphabet associated pendencies. Once a probability distribution has been with the viewpoint. Basic viewpoints are projection generated in a particular context, the entropy of that functions associated with the attributes of the events distribution reflects the model’s uncertainty about the (i. e., pitch, onset, duration and loudness in the example following musical event before it arrives while the in- above). The framework also allows the specification of formation content (the negative log probability) reflects derived viewpoints (e.g., pitch interval, contour, scale ]. and 49 , 48 , 46], text Raczynski , 34 45 ], for an ex- Rohrmeier 53 25); in an analo- ] found that a sim- ]aswellas ]), which correspond to 34 51 43 ]; for an introduction see the [26. ]) generalize HMMs and con- 42 , 50 41 et al. [26. Graepel ] implemented a DBN modeling Jazz ], meter and rhythm [26. ] and harmonic structure [26. Markov models described above), but as 34 45 and ] to model polyphonic pitch structure. Fur- , 47 Paiement 52 44 [26. visible Most of the models of sequence processing dis- HMMs have been employed to model various as- One well-known example is hidden Markov mod- harmony using features ofprove predictive duration power and though modeextend the to to derived approach viewpoints. im- does not cussed so far drew theirof motivation from musical the expectation modeling (see also [26. setting [26. et al. [26. Modeling harmony inRohrmeier a corpus of jazz standards, ple HMM modeling chordany sequences overfitting exhibited of barely itsnetworks training (DBNs, data. [26. Dynamic Bayesian stitute a familydependency of structure graphical ofstructural models features. different that DBNs model (temporal)music by were the deep- applied to modeling transitions between deep structuralthemselves (hidden) emit states surface symbols that fromsion associated emis- distributions over thewords, terminal it alphabet. assumes In adeep other structure Markov of model states that as governtion the the over symbol subsections distribu- underlying of the sequence. pects of music, including,ture for instance, [26. melodic struc- 28]) and on(e.g., representations phrase of classes). However, although higher-order someformal structure of these limitations indressed in expressive part power withmore the may expressive multiple-viewpoint models be approach, been of ad- developed sequential in machine structurewhich learning have have research, been many applied of toerarchy, music. the In the different Chomskyto hi- model finite-context) classes assume (from an(represented underlying finite-state using deep nonterminal structure symbols)the that surface predicts terminal symbols (Chap. Graepel tensive account ofperception). the Models role with aever, of may rich be expectation used deep to in structure, understand other music how- aspects of music thermore, DBNs canmodeling be the straightforwardly adapted interactionstreams to of in different the parallel framework feature- of HMMs. probabilistic extensions of finite-stateextension automata. of Markov As models, an the hidden Markovassumes model the Markov transitioncharacterizing matrix transitions between not surface asin symbols a the (as model comprehensive review by [26. gous way, many modelingof approaches an take explicit advantage representation of deep structure in music. els (HMMs, e.g., [26. ] , 28 27 25). One im- ]. ] and also ac- 34 , 28 , 33 , 27 -gram family discussed 28 33]. Multiple-viewpoint n , rs’ pitch expectations in 28 , 27 ]. In some cases, the model pa- 40 35– , are essentially equivalent to probabilistic 28 Hidden Markov Models and Dynamic Bayesian Networks 26.1.3 When modeling music with a multiple viewpoint When configured appropriately and trained on rel- versions of strictly local grammars (Chap. portant feature of Markov modelsmodel is local that sequential dependencies they and can do not only any assume underlying deep structurethough (hidden they variables). operate Al- directlypossible on to use surface multiple symbols, viewpoint itsuch frameworks to models is allow to operatenotes on on nonsequential events tactus (e.g., beats or phrase-final events, [26. models have been developed for the domains ofharmony melody, and voice leading [26. rameters that optimize predictionimprove fit performance to human do perception and not vice versa [26. melody [26. count accurately for listene The models of the Markov and 26.1.4 Beyond Simple Markov Models: degree), which are derived from a basicpitch viewpoint in (e.g., this case).undefined Note that at some particular viewpointsfor may locations be the (e.g., first pitchviewpoints event that interval in return a aa melody). Boolean note falls value Test on (e.g., viewpoints apoints whether are tactus represent beat a or base not)at viewpoint and points (e.g., threaded where view- pitch a interval) terval test viewpoint between is notes trueallowing (e.g., for falling sequences pitch of on nonsequential in- events. Finally, tactuslinked beats) viewpoints thereby representtwo the or Cartesian more primitive product viewpoints –between of for pitch example, interval a and link scalephabet degree composed will of pairs, have whose an first al- elementinterval is and a whose pitch second is a scale degree. system, separate models are constructedpoint for included each view- inbutions the are system combined andlong-term in the and short-term much resulting model the distri- outputsThe are same combined. distributions way are astions first over the mapped the alphabet back associated withfrom into the which basic distribu- feature they areexamples) derived so (e.g., that pitchthe they in viewpoint the can models above separately be are for the combined. combined long- Typically, are in and then short-term a models, combined which first [26. stage in Sect. evant corpora, these methodsperformance both improve of prediction the models [26. suggesting that human expectationconstraints (such may as be memory or subject representationaltions) limita- to that prevent optimal prediction. Music Psychology – Physiology

Part C

Part C | 26.1 490 Musical Syntax II: Empirical Perspectives 26.1 Computational Research 491 processing such as inferring information encoded in the methods from generative linguistics and formal lan- deep structure of the sequence. For instance Raphael guage theory [26.5, 6, 60, 61]. Each of these formal and Stoddard [26.48] used a type of DBN for the infer- approaches have inspired efforts to build computational ence of harmonic structure from the surface sequence models. of events. Mavromatis [26.45] employed a model selec- tion procedure to find the optimal topology for a HMM, Schenkerian Analysis and Derivative Models using this to draw theoretical and cognitive conclusions Schenkerian theory constitutes one of the earliest and regarding representation of deep structure. He applied most comprehensive formal approaches to syntax in the model to two cases, the statistical learning and seg- tonal music and remains dominant in music-theoretical mentation paradigm used by Saffran and colleagues teaching. It has been the object of several computational (e.g., [26.54]) and metrical induction from rhythmi- approaches from the early days of computing to the cal patterns in a corpus of Palestrina’s vocal music. present day. Mavromatis [26.46] extended this approach and drew Early work by Kassler focused on understanding computationally informed conclusions regarding a dis- Schenkerian-like operations of analysis from the per- cussion surrounding Renaissance meter. spective of formal languages [26.62]. He formalized In summary, while the application of deep structure a subset of Schenkerian derivations with primitive- models and DBNs in cognitive music research is grow- recursive functions [26.63] and described a basic im- ing, there is a great potential to employ these models plementation of a (presumably two-voice) model of (in combination with model selection) to understand the Schenkerian analysis in terms of primitive operations role of deep structure in the representation and process- on a matrix representation of pitch sequence, such ing of musical syntax. as an Ursatz axiom, arpeggiation, neighbor note pro- longation, simplified interruption (termed articulation), 26.1.5 Hierarchical Models octave adjustment, bass ascent, mixture, etc. [26.62, 64, 65]. In a separate modeling attempt based on functional While computational implementations of Markovian programming, Smoliar and colleagues used a recursive approaches (and derived approaches such as HMMs and list structure of musical events that encoded a set of DBNs) have largely addressed the problem of modeling Schenkerian elaboration operations in terms of Lisp effects of expectancy and prediction, many computa- function calls. Drawing a direct formal analogy be- tional approaches have sought explicitly to implement tween (generative) linguistic parse trees and musical hierarchical generative models of music. The motiva- structure, they developed a tree-based structural repre- tion derives from theoretical insights that tonal music sentation of a Schenkerian reductive analysis [26.66– is organized in a hierarchical fashion and, accordingly, 68]. In an approach similar to Schenkerian analysis cognitive models of music processing should be able techniques, Baroni and colleagues applied formal gram- to account for such structural complexity. Moreover, mars to melodic structure [26.69, 70]. However, none theoretical hierarchical accounts of music stress that hu- of these early approaches resulted in a complete, fully 26.1 | C Part man music cognition involves substantially more than automatic functioning model of Schenkerian analysis. computation of sequential predictions including, in par- Marsden [26.71] suggested that this may be due to to ticular, the perception of large-scale processes (e.g., the massive explosion of combinatorial complexity that musical form), reductive listening, experience of hierar- arises when encoding a formal account of Schenkerian- chical tension and recognizing similarity (see Chap. 25 style reductive analysis at the level of notes. for a discussion of various ways to motivate and ex- With greater computational resources available, plore the understanding of musical syntax). Accord- a number of researchers have recently returned to ingly, computational models of hierarchical structure the problem of implementing Schenkerian analysis. have been inspired by a diverse array of modeling goals. Mavromatis and Brown [26.72] proposed a theoreti- The hierarchical and generative branches of music cal approach to modeling Schenkerian analysis with theory mainly trace back to Schenkerian theory [26.55, probabilistic context-free grammars. However, their 56]. Apart from Schenkerian theory itself, there have model did not reach the stage of a full implementa- been three major lines of hierarchical modeling re- tion due to issues of complexity (see [26.71]). Mars- search: the generative theory of tonal music (GTTM) den [26.73] proposed a graph-theoretical representation and tonal pitch space [26.57, 58], which originated of reductive structures (termed E-Graph) suitable for from the goal of framing Schenkerian analysis in computational implementation. Subsequently, Mars- terms of formal linguistic approaches; Narmour’s the- den [26.74] proposed an expanded representation of ory of melodic expectancy and complexity [26.13, Schenkerian reduction in a generative framework. Us- 59]; and approaches to tonal harmony that employ ing the limited case of short musical phrases, Marsden ] , ]). ex- 58 85 and 89 ]and , ]. The 6 [26. , 88 5 58 ]). that define 83 70, [26. , [26. 69 Lerdahl Granroth-Wilding Steedman and colleagues [26. ], well-formedness rules 90 Jackendoff , Ch. 1] for a review); these ]. To date, the most exten- ]. Following the formaliza- preference rules 58 84 87 ]. Based on Steedman’s categori- Hamanaka [26. 61 ] constitutes probably the most influ- , ] implemented a model of jazz har- 57 60 91 [26. [26. Riemann Generative Grammars for Music While the GTTM is specified to a much higher de- The Generative Theory of Tonal Music (GTTM) mony that extends Steedman’s earlier theoretical gram- Steedman cal grammar formalism [26. Rohrmeier More recent approaches include tion of context-free rewritesky hierarchy, grammars a and number thethese of Chom- earlier techniques approaches applied to music (e.g., [26. model has been founduous to ratings predict of participants’of contin- musical musical tension pieces for [26. a small number sive progress towards an implementation ofhas the been GTTM made by 86]. Many authors have proposed recursive generative gram- mars for modelingharmonic the sequences, hierarchicalback an organization to idea of whose essence goes haustively defining the (very large)candidate set of analyses well-formed and gree of detail andit specificity is than still Schenkerian theory, highlyof imprecise view. For from instance, a itthe computational does not preference point specify rules aman ranking and musical for intuition in for someThese making factors analytical pose cases challenges decisions. for assumes computationalmentations imple- hu- of the GTTM. Nonetheless, has devised a modelon of a musical complete tensiontension that GTTM based is on analysis based local and and predicts global factors musical [26. The GTTM [26. ential model in empirical musicologythe to date. perception It of models musicalteraction of structure four levels in of structure:metrical term grouping structure, of structure, time-span the reductiontional in- reduction and (see prolonga- [26. levels are defined in terms of rules to select thecandidates. best Although analysis it fromtive is the grammars partly well-formed inspired forthat by language, the genera- it GTTM is(tree-defining) is context-free important not rules are a to specified,unclear whether formal and note it could it grammar be is developed into becausemore, one. Further- no the GTTM is aof model a of piece the of final Westernthe tonal representation mind music of as it an might idealizedern appear music. listener, It in enculturated does not in account West- enculturation for any nor effects of make stylistic anynamic predictions nature about of the(this structure dy- was perception later addressed during by listening Eu- ]as ]. The ]), 13 59 ]. Further- ] use super- 37 , 78 36 ]) who has de- , 76 [26. 28 ] presents detailed (MOP) as represen- 59 is based on the implica- [26. Jensen (e.g., [26. ], who proposed the structure and ]. Research has also compared 75 81 ] and building on this theoretical , Kirlin [26. Narmour 74 80 developed a distinct theoretical ap- Kirlin ]. ]. Yust ]. 71 77 82 Furthermore, The Implication-Realization Model Recent research on implementing Schenkerian the- maximal outerplanar graphs more, Grachten and colleagues haveof examined the the implication-realization use model inmodel a computational of similaritytrieval for [26. use in music information re- proposals for the waystures in (pairs which of basicunits intervals) (chains) musical and struc- may larger hierarchicaltions) form units based (transforma- larger on the sequential ideaa structure of does closure, not generate which a occurs strongever, implication. when these How- hierarchical aspects ofsomewhat the neglected theory in have terms been ofing computational and model- empirical evaluation. veloped a corpusresponding Schenkerian of analyses 41 informat machine-readable pieces [26. annotated with cor- tations for the structureand of the Schenkerian work prolongation, of gene Narmour proach to modeling melodic expectancy [26. well as melodic complexity and reduction [26. vised probabilistic learningchical to musical uncover structure,deriving the deep including most hierar- an probableof analysis music. algorithm for for a given piece Breaking with Schenkerian tradition (see [26. 79 implication-realisation model tions that a melodichas for interval the (the following interval implicativepresents (the interval) a realized interval) detailed and classification ofon interval the pairs based size andThe model direction consists of of two thetems independent – perceptual component the sys- intervals. bottom-up and top-downimplication. systems of While melodic the principles ofto the former be are automatic, held unconsciousciples and of universal, the theculture latter prin- are dependent. held Althoughin to the a be model highly learned analytic isevance and manner, because presented hence it it hasperceptual proposes psychological hypotheses principles rel- about thattively general are specified precisely and andinvestigation therefore [26. quantita- amenable to empirical the implication-realization modelMarkov with models variable-order infor listeners’ terms pitch expectations of [26. how well they account of developed a preliminaryrian implementation analysis of [26. Schenke- framework, a first proof-of-conceptveloped [26. prototype was de- ory includes Music Psychology – Physiology Part C

492 Part C | 26.1 Musical Syntax II: Empirical Perspectives 26.1 Computational Research 493 mars for blues harmony [26.6]. Similarly, De Haas et al. simple units, distributed representations, synaptic con- developed an implementation of Rohrmeier’s gram- nectivity, graceful degradation and Hebbian learning). mar for the purposes of music information retrieval, Note, however, that while they are biologically inspired, such as harmonic similarity and transcription [26.92– most neural network models (especially so-called con- 94]. Tidhar [26.95] implemented adapted versions of nectionist models) are not actually biologically plau- a context-free grammar formalism for the parsing of sible models of neural processing. Thus they remain Couperin’s unmeasured preludes. at Marr’s [26.100] algorithmic/representational level Other approaches have attempted to combine rather than at the implementational/physical level of context-free grammars with probabilistic learning. description. At this level, one practical difficulty with Gilbert and Conklin [26.96], for example, have em- neural networks as simulations of cognitive processing ployed a probabilistic context-free grammar for mod- is that the nature of the learned representations are often eling melodic reduction. Bod [26.97] argues for not easily interpretable. a memory-based approach to modeling melodic group- Mozer [26.101], for instance, developed a model ing structure as an alternative to the Gestalt approach based on a recurrent artificial neural network (RANN, based on rules. He used grammar learning techniques to [26.102]) and used psychoacoustic constraints in the induce the annotated phrase structure of the Essen Folk representation of pitch and duration. In particular, the Song Collection [26.98, 99]. Three grammar induction networks operated over predefined, theoretically moti- algorithms were examined: first, the treebank grammar vated multidimensional spatial representations of pitch learning technique, which reads all possible context- (which emphasized a number of pitch relations includ- free rewrite rules from the training set and assigns each ing pitch height, pitch chroma and fifth relatedness, a probability proportional to its relative frequency in the [26.103]) and duration (emphasizing such relations as training set; second, the Markov grammar technique, relative duration and tuplet class). These neural net- which assigns probabilities to context-free rules by de- works are trained within a supervised regime in which composing the rule and its probability by an n-th-order the discrepancy between the activation of the output Markov process, allowing the model to estimate the units (the expected next event) and the desired acti- probability of rules that have not occurred in the train- vation (the actual next event) is used to adjust the ing set; and third, a Markov grammar augmented with network weights at each stage of training. When trained a data-oriented parsing (DOP) method for conditioning and tested on sets of simple artificial pitch sequences the probability of a rule over the rule occurring higher with a split-sample experimental paradigm, the RANN in the parse tree. A best-first parsing algorithm based model outperformed simple digram models. In particu- on Viterbi optimization was used to generate the most lar, the use of cognitively motivated multidimensional probable parse for each melody in the test set given each spatial representations led to significant benefits (over of the three models. The results demonstrated that the a local pitch representation) in the training of the net- treebank technique yielded moderately high precision works. However, the results were less than satisfactory but very low recall (F D 0:065), the Markov grammar when the model was trained on a set of melodic lines 26.1 | C Part yielded slightly lower precision but much higher recall from ten compositions by J.S. Bach and used to gen- (F D 0:706), while the Markov-DOP technique yielded erate new melodies; the neural network architecture the highest precision and recall (F D 0:810). A qualita- appeared unable to capture the higher-level structure in tive examination of the folk song data reveals a number these longer pieces of music. of cases (15% of the phrase boundaries in the test set) The question arises of whether these neural network where the annotated phrase boundary cannot be ac- models provide good simulations of human process- counted for by Gestalt principles but that are predicted ing. In an artificial grammar learning paradigm of by the Markov-DOP parser. melodic structure [26.40, 104], a simple recurrent net- work model [26.102] was compared with an n-gram 26.1.6 Neural Networks model [26.28] and a chunking model [26.105]. Results indicate that the n-gram model achieved by far the best Neural networks represent a different class of models performance, yet the simple recurrent network exhib- that have been used to understand the representation ited characteristic patterns of performance (including and processing of musical syntax. Rather than basing errors) that were closer to the human level [26.106, the model on a specific representation of musical struc- 107]. ture (e.g., n-grams, production rules, music-theoretic One approach to addressing the apparent inability principles), neural networks are biologically inspired of RANNs to represent recursive constituent structure in the sense that they take their motivation from basic in music involves what is called auto-association. An properties of the brain (e.g., parallel processing across auto-associative network is simply one that is trained – ]. 114 112 , ]. However, 112 [26. 26.1.3 25, harmony constitutes one ] and cognitive representations of 57 ], a point to whichPsychological we return research below. has established that encul- Recent developments in neural networks have led As discussed inof Chap. the coreand building it blocks hastheories in been of Western studied syntax. tonalexperimental Accordingly, in studies music a have theof focused large context harmonic on number structure of the of from perception different a variety of perspectives. 26.2.1 Perception of Local Dependencies 116 turated listeners are sensitivemusic; to sequential however, structure research in paradigmsbeen have driven sometimes bysumptions. (overly) We will simpleharmonic examine representational movement, melody the and as- evidence high-level form.topic relating One that to continues towhich listeners attract debate arecal, is capable nonsequential the of relationships in extent representing music.research to hierarchi- We on examine this question below. which are bothcommunication, sequential may share auditory similarities at formslevels more of abstract of cognitive human and neural representation [26. efficiently (and reproduced more accurately)events. than Furthermore, other the trained networkability showed to some generalizevariant beyond and the novel training melodiesformance examples although, was to affected in by the general,used depth per- of to the tree represent structure thegrees of input hierarchical melodies nesting with leadingduction to greater of impaired input de- repro- melodies. However,which the certainty the with trained networklated reconstructed well events with corre- music-theoretic predictionsimportance of structural [26. structural importance as assessed bythe empirical events data on retainedvised by variations on trained the melodies. pianists across impro- to successful modeling ofstricted musical Boltzmann structure using machines re- (RBMs) [26. 111 these neural networkand models it are remains to difficultin be to modeling seen cognitive analyze howNonetheless, successful processing they they do of at will least musical be demonstratecessing that syntax. can such pro- be implemented(and potentially using nonsymbolic) representations. parallel distributed RBMs appear to be approachingmance the of prediction perfor- themodels best-performing described variable-order in Markov Sect. ] ]. 110 108 ]. Fur- 113 et al. [26. Large 25). Although there are ] introduced an extension of auto- 109 ]. It was found that the trained models 57 [26. notable differences between language andin terms music of (e.g., lexical categories andcontent), the cognitive nature scientific of research semantic onan music analogous approach, follows exploring processing via a com- bination of theoretical inquiry, computational modeling and psychological/neuroscientific testing [26. 26.2 Psychological Research Computational models shedof light specific assumptions on about,nature the and of constraints plausibility the on, cognitive the underlying representations and music algorithms perceptiontools and to explore the serve types of asin syntactic structures analytical music. present However,tactic to structures understand that the arelisteners, kinds perceived it and of represented is syn- output by of important a to model withpirical compare the research responses empirically on of musical listeners. the the Em- listening power, the can limits help andtion constraints identify of and human cognition percep- ofcomputational musical models syntax. are Inpotheses useful and this selecting context, for stimuli thatin generating differ terms quantitatively hy- oficality, their uniformity, syntactic see properties Chap. (e.g., grammat- to reproduce onto its its output layer inputrepresentation of a layer, the pattern input generally onample, presented its forming training hidden layer. a a Foroutput layers ex- network compressed separated by a with three-unit hidden eight-unit layerthe with input eight and one-bit-in-eighta patterns three-bit typically binary results code in on the hidden units [26. Pollack thermore it has been suggested that music and language, association called recursive(RAAM), auto-associative which is memory capable ofresentations learning for fixed-width rep- compositional treerepeated structures compression. through Thesists of RAAM two separate networks: architecture first, anthat con- encoder constructs network a fixed-dimensional code byprocessing recursively the nodes oftom a up; symbolic and second, tree a from decoderdecompresses the network this that bot- code recursively intoit its terminates component parts in until from symbols, the thus top reconstructingtandem down. as the The a tree single two auto-associator. networks are trained in examined the ability of RAAMresentations of to Western acquire children’s melodies reduced represented rep- as tree structures accordingtions to [26. music-theoretic predic- acquired compressed representations of the melodieswhich in structurally salient events are represented more Music Psychology – Physiology Part C

494 Part C | 26.2 Musical Syntax II: Empirical Perspectives 26.2 Psychological Research 495

At a local level, listeners are able to detect harmonic regularities in the immediate context [26.133]. As with relations between successive chords as evidenced by harmonic expectations, computational models relying longer reaction times to in-key than out-of-key har- on nonhierarchical statistical learning of regularities monic transitions in the paradigm of musical priming have proved highly successful in predicting listeners’ studies ([26.117, 118], see [26.119] for a review). These melodic expectations [26.28, 37, 38]. findings have been replicated with longer sequences of All of these effects can be accounted for by lo- chords and more complex harmonic relations [26.120, cal models corresponding to strictly local grammars. 121]. Furthermore, there is evidence that these har- Therefore, it is crucial to ask which aspects of hierar- monic priming effects are affected both by the global chical and nonlocal structures affect music listening and context of the prevailing key (which determines the processing. overall stability of a chord), by the local harmonic context and by the rhythmic organization of the pro- 26.2.2 Perception of Nonlocal Dependencies gression [26.120]. Research has also examined whether these effects are best explained by learned properties of There are several ways in which nonlocal dependencies harmonic movement or by sensory properties of the tar- can be expressed in music, ranging from short chord get chords [26.121–123]. The results consistently show sequences (such as “I IV V/ii ii V I”, in which the that the structural properties of harmonic movement IV chord implies the V chord and not V/ii, and the have a stronger effect than sensory influences such as second I chord prolongs the initial one, rendering the repetition priming [26.124, 125], even when efforts are whole sequence a constituent of a prolonged tonic) to made to make these sensory influences very strong. Fur- dependencies between (sub)phrases (e.g., antecedent- thermore, Tillmann et al. [26.126] used self-organizing consequent patterns) and between larger formal units maps (a variety of neural network) to argue that these like the parts of a minuet or a sonata [26.143]. It may priming effects can be explained by learning of tonal even be possible to understand these dependencies at organization through musical exposure. different timescales (chords, parts of phrases, phrases, We can ask similar questions about the percep- movements) as recursive instantiations of similar struc- tion of structure in melody. Early work on predictive tures [26.57, 143]. processing of musical structure focused on rules of Deutsch [26.144] describes experiments in which melodic organization and how they influence pitch ex- subjects were presented with sequences of 12 notes, pectations [26.13, 80, 81]. Empirical studies of these which they recalled in musical notation. Half the se- rules have found that listeners’ melodic expectations quences were structured in accordance with the model do generally exhibit influences of pitch proximity of Deutsch and Feroe [26.10], described above, such (smaller intervals are more expected) and pitch re- that a higher-level subsequence of four elements acted versal (large pitch intervals are expected to be fol- on a lower-level subsequence of three or four elements lowed by smaller ones in the opposite registral direc- while the remaining sequences were unstructured. Se- tion) [26.53, 127]. Actual melodies also exhibit these quences were presented in one of three conditions: 26.2 | C Part properties [26.128], possibly reflecting physical con- first, with no temporal segmentation; second, tempo- straints of performance – the difficulty of produc- rally segmented in accordance with tonal structure; and ing large intervals accurately and tessitura constraints third, temporally segmented in violation of tonal struc- producing regression to the mean after large inter- ture. The results demonstrated that recall was high in vals [26.129–131]. Therefore, it remains possible that the first and second conditions and low in the third listeners learn these regularities (or variants of them, condition and for unstructured sequences, suggesting subject to cognitive constraints, [26.53, 132]), which that hierarchically structured sequences are better en- are then reflected in their pitch expectations. coded in memory. However, on a similar task, Boltz and In fact, there is empirical evidence of implicit Jones [26.145] have found that rule recursion has only learning of regularities in musical melody and other a modest effect on memory for melodies and only in sequences of pitched events ([26.40, 54, 104, 133]; certain conditions. see [26.8] for a review on implicit learning of music). Regarding large-scale form, researchers have stud- Consistent with an approach based on statistical learn- ied the aesthetic judgments of musicians and nonmu- ing, melodic pitch expectations vary between musical sicians for pieces of music in which the large-scale styles [26.134] and cultures [26.135–140], throughout tonal form has been disrupted by rearranging or rewrit- development [26.141] and across degrees of musical ing certain parts. These studies have been conducted training and familiarity [26.36, 37, 134]. Furthermore, by rearranging the various movements of Beethoven pitch expectations appear to be informed both by long- sonatas and string quartets [26.146], reordering the vari- term exposure to music [26.142] and by the encoding of ations making up Bach’s Goldberg variations [26.147], – – 157 167 ]. The EAN 161 , 157 ]. The amplitude of 163 ] suggesting that it reflects ]. above). Consistent with this 166 159 , 165 26.1.3 600 ms at central and posterior sites in 280 ms (sometimes right lateralized and ) to the final chord. In addition, behav- 26.3.2 -gram or Markov models could explicitly represent ]. Two characteristic brain responses have been ] identified a late positive component (LPC) peaking To date, less is known about the neural correlates n the EAN is related toity the of long-term the transition chord probabil- [26. is thought to reflect thetion, violation while of the harmonic N5 expecta- cessing is effort thought needed to to integrate reflect unexpected harmoniesinto the higher the pro- ongoing context [26. ioral measures showed that thereemotional were no response differences (valence in andversions, arousal) suggesting between that the nonlocal harmonic dependen- cies are independent ofbehavioral emotional measures showed expression. that Further theoriginal final version chord was of judged the toter than close the the final sequence chord of bet- the the difference altered was versions relatively (although small, pointinguse towards the of implicitdamentally, rather because the than harmonic explicit dependenciesstudy knowledge). in exceed ten this Fun- chords, itcal is impossible that nonveridi- 164 reported: an early anteriortency negativity of (EAN) 150 with areferred la- to as the ERANity), – and early a right anterior later(N5) negativ- bilateral with a or latency right-lateralized of negativity 500 ms [26. in a pairphrases, of together chorale with melodies modifiedfirst composed versions in of phrase which two was the the sub- final altered phrase. The to resultstic showed break differences strong tonal characteris- between theresponse closure two (the with versions early in rightSect. the anterior neural negativity (ERAN), responses to violations of harmonic syntax [26. 171 between 300 response to stylistically unexpected notes in a melody. of structural processing in otheras aspects pitch, of music rhythm such and timbre. Early studies [26. implicit learning ofperience harmonic (see movement Sect. through ex- proposal are findings that the EAN isstill attenuated (though present) in fiveto to adults six-year-old and accentuated children in compared adult adult nonmusicians musicians, [26. relative to the difference. Accordingly, this evidencedependencies for nonlocal affecting thesure perception falsifies of the phrase assumptionof clo- harmony that can simple adequately model local humanmusical perception models syntax. of Koelsch ]. Clearly, cally criti- 156 – 153 ]) and using rearranged ]; methodologi ]. The results consistently ] found that both versions 149 148 150 151 [26. ]. In a study of musicians listening 6 min) such that they start and end  150 , ], examined responses to the final chord 148 Gjerdingen 152 in Music However, there is crucial evidence that listeners were deemed equally conformingtions to and stylistic accuracy expecta- inending judging key whether were the the same starting was atsuggest and chance. that These there findings are severe constraintsbuild on cognitive the representations ability of to large-scale formallationships re- in music, even forof musicians these (though studies none examined professionala musicians very high with level of expertise). are sensitive to long-distance hierarchical dependencies at the phrase-levelet and al. midrange [26. timescales. to original andworks rearranged by versions Handel of [26. six keyboard Neuroscientific research has used electroencephalogra- phy (EEG) to investigate event-related-potential (ERP) 26.3.2 Neural Basis of Syntactic Processing 26.3.1 Introduction It is pertinent totell ask us what over cognitivechology and neuroscience and above can cognitive research science in [26. experimental psy- 26.3 Neuroscientific Research altering the endingsClassical of works (1 excerpts from Romantic and cized by suggest that listeners gain as much pleasuretered from versions the al- as theare unaltered versions no and more musicians affectedruptions than [26. nonmusicians by these dis- in a different key ([26. versions of the openingphony movement in of G Mozart’s Minor Sym- [26. it can tell uschological something processes; about less the obviously, but neuralimportantly, perhaps basis it more of can psy- alsopsychological tell processes themselves. us Once something asponse about neural those re- has beenprocess (and linked this to itselftablish), may a then require we specific great canof psychological effort use that to it process, es- asneural alongside responses an have behavioral the additional potential measures. advantages that measure Such may they be more sensitive and less pronebias to various than kinds of behavioral measures. Potentialinclude the disadvantages difficulty of establishinglationships between direct, features specific of re- the neuralparticular response psychological and processes. Music Psychology – Physiology

Part C

Part C | 26.3 496 Musical Syntax II: Empirical Perspectives 26.3 Neuroscientific Research 497

The amplitude and latency of the LPC are sensitive to musical and linguistic syntax. Violations of syntax in musical expertise, the familiarity of the melody, the de- language often generate two characteristic ERPs: a left gree of expectancy violation [26.167],andalsotothe anterior negativity (LAN) peaking around 300450 ms timing of the unexpected note [26.168]. at frontal scalp locations, and a positive-going deflec- There is an important distinction between schematic tion termed the P600 at a latency of about 600 ms with representations of the syntactic rules governing a mu- a posterior distribution. An early study suggested that sical style and veridical memory for the structure of violations of harmonic syntax generate an increased a familiar piece of music [26.172]. Miranda and Ull- P600, which is very similar to that induced by viola- man [26.173] describe a functional dissociation be- tions of linguistic syntax [26.162]. More recent research tween two ERP components: an early (150270 ms) has presented music synchronously with visually pre- anterior-central negativity (i. e., the EAN) associated sented sentences where each word coincides temporally with out-of-key violations in both familiar and unfamil- with a note or chord in the music. Introducing syn- iar melodies, and a subsequent (220380 ms) posterior tactic violations in the music and language allows the negativity elicited by both in-key and out-of-key vio- investigation of conditions where the violations are lations of familiar melodies only. They suggested that congruent or incongruent in the two domains. This re- these two components are driven by violations of musi- search suggests that the LAN to syntactic violations in cal rules (of tonality/harmony) and of veridical memory language is reduced by unusual harmonic movement representations of familiar melodies respectively. In or- (e.g., a Neapolitan chord, [26.175])andalsobylow- der to focus exclusively on schematic acquisition of probability notes in melodic phrases [26.176]. There is syntax, Loui et al. [26.166] examined neural responses also evidence that the ERAN is reduced when presented to deviant melodic endings in pitch sequences gener- concurrently with a linguistic violation [26.177]. Inter- ated according to an artificial (strictly local) grammar, estingly, these interactive effects are not in evidence using pitches taken from the unfamiliar Bohlen–Pierce when musical violations are paired with semantic in- scale. They report an EAN, whose amplitude increased congruities in language [26.176, 177]. with greater learning of the grammar (measured by Analogies have been drawn between the ERAN and degree of exposure and performance in a grammat- the early left anterior negativity (ELAN) often observed icality decision task). Again, this suggests that the in response to violations of syntax in language [26.178]. EAN reflects a process of implicit statistical learning Interestingly, children with specific language impair- of sequential dependencies in the auditory environment ment not only show characteristic changes in the ELAN (Sect. 26.1.3). to language [26.179] but also a reduced ERAN to The EAN to violations of melodic syntax tends harmonic violations [26.180]. Broca’s aphasics also to occur earlier (around 100 ms) than to violations of show reduced neural responses to harmonic violations harmonic syntax (circa 180 ms) [26.37, 38, 174], per- in music [26.181] and there is evidence from magne- haps indicating that single notes are processed more toencephalography (MEG) research that the ERAN to quickly than chords. Using a computational model of violations of harmonic syntax originates in Broca’s area 26.3 | C Part auditory expectation [26.28], which makes probabilis- and its right hemisphere homologue [26.182]. Further- tic pitch predictions based on statistical learning, to more, functional magnetic resonance imaging (fMRI) identify notes in melodies that varied systematically in studies [26.183–186] suggest that violations of har- information content (IC), Omigie et al. [26.39]showed monic syntax induce activation in the inferior frontal a linear relationship between the amplitude of the EAN cortex, which is also suggestive of a relationship be- and IC. Pearce et al. [26.37] reported an increase in tween the neural processing of syntax in music and the amplitude of beta oscillations for high information language [26.114]. content (low probability) notes at a latency of around 500 ms in centroparietal regions and in phase locking 26.3.4 Grouping Structure in the same time window between electrodes located over centroparietal and occipital regions. Again these One other aspect of musical structure that has been results are consistent with the proposal that these neural studied from the perspective of cognitive neuroscience indicators of syntactic processing reflect probabilistic is grouping structure. Using EEG and MEG, Knösche inference based on implicit statistical learning. et al. [26.187] found that phrase boundaries in melodies generated a late (500600 ms for EEG, 400700 ms for 26.3.3 Syntax in Music and Language MEG) positive deflection, which they termed the clo- sure positive shift (CPS). Source localization suggested EEG research has also addressed the question of that the CPS was generated by structures in the limbic whether there exists an overlap in neural processing of system, including anterior/posterior cingulate and pos- , 198 [26. ]. Building on Meyer had he not gone ], suggesting that 53 ], 191 ] for further discus- 199 rential and propositional 197 [26. , ]. In particular, the syntactic 196 , 198 113 Hanslick (see [26. ] examines from a theoretical perspective the dy- There is one prominent theoretical perspective on 200 namic cognitive processes in operationto when we music listen andthe how listener’s these understanding processesalso of not give musical only rise structure underlie toperception but the of meaning communication in ofmeaning music. affect arises Meyer and through proposes the the that structures manner activate, in inhibit which andthe musical resolve listener expectations about in forthcomingnotes musical that these structures. expectations He mayin differ terms independently of the degreetive, to their which strength they are andin passive their particular, or that specificity. ac- He affectpectation is contends, induced aroused by when amade antecedent passive active musical ex- by structures itmanently is being blocked temporarily by inhibitedMeyer or consequent discusses three per- musical ways in structures. pectations which may the be listener’s ex- violated. The first occurs when the ferent from natural languagedo in not that usually musical carry elements clear refe semantics in thesometimes possible way to establish that indexical references for linguisticappropriately enculturated atoms listeners – do. thesea, old It a castle, the is storm, theBrunhilde’s spring, leitmotif love, being James good examples Bond’svarious taken theme pieces from or of music.communicate complex However, statements it like isto impossible sea to lasttle, spring James and Bond thenBrunhilde would returned not to have the fallen old in cas- love with arguments made by structure of musicterns affords of the tension communication and resolution,manipulation of through of pat- the the systematic based listener’s structural in expectations, turn ontations of their the internalized style. syntactic represen- affective responses to music thatit is relevant relates here the since to expression predictive and processinginformation-theoretic perception of principles of musical [26. meaning structure, using strategies for segmenting music aresical influenced training. by mu- CPS to musical phrasesicians than boundaries in is nonmusicians stronger [26. in mu- sion). Therefore, meaning inis musical borne communication tothe a listener’s perception large of extent structuralmusical relations by elements between syntactic [26. structure and ]. 195 , 37 ]. Psychological ]. Therefore, future research xplicitly between computa- 194 , 193 ]. A subsequent study showed that the ], which seem to be related to prosodic 190 188 , 189 tional modeling, psychologicalcognitive-neuroscientific investigation experimentation in further and devel- oping our understanding of musical syntax [26. should triangulate more e Musical styles cananalogous way be to said thatprogramming to in languages) which possess do. natural syntax languages However, (or music in is an dif- 26.4.2 Syntax, Semantics and Emotion 26.4.1 Convergence Between Approaches One of theto key integrate insights challenges from facingical these future different and methodolog- research epistemologicalmodeling is allows approaches. us Computational tostructure implement and theories syntax of andimplemented musical examine algorithm when the supplied with behavior musicalamples. of ex- This can the provide insights into optimalrepresentations syntactic and absolute constraints ontic the structure syntac- of a musicalresearch, style192 [26. on theof other syntactic forms hand, andcapable can relationships of indicate that perceiving, listeners representing theplementing are and a kinds learning. psychological theory Im- model as a requires computational theand theory also to allows bethrough the quantitative precisely comparison theory expressed ofresponses. to human It and also be model allows tested comparisonmance of at and human different perfor- refined levelswith of optimal syntactic experience parsing. andresearch Finally, provides expertise neuroscientific information about the neuralsyntactic basis processing, of which canhuman impose syntactic constraints processing on andis also more provide sensitive data to implicit that knowledgemethods than (e.g., behavioral [26. We have reviewed empirical research on musicalfrom syntax computational, psychological and neuroscientific perspectives. We closesues with that naturally a arise during discussionextent this discussion: to of first, the which twosyntax the is- converge; different and second, perspectives thesyntax on relationship and between musical semantics in music,the which question naturally of invokes affective responses to music. 26.4 Implications and Issues terior mediotemporal cortex. Similarhave neural been responses observed atlanguage [26. syntactic phrase boundaries in cues [26. Music Psychology – Physiology

Part C

Part C | 26.4 498 Musical Syntax II: Empirical Perspectives References 499 expected consequent event is delayed, the second when the feedback of information such that future behavior the antecedent context generates ambiguous expecta- is conditioned by the results of past events. tions about consequent events, and the third when the Meyer notes that uncertainty may arise from dif- consequent is unexpected. While the particular effect of ferent sources. Thus, systemic uncertainty decreases music is clearly dependent on the strength of the expec- throughout the experience of a piece of music as the tation, Meyer argues that it is also conditioned by the listener’s model develops and the composer may delib- specificity of the expectation. erately introduce designed uncertainty to combat this Meyer [26.200] discusses the relationship between effect. Furthermore, the redundancy (lack of uncer- his theory of musical expectancy and concepts in tainty) inherent in a style serves to combat noise, be information theory. He starts with the suggestion it cultural (resulting from discrepancies between the that [26.200, p. 414]: habit responses of a given listener and those operating in the style) or acoustical. Witten et al. [26.201, p.71] once a musical style has become part of the habit make a similar distinction between perceptual uncer- responses of composers, performers and practiced tainty (that which is relative to a particular listener’s listeners it may be regarded as a complex system of model) and stylistic uncertainty (that which is inherent probabilities in the musical style). and that expectations arise out of these internalized As noted above, the most obvious emotional re- probability systems. In particular, he suggests that sponse to expectancy violation and uncertainty in a musical style may be conceived as a Markov process music is tension but according to Meyer [26.198], (Sect. 26.1.3) and that experienced listeners possess in- these processes may also give rise to a range of spe- ternalized models of that process. The degree to which cific emotional experiences including apprehension/ hypothetical meanings provide ambiguous expectations anxiety (p.27), hope (p.29), and disappointment (p.182) about consequent structures can be measured by en- (see also [26.202]). Convergingly, empirical research tropy (or uncertainty) [26.200, p. 416]: has found that stylistically unusual chord progressions do stimulate increases in physiological arousal [26.177] The lower the probability of a particular consequent while notes that have a low probability of occurrence [...] the greater the uncertainty (and information) in performed melodic music [26.28] have been found involved in the antecedent-consequent relation. to be associated with increased arousal, reduced va- An unexpected consequent conveys a maximum of in- lence, increased skin conductance and reduction in formation. The process of revaluation corresponds to heart rate [26.35].

References atC|26 | C Part 26.1 P.N. Johnson-Laird: Mental Models (Harvard Univ. 26.8 M. Rohrmeier, P. Rebuschat: Implicit learning and Press, Cambridge 1983) acquisition of music, Top. Cogn. Sci. 4(4), 525–553 26.2 H.C. Longuet-Higgins: Artificial intelligence – (2012) A new theoretical psychology?, Cognition 10(1–3), 26.9 N. Chomsky: Aspects of the Theory of Syntax (MIT 197–200 (1981) Press, Cambridge 1965) 26.3 H.A. Simon, C.A. Kaplan: Foundations of cognitive 26.10 D. Deutsch, J. Feroe: The internal representation science. In: Foundations of Cognitive Science,ed. of pitch sequences in tonal music, Psychol. Rev. by M.I. Posner (MIT Press, Cambridge 1989) pp. 1– 88(6), 503–522 (1981) 47 26.11 H.A. Simon, R.K. Sumner: Pattern in music. In: 26.4 A. Newell, H.A. Simon: Computer science as em- Formal Representation of Human Judgement,ed. pirical enquiry: Symbols and search, Commun. by B. Kleinmuntz (Wiley, New York 1968) pp. 219– ACM 19(3), 113–126 (1976) 250 26.5 M. Steedman: A generative grammar for jazz 26.12 L.B. Meyer: Explaining Music: Essays and Explo- chord sequences, Music Percept. 2(1), 52–77 (1984) rations (University of Chicago Press, Chicago 1973) 26.6 M. Steedman: The blues and the abstract truth: 26.13 E. Narmour: The Analysis and Cognition of Basic Music and mental models. In: Mental Models in Melodic Structures: The Implication-realisation Cognitive Science, ed. by A. Garnham, J. Oakhill Model (University of Chicago Press, Chicago 1990) (Erlbaum, Mahwah 1996) pp. 305–318 26.14 C. Ames: Automated composition in retrospect: 26.7 A. Clark: Learning trees from strings: A strong 1956–1986, Leonardo 20(2), 169–185 (1987) learning algorithm for some context-free gram- 26.15 C. Ames: The Markov process as a compositional mars,J.Mach.Learn.Res.14, 3537–3559 (2013) model: A survey and tutorial, Leonardo 22(2), 175– 187 (1989) (3), 3 ,253–272 21 56(2), 169– Mathemat- , Doctoral Dis- , Communications (1), 164–171 (1970) https://doi.org/10.3389/ 41 rrelates of melodic pro- ndvalet, S. Bengio: Predic- , 533–553 (2013) (6), 1554–1563 (1966) 13 37 , 1483–1493 (2012) (7), e66174 (2013) 50 8 Dynamic Bayesian Networks: Repre- (2), 257–286 (1989) , 302–313 (2010) ,1052(2014), 77 5 50 (3), 45–52 (2004) , 93–112 (2005) 28 ,Vol.5(2005)pp.304–311 14 , 1749–1762 (2013) violation predict psychophysiologicalresponses emotional to liveBehav. Neurosci. concert music, Cogn. Affect. pitch probabilities inropsychologia congenital amusia, Neu- Electrophysiological co cessing in congenital51 amusia, Neuropsychologia of melody istency: Narmour’s constrained principles affect by melodic learn- ing, melodic PLOS ONE inconsis- fpsyg.2014.01052 J. Bhattacharya: Unsupervised statistical learning underpins computational, behavioural and neu- ral manifestations of musicalroImage expectation, Neu- tainty in auditoryPsychol. sequence processing, Front. abilistic functions ofAnn. finite Math. state Stat. Markov chains, imization techniquecal occurring analysis in ofchains, probabilistic the Ann. functions Math. statisti- of Stat. Markov production in greek church chant, Comput.col. Musi- and selected applications inProc. speech IEEE recognition, Identification of hiddenvariables throughogy-sensitive topol- modelics selection. and In: Computationin in Computer Music and InformationBerlin, Science, Heidelberg 2009) (Springer, pp. 205–217 Palestrina style. Agrammar induction, case J. study Music Theory in probabilistic modeling of musical structure, J. Math. Music 117–136 (2009) analysis using probabilistic models, Comput. Mu- sic J. tation for harmonic content inISMIR music signals. In: sentation, Inference and Learning sertation (Univ. of California, Berkeley 2002) tive models for music, Connect. Sci. (2009) 223 (2012) 26.38 D. Omigie, M.T. Pearce, L. Stewart: Tracking of 26.39 D. Omigie, M.T. Pearce, V. Williamson, L. Stewart: 26.40 M. Rohrmeier, I. Cross: Artificial grammar learning 26.37 M.T. Pearce, M.H. Ruiz, S. Kapasi, G.A. Wiggins, 26.36 N.C. Hansen, M.T. Pearce: Predictive uncer- 26.41 L.E. Baum, T. Petrie: Statistical inference for prob- 26.42 L.E. Baum, T. Petrie, G. Soules, N. Weiss: A max- 26.44 P. Mavromatis: A hidden Markov model of melody 26.43 L.R. Rabiner: A tutorial on hidden Markov models 26.46 P. Mavromatis: Exploring the rhythm of the 26.45 P. Mavromatis: HMM analysis of musical structure: 26.47 P. Mavromatis: Minimum description length 26.48 C. Raphael, J. Stoddard: Functional harmonic 26.49 J.P. Bello, J. Pickens: A robust mid-level represen- 26.50 K.P. Murphy: 26.51 J.-F. Paiement, Y. Gra (1), and 11(1), ,ed. ,ed. 24 Proc. 1st (3), 43–51 10(1), 96– 12 mplexity , 625–652 (2012) 4 (1988) pp. 125–137 (Springer, London (2), 164–175 (2012) Proc. 9th Int. Symp. (3), 379–423 (1948), , Ph.D. Thesis (Lon- Experimental Music 27 83 (Cambridge Univ. Press, ems: Time co : Modelling and generat- The Computer and Music Information Theory, Inference, , 237–266 (2013) The Construction and Evaluation of (2), 77–86 (1956) Prediction and Entropy of Music, Mas- 42 194 ,24–35(1958) 2 7(2), 137–163 (1962) Computer Music Modelling and Retrieval Webern’s Symphonie, Op. 21, J.60–115 Music (1967) Theory Sci. Am. Sci. Statistical Models ofPerception Melodic and Structure Composition indon City Music University, London 2005) ods for combiningIn: statistical models of music. 51–73 (1995) Theory torical survey. In: tems for music prediction, J. New Music Res. (McGraw–Hill, New York 1959) four sonata expositions, J. Music Theory ter’s dissertation (University1990) of Calgary, Calgary by U.K. Wiilpp. 295–312 (Springer, Berlin, Heidelberg 2005) ing music using multiple viewpoints.Workshop In: AI Music, Menlo Park The information dynamicsand cognition, of TopiCS Cogn. music Sci. perception and 623–656 tion processing in musicview,Int.J.Psychophysiol. cognition. A critical re- by H.B. Lincoln (Cornellpp. Univ. 42–96 Press, Cornell 1970) nication, Bell Syst. Tech. J. Cambridge 2003) and Learning Algorithms four–part chorales, Comput. Music J. Comput. Music2012) Model. pp. Retr. 357–370 based models of harmony. In: McAdams: Probabilistic models of expectation the construction of domains forviewpoints complex in musical the harmonisation problem,Music J. New Res. (1988) Multiple viewpoint syst 137 (1966) 26.25 R.C. Pinkerton: Information theory and melody, 26.24 L. Hiller, R. Fuller: Structure and information in 26.16 J.E. Cohen: Information theory and music, Behav. 26.29 M.T. Pearce, D. Conklin, G.A. Wiggins: Meth- 26.28 M.T. Pearce: 26.26 J.E. Youngblood: Style as information, J. Music 26.17 L. Hiller: Music composed with computers – A his- 26.27 D. Conklin, I.H. Witten: Multiple viewpoint sys- 26.23 L. Hiller, C. Bean: Information theory analyses of 26.31 D. Conklin, J.G. Cleary 26.30 D. Conklin: 26.20 M.T. Pearce, G.A. Wiggins: Auditory expectation: 26.19 M.A. Rohrmeier, S. Koelsch: Predictive informa- 26.18 C.E. Shannon: A mathematical theory of commu- 26.22 L. Hiller, L. Isaacson: 26.21 D.J.C. MacKay: 26.32 K. Ebcioğlu: An expert system for harmonising 26.34 M. Rohrmeier, T. Graepel: Comparing feature- 26.35 H. Egermann, M.T. Pearce, G.A. Wiggins: 26.33 R. Whorley, G. Wiggins, C. Rhodes, M.T. Pearce: Music Psychology – Physiology

Part C

Part C | 26 500 Musical Syntax II: Empirical Perspectives References 501

26.52 S.A. Raczyński, S. Fukayama, E. Vincent: Melody sic Percept. Cogn., Evanston (2004) pp. 414–415 harmonization with interpolated probabilistic 26.73 A. Marsden: Representing melodic patterns as models, J. New Music Res. 42(3), 223–235 (2013) networks of elaborations, Comput. Humanit. 35, 26.53 D. Huron: Sweet Anticipation: Music and the 37–54 (2001) Psychology of Expectation (MIT Press, Cambridge 26.74 A. Marsden: Generative structural representation 2006) of tonal music, J. New Music Res. 34,409–428 26.54 J.R. Saffran, E.K. Johnson, R.N. Aslin, E.L. New- (2005) port: Statistical learning of tone sequences by 26.75 J.D. Yust: Formal Models of Prolongation, Ph.D. human infants and adults, Cognition 70(1), 27–52 Thesis (University of Washington, Washington (1999) 2006) 26.55 H. Schenker: Der Freie Satz. Neue Musikalische 26.76 P.B. Kirlin: Using harmonic and melodic analy- Theorien und Phantasien (Margada, Liège 1935) ses to automate the initial stages of Schenkerian 26.56 A. Forte, S.E. Gilbert: Introduction to Schenkerian analysis. In: Proc. Int. Conf. Music Inf. Retr. (IS- Analysis (Norton, New York 1982) MIR), Kobe (2009) pp. 423–428 26.57 F. Lerdahl, R. Jackendoff: A Generative Theory of 26.77 P.B. Kirlin: A data set for computational studies of Tonal Music (MIT Press, Cambridge 1983) Schenkerian analysis. In: Proc. 15th Int. Soc. Music 26.58 F. Lerdahl: Tonal Pitch Space (Oxford Univ. Press, Inf. Retr. Conf (2014) pp. 213–218 Oxford 2001) 26.78 P.B. Kirlin, D.D. Jensen: Using supervised learning 26.59 E. Narmour: The Analysis and Cognition of to uncover deep musical structure. In: Proc. 29th Melodic Complexity: The Implication-realisation AAAI Conf. Artif. Intell. (2015) pp. 1770–1776 Model (University of Chicago Press, Chicago 1992) 26.79 E. Narmour: Beyond Schenkerism: The Need for Al- 26.60 M. Rohrmeier: A generative grammar approach to ternatives in Music Analysis (University of Chicago diatonic harmonic structure. In: Proc. 4th Sound Press, Chicago 1975) Music Comput. Conf. (SMC07), ed. by C. Spyridis, 26.80 C.L. Krumhansl: Music psychology and music the- A. Georgaki, G. Kouroupetroglou, C. Anagnos- ory: Problems and prospects, Music Theory Spectr. topoulou (2007) pp. 97–100 17,53–90(1995) 26.61 M. Rohrmeier: Towards a generative syntax of 26.81 E.G. Schellenberg: Expectancy in melody: Tests tonal harmony, J. Math. Music 5(1), 35–53 (2011) of the implication-realisation model, Cognition 26.62 M. Kassler: Proving musical theorems I: The mid- 58(1), 75–125 (1996) dleground of Hienrich Schenker’s theory of tonal- 26.82 M. Grachten, J.L. Arcos, R.L. de Mántaras: Melody ity, Technical Report no. 103, (University of Sydney, retrieval using the Implication/Realization model. Sydney 1975) In: Proc. 6th Int. Conf. Music Inf. Retr (Queen Mary 26.63 M. Kassler: A Trinity of Essays, Ph.D. Thesis University of London, London 2005) (Princeton University, Princeton 1967) 26.83 R. Jackendoff: Musical parsing and musical affect, 26.64 M. Kassler: Explication of the middleground of Music Percept. 9(2), 199–229 (1991) Schenker’s theory of tonality, Misc. Musicol. Adel. 26.84 F. Lerdahl, C.L. Krumhansl: Modeling tonal ten- Stud. Music. 9, 72–81 (1977) sion, Music Percept. 24, 329–366 (2007) 26.65 M. Kassler: APL applied in music theory, APL Quote 26.85 M. Hamanaka, K. Hirata, S. Tojo: Implementing Quad 18,209–214(1988) a generative theory of tonal music, J. New Mu-

26.66 R.E. Frankel, S.J. Rosenschein, S.W. Smoliar: sic Res. 35, 249–277 (2006) 26 | C Part Schenker’s theory of tonal music – its explication 26.86 M. Hamanaka, K. Hirata, S. Tojo: FATTA: Full au- through computational processes, Int. J. Man– tomatic time-span tree analyzer. In: Proc. Int. Mach. Stud. 10, 121–138 (1976) Comput. Music Conf. (ICMC), Copenhagen (2007) 26.67 R.E. Frankel, S.J. Rosenschein, S.W. Smoliar: pp. 153–156 A LISP-based system for the study of Schenkerian 26.87 H. Riemann: Musikalische Syntaxis. Grundriss analysis, Comput. Humanit. 10,21–32(1978) einer harmonischen Satzbildungslehre (Breitkopf 26.68 S.W. Smoliar: A computer aid for Schenkerian Härtel, Leipzig 1877) analysis, Comput. Music J. 4, 41–59 (1980) 26.88 T. Winograd: Linguistics and computer analysis of 26.69 M. Baroni: The concept of musical grammar tonal harmony, J. Music Theory 12, 3–49 (1968) (translated by S. Maguire with the assistance of 26.89 J. Sundberg, B. Lindblom: Generative theories in W. Drabkin), Music Anal. 2, 175–208 (1983) language and music descriptions, Cognition 4, 26.70 M. Baroni, R. Dalmonte, C. Jacobini: Theory and 99–122 (1976) analysis of European melody. In: Computer Rep- 26.90 M. Steedman: The Syntactic Process (MIT Press, resentations and Models in Music, ed. by A. Mars- Cambridge 2000) den, A. Pople (Academic Press, London 1992) 26.91 M. Granroth-Wilding, M. Steedman: A robust pp. 187–206 parser-interpreter for jazz chord sequences, 26.71 A. Marsden: Schenkerian analysis by computer: J. New Music Res. 43, 355–374 (2014) A proof of concept, J. New Music Res. 39, 269–289 26.92 W.B. De Haas: Music Information Retrieval Based (2010) on Tonal Harmony, Doctoral Dissertation (Utrecht 26.72 P. Mavromatis, M. Brown: Parsing context-free Univ., Utrecht 2012) grammars for music: A computational model of 26.93 B. De Haas, M. Rohrmeier, R. Veltkamp, F. Wiering: Schenkerian analysis. In: Proc. 8th Int. Conf. Mu- Modeling harmonic similarity using a generative (1), 19 ,468– (2012) 4 (OUP, Ox- Proc. 14th Proc. 29th , B11–B20 81 (6), 519–524 (8), 1347–1375 41 58 , 3095–3104 (2011) (Wiley, Chichester (4), 403–410 (1986) 23 12 B. Poulin-Charronnat, , 142 (2011) 2 (1), 159–171 (2003) 29 (1), 184–197 (1999) 25 ,1–11(2005) ,107 885–913 (2000) Brain and Music (1), 159–171 (2003) (3), 295–318 (1997) 1060 29 14 Music, Language and the Brain (2013) Learning distributed representationsple-viewpoint for melodic multi- prediction. In: Int. Soc. Music Inf. Retriev.ritiba Conf. (ISMIR 2013), Cu- ory representations for music, Cogn. Sci. the cognitive sciences, TopiCS Cogn. Sci. 53–96 (1995) Modeling temporalmensional dependencies sequences: Application in to polyphonic music high-di- generation and transcription. In: Int. Conf. Mach. Learn. (ICML), Edinburgh 484 (2012) (1987) knowledge in nonmusicians listeners,Acad. Ann. Sci. N. Y. 2012) sical expectancy: Priming ofchol. Hum. chords, Percept. Perform. J. Exp. Psy- fect of global structure and temporal organization on chord processing,cept. J. Perform. Exp. Psychol. Hum. Per- ford 2008) the neural encodingpothesis, of Front. speech? Psychol. The OPERA hy- Spreading activation orspectra?, overlapping Percept. frequency Psychophys. tions influence earlyfrom pitch ERPs,processing: J. Cogn. Evidence Neurosci. The effect of harmonic context onitoring phoneme mon- in vocal music, Cognition (2005) D.A. D’Adamo: Sensory versusnents in cognitive harmonic compo- priming, J.Percept. Exp. Psychol. Perform. Hum. realisation model ofPercept. melodic expectancy, Music learning of music:Psychol. A Rev. self-organizing approach, D. Manderlier: Repetition priming:cial?, Is Q. music J. spe- Exp. Psychol. Sect. A (2001) Cognitive versus sensory components in harmonic priming effects, Perform. J. Exp. Psychol. Percept. Hum. 26.112 S. Cherla, T. Weyde, A. d’Avila Garcez, M.T. Pearce: 26.110 E.W. Large, C. Palmer, J.B. Pollack: Reduced mem- 26.113 M.T. Pearce, M. Rohrmeier: Music cognition and 26.111 N. Boulanger-Lewandowski, Y. Bengio, P. Vincent: 26.114 A. Patel: 26.119 B. Tillmann: Implicit investigations of tonal 26.117 J.J. Bharucha, K. Stoeckig: Reaction time and mu- 26.120 E. Bigand, F. Madurell, B. Tillmann, M. Pineau: Ef- 26.115 A.D. Patel: Why would musical training benefit 26.116 S. Koelsch: 26.118 J.J. Bharucha, K. Stoeckig: Priming of chords: 26.121 F. Marmel, F. Perrin, B. Tillmann: Tonal expecta- 26.122 E. Bigand, B. Tillmann, B. Poulin, D.A. D’Adamo: 26.125 E. Bigand, B. Poulin, B. Tillmann, F. Madurell, 26.127 E.G. Schellenberg: Simplifying the implication- 26.126 B. Tillmann, J.J. Bharucha, E. Bigand: Implicit 26.123 E. Bigand, B. Poulin, B. Tillmann, D. D’Adamo: 26.124 E. Bigand, B. Tillmann, (2012) ,Vol.1, Proc. Int. ,ed.by Proc. 13th , ed. by M. Goto Proc. 10th Int. Soc. , 66 (1992) (2007) pp. 83–94 8 electronic songbooks, , Ph.D. Thesis (Technische al representations of mu- , 592–608 (1990) (1), 77–105 (1990) (W.H. Freeman, San Francisco ,78(1994) 16 46 9 Psychology of Music (2), 214–222 (2011) Implicit Learning of Musical Struc- (2/3), 247–280 (1994) 20 , 27–37 (2001) 6 , Doctoral Dissertation (University of A Hierarchical and Deterministic Ap- 30 Vision: A Computational Investigation Parallel Distributed Processing: Experi- , ed. by J. Louhivuori, T. Eerola, S. Saarikallio, ,179–211(1990) ture: Experimental and Computational Modelling Approaches Cambridge, Cambridge 2010) learning of context-free harmonicProc. structure. 7th In: Trienn.sic Conf. Eur. Soc.T. Cogn. Sci. Himberg, Mu- Jyväskylä P.-S. 2009) pp. Eerola 443–452 (Univ. ofinternal representations Jyväskylä, through errortion. propaga- In: ments in theed. Microstructure of by D.E. Cognition Rumelhart,search Group J.L. (MIT McClelland, Press, Cambridge PDP 1986)40 pp. Re- 25– tions, Artif. Intell. mechanism of implicit learning, J.Learn. Exp. Mem. Psychol.: Cogn. D. Deutschpp. (Academic 343–390 Press, Newand York online 1982) learningscious. of Cogn. melodic structure, Con- into the Human Representation and ProcessingVisual of Information 1982) by prediction: Exploring the benefitscoustic of constraints psychoa- andConnect. Sci. multi–scale processing, sical pitch. In: 14 Comput. Musicol. Universität Berlin, Berlin 2005) proach to Musicto Grammars Unmeasured Preludes and its Application free grammar for melodic reduction. In: Workshop Artif. Intell. Music, 20th Int.tif. Jt. Intell. Conf. Ar- (IJCAI), Hyderabad ysis: Challenging theMusic Gestalt Res. principles, J. New (2009) pp. 549–554 proving audio chordharmonic transcription and by metric exploiting Int. knowledge. Soc. In: Music Inf. Retr. Conf. (ISMIR 2012) Music Inf. Retr. Conf. (ISMIR 2009) grammar of tonal harmony. In: pp. 295–300 software, Comput. Musicol. 26.107 M. Rohrmeier, I. Cross: Tacit tonality: Implicit 26.108 D.E. Rumelhart, G. Hinton, R. Williams: Learning 26.109 J.B. Pollack: Recursive distributed representa- 26.106 M. Rohrmeier: 26.105 E. Servan-Schreiber, J.R. Anderson: Chunking as a 26.104 M. Rohrmeier, P. Rebuschat, I. Cross: Incidental 26.101 M.C. Mozer: Neural network music composition 26.103 R.N. Shepard: Structur 26.102 J.L. Elman: Finding structure in time, Cogn. Sci. 26.99 H. Schaffrath: The ESAC 26.100 D. Marr: 26.96 E. Gilbert, D. Conklin: A probabilistic context- 26.97 R. Bod: Memory-based models of melodic anal- 26.94 W.B. De Haas, J.P. Magalhães, F. Wiering: Im- 26.95 D. Tidhar: 26.98 H. Schaffrath: The ESAC databases and MAPPET Music Psychology – Physiology Part C

502 Part C | 26 Musical Syntax II: Empirical Perspectives References 503

26.128 W.F. Thompson, M. Stainton: Expectancy in Bo- 26.145 M.G. Boltz, M.R. Jones: Does rule recursion make hemian folk song melodies: Evaluation of im- melodies easier to reproduce? If not, what does?, plicative principles for implicative and closural Cogn. Psychol. 18(4), 389–431 (1986) intervals, Music Percept. 15(3), 231–252 (1998) 26.146 V.J. Konečni: Elusive effects of artists’ “mes- 26.129 P.T. von Hippel, D. Huron: Why do skips pre- sages”. In: Cognitive Processes in the Perception cede reversals? The effects of tessitura on melodic of Art, ed. by W.R. Crozier, A.J. Chapman (North structure, Music Percept. 18(1), 59–85 (2000) Holland, Amsterdam 1984) pp. 71–93 26.130 F.A. Russo, L.L. Cuddy: A common origin for vo- 26.147 H. Gotlieb, V.J. Konečni: The effects of instru- cal accuracy and melodic expectancy: Vocal con- mentation, playing style, and structure in the straints, J. Acoust. Soc. Am. 105, 1217 (1999) Goldberg Variations by Johann Sebastian Bach, 26.131 A. Tierney, F.A. Russo, A.D. Patel: The motor origins Music Percept. 3, 87–102 (1985) of human and avian song structure, Proc. Natl. 26.148 N. Cook: The perception of large-scale tonal clo- Acad. Sci. (PNAS) USA 108, 15510–15515 (2011) sure, Music Percept. 5(2), 197–206 (1987) 26.132 P.T. von Hippel: Melodic-expectation rules as 26.149 R.O. Gjerdingen: An experimental music theory? learned heuristics. In: Proc. 7th Int. Conf. Music In: Rethinking Music, ed. by M. Everist, N. Cook Percept. Cogn., ed. by C. Stevens, D. Burnham, (Oxford Univ. Press, Oxford 1999) pp. 161–170 E. Schubert, J. Renwick (Causal Productions, Ade- 26.150 M. Karno, V.J. Konečni: The effects of structural laide 2002) pp. 315–317 interventions in the First Movement of Mozart’s 26.133 N. Oram, L.L. Cuddy: Responsiveness of West- Symphony in G-Minor K. 550 on aesthetic prefer- ern adults to pitch-distributional information in ence, Music Percept. 10, 63–72 (1992) melodic sequences, Psychol. Res. 57(2), 103–118 26.151 E.W. Marvin, A. Brinkman: The effect of modu- (1995) lation and formal manipulation on perception of 26.134 C.L. Krumhansl, P. Toivanen, T. Eerola, P. Toivi- tonic closure by expert listeners, Music Percept. ainen, T. Järvinen, J. Louhivuori: Cross-cultural 16, 389–408 (1999) music cognition: Cognitive methodology applied 26.152 S. Koelsch, M. Rohrmeier, R. Torrecuso, to North Sami yoiks, Cognition 76(1), 13–58 (2000) S. Jentschke: Processing of hierarchical syn- 26.135 T. Eerola: Data-driven influences on melodic ex- tactic structure in music, Proc. Natl. Acad. Sci. pectancy: Continuations in North Sami Yoiks rated Am. 110, 15443–15448 (2013) by South African traditional healers. In: Proc. 8th 26.153 S.M. Kosslyn: If neuroimaging is the answer, what Int. Conf. Music Percept. Cogn,ed.byS.D.Lip- is the question?, Proc. R. Soc. B 354, 1283–1294 scomb, R. Ashley, R.O. Gjerdingen, P. Webster (1999) (Causal Productions, Adelaide 2004) pp. 83–87 26.154 M. Mather, J.T. Cacioppo, N. Kanwisher: How fMRI 26.136 J.C. Carlsen: Some factors which influence can inform cognitive theories, Perspect. Psychol. melodic expectancy, Psychomusicology 1(1), 12–29 Sci. 8, 108–113 (2013) (1981) 26.155 R.A. Poldrack: Can cognitive processes be inferred 26.137 M.A. Castellano, J.J. Bharucha, C.L. Krumhansl: from neuroimaging data?, Trends Cogn. Sci. 10, Tonal hierarchies in the music of North India, 59–63 (2006) J. Exp. Psychol. Gen. 113(3), 394–412 (1984) 26.156 C.N. White, R.A. Poldrack: Using fMRI to constrain 26.138 E.J. Kessler, C. Hansen, R.N. Shepard: Tonal theories of cognition, Perspect. Psychol. Sci. 8,

schemata in the perception of music in Bali and 79–83 (2013) 26 | C Part the West, Music Percept. 2(2), 131–165 (1984) 26.157 S. Koelsch, T. Gunter, A.D. Friederici: Brain indices 26.139 C.L. Krumhansl, J. Louhivuori, P. Toiviainen, of music processing: “Nonmusicians” are musi- T. Järvinen, T. Eerola: Melodic expectation in cal, J. Cogn. Neurosci. 12(3), 520–541 (2000) Finnish spiritual hymns: Convergence of statisti- 26.158 S. Koelsch, S. Kilches, N. Steinbeis, S. Schelinski: cal, behavioural and computational approaches, Effects of unexpected chords and of performer’s Music Percept. 17(2), 151–195 (1999) expression on brain responses and electrodermal 26.140 M. Rohrmeier, R. Widdess: Incidental learning of activity, PLoS One 3(7), e2631 (2008) melodic structure of North Indian music, Cogn. 26.159 S. Koelsch, B.-H. Schmidt, J. Kansok: Effects of Sci. (2016), https://doi.org/10.1111/cogs.12404 musical expertise on the early right anterior neg- 26.141 E.G. Schellenberg, M. Adachi, K.T. Purdy, ativity: An event-related brain potential study, M.C. McKinnon: Expectancy in melody: Tests Psychophysiology 39(5), 657–663 (2002) of children and adults, J. Exp. Psychol.: Gen. 26.160 S. Leino, E. Brattico, M. Tervaniemi, P. Vuust: Rep- 131(4), 511 (2002) resentation of harmony rules in the human brain: 26.142 C.L. Krumhansl: Cognitive Foundations of Musical Further evidence from event-related potentials, Pitch (Oxford Univ. Press, Oxford 1990) Brain Res. 1142, 169–177 (2007) 26.143 W.E. Caplin: Classical Form: A Theory of Formal 26.161 P. Loui, T. Grent, D. Torpey, M.G. Woldorff: Effects Functions for the Instrumental Music of Haydn, of attention on the neural processing of harmonic Mozart, and Beethoven (Oxford Univ. Press, Ox- syntax in western music, Cogn. Brain Res. 25,678– ford 1998) 687 (2005) 26.144 D. Deutsch: The processing of structured and un- 26.162 A.D. Patel, E. Gibson, J. Ratner, M. Besson, structured tonal sequences, Percept. Psychophys. P.J. Holcomb: Processing syntactic relations in 28(5), 381–389 (1980) language and music: An event-related potential , , 2 18 (2), 89–105 8 Representing Mu- , 540–545 (2001) 4 , 2357–2365 (2011) , 267–295 (2001) 21 (2), 164–175 (2012) 30 22,776–789(2008) 83 , 2142–2152 (2003) (11), 1940–1951 (2008) 17, 956–966 (2002) , 145–161 (2003) 20 20 17, 407–421 (2005) 16 , 1277–1286 (2009) , 2791–2803 (2006) , ed. by P. Howell, R. West, I. Cross 132 23 , 259–273 (2005) Musical syntax processing inaphasia, Aphasiology agrammatic Broca’s of musical expertisephrase and perception in boundary music, J. markers Cogn. Neurosci. on language side by side inthe the generation brain: A of PETJ. melodies study Neurosci. of and sentences, Eur. Children with specific languageshow impairment impairment also of music-syntacticJ. processing, Cogn. Neurosci. G. Lohmann, A.D. Friederici:tical Bach ‘language-network’ speaks: serves A the cor- processingmusic, of Neuroimage musical semantics, Phys. Life Rev. cessed in “language” areas ofble the role brain: for A Brodmann possi- area 47ence, in NeuroImage temporal coher- ‘Musical syntax’ is processedMEG-study, in Nat. Neurosci. Broca’s area: An A.D. Friederici:cessing: Prosody-driven An event-related sentence brainJ. Cogn. potential pro- Neurosci. study, 191–196 (1999) aries, comma rules,closure positive and shift in ERPs brainfor as responses: a universal prosodic marker The phrasingJ. in Psycholinguist. Res. listeners and readers, (2011) B. Maess, A.D. Friederici, O.of Witte: phrase The perception structure in24 music, Hum. Brain Mapp. of the inferior frontalCogn. cortex Brain in Res. musical priming, The amusic brain:aware, In Brain tune, out of key, and un- processing in musicInt. cognition. J. A Psychophysiol. critical review, tentials indicate immediate usein of prosodic natural cues speech processing, Nat. Neurosci. 472–493 (2006) at the computational level. In: sical Structure (Academic Press, London 1991) pp. 291–325 implicit learning of bothstructures, Cereb. musical Cortex and linguistic 26.181 A.D. Patel, J.R. Iversen, M. Wassenaar, P. Hagoort: 26.191 C. Neuhaus, T. Knösche, A. Friederici: Effects 26.186 S. Brown, M.J. Martinez, L.M. Parsons: Music and 26.180 S. Jentschke, S. Koelsch, S. Sallat, A.D. Friederici: 26.183 S. Koelsch, T.C. Gunter, D.Y. von Cramon, S. Zysset, 26.196 S. Koelsch: Towards a neural basis of processing 26.185 D. Levitin, V. Menon: Musical structure is pro- 26.182 B. Maess, S. Koelsch, T.C. Gunter, A.D. Friederici: 26.190 A. Pannekamp, U. Toepel, K. Alter, A. Hahne, 26.189 K. Steinhauer, A.D. Friederici: Prosodic bound- 26.187 T.R. Knösche, C. Neuhaus, J. Haueisen, K. Alter, 26.184 B. Tillmann, P. Janata, J.J. Bharucha: Activation 26.194 I. Peretz, E. Brattico, M. Järvenpää, M. Tervaniemi: 26.195 M. Rohrmeier, S. Koelsch: Predictive information 26.188 K. Steinhauer, K. Alter, A.D. Friederici: Brain po- 26.192 P.N. Johnson-Laird: Jazz improvisation: A theory 26.193 C. Francois, D. Schön: Musical expertise boosts 17, ,78–84 (2), 331– 6 (3), e1832 (8), 1380– 3 38 18 , e17337 (2011) 6 (2), 202–206 (1992) 29 , 1278–1296 (1995) 10, 717–733 (1998) ts with neural responses 21 udy, J. Cogn. Neurosci. (5), 1169–1178 (2008) h, J.A. Sloboda: The role of (2), 454–459 (2009) ,38–48(2002) 18 29 39 , 1–16 (2000) 22(10), 2251–2262 (2010) 52 , 1–30 (1987) 17(10), 1565–1577 (2005) 5 , 2186–2200 (2012) (2–4), 171–179 (1990) harmonic expectancy violations in musicaltions: emo- Evidence from subjective, physiological and neural responses, J. Cogn. Neurosci. study, J. Cogn. Neurosci. ditional probability of chord progressionresponse: on An brain MEG study, PLoS ONE eralized mechanism for perceptionterns, of J. pitch Neurosci. pat- between rules and memoryrelated in music: potential An study, event- NeuroImage 694–705 (2005) pected, awaited, or55 arousing?, Int. J. Neurosci. cilitation: A connectionist framework, Musiccept. Per- expectancy in musicelectrophysiological reading: st A behavioral and (ERP) study ofof musical musicians with expectancy: nonmusicians, Comparison Hum. J. Percept. Exp. Perform. Psychol. ters: Preattentive musicality of thePsychophysiology human brain, Event-related potentials elicitedminal by notes: Effects wrong of ter- Psychol. temporal disruption, Biol. pitch expectation interac to syntactic but49 not semantic violations, Cortex between music andtic language processing of indicate musical tension-resolutionterns, seman- pat- Cereb. Cortex sentence processing, Trends Cogn. Sci. brain responses to melodies andNeurosci. chords, J. Cogn. Interaction between syntaxguage processing and in in music:rosci. An lan- ERP study, J. Cogn. Neu- lated potentials elicitedmelodies, Psychophysiology by deviant endings to (2002) responses in language-impairedgrammar-specific children reveal deficits, PLoS One (2008) 1393 (2006) 345 (2007) 26.163 N. Steinbeis, S. Koelsc 26.165 S.G. Kim, J.S. Kim, C.K. Chung: The effect of con- 26.166 P. Loui, E.H. Wu, D.L. Wessel, R.T. Knight: A gen- 26.173 R.A. Miranda, M.T. Ullman: Double dissociation 26.171 R. Verleger: P3-evoking wrong notes: Unex- 26.172 J.J. Bharucha: Music cognition and perceptual fa- 26.170 D. Schön, M. Besson: Visually induced auditory 26.167 M. Besson, F. Faïta: An event-related potential 26.164 S. Koelsch, E. Schroger, T.C. Gunter: Music mat- 26.168 H. Nittono, T. Bito, M. Hayashi, S. Sakata, T. Hori: 26.176 E. Carrus, M.T. Pearce, J. Bhattacharya: Melodic 26.177 N. Steinbeis, S. Koelsch: Shared neural resources 26.178 A.D. Friederici: Towards a neural basis of auditory 26.174 S. Koelsch, S. Jentschke: Differences in electric 26.175 S. Koelsch, T.C. Gunter, M. Wittfoth, D. Sammler: 26.169 K.A. Paller, G. McCarthy, C.C. Wood: Event-re- 26.179 E. Fonteneau, H.K.J. van der Lely: Electrical brain Music Psychology – Physiology

Part C

Part C | 26 504 Musical Syntax II: Empirical Perspectives References 505

26.197 U. Reich: The meanings of semantics: Comment 26.200 L.B. Meyer: Meaning in music and information on “Towards a neural basis of processing musical theory, J. Aesthet. Art Crit. 15(4), 412–424 (1957) semantics” by Stefan Koelsch, Phys. Life Rev. 8(2), 26.201 I.H. Witten, L.C. Manzara, D. Conklin: Comparing 120–121 (2011) human and computational models of music pre- 26.198 L.B. Meyer: Emotion and Meaning in Music (Uni- diction, Comput. Music J. 18(1), 70–80 (1994) versity of Chicago Press, Chicago 1956) 26.202 P.N. Juslin, D. Västfjäll: Emotional responses to 26.199 E. Hanslick: Vom Musikalisch-Schönen (R. Weigel, music: The need to consider underlying mech- Leipzig 1854), Reprint: Darmstadt 1965 anisms, Behav. Brain Sci. 31(5), 559–575 (2008), discussion pp. 575–621 atC|26 | C Part