A standard format proposal for hierarchical analyses and representations David Rizo Alan Marsden Department of Software and Computing Lancaster Institute for the Contemporary Arts, Systems, University of Alicante Lancaster University Instituto Superior de Enseñanzas Artísticas de la Bailrigg Comunidad Valenciana Lancaster LA1 4YW, UK Ctra. San Vicente s/n. E-03690 [email protected] San Vicente del Raspeig, Alicante, Spain [email protected] ABSTRACT portant place in historical research, in pedagogy, and even in such In the realm of digital musicology, standardizations efforts to date humble publications as concert programme notes. have mostly concentrated on the representation of music. Anal- Standard text is generally a crucial part of music analyses, often yses of music are increasingly being generated or communicated augmented by specialist terminology, and frequently accompanied by digital means. We demonstrate that the same arguments for the by some form of diagram or other graphical representation. There desirability of standardization in the representation of music apply are well established ways of representing texts in digital form, but also to the representation of analyses of music: proper preservation, not for the representation of analytical diagrams. In some cases sharing of data, and facilitation of digital processing. We concen- the diagrams are very like music notation, with just the addition of trate here on analyses which can be described as hierarchical and textual labels, brackets or a few other symbols (see Figure 1). In show that this covers a broad range of existing analytical formats. other cases they are more elaborate, and might not make direct use We propose an extension of MEI (Music Encoding Initiative) to al- of musical notation at all. It is possible to digitise these analytical low the encoding of analyses unambiguously associated with and diagrams as images, just as it is possible to convert a score to a aligned to a representation of the music analysed, making use of digital image, but with exactly the same disadvantages: the essen- existing mechanisms within MEI’s parent TEI (Text Encoding Ini- tial content of the diagram remains hidden in the image and is not tiative) for the representation of trees and graphs. exposed for explicit and unambiguous processing. Keywords Encodings, standards, music analysis, music representations 1. INTRODUCTION: DESIRABILITY OF AN ANALYSIS ENCODING STANDARD Musical scholarship in the digital age requires not just digital representations of music, but also digital representations of data as- sociated with that music. For some time, it has been recognised that standards are required for recording bibliographic and related metadata about music.1 Less attention has been paid to the digi- tal representation of analyses of pieces of music, despite the fact that a great deal of musical scholarship depends on the recording and communication of such commentary on pieces of music. This Figure 1: A Classic Turn of Phrase: Music and the Psychology is evident of course in specialist academic publications dedicated of Convention by Robert O. Gjerdingen. to music analysis, but communication of analysis also has an im- Recent developments in digital musicology make the develop- ment of standards for the representation of analytical information important. Much research in the field of Music Information Re- trieval relies on collections of data for the development and testing of algorithms. Systems involving machine learning (which have DLfM August 12, 2016, New York, USA now become standard fare in MIR) depend for their effectiveness on large sets of reliable data. To properly compare the performance ACM ISBN . of different systems designed to carry out similar tasks requires DOI: those systems to be supplied with the same sets of data. It is there- fore becoming increasingly common for researchers to make sets of 1See for example the Variations projects at the Indiana Univer- digital music-analytical data available. Examples include segmen- sity, http://www.dlib.indiana.edu/projects/variations3/ tation and chord-labelling data for Beatles songs and other popu- lar music,2 segmentation data arising from the SALAMI project,3 over what information is maintained from one level to the next, analyses in the style of Lerdahl and Jackendoff [5], Schenkerian see [20]. Other issues are discussed in [5] which also demonstrates analyses from text books and the like [15], and chord labelling and some of the difficulties over systematic treatment of Lerdahl and phrase structures in theme and variations.4 While all of these give a Jackendoff’s ‘preference rules’. clear definition of the structure of their data, they each use different representation schemes, bringing with them the common difficul- ties in reusing the data and potential for misinterpretations. Advances in technology always have an impact on music, and we have seen a transformation in the ways in which music is produced, recorded and communicated in the past few decades. Advances which envisage yet more sophisticated digital processing of mu- sic are envisaged (automatic composition, variation, arrangement, etc.). In the recent past the concepts by which music-processing has been organised have been derived from the world of sound recording (‘tracks’, ‘mixes’ and the like). Composers and perform- ers, however, tend to deal with concepts such as ‘voice’, ‘phrase’, ‘chord’ and so on, which are closer to the elements of music anal- ysis. Future, properly ‘musical’, software will need to operate with this kind of concept and so data of that nature will need to be made available. In this respect, our concerns align with those who aim to give adequate computational definitions of these high-level mu- sicological concepts [8]. Therefore, the same reasons which require standards for the dig- Figure 2: Generative theory of tonal music (GTTM) reduction ital representation of music also apply to the digital representation at different levels (extracted from [17], figure 5.8) of music analyses: to preserve not just the images of analyses but also the knowledge embodied in them, to allow the sharing of data among researchers, and to facilitate automatic processing of that data. There is no limit to forms which analytical commentary on music may take, so it is unrealistic to seek standards for the rep- resentation of all kinds of analysis. In this project, we concentrate on those classes of analysis which are hierarchical, generally rep- resented in the forms of trees and graphs. (a) Schenker’s analysis of the first phrase of Mozart’s piano sonata in A, K.331 (Schenker, 1935, Fig. 157) 2. HIERARCHICAL ANALYSIS AND REPRESENTATION Hierarchies have had an important role both in musical analy- sis and, more recently, in representations of music in tasks such as computer assisted composition or music information retrieval. By ‘hierarchical analysis’ or ‘hierarchical representation’, we mean an analysis or representation which presents the structure of the mu- sic on different levels, in which higher levels correspond to larger spans of the music. Relations between different elements are shown, but only between neighbouring elements, either above or below in the hierarchy, or side-to-side from one ‘branch’ to another. An anal- ysis which is based on the relationships of distant segments of mu- sic, such as a ‘paradigmatic analysis’ in the style of Nattiez [22], is not hierarchical according to our definition. While hierarchical analyses and representations are often presented as trees, it is not essential for them to be so, as will become clear from the examples (b) Tree representation derived from below. Schenker’s analysis The two most famously hierarchical kinds of music analysis are those of Schenker and of Lerdahl and Jackendoff. The well known Generative Theory of Tonal Music (GTTM) [17] proposes several Figure 3: Marsden’s tree representation Schenker’s analysis as viewpoints for analyzing tonal music based on generative linguis- proposed in [20] (figures 5 and 6). tic theory, i.e., modeling analysis using formal grammars (see fig- ures 2, 10, 11, 12, and 14). Although the theory is presented as a set of rules, and the results are presented in tree diagrams, the anal- While the analyses of Heinrich Schenker were not presented in yses are not so systematic as this might suggest. For discussion trees, levels of hierarchy are explicit, and the analyses can (with the of some of the difficulties concerning polyphony and uncertainties loss of some information) be converted to trees, as demonstrated in [20] (see figure 3). Further systematisations of Schenkerian anal- 2http://isophonics.net/content/reference-annotations ysis are found in [19, 21] and [15]. The latter presents an analysis 3http://ddmal.music.mcgill.ca/research/salami/annotations not as a tree but as a Maximal outerplanar graph (MOP), a rep- 4http://u.osu.edu/tavern/ resentation for Schenkerian analyses introduced by Yust [27] (see 5 Figure 16). 4 Less obviously, traditional tonal-harmonic analysis of music is 4 also hierarchical: spans of music are identified as being in a key, (a) Source melody. which are identified as subordinate to longer spans which might be r6 F goverened by a different, related, key. This is most evident in the ‘keyscale plots’ presented by Sapp [25], whose triangular shape F r2 G r5 r4 shows the structural similarities to the reduction trees of Lerdahl p=2 s r3 F r3 A G and Jackendoff (see Figure 4). F F r4 A G r4 A G A G (b) Propagation and prunning. Figure 6: Rizo’s metric trees propagation and prunning [24] (figure 3.28). Figure 4: Sapp’s keyscape plot (extracted from [25], figure 3.14) . Figure 7: Pinto’s graph encoding of a sequence. Figure 6 in [23] 3. EXISTING HIERARCHICAL REPRESENTATIONS Many of the approaches for hierarchical analysis and represen- Figure 5: Tree rewrite sequence proposed by Jacquemard et tations introduced above have points in common that could foster al.