Associations Between Musicology and Music Information Retrieval

12th International Society for Music Information Retrieval Conference (ISMIR 2011)

ASSOCIATIONS BETWEEN MUSICOLOGY AND MUSIC INFORMATION RETRIEVAL

Kerstin Neubarth Mathieu Bergeron Darrell Conklin Canterbury Christ Church University CIRMMT Universidad del Pa´ıs Vasco Canterbury, UK McGill University San Sebastian,´ Spain [email protected] Montreal, Canada and IKERBASQUE, Basque Foundation for Science

ABSTRACT formed a citation analysis of papers within ISMIR, counting references to individual authors and papers. It briefly ad- A higher level of interdisciplinary collaboration between dressed citer motivations such as identification of data and music information retrieval (MIR) and musicology has been methods or paying homage, and concluded that: “Without a proposed both in terms of MIR tools for musicology, and more in-depth analysis of the individual contexts surround- musicological motivation and interpretation of MIR research. ing each citation, it is difficult to tease out the precise mo- Applying association mining and content citation analysis tivations for all the references” (p. 61). Functions of refer- methods to musicology references in ISMIR papers, this pa- ences to one particular study were discussed in the editorial per explores which musicological subject areas are of inter- to the JNMR Special Issue on MIR in 2008 [1]. est to MIR, whether references to specific musicology areas This study extends previous work in several ways: It are significantly over-represented in specific MIR areas, and analyses inter-disciplinary references (musicology cited in precisely why musicology is cited in MIR. MIR) rather than intra-MIR references; also, the references are analysed at the level of MIR and musicology subject cat- 1. INTRODUCTION egories instead of individual papers. The quantitative analy- At the tenth anniversary ISMIR 2009 several contributions sis goes beyond citation counts; association mining is used discussed challenges in the further development of music here to yield interdisciplinary associations. In addition, ci- information retrieval (MIR) as a discipline, including re- tation contexts are analysed in depth to reveal functions of quests for deeper musical motivation and interpretation of musicology citations in ISMIR papers, taking into account MIR questions and results, and envisaging closer interaction both MIR-specific functions and more general referencing with source disciplines such as computer science, cognitive purposes to allow comparison with existing studies. science and musicology [4, 9, 20]. Suggestions for interdisciplinary collaboration have of- 2. DATA SELECTION AND ANALYSIS ten considered musicology as a target discipline, emphasis- ing the usefulness of MIR tools to musicology (e.g. [18]). This section presents the corpus development and the as- Occasionally mutual benefits have been explored (e.g. [15]). sociation mining and citation analysis methods used for ex- The current study addresses associations between MIR and tracting and analysing associations between musicology and musicology as a source discipline. It presents a systematic music information retrieval. analysis of how MIR, as represented at ISMIR, has drawn on musicology so far, by applying data mining and content 2.1 Sampling citation analysis to musicology references in ISMIR publi- From the cumulative ISMIR proceedings (www.ismir.net) cations. as a sampling frame, first all available full papers from 2000 Related quantitative ISMIR surveys mainly analyse top- until 2007, and all oral/plenary session papers for 2008 to ics and trends [3, 7, 10]. The study by Lee et al. [10] per- 2010, were selected. This resulted in 416 papers. Then the reference lists of those papers were screened and a purposive Permission to make digital or hard copies of all or part of this work for sample was taken of all papers which contain references to personal or classroom use is granted without fee provided that copies are musicology as a source discipline, excluding self-citations not made or distributed for profit or commercial advantage and that copies and references to other ISMIR papers. The final analysis bear this notice and the full citation on the first page. corpus consisted of 184 papers. These papers are identified c 2011 International Society for Music Information Retrieval. by their IDs within the cumulative proceedings (e.g. ID135).

429 Poster Session 3

2.2 Encoding History, criticism & philosophy History of music (8) The 184 papers of the analysis corpus were labelled accord- Philosophy of music & music semiotics (3) ing to their MIR research topic and the musicology areas Textual criticism, archival research & bibliography (22) that they cite, using the following categorisations. Electronic & computer music (7) MIR Categories. In a first step of encoding, the 184 Popular & jazz music studies (5) papers were classified into MIR research areas (Table 1). Film music studies (1) As no single standardised and comprehensive taxonomy of Theory & analysis MIR topics exists [3,6], an organisation of topics was devel- Music theory & analysis (36) oped based on ISMIR calls and programs, harmonising cate- Performance studies (6) gories across conferences. Each ISMIR paper is assigned to Ethnomusicology exactly one MIR category. Numbers in brackets in Table 1 Ethnomusicology (non-Western) (5) indicate the number of papers in each category. Ethnomusicology (folk music) (9) Ethnomusicology (other) (1) Systematic Musicology Research paradigms (6) Acoustics (11) Epistemology, interdisciplinarity Psychology of music (perception & cognition) (93) Representation & metrics (24) Psychology of music (emotion & affect) (8) Representation, metrics, similarity Psychology of music (other) (1) Data & metadata (11) Sociology & sociopsychology (18) Databases, data collection & organisation, metadata, anno- tation Transcription (42) Table 2. Thematic categories of musicology. Segmentation, voice & source separation, alignment, beat tracking & tempo estimation, key estimation, pitch tracking & spelling Musicology Categories. In a second step, the musicol- OMR (5) ogy references in the 184 papers were assigned to musicol- Optical music recognition, optical lyrics extraction ogy areas (Table 2). As the interest of this study is in mu- Computational music analysis (21) sicology as a source discipline, the labels used are based Pattern discovery & extraction, summarisation, chord la- on traditional subject organisations (e.g. [11, 14, 16]) rather belling, musical analysis (melody & bassline, harmonic, than more recent developments such as empirical or com- rhythm and form analysis) putational musicology which potentially overlap with music Retrieval (19) information retrieval (e.g. [18]). Category counts in Table 2 Query-by-example refer to the number of ISMIR papers citing this musicology Classification (32) area one or more times. A paper may reference more than Genre classification, geographical classification, artist clas- one musicology category. sification & performer identification, instrument-voice classification (instrument recognition, instrument vs voice dis- 2.3 Association Mining tinction, classification of vocal textures), mood & emotion classification Data mining of the analysis corpus is used to reveal asso- Recommendation (5) ciations between papers in specific MIR categories and pa- Recommendation methods & systems, playlist generation, pers that have citations to specific musicology categories. recommendation contexts For every musicology category A and MIR category B, the Music generation (4) support (number of papers) s(A) and s(B) were computed. Music prediction, improvisation, interactive instruments Also the support s(A, B) of an association hA, Bi (number A Software systems (10) of papers containing references to musicology category B Prototypes & toolboxes, user interfaces & usability, visuali- which are also in MIR category ) and the statistical signif- sation icance of the association were computed. User studies (5) The null hypothesis is that for an association hA, Bi the User behaviour (music discovery, collection organisation) two categories are statistically independent, i.e. that the pro- portion of papers citing musicology category A that are in MIR category B does not differ significantly from the rel- Table 1. Thematic categories and examples of topics of ative frequency of MIR category B in the general popula- MIR research. tion. Given the small corpus and low counts for many cat-

430 12th International Society for Music Information Retrieval Conference (ISMIR 2011)

egories, the appropriate test for statistical independence is perception and cognition research and acoustics are cited by Fisher’s one-tailed exact test on a 2x2 contingency table [5]. different subsets of papers on transcription. For an association hA, Bi with support s(A, B), this gives Comparing Figure 1 against MIR topics in Table 1, addi- the probability (p-value) of finding s(A, B) or more papers tional links could have been expected e.g. between papers of category B in s(A) samples (without replacement) in on data and metadata and references to textual criticism, n = 184 total papers. If the computed p-value is less than archival research and bibliography or history of music [15] the significance level α = 0.05, then we reject the null hy- or between classification and performance studies (for per- pothesis that the categories are independent. former identification), acoustics (for instrument-voice clas- Prior to computing the p-values, the counts of all MIR sification) and popular music studies or history of music (for categories B (and hence the p-values of associations) are genre classification). However, these relations are supported slightly adjusted upwards to account for the fact that only by only one or two papers each and thus do not appear papers citing musicology were included in the sample of in Figure 1. Surprisingly, the category of ethnomusicology n = 184 papers from the larger corpus of 416 papers. Under (folk music) (Table 2) does not feature in associations with the null hypothesis of independence, it is assumed that the MIR categories above the support threshold. larger set of papers has the same distribution of MIR cat- Of the 21 associations shown in Figure 1, nine are statis- egories as the smaller corpus. The adjustment is done by tically significant (Section 2.3). Table 3 enumerates those n 416 s(B) 416 × s(B)/184 increasing to and to . associations that have a p-value less than α = 0.05. In In line with the view of musicology as a source disci- Figure 1 these particular associations are shown in bold lines, pline, significant associations were oriented into rules from with a directed arrow indicating in brackets the confidence musicology to MIR categories. For every significant associ- of the oriented rule. Overall the relatively low confidence hA, Bi confidence A → B ation , the of the oriented rule was values confirm that in general musicology areas are cited s(A, B)/s(A) computed as , indicating the empirical proba- across MIR categories. bility of a paper being in MIR category B given that it cites a paper in musicology category A. Generally an association will be significant if the association support s(A, B) is high relative to the size of one 2.4 Content Citation Analysis of Referencing Functions involved category. Here this applies in particular for small categories like ethnomusicology (non-Western), OMR, user Papers supporting significant associations were analysed in studies or recommendation. Significance becomes harder to more detail to reveal functions of musicology references. achieve for associations between large categories; it is more Studies of citation behaviour have proposed several clas- likely to achieve the observed level of support at random sifications of citer motivation (e.g. [2, 12, 13, 17]). In our given the individual category distributions in the corpus. For analysis, we are mainly interested in (a) the function of the example, perception and cognition research is linked to sev- reference in the citing paper rather than conclusions about eral MIR categories with high support, but the distribution the cited work, and (b) referencing purposes that can be sug- of those MIR categories across the 93 papers citing percep- gested from the content of the citing paper and the co-text of tion and cognition does not differ significantly from their the citation. Musicology references were analysed in their distribution across all sampled papers. context in the ISMIR paper, and recurring referencing functions extracted and linked to existing citation classifications. 3.2 Referencing Functions 3. RESULTS For the content citation analysis we selected the ISMIR pa- This section presents the associations and referencing func- pers supporting the associations in Table 3, as these papers tions uncovered in the corpus. are examples of musicology and MIR categories that are significantly correlated. Of these 47 papers, 17 papers (5 3.1 Associations computational music analysis papers, 8 representation and Figure 1 presents the network of all associations with sup- metrics papers and 4 retrieval papers) also cite perception port ≥ 3 extracted by the association mining method de- and cognition research; these references were also consid- scribed in Section 2.3. The figure highlights that MIR areas ered. The in-depth analysis of citation contexts in these generally draw on more than one musicology discipline. But papers demonstrates that musicology is used for a variety there are differences in the level of co-citation, i.e. occur- of purposes. Figure 2 presents a taxonomy of referencing rences within the same ISMIR papers: For example, eight functions and the references’ contribution in the MIR work out of the ten papers on representation and metrics which (boxes), with examples of co-text. Related features from cite music theory and analysis literature also cite psycholog- the citation analysis literature [2, 12, 13, 17] are included in ical work on perception and cognition. On the other hand, italics.

431 Poster Session 3

User studies

Recommendation

4 (0.2) 3 (0.2)

Sociology & sociopsychology History of music 3

3 (0.4) Software 4 systems

Retrieval 4 Psychology of music 3 Classification (emotion & affect) 6 (0.3) Data 3 (0.4) & metadata 11 19 Textual criticism, Psychology of music archival research 3 (0.6) (perception & cognition) & bibliography 5 (0.2) OMR 15 4 Ethnomusicology (non Western) Representation 11 & metrics 27

Computational music analysis 10 (0.3) Transcription 10 (0.3)

8 Music theory & analysis 5

Acoustics

Figure 1. Associations (support ≥ 3) between musicology categories (dark boxes) and MIR categories. Edges are labelled with the support of the association, and significant (α = 0.05) associations are indicated with dark oriented edges. Rule confidence is indicated in brackets for significant associations.

A B s(A, B) p-value textual criticism, archival research & bibliography omr 5 9.7e-05 sociology & sociopsychology user studies 4 0.00068 music theory & analysis computational music analysis 10 0.0035 psychology of music (emotion & affect) data & metadata 3 0.0078 sociology & sociopsychology recommendation 3 0.0091 music theory & analysis representation & metrics 10 0.01 textual criticism, archival research & bibliography retrieval 6 0.016 history of music retrieval 3 0.037 ethnomusicology (non western) classiﬁcation 3 0.038

Table 3. Signiﬁcant (α = 0.05) associations found in the corpus. A: musicology category; B: MIR category; s(A, B): support of the association; p-value of the association.

432 12th International Society for Music Information Retrieval Conference (ISMIR 2011)

Function Co-text examples Relevance “Repeated patterns [...] represent therefore one of the most salient characteristics of musical works [music theory references]” (ID242) Contribution “This paper addresses systematic differences in the performance of final ritardandi by different pianists [...] the kinetic model is arguably too simple [...] In this work [...] [psychology of music references]” (ID159) Task definition “As stated in [music theory reference] musical analysis is ’the resolution of a musical structure into relatively simpler constituent elements, and the investigation of the functions of these elements within that structure”’ (ID24) General approach “The [basic] idea is motivated by the results of musicological studies, such as [...]” (ID859) Method/algorithm “HMM initialization [...] The covariance matrix should also reflect our musical knowledge [...], gained both from music theory as well as empirical evidence [psychology of music reference]” (ID30) Data “we evaluated both [OMR] systems on the same set of pages to measure their accuracy [...] [textual criticism, archival research & bibliography reference]” (ID729) Related work “dimensions of dissimilarity have been interpreted to be e.g. [...] [psychology of music reference]” (ID345)

Figure 2. Taxonomy of referencing functions (top) and selected examples of co-text (bottom).

4. DISCUSSION AND CONCLUSIONS whether certain MIR areas are over- or under-represented in the corpus of papers citing musicology, or comparing use of The findings presented in this study are based on direct and musicology references against other source disciplines like explicit references to musicology. However, not all refer- computer science or cognitive science. ences are explicit: papers sometimes refer to musicological work reported in earlier MIR publications; incorporate Several observations can be drawn from the results pre- concepts or approaches like music-analytical methods into sented in this paper. First, less than half of the full papers the main text without including specific references; charac- in the cumulative ISMIR proceedings (184 out of 416) cite terise the considered repertoire such as non-Western tradi- musicology. Given the close interdisciplinary links between tions without making explicit whether the description is de- MIR and musicology a larger percentage had been expected. rived from musicological research, common cultural knowl- Second, the most frequently cited category is music per- edge or the researchers’ personal experience; or use music ception and cognition research (93 citing papers across all examples without citing a musicological source. Taking into MIR categories in our corpus). On the other hand, histor- account such references is expected to strengthen rather than ical musicology and especially history of music appear to change the picture of associations presented here. be under-represented in our corpus, compared to their tra- For this study the analysis corpus only contained full IS- ditional weight in musicology [16]. Third, the association MIR papers (Section 2.1). Future work could extend the mining has revealed significant associations between certain corpus to also include posters, in particular those from IS- musicology and MIR categories. However, most pairings MIR 2008 onward (because these are of equal length and are not significant, and this may indicate opportunities for status to full papers); apply multilevel association mining category refinement and for specific interdisciplinary collab- methods [8] to hierarchical subject categorisations; allow a oration. Fourth, the content citation analysis yields a range paper to be within multiple MIR categories; and evaluate of citation purposes, from justifying the MIR topic and spec- whether the associations found here persist and whether new ifying the MIR task, through informing methods or provid- significant associations arise. Furthermore, if an encoding ing data, to references which demonstrate awareness of the of the complete ISMIR proceedings was available, other in- research context but remain without direct implications for teresting types of analysis would be possible, e.g. exploring the MIR work.

433 Poster Session 3

Following the discussions at ISMIR 2009 [4, 9, 20], in In International Society for Music Information Retrieval the further development of MIR we would expect that with Conference, fMIR workshop, 2009. the increasing interest in ethnic music (e.g. [3, 19]) ethnomusicology will more strongly feed not only into classifi- [10] J. Lee, M. Jones, and J. Downie. An analysis of ISMIR cation but also MIR areas such as representation and met- proceedings: Patterns of authorship, topic, and citation. rics, transcription or retrieval; envisage more musicology In International Society for Music Information Retrieval references, including history of music, in defining MIR re- Conference, pages 57–62, 2009. search questions and in interpreting MIR results; and en- [11] Library of Congress. Library of Congress courage more projective references highlighting potential of Classification Outline: Class M – Music. MIR achievements for musicology, beyond providing tech- http://www.loc.gov/catdir/cpso/lcco/. nological tools. The association mining and content analysis methods applied in this paper will be invaluable to study [12] M. Liu. Citation functions and related determinants: A the continuing evolution of the field of music information study of Chinese physics publications. Journal of Li- retrieval. brary and Information Science, 19(1):1–13, 1993. [13] M. Moravcsik and P. Murugesan. Some results on the 5. REFERENCES function and quality of citations. Social Studies of Sci- [1] J.-J. Aucouturier and E. Pampalk. Introduction – From ence, 5(1):86–92, 1975. genres to tags: A little epistemology of music informa- [14] R. Parncutt. Systematic musicology and the history and Journal of New Music Research tion retrieval research. , future of western musical scholarship. Journal of Inter- 37(2):87–92, 2008. disciplinary Music Studies, 1(1):1–32, 2007.

[2] T. A. Brooks. Private acts and public objects: An inves- [15] J. Riley and C. Mayer. Ask a librarian: The role of li- tigation of citer motivations. Journal of the American brarians in the music information retrieval community. Society of Information Science, 36(4):223–229, 1985. In Proceedings of the International Conference on Mu- [3] O. Cornelis, M. Lesaffre, D. Moelants, and M. Leman. sic Information Retrieval, 2006. w/o pages. Access to ethnic music: Advances and perspectives in [16] S. Sadie, editor. The New Grove Dictionary of Music and content-based music information retrieval. Signal Pro- Musicians, chapter “Ethnomusicology”, “Music Analy- cessing, 90:1008–1031, 2010. sis”, “Musicology”, “Psychology of Music”, and “The- [4] J. Downie, D. Byrd, and T. Crawford. Ten years of IS- ory, Theorists. 15: New Theoretical Paradigms, 1980- MIR: Reﬂections on challenges and opportunities. In 2000”. Macmillan, London, 2001. International Society for Music Information Retrieval [17] J. Swales. Citation analysis and discourse analysis. Ap- Conference, pages 13–18, 2009. plied Linguistics, 7(1):39–56, 1986.

[5] S. Falcon and R. Gentleman. Hypergeometric testing [18] G. Tzanetakis, A. Kapur, W. Schloss, and M. Wright. used for gene set enrichment analysis. In F. Hahne, Computational ethnomusicology. Journal of Interdisci- W. Huber, R. Gentleman, and S. Falcon, editors, Biocon- plinary Music Studies, 1(2):1–24, 2007. ductor Case Studies, pages 207–220. Springer, 2008. [19] P. van Kranenburg, J. Garbers, A. Volk, F. Wiering, [6] J. Futrelle and J. Downie. Interdisciplinary research is- L. Grijp, and R. C. Veltkamp. Collaborative perspectives sues in music information retrieval: ISMIR 2000–2002. for folk song research and music information retrieval: Journal of New Music Research, 32(2):121–131, 2003. The indispensable role of computational musicology. Journal of Interdisciplinary Music Studies, 4(1):17–43, [7] M. Grachten, M. Schedl, T. Pohle, and G. Widmer. The 2010. ISMIR cloud: a decade of ISMIR conferences at your ﬁngertips. In International Society for Music Informa- [20] F. Wiering. Meaningful music retrieval. In International tion Retrieval Conference, pages 63–68, 2009. Society for Music Information Retrieval Conference, fMIR workshop, 2009. [8] J. Han and Y.Fu. Discovery of multiple-level association rules from large databases. In International Conference on Very Large Data Bases, pages 420–431, 1995.

[9] P. Herrera, J. Serra,` C. Laurier, E. Guaus, E. Gomez,´ and X. Serra. The discipline formerly known as MIR.

434