A Computational Approach to Content-Based Retrieval of Folk Song Melodies
Total Page:16
File Type:pdf, Size:1020Kb
A Computational Approach to Content-Based Retrieval of Folk Song Melodies Peter van Kranenburg c 2010 Peter van Kranenburg ISBN 978-90-393-5393-6 SIKS Dissertation Series No. 2010-43 The research reported in this thesis has been carried out under the auspices of SIKS, the Dutch Research School for Information and Knowledge Systems. Cover design by Droppert, vld. Photo cover: Agricultural workers in flax field, Groningen, Netherlands, begin- ning of twentieth century. (Private collection, Groningen, Netherlands). Typeset with LATEX and LYX. Printed by Ridderprint Offsetdrukkerij BV (http://ridderprint.nl). All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior written permission by Peter van Kranenburg. A Computational Approach to Content-Based Retrieval of Folk Song Melodies Een computationele aanpak van het zoeken naar volksliederen op melodische inhoud (met een samenvatting in het Nederlands) PROEFSCHRIFT ter verkrijging van de graad van doctor aan de Universiteit Utrecht op gezag van de rector magnificus, prof.dr. J.C. Stoof, ingevolge het besluit van het college voor promoties in het openbaar te verdedigen op maandag 4 oktober 2010 des middags te 4.15 uur door Pieter van Kranenburg geboren op 9 juni 1977 te Schiedam Promotoren: Prof.dr. R.C. Veltkamp Prof.dr. L.P. Grijp Co-promotor: Dr. F. Wiering This research was supported by the Netherlands Organisation for Scientific Re- search (NWO) within the CATCH program (Continuous Access to Cultural Her- itage) under project number 640-003 501. pred7-24.html 9/2/10 1:38 PM !"#$ %&'()*+, ,- +* ./0*'1 2 .-0*'1)3 45 ', '4 %6 7 4 8* .9 /:'; file:///Users/pvk/Documents/Eigenwerk/diss/afbeeldingen/source/pred7-24.html Page 1 of 1 vi Contents Contents vii Preface xiii 1 Introduction 1 1.1 Context: The WITCHCRAFT Project . 1 1.2 Computational Musicology . 2 1.3 Music Information Retrieval . 3 1.4 Folk Song Research . 4 1.5 Organization of the Thesis . 5 2 Collaboration Perspectives for Folk Song Research and Music Infor- mation Retrieval: the Indispensable Role of Computational Musi- cology 7 2.1 Introduction . 8 2.2 Some Past and Current Approaches to Melody in Folk Song Re- search . 9 2.2.1 Oral Transmission and Tune Families . 10 2.2.2 Identification . 13 2.2.3 Features for Classification . 13 2.3 Computational Approaches to Folk Song Melodies . 15 vii 2.3.1 Folk Songs in Computational Musicology . 16 2.3.1.1 The Early Days . 16 2.3.1.2 Contour-Based Approaches . 17 2.3.1.3 Segmentation . 17 2.3.1.4 Other Approaches . 18 2.3.2 Folk Songs in Music Information Retrieval . 18 2.3.2.1 Online Search Engines . 19 2.3.2.2 An Example . 21 2.4 Problems and Challenges . 24 2.5 Directions for Research . 25 2.5.1 Research Question . 26 2.5.2 Cognitive Approaches . 26 2.5.3 The Perspective of Folk Song Research . 27 2.5.4 Music Information Retrieval . 27 2.5.5 Repeating Patterns and Stylistic Studies . 28 2.5.6 Global Features of Melody . 29 2.5.7 Sequence Alignment . 29 2.5.8 Evaluation and Testing . 30 2.6 Collaboration and Integration . 30 2.7 Concluding Remarks . 34 3 A Manual Annotation System for Folk Song Similarity 37 3.1 Introduction . 37 3.2 The Context of the Experiment . 39 3.3 The Annotation Method . 40 3.4 Criteria for the Similarity Annotations . 42 3.4.1 Rhythm . 42 3.4.2 Contour . 43 viii 3.4.3 Motifs . 44 3.4.4 Lyrics . 44 3.5 First experiment on similarity annotations . 45 3.5.1 Agreement among the experts . 45 3.5.2 Comparing dimensions across tune families . 46 3.5.3 Similarity within tune families . 48 3.5.4 Discussion . 49 3.6 Second experiment on similarity annotation . 49 3.7 Concluding Remarks . 52 4 The Study of Melodic Similarity using Global Melody Feature Sets 55 4.1 Data . 56 4.2 Feature Set . 56 4.3 Feature Evaluation Criterion . 57 4.4 Individual Features . 60 4.4.1 Method . 60 4.4.2 Results . 60 4.5 Feature Subsets . 61 4.5.1 Method . 61 4.5.2 Results . 62 4.6 Comparison of global and local features . 64 4.6.1 Method . 66 4.6.1.1 Global Features . 66 4.6.1.2 Local Features . 66 4.6.2 Interval Features . 67 4.6.3 Melody Features . 67 4.6.4 Global Duration-Ratio Features . 67 4.6.5 Rhythmic Features . 67 ix 4.6.6 All Features . 68 4.6.7 Results . 68 4.7 Conclusions . 68 5 On Measuring Musical Style—The Case of Some Disputed Organ Fugues in the J.S. Bach (BWV) Catalog 71 5.1 Bases for Composer Attributions . 72 5.2 A Machine Learning Approach to Stylistic Assessment . 73 5.3 Modeling Musical Style . 73 5.4 The Dataset . 74 5.4.1 Selected Features . 74 5.4.2 The Compositions . 78 5.5 Data-Analysis Methods . 79 5.6 General Findings . 80 5.6.1 J.S. Bach vs. J.L. Krebs . 80 5.6.2 J.S. Bach vs. J.P. Kellner . 83 5.6.3 J.S. Bach vs. W.F. Bach . 84 5.7 Classification of the Disputed Works . 84 5.7.1 BWV 534/2 . 85 5.7.2 BWV 536/2 . 85 5.7.3 BWV 537/2 . 86 5.7.4 BWV 555/2, 557/2, 558/2, 559/2 and 560/2 . 87 5.7.5 BWV 565/2 . 88 5.8 Concluding Remarks . 88 6 Musical Models for Folk Song Melody Alignment 91 6.1 Introduction . 91 6.2 Data . 93 6.2.1 The Data Set . 93 x 6.2.2 Representation of Melodies . 94 6.2.3 Inconsistencies in Transcriptions . 95 6.2.3.1 Absolute Pitch . 96 6.2.3.2 Length of cadence notes . 98 6.2.3.3 Rhythmic Unit . 99 6.2.3.4 Inconsistent Segmentations . 101 6.2.4 Normalization of Alignment Scores . 102 6.3 Substitution Scoring Functions . 104 6.3.1 Single substitution scoring functions . 104 6.3.2 Combinations . 107 6.3.3 Gap Penalty Function . 107 6.4 Evaluation of Substitution Functions . 107 6.4.1 Evaluation of Single Substitution Functions . 109 6.4.2 Evaluation of Combinations of Single Substitution Func- tions . 112 6.4.3 Comparison with Related Methods . 116 6.5 Evaluation of false negatives . 117 6.6 Evaluation of the Full Corpus . 120 6.7 Hard to Classify Melodies . 121 6.8 Conclusion and Future Work . 121 7 Retrieval of Folk Song Recordings using Musically Meaningful Audio Segments 123 7.1 Background . 124 7.2 Data-Set . 125 7.3 Pitch Extraction . 125 7.4 Segmentation . 126 7.5 A similarity measure for segments . 126 7.6 Selecting Representative Segments . 129 7.7 A Classification Experiment . 130 7.8 Discussion and Future Work . 132 xi 8 Conclusions and Future Work 133 8.1 Conclusions . 133 8.2 Implications and Final Remarks . 136 Bibliography 139 A The Contents of the Annotated Corpus 149 B The Set of Global Features 153 Summary 161 Samenvatting 163 Curriculum Vitæ 165 Acknowledgements 167 xii Preface The research that is presented in this thesis was performed in the context of the WITCHCRAFT project, which had as its aim to develop a content-based re- trieval system for folksong melodies (Wiering et al., 2009). The acronym stands for: What Is Topical In Cultural Heritage: Content-based Retrieval Among Folk- song Tunes. This project was part of the CATCH program (Continuous Access to Cultural Heritage) as NWO project no. 640-003 501. The CATCH program con- sists of a number of projects on digitization of and access to cultural heritage. Each of these projects is a collaboration between a research institute, such as a university, and a cultural heritage institute, such as a museum or a national archive. In each project techniques from Information Science are employed to improve accessibility of data and collections from the cultural heritage insti- tutes. The program is founded by The Netherlands Organization for Scientific Research (NWO). The WITCHCRAFT project was a collaboration between Utrecht University and the Meertens Institute, Utrecht University as research institute and the Meer- tens Institute as cultural heritage institute, which hosts the collection of Dutch folk songs for which retrieval methods had to be developed. The project team was located at the Meertens Institute, which enabled close collaboration with the collection specialists. This project offered me the opportunity to collaborate with both musicologists and computer scientists, which is exactly what I want to do. During my mas- ter projects Electrical Engineering at Delft University of Technology and Musi- cology at Utrecht University, I discovered the enormous potential Information Science has for Musicology. Information Science offers many techniques for knowledge discovery in large amounts of data. To apply.