Clef: An Extensible, Experimental Framework for Music Information Retrieval The Harvard community has made this article openly available. Please share how this access benefits you. Your story matters Citation DeCurtins, Max. 2018. Clef: An Extensible, Experimental Framework for Music Information Retrieval. Master's thesis, Harvard Extension School. Citable link https://nrs.harvard.edu/URN-3:HUL.INSTREPOS:37364558 Terms of Use This article was downloaded from Harvard University’s DASH repository, and is made available under the terms and conditions applicable to Other Posted Material, as set forth at http:// nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of- use#LAA Clef: An Extensible, Experimental Framework for Music Information Retrieval Maxwell Spenser DeCurtins A Thesis in the Field of Information Technology for the Degree of Master of Liberal Arts in Extension Studies Harvard University November 2018 © 2018 Maxwell Spenser DeCurtins Abstract Text-based searching of textual data has long been a mainstay of computing, and as search technology has evolved so has interest in searching non-textual data. In recent years eff orts to use image fi les as queries for other image fi les (or for information about what is in the query image fi le) have profi ted from advances in machine learning, as have other alternative search domains. While searching for music using musical data has met with considerable success in audio sampling software such as Shazam, searching machine-readable, music nota- tion-based data—also known as symbolic music data—using queries written in music notation has lagged behind, with most development in this area geared toward academic music researchers or existing in ad hoc implementations only. Music information retriev- al—the fi eld concerned with developing search techniques for music—requires a frame- work that can move beyond predetermined combinations of algorithms and datasets. The Clef system demonstrates that this is possible using container-based services that communicate with each other over HTTP. Clef off ers an extensible approach to building a musical search engine that allows new algorithms and datasets to be accessed through a consistent, music notation-based user interface for query input. Extending the system with a new container for running a music information retrieval algorithm requires signifi cant development, but once operational, new algorithm containers integrate seam- lessly into the user interface. iii About the Author Max hails from Menlo Park, CA, a town at the heart of Silicon Valley known now worldwide as the home of Facebook. Growing up in this location, in tandem with the evolution of home computing, made for an experience that even among his fellow so- called “Older Millennials” stands out to the author as unique. The author has little doubt that this bac kground has allowed him to prosper personally and professionally. For the last nine years Max has lived in Boston and for the last seven has worked in a continually-evolving information technology role in the Center for Executive and Continuing Professional Education at the Harvard T.H. Chan School of Public Health. Despite the author’s Silicon Valley origins, he lacked any formal computer science edu- cation prior to his coursework at Harvard; Max had originally planned to pursue a career as a professor of music. Needless to say, music has remained a serious component of the author’s life despite his shift to a career as a professional software engineer. Max holds a Bachelor of Arts with majors in Music and Linguistics from the Uni- versity of California at Santa Barbara and a Master of Music in Musicology from Boston University’s School of Music. iv Acknowledgments This project represents the culmination of graduate work in software engineering begun—without much advance planning—in earnest in the fall of 2013. Completing a master’s degree on a part-time basis while simultaneously holding a full-time job requires no small amount of persistence and sacrifi ce, but like so many other endeavors, it also requires the help of other individuals. I must fi rst and foremost acknowledge Dr. Michael Scott Cuthbert for his kind help and knowledgeable guidance as thesis director for this project. Given the esoteric nature of the fi eld of music information retrieval, I count myself extremely lucky to have been able to work with one of its leading voices just down the street, at MIT. For her help facilitating introductions with Dr. Cuthbert (technically, a re-introduction), I am deeply grateful to Shannon Rose McAuliff e. I would also like to thank Dr. Amy Marie Carleton for guiding the initial stages of this project during the transitional time following the retirement of Dr. Jeff Parker. Several of the courses I completed during the course of this degree have proven especially useful to me and stand out as highlights of my time at Harvard. I wish to thank Bruce Molay, Charles Sawyer, and Dr. David Sullivan for these engaging and valuable classes. Though I do not know him personally, I must acknowledge Dr. Donald Byrd and his extremely valuable bibliography in music information retrieval, which made it possi- ble to review existing research with exceptional effi ciency and to jump-start the process of developing the current project. v I could not have reached this point were it not for the support of my workplace, the Center for Executive and Continuing Professional Education (ECPE) at the Harvard T.H. Chan School of Public Health, and in particular the support of Erik Jespersen, Mat- thew Denault, and Charles Mapps. Without a doubt the fl exibility of my role at ECPE—to say nothing of the tuition benefi t available to me as a member of the staff of Harvard University—made possible the pursuit and completion not only of this thesis project but of the ALM degree itself. For his tireless encouragement of my education in software development over many years, including more recently many stimulating conversations related to this project, I wish to thank my longtime friend Alex Broadwin; arguably no single person has had a greater impact on my growth as a developer. And to Amelia Drace, who casually asked about “googling for music using mu- sic” in the car one day on the way to San Francisco many years ago: thanks for putting this crazy idea in my head in the fi rst place. vi Contents Abstract .............................................................................................................................. iii About the Author ................................................................................................................ iv Acknowledgments ................................................................................................................v 1 Introduction ....................................................................................................................1 What is Music Information Retrieval? .....................................................................1 Metadata-focused MIR ................................................................................3 Audio Stream-focused MIR .........................................................................3 Notation-focused MIR ................................................................................4 Toward a Musical Search Engine ............................................................................4 A World of Possibilities .............................................................................10 Organization of the Rest of this Thesis ..................................................................11 2 Music Information Retrieval: History, Challenges, Techniques, and Implementations ..............................................................................................14 Challenges of Music Information Retrieval ...........................................................17 A Survey of MIR Techniques for Symbolic Music Data .......................................22 String Techniques for Music Information Retrieval ..................................23 Limitations of String Techniques: An Example .........................................28 Geometric Techniques for Music Information Retrieval ...........................30 Other Techniques .......................................................................................32 Developed Implementations ..................................................................................32 Themefi nder ...............................................................................................32 vii Musipedia ...................................................................................................35 MELDEX ...................................................................................................37 MelodicMatch ............................................................................................37 The Josquin Research Project ....................................................................39 Peachnote ...................................................................................................40 Implementations under Development ....................................................................42 Single Interface for Music Searching and Score Analysis .........................42 Summary ................................................................................................................43 3 The Design of Clef .......................................................................................................45