Music Information Retrieval: Recent Developments and Applications
Total Page:16
File Type:pdf, Size:1020Kb
Music Information Retrieval: Recent Developments and Applications Markus Schedl Johannes Kepler University Linz, Austria [email protected] Emilia Gómez Universitat Pompeu Fabra, Barcelona, Spain [email protected] Julián Urbano Universitat Pompeu Fabra, Barcelona, Spain [email protected] Boston — Delft Foundations and TrendsR in Information Retrieval Published, sold and distributed by: now Publishers Inc. PO Box 1024 Hanover, MA 02339 United States Tel. +1-781-985-4510 www.nowpublishers.com [email protected] Outside North America: now Publishers Inc. PO Box 179 2600 AD Delft The Netherlands Tel. +31-6-51115274 The preferred citation for this publication is M. Schedl , E. Gómez and J. Urbano. Music Information Retrieval: Recent Developments and Applications. Foundations and TrendsR in Information Retrieval, vol. 8, no. 2-3, pp. 127–261, 2014. R This Foundations and Trends issue was typeset in LATEX using a class file designed by Neal Parikh. Printed on acid-free paper. ISBN: 978-1-60198-807-2 c 2014 M. Schedl , E. Gómez and J. Urbano All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, mechanical, photocopying, recording or otherwise, without prior written permission of the publishers. Photocopying. In the USA: This journal is registered at the Copyright Clearance Cen- ter, Inc., 222 Rosewood Drive, Danvers, MA 01923. Authorization to photocopy items for internal or personal use, or the internal or personal use of specific clients, is granted by now Publishers Inc for users registered with the Copyright Clearance Center (CCC). The ‘services’ for users can be found on the internet at: www.copyright.com For those organizations that have been granted a photocopy license, a separate system of payment has been arranged. Authorization does not extend to other kinds of copy- ing, such as that for general distribution, for advertising or promotional purposes, for creating new collective works, or for resale. In the rest of the world: Permission to pho- tocopy must be obtained from the copyright owner. Please apply to now Publishers Inc., PO Box 1024, Hanover, MA 02339, USA; Tel. +1 781 871 0245; www.nowpublishers.com; [email protected] now Publishers Inc. has an exclusive license to publish this material worldwide. Permission to use this content must be obtained from the copyright license holder. Please apply to now Publishers, PO Box 179, 2600 AD Delft, The Netherlands, www.nowpublishers.com; e-mail: [email protected] Foundations and TrendsR in Information Retrieval Volume 8, Issue 2-3, 2014 Editorial Board Editors-in-Chief Douglas W. Oard Mark Sanderson University of Maryland Royal Melbourne Institute of Technology United States Australia Editors Alan Smeaton Justin Zobel Dublin City University University of Melbourne Bruce Croft Maarten de Rijke University of Massachusetts, Amherst University of Amsterdam Charles L.A. Clarke Norbert Fuhr University of Waterloo University of Duisburg-Essen Fabrizio Sebastiani Soumen Chakrabarti Italian National Research Council Indian Institute of Technology Bombay Ian Ruthven Susan Dumais University of Strathclyde Microsoft Research James Allan Tat-Seng Chua University of Massachusetts, Amherst National University of Singapore Jamie Callan William W. Cohen Carnegie Mellon University Carnegie Mellon University Jian-Yun Nie University of Montreal Editorial Scope Topics Foundations and TrendsR in Information Retrieval publishes survey and tutorial articles in the following topics: • Applications of IR • Metasearch, rank aggregation, • Architectures for IR and data fusion • Collaborative filtering and • Natural language processing recommender systems for IR • Cross-lingual and multilingual • Performance issues for IR IR systems, including algorithms, • Distributed IR and federated data structures, optimization search techniques, and scalability • Evaluation issues and test • Question answering collections for IR • Summarization of single • Formal models and language documents, multiple models for IR documents, and corpora • IR on mobile platforms • Text mining • Indexing and retrieval of • Topic detection and tracking structured documents • • Information categorization and Usability, interactivity, and clustering visualization issues in IR • Information extraction • User modelling and user studies for IR • Information filtering and routing • Web search Information for Librarians Foundations and TrendsR in Information Retrieval, 2014, Volume 8, 5 issues. ISSN paper version 1554-0669. ISSN online version 1554-0677. Also available as a combined paper and online subscription. Foundations and TrendsR in Information Retrieval Vol. 8, No. 2-3 (2014) 127–261 c 2014 M. Schedl , E. Gómez and J. Urbano DOI: 10.1561/1500000042 Music Information Retrieval: Recent Developments and Applications Markus Schedl Johannes Kepler University Linz, Austria [email protected] Emilia Gómez Universitat Pompeu Fabra, Barcelona, Spain [email protected] Julián Urbano Universitat Pompeu Fabra, Barcelona, Spain [email protected] Contents 1 Introduction to Music Information Retrieval 2 1.1 Motivation .......................... 2 1.2 History and evolution .................... 3 1.3 Music modalities and representations ............ 4 1.4 Applications ......................... 6 1.5 Research topics and tasks .................. 15 1.6 Scope and related surveys .................. 15 1.7 Organization of this survey ................. 17 2 Music Content Description and Indexing 19 2.1 Music feature extraction ................... 20 2.2 Music similarity ....................... 40 2.3 Music classification and auto-tagging ............ 44 2.4 Discussion and challenges . ................. 46 3 Context-based Music Description and Indexing 48 3.1 Contextual data sources ................... 49 3.2 Extracting information on music entities .......... 50 3.3 Music similarity based on the Vector Space Model . 55 3.4 Music similarity based on Co-occurrence Analysis ..... 59 3.5 Discussion and challenges . ................. 64 ii iii 4 User Properties and User Context 67 4.1 User studies ......................... 68 4.2 Computational user modeling . .............. 70 4.3 User-adapted music similarity ................ 72 4.4 Semantic labeling via games with a purpose ........ 74 4.5 Music discovery systems based on user preferences . 76 4.6 Discussion and challenges . ................. 78 5 Evaluation in Music Information Retrieval 82 5.1 Why evaluation in Music Information Retrieval is hard . 83 5.2 Evaluation initiatives .................... 87 5.3 Research on Music Information Retrieval evaluation .... 94 5.4 Discussion and challenges . ................. 96 6 Conclusions and Open Challenges 100 Acknowledgements 105 References 106 Abstract We provide a survey of the field of Music Information Retrieval (MIR), in particular paying attention to latest developments, such as seman- tic auto-tagging and user-centric retrieval and recommendation ap- proaches. We first elaborate on well-established and proven methods for feature extraction and music indexing, from both the audio sig- nal and contextual data sources about music items, such as web pages or collaborative tags. These in turn enable a wide variety of music retrieval tasks, such as semantic music search or music identification (“query by example”). Subsequently, we review current work on user analysis and modeling in the context of music recommendation and retrieval, addressing the recent trend towards user-centric and adap- tive approaches and systems. A discussion follows about the important aspect of how various MIR approaches to different problems are eval- uated and compared. Eventually, a discussion about the major open challenges concludes the survey. M. Schedl , E. Gómez and J. Urbano. Music Information Retrieval: Recent Developments and Applications. Foundations and TrendsR in Information Retrieval, vol. 8, no. 2-3, pp. 127–261, 2014. DOI: 10.1561/1500000042. 1 Introduction to Music Information Retrieval 1.1 Motivation Music is a pervasive topic in our society as almost everyone enjoys lis- tening to it and many also create. Broadly speaking, the research field of Music Information Retrieval (MIR) is foremost concerned with the extraction and inference of meaningful features from music (from the audio signal, symbolic representation or external sources such as web pages), indexing of music using these features, and the development of different search and retrieval schemes (for instance, content-based search, music recommendation systems, or user interfaces for browsing large music collections), as defined by Downie [52]. As a consequence, MIR aims at making the world’s vast store of music available to individ- uals [52]. To this end, different representations of music-related subjects (e.g., songwriters, composers, performers, consumer) and items (music pieces, albums, video clips, etc.) are considered. Given the relevance of music in our society, it comes as a surprise that the research field of MIR is a relatively young one, having its origin less than two decades ago. However, since then MIR has experienced a constant upward trend as a research field. Some of the most important reasons for its success are (i) the development of audio compression 2 1.2. History and evolution 3 techniques in the late 1990s, (ii) increasing computing power of personal computers, which in turn enabled users and applications to extract music features in a reasonable time, (iii) the widespread