DOCSLIB.ORG

Contextual Music Information Retrieval and Recommendation: State of the Art and Challenges

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Contextual Music Information Retrieval and Recommendation: State of the Art and Challenges COMPUTERSCIENCEREVIEW ( ) ± Available online at www.sciencedirect.com journal homepage: www.elsevier.com/locate/cosrev Survey Contextual music information retrieval and recommendation: State of the art and challenges Marius Kaminskas∗, Francesco Ricci Faculty of Computer Science, Free University of Bozen­Bolzano, Piazza Domenicani 3, 39100 Bolzano, Italy ARTICLEINFO ABSTRACT Article history: Increasing amount of online music content has opened new opportunities for Received 15 September 2011 implementing new effective information access services – commonly known as music Received in revised form recommender systems – that support music navigation, discovery, sharing, and formation 30 March 2012 of user communities. In the recent years a new research area of contextual (or situational) Accepted 7 April 2012 music recommendation and retrieval has emerged. The basic idea is to retrieve and suggest music depending on the user’s actual situation, for instance emotional state, or any other Keywords: contextual conditions that might influence the user’s perception of music. Despite the Music information retrieval high potential of such idea, the development of real­world applications that retrieve or Music recommender systems recommend music depending on the user’s context is still in its early stages. This survey Context­aware services illustrates various tools and techniques that can be used for addressing the research Affective computing challenges posed by context­aware music retrieval and recommendation. This survey Social computing covers a broad range of topics, starting from classical music information retrieval (MIR) and recommender system (RS) techniques, and then focusing on context­aware music applications as well as the newer trends of affective and social computing applied to the music domain. ⃝c 2012 Elsevier Inc. All rights reserved. Contents 1. Introduction .................................................................................................................................................................................2 2. Content­based music information retrieval..................................................................................................................................3 2.1. Query by example..............................................................................................................................................................3 2.2. Query by humming............................................................................................................................................................4 2.3. Genre classification............................................................................................................................................................4 2.4. Multimodal analysis in music information retrieval ..........................................................................................................5 2.5. Summary ...........................................................................................................................................................................6 3. Music recommendation................................................................................................................................................................6 3.1. Collaborative filtering ........................................................................................................................................................6 3.1.1. General techniques...............................................................................................................................................7 3.1.2. Applications in the music domain........................................................................................................................7 ∗ Corresponding author. E­mail addresses: [email protected] (M. Kaminskas), [email protected] (F. Ricci). 1574­0137/$ ­ see front matter ⃝c 2012 Elsevier Inc. All rights reserved. doi:10.1016/j.cosrev.2012.04.002 Please cite this article in press as: M. Kaminskas, F. Ricci, Contextual music information retrieval and recommendation: State of the art and challenges, Computer Science Review (2012), doi:10.1016/j.cosrev.2012.04.002 2 COMPUTERSCIENCEREVIEW ( ) ± 3.1.3. Limitations ...........................................................................................................................................................8 3.2. Content­based approach....................................................................................................................................................8 3.2.1. General techniques...............................................................................................................................................8 3.2.2. Applications in the music domain........................................................................................................................9 3.2.3. Limitations ...........................................................................................................................................................9 3.3. Hybrid approach ................................................................................................................................................................ 10 3.3.1. General techniques............................................................................................................................................... 10 3.3.2. Applications in the music domain........................................................................................................................ 10 3.4. Commercial music recommenders .................................................................................................................................... 11 3.4.1. Collaborative­based systems ................................................................................................................................ 11 3.4.2. Content­based systems ........................................................................................................................................ 11 3.5. Summary ........................................................................................................................................................................... 11 4. Contextual and social music retrieval and recommendation........................................................................................................ 12 4.1. Contextual music recommendation and retrieval.............................................................................................................. 12 4.1.1. Environment­related context................................................................................................................................ 13 4.1.2. User­related context ............................................................................................................................................. 16 4.1.3. Multimedia context .............................................................................................................................................. 17 4.1.4. Summary.............................................................................................................................................................. 18 4.2. Emotion recognition in music............................................................................................................................................ 18 4.2.1. Emotion models for music cognition.................................................................................................................... 18 4.2.2. Machine learning approaches to emotion recognition in music ........................................................................... 19 4.2.3. Summary.............................................................................................................................................................. 21 4.3. Music and the social web................................................................................................................................................... 21 4.3.1. Tag acquisition ..................................................................................................................................................... 22 4.3.2. Tag usage for music recommendation and retrieval............................................................................................. 25 4.3.3. Summary.............................................................................................................................................................. 26 5. Conclusions.................................................................................................................................................................................. 26 References .................................................................................................................................................................................... 27 1. Introduction user’s context, e.g., her mood, or her current location and activity [2]. In fact, a study on users’ musical information Music has always played a major role in human enter­ needs [3] showed that people often seek music for a certain tainment. With the coming of digital music and Internet occasion, event, or an emotional state. Moreover, the authors technologies, a huge amount of music content has become of a similar study [4] concluded that there
Load more

Recommended publications
  • Automatic Music Transcription: an Overview Emmanouil Benetos Member, IEEE, Simon Dixon, Zhiyao Duan Member, IEEE, and Sebastian Ewert Member, IEEE
    1 Automatic Music Transcription: An Overview Emmanouil Benetos Member, IEEE, Simon Dixon, Zhiyao Duan Member, IEEE, and Sebastian Ewert Member, IEEE I. INTRODUCTION IV-F, as well as methods for transcribing specific sources The capability of transcribing music audio into music within a polyphonic mixture such as melody and bass line. notation is a fascinating example of human intelligence. It involves perception (analyzing complex auditory scenes), cog- A. Applications & Impact nition (recognizing musical objects), knowledge representation A successful AMT system would enable a broad range (forming musical structures) and inference (testing alternative of interactions between people and music, including music hypotheses). Automatic Music Transcription (AMT), i.e., the education (e.g., through systems for automatic instrument design of computational algorithms to convert acoustic music tutoring), music creation (e.g., dictating improvised musical signals into some form of music notation, is a challenging task ideas and automatic music accompaniment), music production in signal processing and artificial intelligence. It comprises (e.g., music content visualization and intelligent content-based several subtasks, including (multi-)pitch estimation, onset and editing), music search (e.g., indexing and recommendation of offset detection, instrument recognition, beat and rhythm track- music by melody, bass, rhythm or chord progression), and ing, interpretation of expressive timing and dynamics, and musicology (e.g., analyzing jazz improvisations and other non- score typesetting. Given the number of subtasks it comprises notated music). As such, AMT is an enabling technology with and its wide application range, it is considered a fundamental clear potential for both economic and societal impact. problem in the fields of music signal processing and music AMT is closely related to other music signal processing information retrieval (MIR) [1], [2].
    [Show full text]
  • Real-Time Programming and Processing of Music Signals Arshia Cont
    Real-time Programming and Processing of Music Signals Arshia Cont To cite this version: Arshia Cont. Real-time Programming and Processing of Music Signals. Sound [cs.SD]. Université Pierre et Marie Curie - Paris VI, 2013. tel-00829771 HAL Id: tel-00829771 https://tel.archives-ouvertes.fr/tel-00829771 Submitted on 3 Jun 2013 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Realtime Programming & Processing of Music Signals by ARSHIA CONT Ircam-CNRS-UPMC Mixed Research Unit MuTant Team-Project (INRIA) Musical Representations Team, Ircam-Centre Pompidou 1 Place Igor Stravinsky, 75004 Paris, France. Habilitation à diriger la recherche Defended on May 30th in front of the jury composed of: Gérard Berry Collège de France Professor Roger Dannanberg Carnegie Mellon University Professor Carlos Agon UPMC - Ircam Professor François Pachet Sony CSL Senior Researcher Miller Puckette UCSD Professor Marco Stroppa Composer ii à Marie le sel de ma vie iv CONTENTS 1. Introduction1 1.1. Synthetic Summary .................. 1 1.2. Publication List 2007-2012 ................ 3 1.3. Research Advising Summary ............... 5 2. Realtime Machine Listening7 2.1. Automatic Transcription................. 7 2.2. Automatic Alignment .................. 10 2.2.1.
    [Show full text]
  • 2007–09 Program Requirements (.Pdf)
    1 FOREWORD F ROM THE DEAN http://www.yorku.ca/grads/calendar/ Graduate study involves a level of engagement with subject matter, in Business, Law, Education, Translation and Social Work and in fellow students, and faculty members that marks a high point in health-related disciplines focused through York’s new Faculty of one’s intellectual and creative development. At the master’s and Health. Innovative and unique interdisciplinary programs have Doctoral levels, graduate study in one way or another is at the centre been created in such areas as Environmental Studies, Earth & Space of research and scholarly intensity within the University and provides Science, Social & Political Thought, Interdisciplinary Studies, exciting challenges and opportunities. Women’s Studies, and our most recent programs: Humanities, Human Resources Management, and Critical Studies in Disability. Since its inception in 1963, the Faculty of Graduate Studies has A further innovative dimension has involved the creation of a grown from 11 students in a single graduate program to more number of specialized graduate diplomas—such as Early Childhood than 5000 students in 46 programs. York’s graduate studies are Education, and Environmental/Sustainability Education—which expanding, with five new graduate programs in development, three may be earned concurrently with the master’s or Doctoral degree of which begin this year; 11 more programs are expanding, either in several programs, and which may also be taken as stand-alone adding a doctoral program where there is an existing master’s, graduate diplomas. York offers 32 graduate diplomas. The Faculty or adding new fields or different master’s programs.
    [Show full text]
  • Movie Genome: Alleviating New Item Cold Start in Movie Recommendation
    User Modeling and User-Adapted Interaction https://doi.org/10.1007/s11257-019-09221-y Movie genome: alleviating new item cold start in movie recommendation Yashar Deldjoo, et al. [full author details at the end of the article] Received: 23 December 2017 / Accepted in revised form: 19 January 2019 © The Author(s) 2019 Abstract As of today, most movie recommendation services base their recommendations on col- laborative filtering (CF) and/or content-based filtering (CBF) models that use metadata (e.g., genre or cast). In most video-on-demand and streaming services, however, new movies and TV series are continuously added. CF models are unable to make predic- tions in such a scenario, since the newly added videos lack interactions—a problem technically known as new item cold start (CS). Currently, the most common approach to this problem is to switch to a purely CBF method, usually by exploiting textual meta- data. This approach is known to have lower accuracy than CF because it ignores useful collaborative information and relies on human-generated textual metadata, which are expensive to collect and often prone to errors. User-generated content, such as tags, can also be rare or absent in CS situations. In this paper, we introduce a new movie recommender system that addresses the new item problem in the movie domain by (i) integrating state-of-the-art audio and visual descriptors, which can be automati- cally extracted from video content and constitute what we call the movie genome; (ii) exploiting an effective data fusion method named canonical correlation analysis, which was successfully tested in our previous works Deldjoo et al.
    [Show full text]
  • Music Similarity: Learning Algorithms and Applications
    UNIVERSITY OF CALIFORNIA, SAN DIEGO More like this: machine learning approaches to music similarity A dissertation submitted in partial satisfaction of the requirements for the degree Doctor of Philosophy in Computer Science by Brian McFee Committee in charge: Professor Sanjoy Dasgupta, Co-Chair Professor Gert Lanckriet, Co-Chair Professor Serge Belongie Professor Lawrence Saul Professor Nuno Vasconcelos 2012 Copyright Brian McFee, 2012 All rights reserved. The dissertation of Brian McFee is approved, and it is ac- ceptable in quality and form for publication on microfilm and electronically: Co-Chair Co-Chair University of California, San Diego 2012 iii DEDICATION To my parents. Thanks for the genes, and everything since. iv EPIGRAPH I’m gonna hear my favorite song, if it takes all night.1 Frank Black, “If It Takes All Night.” 1Clearly, the author is lamenting the inefficiencies of broadcast radio programming. v TABLE OF CONTENTS Signature Page................................... iii Dedication...................................... iv Epigraph.......................................v Table of Contents.................................. vi List of Figures....................................x List of Tables.................................... xi Acknowledgements................................. xii Vita......................................... xiv Abstract of the Dissertation............................. xvi Chapter 1 Introduction.............................1 1.1 Music information retrieval..................1 1.2 Summary of contributions..................1
    [Show full text]
  • Intelligent User Interfaces for Music Discovery
    Knees, P., et al. (2020). Intelligent User Interfaces for Music Discovery. Transactions of the International Society for Music Information 7,60,5 Retrieval, 3(1), pp. 165–179. DOI: https://doi.org/10.5334/tismir.60 OVERVIEW ARTICLE Intelligent User Interfaces for Music Discovery Peter Knees*, Markus Schedl† and Masataka Goto‡ Assisting the user in finding music is one of the original motivations that led to the establishment of Music Information Retrieval (MIR) as a research field. This encompasses classic Information Retrieval inspired access to music repositories that aims at meeting an information need of an expert user. Beyond this, however, music as a cultural art form is also connected to an entertainment need of potential listeners, requiring more intuitive and engaging means for music discovery. A central aspect in this process is the user interface. In this article, we reflect on the evolution of MIR-driven intelligent user interfaces for music browsing and discovery over the past two decades. We argue that three major developments have transformed and shaped user interfaces during this period, each connected to a phase of new listening practices. Phase 1 has seen the development of content-based music retrieval interfaces built upon audio processing and content description algorithms facilitating the automatic organization of repositories and finding music according to sound qualities. These interfaces are primarily connected to personal music collections or (still) small commercial catalogs. Phase 2 comprises interfaces incorporating collaborative and automatic semantic description of music, exploiting knowledge captured in user-generated metadata. These interfaces are connected to collective web platforms. Phase 3 is dominated by recommender systems built upon the collection of online music interaction traces on a large scale.
    [Show full text]
  • Recommender Systems User-Facing Decision Support Systems
    Recommender Systems User-Facing Decision Support Systems Michael Hahsler Intelligent Data Analysis Lab (IDA@SMU) CSE, Lyle School of Engineering Southern Methodist University EMIS 5/7357: Decision Support Systems February 22, 2012 Michael Hahsler (IDA@SMU) Recommender Systems EMIS 5/7357: DSS 1 / 44 Michael Hahsler (IDA@SMU) Recommender Systems EMIS 5/7357: DSS 2 / 44 Michael Hahsler (IDA@SMU) Recommender Systems EMIS 5/7357: DSS 3 / 44 Michael Hahsler (IDA@SMU) Recommender Systems EMIS 5/7357: DSS 4 / 44 Table of Contents 1 Recommender Systems & DSS 2 Content-based Approach 3 Collaborative Filtering (CF) Memory-based CF Model-based CF 4 Strategies for the Cold Start Problem 5 Open-Source Implementations 6 Example: recommenderlab for R 7 Empirical Evidence Michael Hahsler (IDA@SMU) Recommender Systems EMIS 5/7357: DSS 5 / 44 Decision Support Systems ? Michael Hahsler (IDA@SMU) Recommender Systems EMIS 5/7357: DSS 6 / 44 Decision Support Systems \Decision Support Systems are defined broadly [...] as interactive computer-based systems that help people use computer communications, data, documents, knowledge, and models to solve problems and make decisions." Power (2002) Main Components: Knowledge base Model User interface Michael Hahsler (IDA@SMU) Recommender Systems EMIS 5/7357: DSS 6 / 44 ! A recommender system is a decision support systems which help a seller to choose suitable items to offer given a limited information channel. Recommender Systems Recommender systems apply statistical and knowledge discovery techniques to the problem of making product recommendations. Sarwar et al. (2000) Advantages of recommender systems (Schafer et al., 2001): Improve conversion rate: Help customers find a product she/he wants to buy.
    [Show full text]
  • Pandora's Music Recommender 1 Introduction to Pandora 2 Description of Pandora Problem Space
    Pandora’s Music Recommender Michael Howe 1 Introduction to Pandora One of the great promises of the internet and Web 2.0 is the opportunity to expose people to new types of content. Companies like Amazon and Netflix provide customers with ideas for new items to purchase based on current or previous selections. For instance, someone who rented “Star Wars” at Netflix might be presented with “The Matrix” as another movie to rent. The challenge in this strategy is to make suggestions in a reasonable amount of time that the user will mostly like based on the known list of what the user already enjoys. People will only pay attention to the recommendations of a service a finite number of times before they lose trust. Repeatedly suggest content that the user hates and the user will look for new content elsewhere. Also, a user will only pay attention to a service’s recommendation if they arrive in a reasonable amount of time. Make the user wait longer than they are willing and they will again turn elsewhere for content suggestions. Accuracy and speed are critical to a service’s success. Pandora exposes people to music with an online radio station where a user builds up “stations” based on musical interests. The user indicates in each station one or more songs or artists that he or she likes. Based on these preferences Pandora plays similar songs that the user might also like. As Pandora plays the user can further refine the station by giving a “thumbs up” or a “thumbs down” to a particular song.
    [Show full text]
  • Music Recommendation Algorithms: Discovering Weekly Or Discovering
    MUSIC RECOMMENDATION ALGORITHMS Music Recommendation Algorithms: Discovering Weekly or Discovering Weakly? Jennie Silber Muhlenberg College, Allentown, PA, USA Media & Communication Department Undergraduate High Honors Thesis April 1, 2019 1 MUSIC RECOMMENDATION ALGORITHMS 2 MUSIC RECOMMENDATION ALGORITHMS Abstract This thesis analyzes and assesses the cultural impact and economic viability that the top music streaming platforms have on the consumption and discovery of music, with a specific focus on recommendation algorithms. Through the support of scholarly and journalistic research as well as my own user experience, I evaluate the known constructs that fuel algorithmic recommendations, but also make educated inferences about the variables concealed from public knowledge. One of the most significant variables delineated throughout this thesis is the power held by human curators and the way they interact with algorithms to frame and legitimize content. Additionally, I execute my own experiment by creating new user profiles on the two streaming platforms popularly used for the purpose of discovery, Spotify and SoundCloud, and record each step of the music discovery process experienced by a new user. After listening to an equal representation of all genre categories within each platform, I then compare the genre, release year, artist status, and content promotion gathered from my listening history to the algorithmically-generated songs listed in my ‘Discover Weekly’ and ‘SoundCloud Weekly’ personalized playlists. The results from this experiment demonstrate that the recommendation algorithms that power these discovery playlists intrinsically facilitate the perpetuation of a star- driven, “winner-take-all” marketplace, where new, popular, trendy, music is favored, despite how diverse of a selection the music being listened to is.
    [Show full text]
  • UNIVERSITY of CALIFORNIA, SAN DIEGO Bag-Of-Concepts As a Movie
    UNIVERSITY OF CALIFORNIA, SAN DIEGO Bag-of-Concepts as a Movie Genome and Representation A Thesis submitted in partial satisfaction of the requirements for the degree Master of Science in Computer Science by Colin Zhou Committee in charge: Professor Shlomo Dubnov, Chair Professor Sanjoy Dasgupta Professor Lawrence Saul 2016 The Thesis of Colin Zhou is approved, and it is acceptable in quality and form for publication on microfilm and electronically: Chair University of California, San Diego 2016 iii DEDICATION For Mom and Dad iv TABLE OF CONTENTS Signature Page........................................................................................................... iii Dedication.................................................................................................................. iv Table of Contents....................................................................................................... v List of Figures and Tables.......................................................................................... vi Abstract of the Thesis................................................................................................ vii Introduction................................................................................................................ 1 Chapter 1 Background............................................................................................... 3 1.1 Vector Space Models............................................................................... 3 1.1.1. Bag-of-Words........................................................................
    [Show full text]
  • A Movie Recommender System: MOVREC Using Machine Learning Techniques Ashrita Kashyap1, Sunita
    ISSN 2321 3361 © 2020 IJESC Research Article Volume 10 Issue No.6 A Movie Recommender System: MOVREC using Machine Learning Techniques Ashrita Kashyap1, Sunita. B2, Sneh Srivastava3, Aishwarya. PH4, Anup Jung Shah5 Department of Computer Science & Engineering SAIT, Bengaluru, Karnataka, India Abstract: This paper discuss about recommendations of the movies. A movie recommendation is important in our social life due to its strength in providing enhanced entertainment. Such a system can suggest a set of movies to users based on their interest, or the popularities of the movies. A recommendation system is used for the purpose of suggesting items to purchase or to see. They direct users towards those items which can meet their needs through cutting down large database of Information. A recommender system, or a recommendation system (sometimes replacing 'system' with a synonym such as platform or engine), is a subclass of information filtering system that seeks to predict the "rating" or "preference" a user would give to an item[1][2]. They are primarily used in commercial applications. MOVREC also help users to find the movies of their choices based on the movie experience of other users in efficient and effective manner without wasting much time in useless browsing. Keywords: Filtering, Recommendation System, Recommender. I. INTRODUCTION emphasizing other attributes when a user "likes" a song. This is an example of a content-based approach. Recommender systems usually make use of either or both collaborative filtering and content-based filtering (also known as Each type of system has its strengths and weaknesses. In the the personality-based approach),[3] as well as other systems above example, Last.fm requires a large amount of information such as knowledge-based systems.
    [Show full text]
  • Five Approaches to Collecting Tags for Music
    FIVE APPROACHES TO COLLECTING TAGS FOR MUSIC Douglas Turnbull Luke Barrington Gert Lanckriet UC San Diego UC San Diego UC San Diego [email protected] [email protected] [email protected] ABSTRACT There are a number of key concepts to consider when com- paring these approaches. The cold start problem refers to the We compare five approaches to collecting tags for music: fact songs that are not annotated cannot be retrieved. This conducting a survey, harvesting social tags, deploying anno- problem is related to popularity bias in that popular songs (in tation games, mining web documents, and autotagging audio the short-head) tend to be annotated more thoroughly than content. The comparison includes a discussion of both scala- unpopular songs (in the long-tail)[11]. This often leads bility (financial cost, human involvement, and computational to a situation in which a short-head song is ranked above resources) and quality (the cold start problem & popularity a long-tail song despite the fact that the long-tail song may bias, strong vs. weak labeling, vocabulary structure & size, be more semantically relevant. We prefer an approach that and annotation accuracy). We then describe one state-of- avoids the cold start problem (e.g., autotagging). If this is the-art system for each approach. The performance of each not possible, we prefer approaches in which we can explic- system is evaluated using a tag-based music information itly control which songs are annotated (e.g., survey, games), retrieval task. Using this task, we are able to quantify the rather than an approach in which only the more popular songs effect of popularity bias on each approach by making use are annotated (e.g., social tags, web documents).
    [Show full text]