D3.5 .1 Music Collection Structuring and Navigation Module Abstract The goal of Work Package 3 is to take the features and meta- data provided by Work Package 2 and provide the technology needed for the intelligent structuring, presentation, and use of large music collections. This deliverable describes 4 approaches to make music collections more accessible. For each approach a demon- stration is implemented. The approaches use different similarity measures (audio-based, web-based), use different types of organizations (fuzzy hierarchical, linear, etc.), and assume different usage scenarios (from active exploration to passive consumption). Version 1.0 Date: March 2006 Editor: E. Pampalk Contributors: E. Pampalk, T. Pohle, J. Petrak, S. Dixon, F. Gouyon, and A. Flexer Reviewers: G. Widmer, P. Herrera, J. Bello, D. Wallace, F. Vignoli Contents 1 Introduction 1 2 Audio-based Organization of Music Collections 5 2.1 Introduction ............................ 5 2.2 Related Work ........................... 7 2.3 The Self-Organizing Map ..................... 8 2.4 Visualizations ............................ 10 2.4.1 Smoothed Data Histograms ................ 10 2.4.2 Weather Charts and Other Visualizations ......... 11 2.5 Aligned Self-Organizing Maps ................... 11 2.5.1 Details ........................... 14 2.5.2 Illustration ......................... 16 2.6 Browsing Different Views (Demonstration) ............ 16 2.7 Conclusions ............................. 21 3 Web-based Fuzzy Hierarchical Structuring of Music Collec- tions at the Artist Level 23 3.1 Introduction ............................ 23 3.2 Background ............................ 24 3.3 Hierarchical Clustering ....................... 29 3.4 Term Selection for Cluster Description .............. 31 3.4.1 Techniques ......................... 32 3.4.2 Domain-Specific Dictionary ................ 34 3.5 Results and Discussion ....................... 35 3.5.1 User Interface ....................... 35 3.5.2 Comparison of Term Selection Techniques ........ 36 3.5.3 Discussion ......................... 36 3.6 Conclusions ............................. 38 4 Navigation via Feedback: Dynamic Playlist Generation 39 4.1 Introduction ............................ 39 4.2 Related Work ........................... 40 4.3 Method ............................... 41 4.4 Evaluation ............................. 42 4.4.1 Procedure (Hypothetical Use Cases) ........... 43 4.4.2 Data ............................ 44 4.4.3 Results ........................... 44 4.5 Implementation and Examples ................... 49 4.6 Conclusions ............................. 51 5 Linear Organization of Music Collections and a New Interface 53 5.1 Introduction ............................ 53 5.2 Literature Review ......................... 54 5.3 Approach .............................. 55 5.4 Algorithms ............................. 55 5.4.1 General Remarks ...................... 56 5.4.2 Domain-Specific Issues ................... 56 5.4.3 Evaluated Algorithms ................... 56 5.5 Evaluation and Results ....................... 58 5.5.1 Data Set .......................... 59 5.5.2 Subjective Evaluation ................... 59 5.5.3 Runtime and Route Length ................ 59 5.5.4 Fluctuations Between Genres ............... 60 5.5.5 Shannon entropy ...................... 60 5.5.6 Long-Term Consistency .................. 61 5.5.7 Artist Filter ........................ 62 5.6 Summary and Discussion ...................... 63 5.7 Conclusions and Future Work ................... 65 6 Conclusions 67 Chapter 1 Introduction This deliverable is part of WP3. The goal of workpackage 3 (WP3) is to take the features and meta-data provided by workpackage 2 (WP2) and provide the technology needed for the intelligent structuring, presentation, and use (query processing and retrieval) of large music collections. This general goal can be broken down into two major task groups: the automatic structuring and organi- sation of large collections of digital music, and intelligent music retrieval in such structured “music spaces”. According to the Technical Annex the specific target of this deliverable is: “Developing methods for automatic (hierarchical) structuring of large music col- lections based on pertinent similarity criteria, and novel (aural and/or graphical) strategies for presentation and navigation in structured music archives.” This deliverable directly uses the outcomes of D341. In particular, similarity measures are the core technology of each of the applications described here. In addition to D341, the updated results summarized in [51] are relevant for this deliverable. Music Similarity Measures There are a number of applications and services for music currently available (e.g, pandora.com, last.fm, Amazon, iTunes, Yahoo! Shoutcast and many more). However, none of these use content-based similarity measures. Instead they rely on expert ratings, or collaborative filtering techniques. Within SIMAC a strong focus has been on the development of alternative approaches. These can complement existing approaches and support applications which would not be feasible otherwise. In particular, within SIMAC audio-based approaches and web-based approaches were developed. This deliverable specifically focuses on applications of these similarity measures. Structuring, Navigation, and Playlist Generation At the outset of SIMAC the intention was to develop interfaces to support active exploration of music collections. Two chapters in this deliverable deal with this. However, there are also two chapters which assume “lazy” users who do not wish to select each piece they want to listen to individually. In particular, these two chapters deal with playlists; how they can be dynamically updated (and improved) based on minimum user feedback, and how they can be initialized as 2 1 Introduction simply as turning the tuner on a radio. These interfaces support the exploration and navigation of music collections while at the same time leaning back and enjoying listening to music. Main Outcomes of this Deliverable This deliverable describes four applications of computational models of music similarity which are based on similarity measures and have been developed as part of WP3. The first application (Chapter 2) is the organization and visu- alization of music collections using a metaphor of geographic maps [54]. The second application (Chapter 3) is the organization of music collections at the artist level. In contrast to the first application a fuzzy hierarchy is used to struc- ture the collection and automatic summaries are created based on words found on web pages containing the artist’s name [58; 55]. The third application (Chap- ter 4) is the dynamic generation of playlists. Feedback given by the users by pushing a skip button is used to dynamically improve a playlist [59]. The fourth application demonstrates an alternative approach to playlist generation [66]. Us- ing a traveling salesman algorithm the shortest path through the whole collection is determined. The path is mapped to a wheel interface. This wheel allows the user to quickly “zap” through the music collection. Similar pieces are located close to each other on the wheel. Classification of Applications There are various applications for music similarity measures. Applications include simple retrieval, recommendation based on user profiles, exploration, and music analysis. Each application assumes specific usage scenarios in which the user plays the central role. There are several categories into which applications can be classified. One is how much interaction they require. Roughly, there are two types. One type allows active exploration to discover new music or new insights into music. The other type is intended for more or less passive consumption of music. The first two applications in this deliverable belong to the first category. The third application belongs to the second category. Another category into which applications can be classified is the users’ fa- miliarity with the music. For example, does the application support the users in discovering something new? Or do they help the users manage their own (and thus to some extend familiar) collections? In this context an interesting ques- tion is also the size of the collections the applications are designed for. Some applications might work best for sizes of a few hundred pieces while others can (theoretically) deal with a few million. 3 User Size of the User’s Familiarity Type of Chapter Interaction Collection with the Music Similarity Level 2 active small any audio track 3 active very large unfamiliar web artist 4 & 5 passive large familiar audio track Table 1.1: Rough categorization of applications in this deliverable. Another difference is the level which the applications work in. In this deliver- able there are two levels, namely, the artist level and the track level. Some ap- proaches can only work at the artist level (e.g. the web-based approach described in deliverable D341). Other approaches, such as the audio-based similarity can deal with any level (track, album, artist). In general, the type of similarity used (e.g. audio or web) also has a strong impact on the application and its usage scenarios. For example, audio-based similarity cannot be used to rate the quality of music, and thus requires additional precautions when using it to discover new music (to avoid flooding the user with low quality music). This deliverable primarily serves to describe some ideas on how similarity measures can be applied. The applications