User Based Hybrid Algorithms for Music Recommendation Systems By Murtadha Sami Luaibi Al-Maliki The thesis is submitted in partial fulfilment of the requirements for the award of the degree of Doctor of Philosophy of the University of Portsmouth September 2018 Declaration Whilst registered as a candidate for the above degree, I have not been registered for any other research award. The results and conclusions embodied in this dissertation are the work of the named candidate and have not been submitted for any other academic award. Murtadha Sami Luaibi Al-Maliki i Acknowledgment I would like to express my gratitude to my country Iraq, represented by the Prime Minister Office – Higher Committee for Education Development (HCED) in Iraq for giving me this full scholarship to study PhD in the University of Portsmouth; This work would not have been possible without their support and encouragement. Foremost, I would like to express my sincere gratitude to my supervisor Dr. Linda Yang for the continuous support of my PhD study and research, for her patience, motivation, enthusiasm, and immense knowledge. Her guidance helped me in all the time of research and writing of this thesis. I could not have imagined having a better supervisor and mentor for my PhD study. Besides my supervisor, I would like to thank the University of Portsmouth, represented by the School of Energy and Electronic Engineering, the University library and the graduate school – represented by Dr. Heather MacKenzie for being very supportive by providing me an excellent environment and wise advice during my PhD study. Last but not the least, I would like to show my deepest thankful and gratitude to my parents, my brothers, my sisters and my friends (Mahmoud Al-Faris, Mohammed Ibrahim, Mohammed Al- Wattar, Haider Murad, Ahmed Ahmed, Muntadher Fadhil, Mohanad Al-Behadili, Wassem and his wife Maha, Mr. & Mrs. Kirby, etc.) for their love, prayers, courage and moral support they gave me throughout the study. ii Dedication Every challenging work needs self-efforts as well as guidance of elders especially those who were very close to our heart. My humble effort I dedicate to my sweet and loving Father & Mother Whose affection, love, encouragement and prays of day and night make me able to get such success and honour, Along with all hard working and respected Teachers I will be forever grateful to whomever teaches me anything worth learning iii Abstract The amount of music available digitally is overwhelmingly increasing. The main purpose of music recommendation systems is to suggest quality relevant songs that fit with the user’s preferences. Currently, most of the streaming music systems recommend songs based on Collaborative Filtering and Content-Based filtering techniques. However these systems fail in dealing with the Cold-Start problem. This thesis presents user-based hybrid algorithms for music recommendation systems to address the Cold-Start problem and to recommend music for both new and existing users based on their context by integrating the social information to provide context aware personalized music recommendation. This thesis makes two major contributions: First, hybrid recommendation algorithms are developed using multi-strategy approach to give more accurate recommendations by combining collaborative filtering, content based and the user’s context obtained from social network in order to provide both new and existing users with an easy way to discover new songs. In this way, the system is able to estimate what artist/song would match user preferences. Second, a generic Context-Aware Personalised Music (CAPM) framework is proposed for supporting the rapid development of context-aware music recommendation systems and for clarifying the whole process of recommendation. As there are myriad approaches of recommendation, there is a need for a generic framework not only to gather these approaches, but also to interpret them under the proposed framework. Recommendation algorithm types differ by the input structure. For example, social recommendation algorithm uses social information, collaborative filtering uses users rating data, whereas content based recommendation uses item’s characteristics. This difference affects enormously the representation of data and consequently the process of recommendation. CAPM is able to present different input data and uniforms the recommendation process. The proposed algorithms and the framework have been successfully evaluated via practical experiments by real users. The practical experiments are carried out by presenting a Context- Aware Personalised Music (CAPMusic) application in Google Play which helps users to discover new artists, albums or songs. Satisfactory results have been obtained which indicate that using the proposed hybrid recommendation algorithms leads to better results compared with using the pure content based and collaborative filtering techniques. iv Table of Content Declaration ....................................................................................................................................... i Acknowledgment ............................................................................................................................ ii Dedication ....................................................................................................................................... iii Abstract .......................................................................................................................................... iv Chapter 1 Introduction..................................................................................................................1 1.1. Motivation ............................................................................................................................2 1.2. Aims and Objectives ..................................................................................................................... 3 1.3. Contributions........................................................................................................................4 1.4. Thesis Outline ......................................................................................................................4 Chapter 2 Background and Related Work ..................................................................................6 2.1. Recommender systems: main approaches and challenges ........................................................6 2.1.1. Collaborative Filtering Models ............................................................................................7 2.1.2. Content-Based Models .........................................................................................................9 2.1.3. Hybrid Recommender systems ..........................................................................................11 2.2. Music Recommendation Systems ...........................................................................................13 2.3. Factors that Affect Music Recommendation .........................................................................17 2.3.1. Music-related factors .........................................................................................................17 2.3.2. Listener-related factors ......................................................................................................19 2.3.3. Listener context-related factors..........................................................................................20 2.3.4. Music use-related factors ...................................................................................................21 2.4. Mood Categories .....................................................................................................................23 2.5. Summary ................................................................................................................................24 Chapter 3 Context-Aware Personalised Music (CAPM) Framework ....................................26 3.1. System Framework and Design .........................................................................................26 3.2. Server Side .........................................................................................................................28 3.2.1. Data Collection ..................................................................................................................28 3.2.1.1. Last.fm Crawler ...........................................................................................................28 3.2.1.2. Lyric Searcher ..............................................................................................................29 3.2.1.3. LDSD Calculator .........................................................................................................30 v I. Linked Data Semantic Distance (Direct) ........................................................................31 II. Linked Data Semantic Distance (Indirect) .....................................................................33 III. Linked Data Semantic Distance (LDSD) .......................................................................33 3.2.1.4. User Profile Manager ...................................................................................................35 3.2.1.4a. Tweets Collector .............................................................................................................37 3.2.1.4b. Facebook Collector .........................................................................................................38