University of Saarland Faculty of Natural Sciences and Technology I Department of Computer Science Master’s Thesis ENTITYRECOMMENDATION BASEDONWIKIPEDIA submitted by dragan milchevski on December 20, 2013 Supervisor: Dr.-Ing. Klaus Berberich Reviewers: Dr.-Ing. Klaus Berberich Prof. Dr.-Ing. Gerhard Weikum Eidesstattliche Erklärung Ich erkläre hiermit an Eides statt, dass ich die vorliegende Ar- beit selbstständig verfasst und keine anderen als die angegebenen Quellen und Hilfsmittel verwendet habe. Statement in Lieu of an Oath I hereby confirm that I have written this thesis on my own and that I have not used any other media or materials than the ones referred to in this thesis. Einverständniserklärung Ich bin damit einverstanden, dass meine (bestandene) Arbeit in beiden Versionen in die Bibliothek der Informatik aufgenommen und damit veröffentlicht wird. Declaration of Consent I agree to make both versions of my thesis (with a passing grade) accessible to the public by having them added to the library of the Computer Science Department. Saarbrücken, December 20, 2013 Dragan Milchevski He who loves practice without theory is like the sailor who boards ship without a rudder and compass and never knows where he may cast. — Leonardo da Vinci ABSTRACT Recommender systems improve the user experience, usually on the Web, by producing personalized suggestions for new products based on past user behavior. Therefore, they are a very popular research and industry topic. Many commercial sites, such as Ama- zon, Netflix and Apple offer recommendations for their products. However, their algorithms work on propriety data and are con- fined to one category. For instance, they recommend music based on listened music, movies based on watched movies and buy-able goods based on purchased goods. In this thesis we present a novel system for personalized en- tity recommendation across different categories by utilizing open- source data, namely Wikipedia and YAGO. We propose a hybrid model that exploits the users’ contributions to Wikipedia articles, the textual similarity between the articles as well as the relations of the entities within the YAGO taxonomy. We introduce a technique for inferring implicit user preferences from the Wikipedia revi- sion history. Further, we propose a similarity measure that uses the WordNet taxonomy from YAGO and the Wikipedia category system and combines it with collaborative and content-based fil- tering algorithms into a hybrid similarity measure. Using offline evaluation and a user study, we demonstrate the effectiveness of our proposed methods. vii ACKNOWLEDGMENTS I would like to sincerely thank my supervisor, Dr.-Ing. Klaus Ber- berich for his invaluable guidance, engagement and support dur- ing all stages of this thesis. He was always open for discussions, providing useful advises and ideas on solutions to problems that I faced during my work. I am also greatly thankful for his careful reading and commenting on the manuscript. I would like to thank Prof. Dr. Gerhard Weikum for giving me the opportunity to work in a great research environment at the D5 department of the Max-Planck Institute for Informatics. I would also like to take this opportunity to thank the members of the de- partment who participated in my user study and helped me with their valuable feedback. I would like to give my special thanks to my parents and my sister for believing in me and giving me their unconditional love and support. A heartfelt thanks goes out to Evica whose support has always been my source of strength and inspiration. This thesis would not have been possible if it wasn’t for her advices and discussions that inspired new ideas. I am especially grateful for her patience dur- ing the last months and for proofreading my thesis. Last, but not least, I would like to thank all my friends for their unconditional support. ix CONTENTS 1 introduction1 1.1 Motivation . 1 1.2 Research Questions . 4 1.3 Structure of the Thesis . 4 2 background5 2.1 Recommender Systems . 5 2.2 Recommendation Techniques . 9 2.2.1 Collaborative Filtering . 10 2.2.2 Content-based Filtering . 15 2.2.3 Knowledge-based Filtering . 17 2.2.4 Hybrid Recommendation . 18 2.3 Knowledge Bases . 20 2.3.1 YAGO . 21 2.4 Wikipedia . 22 3 related work 25 3.1 Collaborative Filtering Recommender Systems . 25 3.2 Content-based Recommender Systems . 26 3.3 Knowledge-based Recommender Systems . 28 3.4 Hybrid Recommender Systems . 30 3.5 Recommendation Algorithms and APIs . 32 3.5.1 LensKit . 32 3.5.2 Mahout . 33 4 data sources 35 4.1 Preprocessing the Wikipedia Dataset . 35 4.2 Studying the Wikipedia Dataset . 38 4.3 YAGO as a Data Source . 42 5 recommending entities 45 5.1 General Idea . 45 5.2 Overall Architecture . 47 5.3 Modeling the User Profile . 48 xi xii contents 5.3.1 Characteristics of Wikipedia Revision History 48 5.3.2 Computing the Preference Values . 49 5.4 Similarity Measures . 53 5.4.1 Item-based Similarity . 53 5.4.2 Text-based Similarity . 56 5.4.3 Taxonomy-based Similarity . 56 5.4.4 Hybridization Similarity . 59 5.5 Generation of Recommendations . 60 6 prototype implementation 63 6.1 System Architecture . 63 6.2 System Implementation . 66 6.2.1 Back-end System – Recommendation Engine 66 6.2.2 Front-end Application . 68 6.3 User Interface . 70 7 experimental evaluation 77 7.1 Evaluation Measures . 77 7.1.1 Prediction Accuracy . 77 7.1.2 Classification Accuracy . 78 7.2 Offline Evaluation . 81 7.2.1 Setup . 82 7.2.2 Results . 82 7.3 User Study . 89 7.3.1 Setup . 89 7.3.2 Results . 90 7.4 Summary . 101 8 conclusion and future work 103 a appendix 105 a.1 Hadoop Commands for Computing Text-based Sim- ilarity Using Mahout . 105 1 INTRODUCTION 1.1 motivation Which movie should I watch next? Which book should I read? Which music album should I buy? Where should I go on vacation? Which web portal offers interesting information for me? The list of questions can grow quite long. The decision making process is especially aggra- vated today where people can find everything and everyone on Internet. In this digitized world we are overloaded with information. The question is how to find something in the ocean of data that is in- teresting for us and suits our needs best? People have used many ways to deal with the decision making process, like search engines, social networks, consulting other friends, consulting experts. How- ever, none of the strategies is good enough for generic cases. The search engines often need to know exactly what are we looking for. Social networks can help sometimes, but we need to have many friends with similar taste. Still, because of a privacy concern issue, people on the social networks often do not share what they like. Experts sometimes tell the same thing to everybody, and the ques- tion is, is that what we like or what they like? Recommender systems (RS) offer a mechanism for dealing with the increasing information overload and helping with the deci- sion making process. Due to their commercial value, they are a very popular research and industry topic. Many commercial sites like Amazon, Apple and Netflix offer recommendations for their services. However, they are all based on propriety data and the recommendations are mostly limited to one category (e.g. music based on listened music, movies based on watched movies, buy- able goods based on purchased goods). On the other hand we have lots of open-data sources (e.g. Wikipedia [8], YAGO [91], Word- Net [36]) that contain knowledge about many different entities from different domains and categories. Furthermore, Wikipedia offers another very valuable information – users’ contributions to 1 2 introduction the Wikipedia articles. These data sources provide unique possi- bilities for learning the characteristics of millions of entities and observing the taste of millions of users for different entities across multiple categories. Nevertheless, to the best of our knowledge, no system provides entity recommendations based on open-data across multiple categories. In this thesis we present a novel ap- plication for personalized entity recommendations that leverages data from Wikipedia and knowledge extracted from YAGO for recommending entities across different categories. We have built a hybrid model that exploits the users’ contributions to Wikipedia articles as well as the relations of the entities within the YAGO taxonomy. Existing recommender systems often use collaborative filtering (CF), content-based filtering (CBF) or knowledge-based filtering (KBF) as recommendation techniques. Collaborative filtering uses the ratings of other users that had similar taste in the past to produce recom- mendations. For example, if you and your friend listened to the same music or watched the same movies, and your friend bought a new book that you do not know about, you might like the book as well. Content-based filtering creates a profile for each product to characterize its nature. Profiles with similar characteristics to the ones we liked in the past are recommended. For example, if you liked fantasy novels in the past, a content-based recommender system will recommend you other fantasy novels. This technique often requires more information about the items. The knowledge- based approaches use external knowledge to filter products based on their attributes. They recommend items based on specific do- main knowledge, about how much certain item feature satisfies the user needs and tastes. There is also a fourth technique hy- brid recommendation that combines two ore more of the above men- tioned techniques. This can mitigate problems such as the “cold start“ problem, when users have not yet rated enough items, or the problem when item characteristics are unknown.