Warsaw University of Technology The Faculty of Electronics and Information Technology Institute of Control and Computation Engineering Facebook user interests exploration and recommendation based on Facebook Social Graph data analysis Author: Jakub Krzemien´ student no. 214616 Supervisor: Mariusz Kamola PhD Warsaw, 2014 Streszczenie Eksploracja i sugerowanie zainteresowa´noparte o analiz¸edanych z Facebook Social Graph Dynamiczny rozw´oji rozpowszechnienie Internetu a tak_zeogromna popularyzacja serwis´owspo leczno´sciowych sprawi ly, _zeogromne ilo´sci informacji na temat u_zytkownik´owInternetu s¸adost¸epnena wyci¸agni¸ecie r¸eki.Najpopularniejsze serwisy spo leczno´sciowe udostepniaj¸awygodne narz¸edziadzi¸ekikt´orymmo_znawchodzi´cw interakcj¸ez u_zytkownikami tych serwis´ow,a tak_zeuzyskiwa´cdost¸epdo danych profilowych. Takie informacje jeszcze kilka lat temu by lyniedost¸epneczy te_znieosi¸agalne. W ostatnich czasach sytuacja ta uleg la zmianie co generuje nowe, niezbadane jeszcze obszary zwi¸azanez analiz¸ai interpretacj¸atego typu danych. R´ownie_zw zwiazku z rozwojem Internetu i ogromem informacji w nim zawartych, sta lo si¸epo_z¸adaneby w jaki´sspos´ob moderowa´cczy selekcjonowa´ctre´scidla u_zytkownika. Mechanizmy serwuj¸acereklamy w serwisach WWW staraj¸asie prezentowa´ctre´sci, kt´oremoga zainteresowa´cdanego u_zytkownika w oparciu o histori¸e jego zapyta´nczy wcze´sniejprzegl¸adanestrony. Serwisy spo leczno´sciowe jak np. Facebook filtruj¸aprezentowane dane, staraj¸acsie pokazywa´c informacje jedynie od najbli_zszych czy najwa_zniejszych znajomych. Sklepy internetowe pr´obuj¸asprzeda´cdodatkowe produkty dopasowuj¸ac je do poprzednich zakup´owdanego u_zytkownika. Wcze´sniejwspomni- ane nowe, niezbadane jeszcze dane stanowi¸aciekaw¸aalternatyw¸ejako podstawa dzialaniadla tego typu mechanizm´ow. Celem niniejszej pracy jest analiza obecnie znanych i u_zywanych mechanizm´owrekomendowania tre´sci, a nast¸epniepr´obastworzenia nowego mechanizmu w oparciu o dane z sieci spo leczno´sciowej, w tym wypadku Facebooka. Praca skupia sie na sugerowaniu muzyk´owi zespo l´owmuzycznych w oparciu o zainteresowania u_zytkownik´ow. Ze wzgl¸eduna subiektywn¸anatur¸etego typu rozwi¸azaniaw ramach pracy powstaly tak_zenarz¸edziamaj¸acena celu weryfikacj¸eotrzymy- wanych wynik´ow. 1 Abstract Because of the dynamic evolution and propagation of the Internet and the popularization of social networks, huge amounts of informa- tion about the users of the Internet are easily accessible to anyone. The most popular social networks provide efficient and easy to use tools that allow to interact with their users, as well as access profile data. This type of information just several years ago was not avail- able or inaccessible. This change generates new, unexplored areas for analysing and interpreting such data. Also because of the evolution and popularization of the Internet and the vast amount of data stored, it has become desirable to moderate and select the content that is be- ing displayed to the user. Mechanisms presenting ads on websites try to present content that can interest the user based on his browsing history or previous queries. Social networks such as e.g. Facebook filter the presented information and try to show only the data related to the closest or most important friends of the user. E-commerce sites try to sell additional goods by suggesting products based on previous purchases made by the user. The aforementioned new, unexplored data is an interesting alternative for the core of such systems. The aim of this paper is to analyze the currently known and used content suggestion mechanisms and then create a new one based on data from a social network, in this case Facebook. In particular the paper focuses on suggesting music bands and musicians based on user interests. Because of the subjective nature of such a solution, as a part of this paper, verification tools have also been developed. 2 Contents 1 Introduction 5 2 Basic information 7 2.1 Introduction to the concept of a social network . .7 2.2 Facebook . .8 2.2.1 Facebook Social Graph and Open Graph concepts . .8 2.2.2 Facebook Platform, Graph API and Facebook Query Language (FQL) . .8 3 Currently used recommendation algorithms 10 3.1 Collaborative Filtering - introduction . 11 3.1.1 Traditional Collaborative Filtering . 12 3.1.2 Cluster Models . 13 3.1.3 Search-based filtering . 13 3.1.4 Slope One algorithm . 14 3.2 Commercially used recommendation algorithms . 15 3.2.1 Amazon - Item-To-Item Collaborative Filtering . 15 3.2.2 Last.fm - Label Propagation Algorithm . 16 4 Social-based recommendation algorithm concept 21 4.1 Data acqusition . 21 4.2 User data analysis . 25 4.3 Building the graph . 34 4.3.1 Basic implementation . 34 4.3.2 Refined implementation . 35 4.4 Initial results . 36 5 Cloudera Oryx 38 5.1 Oryx description . 38 5.2 Oryx architecture . 38 5.3 Oryx Collaborative Filtering algorithm . 40 5.4 PMML documents . 41 5.5 Apache Mahout . 41 5.6 Oryx initial test . 42 5.7 FB Graph data with Oryx . 44 6 Results verification 47 6.1 Verification with Amazon recommendations . 47 6.2 Verification with Last.fm recommendations . 48 6.3 Amazon and Last.fm results comparison . 49 3 6.4 Verification results . 50 6.5 Conclusion . 52 7 Summary 53 8 Bibliography 54 4 1 Introduction Just several years ago the biggest online platforms in the world would pro- tect the valuable information they collect. However, in time a more open approach was born, with public-facing APIs becoming open to the world and specially prepared SDKs being developed to help interact with those end- points. One of the leaders of such a modern, open approach was Facebook. Very quickly Facebook applications became a natural element of the social network ecosystem, with friends inviting other friends to online games or so- cial applications where users could interact with their friends content such as images or events in a new way. A new business was born with social media agencies creating specialized campaigns whose goal was to become viral and spread naturally like a virus. As a result, developers from all over the world gained access to unprecedented amounts of new data - user profiles, their interests, events they are attending, things they like, content they share and much more. Data gathered from social networks can be utilized in various ways, one of which is analyzed and studied in this thesis. Recommendation Systems become more and more necessary for a variety of reasons - the amount of content Internet users are experiencing is growing rapidly and it has to be selected for them because the volume of information is simply overwhelming. For the same reason the attention span of users is shrinking, especially on mobile devices, and the content presented has to be interesting to the user, otherwise it will be disregarded. E-commerce platforms are very competitive and need to search for new ways to sell more products. Social networks users are very active, creating too much data, so it needs to be filtered. Popular cloud-based platforms for music and video try to interest users in new musicians or movies. Mobile and web ads systems need to present ads that can be interesting for the user, otherwise the click- through rate will be very low. These several examples showcase a wide variety of applications for Recommendation Systems. However, these Systems work based on algorithms and mechanics that have been developed before the age of Web 2.0 and the aforementioned freedom in access to social network data. The aim of this paper is to analyze the current Recommendation Systems and algorithms and to create a new one based on data from a social network, in this case Facebook. In particular the paper focuses on suggesting mu- sic bands and musicians. Because suggesting content is a highly suggestive process, additional verification mechanisms were proposed and introduced. Additionally, a popular open source Recommendation System, Oryx, is used to compare results. 5 The paper is divided into several chapters: • Chapter 2 - Basic information about social networks and Facebook • Chapter 3 - An analysis of currently used Recommendation Systems • Chapter 4 - Detailed information about the developed Recommenda- tion System • Chapter 5 - An analysis of a popular open source Recommendation System, Oryx • Chapter 6 - A decription of proposed verification mechanisms, and the results of this verification 6 2 Basic information The success of social networks, understood as online platforms, is a phe- nomenon that created new sources of unprecedented quantities of social data. This allows for development of new concepts, one of which being the topic of this thesis, that could have not been possible just a few years ago. Internet turns out to be the perfect environment for a concept originating from social behavior and network science. 2.1 Introduction to the concept of a social network A social network is a theoretical structure emanating from sociology, psy- chology and statistics. The most basic description defines a set of actors and ties between these actors. 'Actors' do not have to necessarily be individuals - in a social network this term is also applied to groups, organizations and whole societies. The origins of social networking understood as an online platform which is being used to create virtual social networks can be dated back to the early 1970s [1]. From digital bulletin boards the idea evolved and made its appearance in the World Wide Web during mid 1990s. The basic concept back then was to bring people together to interact, for example via simple publishing tools, predecessors of blogs, or through chat rooms. In the next couple of years the focus shifted to user profiles, where people listed their personal information and could look for users with common interests. The first online platform with such functionality was SixDegrees.com, named af- ter the concept of six degrees of separation - the theory suggesting that any two people can be connected through a chain of at most five acquaintances.