CSE 255 Assignment 1 : Movie Rating Prediction using the MovieLens dataset Yashodhan Karandikar [email protected] 1. INTRODUCTION The goal of this project is to predict the rating given a user and a movie, using 3 different methods - linear regression using user and movie features, collaborative filtering and la- tent factor model [22, 23] on the MovieLens 1M data set [6]. We describe results using each of the 3 models and compare them with those obtained using the Apache Mahout machine learning library [1]. We also briefly discuss some results on restaurant ratings data from the Google Local data set [4]. 2. THE MOVIELENS DATA SET In this section, we introduce the MovieLens 1M data set [6]. This is a data set of about 1 million ratings from 6040 users on 3900 movies, thus containing around 165 ratings per user and 256 ratings per movie on an average. Ratings are integers on a 5-star scale. Each user and each movie is identified by a unique id. The data set includes information about the age (7 age groups), gender, occupation (21 types) Figure 1: Average rating for different age groups and zip code for each user, as well as the title and genre (18 types) for each movie. Each user has at least 20 ratings. We normalize the ratings so that each rating is in the closed interval [0; 1]. We perform an exploratory analysis of the data set by plotting the average rating against user and item features. This analysis is shown in figures 1, 2, 3, 4. Figure 1 shows the average rating for each of the age groups. We observe a slightly increasing trend in the average rating as the age group of the user increases, suggesting that the age group of the user might be a useful feature in predicting movie ratings. Figure 2 shows the average rating for male users and female Figure 2: Average rating for male and female users users. We observe a very small difference between the aver- age ratings for male and female users, suggesting that gender of the user is not likely to be a useful feature in prediction pations. We observe that there is a reasonable amount of of movie ratings. variation in the average ratings for various occupations, with the minimum and maximum average ratings being 0.681257 Figure 3 shows the average rating for users in various occu- and 0.757114 respectively. This indicates that the occupa- tion of the user might be a useful feature in prediction of ratings. Figure 4 shows the average rating for various movie genres. We observe that there is a reasonable amount of variation in the average ratings for various genres, with the minimum and maximum average ratings being 0.644848 and 0.815781 respectively. This indicates that the genre of the movie might be a useful feature in prediction of ratings. Figure 3: Average rating for various occupations Figure 4: Average rating for various movie genres 3. MOVIE RATING PREDICTION for the best algorithm to predict user ratings for films, given In this section we discuss the task of predicting the rating only the previous ratings, without any other information given a user and a movie. Given a user u and a movie m, about the users or films. An alternative to these methods is we would like to predict the rating that u will give to m. content filtering [23], which uses user features (such as age, Although this problem is closely related to the problem of gender, etc.) and item features (such as movie genre, cast recommending the best possible items to a user based on etc.) instead of relying on past user behavior. the user's previous purchases or reviews, in this work we fo- cus on accurately predicting the rating itself. Accordingly, Neighborhood approaches and latent factor models are gen- we evaluate the performance of our models using the mean- erally more accurate than content-based techniques [23], since squared-error (MSE) [17] and the coefficient of determina- they are domain-free and capture subtle information (such as tion R2 [15] metrics. These are calculated as follows: a user's preferences) which is hard to extract using content- filtering. On the other hand, neighborhood approaches and latent factor models suffer from the cold start problem i.e. N they are not useful on new users or new items [13]. 1 X 2 MSE = (f(u ; m ) − r(u ; m )) N i i i i i=1 In this work, we compare 3 models - the first one is a content- based linear predictor and is described in section 6. The sec- ond one is a neighborhood model and is described in section MSE R2 = 1 − 7. The last one is a latent-factor model and is described in variance in true ratings section 8. where N is the number of examples, ui is the user in the 5. FEATURES example i, mi is the movie in the example i, f(ui; mi) is the Sections 6, 7, 8 discuss the 3 models we consider, namely predicted rating, r(ui; mi) is the true rating. linear regression, collaborative filtering and latent factor[23, 22]. While collaborative filtering and latent factor models The MSE is a useful metric for comparing 2 or more models do not use the additional information about users and items on the same data. However, the MSE depends on the vari- available in the dataset, the model based on linear regression ance in the data and hence by itself is not a good measure relies on this information. As discussed in section 2, the 2 of how well a model explains the data. The R value, on the user's age and occupation and the movie genre seem to be other hand, is a measure of the fraction of the variance that useful in predicting the rating that a user will give to a is explained by a model, and hence is a better measure of movie. how well a model explains the data. We report both values for each model we consider. The data set includes the user's age as one of 6 values - 1, 18, 25, 35, 45, 50, 56, which respectively denote the age groups As a baseline model, we consider a linear predictor based on under 18, 18-24, 25-34, 35-44, 45-49, 50-55, above 56. During user and item features, which we discuss in section 6. the pre-processing stage, we replace each of these values by an integer in the interval [0; 6]. When we create the feature Further, to confirm the validity of our results, we compare vector for a given user-movie pair, we create one boolean them with the results obtained using the Apache Mahout feature for each age group, where a value of 1 indicates that machine learning library [1] on the same data set. Apache the user falls in the corresponding age group. Thus, we have Mahout contains an implementation of an alternating-least- 7 features to represent the user's age. squares recommender with weighted-λ-regularization (ALS- WR) [28, 8]. The data set includes the user's occupation as an integer in the interval [0; 20], where each integer denotes a particular 4. RELATED WORK occupation as given in [7]. When we create the feature vector The MovieLens data set [6, 7] is a data set collected and for a given user-movie pair, we create one boolean feature made available by the GroupLens Research group [5]. Movie- for each occupation, where a value of 1 indicates that the Lens is a website for personalized movie recommendations user's occupation is the corresponding occupation. Thus, we [10]. The data set contains data from users who joined have 21 features to represent the user's occupation. MovieLens in the year 2000. The data set includes the genres for a movie as a string of Given a set of users, items and ratings given by some of the genres separated by `j'. Each genre in this string is one of 18 users for some of the items, the task of predicting the rating possible genres given in [7]. We represent each genre as an for a given user for a given movie has been studied in the lit- integer in the interval [0; 17]. When we create the feature erature in the form of recommender systems, which addition- vector for a given user-movie pair, we create one boolean ally perform the task of recommending items to users. The feature for each genre, where a value of 1 indicates that state-of-the-art in recommender systems is based on 2 main the corresponding genre is one of the genres listed for that approaches - neighborhood approach and latent factor models movie. Thus, we have 18 features to represent the genres of [21, 23]. Both these methods rely only on past user behav- the movie. ior, without using any features about the users or items. [28] describes an Alternating-Least-Squares with Weighted- Similarly, we represent the user's gender by 2 boolean fea- λ-Regularization (ALS-WR) parallel algorithm designed for tures. Although the user's gender does not seem to be useful, the Netflix Prize challenge [11], a well-known competition we still include it in the features used by the predictor and expect that the the predictor will learn a weight close to 0 for these 2 features. N M ^ 1 X 2 X 2 θ = arg min (r(ui; mi) − Xi · θ) + λ θi θ N We note that though the features just described capture i=1 i=2 useful attributes of users and movies, none of these features capture a sense of compatibility or a match between a user We have the following expressions for the gradient of the and a movie.