Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16) A Relaxed Ranking-Based Factor Model for Recommender System from Implicit Feedback + + Huayu Li , Richang Hong⇤, Defu Lian−, Zhiang Wu⇥, Meng Wang⇤ and Yong Ge + UNC Charlotte, hli38, yong.ge @uncc.edu { } ⇤ Hefei University of Technology, hongrc, wangmeng @hfut.edu.cn { } − University of Electronic Science and Technology of China, [email protected] ⇥ Nanjing University of Finance and Economics, [email protected] Abstract about which items users dislike. This has been a thorny issue for learning task. Implicit feedback based recommendation has re- cently been an important task with the accumulated In the literature, some related work has been proposed to user-item interaction data. However, it is very chal- take advantage of implicit feedback for item recommenda- lenging to produce recommendations from implicit tions. For example, [Hu et al., 2008] regards user’s prefer- feedback due to the sparseness of data and the lack ence for an item as a binary value, where a user’s preference of negative feedback/rating. Although various fac- for observed item and unobserved one are viewed as one and tor models have been proposed to tackle this prob- zero, respectively. Then it fits these predefined ratings with lem, they either focus on rating prediction that may vastly varying confidential levels based on matrix factoriza- lead to inaccurate top-k recommendations or are tion framework. Although it assumes that a user prefers the dependent on the sampling of negative feedback observed items to unobserved ones, the quadratic in the for- that often results in bias. To this end, we propose mulation weakens their instinct ranking order. There is no a Relaxed Ranking-based Factor Model, RRFM, to guarantee that the higher accuracy in rating prediction will re- relax pairwise ranking into a SVM-like task, where sult in the better ranking effectiveness [N. and Qiang, 2008]. positive and negative feedbacks are separated by For instance, the true ratings for two items are 1, 0.5 . The predicted ratings 0.4, 0.6 and 1.6, 0.6 have{ the same} pre- the soft boundaries, and their non-separate property { } { } is employed to capture the characteristic of unob- diction accuracy. But in fact, they lead to totally different served data. A smooth and scalable algorithm is ranking orders of items. To this end, Rendle formulates user’s developed to solve group- and instance- level’s op- consuming behavior into the pairwise ranking problem, i.e., timization and parameter estimation. Extensive ex- users are much more interested in their consumed items than periments based on real-world datasets demonstrate unconsumed ones [Rendle et al., 2009]. Due to a large num- the effectiveness and advantage of our approach. ber of such pairs, it only samples some negative items for the learning procedure. However, there are two limitations. First, the pairwise ranking increases the number of compar- 1 Introduction isons. Second, although Rendle [Rendle and Freudenthaler, Recommender systems have been an important feature to 2014] improves the sampling skill by oversampling the top recommend relevant items to relevant users in many on- ranked items, sampling technique itself easily leads to bias. line communities, e.g. Amazon, Netflix, and Foursquare. It is likely that the sampled negative item is already ranked Some online systems allow users to provide an explicit rat- below the positive one, which as a result has no contribution ing for an item to express how much they like it. A higher to the optimization. (or lower) rating indicates that the user likes (or dislikes) To address these issues, we propose to relax the rank- the item more. Nevertheless, many recommender systems ing model in [Rendle et al., 2009] to eliminate the pair- only have user’s implicit feedback, such as browsing activ- wise ranking. Specifically, the positive and negative feed- ity, purchasing history, watching history, click behavior and backs are separated by the positive and negative boundaries. check-in information. As implicit feedback becomes more In fact, the unobserved implicit feedback is often a mix- and more prevalent, this type of recommender system has at- ture of negative and missed positive data, so a slack vari- tracted many researchers’ attention [Joachims et al., 2005; able is introduced to capture such characteristic, which al- Lim et al., 2015]. However, implicit feedback based recom- lows some negative feedbacks and positive feedbacks to be mender systems suffer from many challenges. For example, non-separate. Furthermore, instead of sampling, a smooth the sparseness of observed data (i.e., only a small percentage and scalable algorithm is designed to learn model’s param- of user-item pairs have implicit feedback) increases the diffi- eters based on group and instance level’s optimization. The culty to learn user’s exact taste on items. Also, different from proposed algorithm allows to take all the unobserved items explicit feedback, only positive preference is observed in im- into account for optimization, and as a result addresses the plicit feedback. In other words, we have no prior knowledge bias caused by the sampling technique in [Rendle et al., 2009; 1683 Rendle and Freudenthaler, 2014]. Finally, the proposed is no negative rating. Most research work argues that one model is evaluated with many state-of-the-art baseline mod- user’s preference for the observed item is supposed to be els and different validation metrics on three real-world data larger than that for any unobserved one [Rendle et al., 2009; sets. The experimental results demonstrate the superiority of Hu et al., 2008], which indicates the presence of ranking be- our model for tackling implicit feedback based recommenda- tween positive and negative ratings. However, the pairwise tions. ranking of a user’s preference for the observed items over the unobserved ones is quite inefficient, especially when the 2 Preliminaries user’s historical data increases. To address this issue, we pro- The recommendation task addressed in this paper is defined pose to relax the pairwise ranking. We treat one user’s pref- as: given the consumption behaviors of N users over M erence for an item as an point, where his positive (or nega- items, we aim at recommending each user with top-K new tive) rating is viewed as the positive (or negative) point. Our items that he might be interested in but has never consumed goal is to make all positive points reside above all negative before. Matrix factorization based models assume that U points. Inspired by the soft margin idea of SVM, we sepa- K N K M 2 rate these two types of points by two different boundaries. In R ⇥ and V R ⇥ are the user and item latent feature 2 other words, user’s preference for the observed items and un- matrices, with column vectors Ui and Vj representing the K- dimension user-specific and item-specific feature vectors of observed ones are separated by the boundaries. Specifically, user i and item j, respectively. The predicted preference (rat- user’s positive rating is located on or above a boundary rep- ˆ resented as a numeric value r+. On the other hand, his nega- ing) of user i for item j, denoted as Rij, is approximated by: tive rating resides on or below another boundary. It not only ˆ T improves the efficiency of comparisons, but also preserves Rij = Ui Vj. (1) the ranking information. Along this line, we relax Eq.(2) for In implicit feedback datasets, only positive feedback is ob- o u j i k i in the following: served. As we lack substantial evidence on which items users 8 2M ^8 2M T dislike, their preference for an unobserved item is regarded Ui Vj r+, T ≥ (3) as a mixture of negative and missing values. Along this line, Ui Vk r + ⇠ik, ⇢ − [Rendle et al., 2009] assumes that each user prefers the ob- where ⇠ik is the slack variable for user i on the unobserved o served items over unobserved ones. Let us denote i as a set item k. Similar to [Hu et al., 2008], r+ and r are set as u M − of items that user i has consumed and as the remaining one and zero, respectively. Even though there may be many Mi items that he never consumed. Then for user i, the ranking unobserved items for a user, it does not necessarily indicate based on user’s preference for an observed item j over an un- that he dislikes them. Probably he may be just unaware of observed item k is given by: them. In other words, some of the unobserved items might be T T o u those users are interested in, while others are actually those U V >U V , j k (2) i j i k 8 2Mi ^8 2Mi they dislike. To capture this characteristic, the slack variable For convenience we will henceforth refer to i as user, j as ⇠ik in Eq.(3) is introduced to allow the mixture of negative observed item, and k as unobserved item unless stated other- feedback and positive feedback in unobserved data. Hence, wise. Eq.(2) models the correlation of user’s preference for we apply the following constraint for ⇠ik as: each pair of observed item and unobserved one. It is actu- r + ⇠ik r+ ⇠ik r+ r . (4) − ≥ ) ≥ − − ally maximizing the Area Under the ROC Curve (AUC) for It is worth to note that for each user i, pairwise ranking in matrix factorization. The optimization problem for pairwise Eq.(2) needs o u comparisons; but our relaxed ranking ranking is formulated as follows: |Mi ||Mi | in Eq.(3) only requires M= o + u comparisons.