Applying Collaborative Filtering Techniques to Movie Search for Better Ranking and Browsing Seung-Taek Park David M. Pennock Dennis DeCoste Yahoo! Research Yahoo! Research Yahoo! Research 3333 Empire Ave 45 West 18th St, 6th floor 3333 Empire Ave Burbank, CA 91504 New York, NY 10011 Burbank, CA 91504 [email protected] [email protected] [email protected] Abstract of documents on the web while HITS(Kleinberg 1998) mea- sures local authorities and hubs in the base set documents In general web search engines, such as Google and Ya- extracted by the given query. However, even though item hoo! Search, document relevance for the given query authority and proximity are widely used together in general and item authority are two major components of the ranking system. However, many information search search engines for better document ranking in search results, tools in ecommerce sites ignore item authority in their item authority is often ignored or partially used in many ranking systems. In part, this may stem from the relative search systems in ecommerce sites. For example, search re- difficulty of generating item authorities due to the dif- sults are often sorted based on only item relevance against ferent characteristics of documents (or items) between the given query. ecommerce sites and the web. Links between docu- There are several challenges for adapting item authority in ments in an ecommerce site often represent relation- these information retrieval systems due to the different char- ship rather than recommendation. For example, two acteristics of documents in commercial sites (e.g., item or documents (items) are connected since both are pro- duced by the same company. We propose a new ranking product information documents) from web documents. The method, which combines recommender systems with power of PageRank and HITS mainly comes from the fea- information search tools for better search and brows- ture of links between web documents. PageRank and HITS ing. Our method uses a collaborative filtering algorithm assume that a link from document i to j represents a rec- to generate personal item authorities for each user and ommendation or endorsement of document j by the owner combines them with item proximities for better rank- of document i. However, in item information pages in com- ing. To demonstrate our approach, we build a prototype mercial sites, links often represent some kind of relationship movie search engine called MAD6 (Movies, Actors and rather than recommendation. For example, two items may Directors; 6 degrees of separation). be linked since both items are produced by the same com- pany. Also, since these item information pages are gener- Introduction ally created by providers rather than users or customers, the documents may contain providers’ perspectives on the items Two types of technologies are widely used to overcome in- rather than those of users or customers. formation overload: information retrieval and recommender On the other hand, recommender systems are widely used systems. Information retrieval systems, e.g. general web 1 2 in ecommerce sites to overcome information overload. Note search engines such as Google and Yahoo! Search , accept that information retrieval systems work somewhat passively a query from a user and return the user relevant items against while recommender systems look for the need of a user more the query. Since the number of returned documents can run actively. Information retrieval systems list relevant items at into the millions, a good ranking algorithm, which ensures higher ranks only if a user asks for it (e.g. when a user sub- high precision in the top ranked documents, is important for mits a query). However, recommender systems predict the the success of a search engine. need of a user based on the his historical activities and rec- In general, the ranking of returned documents in web ommend items that he may like to consume even though the search engines is the combination of the document proxim- user does not specifically request it. ity and authority. Document proximity, sometimes called In this study, we propose a new approach to combine document relevance, denotes the document’s similarity or informational retrieval and recommender system for bet- relevance to the given query. Document authority denotes ter search and browsing. More specifically, we propose to the importance of a document in the given document set. use collaborative filtering algorithms to calculate personal- PageRank (Page et al. 1998) measures global importance ized item authorities in search. This approach has several Copyright c 2006, American Association for Artificial Intelli- benefits. First, user ratings or behavior information (e.g. gence (www.aaai.org). All rights reserved. user click logs) better represent user’s recommendation than 1http://www.google.com links in the item information pages. Second, this informa- 2http://search.yahoo.com tion is biased to the customers’ perspectives on items rather than those of providers. Third, many ecommerce sites pro- collaborative filtering systems enables serendipitous discov- vide users both information retrieval and recommender sys- eries by using historical user data. tems. Calculating item authorities using these already ex- Collaborative filtering systems can be divided into isting recommender systems in ecommerce sites does not two classes: memory-based and model-based algorithms require much work and resources. Fourth, using both item (Breese, Heckerman, & Kadie 1998). Memory-based algo- authorities and proximities, search results can be improved. rithms (Resnick et al. 1994; Breese, Heckerman, & Kadie Last, since collaborative filtering algorithms provide person- 1998) store all historical user information in memory and alized item authorities, the system can provide a better per- use a statistical technique to find a set of closest neighbors sonalized user experience. of the target user. Then, the system combines the prefer- To demonstrate our approach, we build a prototype per- ences of neighbors to generate predictions of unrated items. sonalized movie search engine called MAD6. The name is Model-based algorithms first build a model of user ratings. an acronym for Movies, Actors, and Directors with 6 de- This model can be built by using Bayesian networks (Breese, grees of separation.3. MAD6 combines both information re- Heckerman, & Kadie 1998), clustering (Breese, Heckerman, trieval and collaborative filtering techniques for better search & Kadie 1998; Ungar & Foster 1998), or classifiers (Billsus and navigation. MAD6 is different from general web search & Pazzani 1998; Miyahara & Pazzani 2000). engines since it exploits users’ ratings on items rather than Collaborative filtering algorithms range from the simple the link structures for generating item authorities. More- nearest-neighbor methods (Breese, Heckerman, & Kadie over, using the users’ historical preference data and expected 1998; Resnick et al. 1994; Sarwar et al. 2001) to more preferences on items, MAD6 provides a personalized search complex machine learning based methods such as graph ranking for each user. Even though we apply our ranking based methods (Aggarwal et al. 1999; Huang, Chen, & method to one specific domain, we believe that our ranking Zeng 2004), linear algebra based methods (Billsus & Paz- approach is general enough and it can be applied to other zani 1998; Sarwar et al. 2000; Goldberg et al. 2001; domains, including web search, by using fast and scalable Marlin & Zemel 2004; Rennie & Srebro 2005; DeCoste collaborative filtering algorithms. 2006), and probabilistic methods (Hofmann & Puzicha 1999; Pennock et al. 2000; Popescul et al. 2001; Karypis Related Work 2001; Deshpande & Karypis 2004). A few variations of filterbot-based algorithms (Good et al. 1999; Park et al. Page et al. (1998) and Kleinberg (1998) first proposed a 2005) and hybrid methods (Balabanovic & Shoham 1997; new concept of document relevance—often called docu- Popescul et al. 2001; Melville, Mooney, & Nagarajan 2002; ment authority—and proposed PageRank and HITS algo- Basilico & Hofmann 2004a; 2004b) that combine content rithms for better precision in web search (Kleinberg 1998; and a collaborative filtering have also been proposed to at- Page et al. 1998). Both algorithms analyze the link struc- tack the cold start problem. ture of the web and calculate document authorities similarly. Tapestry (Goldberg et al. 1992) is one of the earliest rec- Later, Haveliwala (2002; 2003) proposed topic-sensitive ommender systems. In this system, each user records their PageRank, which generates multiple document authorities opinions (annotations) of documents they read, and these biased to each specific topic for better document ranking. annotations are accessed by others’ filters. GroupLens4 Recommender systems can be built in three ways: (Resnick et al. 1994; Konstan et al. 1997; Miller, Riedl, content-based filtering, collaborative filtering and hybrid & Konstan 1997), Ringo (Shardanand & Maes 1995) and recommender systems. Content-based recommender sys- Video Recommender (W. Hill & Furnas 1995) are the ear- tems, sometimes called information filtering systems, use liest fully automatic recommender systems, which provide behavioral user data for a single user in order to try to in- recommendations of news, music, and movies. PHOAKS fer the types of item attributes that the user is interested in. (People Helping One Another Know Stuff) (Terveen et al. Collaborative filtering compares one user’s behavior against