Application of Dimensionality Reduction in Recommender System -- A Case Study Badrul M. Sarwar, George Karypis, Joseph A. Konstan, John T. Riedl Department of Computer Science and Engineering / Army HPC Research Center University of Minnesota Minneapolis, MN 55455 +1 612 625-4002 {sarwar, karypis, konstan, riedl}@cs.umn.edu meet many of the challenges of recommender Abstract systems, under certain conditions. We investigate the use of dimensionality reduction to 1 Introduction improve performance for a new class of data analysis software called “recommender systems”. Recommender systems have evolved in the extremely Recommender systems apply knowledge discovery interactive environment of the Web. They apply data techniques to the problem of making product analysis techniques to the problem of helping recommendations during a live customer interaction. customers find which products they would like to These systems are achieving widespread success in purchase at E-Commerce sites. For instance, a E-commerce nowadays, especially with the advent of recommender system on Amazon.com the Internet. The tremendous growth of customers (www.amazon.com) suggests books to customers and products poses three key challenges for based on other books the customers have told recommender systems in the E-commerce domain. Amazon they like. Another recommender system on These are: producing high quality recommendations, CDnow (www.cdnow.com) helps customers choose performing many recommendations per second for CDs to purchase as gifts, based on other CDs the millions of customers and products, and achieving recipient has liked in the past. In a sense, high coverage in the face of data sparsity. One recommender systems are an application of a successful recommender system technology is particular type of Knowledge Discovery in Databases collaborative filtering, which works by matching (KDD) (Fayyad et al. 1996) technique. KDD customer preferences to other customers in making systems use many subtle data analysis techniques to recommendations. Collaborative filtering has been achieve two unsubtle goals. They are: (i) to save shown to produce high quality recommendations, but money by discovering the potential for efficiencies, the performance degrades with the number of or (ii) to make more money by discovering ways to customers and products. New recommender system sell more products to customers. For instance, technologies are needed that can quickly produce companies are using KDD to discover which high quality recommendations, even for very large- products sell well at which times of year, so they can scale problems. manage their retail store inventory more efficiently, This paper presents two different experiments where potentially saving millions of dollars a year we have explored one technology called Singular (Brachman et al. 1996). Other companies are using Value Decomposition (SVD) to reduce the KDD to discover which customers will be most dimensionality of recommender system databases. interested in a special offer, reducing the costs of Each experiment compares the quality of a direct mail or outbound telephone campaigns by recommender system using SVD with the quality of a hundreds of thousands of dollars a year recommender system using collaborative filtering. (Bhattacharyya 1998, Ling et al. 1998). These The first experiment compares the effectiveness of applications typically involve using KDD to discover the two recommender systems at predicting consumer a new model, and having an analyst apply the model preferences based on a database of explicit ratings of to the application. However, the most direct benefit products. The second experiment compares the of KDD to businesses is increasing sales of existing effectiveness of the two recommender systems at products by matching customers to the products they producing Top-N lists based on a real-life customer will be most likely to purchase. The Web presents purchase database from an E-Commerce site. Our new opportunities for KDD, but challenges KDD experience suggests that SVD has the potential to systems to perform interactively. While a customer is at the E-Commerce site, the recommender system depicts the neighborhood formation using a nearest- must learn from the customer’s behavior, develop a neighbor technique in a very simple two dimensional model of that behavior, and apply that model to space. Notice that each user’s neighborhood is those recommend products to the customer. Recommender other users who are most similar to him, as identified systems directly realize this benefit of KDD systems by the proximity measure. Neighborhoods need not in E-Commerce. They help consumers find the be symmetric. Each user has the best neighborhood products they wish to buy at the E-Commerce site. for him. Once a neighborhood of users is found, Collaborative filtering is the most successful particular products can be evaluated by forming a recommender system technology to date, and is used weighted composite of the neighbors’ opinions of in many of the most successful recommender systems that document. on the Web, including those at Amazon.com and These statistical approaches, known as automated CDnow.com. collaborative filtering, typically rely upon ratings as The earliest implementations of collaborative numerical expressions of user preference. Several filtering, in systems such as Tapestry (Goldberg et ratings-based automated collaborative filtering al., 1992), relied on the opinions of people from a systems have been developed. The GroupLens close-knit community, such as an office workgroup. Research system (Resnick et al. 1994) provides a However, collaborative filtering for large pseudonymous collaborative filtering solution for 1 2 3 5 4 Figure 1: Illustration of the neighborhood formation process. The distance between the target user and every other user is computed and the closest-k users are chosen as the neighbors (for this diagram k = 5). communities cannot depend on each person knowing Usenet news and movies. Ringo (Shardanand et al. the others. Several systems use statistical techniques 1995) and Video Recommender (Hill et al. 1995) are to provide personal recommendations of documents email and web systems that generate by finding a group of other users, known as recommendations on music and movies respectively. neighbors that have a history of agreeing with the Here we present the schematic diagram of the target user. Usually, neighborhoods are formed by architecture of the GroupLens Research collaborative Recommender System Reques t Ratings Ratings WWW Dynamic HTML Respons e Server generator Recomm- Recomm- endations Customer endations Correlation Ratings Figure 2. Recommender System Architecture Database Database applying proximity measures such as the Pearson filtering engine in figure 2. The user interacts with a correlation between the opinions of the users. These Web interface. The Web server software are called nearest-neighbor techniques. Figure 1 communicates with the recommender system to choose products to suggest to the user. The 2 Existing Recommender Systems recommender system, in this case a collaborative Approaches and their Limitations filtering system, uses its database of ratings of products to form neighborhoods and make Most collaborative filtering based recommender recommendations. The Web server software displays systems build a neighborhood of likeminded the recommended products to the user. customers. The Neighborhood formation scheme The largest Web sites operate at a scale that stresses usually uses Pearson correlation or cosine similarity the direct implementation of collaborative filtering. as a measure of proximity (Shardanand et al. 1995, Model-based techniques (Fayyad et al., 1996) have Resnick et al. 1994). Once these systems determine the potential to contribute to recommender systems the proximity neighborhood they produce two types that can operate at the scale of these sites. However, of recommendations. these techniques must be adapted to the real-time 1. Prediction of how much a customer C will like a needs of the Web, and they must be tested in realistic product P. In case of correlation based problems derived from Web access patterns. The algorithm, prediction on product ‘P’ for present paper describes our experimental results in customer ‘C’ is computed by computing a applying a model-based technique, Latent Semantic weighted sum of co-rated items between C and Indexing (LSI), that uses a dimensionality reduction all his neighbors and then by adding C's average technique, Singular Value Decomposition (SVD), to rating to that. This can be expressed by the our recommender system. We use two data sets in following formula (Resnick et al., 1994): our experiments to test the performance of the model- based technique: a movie dataset and an e-commerce - åJÎrates (J P J )rCJ dataset. C = C + P pred r The contributions of this paper are: åJ CJ 1. The details of how one model-based Here, rCJ denotes the correlation between user C technology, LSI/SVD, was applied to and neighbor J. JP is J's ratings on product P. reduce dimensionality in recommender J and C are J and C's average ratings. The systems for generating predictions. prediction is personalized for the customer C. There are, however, some naive non- 2. Using low dimensional representation personalized prediction schemes where to compute neighborhood for generating prediction, for example, is computed simply by recommendations. taking the average ratings of items being predicted over