AWESOME – A Data Warehouse-based System for Adaptive Website Recommendations Andreas Thor Erhard Rahm University of Leipzig, Germany {thor, rahm}@informatik.uni-leipzig.de Abstract There are many ways to automatically generate rec- ommendations taking into account different types of in- Recommendations are crucial for the success of formation (e.g. product characteristics, user characteris- large websites. While there are many ways to de- tics, or buying history) and applying different statistical or termine recommendations, the relative quality of data mining approaches ([JKR02], [KDA02]). Sample these recommenders depends on many factors approaches include recommendations of top-selling prod- and is largely unknown. We propose a new clas- ucts (overall or per product category), new products, simi- sification of recommenders and comparatively lar products, products bought together by customers, evaluate their relative quality for a sample web- products viewed together in the same web session, or site. The evaluation is performed with products bought by similar customers. Obviously, the AWESOME (Adaptive website recommenda- relative utility of these recommendation approaches (rec- tions), a new data warehouse-based recommen- ommenders for short) depends on the website, its users dation system capturing and evaluating user and other factors so that there cannot be a single best ap- feedback on presented recommendations. More- proach. Website developers thus have to decide about over, we show how AWESOME performs an which approaches they should support and where and automatic and adaptive closed-loop website op- when they should be applied. Surprisingly, little informa- timization by dynamically selecting the most tion is available in the open literature on the relative qual- promising recommenders based on continuously ity of different recommenders. Hence, one focus of our measured recommendation feedback. We pro- work is an approach for comparative quantitative evalua- pose and evaluate several alternatives for dy- tions of different recommenders. namic recommender selection including a power- Advanced websites, such as Amazon [LSY03], sup- ful machine learning approach. port many recommenders but apparently are unable to select the most effective approach per user or product. 1 Introduction They overwhelm the user with many different types of recommendations leading to huge web pages and reduced Recommendations are crucial for the success of large web usability. While commercial websites often consider the sites to effectively guide users to relevant information. E- buying behaviour for generating recommendations, the commerce sites offering thousands of products cannot usage (navigation) behaviour on the website remains solely rely on standard navigation and search features but largely unexploited. We believe this a major shortcoming need to apply recommendations to help users quickly find since the navigation behaviour contains detailed informa- “interesting” products or services. With many users and tion on the users’ interests not reflected in the purchase products manual generation of recommendations is much data. Moreover, the web usage behaviour contains valu- too laborious and ineffective. Hence a key question be- able user feedback not only on products or other content comes how should recommendations be generated auto- but also on the presented recommendations. The utiliza- matically to optimally serve the users of a website. tion of this feedback to automatically and adaptively im- prove recommendation quality is a major goal of our Permission to copy without fee all or part of this material is granted work. provided that the copies are not made or distributed for direct commercial advantage, the VLDB copyright notice and the title of the AWESOME (Adaptive website recommendations) is a publication and its date appear, and notice is given that copying is by new data warehouse-based website evaluation and rec- permission of the Very Large Data Base Endowment. To copy ommendation system under development at the University otherwise, or to republish, requires a fee and/or special permission from of Leipzig. It contains an extensible library of recommen- the Endowment Proceedings of the 30th VLDB Conference, der algorithms that can be comparatively evaluated for Toronto, Canada, 2004 real websites based on user feedback. Moreover, 384 Record web access and recommendation feedback Recommendations Rec. Filter Current context Application Server User history Recommender Selection Recom- Recommender Precomputed Selection Presented Library mender Recommendations Recommendations Web Usage Analysis Recommendation User history Content User Web Data Data feedback Data Warehouse User history & Recommender ETL process Recommendation feedback Evaluation Figure 1: AWESOME architecture AWESOME can perform an automatic closed-loop web- lated work is briefly reviewed in Section 6 before we con- site optimization by dynamically selecting the most prom- clude. ising recommenders for a website access. This selection is based on the continuously measured recommendation 2 Architecture quality of the different recommenders so that AWESOME automatically adapts to changing user interests and chang- 2.1 Overview ing content. To support high performance and scalability, quality characteristics of recommenders and recommenda- Fig. 1 illustrates the overall architecture of AWESOME tions are largely precomputed. AWESOME is fully op- which is closely integrated with the application server erational and in continuous use at a sample website; adop- running the website. AWESOME is invoked for every tion to further sites is in preparation. website access, specified by a so-called context including The main contributions of this paper are as follows: information from the current HTTP request such as URL, - Presentation of the AWESOME architecture for timestamp and user-related data. For such a context, warehouse-based recommender evaluation and for AWESOME dynamically generates a list of recommenda- scaleable adaptive website recommendations tions which are displayed by the application server to- - A new classification of recommenders for websites gether with the requested website content. Recommenda- supporting a comparison of different approaches. tions are automatically determined by a variety of algo- We show how sample approaches fit the classifica- rithms from an extensible recommender library. The rec- tion and propose a new recommender for users ommenders use information on the usage history of the coming from search engines. website and additional information maintained in a web - A comparative quantitative evaluation of several data warehouse. The recommendations are subject to a recommenders for a sample website. The consid- final filter step to avoid the presentation of unsuitable or ered recommenders cover a large part of our classi- irrelevant recommendations (e.g., recommendation of the fication’s design space. current page or the homepage). - Description and comparative evaluation of several Dynamic selection of recommendations is a two-step rule-based approaches for dynamic recommender process. For a given context, AWESOME first selects the selection. In particular, a machine learning ap- most appropriate recommender(s). This recommender proach for feedback-based recommender selection selection is controlled by a moderate number of selection is presented. rules. For evaluation purposes, we support several selec- In the next section we present the AWESOME archi- tion strategies for determining and adapting these rules, in tecture and the underlying data warehouse approach. We particular automatic approaches based on user feedback then outline our recommender classification and sample on previously presented recommendations. This recom- recommenders (Section 3). Section 4 contains the com- mendation feedback is also recorded in the web data parative evaluation of several recommenders for a non- warehouse. For the chosen recommender(s), the best rec- commercial website. In Section 5 we describe and evalu- ommendations for the current context are selected in the ate approaches for dynamic recommender selection. Re- second step. For performance reasons, these recommenda- 385 tions are precomputed (and periodically refreshed) and they have been followed. We thus decided to use tailored can thus quickly be looked up at runtime. application server logging to record this information. Ap- Separating the selection of recommenders and recom- plication server logging also enables us to apply effective mendations makes it easy to add new recommenders. approaches for session and user identification and early Moreover, using recommendation feedback at the level of elimination of crawler accesses, thus supporting high data recommenders is simpler and more stable than trying to quality. use this feedback for individual recommendations, e.g. The AWESOME extensions of the application server specific web pages or products. One problem with the are implemented by PHP programs and run together with latter approach is that individual pages/products are fre- standard web servers such as Apache. We use two log quently added and that there is no feedback available for files: a web usage and a recommendation log file with the such new content. Conversely, removing content would formats shown in Table 1. The recommendation log file result in a loss of the associated recommendation feed- records all presented recommendations and is required for back. our recommender