Supervised and Active Learning for Recommender Systems by Laurent Charlin A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy Graduate Department of Computer Science University of Toronto © Copyright 2014 by Laurent Charlin Abstract Supervised and Active Learning for Recommender Systems Laurent Charlin Doctor of Philosophy Graduate Department of Computer Science University of Toronto 2014 Traditional approaches to recommender systems have often focused on the collaborative filtering problem: using users’ past preferences in order to predict their future preferences. Although essential, rating prediction is only one of the components of a successful recommender system. One important problem is how to translate predicted ratings into actual recommendations. Furthermore, considering additional information either about users or items may offer substantial gains in performance while allowing the system to provide good recommendations to new users. We develop machine learning methods in response to some of the limitations of current recommender- systems’ research. Specifically, we propose a three-stage framework to model recommender systems. We first propose an elicitation step which serves as a way to collect user information beneficial to the recommendation task. In this thesis we framed the elicitation process as one of active learning. We developed several active elicitation methods which, unlike previous approaches which exclusively focus improving the learning model, directly aim at improving the recommendation objective. The second stage of our framework uses the elicited user information to inform models that predict user-item preferences. We focus user-preference prediction for a document recommendation problem for which we introduce a novel graphical model over the space of user side-information, item (document) contents, and user-item preferences. Our model is able to smoothly tradeoff its usage of side information and of user-item preferences to make good document recommendations in both cold-start and non-cold- start data regimes. The final step of our framework consists of the recommendation procedure. In particular, we focus on a matching instantiation and explore different natural matching objectives and constraints for the paper- to-reviewer matching problem. Further, we explore and analyze the synergy between the recommendation objective and the learning objective. In all stages of our work we experimentally validate our models on a variety of datasets from differ- ent domains. Of particular interest are several datasets containing reviewer preferences about papers submitted to conferences. These datasets were collected using the Toronto Paper Matching System, a system we built to help conference organizers in the task of matching reviewers to submitted papers. ii Acknowledgements I am most in debt to my supervisors Richard Zemel and Craig Boutilier. Without their advice, their continued encouragements and their help and support this work would not have been possible. I am glad we made this co-supervision work. I am especially grateful to Rich, whom as the NIPS’10 program chair provided the initial motivation and momentum behind this thesis. Furthermore, Rich’s ideas, presence and experience were determinant in our joint creation of the Toronto paper matching system. Throughout these projects I found a great mentor and I have been privileged to work closely to Rich. Rich has thought me a lot about how to pick and approach research problems as well as about how to model them. I am also very grateful to have been able to work with Craig. I have learned a lot from Craig. His vision, his curiosity and his scientific rigour are qualities that I strive for. Craig’s ideas were also the ones that initially helped foster this research and his insights and ideas throughout have provided great balance to my work. Our interactions through COGS have further widen my research interests. I would also like to thank my first mentor, Pascal Poupart, who showed me how exciting research could be and gave me some of the tools to succeed at it. My thanks also go to the members of my thesis committee, Sheila Mcllraith and Geoffrey Hinton, for their precise comments and questions throughout my PhD. Geoff’s enthusiasm and presence in the lab were also very motivating to me. I would also like to thank my external advisor, Andrew McCallum for the appropriateness of his comments regarding my work and also for pointing out important immediate future steps of great benefit. Finally, I am thankful to Ruslan Salakhutdinov and Anna Goldenberg for reading and commenting on the final copy of my thesis. The constant support and love of Anne were also determinant in undertaking and successfully finishing this PhD. Her reassuring words have helped me in many occasions. I am especially thankful for her ideas and her outlook on life which she selflessly shares with me and which I have learned so much from. Further, I want to dedicate this thesis to Viviane, the next big project in our lives. Although their involvement was more indirect I learned a lot from postdocs that have tenured in Toronto, specifically I want to thank Iain, Ryan, Marc’Aurelio and, of course, Hugo who has become a good friend and collaborator. Finally, the machine learning group at Toronto was an extremely stimulating and pleasant place to work at thanks to collaborators, close colleagues and friends: Kevin R., Jasper, Danny, Ilya, Kevin S., Jen, Fernando, Eric, Maks, Darius, Bowen, Charlie, Tijmen, Graham, John, Vlad, Deep, Nitish, George, Andriy, Tyler, Justin, Chris, Niail, Phil, and Genevi`eve. Special thanks to Kevin R., Jasper, Danny and Ilya for many interesting discussions about everything throughout our graduate years. iii Contents 1 Introduction 1 1.1 Recommender Systems ...................................... 1 1.1.1 Constrained Recommender Systems .......................... 2 1.2 Contributions ........................................... 3 1.3 Outline .............................................. 4 2 Background 6 2.1 Preliminaries and Conventions ................................. 6 2.1.1 Learning .......................................... 7 2.2 Preference Modelling and Predictions .............................. 7 2.2.1 Collaborative Filtering .................................. 8 2.2.2 CF for Recommendations ................................ 14 2.3 Active Preference Collection ................................... 18 2.3.1 Uncertainty Sampling .................................. 19 2.3.2 Query by Committee ................................... 20 2.3.3 Expected Model Change ................................. 21 2.3.4 Expected Error Reduction ................................ 21 2.3.5 Batch Queries ....................................... 23 2.3.6 Stopping Criteria ..................................... 24 2.4 Matching ............................................. 24 3 Paper-to-Reviewer Matching 26 3.1 Paper Matching System ..................................... 27 3.1.1 Overview of the System Framework .......................... 27 3.1.2 Active Expertise Elicitation ............................... 29 3.1.3 Software Architecture .................................. 29 3.2 Learning and Testing the Model ................................ 31 3.2.1 Initial Score Models ................................... 31 3.2.2 Supervised Score-Prediction Models .......................... 32 3.2.3 Evaluation ........................................ 33 3.3 Related Work ........................................... 36 3.3.1 Expertise Retrieval and Modelling ........................... 37 3.4 Other Possible Applications ................................... 37 3.5 Conclusion and Future Opportunities .............................. 37 iv 4 Collaborative Filtering with Textual Side-Information 39 4.1 Side Information in Collaborative Filtering .......................... 39 4.2 Problem Definition ........................................ 42 4.3 Background ............................................ 43 4.3.1 Variational Inference in Topic Models ......................... 44 4.4 Collaborative Score Topic Model (CSTM) ........................... 46 4.4.1 The Relationship Between CSTM and Standard Models ............... 48 4.4.2 Learning and Inference .................................. 48 4.5 Related Work ........................................... 52 4.6 Experiments ............................................ 53 4.6.1 Datasets .......................................... 53 4.6.2 Competing Models .................................... 53 4.6.3 Results .......................................... 55 4.7 Conclusion and Future Opportunities .............................. 60 5 Learning and Matching in the Constrained Recommendation Framework 63 5.1 Learning and Recommendations ................................ 63 5.2 Matching Instantiation ...................................... 64 5.2.1 Matching Objectives ................................... 65 5.3 Related Work on Matching Expert Users to Items ...................... 67 5.4 Empirical Results ......................................... 68 5.4.1 Data ............................................ 68 5.4.2 Suitability Prediction Experimental Methodology ................... 69 5.4.3 Match Quality ...................................... 69 5.4.4 Transformed Matching and Learning .......................... 74 5.5 Conclusion and Future Opportunities .............................. 75 6 Task-Directed Active Learning 77 6.1 Related work ..........................................