User-Centric Design and Evaluation of Online Interactive Recommender Systems A THESIS SUBMITTED TO THE FACULTY OF THE GRADUATE SCHOOL OF THE UNIVERSITY OF MINNESOTA BY Qian Zhao IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF Doctor of Philosophy Joseph A. Konstan May, 2018 c Qian Zhao 2018 ALL RIGHTS RESERVED Acknowledgements My Ph.D. at the University of Minnesota transformed my way of thinking and changed how I plan to live for the rest of my life. To many people I owe thanks in this process of transformation, which I deeply appreciate. Without their guide, help, patience and love, I would not feel the joy, peace, confidence and hope that I'm full of now. I am so grateful to my advisor Prof. Joseph Konstan for his advising across my Ph.D. His wisdom, his deep knowledge and his kind support for me to freely explore and think as an independent researcher plays a key role to my transformation. I always feel honored and lucky to be his student. I am thankful for his patience in my mistakes and sometimes stubborn insistence. I will miss the meetings with him where it almost always takes less than 30 seconds before he gives me insightful feedback no matter how horrible my presentation was. I was amazed that the depth of the feedback sometimes took me a year to digest (\Aha! That's what he was talking about one year ago!"). I am also grateful for his kindness to support me and also my family (wife and daughter) so that I can not only pursue a Ph.D. but also have a happy life in these five years. I am grateful to my wife Tianjiao Jin for being such a great mother taking care of our daughter. I am grateful to my host family Bonnie and Bob Oleson. We had so many sweet and unforgettable memories. They helped me appreciate the stunning beauty of Minnesota. Most importantly, from them, I got to know God's love for the whole world. I am grateful to Prof. Gedas Adomavicius who spent hours discussing research projects with me and showed me how to think and conduct research rigorously. I am grateful to Prof. Loren Terveen who showed me how to design rigorous scientific experiments in the beginning years establishing a foundation of my Ph.D. research. I am grateful to Research Scientist Dr. F. Max Harper who all the way led me and encouraged me through many of my projects. He taught me how to design and develop i better softwares. I am grateful to Prof. George Karypis who inspired and guided me to think about the sciences of data modeling and computation. I am grateful to Prof. Martijn Willemsen who helped me gain a better understanding on the psychological perspective of recommender system research. I am grateful to Prof. Sashank Varma who guided me in exploring the fields of educational psychology and cognitive sciences. I am grateful to my mentors of several internships, especially Dr. Paul Bennett, Dr. Adam Fourney, Dr. Susan Dumais, Dr. Liangjie Hong, Dr. Shi Yue, Dr. Minmin Chen and Dr. Jilin Chen, who in one way or another shaped my Ph.D. research and my perspectives on the fields. I first heard about GroupLens from my dear friend and mentor Dr. Zhenhua Dong, who led me into the field of recommender systems. I am grateful to him for his kindness, patience and guidance. I am lucky to be a member of the GroupLens research lab. I felt the warm care and support for each other here as a bigger family. I am grateful for and will miss the numerous times I received detailed and constructive feedback from my labmates. I am grateful to the guidance and help of Dr. Shuo Chang. I am grateful to Raghav Karumur who sit next to me in the lab as a companion and has made the past five years full of fun conversations. ii Dedication To my parents Liangren Zhao and Shuange Wang who unconditionally love me and cultivated the foundation of my personality. To my aunt Suoe Wang and her husband Jingshi Ren. To my aunt Liangmei Zhao. To my uncle Jinwei Zhao and his wife Jiqin Shi. To my uncle Liangsi Zhao. To all my relatives who cared and supported me in Jiexiu, China. iii Abstract User interaction is present in all user interfaces including recommender systems. Un- derstanding user factors in interactive recommender systems is important for achieving better user experience and overall user satisfaction. Many prior works in recommender systems consider recommendation as a content selection process and there is not much prior work focusing on studying user interaction, except user on-boarding interaction design, rating interface design etc. Even for the content selection part, however, it seems obvious that there are a fair amount of factors lying in the scope of user interaction as well, to name a few, visual attention and item exposure, perceived temporal change, reactivity, confusion; i.e., factors regarding content browsing in a typical information system. My research studies several factors while real users are interacting with online recommender systems and answers a series of questions regarding those factors. Specif- ically, my research focuses on gaining a better understanding on a) whether users pay attention to grids of recommendations displayed in modern recommender interfaces; b) how to interpret and infer user inaction after we show those recommendations to users and further utilize this inaction model to improve recommendation; c) how to organize and present the top-N recommendations to better utilize user attention and increase user engagement; d) how does recommenders optimizing for being engaging (i.e., as many user interactions as possible) affect user experience compared with recommenders optimizing for being right in estimating user preference and maximizing the preference of users on recommendations displayed; e) how to better support work that combines user-centric design, evaluation and building complex, scalable recommendation models going from offline settings into the online environments of providing interactive real-time responses to user recommendation requests, by building a generic recommender server framework. iv Contents Acknowledgements i Dedication iii Abstract iv List of Tables ix List of Figures xii 1 Introduction 1 1.1 Interactivity of Online Recommender Systems . 1 1.2 Problems and Approaches . 2 1.3 The Research Platform: MovieLens . 5 1.4 Thesis Overview . 5 1.4.1 Do users see the recommendations? . 6 1.4.2 What does user inaction mean? . 7 1.4.3 Should we always show the best? . 7 1.4.4 Accuracy, engagement or satisfaction? . 8 1.4.5 How to support going from offline to online? . 9 2 Related Work 10 2.1 Classical Works in Recommender Systems . 10 2.2 Machine Learning Models . 12 2.3 User Modeling . 15 v 2.4 User-Centric Evaluation . 18 2.5 Recommender Toolkits and Machine Learning Libraries . 20 3 Gaze Modeling in Grid-Based Interfaces 21 3.1 Introduction . 21 3.2 Related Work . 23 3.3 Building Models for the Gaze Prediction Problem . 26 3.3.1 Building Linear Models . 27 3.3.2 Building Hidden Markov Models . 28 3.4 Methods . 29 3.4.1 User Browsing Dataset in MovieLens . 29 3.4.2 Eye Tracking Protocol Design and Dataset . 30 3.4.3 Evaluation . 31 3.5 Results . 31 3.6 Discussion . 35 4 Interpreting User Inaction Feedback 38 4.1 Introduction . 38 4.2 Related Work . 40 4.3 Data Collection . 43 4.4 Interpreting User Inaction . 48 4.5 Future Recommendation . 48 4.6 Classifying User Inaction . 49 4.7 Improve Recommendation . 55 4.8 Discussion . 56 5 Cycling and Serpentining of Top-N Lists 59 5.1 Introduction . 59 5.2 Related Work . 61 5.3 Experiment Design . 64 5.3.1 Measurements . 68 5.4 Results . 70 5.5 Discussion . 73 vi 6 Optimizing for User Interaction 78 6.1 Introduction . 78 6.2 Background of User-Centric Research . 81 6.3 Background of Machine Learning . 82 6.3.1 Empirical Risk Minimization . 82 6.3.2 Matrix Factorization . 83 6.3.3 Q Learning . 84 6.3.4 Regret Minimization . 85 6.3.5 Contextual Bandit and LinearUCB . 86 6.4 Method . 87 6.4.1 The Six Recommenders . 88 6.4.2 Objective Measurements . 93 6.4.3 Subjective Measurements . 94 6.5 Results . 95 6.5.1 Measurements Interpretation . 97 6.5.2 RQ1 . 97 6.5.3 RQ2 . 98 6.6 Discussion . 99 7 A Generic Recommender Server 101 7.1 Introduction . 101 7.2 Related Work . 103 7.3 The Generic Server Design . 105 7.3.1 Recommender Components and Extensibility . 107 7.3.2 Server Interface, Architecture and Scalability . 109 7.3.3 Using the Server . 112 7.4 Case Studies . 115 7.4.1 Extension and Integration . 115 7.4.2 Online Recommender Blending . 118 7.4.3 System-Level Cold-Start . 121 7.5 Discussion . 122 vii 8 Conclusion 124 8.1 Contribution . 126 8.2 Future Work . 128 8.3 Implication . 130 References 131 viii List of Tables 1.1 A toy example of the data that a movie recommender system has. 3 3.1 Different models for the gaze prediction problem, in which bl denotes baseline, ub denotes training only on user browsing data (or training without fixation) and et denotes training on eye tracking data (or training with fixation) as well. 37 4.1 The confusion matrix of the inaction model (rows are the predicted classes and columns are the actual classes) and the accuracy in terms of AUC for each class (binary classification of one vs.