!" #$$ % &$! !'($!!!!!( )*+*, - . !" / 0123 & 3 3 4& + 5 #$$ % &$! !'($!!!!!( Journal of Neuroscience, Psychology, and Economics © 2019 American Psychological Association 2019, Vol. 1, No. 999, 000 1937-321X/19/$12.00 http://dx.doi.org/10.1037/npe0000107 Gaze-Informed Modeling of Preference Learning and Prediction Stephanie M. Smith and Ian Krajbich The Ohio State University Learning other people’s preferences is a basic skill required to function effectively in society. However, the process underlying this behavior has been left largely unstudied. Here we aimed to characterize this process, using eye-tracking and computational modeling to study people while they estimated another person’s film preferences. In the first half of the study, subjects received immediate feedback after their guess, whereas in the second half, subjects were presented with four random first-half outcomes to aid them with their current estimation. From a variety of learning models, we identified two that best fit subjects’ behavior and eye movements: k-nearest neighbor and beauty contest. These results indicate that although some people attempt to form a high- dimensional representation of other people’s preferences, others simply go with the average opinion. These strategies can be distinguished by looking at a person’s eye movements. The results also demonstrate subjects’ ability to appropriately weight feedback in their estimates. Keywords: preference estimation, computational modeling, eye-tracking, attention, individual differences Supplemental materials: http://dx.doi.org/10.1037/npe0000107.supp People often need to learn the preferences of omy, the surrogate must know the recipient’s their friends, family, and colleagues, so as to preferences. An important, unanswered ques- improve their recommendations, gift-giving tion, though, is precisely how a person learns abilities, and, more generally, their relation- another’s preferences. ships. Estimating another’s preferences can be a For example, if you had to recommend a vital task, especially if one should need to make restaurant to someone you barely knew, what a decision on their behalf. Maintaining the in- would you do? You might, for instance, start dividual’s autonomy is heralded as the gold with your personal favorite place. Alternatively, standard when surrogates make decisions for you might suggest the general consensus among another (Minogue, 1996); to preserve auton- your friends. Then, after hearing this diner’s evaluation of your idea, how would you adapt your next recommendation? How would you integrate their feedback with your own knowl- edge and opinions? Clearly, the process in- Stephanie M. Smith, Department of Psychology, The volved in learning another’s preferences is quite This document is copyrighted by the American Psychological Association or one of its allied publishers. Ohio State University; X Ian Krajbich, Departments of nuanced and complex. This article is intended solely for the personal use of the individualPsychology user and is not to be disseminated broadly. and Economics, The Ohio State University. Computer scientists have been developing This work was supported by NSF Career Grant 1554837 machine learning techniques for predicting peo- (to Ian Krajbich) and NSF GRFP Grant DGE-1343012 (to ple’s preferences. For example, Netflix predicts Stephanie M. Smith). We thank R. Gwinn for sharing ex- periment code and A. Hedstrom for film advice. The data movie preferences (and makes suggestions) sets analyzed for this study can be found in the Open based on the feedback it receives. It uses rating, Science Framework (https://osf.io/k7rjz/?view_onlyϭa917 viewing, scrolling, and search behavior as well 8b2938794f7cbd19d7bdf037598e). as the similarities and differences between Correspondence concerning this article should be ad- dressed to Ian Krajbich, Departments of Psychology and watched and unwatched films to determine the Economics, The Ohio State University, 1827 Neil Ave- predicted rating for another film (and to calcu- nue, Columbus, OH 43210. E-mail: [email protected] late, ultimately, whether or not to suggest this 1 2 SMITH AND KRAJBICH other film; Vanderbilt, 2013). There is substan- Previous research has demonstrated the use- tial overlap between these machine-learning fulness of eye-tracking and other process- techniques and the models used in research on tracing data for inferring choice processes (Ai- human learning, ranging from basic reinforce- mone, Ball, & King-Casas, 2016; Ashby, ment learning (Sutton & Barto, 1998) to more Dickert, & Glöckner, 2012; Gharib, Mier, Adol- complex models like k-nearest neighbor (Aha, phs, & Shimojo, 2015; Johnson, Camerer, Sen, Kibler, & Albert, 1991). Thus, these models & Rymon, 2002; Kim, Seligman, & Kable, provide possible frameworks for understanding 2012; Knoepfle, Wang, & Camerer, 2009; Kon- human preference estimation. For example, ovalov & Krajbich, 2016; Lindner et al., 2014; people might expect others to have an underly- Lohse & Johnson, 1996; Polonio, Di Guida, & ing preference schematic—an internal, multidi- Coricelli, 2014; Russo & Leclerc, 1994; Ven- mensional map of films, for instance—where katraman, Payne, & Huettel, 2014). In particu- the similarity between any two films relates lar, this work has demonstrated that information inversely to the difference in preference be- acquisition, as indexed by looking, has a signif- tween them. If people do have such representa- icant impact on people’s choices (Cavanagh, tions of their preferences, then others might try Wiecki, Kochar, & Frank, 2014; Fiedler & to learn (and use) this underlying structure. Glöckner, 2012; Fisher, 2017; Krajbich, Armel, At the same time, the existing research is less & Rangel, 2010; Krajbich, Lu, Camerer, & Ran- than encouraging when it comes to human com- gel, 2012; Krajbich & Smith, 2015; Milosav- petence in predicting others’ preferences. For ljevic, Navalpakkam, Koch, & Rangel, 2012; instance, some findings indicate that people pre- Orquin & Mueller Loose, 2013; Pärnamets et dict others to be more risk-seeking than them- al., 2015; Shimojo, Simion, Shimojo, & selves for gambles over gains and losses (Hsee Scheier, 2003; Smith & Krajbich, 2018, 2019; & Weber, 1997), whereas others show that peo- Stewart, Hermens, Matthews, 2015; Towal, ple predict others’ preferences to be closer to Mormann, & Koch, 2013; Vaidya & Fellows, risk neutrality (Faro & Rottenstreich, 2006). In 2015). Specifically, when subjects spend longer a more applied context, gift-givers often fail to gazing at an option, they gather more evidence give their recipients the preferred gift, even about said option and are subsequently more when the giver and recipient know each other likely to choose it from a set of alternatives; this very well (Givi & Galak, 2017). Perhaps the finding holds even in perceptual judgments (Ta- most alarming research demonstrates that when vares, Perona, & Rangel, 2017). participants are asked to predict whether a fam- In this study, we sought to combine compu- ily member would want life-sustaining treat- tational modeling with eye-tracking data to ment in a variety of health scenarios, their esti- study the preference estimation process. To this mations were largely inaccurate and more end, we studied human subjects as they at- closely mimicked the estimator’s preferences tempted to guess a passive subject’s values for than the recipient’s (Fagerlin, Ditto, Danks, a variety of movies. The experiment consisted Houts, & Smucker, 2001). It is therefore also of two blocks. In the first block, subjects made possible that in a context such as this, people do predictions for 100 movies, receiving feedback not learn or adapt, but instead use heuristics. For about the true value after each guess. In the instance, in predicting the preferences of some- second block, subjects made prediction for This document is copyrighted by the American Psychological Association or one of its allied publishers. one that they do not know well, a reasonable 100 more movies, this time without feedback This article is intended solely for the personal use of the individualstrategy user and is not to be disseminated broadly. might be to guess the average individ- between trials. However, here we provided ual’s preference. With some knowledge about subjects with onscreen feedback about prior the other person, one might instead use the guesses and true values. We tracked their average as the starting point in an anchor-and- eye-movements while they inspected this in- adjustment process (Tamir & Mitchell, 2010). formation and incorporated it into their pre- It seems plausible then that humans might dictions. The aim of the study was to use the utilize some combination of learning techniques choice data from the first block of trials to and simple heuristics. The question is which identify the best-fitting learning model (out of models most closely match peoples’ actual our candidate set) and then to validate this choice processes? classification and characterize the different PREFERENCE LEARNING 3 learning processes using the eye-tracking data jects received immediate feedback (the actual in the second block. WTP) after each estimate. We find that different subjects do seem to use In the second block, subjects estimated the different strategies. With the choice data, we passive subject’s WTP for the remaining 100 show that some subjects mimicked a simple, films from the sample (Figure 1b). These esti- static heuristic to make