The Role of Attention in Preference-Based Choice: Evidence from Behavioral, Neural, and Auditory

Home , Attention Attention, Despacito, Labrinth, Wannabe

The role of attention in preference-based choice: Evidence from behavioral, neural, and auditory

domains

Dissertation

Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the

Graduate School of The Ohio State University

Rachael Gwinn

Graduate Program in Psychology The Ohio State University

2019

Dissertation Committee

Ian Krajbich Ph.D., Advisor

Julie Golomb Ph.D.

Andrew Leber Ph.D.

Copyrighted by

Rachael Gwinn

2019

Abstract

What role does attention play in decision-making? This dissertation will explore this question in three chapters exploring the causality of attention, the neural basis for the effects of attention, and the role of attention in auditory choices.

Prior research has demonstrated a link between visual attention and value-based choice, but the direction of causality is still unclear. Here we aimed to demonstrate that attention has a causal influence on choice. We tested whether spatially biasing attention in a visual search task would produce choice biases in a later choice task. We ran four experiments where the search target was more likely to appear on one “rich” side of the screen. In the subsequent choice tasks, subjects were more likely to choose items appearing on the rich side and the average choice bias depended on how well subjects learned the regularity in the search task. Additionally, eye- tracking data revealed a first-fixation bias toward the rich side, which in turn influenced choices.

Taken together, these results provide novel support for a causal effect of attention on choice.

While attention appears to be causal, little is known about the choice process’s neural substrates or how attention affects the integrated evidence representations. We conducted a simultaneous eye-tracking and fMRI experiment in which subjects gradually learned about the value of two lotteries. With this design we were able to extend decisions over a long period of time, manipulate the time course of evidence, and thus dissociate instantaneous and integrated evidence. We found that instantaneous evidence was represented in ventromedial prefrontal cortex and striatum, while integrated evidence was instead represented in dorsomedial prefrontal

iii cortex and parietal cortex. In line with our computational model, both instantaneous and integrated evidence signals in the prefrontal cortex were modulated by gaze direction, with attended options receiving more evidence and thus higher choice proportions.

Finally, we explored the role of attention in the auditory domain. While we do make visual, preference-based decisions, like what to wear, every day, we don’t live in a purely visual world. We frequently make auditory decisions, like what song to listen to, yet these decision processes are not as well understood. To investigate such processes, including the role of attention, we had subjects complete a binary choice task. On each trial they controlled which of two songs played at any moment and ultimately decided which song they preferred. We found striking similarities between vision-based and auditory-based decisions, namely that attention played an important role in determining which song was chosen. However, unlike in visual choices, these attentional effects appear to be driven by a strong final-fixation effect.

Acknowledgments

I would like to acknowledge my advisor, Ian Krajbich, for all of the help and support (and terrifying jokes) he’s given throughout my graduate program. I would also like to acknowledge my lab mates, in particular Stephanie Smith, for their support, advice, and teamwork. Finally, I’d like to acknowledge my parents, Lynne and Paul Valentich-Scott and Charles and Ann Gwinn, as well as my partner, Nathan Biggs, for the love and support they’ve given me even when I was sure I was going to fail.

Vita

2008...... Dos Pueblos High School

2012...... B.A. Psychology, Linguistics, Italian,

University of California, Davis

2014-2015 ...... Graduate Fellow, The Ohio State University

2015-2016 ...... Graduate Teach Associate, Department of

Psychology, The Ohio State University

2016...... M.A. Psychology, The Ohio State

University

2017-2018 ...... Graduate Research Associate, Fisher

Business School, The Ohio state University

2018-2019 ...... Graduate Teaching Associate, Department

of Psychology, The Ohio State University

Publications

Gwinn, R.E., Leber, A.B., Krajbich, I. (2019). The spillover effects of attentional learning on

value-based choice. Cognition.

Leber, A. B., Gwinn, R. E., Hong, Y., & O’Toole, R. J. (2016). Implicitly learned suppression of

irrelevant spatial locations. Psychonomic Bulletin & Review, 1-9.

Fields of Study

Major Field: Psychology

vii

Table of Contents

Abstract ...... iii Acknowledgments ...... v Vita ...... vi Table of Contents ...... viii List of Tables ...... x List of Figures ...... xi Chapter 1: Introduction ...... 1 The Causal Role of Attention in Choice ...... 5 Attention and Value in the Brain ...... 6 Attention in Audition ...... 7 Chapter 2: The spillover effects of attentional learning on value-based choice ...... 9 Introduction ...... 9 Experiment 1 ...... 14 Materials and Methods ...... 14 Results ...... 19 Experiment 2 ...... 27 Methods ...... 27 Results ...... 28 Experiment 3 ...... 31 Methods ...... 31 Results ...... 32 Experiment 4 ...... 34 Methods ...... 34 Results ...... 35 Discussion ...... 44 Chapter 3: The Neural Computation and Comparison of Value in Simple Choice ...... 49 Introduction ...... 49 viii

Materials and Methods ...... 52 Results ...... 58 Discussion ...... 71 Chapter 4: The role of attention in auditory choice ...... 74 Introduction ...... 74 Materials and Methods ...... 78 Results ...... 81 Discussion ...... 95 Chapter 5: Discussion ...... 99 Limitations ...... 100 Future Directions ...... 102 Conclusions ...... 103 Bibliography ...... 105 Appendix A: Additional Analyses for Chapter 2, Experiment 3 ...... 115 Appendix B: Additional Analyses from Chapter 2, Experiment 4 ...... 118 Appendix C: Additional Regressions for Chapter 2 ...... 119 Appendix D: BRMS Code ...... 129 Appendix E: Song List ...... 131

List of Tables

Table 2.1: Meta-analysis result ...... 43 Table 2.2: No reduction in training effectiveness over time ...... 44

Table C. 1: Experiment 1 Models, Excluding Subjects not Above Chance ...... 119 Table C. 2: Experiment 1 Models, No Exclusions ...... 119 Table C. 3: Experiment 1 Complex Models, Excluding Subjects not Above Chance .... 120 Table C. 4: Experiment 2 Models – Second Choices Only ...... 120 Table C. 5: Experiment 2 Models – Second Choices Only, No Exclusions ...... 120 Table C. 6: Experiment 2 Models – All Choices, No Exclusions ...... 121 Table C. 7: Experiment 3 Models – First Choices Only ...... 122 Table C. 8: Experiment 3 Models – First Choices Only, No Exclusions ...... 122 Table C. 9: Experiment 3 Models – Second Choices Only ...... 122 Table C. 10: Experiment 3 Models – Second Choices Only, No Exclusions ...... 123 Table C. 11: Experiment 3 Models – All Choices, No Exclusions ...... 124 Table C. 12: Experiment 4 Models – First Choices Only, Choice Regressions ...... 125 Table C. 13: Experiment 4 Models – First Choices Only, First Fixation ...... 125 Table C. 14: Experiment 4 Linear Models – First Choices Only, Dwell Time Advantage ...... 126 Table C. 15: Experiment 4 Models – Second Choice Only ...... 126 Table C. 16: Experiment 4 Models – Second Choices Only, First Fixation ...... 127 Table C. 17: Experiment 4 Linear Models – Second Choices Only, Dwell Time Advantage ...... 127 Table C. 18: Experiment 4 Models – All Choices ...... 127 Table C. 19: Experiment 4 Models – All Choices, First Fixation ...... 128 Table C. 20: Experiment 4 Linear Models – All Choices, Dwell Time Advantage ...... 128 Table C. 21: All Studies Combined Models ...... 128

List of Figures

Figure 2.1 Experiment components ...... 17 Figure 2.2: Search task results...... 21 Figure 2.3 Choice bias as a function of attention bias...... 25 Figure 2.4 Bayesian Logistic Regression Results...... 41 Figure 3.1 Task timeline...... 55 Figure 3.2 Example trial with instantaneous and cumulative value differences...... 56 Figure 3.3 Choice data...... 59 Figure 3.4 Regions responding to instantaneous and cumulative value...... 62 Figure 3.5 Beta plots from the vmPFC, striatum, dmPFC, IPS, and dlPFC...... 63 Figure 3.6 Regions responding to gaze-weighted instantaneous (IGWV) and cumulative value (CGWV)...... 65 Figure 3.7 Gaze contrast results...... 68 Figure 3.8 Motor cortex activity reveals the winning lottery...... 69 Figure 4.1 Rating and Choice Tasks...... 80 Figure 4.2 Choice and RT Patterns...... 83 Figure 4.3 Average Fixation Length per Subject...... 84 Figure 4.4 Attention on Choice...... 86 Figure 4.5 Listening Duration Dependent on Rating...... 89 Figure 4.6 Model Fits...... 93

Chapter 1: Introduction

As humans, we make so many decisions throughout our lives that some of them seem to come naturally, such as what to eat for breakfast, what to wear, or what song to listen to on our way to work. Despite the seeming ease behind such decisions, there are many factors that go into our final choice, over and above our preferences for each of the items. One of these factors is attention. In this dissertation, I explore whether attention has a causal role in choice, how attention is related to the neural activity underlying the preference for each choice item, and whether attention also plays a role in the auditory domain.

Attention has been implicated in a wide range of human behaviors, from memory to search tasks (Chun, Golomb, & Turk-Browne, 2011). While attention is modality-independent, that is attention is shared across senses such as vision and audition, it has most often been studied in the visual domain (Eimer, Velzen, & Driver, 2002). Typically, attention is measured using eye tracking (Corbetta et al., 1998). People have been interested in the role of attention in choice for quite some time. In fact, the relationship between attention and preference-based choice was studied even before eye-tracking was readily available. In such studies, researchers often used verbal protocols in which subjects were encouraged to verbalize each thought they had. This would typically allow researchers to track what attributes a subject was attending to during the choice process (Payne, Braunstein, & Carroll, 1978).

The role of attention in choice has been studied in many domains, including risky decisions (Fiedler & Glöckner, 2012; Franco-Watkins & Johnson, 2011; Glöckner & Herbold,

2011; Kim, Seligman, & Kable, 2012; Stewart, Hermens, & Matthews, 2015; Vlaev, Chater, &

Stewart, 2008), social preferences (Fiedler, Glöckner, Nicklisch, & Dickert, 2013; Krajbich,

Hare, Bartling, Morishima, & Fehr, 2015), and choices between goods (Krajbich, Armel, &

Rangel, 2010; Rosen & Rosenkoetter, 1976; Shi, Wedel, & Pieters, 2012; Shimojo, Simion,

Shimojo, & Scheier, 2003; Smith & Krajbich, 2018). While there has been some debate about whether attention influences choice, or choice influences attention, there is an undeniable relationship between the two. In the preference-based choice literature, studies have repeatedly found that increased attention to an item is correlated with choosing that item (Cavanagh,

Wiecki, Kochar, & Frank, 2014; Krajbich & Rangel, 2011; Krajbich et al., 2010; Pärnamets et al., 2015; Shimojo et al., 2003). In fact, several mathematical models have been proposed that take this relationship into account when capturing patterns found in preference-based decision- making.

These models are typically sequential sampling models (SSMs), which assume that, when faced with a decision, people accumulate evidence for each of the choice options over the course of the decision. Once sufficient evidence has been gathered for any one particular item, the choice process terminates, and the item with the most evidence in favor of it is chosen. These types of models have been used in the perceptual decision-making literature to capture patterns often seen in lexical decision tasks, in which subjects decide if a string of letters is a word or not; memory tasks, where subjects decide if the presented word was on a previously studied list or not; random dot kinematogram tasks, where subjects decide which direction the majority of dots are moving in an array of randomly moving dots; brightness discrimination tasks, in which a

2 subject chooses which item is brightest among a set of items; and many more types of tasks

(Ratcliff, 1978, 2002; Ratcliff & McKoon, 1996; Ratcliff & Starns, 2013; Ratcliff, Thapar, &

McKoon, 2004; Shadlen & Shohamy, 2016; Usher & McClelland, 2001). These models have also been adapted to the preference-based decision-making domain (Johnson & Busemeyer,

2005; Roe, Busemeyer, & Townsend, 2001).

Recent literature has begun to include attention into these SSMs — specifically, a version called the Drift Diffusion Model (DDM), which was first introduced to the memory-based decision-making literature by Ratcliff (1978). One specific model that includes attention is the attentional Drift Diffusion Model (aDDM), created by Krajbich et al. (2010). The aDDM adds an additional parameter to the standard DDM, such that the rate that evidence is gathered, the drift rate, is a direct result of not only the values of each choice item, but also a discounting parameter multiplied with the unattended item. This discounting parameter, �, is constrained between 0 and

1. A value of 0 means full attentional discounting. That is, if an item is unattended, it’s as if it doesn’t exist (i.e. no evidence is accumulated for that item). A value of 1 reduces the aDDM to a standard DDM. More information on this can be found in the introduction to Chapter 4.

Importantly, there are two aspects of preference-based choice data that are often associated with the aDDM. The first aspect is that the first fixation is not influenced by the values of the items. The second aspect is that the length of each fixation is not influenced by the values of the items (Krajbich et al., 2010; Smith & Krajbich, 2018). Together, these features of the data suggest that attention plays a causal role in choice, rather than vice-versa. The model gives us mechanism as to how attention has a role in choices: it directly modifies the incoming signals about the items. While this model fits the data well, it may not reflect how humans are actually making their decisions or the true role of attention in those decisions. However,

3 researchers have used brain data to corroborate the assumption that attention modulates the valuations of items.

Brain data from monkeys using single unit recording, and from humans using electroencephalography (EEG) and functional Magnetic Resonance Imaging (fMRI) have been used to study how preference-based decisions are made, even without the added element of attention. Specifically in reference to the neural plausibility of SSMs, and in particular the DDM, several brain regions have been found to correlate with parameters from the model, including drift rate, have been found in both monkeys and humans (Gluth, Rieskamp, & Büchel, 2012;

Gold & Shadlen, 2001, 2007; Hanks & Summerfield, 2017; Hanks, Kiani, & Shadlen, 2014;

Mulder, van Maanen, & Forstmann, 2014; Rodriguez, Turner, Zandt, & McClure, 2015). In addition, specific areas of the brain, mostly located in the pre-frontal cortex (e.g. ventromedial prefrontal cortex (vmPFC)) respond more strongly for higher valued items, reflecting a valuation system in the brain (Bartra, McGuire, & Kable, 2013; Chib, Rangel, Shimojo, & O’Doherty,

2009; De Martino, 2006; Hare, Malmaud, & Rangel, 2011; Hare, Camerer, & Rangel, 2009;

Kable & Glimcher, 2007; Pisauro, Fouragnan, Retzler, & Philiastides, 2017; Polania, Krajbich,

Grueschow, & Ruff, 2014; Polanía, Moisa, Opitz, Grueschow, & Ruff, 2015).

Returning now to the aDDM, there is some evidence that attention modulates the valuation of items, so that items that people attend to show a larger amount of brain activation than unattended items (Lim, O’Doherty, & Rangel, 2011). While the effect of attention has been shown to occur not only behaviorally but also in the brain, there is a lot left to be learned about the exact role of attention in choice; how attention affects valuation on a neural level; and whether attention only affects choices within the visual domain or whether we can observe its effects in other sensory modalities, such as audition.

The Causal Role of Attention in Choice

As previously mentioned, there is a relationship between attention and choice in that attending to an item longer is correlated with the probability of choosing it. Some studies, such as Shimojo et al.’s (2003) gaze cascade effect have not claimed a uni-directional, causal relationship, but rather that attention is both drawn to the preferred item and increases one’s preference for that item, thus leading to more frequent fixations. This translates to an increase in total gaze duration towards the eventually chosen item. Preference-based choice data do not entirely support the latter mechanism. Rather, patterns in the data show that attention is not affected by the values of the items one is choosing between.

There is quite a bit of evidence in support of a causal role of attention in choice. This has been demonstrated with a modeling approach, much like the aDDM, to show that accounting for the natural salience of the items in the decision better predicts choice, as attention allocated to these more salient items biases choices towards them (Towal, Mormann, & Koch, 2013).

Another approach has shown that manually manipulating the brightness of a choice item can bias people to choose it, regardless of their stated preferences (Mormann, Navalpakkam, Koch, &

Rangel, 2012). Others have directly manipulated the duration that each choice item is shown or has manipulated eye movements to favor one item over another. In both cases, these studies have demonstrated an increased probability of choosing the item attended to for longer (Armel,

Beaumel, & Rangel, 2008; Lim et al., 2011; Pärnamets et al., 2015).

However, despite this mounting evidence in favor of a causal role of attention in preference-based choice, it is possible that manipulating the choice environment--either by artificially increasing the brightness of a stimulus or forcing unnatural eye movements-- 5 influences something other than attention. For instance, brighter items may be easier to identify, or subjects may be aware of the manipulation. In the second chapter of this thesis, I will address this concern by manipulating attention outside of the choice environment and demonstrate that there is a causal effect of attention on choice. To do so, I will use an attentional learning paradigm in which subjects are taught, without conscious knowledge, to attend to one side of the computer screen first. This then should induce a bias to choose items presented in a separate choice task which appear on that side of the screen.

Attention and Value in the Brain

Having demonstrated a causal effect of attention in choice, I explore how attention affects the evidence accumulation process proposed by the aDDM in the brain. Namely, I attempt to separate the input that contributes to the accumulated evidence from the total amount of accumulated evidence. Here, I refer to the input as instantaneous value, that is the ratings of each item in the decision. Accumulated evidence can be conceptualized as how close a person is to choosing item A or item B, based on where they are in their decision process. This is the value that evolves throughout time in the decision. In addition, I explore how attention modulates these two types of input. While, as mentioned previously, attention has been shown to affect the value of the instantaneous input (what is currently on the screen), it’s difficult to know if this attentional modulation carries through to the cumulative evidence (Lim et al., 2011).

There has already been some research separating out this instantaneous value signal from a cumulative value signal and has found distinctive areas for each (Hare, Schultz, Camerer,

O’Doherty, & Rangel, 2011). Namely, the ventromedial prefrontal cortex (vmPFC) and striatum have been implicated in instantaneous value computations while the dorsomedial prefrontal 6 cortex (dmPFC) and intraparietal sulcus (IPS) have been implicated in cumulative value computations. However, this study did not look at how attention influenced the brain’s response to the cumulative valuation in these regions. Even more importantly, cumulative value was not directly measured, but rather estimated using model predictions. These predictions were then correlated with brain activity, as opposed to measuring activity in real time as the decision itself unfolded. Thus, in my third chapter, I will instead use fMRI to observe how both instantaneous and cumulative values are represented in the brain as the decision process unfolds in time.

Attention in Audition

A common thread throughout the preference-based decision-making literature is that choices are presented to subjects visually, which greatly helps with measuring attention, in that we can track eye movements. It is also much easier to present visual stimuli to subjects in an fMRI scanner than it is to present most other sensory modalities. However, this raises the question of whether attention affects choices made in other modalities in the same way that it affects choices in the visual domain.

Some studies have looked at the overlap between audition and vision. In one such study, auditory cues were used to facilitate a visual search task, indicating that auditory attention can guide visual search (Knoeferle, Knoeferle, Velasco, & Spence, 2016). Other studies have shown that auditory cues can influence visual, preference-based decisions. For instance, one study used an auditory cue to draw attention to an item during a go/no-go task in which subjects viewed food items and pressed a button whenever they heard a tone. In a later choice task, subjects were more likely to choose items that had been paired with the tone (Schonberg et al., 2014). Still other studies have shown that playing brand-consistent music or jingles in stores can increase 7 sales associated to that brand (Arnd, Claudiu, Karin, & Leder, 2012; North, Hargreaves, &

McKendrick, 1999).

While these studies hint at a role of auditory attention in choice, they retained a visual element. Thus, in the fourth chapter of this dissertation, I will demonstrate that auditory attention plays a role in choices made in the auditory domain, with no visual input related to the choice items.

Chapter 2: The spillover effects of attentional learning on value-based choice

Introduction

One of the most fundamental challenges we face as humans is to efficiently process the information that we are surrounded by. Attention allows us to prioritize behaviorally relevant information while ignoring irrelevant information (Chun et al., 2011; Egeth & Yantis, 1997). An abundance of research suggests that attention interacts with essentially every known cognitive function (Baddeley, Lewis, Eldridge, & Thomson, 1984; Chun et al., 2011; Chun & Johnson, 2011;

Hillyard, Vogel, & Luck, 1998; Hillyard et al., 1998; Kane & Engle, 2000; Woldorff et al., 1993).

Attention is also thought to play a critical role in decision-making, influencing which aspects of a choice problem are evaluated from moment to moment (Roe et al., 2001), though it may also limit our ability to simultaneously compare options (Krajbich et al. 2010). For instance, when deciding what to eat for lunch, we may imagine at one moment what it would be like to eat a cheeseburger while imagining at another moment what it would be like to eat a salad. However, it is still not well understood whether attention causally determines the outcomes of decisions or merely reflects the emerging preference.

Many models of the decision process assume serial processing of information, including seminal models such as satisficing (Simon 1955), elimination-by-aspects (Tversky 1972), decision field theory (Busemeyer & Townsend 1993; Diederich 1997; Roe, Busemeyer, Townsend 2001), fast-and-frugal heuristics (Gigerenzer & Goldstein 1999), and query theory (Weber & Johnson 9

2006). In these models, attention to attributes or alternatives varies over time, influencing the extent to which they affect the final decision.

In many of these models, attention is thought to be attracted to more important or predictive attributes/alternatives (Aschenbrenner, Albert, & Schmalhofer, 1984; Bordalo, Gennaioli, &

Shleifer, 2012; Cassey, Evens, Bogacz, Marshall, & Ludwig, 2013; Khodadadi, Fakhari, &

Busemeyer, 2017; Wallsten & Barton, 1982). At the same time, other (potentially irrelevant) factors such as visual saliency (Mormann et al., 2012), even when made salient after the decision process has begun (Bear & Bloom, 2016), or emotional content (Vuilleumier, 2015) might also attract attention and thus affect the decision outcome.

In a related literature, perceptual fluency, or the ease with which one perceives information, is also thought to influence preferences. Prior studies have demonstrated that positive affective judgments are increased by prior exposure (Zajonc, 1968), primes that facilitate perception

(Winkielman & Cacioppo, 2001), and higher contrast (Reber, Winkielman, & Schwarz, 1998).

Previously ignored stimuli are also devalued (Raymond, Fenske, & Tavassoli, 2003), though it has been argued that this is more likely due to attentional inhibition than perceptual fluency (Fenske

& Raymond, 2006).

To more systematically investigate the link between attention and decision making, some researchers have employed eye tracking. For example, Shimojo and colleagues showed that, over time, gaze tends to shift towards the option that is eventually chosen; a phenomenon referred to as the gaze cascade effect (Shimojo et al., 2003). Because eye position is a reliable overt measure of attentional allocation (Corbetta et al., 1998; Hoffman & Subramaniam, 1995), this effect provides an important demonstration of the interplay between choice and attention.

To model the relationship between attention and choice, Krajbich, Armel, and Rangel

(2010) proposed an attentional drift diffusion model (aDDM) in which evidence for each option is accumulated and compared over time until one item gains sufficiently more evidence than the other. The key, novel feature of this model was that evidence is accumulated more quickly for an item when it is being looked at than when it is not. Using a binary food choice task, the authors demonstrated that the model could quantitatively capture many complex relationships between choices, response times, and gaze data. In particular, it was able to predict the gaze cascade effect without assuming that attention is drawn to the emerging favorite. The aDDM itself is agnostic about the direction of causality, but other features of the data suggest a causal link from attention to choice. For instance, the authors found no correlation between gaze time and independently measured valuations of the items, but they did find that gaze time was predictive of choice.

Other studies have also implicated an important role for eye movements in choice (Ashby,

Dickert, & Glöckner, 2012; Ashby, Jekel, Dickert, & Glöckner, 2016; Cavanagh, Wiecki, Kochar,

& Frank, 2014; Fiedler & Glöckner, 2012; Fiedler, Glöckner, Nicklisch, & Dickert, 2013; Fisher,

2017; Folke, Jacobsen, Fleming, & De Martino, 2016; Franco-Watkins & Johnson, 2011; Glaholt

& Reingold, 2011; Isham & Geng, 2013; Janiszewski, Kuo, & Tavassoli, 2013; Kim, Seligman, &

Kable, 2012; Konovalov & Krajbich, 2016; Kovach, Sutterer, Rushia, Teriakidis, & Jenison, 2014;

Krajbich & Rangel, 2011; Krajbich, Lu, Camerer, & Rangel, 2012; Mullett & Stewart, 2016;

Noguchi & Stewart, 2014; Orquin & Mueller Loose, 2013; Pärnamets, Johansson, Gidlöf, &

Wallin, 2016; Polonio, Di Guida, & Coricelli, 2015; Reutskaja, Nagel, Camerer, & Rangel, 2011;

Russo & Leclerc, 1994; Shi, Wedel, & Pieters, 2012; Stewart, Hermens, & Matthews, 2015;

Tavares, Perona, & Rangel, 2017; Vaidya & Fellows, 2015; Wang, Spezio, & Camerer, 2009;

Willemsen, Böckenholt, & Johnson, 2011). Still, these studies have focused on correlations

11 between visual attention and choices, so they cannot fully address the issue of causality, i.e. whether attention is driving preference or preference is driving attention.

To address this problem, other studies have attempted to influence attention exogenously.

Armel et al (2008) displayed one option at a time and thus were able to manipulate relative exposure times. Participants were more likely to pick the item that appeared on the screen for a longer duration. Lim et al (Lim et al., 2011) used an analogous paradigm, but kept both choice items on the screen and directed gaze using exogenous cueing. Again, items receiving more attention were more likely to be chosen.

Another set of studies attempted to physically alter the salience properties of the stimuli in order to more subtly influence attention. In the first study (Mormann et al., 2012) the researchers increased the brightness of one of the items so that it would be more salient than the other. This manipulation did increase choices for the more salient item, with the strongest effects at shorter presentation durations (on the order of 100 ms). In a follow-up paper, Towal et al (2013) introduced a choice model which takes into account the salience of an item in relation to its surroundings. They found that a model accounting for both salience and value of each item was best able to predict decisions. Still other studies have shown that one can bias choice by prompting participants to decide when their attention has been particularly devoted to one option over the other (Pärnamets et al., 2015), or by making options in one location more valuable than in other locations (Colas & Lu, 2017).

While these studies have made important strides in establishing the causal link between attention and choice, they utilize techniques that directly interfere with the natural choice process

(Armel et al., 2008; Lim et al., 2011; Pärnamets et al., 2015), alter the properties of the choice

12 options (Mormann et al., 2012; Towal et al., 2013), or manipulate participants’ expectations (Colas

& Lu, 2017). Thus, we cannot rule out alternative explanations for the results.

The attention literature has provided several techniques for experimentally manipulating attention. For instance, in probability cueing, targets are presented more frequently in one spatial location compared to others (either a specific location or a general region of the display). This has been shown to influence attentional allocation through shorter reaction times (RT) and eye movements directed towards targets appearing in more probable locations, even after the probability manipulation has ceased (Druker & Anderson, 2010; Geng & Behrmann, 2005;

Jiang, Swallow, & Rosenbaum, 2013; Jiang, Won, & Swallow, 2014).

Here, we aimed to use this attentional learning technique to provide definitive evidence that attention influences choice. We did so by introducing attentional biases without altering the presented choice stimuli in any way, and without unnaturally forcing eye movements or decision times. Instead, we used a separate attentional learning task to induce a spatial bias in attention, and then tested whether that spatial bias would spill over into a later, independent choice task.

In Experiment 1, we aimed to provide evidence that attention causally influences choice by using probability cueing to induce a spatial bias in attention. We hypothesized that this spatial bias would spill over into a later, independent choice task where participants chose which of two food items they would prefer to eat. Moreover, we also hypothesized that the extent of each individual’s attentional learning, as captured by the RT-difference between spatial locations, would predict the size of their subsequent choice bias. In two additional experiments

(Experiments 2 & 3) we investigated whether these spatial biases could be induced or reversed in a second set of food choices following the first food-choice task. These experiments probed the malleability and limits of attentional learning while also controlling for potential baseline spatial

13 biases. Finally, in Experiment 4 we collected eye-tracking data to directly establish that probability cueing affected subsequent eye-movements, and therefore choice.

Experiment 1

Materials and Methods

Participants

42 undergraduate students at The Ohio State University participated in the initial experiment. One participant failed to complete the binary choice task due to insufficient positively rated items. One other participant was excluded due to performing significantly below chance during the visual search task. Participants earned a show-up fee of $7. In addition to this, participants earned an average of $7.59 during the probability manipulation as well as the food item from one randomly selected choice trial.

Apparatus

All images were created and displayed using Matlab (Mathworks) in conjunction with

Psychtoolbox (Brainard, 1997). Participants sat approximately 101 cm away from the screen and used a standard U.S. keyboard to indicate their responses.

Obtaining Value

The first task that participants completed was a rating task (Fig. 2.1A). Participants saw an individual image of each of the snack items (91 in total for Experiment 1) and a rating scale from -10 to +10 in increments of one. Participants used the right and left arrow keys to move the slider on this number scale to indicate their desired rating, at which point they pressed “enter” on the keyboard to confirm. A rating of “-10” indicated that the item was very disliked, “+10” indicated that the item was very liked, and “0” indicated that the item was neither liked nor disliked. Items were presented in a random order and transitioned immediately between ratings.

Food images were procured from a database made available by the Rangel lab (Plassmann,

O’Doherty, & Rangel, 2007) and supplemented with additional images.

Training – Attentional Biasing

In order to manipulate attentional allocation, we used a visual search task (Fig. 2.1B). In this task, we presented participants with 12 rotated L’s (distractors) and one T (target) which was rotated 90° left or right. The task was to report whether the target T was rotated to the left (press the left arrow key) or to the right (press the right arrow key). The key attentional manipulation was that the “rich” side of the display was more likely to contain the target (75%) than the other

“sparse” side (25%).

At the beginning of each trial, participants saw a fixation cross at the center of the screen, which lasted for 1 second. They were instructed to look at the fixation cross until the search display appeared.

In the search display, T’s and L’s were matched for size, each letter subtending .72° visual angle and appearing at a minimum eccentricity of 1.20° from the center of the screen and a maximum eccentricity of 7.23° from the center of the screen. Each T and L was randomly and 15 independently colored blue, red, yellow, or green. Each L was randomly rotated by 90 or 270 degrees and randomly flipped over the horizontal axis or not. To increase search difficulty, we slightly offset the horizontal line segment of each L to make it more similar to a sideways T. A dashed grey line vertically bisected the screen, so as to clearly separate the left from the right, in order to encourage learning. The search display was presented until response or for 8 seconds, whichever came first.

Figure 2.1 Experiment components (a) Rating task: Participants rated each item on a scale from -10 to +10 based on how much they would like to eat the item. (b) Visual search task: participants had 8 seconds to locate a rotated T among rotated Ls and indicate whether the T was rotated 90° to the left or to the right. A correct answer was given a reward of 4 points (c) Binary food-choice task: Participants indicated which of two food items they preferred.

Each participant was assigned a different rich side of the display; 47.5% of the participants had the rich side on the left. While we did not tell participants about the probability manipulation, prior research indicates that most participants do not become aware of it (Druker

& Anderson, 2010; Geng & Behrmann, 2005; Jiang, Swallow, Rosenbaum, & Herzig, 2013).

Participants completed 200 search trials, with a break halfway through. Accurate responses to targets yielded a reward of 4 points. Immediately upon responding, participants were shown how many points they had earned on that trial. The total number of points earned was also displayed at the resting point 100 trials into the task and was displayed again at the end of the task. Points were translated to dollars at the rate of 100 points per dollar. Maximum earnings were $8.

Test – Choice

To test the effects of our attention manipulation we used a binary food-choice task. In each trial, participants saw two food items on the screen and were told to choose the item that they preferred. These choices were incentivized. That is, a trial was drawn at random at the end of the study, and the participant received the item they had chosen on that trial. If the randomly selected food item was not in stock, another trial was drawn at random until we were able to provide the participant with a food item they had chosen.

Choice trials were created by selecting every possible pair of items with an absolute rating difference of 1 or less. Pairs were then randomly selected from this master list, attempting to minimize the number of times any one item was seen. On average, the maximum number of times any one item was seen was 5.9.

As in the search task, a dashed grey line once again appeared down the center of the screen in order to define the left from the right side. Two food items appeared on the screen, each subtending 6.94° and appearing 1.34° from the center (Fig. 2.1C). Participants chose their preferred food item using the left and right arrow keys. Food items remained on the screen until a choice was made. Once a choice was made, a blue outline square appeared around the chosen item for 500ms.

Participants completed 130 binary choice trials. We did not include any items that received a rating less than 0 (we did not mention this to the participants). Additionally, we only used pairs of items with a maximum rating difference of 1; we did this to focus on difficult choices, in which the effects of attention would be most noticeable. The percentage of trials with a rating difference of 0 was 39-44% in all four experiments.

We informed participants at the beginning of the study that they would get to keep the food item that they chose from one randomly selected trial. We could not guarantee that all food items were always available and so if a participant won an item that was out of stock, we randomly selected a new trial. Participants were aware of this possibility.

Results

In this study, the target during the search task was more likely to appear on the rich side

(75%) than the sparse side (25%), which has been shown to increase attentional allocation to the rich side (Geng & Behrmann, 2002). We hypothesized that this spatial prioritization of attention

19 toward the rich side would then carry over into the food choice task, biasing participants’ choices towards the rich side.

Training

First, we analyzed behavior during the visual search task. Mean accuracy was 91.32%

(s.e. = 2.7%) and did not differ between targets on the rich vs. sparse side (mean difference =

.40%, t= 0.48, 95% CI [-1.30%, 2.09%], d = 0.02). RTs were significantly shorter for targets appearing on the rich vs. sparse side (mean difference = 475 ms, t= 6.14, 95% CI [318 ms, 631 ms], d = .85) (Fig. 2.2A). This replicated previous probability cueing results, demonstrating that the participants learned to attentionally prioritize the rich side of the display.

A) B)

C) D)

Figure 2.2: Search task results. Reaction times (RT) from the search task in (a) Experiment 1 (b) Experiment 2 (c) Experiment 3, first search task (d) Experiment 3, second search task, separated into blocks of 50 trials (e) Experiment 4, first search task (f) Experiment 4, second search task, separated into blocks of 50 trials (g) Experiment 4, first search task initial fixations (h) Experiment 4, second search task initial fixations by blocks of 50 trials. Rich and sparse refer to the assignment in the original search task. In (d), (f), and (h) the original sparse side (orange) is now the rich side, and so its RTs, as well as first fixations to the original rich side, decline as participants learn the new assignment. Bars indicate 95% confidence interval.

E) F)

G) H)

Test – Main Effect We next investigated whether there was an effect of the attentional manipulation on the food choice task. Because we were interested in the effect of the attentional manipulation, we excluded participants who did not score significantly above chance during the visual search task

(N = 2); we reasoned that these individuals had not learned the task and thus would not display biases induced by the manipulation.

We used a mixed-effects logistic regression with the probability of choosing the food item from the rich side as a function of the value difference between the rich item and the sparse item (i.e. the item on the rich side minus the item on the sparse side), with random effects for all regressors. In this regression, the value-difference variable indicates how well participants’ choices aligned with their earlier ratings, and the intercept indicates any bias toward choosing the rich item or the sparse item. Specifically, a positive intercept indicates a bias toward choosing the rich item.

The effect of value difference was highly significant (� = 0.42, 95% CI [.33, .51]), indicating that participants’ choices were indeed correlated with their earlier liking ratings. Note that across all experiments, this regressor was always highly significant and so for brevity we omit reporting it in subsequent results sections (although full regression results for all analyses are available in the supplementary material).

In line with our hypothesis, we found a positive intercept (�=0.07, 95% CI [0.02, 0.13]).

This indicates that participants’ decisions were biased toward the rich side of the display (Table

1 in Appendix 1).

Test – Establishing the role of attention

In order to establish that our observed choice bias was due to attention and not other possible factors, we sought to test whether there was a correlation between the degree of attentional learning during the search task and the size of the choice bias during the decision task. For this we used the RT difference between the sparse and rich sides during training.

It was also suggested to us that accuracy during the search task might be predictive of the subsequent choice bias, as it might serve as an alternative measure of learning in the search task.

We also suspected that overall value, that is the sum of the ratings for the left and right items, might affect the choice bias. This last prediction comes from a subtle feature of the aDDM. In the model, evidence for an unattended item is discounted by a factor of �. This means that items with higher values are discounted more in absolute terms, leading to a larger effect of attention on choice.

To best characterize which, if any, of these variables were important in determining final choice, we ran several mixed-effects logistic regressions with all possible combinations of these three variables plus rating difference and intercept and used the Akaike Information Criterion

(AIC) to compare them (see Table 2 in Appendix C). Because we later conducted a meta- analysis across all four experiments, the goal here was simply to rule out variables/models that provided clearly worse fits to the data. Therefore, here (and later for Experiments 2 & 3) we report the results from the most complex model whose AIC fell within 2 of the best-fitting model

(Posada & Buckley, 2004). Later, in the meta-analysis, we focus exclusively on the best-fitting model.

For these analyses we ran the regressions including the participants who did not score significantly above chance on the search task, since they essentially serve as control participants that did not learn the attention manipulation. In other words, it helps to compare participants who did not learn the attentional manipulation to those who did (but see Table 3 in Appendix C for results without these inclusions). We continued to exclude participants who scored significantly below chance (N=1), as these participants seem to have learned the manipulation but were not following directions.

A) B)

Figure 2.3 Choice bias as a function of attention bias. (a-d) Mean probability of choosing the food item from the rich side of the display, as a function of the RT difference from the search task. Subjects were split into five even-sized groups (quintiles) within each experiment. (e) Subjects were then combined across experiments. In other words, the far-right bin represents the most attentionally biased 20% of the subjects from Experiment 1, 20% of the subjects from Experiment 2, 20% of the subjects from Experiment 3, and 20% of the subjects from Experiment 4. We observe a fairly consistent increase in the choice bias as the RT difference increased. (f) Probability of choosing from the rich side for each study. Bars indicate 95% confidence intervals.

C) D)

E) F)

Our AIC rule selected the model that included both accuracy and RT difference, in addition to rating difference and intercept. RT difference and accuracy, both proxies for the degree of attentional learning, positively predicted choosing from the rich side, though RT difference was not quite significant (� = 0.11, 95% CI [-0.004, 0.22] and � = 0.64, 95% CI

[0.21, 1.06] respectively). In other words, participants who displayed bigger learning effects in the search task were more biased in their later food choices (Fig. 2.3A).

After accounting for the degree of attentional learning, we should not expect any remaining choice bias. Indeed, this more complex model yielded a negative intercept (� = -0.58,

95% CI [-0.99, -0.18]). Because the accuracy variable was coded from 0 to 100%, with 50% being chance level, the negative intercept in this complex model simply indicates that, in theory, a participant with 0% accuracy would be biased towards choosing from the poor side. In other words, this merely confirms the effect of accuracy on choosing from the rich side.

Experiment 2

In this pre-registered experiment, we sought to replicate our first experiment and test whether the attentional bias could be detected relative to participants’ baseline behavior on a choice task prior to the search task. It was unclear whether the attentional manipulation would still succeed in this context, since participants might become less susceptible to the manipulation after already going through the choice task. To preview the results, we do not replicate the choice bias, though the effect is in the predicted direction. We do however observe some new effects when comparing pre- and post-training choices.

Methods

Participants

As detailed in the pre-registration of this study, we used a power analysis to determine that we should use a sample size of at least 135 participants to achieve power of 0.9 (Gwinn,

Krajbich, & Leber, 2016, February 24). We ran these experiments in a 30-person experimental 27 lab at the Ohio State University and our stopping rule was to invite full sessions until we had run

135 participants. We ended up with 163 undergraduate students. Nine participants were excluded for having too few positively rated food items. An additional 3 were excluded due to computer crashes. One final participant was excluded from all analyses for scoring significantly below chance on the search task. Participants earned an average of $7.89 during the search task.

Task

The procedure for this experiment was the same as Experiment 1, except as noted below.

The main difference from Experiment 1 was that between the food rating task and the visual search task, participants completed 130 trials of the binary choice task. This was done to establish participants’ baseline behavior on the choice task. In the second, manipulated choice task (also 130 trials), we made sure to not repeat any of the pairings from the first choice task.

To accommodate these additional choice trials, we expanded the number of food items to 147.

49% of the non-excluded participants had their rich side on the left. On average, the maximum number of times any one item was seen was 6.81.

Results

Training

As in Experiment 1, we first analyzed the results from the search task (section 2.2.1).

Average accuracy was 96.28% (s.e. = 0.65%) and was greater for targets on the rich side (mean accuracy difference = .99%, t = 3.35, 95% CI [0.41%, 1.60%], d = 0.12). As expected, RTs to items appearing on the rich side were significantly shorter (mean difference = 517 ms, t = 13.99,

95% CI [444 ms, 590 ms], d = 0.97), replicating the probability-cueing effect (Fig. 2.2B). This again indicates that participants learned the attentional manipulation.

Test – Main Effect

Next we wanted to know if the attentional manipulation influenced choices. To do this we first looked at only the second choice task, which occurred after the attentional manipulation.

This analysis simply replicated that from the first experiment. As before, we ran a mixed-effects logistic regression of the probability of choosing the rich item, controlling for the rating difference between the items on the rich and sparse sides. In contrast to Experiment 1 the intercept was not significant (� = 0.011, 95% CI [-0.03, 0.05]) (Table 4 in Appendix C). Hence, there was no significant choice bias in this study.

Test – Establishing the role of attention

To investigate further, we again expanded our regression models to include RT difference, accuracy, and overall value as independent variables. Here the AIC-selected model included only RT difference as an additional predictor (Table 5 in Appendix C). RT difference did not significantly predict choosing from the rich side, though the effect was in the predicted direction (� = 0.06, 95% CI [-0.02, 0.14]) (Fig. 2.3B). Again, after accounting for the degree of attentional learning, we expect and find no remaining choice bias (� = -0.02, 95% CI [-0.07,

0.04]).

Test – Comparing pre-training and post-training

In accordance with our registration for this study, we ran a mixed-effects logistic regression of the probability of choosing the rich side on every combination of the following parameters: rating difference, RT difference, accuracy, overall value, and treatment (a binary variable coding for whether choices occurred before (0) or after (1) the search task), as well as models including two and three way interactions with the treatment variable.

Out of these models, the regression with rating difference, overall value, treatment, and the interaction between treatment and rating difference produced the best fit (Table 6 in

Appendix C). There was no significant coefficient on the intercept (� = 0.032, 95% CI [-0.01,

0.08]), overall value (� = -0.003, 95% CI [-0.006, 0.001]), or on treatment (� = 0.004, 95% CI [-

0.04, 0.05]). However, there was a significant negative interaction between treatment and rating difference (� = -0.15, 95% CI [-0.20, -0.09]), indicating that after the search task, participants were less likely to choose the higher rated foods.

This last result suggests that our attentional manipulation did have an effect on participants’ food choices, though not in the exact same way as in Experiment 1. One concern is that the increased delay between the rating task and the second choice task (and possibly fatigue) could be responsible for the less consistent choices. To test these competing hypotheses we again included the RT-difference variable as a way to capture the amount of learning during the search task. We reasoned that if the RT-difference variable modulated the effect of the treatment on choice consistency, then that would support the hypothesis that the change in behavior was due to the treatment and not due to time delay or fatigue.

Indeed, in this analysis we see that there was a significant, three-way, negative interaction between rating difference, RT difference, and treatment (� = -0.16, 95% CI [-0.30, -0.02]). This means that after the search task, participants who displayed a bigger attentional bias were less 30 likely to choose the higher rated items. All other coefficients and interactions were non- significant (intercept: � = 0.01, 95% CI [-0.05, 0.07]; treatment: � = -0.3, 95% CI [-0.03, 0.09];

RT difference: � = -0.01, 95% CI [-0.09, 0.07]; rating difference x RT difference: � = 0.04, 95%

CI [-0.08, 0.16]; treatment x RT difference: � = 0.07, 95% CI [-0.03, 0.17]; treatment x rating difference: � = -0.06, 95% CI [-0.14, 0.02]).

Experiment 3

Methods

Participants

126 undergraduate students at the Ohio State University participated in this study, as outlined in a second pre-registration (Gwinn, Krajbich, & Leber, 2016, March 14). Two participants were excluded from the analyses for having an insufficient number of positively rated items. Two additional participants experienced computer crashes and were unable to finish the study.

To compensate for the longer experiment, participants in this experiment earned a show- up fee of $5 for the study in addition to an average of $15.79 during the search tasks (as well as one food item from a randomly selected trial).

Task Order

The procedure for this experiment was the same as Experiment 1, except as noted below.

The main difference from Experiment 1 was that after completing the rating task, the search task and then the 130 trials of the choice task, participants then completed another 200 trials of the search task with the rich side reversed from before, so that if it originally was on the

31 left, it would now be on the right, and vice-versa. This has been shown to eliminate attentional bias effects (Jiang, Swallow, Rosenbaum, et al., 2013). This was followed by another 130 trials of the choice task. As in Experiment 2, we made sure to not repeat any of the pairings from the first choice task and we used 147 food items. In this experiment exactly 50% of the participants had their rich side on the left during the first search task. On average, the maximum number of times any one item was seen was 8.2.

Results

Training

As before, we describe the results from the search tasks first. Mean accuracy during the first search task was 96.66% (s.e. = 0.68%) and was significantly higher on the rich side (mean accuracy difference = 0.68%, t = 2.93, 95% CI [0.22%, 1.14%], d = 0.09), unlike Experiment 1 but replicating Experiment 2. As before, RTs showed a significant difference between sparse and rich sides (mean difference = 492 ms, t = 10.74, 95% CI [402 ms, 583 ms], d = 0.98) (Fig. 2.2C).

In the second search task, accuracy was 97.57% (s.e. = 0.72%) and was not significantly different between sides (mean accuracy difference = 0.26%, t=1.35, 95% CI [-0.12%, 0.63%], d

= 0.03). For simplicity we will always refer to the rich and sparse sides based on how they were assigned in the first search task. In other words, for one participant the “rich” side label would be the left side in both the first and second search tasks, despite the actual reversal. As predicted, the RT difference between the sparse and rich sides disappeared (mean difference = 1.91 ms, t =

0.042, 95% CI [-86 ms, 90 ms], d = 0). For a more fine-grained look at the extinction of the RT effect we examined behavior in the second search task in blocks of 50 trials. Participants remained significantly faster at detecting targets on the rich side for the first 50 trials (mean

32 difference = 266 ms, t = 5.06, 95% CI [162 ms, 370 ms], d = 0.54), were not significantly faster in either direction for the next two blocks of 50 trials (mean difference = 10 ms, -10ms, t = 0.17,

-1.74, 95% CI [-108 ms, 128 ms], [-210 ms, 14 ms], d =0.02, -0.2), and were significantly faster for targets on the sparse side for the last block of 50 trials, (mean difference = -173 ms, t = -3.84,

95% CI [-263 ms, -84 ms], d = -0.37) (Fig. 2.2D).

Test – Main effect

Next, we turned to the choice data. Looking only at the first 130 choice trials, and using data taken from only the first search task, we ran the same mixed-effects logistic regression, looking at the probability of choosing from the rich side based on rating difference. As before, anyone with search accuracy not significantly above chance was excluded (N = 3). Here we found that the intercept was marginally significant (� = 0.04, 95% CI [-0.005, 0.09]) (Table 7 in

Appendix C).

Test – establishing the role of attention

To investigate further, we again expanded our regression models to include RT difference, accuracy, and overall value as predictors. As in the prior experiments, the three excluded participants were re-included in this analysis. Here the AIC-selected model included overall value and RT difference as additional predictors (Table 8 in Appendix C). RT difference, our measure of attention, was in the correct direction but not significant (� = 0.06, 95% CI [-

0.03, 0.15]) (Fig. 2.3C). Overall value was marginally significant (� = 0.005, 95% CI [-0.0003,

0.01]). Again, accounting for the degree of attentional learning, we expect and find no remaining choice bias (� = -0.04, 95% CI [-0.11, 0.04]). 33

Test – Post Extinction

We next investigated how these same regressions differed after the reversed search task.

First, we ran the simple regression of choosing the original rich side on rating difference, again excluding those participants who did not perform above chance on the search task (N=3). The bias to choose the original rich side disappeared (in fact slightly changed sign) (intercept � = -

0.005, 95% CI [-0.066, 0.056]) (Table 9 in Appendix C).

Additional regressions including RT-difference from the two search tasks paint a complex picture and are described in detail in the supplements. Also, direct comparisons between the two choice tasks failed to reveal any significant differences (see Appendix A and

Table 10-11 in Appendix C).

Experiment 4

In the fourth experiment, we collected eye-tracking data to test whether the effects we had found in the prior experiments were indeed due to attention. We replicated the procedure for

Experiment 3, testing whether the probability cueing task would bias participants’ first fixations or dwell times towards the rich side of the display.

Methods

Participants

43 undergraduate students at the Ohio State University participated in this study. Eight participants were excluded for having an insufficient number of positively rated items, leaving us

34 with 35 valid participants, as outlined in our pre-registration (Gwinn, Krajbich, & Leber, 2017,

August 11).

As in Experiment 3, participants in this experiment earned a show-up fee of $5 for the study in addition to an average of $15.88 during the search tasks (as well as one food item from a randomly selected trial).

Task Order

This study was identical to Experiment 3 in task order, with the addition of eye-tracking during both search and choice tasks. Participants’ left-eye movements were recorded at 1000 Hz using an Eyelink 1000 Plus (SR Research, Osgoode, ON, Canada) eye-tracker, located 40.5 cm in front of the participant. We used a chinrest provided by the manufacturer to minimize head movement. All stimuli were presented on an LCD monitor (24’ XL2420TE, BenQ), located 79 cm in front of the participant. After the food rating task, participants were calibrated using the standard nine-dot calibration procedure provided by the manufacturer.

In this experiment 46% of the participants had their rich side on the left during the first search task. On average, the maximum number of times any one item was seen was 14.32.

Results

Training

We first checked the accuracy during the first and second search tasks, as in Experiment

3. During the first search task, mean accuracy was 96.29% (se = 1.50%) and did not differ between the rich and sparse side in a two-sided t-test (mean accuracy difference = 0.69%, t =

1.23, 96% CI [-0.45%, 1.82%], d = 0.21). In contrast to accuracy, but in line with all of the prior

35 studies, RT did significantly differ between the rich and sparse sides during the first search task

(mean difference = 426ms, t = 5.94, 95% CI [280 ms, 572 ms], d = 1.003) (Fig. 2.2E).

In line with (Jiang et al., 2014), we found that participants were significantly more likely to look at the rich side first during the search task (mean proportion of trials = .66, t = 4.37, 95%

CI [.59, .74]) (Fig. 2G). This confirms that the attentional biasing indeed affected participants’ eye movements.

In the second search task, accuracy was 99.01% (se = 0.24%) and did not differ between sides (mean accuracy difference = 0.32%, t = 1.33, 95% CI [-0.17%, 0.82%], d = 0.22). As in

Experiment 3, the terms “rich” and “sparse” always refer to the original rich and sparse sides unless otherwise specified. As expected, and in line with Experiment 3 (section 4.2.1), the RT difference between the rich and sparse side was now negligible (mean difference = 34 ms, t =

0.50, 95% CI [-103 ms, 171 ms], d = 0.09). When we break the RT difference down to 50-trial blocks, we see that participants quickly un-learned the original training within the first 50 trials, as there was no significant difference in RT between sides (mean difference = 162 ms, t = 1.62,

95% CI [-366 ms, 41 ms], d = -0.27). By the final 50 trials, participants had learned the new probability structure and the RT difference reversed (participants were faster to respond to targets on the sparse side), although it was still not significantly different from zero (mean difference = 153 ms, t = 1.70, 95% CI [-30 ms, 338 ms], d = 0.29) (Fig. 2.2F).

In a similar fashion, first fixations showed no bias towards either side until the final block of 50 trials where they were biased towards the sparse side (mean proportion of trials in the final block = 0.59, 95% CI [0.51, 0.67]) (Fig. 2.2H). Thus, participants’ eye-movements mirrored their RT differences in the search tasks.

Test – Main effect

Looking at only the first 130 choice trials, we ran a simple mixed-effects logistic regression of fixating the rich side first on rating difference. Here we find no significant effect of rating difference on first fixations (� = 0.020, 95% CI [-0.064, 0.104]), which is in line with previous literature (Krajbich et al., 2010).

Importantly, we do find a significant bias to fixate the rich side first, as evidenced by a positive intercept (� = 0.335, 95% CI [0.023, 0.647]), indicating that the attentional bias did transfer to the choice task (Table 13 in Appendix C).

We next investigated how this first-fixation bias translated into behavior by running the same mixed-effects logistic regression as that in Experiment 3: choosing the rich side on rating difference. Participants showed a significant bias to choose the item on the rich side (intercept �

= 0.087, 95% CI [0.028, 0.146]) (Table 12 in Appendix C).

Test – Establishing the role of attention

While we have shown that our manipulation biases both first-fixation location and choice, it is not yet clear that there is a trial-level effect of first-fixation location on choice. To establish the link between first fixation and choice, we ran another mixed-effects logistic regression of choosing the rich side on rating difference and first-fixation location. Critically, the effect of first-fixation location on choice was strongly positive (� = 0.354, 95% CI [0.215,

0.493]), and the choice bias for trials with the first fixation to the sparse side was negative (� = –

0.114, 95% CI [–0.220, –0.008]).

It is important to note at the outset that the following analyses are underpowered, since the sole aim of this experiment was simply to establish the eye-tracking effects. With only 35 37 participants, we should not expect to reliably demonstrate the individual-difference effects seen with greater samples sizes in the previous experiments. Nevertheless, for completeness, we report the results of those analyses here.

Focusing on the first 130 choice trials, we added the RT difference from the first search task into the aforementioned regressions. We first investigated whether those who learned the probabilities in the first search task, as measured by a larger RT difference, also demonstrated more of a first-fixation bias while controlling for the rating difference. Indeed, we found a strong positive relationship between RT difference and the first-fixation bias (� = 0.819, 95% CI

[0.143, 1.495]) (Fig. 2.3D). This indicates that those who better learned the attention manipulation were more likely to look at the rich side first during the choice task.

As before, the rating difference did not predict first fixations (� = 0.020, 95% CI [-0.064,

0.104]). The intercept also becomes non-significant (� = -0.005, 95% CI [-0.401, 0.391]), as expected after accounting for the degree of attentional learning (RT difference) (Table 13 in

Appendix C).

We looked at whether this pattern of results held for the choice behavior as well by running the mixed-effects logistic regression of choice on rating difference and RT difference.

The RT difference effect was in the predicted direction, though it did not reach significance (� =

0.136, 95% CI [-0.062, 0.334]). Again, we expect and find no overall choice bias (� = 0.036,

95% CI [-0.064, 0.136]) after accounting for the degree of attentional learning (Table 12 in

Appendix C).

Test – Post Extinction

We next ran these same analyses on the second set of 130 choice trials. When predicting first fixations from rating difference, there is again no significant effect of rating difference (� =

0.012, 95% CI [-0.086, 0.110]), and participants no longer showed a significant tendency to fixate the rich side first (� = 0.175, 95% CI [-0.141, 0.491]). Similarly, choice behavior was also no longer significantly biased towards the rich side (intercept � = 0.015, 95% CI [-0.079, 0.109])

(Tables 15-16 in Appendix C).

As with Experiment 3, additional regressions including RT difference from the two search tasks paint a complex picture and are described in detail in Appendix B. Again, direct comparisons between the two choice tasks failed to reveal any significant differences (Tables 18

– 20 in Appendix C).

Dwell time effects

In accordance with our pre-registration, we also investigated whether there were biases in the total dwell time advantages to the rich side in either choice task. None of these effects approached significance, and we report them in the supplements (Tables 14, 17, & 20 in

Appendix C). Moreover, accounting for the dwell time advantage in the choice regressions did not reduce the choice bias, if anything it increased it (Table 12 in Appendix C). These results confirm past findings on the effect of dwell time on choice but indicate that our attention manipulation operated through a different mechanism, namely an initial bias due to first-fixation location.

Pooled data analysis

A recent paper by Scheibehenne et al. (Scheibehenne, Jamil, & Wagenmakers, 2016) highlighted a method to study the consistency of results across studies. This type of analysis is useful for interpreting our data, given the fact that our effects were consistently in the same direction, but in some cases were not statistically significant. Here we present such an analysis, in which we used only the one choice task from Experiment 1, the second choice task from

Experiment 2, and the first choice task from Experiment 3 and 4, since these were the tasks where we expected participants to choose more often from the rich side. All analyses were run using the BRMS package for R (For BRMS code, see Appendix D).

First we tested the simple model from each study, which is the mixed-effects logistic regression of choosing from the rich side as a function of the rating difference. Here we focused on the intercept and computed the Bayes factor testing whether participants had a bias to choose from the rich side vs. the alternative hypothesis of no bias. In order to avoid any biasing of the outcome, we used improper, uniform distributions over all real numbers as the priors on the intercept and rating difference coefficients.

Considering all four tasks, the Bayes factor for the intercept was 332. According to

Scheibehenne et al. (2016), this indicates “extreme” evidence for a choice bias (see Fig. 2.4A for

40 the evolution of the Bayes factor across studies; also Table 21 in Appendix C). A more traditional frequentist t-test was also significant (p=0.005) (Fig. 2.3F).

A) B)

Figure 2.4 Bayesian Logistic Regression Results. Mixed-effects Bayesian logistic regression results from experiments 1, 2, 3 and 4 combined cumulatively. (A) Bayes Factor of the intercept being greater than 0 from the simple model p(choose rich side) ~ Rating Difference. (B) Bayes Factor of each coefficient of the full model p(choose rich side) ~ Rating Difference + Accuracy + Overall Value + RT Difference. Rating Difference is not shown here as the Bayes Factor was 9999 across all four studies.

Establishing the role of attention

Next we tested a more complex version of the model, which included all the variables that were selected by our AIC comparisons in any of the experiments. This model thus included rating difference, accuracy, overall value, and RT difference. We again computed the Bayes factors for each variable being greater than zero and used uniform distributions over all real numbers as the priors on all the variables.

Considering all four tasks, the overall Bayes factors for the intercept, rating difference, accuracy, overall value, and RT difference were (respectively): 0.48 (anecdotal evidence 41 against), 9999 (extreme), 2.04 (anecdotal), 1.194 (anecdotal), and 141.857 (extreme) (Fig 2.4B).

Thus we find extreme support for only rating difference and RT difference. A frequentist t-test on RT difference was also significant (p=0.015) (Fig. 2.3E). Again, we expected the zero-to- negative intercept in this model after accounting for RT difference and accuracy.

This conclusion is supported when we compare AIC values for all the combinations of these variables in the meta-analysis (Table 2.1). From the pooled analysis we see that the best- fitting model includes only rating difference and RT difference. This confirms the overarching story that our search task had a significant effect on participants’ food choices and that this effect was modulated by their attention bias (Fig. 2.4B). We see that for participants in the top 20% in terms of RT difference, their choice bias was 2.3%. To get a sense for the subjective magnitude of this effect, we went back to the original study by Krajbich et al. (2010), which used a very similar binary food-choice task. For a proper comparison with our current study, we focused solely on trials with the same range of ratings differences (-1 to +1). In those trials, participants looked at the left item first on 75.4% of trials and chose the left item on 52.3% of trials. Thus, for our most influenced participants, it seems that with our simple 8-minute search task, we were able to erase or double (depending on the rich side) their average ‘left-bias’, a bias that has developed over their entire lives.

Table 2.1: Meta-analysis result

Dependent Intercept Rating Overall Training Training RT AIC Variable (Bias) Diff. Value Accuracy Diff. 0.034 0.401 61255 (0.009, 0.059) (0.370, 0.432) 0.399 61327 (0.368, 0.430) 0.033 0.401 0.001 61253 (-0.004, 0.070) (0.370, 0.432) (-0.032, 0.034) -0.092 0.401 0.130 61256 (-0.376, 0.192) (0.370, 0.432) (-0.166, 0.426) -0.002 0.401 0.074 Choose Rich Side 61252 (-0.037, 0.033) (0.370, 0.432) (0.021, 0.127) -0.108 0.401 0.001 61256 (-0.384, 0.168) (0.370, 0.432) (-0.032, 0.034) -0.005 0.401 0.002 0.074 61249 (-0.049, 0.041) (0.370, 0.432) (-0.031, 0.035) (0.021, 0.127) -0.111 0.041 0.113 0.073 61255 (-0.385, 0.163) (0.370, 0.432) (-0.171, 0.397) (0.022, 0.124) -0.131 0.401 0.002 0.132 0.074 61252 (-0.407, 0.145) (0.370, 0.432) (-0.031, 0.035) (-0.154, 0.418) (0.023, 0.125)

Decay over trials

A natural question to ask is whether the effects of our attention manipulation decay over time. That is, are participants’ choices more spatially biased at the beginning of the choice block, compared to at the end? We examined this in each dataset by running mixed-effects logistic regressions predicting choice of the rich-side item as function of rating difference and trial number. As before, we used only the one choice task from Experiment 1, the second choice task from Experiment 2, and the first choice task from Experiment 3 and 4, since these were the tasks where we expected participants to choose more often from the rich side. We find no evidence of a reduction in choosing the rich side over trials, in any dataset (Table 2.2).

Table 2.2: No reduction in training effectiveness over time

Dependent Variable P(Choose Rich Side) Exp. 1 Exp. 2 Exp. 3 Exp. 4 Intercept 0.106* 0.022 0.070** -0.009 (0.058) (0.029) (0.033) (0.062) Rating Difference 0.418*** 0.352*** 0.447*** 0.492*** (0.038) (0.025) (0.028) (0.040) Trial Number -0.005 -0.002 -0.005 0.013 (0.008) (0.004) (0.005) (0.009) Observations 4,940 19,110 15,470 4,420 Log Likelihood -3,358.947 -13,065.730 -10,482.330 -2,979.034 Alaike Inf. Crit. 6,729.894 26,143.460 20,976.660 5,970.068 Bayesian Inf. Crit. 6,768.925 26,190.610 21,022.540 6,008.431 Note: *p<0.1; **p<0.05; ***p<0.01

Discussion

Here we have shown that manipulating spatial attention can influence which item a person will choose. In four separate experiments we manipulated attention using a spatially biased visual search task and then tested for a corresponding spatial bias in a subsequent food- choice task. Across the four experiments we found varying degrees of evidence for the hypothesized choice bias, which while small in size, was 332 times more likely to exist than to not, according to our Bayesian meta-analysis. We were also able to establish how well participants learned the attentional manipulation by measuring their spatial RT difference during the search task. This measure fully determined the effect of the search task on the choice behavior. It is worth noting that these effects are clearly driven by our manipulation and not

44 reflective of any naturally occurring spatial biases, as the biases were randomly assigned and disappeared after the reversed search task (Experiment 3-4).

In addition to the RT effect, we also presented eye-tracking evidence, which mirrors the results of prior work using probability cueing, namely that it biases initial fixations (Jiang et al.,

2014). Consistent with our current results, past research with the aDDM has demonstrated an effect of the first fixation on choice. For example, participants who are more likely to look left first are more likely to choose items on the left. Interestingly, here first fixations did not influence choice via dwell time, perhaps suggesting a non-linearity in the decision process that favors early information. Such non-linearities are a feature of several prominent sequential sampling models, including the Ornstein-Uhlenbeck model (Ratcliff & Smith, 2004) and the

Wang model (Wang, 2002), though notably not the standard DDM. It is worth noting that some prior research with the aDDM has indicated that the effect of the first fixation may be larger than predicted by the model (Krajbich et al., 2010, but see Krajbich & Rangel, 2011 and Cavanagh et al., 2014).

Our results provide novel evidence for a causal effect of attention on choice. From a modelling standpoint, the aDDM merely captures a mathematical relationship between gaze and choice; it is agnostic about the direction of causality. Its key insight is that the value information being sampled over the course of the decision is not i.i.d., as assumed by most other sequential sampling models (though not decision field theory). Instead, value information is preferentially biased towards one alternative during certain time intervals, and these biases are tied to gaze. It could be that gaze merely reflects the shift in information sampling, or it could be that gaze causes the shift in sampling.

Proponents of the gaze-cascade effect have argued that both mechanisms are at play, with preference driving gaze (Shimojo et al., 2003). Proponents of the aDDM have instead argued that it is primarily gaze that drives preference, based in part on a lack of correlation between dwell times and independently measured stimulus values (Krajbich et al., 2010). Either way, we would expect that gaze manipulations would influence choice.

While a few papers have provided such evidence, ours overcomes some of their potential limitations and extends the scope of this link to temporally distinct settings. Prior work has manipulated gaze time (Armel et al., 2008; Lim et al., 2011; Shimojo et al., 2003), prompted a decision once a certain gaze criterion is fulfilled (Pärnamets et al., 2015; Tavares, Perona, &

Rangel, 2017), made certain items more visually salient (Mormann et al., 2012; Towal et al.,

2013) or systematically placed better items on one side of the screen (Colas & Lu, 2017). While it is reassuring that all of these manipulations lead to the predicted choice biases, there are some concerns with each of these manipulations. The primary concerns include interfering with the natural choice process, altering the stimuli, and manipulating value expectations.

Interfering with the natural choice process might be problematic, because while we control which item the subject is looking at, this does not mean that we control which item the subject is attending to. The subject may be forced to look at Coke, but still be focused on Pepsi.

We see this possibility as very unlikely when subjects are free to look at Coke and Pepsi as they please. Related literatures have worried about the effects of interfering with cognitive processes, for example using Mouselab to reveal information with mouse clicks (Lohse & Johnson, 1996), or “think-aloud” paradigms (Leow & Morgan-Short, 2004). Thus, there is precedent to be concerned about experimenter manipulations of the choice process. Our study avoids this issue by placing no constraints on our participants during the choice tasks.

Altering the stimuli themselves may be problematic for the simple reason that it might facilitate identification (e.g. through perceptual fluency), explaining why, for example, people greatly prefer the salient item under very short exposure, but less so with longer exposure

(Mormann et al. 2012). We know that people are generally uncertainty-averse, and so if faced with two items, one known and one unknown, people will generally prefer the known item. Our study avoids this issue by placing no time constraints on our participants, allowing them time to identify both alternatives.

Manipulating value expectations is problematic because adaptive decision makers should develop a response bias, based on their belief that this side of the screen is more likely to contain the better option. Our study avoids this issue by randomly assigning options to the rich and poor sides during the choice task, and by holding constant the value of finding a target on each side during the search task.

An additional feature of our study is that it demonstrates that attentional biases developed in one setting can have impacts in other settings. For example, cultural differences in reading direction might lead to biases in whether people look left or right first, which in turn would influence how often they choose items on the left.

One potential concern with our study, as well as results from other attention-manipulation studies, is experimenter demand. It is possible that participants thought that they should choose items on the rich side of the display. We took steps to reduce this possibility by incentivizing the decisions, which is a standard approach for combatting demand effects in experimental economics. We also avoided interleaving the search and choice tasks, to minimize the likelihood that subjects would form a connection between the two. Finally, our analyses demonstrated that

47 measures of attentional biasing (training RTs and initial fixations) predicted the size of subjects’ choice biases, arguing against a simple response bias.

Our results are also consistent with the literature on perceptual fluency. That literature has argued that the ease with which information is perceived, influences preferences. Factors that facilitate perception such as prior exposure, primes, or visual contrast, appear to affect preference judgments (Reber et al., 1998; Winkielman & Cacioppo, 2001; Zajonc, 1968). Perceptual fluency might be one the mechanisms by which attention increases the rate of evidence accumulation and thus biases choices. The distractor devaluation effect is likely also closely related to our work, as there attentional inhibition of a distractor stimulus leads to reduced preferences for those stimuli

(Fenske & Raymond, 2006).

In conclusion, our results support a causal mechanism from attention to choice.

Therefore, inherent biases or exogenous manipulations of attention will, in turn, result in choice biases. A better understanding of attentional processes, biases, and salience are thus likely critical to the study of decision-making.

Chapter 3: The Neural Computation and Comparison of Value in Simple Choice

Introduction

Simple choices like what cereal to eat for breakfast, seem almost automatic. In reality they involve a process of value computation and comparison between the alternatives that is far from instantaneous. Such choices are thought to rely on an evidence-accumulation-to-bound process (also known as sequential sampling models) that depend on factors such as the value difference between the alternatives and shifts in attention. These factors produce reliable patterns in choice data. In general, smaller value differences lead to slower decisions

(Busemeyer & Townsend, 1993), response-time (RT) distributions are right skewed (Rodriguez et al., 2015), imposing speed instructions leads to worse performance while accuracy demands make decisions take longer (Mormann et al. 2010), and people tend to choose options that they look at longer and look at last (Shimojo et al. 2003; Krajbich et al. 2010; Mullet & Stewart,

2016; for a review see Krajbich 2018).

Sequential sampling models (SSM) vary in specific details but generally share some core features. When faced with a decision, people begin to sample evidence in favor of each option.

Evidence builds up over time until there is enough to commit to one option (Ratcliff & Smith,

2004). These models can mostly be broken down into two key components: inputs and integrators (Bogacz, 2006).

Each option has its own input. An input represents the relevant stimulus magnitude for that option. In our context of value-based decisions, the input represents an option’s subjective

49 value. For a given option, the average input value is generally assumed to be constant over the course of the decision. From one instant to the next the input value does vary, but it does so randomly and i.i.d.

The stochastic sequences of input values are accumulated by the integrators. Typically, each option has its own integrator (Roe, Busemeyer, Townsend 2001; Gold & Shadlen, 2007;

Krajbich & Rangel, 2011; Usher & McClelland, 2001; Wang, 2002), which accumulates the evidence from that option’s input but is also inhibited by the other options’ inputs and/or integrators. Thus, each integrator represents the relative evidence favoring a given option.

Once any of the integrator values reaches a pre-determined threshold, the corresponding option is chosen. In contrast to the input values, these integrator values evolve continuously over the course of the decision.

While SSMs capture choice behavior and RTs extremely well, most do not consider the effects of attention. Attention is thought to shift over the course of the decision, amplifying the attended inputs and/or inhibiting the unattended inputs (Roe, Busemeyer, Townsend, 2001;

Diederich, 1997; Johnson & Busemeyer, 2005). These shifts in attention are reflected in eye- movements (Hoffman & Subramaniam, 1995), which affect choice outcomes (Chen & Krajbich,

2017; Fiedler & Glöckner, 2012; Fiedler et al., 2013; Folke et al., 2016; Glaholt & Reingold,

2011; Gwinn, Leber, & Krajbich, 2019; Janiszewski et al., 2013; T. Jiang, Potters, & Funaki,

2016; Kim et al., 2012; Konovalov & Krajbich, 2016; Orquin & Mueller Loose, 2013; Pärnamets et al., 2015; Polonio, Di Guida, & Coricelli, 2014; Russo & Leclerc, 1994; Shi, Wedel, &

Pieters, 2013; S. M. Smith & Krajbich, 2019; Stewart et al., 2015; Tavares et al., 2017; Towal et al., 2013; Vaidya & Fellows, 2015; Venkatraman, Payne, & Huettel, 2014; J. T. Wang, Spezio,

& Camerer, 2010), and their effect on the choice process is captured by the attentional drift

50 diffusion model (aDDM) and other related SSMs (Ashby, Jekel, Dickert, & Glöckner, 2016b;

Cavanagh, Wiecki, Kochar, & Frank, 2014b; Fisher, 2017; Ian Krajbich et al., 2010; S. M. Smith

& Krajbich, 2019). However, what remains unclear is how this attention-guided comparison process is implemented in the brain.

Identifying the neural mechanisms underlying choice processes is a fundamental goal in decision neuroscience. While this venture has been very successful in perceptual decision making (for recent reviews see: O’Connell et al. 2018, Forstmann et al. 2016; Ratcliff et al.

2016; Hanks & Summerfield 2017), it has proven more challenging for value-based decisions.

The main reason is that decisions are typically very quick, with RTs shorter than the time resolution of functional magnetic resonance imaging (fMRI) (but see Gluth et al., 2012).

Evidence for the neural substrates of value-based SSMs have thus typically come in two varieties: trial-level measures correlating with model parameters (Hare et al., 2011; Rodriguez et al., 2015), scalp-level electric activity from electroencephalography (EEG) (Polania et al. 2014), or a combination of the two (Pisauro et al., 2017). Taken together, these studies have implicated a fronto-parietal network underlying value-based decision-making, with more ventral/frontal regions acting as inputs and more dorsal/parietal regions acting as integrators.

Despite their many strengths, these past experiments are limited by the inability to localize activity from EEG signals and by reliance on particular assumptions of specific SSMs or how their parameters should be reflected in brain activity. Furthermore, while Lim et al. (2011) demonstrate that value signals in ventromedial prefrontal cortex (vmPFC) (for reviews see:

Bartra et al., 2013; Clithero & Rangel, 2013) are modulated by attention, it has yet to be shown that these attention-modulated signals are integrated into the decision values and are not just epiphenomenal.

Here, we present the results of an fMRI experiment designed to address these key issues.

Our approach was to slow down the decision-making process by presenting choice-relevant information gradually over time. Such “expanded judgment” tasks have been used to study SSM assumptions in perceptual decision-making, more recently in combination with neural recordings

(mostly electrophysiology in rats and monkeys) (Brunton et al., 2013; Cisek, Puskas, & El-Murr,

2009; Gluth et al., 2012; Tsetsos, Chater, & Usher, 2012; Yang & Shadlen, 2007). This task allowed us to extend the decision-making period to approximately a minute, while also allowing us to dissociate the input values from the integrator values. The inputs represent the stimuli currently on the screen, while the integrators represent everything that has been presented so far.

We simultaneously collected eye-tracking data, allowing us to further test whether gaze modulates input and integrator representations, both within the current pair of stimuli, and over the whole course of the decision.

We tested the following hypotheses: (1) vmPFC and striatum contain input but not integrator representations (2) the vmPFC and striatum input representations are modulated by gaze, (3) dorsomedial PFC (dmPFC), the intraparietal sulci (IPS), and dorsolateral PFC (dlPFC) contain integrator but not input representations (4) the dmPFC, IPS, and dlPFC integrator representations are modulated by cumulative gaze (5) there is an increase in coupling between dmPFC/IPS and vmPFC/striatum activity modulated by input values.

Materials and Methods

Subjects

Twenty-eight undergraduate students at the Ohio State University participated in this study. Due to time constraints, 4 subjects were unable to finish all 3 runs of the scan. One subject 52 was excluded for not choosing in line with their ratings. This left 23 subjects in the analysis (14 men, 9 women, average age: 22.61). Another 3 subjects could not be calibrated on the eye- tracker and so are discarded from any analyses involving eye tracking (leaving 13 men, 7 women, average age: 22.9). All subjects were right handed, had normal or corrected-to-normal vision, and no history of neurological disorders. This study was approved by the Ohio State

Biomedical Sciences IRB.

Stimuli and Tasks

Rating Task: This task was done outside of the scanner. Subjects rated 148 food items using a continuous rating scale from -10 (extreme dislike) to 10 (extreme like) with 0 being indifference about the item. To choose their preferred rating, subjects moved the mouse back and forth on the screen to scroll across the rating scale and clicked the left mouse button when the cursor was at their desired value for the item. We used an incentivization procedure for these ratings. There was a 50% chance that the rating task would be used to determine the subject’s reward. In such cases, the computer would randomly select two foods and the subject would receive the one with the higher rating. If both items were rated negatively, the subject would not receive any food. Only positively rated items (0 to 10) were used in the choice task.

Choice Task: Once in the MRI scanner, subjects chose between pairs of lotteries.

Lotteries were constructed by creating 1000 potential lotteries of randomly selected items evenly split between having 3, 4, 5, or 6 items. Each item in a lottery was then assigned a probability of being drawn, with probabilities summing up to 1 per lottery. For each of these lotteries we

53 calculated its expected value by multiplying the value of each item (�) in the lottery by its associated probability (�) of being drawn, and summing the results:

�� = � ∗ � where N is the total number of items in that lottery. Lotteries were then selected by identifying pairs that had the smallest correlation between |IV| and |CV|.

We also tracked subjects’ eye movements in the scanner using an Eyelink 1000 plus (SR

Research) during this task. Eye position was monitored using an MR compatible Eyelink 1000

Eye Tracker, with the camera and infrared source reflected in the mirror attached to the head coil and recorded at 500 Hz. The eye tracker was calibrated at the beginning of the session.

Items were sampled from the aforementioned lotteries and presented on the screen one pair at a time. Each draw was presented on the screen for 2 seconds and followed by a fixation cross for 2 – 6 seconds. Once subjects were ready to choose a lottery, they used the index finger of their left (right) hand to press a button corresponding to the left (right) lottery. They were then presented with a random item from the lottery they had chosen (on the same side of the screen as the chosen basket) for 2 seconds, indicating the food they would receive from this trial, should it be randomly selected at the end of the study (Fig 3.1).

Figure 3.1 Task timeline. Each trial subjects chose between two lotteries. Subjects learned about the lotteries through random draws. Every 4-8 seconds, subjects saw a new sample from each lottery. They were allowed to sample as many times as they wanted, but were incentivized to average 7 samples per trial. Items were presented for 2 seconds, followed by a fixation cross appearing for 2 – 6 seconds with random jitter. The trial ended when a subject chose the left (right) lottery by using their left (right) index finger. Upon making their choice, subjects were presented with the item that they won from their chosen lottery.

We constructed each trial’s sequence of items in a pseudorandom way, in an attempt to minimize the correlation between the instantaneous value difference (i.e. the absolute difference in input values) and the cumulative value difference (i.e. the absolute difference in integrator values). For the first screen within each trial, instantaneous and cumulative value difference are equal. On subsequent screens, the instantaneous value difference diverges from the cumulative value difference, yielding two distinct time courses to look for in the fMRI data (Fig. 3.2).

Figure 3.2 Example trial with instantaneous and cumulative value differences. The instantaneous value difference (IV; red) and cumulative value difference (CV; black) are plotted for this example trial. During the first sample, the IV and CV are identical. However, as the trial proceeds, the two signals diverge. In the model, a choice is made when the CV reaches a pre-specified decision boundary.

Subjects were incentivized to average 7 samples per trial; they had 45 minutes to make

60 choices. Any trials that they did not complete by the end of the experiment were made for them randomly by the computer. Any trials they completed beyond 60 were simply added to the pool of potentially rewarded trials. At the end of the study, there was a 50% chance that one of the choice trials would be randomly selected for payment (otherwise the rating task was selected for payment), in which case the subject received the corresponding food from that trial.

Outside of the scanner, subjects first completed a 5 minute practice section where they chose between baskets made up of cars. After each trial, they were given feedback on how many samples they took, and reminded that the goal was to take 7 samples on average. These choices were not incentivized.

fMRI Data Acquisition and Analysis

MRI scanning was carried out at the OSU Center for Cognitive and Behavioral Brain

Imaging. We used a 3T Siemens Magnetom Prisma scanner with a 32-channel head array coil to collect the neural data. Functional data were acquired with a T2*-weighted gradient-echo 56 sequence (48 slices, interleaved, with a field of view of 1554x1554, with an in-plane resolution of 3 mm isotropic and a 3mm slice gap, TR = 2600 ms, TE = 28 ms, 80° flip angle). Slices were oriented such that the anterior side of the acquisition was raised dorsally by 30 degrees compared to the line formed by joining the anterior commissure to the posterior commissure. A high- resolution MPRAGE anatomical scan (256 slices, field of view 224x256, with a in plane resolution of 1 mm and no slice gap, TR = 1900 ms, TE = 4.44 ms, 12° flip angle) was also acquired for each participant. Each participant was scanned in one 1.5-hour session, which included the three experimental runs (15 minutes each) and the high-resolution MPRAGE anatomical scan. Additionally, a resting state scan (5 minutes) and a DTI scan were acquired, but these data are not presented here. Stimuli were presented using Psychtoolbox (Brainard, 1997;

Pelli, 1997; Kleiner, Brainard, Pelli 2007) for MATLAB (MathWorks) and displayed with a DLP projector onto a screen mounted in the rear of the scanner bore.

Statistical parametric mapping (SPM12, Update Rev. Nr. 6905; Functional Imaging

Laboratory, University College London) was used to carry out the preprocessing of fMRI data.

First, we corrected for the different slice times per echo planar image (EPI) across the total volume (using the bottom slice as a reference) and then realigned each volume in a run to the mean EPI volume from that run. Next, the anatomical scan was coregistered with the MNI average of 152 brains template, and the mean EPI per run was used to coregister all functional scans to this coregeistered anatomical scan. In order to warp the EPIs to MNI space, SPM12s normalise function was applied to the coregistered anatomical scan and the resulting warping parameters were applied to the coregistered EPIs. The resulting images were smoothed using an isometric Gaussian kernel (8 mm full width at half maximum).

First level GLMs were run using SPM on each subject individually, including contrasts of interest. We then used FSL’s randomise function to run second-level, non-parametric significance tests with threshold-free clustering and family wise error (FWE) correction to find significant clusters for the described effects.

Results

Behavioral results

A core assumption of SSMs is that individuals decide based on the evidence accumulated over the course of the decision. We thus anticipated that subjects would choose in line with the cumulative value difference and not just the instantaneous value difference. We tested this key assumption with a mixed-effects logistic regression of choosing the left lottery on left–right instantaneous value and left–right cumulative value at the time of choice. Subjects chose in line with both (cumulative value difference (excluding the final samples) � = 0.06, s.e. = 0.009, p <

0.001; instantaneous value difference � = 0.25, s.e. = 0.02, p < 0.001) (Fig. 3.3a).

A) B)

Figure 3.3 Choice data. A) The probability of choosing left based on the left – right value difference for both CV and IV. As the value difference becomes greater in favor of one option, the probability of choosing that option increases, for both IV and CV. B) The effect of attention on choice. The longer that subjects looked at one lottery over the other, over the course of the whole trial, the more likely they were to choose that lottery. Error bars indicate s.e.

A second feature of SSM data is that easier decisions generally take less time, that is we

expect a negative correlation between RT and strength-of-preference (i.e. absolute value

difference). A SSM would handle our experiment’s time constraints by setting evidence

thresholds such that decisions would take an average of seven samples but would vary based on

the values of the lotteries in a given trial. Indeed, we observed substantial variability in the

number of samples per trial within most of our subjects (mean number of samples = 6.37 with

mean s.d. = 2.61). A mixed-effects regression of log(# of samples) on the absolute expected

value difference between the two lotteries revealed a marginal relationship (� = -0.03, s.e. =

0.02, p=0.10), confirming that our subjects sampled less on easier trials.

A third behavioral pattern predicted by the aDDM and other attention-based SSMs is that

individuals should generally choose options that have been attended to more. We thus

59 anticipated a positive correlation between choice and relative dwell time over the course of the whole trial. We added dwell advantage (left – right dwell time) to the previous choice regression and observed a positive effect on choosing left (� = 0.307, s.e. = 0.08, p < 0.001).

We were concerned that this gaze effect might be driven solely by the last pair of samples on the screen at the time of choice. Therefore, we ran another logistic regression to predict choice from the previous sample’s cumulative value difference and left-right dwell time, that is, excluding data from the last sample of each trial. Both regressors were significant and positive

(cumulative value difference � = 0.081, s.e. = 0.009, p < 0.001; left-right dwell time � = 0.220, s.e. = 0.050, p < 0.001). Thus, the influence of dwell time on choice occurred over the course of the entire decision, not simply on the last sample (Fig. 3.3B).

Neuroimaging results

Our general strategy for the fMRI data was to identify regions with BOLD activity correlating with the time series of instantaneous or cumulative evidence. In this way we could identify the key components of the SSM choice process: the inputs and the integrators. We then went on to test whether these representations were modulated by gaze. Finally, we investigated functional connectivity between inputs and integrators.

All GLMS, in addition to the regressors of interest, contained a stick function for the button press onset, modulated by |CV|, as a nuisance regressor event, as well as a boxcar function during the feedback screen, modulated by the value of the received item. We also added motion parameter time series to account for variation due to motion.

GLM 1: Instantaneous and Cumulative Value Results

The variables of interest in GLM 1 were absolute instantaneous value difference (|IV|)

and absolute cumulative value difference (|CV|) where |��| = ∑ |��| , where t is in units of screens (i.e. pairs of samples) and T is the current screen. Summing |CV| to T-1 helps to de- correlate |IV| and |CV| and ensures that any correlations with |CV| are not spuriously due to correlations with items currently on the screen. We focus on absolute value difference since we are interested in regions that represent the amount of evidence for any option, rather than the left or right option in particular (but see “Signed Value Results” below).

We first investigated the vmPFC and striatum, regions we hypothesized represent the inputs, i.e. instantaneous evidence. Looking specifically at BOLD activity in the vmPFC ROI defined in Bartra et al. (2013), we found a positive correlation with |IV| (peak voxel x = -8, y =

56, z = -2; p < 0.05) but no correlation with |CV| (peak voxel x = 6, y = 38, z = 26; p = 0.14)

(Fig. 3.4A). In contrast, the striatum showed no significant relationship with either |IV| or |CV|

(peak voxel: x = 16, y = 20, z = 12, p = 0.18 and peak voxel: x = 28, y = -10, z = -8, p = 0.13, respectively).

We next investigated the dmPFC, IPS, and dlPFC regions we hypothesized represent the integrators, i.e. cumulative evidence. In the dmPFC, whose ROI was defined by Hare et al.,

(2011), we found a significant, positive relationship between BOLD activity and |CV| (peak

61 voxel: x = 4, y = 12, z = 50, p < 0.001), but no relationship between BOLD activity and |IV|

(peak voxel: x = 8, y = 16, z = 54, p = 0.56) (Fig 4B). In the IPS, identified with the FSL atlases,A) we also saw a significant increaseB) in activity as |CV| increases (peak voxel: x = 42, y = - 48, z = 38, p < 0.001), but no relationship with |IV| (peak voxel: x = 38, y = -66, z = 46, p = 0.30)

(Fig. 4B). In the dlPFC we find the same pattern as the dmPFC and striatum. There is a significantly positive relationship between BOLD activity and |CV| (peak voxel: x = 42, y = 34, z

= 28, p < 0.001), and no relationship between BOLD activity and |IV| (peak voxel: x = 40, y =

38, z = 26, p = 0.55).

dmPFC vmPFC IPS

x=-8, y=56, z=-2 x=6, y=56, z=38 Figure 3.4 Regions responding to instantaneous and cumulative value. A) vmPFC showed a significantly positive correlation with |IV|, but did not respond to |CV|. B) Both dmPFC and IPS showed a significantly positive correlation with |CV|, but no correlation with |IV|.

In summary, we found evidence that vmPFC represents inputs but not integrators, while dmPFC and IPS represent integrators but not inputs (Fig. 3.5).

Figure 3.5 Beta plots from the vmPFC, striatum, dmPFC, IPS, and dlPFC. Displayed are regression coefficients from each region for absolute instantaneous value difference (|IV|) and absolute cumulative value difference (|CV|). Both vmPFC and striatum show a similar pattern of BOLD activity that scales positively with |IV|, but does not respond to |CV|. The opposite pattern can be seen in the dmPFC, IPS, and dlPFC which both show a strong positive correlation between BOLD activity and |CV|, but no relationship to |IV|.

GLM 2: Gaze Weighted Instantaneous and Cumulative Values

Having identified input and integrator regions, we next asked whether the activity in these regions was affected by attention. The aDDM (and other attention weighted SSMs) predict that attention to one option should amplify that option’s value relative to the unattended option.

Consider a simple model where an option’s value is weighted by the proportion of time during which it is attended. Imagine two trials with the same pair of values, 7 on the left and 3 on the right. In Trial A, the subject looks left 30% of the time and right 70% of the time. In Trial B, the subject looks left 70% of the time and right 30% of the time. In Trial A, the net input value

(“drift rate”) would be |0.3*7 – 0.7*3| = 0. In Trial B, the drift rate would be |0.7*7 – 0.3*3| = 4.

In Trial A, the value advantage for the left option is canceled out by the gaze advantage for the right option. In Trial B, both the value and gaze advantage favor the left option, leading to strong evidence in favor of left. In sum, there is stronger evidence when gaze difference is aligned with value difference. This should be true for both IV and CV, with IV only affected by gaze during the current screen, but CV affected by gaze during the entire trial.

To test this prediction, GLM 2 used the gaze-weighted values of the items, looking at the

BOLD signal for the entire duration of each presentation of food pairs while also incorporating gaze into the values. The instantaneous, gaze-weighted values (IGWV) were derived, as in the example above, by multiplying the proportion of time spent looking left for each sample pair with the left instantaneous value, and the proportion of time spent looking right with the right instantaneous value. Cumulative, gaze-weighted values (CGWV) were the sums of these IGWVs across samples. GLM 2 included both |IGWV difference| and |CGWV difference|. Again, to reduce the correlation between instantaneous and cumulative values, CGWV does not include the current sample pair.

In the vmPFC, we found a significant correlation with |IGWV difference| (peak voxel: x

= -4, y = 36, z = 4, p < 0.005) but no effect of |CGWV difference| (peak voxel: x = -8, y = 36, z =

2, negative beta, p = 0.42). The striatum also showed a correlation with |IGWV difference| (peak voxel: x = 8, y = 10, z = -6, p < 0.01), but no corresponding effect for |CGWV difference|

(negative beta, peak voxel: x = -20, y = 4, z = -6, p = 0.18).

x = -1, y = 42, z = 0

x = 8, y = 10, z = -6

x = 1, y = 17, z = 55

Figure 3.6 Regions responding to gaze-weighted instantaneous (IGWV) and cumulative value (CGWV). IGWV correlates with activity in (a) vmPFC, and (b) striatum, while CGWV correlates with activity in (c) dmPFC.

In contrast, in the dmPFC we found a significantly positive relationship with |CGWV difference| (peak voxel: x = 4, y = 18, z = 54, p = 0.03), but only a marginal correlation with

|IGWV difference| (peak voxel: x = -8, y = 16, z = 54, p = 0.06). The latter result could simply

65 be due to the fact that some of the current sample pair’s value information is quickly integrated into the cumulative values. The IPS, on the other hand did not seem to respond to either |IGWV| or |CGWV| differences (peak voxel: x = -14, y = -54, z = 48, p = 0.12, and peak voxel: x = 48, y

= -34, z = 44, p = 0.48 respectively) (Fig. 3.6). Similar to the IPS, the dlPFC does not appear to incorporate attention, responding to neither IGWV or CGWV differences (peak voxel: x = 42, y

= 28, z = 26, p = 0.20; peak voxel: x = -46, y = 26, z = 18, p = 0.20, respectively).

GLM 3: Gaze Contrast Results

Another way to test for the effect of attention on the value representations is to directly contrast cases where gaze is to the better option with cases where gaze is to the worse option. As described in the previous section, there should be more evidence when the gaze bias is aligned with the value bias than when they are misaligned. Thus, regions encoding net evidence should show a significant effect in this contrast. The sign of that effect is obvious in the case of the inputs and instantaneous values. If the current left item is better than the right item, then we should see stronger input activity when the subject is currently looking left compared to right.

The prediction is less obvious for the integrators and cumulative values. Suppose the left lottery is better than the right lottery; how should the current gaze location affect the integrator activity? What matters for the integrator activity is where the subject has looked more in the past, since the integrator represents all of the evidence accumulated so far. As it turns out, the current gaze location is negatively correlated with the amount of time spent looking at that side so far in the trial. That is, if the subject is currently looking left then she has, on average, spent more time looking right up to that point.

We established this fact with a mixed-effects logistic regression of whether subjects were currently looking left (1) or right (0) on left-right cumulative value difference (CV), left-right instantaneous value difference (IV), and left-right dwell time for all fixations preceding the current one. The regression revealed a significantly negative beta on left-right dwell time (� = -

0.308, p < 0.001), indicating that subjects had looked at the currently attended lottery less than the unattended lottery. What this means is that if the left lottery is better than the right lottery, we should see stronger integrator activity when the subject is currently looking right compared to left.

To test these predictions, we ran a third GLM (GLM3) that included left - right IV, left - right CV, a dummy variable for current gaze location (left =1, right =0), and the interaction of this dummy with the two value-difference regressors.

We also looked for these effects in our integrator regions. Importantly we found no sign of the expected attention modulation of IV (the effects were in the opposite direction) in the dmPFC (negative beta, peak voxel: x = 4, y = 8, z = 52, p = 0.14), the IPS (negative beta, peak voxel: x = 34, y = -50, z = 46, p = 0.07), or the dlPFC (negative beta, peak voxel: x = -46, y =

26, z = 18, p = 0.09).

For the first hypothesis we focused on the attended vs. unattended contrast for instantaneous value (IV*gaze left > IV*gaze right). Here we found that activity was significantly higher in the vmPFC for attended vs. unattended items (peak voxel: x = 4, y = 54, z = -2, p =

0.05 (one-sided test since this is a replication of Lim et al. 2011)) (Fig. 3.7). We found no such effect in the striatum (peak voxel: x = -16, y = 18, z = -6, negative beta, one-sided p = 0.17).

dmPFC

vmPFC

x = -6, y = 14, z = 60

Figure 3.7 Gaze contrast results. The vmPFC shows a positive interaction between IV and gaze location, while the dmPFC shows a negative interaction between CV and gaze location. Both results are consistent with gaze enhancing the value of attended items.

For the second hypothesis, we focused on the attended vs. unattended contrast for cumulative value (CV*gaze left > CV*gaze right). Here we found, as expected, that activity was marginally lower for attended vs. unattended lotteries in the dmPFC (peak voxel: x = -6, y = 14, z = 58, p = 0.06), and non-significantly in the IPS (peak voxel: x = 36, y = -60, z = 30, p = 0.12)

(Fig. 6) and the dlPFC (peak voxel: x = 42, y = 28, z = 26, p = 0.17).

We also looked for these effects in our input regions. Here we found no effects in the vmPFC (peak voxel: x = 4, y = 58, z = -2 with p = 0.13) and marginal effects in the striatum

(peak voxel: x = 32, y = -10, z = -10, p = 0.07). These perhaps reflect a trace of the input activity from the previous sample.

GLM 4: Motor Cortex Results

The act of accumulating evidence for a decision means nothing if you are unable to then act on that decision. We had subjects indicate their decision with a button press using either their 68 left or right index finger. The absolute evidence accumulating in the dmPFC should thus translate into increased activity in the right or left motor cortex, respectively, which we defined using the FSL atlases (Gluth et al. 2012). GLM 4 included both the signed IV and CV, with more positive values indicating more evidence for the left option.

As expected, there was a positive correlation between CV and BOLD activity in the right motor cortex (peak voxel: x = 48, y = -22, z = 54; p = 0.02) and a negative correlation with CV in the left motor cortex (peak voxel: -36, y = -30, z = 68; p = 0.04). These same ROIs showed no relationship with the signed IV (peak voxel: x = -52, y = -28, z = 56; p = 0.42) (Fig. 3.8).

the second hypothesis, we focused on the attended vs. unattended contrast for cumulative value

(CV*gaze left > CV*gaze right). Here we found, as expected, that activity was marginally lower for attended vs. unattended lotteries in the dmPFC (peak voxel: x = -6, y = 14, z = 58, p = 0.06), and non-significantly in the IPS (peak voxel: x = 36, y = -60, z = 30, p = 0.12) (Fig. 6) and the dlPFC (peak voxel: x = 42, y = 28, z = 26, p = 0.17).

x = 48, y = -22, z = 54

Figure 3.8 Motor cortex activity reveals the winning lottery. Signed cumulative value difference (CV), left – right, shows positive correlations in right motor cortex and negative correlations in left motor cortex. As a reminder, subjects used their left/right hand to choose the lottery on the left/right side of the screen.

PPI Analysis

To investigate whether the dmPFC is receiving input from the vmPFC, as we would expect if the dmPFC were integrating evidence into a cumulative value signal, we ran a psychophysiological interaction (PPI). A seed region was chosen for each subject independently by creating a sphere around the peak voxel in the vmPFC for |IV| with a radius of 4mm. We then interacted the activity in this seed region with the parametric modulator for our variable of interest (|IV| and |CV|). This interaction, as well as the seed region time course, were then entered into GLM 1 as additional regressors.

In line with our predictions that the vmPFC is transferring the instantaneous value to the dmPFC, we see an increase in coupling between these two regions in response to the |IV| (p <

0.01). However, we also see an increase in coupling for |CV| (p < 0.05). This is not entirely surprising, as cumulative values, by nature, contain the instantaneous values. Thus, the key comparison is whether there is more coupling for |IV| compared to |CV|. This contrast (|IV| >

|CV|) is significantly positive (p = 0.03). As the IPS also seems to be an integrator region, we also investigated whether there was higher connectivity between the vmPFC and the IPS for |IV|.

We do see a marginal positive relationship in connectivity between the vmPFC and the right IPS for |IV| (p = 0.075). Unlike with the dmPFC, there was no significant coupling between the vmPFC and the IPS for |CV| (p = 0.15), however the |IV| > |CV| contrast failed to reach significance (p = 0.23). Thus we find weaker evidence for the transfer of value between the vmPFC and IPS.

Next we looked for similar connections between the striatum and dmPFC/IPS. Here we found a similar relationship with the dmPFC in that there was a significant increase in coupling for |IV| (p < 0.001) as well as with |CV| (p < 0.001), while the contrast of |IV| > |CV|, was 70 marginally significant (p = 0.09). We observed a very similar pattern between striatum and IPS, with a significant increase in coupling for |IV| (p < 0.001) as well as for |CV| (p = 0.01), and a marginally positive contrast for |IV| > |CV| (p = 0.09).

Discussion

In this paper we presented results from a simultaneous eye-tracking and fMRI study of value-based decision-making, using an expanded-judgment task where subjects sampled from and then chose between food lotteries. We found that the vmPFC, and to a lesser extent the striatum, appear to compute instantaneous values, while the dmPFC and IPS appear to compute cumulative values, and the motor cortex reflects the direction of the evidence. We found that instantaneous value signals in vmPFC and striatum are modulated by gaze allocation (Lim et al.,

2011; McGinty, Rangel, & Newsome, 2016), and more importantly, we found that this attention modulation extends to the cumulative value signals in dmPFC. Finally, a PPI analysis demonstrated links between the vmPFC/striatum and dmPFC/IPS, specifically through increased connectivity for higher instantaneous values. This effect was particularly strong for the link between vmPFC and dmPFC.

These results provide novel evidence for the neural mechanisms underlying the attention- guided sequential-sampling process that appears to govern many types of decisions, as exemplified by the aDDM (Krajbich et al. 2010; Smith & Krajbich 2018). Neural implementations of sequential sampling models generally require at least two sets of neurons, one set to represent the current information from the stimuli and a second set to integrate that information over time. In the current study, the information being used to make decisions was subjective value (i.e. utility). A large body of work has implicated the vmPFC and striatum in

71 representing value (Chib et al. 2009; Levy & Glimcher 2011; Bartra et al. 2013). Interestingly, our results confirm that these two regions do represent value information, but primarily just for the current stimuli. The integrated value information, which is what ultimately determines the decision, is instead encoded in the dmPFC and IPS, regions that have received much less attention in the literature.

Our results fall well in line with a neural model proposed by Hare et al. (2011) in which the vmPFC provides inputs to the dmPFC and IPS. That model was based on dynamic causal modeling results, while our task provides a more direct test of the neural model. Our eye- tracking data also allow us to identify additional features of the network. First, we find that the striatum also appears to function as an input region, a result that did not appear until we accounted for gaze modulation of value. Second, we find that while the cumulative value signals are amplified by gaze in the dmPFC, they do not appear to be affected by gaze in the IPS. The reason for this distinction between the dmPFC and IPS is unclear, but it does suggest that the dmPFC is more likely to be the final decision-making region, consistent with other recent results

(Rodriguez et al. 2015; Pisauro et al.2017). Of course, the recruitment of the dmPFC may be due to the fact that our subjects made their decisions with a button press, which is supported by the motor cortex activity mimicking the activity seen in the dmPFC. Had our study required eye- movements to indicate a choice, we may very well have observed integrator activity in other regions such as the frontal eye fields or posterior parietal cortex (O’Connell et al. 2018).

The striatum activity is difficult to interpret. It showed no correlation with instantaneous or cumulative value when attention was not included in the model. However, once attention was included, the results seem contradictory. Namely, when looking at attention at the dwell level, the striatum responds more to cumulative values, but when attention is used as a modifier to the

72 true value of each item, we find the striatum responds strongly to the instantaneous value. This may be due to the limitations of running a GLM on the dwell level, since our TR was 2.6 seconds and dwells lasted for 0.66 seconds (se = 0.03 seconds).

Taken together, we managed to separate instantaneous values from an overall decision value, or cumulative value, and found a network of brain regions that are involved in an aDDM- like process. This process involves passing the instantaneous values to an accumulator which responds to not only the values themselves, but the attention-modulated values. These results indicate that attentional effects on value representations are not epiphenomenal, but rather the effect of attention is incorporated into the decision process, affecting how we perceive the value of each option over the course of the entire decision.

Chapter 4: The role of attention in auditory choice

Introduction

People are no strangers to making decisions, whether it’s what food to eat, what to watch on TV, or something more consequential, like which house to buy. While these are visually- based decisions, we don’t live in a purely visual world. In fact, we also make auditory decisions, such as what conversation to tune into at a party or what songs to listen to. While visually-based decision-making has been extensively studied and modelled, the processes behind auditory decision-making are not as well understood. Here we investigate whether the well-established patterns observed in visual, preference-based decisions also occur in the auditory domain.

Specifically, we are interested in whether increased attention to an item also increases the probability of choosing that item.

Visual, preference-based choices have been studied in much the same way as purely perceptual choices. In fact, many of the patterns observed in perceptual choices can also be found in preference-based choices. Specifically, both behaviorally and in terms of attention, these patterns include longer reaction times (RTs) for more difficult choices, a higher probability of choosing items one has fixated longer, and a robust tendency to choose the last item one fixated (Fiedler & Glöckner, 2012; Fiedler et al., 2013; Ian Krajbich et al., 2010; Krajbich, Lu,

Camerer, & Rangel, 2012; Tavares et al., 2017).

Sequential sampling models (SSMs) have capitalized on these patterns to characterize both perceptual and preference-based decisions with computational modeling. These assume that 74 a decision-maker accumulates samples of evidence from each of the possible options. Once sufficient evidence has been gathered for a specific choice item, the decision-maker stops the evidence accumulation process, and chooses the item favored by the sampled evidence (Bogacz,

Brown, Moehlis, Holmes, & Cohen, 2006; Brown & Heathcote, 2008; Forstmann, Ratcliff, &

Wagenmakers, 2016; Johnson & Busemeyer, 2005; Mulder et al., 2014; Ratcliff, 1978; Usher &

McClelland, 2001). A specific type of SSM has gained particular traction in the perceptual decision-making literature, called the Drift Diffusion Model (DDM), first outlined by Roger

Ratcliff (1978). The DDM has been successfully used to explain results from memory tasks, random dot kinematogram (RDK) tasks, brightness tasks, and more (Ratcliff, 2002; Ratcliff,

Gomez, & McKoon, 2004; Ratcliff & McKoon, 1996).

The DDM has four main parameters: drift rate, boundary separation, starting point, and non-decision time. The drift rate is the speed with which one gathers evidence from one choice item relative to the other. The higher the drift rate, the easier the decision. Boundary separation refers to how much relative evidence one must collect before making a decision. Wider boundaries indicate a desire for accuracy rather than speed, while boundaries that are closer together indicate an emphasis on speed at the expense of accuracy (Bogacz, Hu, Holmes, &

Cohen, 2010). Starting point indicates a bias towards an alternative, even before the decision begins. Finally, non-decision time captures the amount of time during a decision where no decision process is taking place. Specifically, this parameter encompasses the processes behind stimulus encoding and enacting the motor response necessary to indicate one’s choice.

However, the DDM does not take attention into account when modelling the choice process. While less prevalent in perceptual decision-making, the effect of attention on preference-based decision making has been well studied. While studies have demonstrated a

75 correlation between increased attention to an item, and choosing that item (Shimojo et al., 2003), others have also demonstrated a causal link between attention and choice. For instance,

Mormann et al. (2012) demonstrated that increasing the salience of an item also increased the probability of choosing that item. Other studies have increased attention to items through changing the duration each item can be viewed, taking natural item salience into account when modeling choice, or manipulating attention outside of the choice environment (Armel et al.,

2008; Gwinn et al., 2019; Lim et al., 2011; Pärnamets et al., 2015; Towal et al., 2013).

A specific adaptation to the traditional DDM has gained traction in the preference-based literature, as it’s able to account for the attentional effects typically observed in these decisions.

The attentional Drift Diffusion Model outlined by Krajbich et al. (2010) assumes that the rate of evidence accumulation, the drift rate, is based on the difference in subjective value between the two items, and that attending to any one item discounts the evidence sampled from the unattended item. Formally, this is conceptualized as:

When attending left:

�� = � ∗ �� − � ∗ �� + �

When attending right:

�� = � ∗ � ∗ �� − �� + �

Where d is a scaling parameter, �, denotes the noise in the process, and � is the attentional discounting parameter modifying the unattended item, which is bounded between 0 and 1, where

0 denotes full attentional discounting and a value of 1 converges on the standard DDM. In the original paper defining this model, Krajbich and colleagues found a best fitting theta of 0.3, and other studies using the aDDM have also found values within the range of 0.2 to 0.4 (Krajbich &

Rangel, 2011; S. M. Smith & Krajbich, 2018, 2019; Tavares et al., 2017). As the effect of 76 attention on choice, as measured by this parameter, appear to be fairly stable across studies, we expect that if auditory attention and visual attention have similar influences on choice, we will find a similar parameter in the current study.

While it’s apparent that visual decision-making has an extensive literature, the auditory, preference-based domain is less explored. Some research does hint that audition can be used to guide attention towards a visual stimulus. A study by Knoeferle et al. (2016) found that sounds that were congruent with a search item, for instance a jingle associated with a specific brand of laundry detergent, could guide visual search towards the congruent target. Even more than audition guiding visual attention, audition could play a role in visual, preference-based choices.

Two separate studies have shown that playing songs that are associated with certain products increase the purchase rates of those products (Arnd et al., 2012; North et al., 1999).

However, no research has yet investigated the role of auditory attention in auditory choices. Thus, to investigate whether attention plays a similar role in auditory, preference-based decisions, we created a 2-alternative forced-choice task in which subjects heard two different songs, one played into each ear. In addition to the music, white noise played over the songs, but could be removed in only one ear at a time by the subject, thus allowing the song to play clearly on that side. We equate this to attending to the song no longer covered by white noise. Using this paradigm, we found very similar patterns as those observed in visual, preference-based choices in terms of choice, RT, and attention patterns. We additionally found similar attentional discounting parameter estimates using the aDDM. However, looking at the data in new ways revealed that in the auditory - but not visual - domain, the effects of attention can be attributed to participants simply choosing the last thing they listened to.

Materials and Methods

Subjects

47 undergraduate students at The Ohio State University participated in this experiment.

12 subjects failed to choose in line with their preferences, and thus were excluded from all analyses. Of the remaining 35 subjects, 3 were excluded for not attending to either stimulus on more than 50% of their choice trials, in line with our pre-registration. This left 32 subjects in our final analyses. Subjects earned $20 for their participation. This study was approved by The Ohio

State University IRB.

Materials

Images and sounds were created and presented using Matlab (Mathworks) in conjunction with Psychtoolbox (Brainard, 1997; Pelli, 1997). Songs were presented using Hamilton Buhl over-ear headphones. Subjects indicated their responses using both a standard U.S. keyboard and a mouse.

We compiled an initial list of 205 songs. 100 were taken from the billboard top 100 for

January, 2019. The remaining 105 songs were gathered from popular movies and previous billboard top 100 charts spanning back to the year 2000. We then asked 5 pilot subjects to indicate whether they recognized each listed song, and whether they would listen to that song, if given the chance. Only songs that were both recognized and liked by at least 75% of our pilot subjects were kept. This resulted in a final list of 136 songs (the full list of songs used in this study can be found in Appendix E).

Rating Task

First, subjects rated how much they liked 136 popular songs while listening to the chorus on repeat over headphones. The average length of the song clips was 22.92 seconds (s.d. = 32 seconds). The choruses, when possible, contained the name of the song, or were chosen to contain a highly recognizable piece of the song in cases where there were no lyrics (i.e. the

Mission Impossible Theme). The title and artist of the song were displayed on the screen to facilitate identification of the sound clip. While the rating scale included only positive numbers

(0 – 10), subjects could also indicate that they did not like a song, by pressing “h,” or did not recognize a song, by press “n,” in which case that song was removed from the rest of the study.

To select a rating, subjects moved a red tick mark along a number line using the arrow keys.

Once they had placed the tick mark on their rating for that song, they pressed the spacebar to select it and continue to the next song (Fig. 4.1a).

Figure 4.1 Rating and Choice Tasks. A) During the rating task, subjects hear the chorus of a song play on repeat in both ears. They could move a red slider back and forth along a number line to choose their liking rating for the current song, or use a button press to state that they either hated or were unfamiliar with the song. To facilitate song identification, the artist and song name were displayed. B) Subjects then completed a binary choice task, choosing which song they preferred on each trial. During the choice the chorus of two different songs played simultaneously, one in each ear. Initially, white noise covered both songs, but subjects could switch back and forth to remove the white noise from one ear at any given time. No information about the songs was presented on the screen.

Choice Task

Next, subjects completed 200 binary-choice trials where they chose which of two songs they preferred. On each trial, subjects heard the chorus of two different songs played through headphones, one in each ear. Importantly, both songs were overlaid with white noise. White noise was played at 5 dB, which was titrated in a small pilot study of 5 subjects to achieve a comfortable amount of sound, without allowing for both underlying songs to be heard clearly.

Subjects could then press “1” to listen to the left song – removing the white noise only on the left side - or “2” to listen to the right song. They could switch between the songs as much as they liked, allowing us to record which song they were attending to at any given time. On the screen, subjects saw a horizontal line, bisected by a vertical, dashed line, which indicated the middle of the screen. A red tick mark automatically moved to the far side of the horizontal line corresponding to the song the subject was currently attending (Fig. 4.1b). The choruses looped indefinitely until subjects made their choice by using the mouse to click on the corresponding side of the screen. That is, to choose the song on the left, subjects clicked left of the vertical, dashed line, and vice versa for the song on the right. To incentivize choices, we selected a random trial at the end of the study for each subject, and each subject listened to the entire song they had chosen on that trial.

Results

If attention plays a similar role in auditory choice as it does in visually-based choices, we would expect to find specific patterns in our data. We investigated analogs to the following, well-known visual effects within our auditory data: 1) Subjects choose in line with their preferences, 2) more difficult decisions take more time, 3) subjects tend to choose items they’ve

81 attended to longer and attended to last, and 4) longer listening durations correlate with an increased probability of choosing the currently attended item, and 5) subjects don’t simply attend to more-liked items for longer, hinting at a causal relationship between attention and choice. All analyses rely on mixed-effects models, with random effects at the subject level, unless otherwise stated.

The role of preferences in choice

As in the visual domain, subjects chose in line with their stated preferences. We verified this with a logistic regression of choosing left on left – right rating using our full sample, with no exclusions (n = 47) and found a significant coefficient on rating difference (� = 0.25, p < 0.001).

This coefficient increases once we apply our exclusion criteria (� = 0.38; p < 0.001) (Fig. 4.2a).

In addition to noting that subjects, on average, chose in line with their ratings, we also noted that a majority of our sample chose consistently with their ratings (74.47%).

Difficulty and Response Time

Again, replicating the visual choice literature, we found a significant decrease in RTs for easier choices. We confirmed this with a linear regression of log RT on absolute rating difference, using clustered standard error, due to high correlations between random effects. We found a significantly negative beta on absolute rating difference indicating shorter RTs for easier choices (i.e. those with a greater rating differences between items) (� = -0.01, p = 0.03) (Fig.

4.2b).

A) B)

Figure 4.2 Choice and RT Patterns. A) Subjects chose in line with their stated ratings. As the rating difference begins to favor the left-hand side, there is an increase in choices to the left, and vice versa as the rating difference begins to favor the right-hand side. B) As the absolute rating difference increases, RTs become shorter, as those decisions are easier to make. We find, roughly, a 101 ms decrease in RT for each increase in rating difference.

Attention and Choice

Before looking at the relationship between attention and choice, we first focused on

whether subjects switched their attention back and forth as they would in the visual domain.

Therefore, we focused on how many “fixations” they made to each item. On average, subjects

made 2.01 fixations per trial (s.e. = 0.07), which is much less than one would expect in the visual

domain (Fig. 4.3a). However, this is not surprising as manually switching between audio streams

is much less natural than making eye movements.

In addition to the number of fixations, we also looked at the average length of each

fixation, depending on whether it was an initial fixation, middle fixation, or final fixation, as

previous literature has shown that first and final fixations are shorter than middle fixations

(Krajbich et al., 2010). While we found a significant difference in duration between first and middle fixations (paired t-test: 0.45 seconds, p = 0.02), there was no analogous difference between middle and final fixations (paired t-test: 0.10 seconds, p = 0.52) (Fig. 4.3b). This could be due to the fact that there are not many middle fixations, as the average number of fixations was 2.

While not all eye movement patterns translated into the auditory domain, subjects did seem to be switching their attention between the two items, meaning there could still be a relationship between attention and choice. Thus, we investigated whether subjects chose items that they had listened to longer.

A) B)

Figure 4.3 Average Fixation Length per Subject. A) On average, subjects made 2.01 fixations per trial (s.e. = 0.07), although there was individual variation in how many times subjects listened to each song. B) The average listening duration based on “fixation” order. Error bars indicate s.e.

We looked at listening advantage – the total time, in seconds, spent listening to the left item minus the time listening to the right item – as a measure of attentional advantage on each trial. Then we used a logistic regression of choosing the left item on listening advantage and left– right rating. We found a significant coefficient on both rating difference (� = 0.38, p < 0.001) and listening advantage (� = 0.15, p < 0.001), indicating that, even after controlling for the ratings, subjects were more likely to choose the item they listened to longer (Fig. 4.4a).

In the visual domain, not only do subjects choose items they’ve attended to longer, but they also have a final fixation bias. That is, they’re more likely to choose the item they last fixated. We looked at this tendency in the auditory domain as well, using a logistic regression with clustered standard error, because mixed effects models did not converge. Our model used choosing the best item as the dependent variable, and best–worst rating and listening to the best item last as independent variables. We found both the expected significant beta on the rating difference (� = 0.03, p < 0.001), and a significant beta on last fixation (� = 1.86, p < 0.001) (Fig.

4.4b). This indicates that participants were far more likely to choose the last item that they listened to.

A) B)

Figure 4.4 Attention on Choice. A) As subjects listen to the left song for longer than the right song, we see an increase in the probability of choosing the left item. B) Subjects were more likely to choose the last item they attended to. In fact, even if the best item was rated 5 points above the worst item, attending to the worst item last increased its chances of being chosen to almost 50%.

Surprisingly, despite replicating this final fixation bias previously seen in the literature, there was no corresponding first-fixation bias. That is, subjects that were more likely to listen to the left item first were not also more likely to choose the left item (r = -0.008, p = 0.97).

Listening duration and choice

In addition to choosing items which have garnered relatively more attention, the visual decision-making literature also shows that longer individual fixations on an item correlate with choosing that item (Cavanagh et al., 2014b; Ian Krajbich et al., 2010). We investigated this in our own data using a logistic regression, with clustered standard errors, of choosing the currently 86 attended item on listening duration, attended rating, unattended rating, and difficulty (absolute rating difference). While most regressors were significant in the expected direction (attended rating � = 0.34, p < 0.001; unattended rating � = -0.30, p < 0.001) or not significant as expected

(absolute rating difference � = 0.03, p = 0.10), listening duration had a significantly negative coefficient (� = -0.10, p < 0.001). This result was unexpected and seemingly contradicts the overall positive effect that attention had on choice.

To dig deeper, we investigated how the effect of listening duration changes over the course of the decision. To do so, we ran the same regression as before, but included “fixation number” interacted with listening duration. We found almost identical coefficients on all of our regressors (attended rating � = 0.33, p < 0.001; unattended rating � = -0.30, p < 0.001; absolute rating difference � = 0.02, p = 0.001), as well as a significantly positive coefficient on fixation number (� = 0.46, p < 0.001), indicating that the likelihood of choosing the item one is currently attending to goes up as the decision progresses. Importantly, in this regression, we found a positive effect of attention on choosing the currently attended item (listening duration � = 0.12, p

= 0.006). The interaction term, instead, had a negative coefficient of -0.13 (p < 0.001), indicating that long fixations late in the decision process are detrimental to choosing the currently attended item. One likely explanation for this effect is that, as the decision progresses, the decision variable should approach one of the boundaries. At those later points in time, a long fixation to an item should trigger a decision, unless it is to the worse item. Thus, later in the decision, a long fixation may signal that the fixated item is the non-preferred alternative. It is also important to note that the coefficient on the interaction term is nearly identical to the coefficient on listening duration. This indicates that there is no relationship between attention and choice in the first fixation, which suggests that our attentional effects might have been due to the number of 87 fixations rather than the amount of time attending to each song. We investigate this possibility below.

The causality of attention

While our research design did not allow us to determine the exact role of auditory attention in choice, that is whether auditory attention changes our choice process in some way, or whether it is epiphenomenal, we can make some educated guesses based off of certain patterns that we observed in our data. A common check for causality is whether subjects listened more to items that they liked more. In the visual domain, we typically do not see any difference in fixation time to items with different ratings (Ian Krajbich et al., 2010; S. M. Smith & Krajbich,

2018, 2019). We also looked at this in our data using a mixed-effects logistic regression of log listening time, in seconds, on the rating of the attended item, the rating of the unattended item, and the absolute rating difference. Contrary to the visual literature, we found a slight decrease in amount of time spent listening to higher rated songs (attended item rating � = -0.01, p = 0.02; unattended item rating � = 0.01, p = 0.16; absolute rating difference � = -0.01, p = 0.30) (Fig.

4.5).

Figure 4.5 Listening Duration Dependent on Rating. Unlike the visual domain, where subjects show no difference in the length of fixations depending on the rating of the item, we found a slight, but significant, decrease in the length of each listening duration as the rating of the item increased. This occurred on the order of a decrease of 10ms per each rating point. Error bars indicate s.e.

One reason for this pattern may be due to using all available “fixations,” whereas in the visual choice literature, only middle fixations are used in these analyses. This is because final fixations are artificially cut short by the decision variable reaching a boundary and triggering a decision. We included final fixations in these analyses as subjects only attended to each song once, on average, meaning that there were very few middle fixations. Nevertheless, we did subset the data to look at only middle fixations. This was a drastic cut, leaving only 12% of our original data set, which translates to 1,918 observations. Using this subset of the data, we ran the same analysis as before, this time finding results more in line with the visual choice literature.

Namely, there was no effect of attended or unattended rating on the length of middle fixations

(attended item rating � = -0.01, p = 0.45; unattended item rating � = 0.01, p = 0.31; absolute

89 rating difference � = -0.004, p = 0.80), though it should be noted that the size of the attended- rating effect did not change.

Regardless of whether we use this subset of the data, or the full data set, these results actually indicate that, if attention were not causal, we should see people choosing items they attended to less (since higher rated items are receiving slightly less attention). We do not observe this in the data.

Because a wide variety of known effects in the visual domain appear to transfer into the auditory domain, it stands to reason that auditory choices can be modelled in much the same way that visual choices are. This allows us to more directly compare the effect of attention to past, visual research.

Modelling with the aDDM

Given the results of the prior analyses that replicated most patterns seen in the visual domain, we turned to sequential sampling models (SSMs) to look at the underlying cognitive processes in these decisions. In particular, we were interested in the role of attention in choice and thus used the attentional Drift Diffusion Model (aDDM) to focus on how attention affected the choice process.

To fit the aDDM, we used a Python package for Hierarchical Bayesian estimation of the

Drift-Diffusion Model (hDDM) (Wiecki, Sofer, & Frank, 2013). This package has no native component to fit the aDDM, but using a technique developed in Cavanagh et al., (2014), we estimated the attentional discounting parameter by using a linear combination of regressors to estimate the drift rate. The regression coefficients � and �, identified by Cavanagh et al. correspond to the following regressors, respectively: (�� ∗ �� − �� ∗ 90

��), and (�� ∗ �� − �� ∗ ��), where “opt” refers to the best item and “sub” refers to the worst item. We then estimated theta as the ratio between the first and second coefficient, or �/�.

Using this estimation method, we fit a hierarchical model with individual-level estimates for all parameters, as well as group estimates. Here, we only focus on the group estimates. We assumed there was no bias to choose one song over the other, and thus set the starting point to lie in the middle of the two boundaries. While we’re primarily interested in the attentional parameter, it is useful to look at the other parameters of the model to make sure that they are reasonable.

The RTs in our data are longer than what we typically see in the visual domain, most likely due to the temporal nature of the song stimuli, as well as the effortful switching between items, compared to much more natural eye movements. It is not surprising then that we found larger boundary separation and non-decision time parameter estimates relative to the visual domain. Specifically, boundary separation was 4.84, as opposed to close to 2 (Cavanagh et al.,

2011; Shevlin, Smith, Hausfeld, & Krajbich, n.d.; Zhang & Rowe, 2014), and non-decision time was 1.63 seconds, as compared to 0.425 seconds in the visual literature (S. M. Smith & Krajbich,

2018). The larger boundaries here reflect the need for more evidence before making a decision.

The longer non-decision time could indicate that songs take longer to identify than visual stimuli, or it could be an artifact of using the mouse to indicate choices, which takes longer than a button press.

The drift rate was comparable to what is typically seen in the visual literature, 0.23 as compared to 0.3 (Shevlin et al., n.d.). This is again most likely an artifact of the longer RTs, as lower drift rates lead to longer decisions (Smith, Krajbich, & Webb, 2019).

Importantly, however, we found a � of 0.40 (s.e. = 0.08), which is extremely close to the typically best-fitting found in the visual, preference-based choice domain of 0.3 (Krajbich &

Rangel, 2011; S. M. Smith & Krajbich, 2019; Tavares et al., 2017). This indicates that the unattended song’s rating is multiplied by 0.4. For example, if a subject were deciding between two items both rated 10, a standard DDM would say the total drift rate should be 0 plus noise, as drift rate is based off of the left – right rating. However, in the aDDM, as the rating difference has now been changed to 10 – 0.4* 10, the subject would be effectively deciding between an item rated 10 and an item rated 4, making the decision much easier.

To be sure we had found the best fitting model, we simulated a data set using hDDM, where each trial was simulated 500 times using the subject-level parameters identified by hDDM. Visually, the simulated data captures all of the patterns we observed in our data’s RT distributions and choice curve (Fig. 4.6a-c).

A) B)

Figure 4.6 Model Fits. A) The simulated data, using the parameters that best explain our data, fit the RT quintiles extremely well, with the exception of the 90th percentile. However, this is extremely common as the 90th percentile can contain outlier data, even after accounting for outlier rate and removing obvious outliers from the data before model fitting. B) The simulated data is also able to capture the choice curve from our data set. C) The effect of listening time on choice is also captured in data simulated from our parameters. All error bars indicate s.e.

While this model fits our data extremely well, we wanted to be sure that the data could not also be captured by a simple DDM. Thus, we ran the same model in hDDM, basing the drift 93 rate on only the rating difference between the two songs. We then used the Deviance Information

Criterion (DIC) to understand the difference in fit between the two models. The DIC is similar to the Aikake Information Criteria (AIC), but used in Bayesian analyses where it relies on the likelihood function (p(data|hypothesis)) and the number of parameters in the model. The lower the DIC, the better the model fits the data. This holds true even if a model is more complex, as the DIC penalizes extra parameters to reduce the risk of over-fitting (Spiegelhalter, Best, Carlin,

& Linde, 2002). This measure indicated that the attentional model fit our data better than assuming no effect of attention (non-attentional – attentional model DIC = 632.91).

Another possibility is that the effects we’ve seen are simply due to the final fixations being directed towards the chosen option. This possibility was suggested by the results we reported above looking at the probability of choosing the currently attended item as a function of the length of the current fixation interacted with the fixation number. As mentioned in that analysis, the interaction term and the coefficient on listening duration were equal and opposite, indicating no effect of attention on choice during the first fixation, and a negative effect in subsequent fixations. More specifically, we were concerned that our results could be entirely explained by a bias to choose the last fixated item.

To investigate this further, we split our trials based on the total number of fixations. That is, we looked at trials in which subjects only fixated once, twice, three times, or four times.

Within these subsets we looked at how likely subjects were to choose the last-fixated item.

Subjects chose the last fixated item 79.92% (s.e. = 3.01%) of the time in single-fixation trials,

73.89% (s.e. = 2.91%) of the time in two-fixation trials, 80.07% (s.e. = 3.99%) of the time in three-fixation trials, and 77.90% (s.e. = 5.69%) of the time in four-fixation trials. These results

94 confirm that subjects were overwhelmingly choosing the last fixated item. However, this does not rule out that fixation time still plays some role.

To follow up on the earlier listening duration analysis, we decided to focus on the effect of first fixation duration on choosing the first fixated item. We excluded single-fixation trials, since we did not want to include final fixations in this analysis. In this analysis we find no relation between the length of first fixations and choosing the first-fixated item. This was tested in a logistic regression predicting choice of the first-fixated item on the first fixation duration and the rating of the first attended item (with standard errors clustered by subject) (first fixation duration � = -0.03, p = 0.25; rating of first-fixated item � = 0.24, p < 0.001; intercept = -1.14, p

< 0.001). This indicates that listening time is not a factor, at least for the first fixations where we have the most data. This further suggests that subjects were simply choosing the last item they listened to. Thus, the listening-time advantage we found in previous analyses is most likely due to odd numbers of fixations equating to longer overall listening times, creating a spurious correlation between overall listening times and choice.

Discussion

In this study, we’ve replicated most patterns seen in the visual, preference-based decision-making literature both in terms of choice behavior, and the role of attention during the decision process. Namely, we have demonstrated that subjects chose in line with their ratings and that more difficult decisions took longer. In terms of the role of attention in choice, we replicated findings in the visual literature that subjects tend to choose items they attend to longer. However, in the current study and unlike in the visual domain, this effect appears to be entirely due to a robust “final fixation” effect.

Although none of these results indicate directional effects of attention on choice, we did investigate whether subjects spent more time listening to higher rated songs. If this were the case, it could indicate that value of the song captured attention. Alternatively, it could indicate that subjects simply chose the higher rated item, which garnered more attention due to its value, thus making the longer listening durations epiphenomenal to the choice process. However, if anything we found a slight negative relationship between rating and listening durations.

While the visual choice literature typically shows no relationship between fixation durations and rating, it’s possible that this slight negative relationship is due to the speed with which the song is identified. That is, higher rated songs may be more recognizable, or accessible, thus leading to a decrease in the amount of time needed to identify the stimulus. Further study would be required to test this hypothesis, however, as the current study does not have sufficient data to explore this topic. Regardless of the mechanism, this slight negative relationship contradicts the argument that attention is epiphenomenal to the decision-process, and thus provides some evidence of a causal relationship between attention and choice.

Finally, we used the aDDM to examine the extent to which attention influenced choices within a DDM framework. While we found slightly larger parameter estimates for most items, due to the generally longer RTs required for auditory choices, we found an attentional discounting parameter, �, that was remarkably similar to that found in visual choices. In our model fitting, the best fitting, group-level � parameter was 0.40, which is extremely close to the standard � found in the visual, preference-based choice literature of 0.30. Despite apparent similarities between the two domains, we can’t directly compare these two parameter distributions, as they were estimated from different populations separated not only by physical distance but also in time. Given our final set of analyses investigating listening time vs. last

96 fixation effects, we suspect that attention is most likely operating in a different capacity in the auditory domain, yet this is not apparent in our aDDM results. This speaks to the importance of carefully studying and understanding one’s data before applying complex models.

While attention seems to be playing some role in auditory choices, namely that people are more likely to choose the last item they have listened to, the effects we found are different from those in the visual domain. One large reason that this may be the case is that switching between songs required more effort than the eye-movements measured in the visual domain. In fact, some research in the visual domain suggest that eye-movements, although less effortful than button presses, still come at some cost, either in effort or in time, and thus subjects attempt to optimize the number of fixations before a choice so that they can make the fewest eye movements while gathering a maximum amount of evidence (Cassey et al., 2013; Song, Wang, Zhang, & Li,

2019). Due to the more effortful switching in our auditory task, subjects may have been deciding whether a song was good enough to choose, or whether they should listen to the other song. This could cause a final fixation effect, without any effect of individual fixation length.

Additionally, we may not be accurately measuring attention with this paradigm. Instead, subjects may be able to attend to songs without switching the white noise once they have identified both songs. That is, we may be unable to measure internal attentional allocation with the current paradigm.

This study is a first step towards understanding how decisions are made in different modalities. While this study has specifically focused on the role of attention in binary choices, it could easily be expanded to answer other questions within the choice literature. One such area of research is in multi-attribute decision-making. As music is a complex stimulus with many features, attending selectively to one feature over another could influence the final choice

(Busemeyer & Townsend, 1993; Cohen, Kang, & Leise, 2017; Krucien, Ryan, & Hermens,

2017; Rosen & Rosenkoetter, 1976). The auditory, preference-based choice literature is also in its infancy, meaning that the exact role of attention in auditory choice has not yet been explored.

While the current study made attempts at a causal explanation, it would be simple to alter the length of time subjects could listen to each option, and manipulate choices in this manner (Armel et al., 2008).

While our study leaves many questions open as to the exact role of auditory attention in auditory choice, it does establish that attention plays a part in preference-based decision-making, although the exact mechanism through which attention influences choice appears to differ between the auditory and visual domain.

Chapter 5: Discussion

In this dissertation, I have explored the concept of attention as a causal influence in preference-based choices. I have demonstrated that this influence is due, at least in part, to the fact that attention modifies the neural response to the values of items. Additionally, I have shown that the role of attention is not limited to visual choices, but in fact spans modalities into the auditory domain.

In the first set of studies, I showed that manipulating attention in a seemingly unrelated search task can spill over into a choice task, thus directing attention without altering the choice task with potentially noticeable changes. This attentional manipulation, which occurs outside of the choice task, affects subsequent choices, with seemingly no decrease in effectiveness over the course of the task. In addition, the extent to which an individual learned the attentional manipulation in the search task translated to an increase in the effect of attention in the choice task. These findings are further corroborated by evidence from eye-tracking, which showed a robust increase in first fixations to the side of the screen that subjects had learned to attend to.

While the first set of studies provides extensive evidence that attention plays a causal role in choice, the mechanism behind this causal role of attention is left unexplored. Thus, in the second study, we used fMRI to observe, as directly as possible, how attention modified the value signals in the brain. We did this on two levels, the first being at the instantaneous level (i.e. the incoming value signal) as well as in the cumulative signal (i.e. the total, relative evidence accumulated during the choice process). We showed that not only does the activity for the

99 attended item’s value increase as subjects attend to it, but that this increase is then integrated into the cumulative value signal, thus affecting the choice process itself.

In the final study, I showed that many of the patterns observed between attention and preference-based choice in the visual domain also transfer into a purely auditory modality. These patterns include choosing in line with one’s preferences, taking longer to make more difficult decisions, and choosing items that one has attended to last. Unlike in the visual domain, however, once one controls for final fixations, there is no effect of listening time on choice. This indicates that while attention plays a role in both visual and auditory choices, the exact mechanisms may differ between the two modalities.

Limitations

The studies in Chapter 2 covered an extensive number of manipulations and techniques to ensure that we were manipulating and measuring attentional learning and spillover into the choice task. Despite these precautions, it still has its limitations. The first of these limitations comes from the search task itself. While prior literature in probability cueing asserts that this type of learning tends to be implicit, that is it occurs without the learner’s conscious awareness, we did not directly test this assumption (Jiang et al., 2014). While it seems unlikely that conscious awareness of the manipulation could influence our final results, it is possible that those who were aware of the manipulation could have experienced some demand effects. We believe that this is unlikely, however, given that the extent to which subjects learned the probabilities predicted the extent to which it affected their choices.

In addition to possible demand effects, Chapter 2 does not explain the mechanism behind the attentional spillover. Namely, we showed that the manipulation did not influence subjects to

100 attend to the rich side longer, but rather it influenced first fixations towards the rich side. This seems to be sufficient to create a choice bias, but prior literature does not show that the first fixation has any influence above and beyond any other fixation in the choice process (Krajbich et al., 2010). Further research is needed to explore the possibility that there is a primacy effect, in that items that are attended to first receive a boost in evidence accumulation beyond that of any other fixation.

Chapter 3, simply due to its use of brain imaging, is also limited in interpretation. While it’s important to know where value, both instantaneous and cumulative, is calculated in the brain, and how attention might influence these calculations at the neural level, it’s important to acknowledge that brain data is inherently correlational. The vmPFC, striatum, dmPFC, and IPS are all clearly implicated in the calculation of value, and attention seems to influence this calculation. However, without directly manipulating the brain activity, we cannot definitively state that these areas are necessary for the calculation of value, or that attention is directly influencing the calculations here. In fact, the boost in signal could be reflecting changes in signal elsewhere in the brain. This is most clearly demonstrated by the motor cortex results. While it’s unlikely that the motor cortex is calculating the cumulative value of the decision, the activity here reflects that of the dmPFC, which could in turn be epiphenomenal to another brain area responsible for the calculation of cumulative value.

Finally, Chapter 4’s study examined the role of attention in auditory choices. While this study made large strides into how auditory decisions are made, it too is limited in scope. Namely, the cost of switching between songs was effortful and unnatural, unlike eye-movements. This may have caused some artifacts in our data such as a strong bias for subjects to choose the last song they listened to, and the low number of overall fixations to each item compared to the

101 visual domain. This could also explain why we find a robust final fixation effect without an accompanying listening advantage effect. Such questions could be explored by manipulating the amount of time subjects listened to each song. This would completely eliminate any effects that are due to switching costs, as attention shifts would be induced by the experimenter, as well as address whether attention could play a causal role in choice in the auditory domain. In addition, while the study was designed to mimic the visual choice literature as closely as possible, to more directly compare the effects of attention in the auditory domain to those in the visual domain, our design does leave open questions about generalizability. Namely, it is rare to make decisions between two auditory stimuli that are both playing at the same time. One might encounter this scenario at a party, when deciding which group conversation, one would like to join, but most frequently, auditory decisions are made sequentially. That is, one auditory stimulus plays, and then stops while the next stimulus plays. These variations in choice environment could easily be explored in future studies.

Future Directions

Although this dissertation does have limitations, this opens up several avenues of research for future studies. In particular, how attention affects the calculation of value in the brain is a fairly new area of study, leaving many questions. One such question, addressed earlier, is whether the brain regions identified in these studies and in this dissertation are required to calculate value, and whether the effect of attention occurs even in the absence of one or more of these areas. This is an extremely difficult avenue of research, but there are methods that can enhance or disrupt brain functioning in cortical areas. Some of these have already been applied in the realm of decision making, such as transcranial magnetic stimulation (TMS), transcranial

102 direct current stimulation (tDCS), and transcranial alternating current stimulation (tACS) (Camus et al., 2009; Knoch et al., 2006; Philiastides, Auksztulewicz, Heekeren, & Blankenburg, 2011;

Polanía et al., 2015). By using one of these methods, one can determine whether a specific brain area is necessary in the calculation of value, and whether the effects of attention on choice depend on a specific brain region.

Another area that requires much more extensive research is the realm of auditory decision making. This area is relatively unexplored, despite humans making many auditory choices throughout their lifetime. One of the more straightforward avenues of research is to directly compare auditory and visual preference-based decision-making within subjects. This would offer a direct comparison of the role of attention in one modality to the other. It would also allow for the direct comparison of model parameters in the aDDM, which would offer a statistical analysis of whether these parameters were significantly different or not. But first we must explore whether other paradigms might induce a relationship between listen duration and choice in the auditory domain. One possible avenue to address this would be through direct manipulation of the listening duration to each auditory stimulus.

Conclusions

While modality does seem to matter when investigating the exact role of attention in choice, it is clear that preferences are not the sole determinant of the outcome. Instead, other factors can influence the decision process. In this dissertation, I specifically focused on the role of attention in these preference-based choices., and the neural mechanisms behind attention’s influence in the choice process. The findings outlined here demonstrate a causal role of visual attention in visual, preference-based choice, a clear neural mechanism through which attention

103 affects the choice process, and that attention affects choices across multiple modalities, albeit not in the same way.

104

Bibliography

Armel, K. C., Beaumel, A., & Rangel, A. (2008). Biasing simple choices by manipulating relative visual attention. Judgment and Decision Making, 3(5), 396–403. Arnd, F., Claudiu, D., Karin, R., & Leder, S. (2012). Brand-Related Background Music and Consumer Choice. ACR North American Advances, NA-40. Retrieved from http://acrwebsite.org/volumes/1011762/volumes/v40/NA-40 Aschenbrenner, K. M., Albert, D., & Schmalhofer, F. (1984). Stochastic choice heuristics. Acta Psychologica, 56(1), 153–166. https://doi.org/10.1016/0001-6918(84)90015-5 Ashby, N. J. S., Dickert, S., & Glöckner, A. (2012). Focusing on what you own: Biased information uptake due to ownership. Judgment & Decision Making, 7(3), 1–24. Ashby, N. J. S., Jekel, M., Dickert, S., & Glöckner, A. (2016). Finding the right fit: A comparison of process assumptions underlying popular drift-diffusion models. Journal of Experimental Psychology: Learning, Memory, and Cognition, 42(12), 1982–1993. https://doi.org/10.1037/xlm0000279 Baddeley, A., Lewis, V., Eldridge, M., & Thomson, N. (1984). Attention and retrieval from long-term memory. Journal of Experimental Psychology: General, 113(4), 518–540. https://doi.org/10.1037/0096-3445.113.4.518 Bartra, O., McGuire, J. T., & Kable, J. W. (2013). The valuation system: A coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value. NeuroImage, 76, 412–427. https://doi.org/10.1016/j.neuroimage.2013.02.063 Bear, A., & Bloom, P. (2016). A Simple Task Uncovers a Postdictive Illusion of Choice. Psychological Science, 27(6), 914–922. https://doi.org/10.1177/0956797616641943 Bogacz, R., Brown, E., Moehlis, J., Holmes, P., & Cohen, J. D. (2006). The physics of optimal decision making: A formal analysis of models of performance in two-alternative forced choice tasks. Psychological Review, 113(4), 700–765. Bogacz, R., Hu, P. T., Holmes, P. J., & Cohen, J. D. (2010). Do humans produce the speed– accuracy trade-off that maximizes reward rate? Quarterly Journal of Experimental Psychology, 63(5), 863–891. https://doi.org/10.1080/17470210903091643 Bordalo, P., Gennaioli, N., & Shleifer, A. (2012). Salience Theory of Choice Under Risk. The Quarterly Journal of Economics, 127(3), 1243–1285. https://doi.org/10.1093/qje/qjs018 Brainard, D. H. (1997). The psychophysics toolbox. Spatial Vision, 10, 433–436. Brown, S. D., & Heathcote, A. (2008). The simplest complete model of choice response time: Linear ballistic accumulation. Cognitive Psychology, 57(3), 153–178. https://doi.org/10.1016/j.cogpsych.2007.12.002 Busemeyer, J. R., & Townsend, J. T. (1993). Decision field theory: A dynamic-cognitive approach to decision making in an uncertain environment. Psychological Review, 100(3), 432–459. https://doi.org/10.1037/0033-295X.100.3.432

105

Camus, M., Halelamien, N., Plassmann, H., Shimojo, S., O’Doherty, J., Camerer, C., & Rangel, A. (2009). Repetitive transcranial magnetic stimulation over the right dorsolateral prefrontal cortex decreases valuations during food choices. European Journal of Neuroscience, 30(10), 1980–1988. https://doi.org/10.1111/j.1460-9568.2009.06991.x Cassey, T. C., Evens, D. R., Bogacz, R., Marshall, J. A. R., & Ludwig, C. J. H. (2013). Adaptive Sampling of Information in Perceptual Decision-Making. PLoS ONE, 8(11). https://doi.org/10.1371/journal.pone.0078993 Cavanagh, J. F., Wiecki, T. V., Cohen, M. X., Figueroa, C. M., Samanta, J., Sherman, S. J., & Frank, M. J. (2011). Subthalamic nucleus stimulation reverses mediofrontal influence over decision threshold. Nature Neuroscience, 14(11), 1462–1467. https://doi.org/10.1038/nn.2925 Cavanagh, J. F., Wiecki, T. V., Kochar, A., & Frank, M. J. (2014). Eye tracking and pupillometry are indicators of dissociable latent decision processes. Journal of Experimental Psychology: General, 143(4), 1476–1488. https://doi.org/10.1037/a0035813 Chen, W. J., & Krajbich, I. (2017). Computational modeling of epiphany learning. Proceedings of the National Academy of Sciences, 114(18), 4637–4642. https://doi.org/10.1073/pnas.1618161114 Chib, V. S., Rangel, A., Shimojo, S., & O’Doherty, J. P. (2009). Evidence for a Common Representation of Decision Values for Dissimilar Goods in Human Ventromedial Prefrontal Cortex. The Journal of Neuroscience, 29(39), 12315–12320. https://doi.org/10.1523/JNEUROSCI.2575-09.2009 Chun, M. M., Golomb, J. D., & Turk-Browne, N. B. (2011). A Taxonomy of External and Internal Attention. Annual Review of Psychology, 62(1), 73–101. https://doi.org/10.1146/annurev.psych.093008.100427 Chun, M. M., & Johnson, M. K. (2011). Memory: Enduring Traces of Perceptual and Reflective Attention. Neuron, 72(4), 520–535. https://doi.org/10.1016/j.neuron.2011.10.026 Cohen, A. L., Kang, N., & Leise, T. L. (2017). Multi-attribute, multi-alternative models of choice: Choice, reaction time, and process tracing. Cognitive Psychology, 98, 45–72. https://doi.org/10.1016/j.cogpsych.2017.08.001 Colas, J. T., & Lu, J. (2017). Learning Where to Look for High Value Improves Decision Making Asymmetrically. Frontiers in Psychology, 8. https://doi.org/10.3389/fpsyg.2017.02000 Corbetta, M., Akbudak, E., Conturo, T. E., Snyder, A. Z., Ollinger, J. M., Drury, H. A., … Shulman, G. L. (1998). A Common Network of Functional Areas for Attention and Eye Movements. Neuron, 21(4), 761–773. https://doi.org/10.1016/S0896-6273(00)80593-0 De Martino, B. (2006). Frames, Biases, and Rational Decision-Making in the Human Brain. Science, 313(5787), 684–687. https://doi.org/10.1126/science.1128356 Druker, M., & Anderson, B. (2010). Spatial probability aids visual stimulus discrimination. Frontiers in Human Neuroscience, 4, 63. https://doi.org/10.3389/fnhum.2010.00063 Egeth, H. E., & Yantis, S. (1997). Visual attention: Control, representation, and time course. Annual Review of Psychology, 48(1), 269–297. Eimer, M., Velzen, J. van, & Driver, J. (2002). Cross-Modal Interactions between Audition, Touch, and Vision in Endogenous Spatial Attention: ERP Evidence on Preparatory States and Sensory Modulations. Journal of Cognitive Neuroscience, 14(2), 254–271. https://doi.org/10.1162/089892902317236885 106

Fenske, M. J., & Raymond, J. E. (2006). Affective Influences of Selective Attention. Current Directions in Psychological Science, 15(6), 312–316. https://doi.org/10.1111/j.1467-8721.2006.00459.x Fiedler, S., & Glöckner, A. (2012). The Dynamics of Decision Making in Risky Choice: An Eye- Tracking Analysis. Frontiers in Psychology, 3. https://doi.org/10.3389/fpsyg.2012.00335 Fiedler, S., Glöckner, A., Nicklisch, A., & Dickert, S. (2013). Social Value Orientation and information search in social dilemmas: An eye-tracking analysis. Organizational Behavior and Human Decision Processes, 120(2), 272–284. https://doi.org/10.1016/j.obhdp.2012.07.002 Fisher, G. (2017). An attentional drift diffusion model over binary-attribute choice. Cognition, 168, 34–45. https://doi.org/10.1016/j.cognition.2017.06.007 Folke, T., Jacobsen, C., Fleming, S. M., & De Martino, B. (2016). Explicit representation of confidence informs future value-based decisions. Nature Human Behaviour, 1(1), 0002. https://doi.org/10.1038/s41562-016-0002 Forstmann, B. U., Ratcliff, R., & Wagenmakers, E. J. (2016). Sequential Sampling Models in Cognitive Neuroscience: Advantages, Applications, and Extensions. Annual Review of Psychology, 67(1), 641–666. https://doi.org/10.1146/annurev-psych-122414-033645 Franco-Watkins, A. M., & Johnson, J. G. (2011). Applying the decision moving window to risky choice: Comparison of eye-tracking and mouse-tracing methods. Judgment & Decision Making, 6(8), 740–749. Geng, J. J., & Behrmann, M. (2002). Probability Cuing of Target Location Facilitates Visual Search Implicitly in Normal Participants and Patients with Hemispatial Neglect. Psychological Science, 13(6), 520–525. https://doi.org/10.1111/1467-9280.00491 Geng, J. J., & Behrmann, M. (2005). Spatial probability as an attentional cue in visual search. Perception & Psychophysics, 67(7), 1252–1268. Glaholt, M. G., & Reingold, E. M. (2011). Eye movement monitoring as a process tracing methodology in decision making research. Journal of Neuroscience, Psychology, and Economics, 4(2), 125–146. https://doi.org/10.1037/a0020692 Glöckner, A., & Herbold, A.-K. (2011). An eye-tracking study on information processing in risky decisions: Evidence for compensatory strategies based on automatic processes. Journal of Behavioral Decision Making, 24(1), 71–98. https://doi.org/10.1002/bdm.684 Gluth, S., Rieskamp, J., & Büchel, C. (2012). Deciding When to Decide: Time-Variant Sequential Sampling Models Explain the Emergence of Value-Based Decisions in the Human Brain. Journal of Neuroscience, 32(31), 10686–10698. https://doi.org/10.1523/JNEUROSCI.0727-12.2012 Gold, J. I., & Shadlen, M. N. (2001). Neural computations that underlie decisions about sensory stimuli. Trends in Cognitive Sciences, 5(1), 10–16. https://doi.org/10.1016/S1364- 6613(00)01567-9 Gold, J. I., & Shadlen, M. N. (2007). The Neural Basis of Decision Making. Annual Review of Neuroscience, 30(1), 535–574. https://doi.org/10.1146/annurev.neuro.29.051605.113038 Gwinn, R., Leber, A. B., & Krajbich, I. (2019). The spillover effects of attentional learning on value-based choice. Cognition, 182, 294–306. https://doi.org/10.1016/j.cognition.2018.10.012 Hanks, T. D., & Summerfield, C. (2017). Perceptual Decision Making in Rodents, Monkeys, and Humans. Neuron, 93(1), 15–31. https://doi.org/10.1016/j.neuron.2016.12.003

107

Hanks, T., Kiani, R., & Shadlen, M. N. (2014). A neural mechanism of speed-accuracy tradeoff in macaque area LIP. ELife, 3, e02260. https://doi.org/10.7554/eLife.02260 Hare, T. A., Malmaud, J., & Rangel, A. (2011). Focusing Attention on the Health Aspects of Foods Changes Value Signals in vmPFC and Improves Dietary Choice. The Journal of Neuroscience, 31(30), 11077–11087. https://doi.org/10.1523/JNEUROSCI.6383-10.2011 Hare, T. A., Schultz, W., Camerer, C. F., O’Doherty, J. P., & Rangel, A. (2011). Transformation of stimulus value signals into motor commands during simple choice. Proceedings of the National Academy of Sciences, 108(44), 18120–18125. https://doi.org/10.1073/pnas.1109322108 Hare, T., Camerer, C., & Rangel, A. (2009). Self-Control in Decision-Making Invovles Modulation of the vmPFC Valuation System. Science, 324(5927), 644–648. https://doi.org/10.1126/science.1169957 Hillyard, S. A., Vogel, E. K., & Luck, S. J. (1998). Sensory gain control (amplification) as a mechanism of selective attention: electrophysiological and neuroimaging evidence. Philosophical Transactions of the Royal Society of London B: Biological Sciences, 353(1373), 1257–1270. https://doi.org/10.1098/rstb.1998.0281 Hoffman, J. E., & Subramaniam, B. (1995). The role of visual attention in saccadic eye movements. Perception & Psychophysics, 57(6), 787–795. https://doi.org/10.3758/BF03206794 Isham, E. A., & Geng, J. J. (2013). Looking Time Predicts Choice but Not Aesthetic Value. PLoS ONE, 8(8), e71698. https://doi.org/10.1371/journal.pone.0071698 Janiszewski, C., Kuo, A., & Tavassoli, N. T. (2013). The Influence of Selective Attention and Inattention to Products on Subsequent Choice. Journal of Consumer Research, 39(6), 1258–1274. https://doi.org/10.1086/668234 Jiang, T., Potters, J., & Funaki, Y. (2016). Eye-tracking Social Preferences: Eye-tracking Social Preferences. Journal of Behavioral Decision Making, 29(2–3), 157–168. https://doi.org/10.1002/bdm.1899 Jiang, Y. V., Swallow, K. M., & Rosenbaum, G. M. (2013). Guidance of spatial attention by incidental learning and endogenous cuing. Journal of Experimental Psychology: Human Perception and Performance, 39(1), 285–297. https://doi.org/10.1037/a0028022 Jiang, Y. V., Swallow, K. M., Rosenbaum, G. M., & Herzig, C. (2013). Rapid acquisition but slow extinction of an attentional bias in space. Journal of Experimental Psychology: Human Perception and Performance, 39(1), 87–99. https://doi.org/10.1037/a0027611 Jiang, Y. V., Won, B.-Y., & Swallow, K. M. (2014). First saccadic eye movement reveals persistent attentional guidance by implicit learning. Journal of Experimental Psychology. Human Perception and Performance, 40(3), 1161–1173. https://doi.org/10.1037/a0035961 Johnson, J. G., & Busemeyer, J. R. (2005). A Dynamic, Stochastic, Computational Model of Preference Reversal Phenomena. Psychological Review, 112(4), 841–861. https://doi.org/10.1037/0033-295X.112.4.841 Kable, J. W., & Glimcher, P. W. (2007). The neural correlates of subjective value during intertemporal choice. Nature Neuroscience, 10(12), 1625–1633. https://doi.org/10.1038/nn2007 Kane, M. J., & Engle, R. W. (2000). Working-memory capacity, proactive interference, and divided attention: Limits on long-term memory retrieval. Journal of Experimental

108

Psychology: Learning, Memory, and Cognition, 26(2), 336–358. https://doi.org/10.1037/0278-7393.26.2.336 Khodadadi, A., Fakhari, P., & Busemeyer, J. R. (2017). Learning to allocate limited time to decisions with different expected outcomes. Cognitive Psychology, 95, 17–49. https://doi.org/10.1016/j.cogpsych.2017.03.002 Kim, B. E., Seligman, D., & Kable, J. W. (2012). Preference Reversals in Decision Making Under Risk are Accompanied by Changes in Attention to Different Attributes. Frontiers in Neuroscience, 6. https://doi.org/10.3389/fnins.2012.00109 Knoch, D., Gianotti, L. R. R., Pascual-Leone, A., Treyer, V., Regard, M., Hohmann, M., & Brugger, P. (2006). Disruption of Right Prefrontal Cortex by Low-Frequency Repetitive Transcranial Magnetic Stimulation Induces Risk-Taking Behavior. Journal of Neuroscience, 26(24), 6469–6472. https://doi.org/10.1523/JNEUROSCI.0804-06.2006 Knoeferle, K. M., Knoeferle, P., Velasco, C., & Spence, C. (2016). Multisensory brand search: How the meaning of sounds guides consumers’ visual attention. Journal of Experimental Psychology: Applied, 22(2), 196–210. https://doi.org/10.1037/xap0000084 Konovalov, A., & Krajbich, I. (2016). Gaze data reveal distinct choice processes underlying model-based and model-free reinforcement learning. Nature Communications, 7, 12438. https://doi.org/10.1038/ncomms12438 Kovach, C. K., Sutterer, M. J., Rushia, S. N., Teriakidis, A., & Jenison, R. L. (2014). Two systems drive attention to rewards. Frontiers in Psychology, 5. https://doi.org/10.3389/fpsyg.2014.00046 Krajbich, I., & Rangel, A. (2011). Multialternative drift-diffusion model predicts the relationship between visual fixations and choice in value-based decisions. Proceedings of the National Academy of Sciences, 108(33), 13852–13857. https://doi.org/10.1073/pnas.1101328108 Krajbich, Ian, Armel, C., & Rangel, A. (2010). Visual fixations and the computation and comparison of value in simple choice. Nature Neuroscience, 13(10), 1292–1298. https://doi.org/10.1038/nn.2635 Krajbich, Ian, Hare, T., Bartling, B., Morishima, Y., & Fehr, E. (2015). A Common Mechanism Underlying Food Choice and Social Decisions. PLOS Computational Biology, 11(10), e1004371. https://doi.org/10.1371/journal.pcbi.1004371 Krajbich, Ian, Lu, D., Camerer, C., & Rangel, A. (2012). The Attentional Drift-Diffusion Model Extends to Simple Purchasing Decisions. Frontiers in Psychology, 3. https://doi.org/10.3389/fpsyg.2012.00193 Krucien, N., Ryan, M., & Hermens, F. (2017). Visual attention in multi-attributes choices: What can eye-tracking tell us? Journal of Economic Behavior & Organization, 135, 251–267. https://doi.org/10.1016/j.jebo.2017.01.018 Leow, R. P., & Morgan-Short, K. (2004). To think aloud or not to think aloud: The Issue of Reactivity in SLA Research Methodology. Studies in Second Language Acquisition, 26(1), 35–57. https://doi.org/10.1017/S0272263104026129 Lim, S.-L., O’Doherty, J. P., & Rangel, A. (2011). The Decision Value Computations in the vmPFC and Striatum Use a Relative Value Code That is Guided by Visual Attention. Journal of Neuroscience, 31(37), 13214–13223. https://doi.org/10.1523/JNEUROSCI.1246-11.2011

109

Lohse, G. L., & Johnson, E. J. (1996). A Comparison of Two Process Tracing Methods for Choice Tasks. Organizational Behavior and Human Decision Processes, 68(1), 28–43. https://doi.org/10.1006/obhd.1996.0087 McGinty, V. B., Rangel, A., & Newsome, W. T. (2016). Orbitofrontal Cortex Value Signals Depend on Fixation Location during Free Viewing. Neuron, 90(6), 1299–1311. https://doi.org/10.1016/j.neuron.2016.04.045 Mormann, M., Navalpakkam, V., Koch, C., & Rangel, A. (2012). Relative visual saliency differences induce sizable bias in consumer choice. Journal of Consumer Psychology, 22(1), 67–74. https://doi.org/10.1016/j.jcps.2011.10.002 Mulder, M. J., van Maanen, L., & Forstmann, B. U. (2014). Perceptual decision neurosciences – A model-based review. Neuroscience, 277, 872–884. https://doi.org/10.1016/j.neuroscience.2014.07.031 Mullett, T. L., & Stewart, N. (2016). Implications of visual attention phenomena for models of preferential choice. Decision, 3(4), 231–253. https://doi.org/10.1037/dec0000049 Noguchi, T., & Stewart, N. (2014). In the attraction, compromise, and similarity effects, alternatives are repeatedly compared in pairs on single dimensions. Cognition, 132(1), 44–56. https://doi.org/10.1016/j.cognition.2014.03.006 North, A. C., Hargreaves, D. J., & McKendrick, J. (1999). The influence of in-store music on wine selections. Journal of Applied Psychology, 84(2), 271. https://doi.org/10.1037/0021- 9010.84.2.271 Orquin, J. L., & Mueller Loose, S. (2013). Attention and choice: A review on eye movements in decision making. Acta Psychologica, 144(1), 190–206. https://doi.org/10.1016/j.actpsy.2013.06.003 Pärnamets, P., Johansson, P., Hall, L., Balkenius, C., Spivey, M. J., & Richardson, D. C. (2015). Biasing moral decisions by exploiting the dynamics of eye gaze. Proceedings of the National Academy of Sciences, 112(13), 4170–4175. https://doi.org/10.1073/pnas.1415250112 Pärnamets, P., Johansson, R., Gidlöf, K., & Wallin, A. (2016). How Information Availability Interacts with Visual Attention during Judgment and Decision Tasks: Information Availability and Visual Attention. Journal of Behavioral Decision Making, 29(2–3), 218– 231. https://doi.org/10.1002/bdm.1902 Payne, J. W., Braunstein, M. L., & Carroll, J. S. (1978). Exploring predecisional behavior: An alternative approach to decision research. Organizational Behavior and Human Performance, 22(1), 17–44. https://doi.org/10.1016/0030-5073(78)90003-X Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spatial Vision, 10(4), 437–442. https://doi.org/10.1163/156856897X00366 Philiastides, M. G., Auksztulewicz, R., Heekeren, H. R., & Blankenburg, F. (2011). Causal Role of Dorsolateral Prefrontal Cortex in Human Perceptual Decision Making. Current Biology, 21(11), 980–983. https://doi.org/10.1016/j.cub.2011.04.034 Pisauro, M. A., Fouragnan, E., Retzler, C., & Philiastides, M. G. (2017). Neural correlates of evidence accumulation during value-based decisions revealed via simultaneous EEG- fMRI. Nature Communications, 8, 15808. https://doi.org/10.1038/ncomms15808 Plassmann, H., O’Doherty, J., & Rangel, A. (2007). Orbitofrontal Cortex Encodes Willingness to Pay in Everyday Economic Transactions. Journal of Neuroscience, 27(37), 9984–9988. https://doi.org/10.1523/JNEUROSCI.2131-07.2007

110

Polania, R., Krajbich, I., Grueschow, M., & Ruff, C. C. (2014). Neural oscillations and synchronization differentially support evidence accumulation in perceptual and value- based decision making. Neuron, 82, 709–720. Polanía, R., Moisa, M., Opitz, A., Grueschow, M., & Ruff, C. C. (2015). The precision of value- based choices depends causally on fronto-parietal phase coupling. Nature Communications, 6. https://doi.org/10.1038/ncomms9090 Polonio, L., Di Guida, S., & Coricelli, G. (2014). Strategic Sophistication and Attention in Games: an Eye-Tracking Study. Retrieved from http://hdl.handle.net/2013/ULB- DIPOT:oai:dipot.ulb.ac.be:2013/159867 Polonio, L., Di Guida, S., & Coricelli, G. (2015). Strategic sophistication and attention in games: An eye-tracking study. Games and Economic Behavior, 94, 80–96. https://doi.org/10.1016/j.geb.2015.09.003 Posada, D., & Buckley, T. R. (2004). Model Selection and Model Averaging in Phylogenetics: Advantages of Akaike Information Criterion and Bayesian Approaches Over Likelihood Ratio Tests. Systematic Biology, 53(5), 793–808. https://doi.org/10.1080/10635150490522304 Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85(2), 59. Ratcliff, R. (2002). A diffusion model account of response time and accuracy in a brightness discrimination task: Fitting real data and failing to fit fake but plausible data. Psychonomic Bulletin & Review, 9(2), 278–291. https://doi.org/10.3758/BF03196283 Ratcliff, R., Gomez, P., & McKoon, G. (2004). A Diffusion Model Account of the Lexical Decision Task. Psychological Review, 111(1), 159–182. https://doi.org/10.1037/0033- 295X.111.1.159 Ratcliff, R., & McKoon, G. (1996). Bias effects in implicit memory tasks. Journal of Experimental Psychology: General, 125(4), 403. Ratcliff, R., & Smith, P. L. (2004). A Comparison of Sequential Sampling Models for Two- Choice Reaction Time. Psychological Review, 111(2), 333–367. https://doi.org/10.1037/0033-295X.111.2.333 Ratcliff, R., & Starns, J. J. (2013). Modeling confidence judgments, response times, and multiple choices in decision making: Recognition memory and motion discrimination. Psychological Review, 120(3), 697–719. https://doi.org/10.1037/a0033152 Ratcliff, R., Thapar, A., & McKoon, G. (2004). A diffusion model analysis of the effects of aging on recognition memory. Journal of Memory and Language, 50(4), 408–424. https://doi.org/10.1016/j.jml.2003.11.002 Raymond, J. E., Fenske, M. J., & Tavassoli, N. T. (2003). Selective Attention Determines Emotional Responses to Novel Visual Stimuli. Psychological Science, 14(6), 537–542. https://doi.org/10.1046/j.0956-7976.2003.psci_1462.x Reber, R., Winkielman, P., & Schwarz, N. (1998). Effects of Perceptual Fluency on Affective Judgments. Psychological Science, 9(1), 45–48. https://doi.org/10.1111/1467-9280.00008 Reutskaja, E., Nagel, R., Camerer, C. F., & Rangel, A. (2011). Search Dynamics in Consumer Choice under Time Pressure: An Eye- Tracking Study. The American Economic Review, 101(2), 900–926. Rodriguez, C. A., Turner, B. M., Zandt, T. V., & McClure, S. M. (2015). The neural basis of value accumulation in intertemporal choice. European Journal of Neuroscience, 42(5), 2179–2189. https://doi.org/10.1111/ejn.12997

111

Roe, R. M., Busemeyer, J. R., & Townsend, J. T. (2001). Multialternative decision field theory: A dynamic connectionst model of decision making. Psychological Review, 108(2), 370– 392. https://doi.org/10.1037//0033-295X.108.2.370 Rosen, L. D., & Rosenkoetter, P. (1976). An eye fixation analysis of choice and judgment with multiattribute stimuli. Memory & Cognition, 4(6), 747–752. https://doi.org/10.3758/BF03213243 Russo, J. E., & Leclerc, F. (1994). An Eye-Fixation Analysis of Choice Processes for Consumer Nondurables. Journal of Consumer Research, 21(2), 274–290. Scheibehenne, B., Jamil, T., & Wagenmakers, E.-J. (2016). Bayesian Evidence Synthesis Can Reconcile Seemingly Inconsistent Results: The Case of Hotel Towel Reuse. Psychological Science. https://doi.org/10.1177/0956797616644081 Schonberg, T., Bakkour, A., Hover, A. M., Mumford, J. A., Nagar, L., Perez, J., & Poldrack, R. A. (2014). Changing value through cued approach: an automatic mechanism of behavior change. Nature Neuroscience, 17(4), 625–630. https://doi.org/10.1038/nn.3673 Shadlen, M. N., & Shohamy, D. (2016). Decision Making and Sequential Sampling from Memory. Neuron, 90(5), 927–939. https://doi.org/10.1016/j.neuron.2016.04.036 Shevlin, B., Smith, S., Hausfeld, J., & Krajbich, I. (n.d.). Influence of Explicit Value Cues on the Decision Process. Shi, S. W., Wedel, M., & Pieters, F. G. M. (Rik). (2012). Information Acquisition During Online Decision Making: A Model-Based Exploration Using Eye-Tracking Data. Management Science, 59(5), 1009–1026. https://doi.org/10.1287/mnsc.1120.1625 Shi, S. W., Wedel, M., & Pieters, F. G. M. (Rik). (2013). Information Acquisition During Online Decision Making: A Model-Based Exploration Using Eye-Tracking Data. Management Science, 59(5), 1009–1026. https://doi.org/10.1287/mnsc.1120.1625 Shimojo, S., Simion, C., Shimojo, E., & Scheier, C. (2003). Gaze bias both reflects and influences preference. Nature Neuroscience, 6(12), 1317–1322. https://doi.org/10.1038/nn1150 Smith, S., Krajbich, I., & Webb, R. (2019). Estimating the dynamic role of attention via random utility. Retrieved from https://flow- attachments.s3.amazonaws.com/files/original/4fd32808-eb4e-4268-8633- 997efae0b535/JESA-S-18- 00081.pdf?AWSAccessKeyId=AKIAIBTSCHX5TTCP3XQA&Expires=1551126160&S ignature=WJTaKQYiOGv2HjQDeVFCrt6WQc0%3D Smith, S. M., & Krajbich, I. (2018). Attention and choice across domains. Journal of Experimental Psychology: General, 147(12), 1810. https://doi.org/10.1037/xge0000482 Smith, S. M., & Krajbich, I. (2019). Gaze Amplifies Value in Decision Making. Psychological Science, 30(1), 116–128. https://doi.org/10.1177/0956797618810521 Song, M., Wang, X., Zhang, H., & Li, J. (2019). Proactive Information Sampling in Value-Based Decision-Making: Deciding When and Where to Saccade. Frontiers in Human Neuroscience, 13. https://doi.org/10.3389/fnhum.2019.00035 Spiegelhalter, D. J., Best, N. G., Carlin, B. P., & Linde, A. V. D. (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 64(4), 583–639. https://doi.org/10.1111/1467-9868.00353 Stewart, N., Hermens, F., & Matthews, W. J. (2015). Eye Movements in Risky Choice: Eye Movements in Risky Choice. Journal of Behavioral Decision Making, n/a-n/a. https://doi.org/10.1002/bdm.1854 112

Tavares, G., Perona, P., & Rangel, A. (2017). The Attentional Drift Diffusion Model of Simple Perceptual Decision-Making. Frontiers in Neuroscience, 11. https://doi.org/10.3389/fnins.2017.00468 Towal, R. B., Mormann, M., & Koch, C. (2013). Simultaneous modeling of visual saliency and value computation improves predictions of economic choice. Proceedings of the National Academy of Sciences, 110(40), E3858–E3867. https://doi.org/10.1073/pnas.1304429110 Usher, M., & McClelland, J. L. (2001). The time course of perceptual choice: the leaky, competing accumulator model. Psychological Review, 108(3), 550. Vaidya, A. R., & Fellows, L. K. (2015). Testing necessary regional frontal contributions to value assessment and fixation-based updating. Nature Communications, 6, 10120. https://doi.org/10.1038/ncomms10120 Venkatraman, V., Payne, J. W., & Huettel, S. A. (2014). An overall probability of winning heuristic for complex risky decisions: Choice and eye fixation evidence. Organizational Behavior and Human Decision Processes, 125(2), 73–87. https://doi.org/10.1016/j.obhdp.2014.06.003 Vlaev, I., Chater, N., & Stewart, N. (2008). Seeing is not enough: manipulating choice options causes focusing and preference change in multiattribute risky decision-making. Journal of Behavioral Decision Making, 21(5), 556–574. https://doi.org/10.1002/bdm.601 Vuilleumier, P. (2015). Affective and motivational control of vision. Current Opinion in Neurology, 28(1), 29. https://doi.org/10.1097/WCO.0000000000000159 Wallsten, T. S., & Barton, C. (1982). Processing probabilistic multidimensional information for decisions. Journal of Experimental Psychology: Learning, Memory, and Cognition, 8(5), 361–384. https://doi.org/10.1037/0278-7393.8.5.361 Wang, J. T., Spezio, M., & Camerer, C. (2009). Pinocchio’s Pupil: Using Eyetracking and Pupil Dilation to Understand Truth-Telling and Deception in Sender-Receiver Game (SSRN Scholarly Paper No. ID 1466532). Retrieved from Social Science Research Network website: http://papers.ssrn.com/abstract=1466532 Wang, J. T., Spezio, M., & Camerer, C. F. (2010). Pinocchio’s Pupil: Using Eyetracking and Pupil Dilation to Understand Truth Telling and Deception in Sender-Receiver Games. The American Economic Review, 100(3), 984–1007. https://doi.org/10.1257/aer.100.3.984 Wang, X.-J. (2002). Probabilistic Decision Making by Slow Reverberation in Cortical Circuits. Neuron, 36(5), 955–968. https://doi.org/10.1016/S0896-6273(02)01092-9 Wiecki, T. V., Sofer, I., & Frank, M. J. (2013). HDDM: Hierarchical Bayesian estimation of the Drift-Diffusion Model in Python. Frontiers in Neuroinformatics, 7. https://doi.org/10.3389/fninf.2013.00014 Willemsen, M. C., Böckenholt, U., & Johnson, E. J. (2011). Choice by value encoding and value construction: Processes of loss aversion. Journal of Experimental Psychology: General, 140(3), 303. Winkielman, P., & Cacioppo, J. T. (2001). Mind at ease puts a smile on the face: Psychophysiological evidence that processing facilitation elicits positive affect. Journal of Personality and Social Psychology, 81(6), 989–1000. https://doi.org/10.1037//0022- 3514.81.6.989 Woldorff, M. G., Gallen, C. C., Hampson, S. A., Hillyard, S. A., Pantev, C., Sobel, D., & Bloom, F. E. (1993). Modulation of early sensory processing in human auditory cortex during

113

auditory selective attention. Proceedings of the National Academy of Sciences, 90(18), 8722–8726. Zajonc, R. B. (1968). Attitudinal effects of mere exposure. Journal of Personality and Social Psychology, 9(2, Pt.2), 1–27. https://doi.org/10.1037/h0025848 Zhang, J., & Rowe, J. B. (2014). Dissociable mechanisms of speed-accuracy tradeoff during visual perceptual learning are revealed by a hierarchical drift-diffusion model. Frontiers in Neuroscience, 8. https://doi.org/10.3389/fnins.2014.00069

114

Appendix A: Additional Analyses for Chapter 2, Experiment 3

Because RT difference had been a consistent modulator of our attentional manipulation in the prior experiments, we chose to investigate its effects more thoroughly. In particular, it occurred to us at this point that it might be important to account not just for the initial RT difference, but for the RT difference in the second search task as well. We reasoned that the degree to which subjects learned the new probability manipulation would impact the reversal of the choice bias. Therefore, we tested another model that additionally included the RT differences from both search tasks interacted with extinction.

Rating difference was a significant predictor of choice (� = 0.44, 95% CI [0.38, 0.50]) but again significantly decreased in the second choice task (� =-0.088, 95% CI [-0.15, -0.025]).

As in the prior experiments' models with RT difference, the intercept was no longer significant in the first choice task (� = 0.011, 95% CI [-0.058, 0.080]), nor did it significantly change in the second choice task (� = 0.047, 95% CI [-0.045, 0.14]). Of particular interest are the RT- difference variables. Neither RT-difference variable was significant when looking at the first choice task (first search task: � = 0.067, 95% CI [-0.043, 0.180], second search task: � = -0.007,

95% CI [-0.12, 0.11]). The first effect is in the expected direction and actually larger in magnitude than in Experiment 2, but fails to reach significance, presumably because of all the variables in the model. The second effect should indeed be zero since the RT difference from the second search task should not have any effect on the choice task that came before it, at least after accounting for the first RT difference.

However, when interacted with the extinction variable (i.e. looking at the second choice task), both RT-difference variables were highly significant (first search task: � = -0.20, 95% CI

115

[-0.34, -0.061], second search task: � = 0.19, 95% CI [0.039, 0.34]). Because both RT- difference variables were coded using the original rich and sparse sides, the second effect indicates that subjects who were less influenced by the second search task (more positive RT difference) were significantly more likely to choose from the original rich side. Moreover, the first regressor indicates that, after controlling for these other effects, subjects who were more susceptible to the attentional learning (those with a bigger RT difference in the first task) were significantly less likely to choose from the rich side. In other words, these subjects were more strongly influenced by the second search task and thus more likely to unlearn (or reverse) the spatial bias in their choices.

Test – comparing pre-reversal to post-reversal

As outlined in our third pre-registration, we turned to our second choice task to investigate the effect of the attentional-learning extinction/reversal. To do so we used data from both choice tasks in several mixed effects logistic regressions of choosing the rich side using every combination of rating difference, overall value, accuracy, RT difference (from the first search task), extinction (a binary variable coding for whether the choices happen before extinction (0) or after (1)), and the interaction of extinction with each of these variables.

The model with the lowest AIC included rating difference, extinction, and their interaction (Table S8). Rating difference was, as always, significant (� = 0.44, 95% CI [0.39,

0.50]). Critically, the intercept was significantly positive, indicating that participants initially had a bias to choose from the rich side (�= 0.04, 95% CI [0.00, 0.09]), while the extinction variable was negative, albeit not significantly so, suggesting that the bias disappeared, and slightly reversed, in the second choice task (� = -0.05, 95% CI [-0.11, 0.01]). Similar to our result from 116

Experiment 2, we also find that subjects were less likely to choose the higher rated items during the second set of choice trials (�= -0.09, 95% CI [-0.15, -0.03])

117

Appendix B: Additional Analyses from Chapter 2, Experiment 4

Test – Comparing pre-reversal to post-reversal

As stated in the pre-registration we also ran several analyses designed to compare pre and post reversal by combining both choice tasks and re-running our simple regressions of first fixation to the rich side on rating difference interacted with a dummy variable coding for the reversal (0 = pre-reversal choices, 1 = post-reversal choices). All analyses are in terms of the original rich side. In this regression, we find a significant bias to fixate the rich side in the first set of choices (intercept � = 0.335, 95% CI [0.029, 0.641]) and a non-significant decrease in this bias in the second choice task (reversal � = -0.159, 95% CI [-0.426, 0.108]). As before, rating difference was not a significant predictor of first fixations either before or after the reversal

(rating difference � = 0.020, 95% CI [-0.066, 0.106]; reversal x rating difference � = -0.012,

95% CI [-0.137, 0.113]).

Turning to choice behavior we see a similar pattern of results. Rating difference predicts choices in both pre and post reversal choices (rating difference � = 0.486, 95% CI [0.402,

0.572]; reversal x rating difference � = -0.072, 95% CI [-0.195, 0.051]). There is also a significant bias to choose from the rich side in the first set of choices (intercept � = 0.087, 95%

CI [0.009, 0.165]) that non-significantly decreases post-reversal (reversal � = -0.073, 95% CI [-

0.173, 0.027]). The lack of significant interactions with reversals may be due to the slow rate of re-learning seen in the second search task.

Complex regressions including the RT difference show a similar pattern of results, namely that post-reversal dependent variables do not show any informative results, but are available in Appendix C.

118

Appendix C: Additional Regressions for Chapter 2

All regressions are mixed-effect logistic regressions unless otherwise stated. Highlighted rows indicate the best fitting model. Bolded coefficients are significant at p < 0.05.

Table C. 1: Experiment 1 Models, Excluding Subjects not Above Chance

Table C. 2: Experiment 1 Models, No Exclusions

119

Table C. 3: Experiment 1 Complex Models, Excluding Subjects not Above Chance

Table C. 4: Experiment 2 Models – Second Choices Only

Table C. 5: Experiment 2 Models – Second Choices Only, No Exclusions

120

Table C. 6: Experiment 2 Models – All Choices, No Exclusions

121

Table C. 7: Experiment 3 Models – First Choices Only

Table C. 8: Experiment 3 Models – First Choices Only, No Exclusions

Table C. 9: Experiment 3 Models – Second Choices Only

122

Table C. 10: Experiment 3 Models – Second Choices Only, No Exclusions

123

Table C. 11: Experiment 3 Models – All Choices, No Exclusions

124

Table C. 12: Experiment 4 Models – First Choices Only, Choice Regressions

Table C. 13: Experiment 4 Models – First Choices Only, First Fixation

125

Table C. 14: Experiment 4 Linear Models – First Choices Only, Dwell Time Advantage

Table C. 15: Experiment 4 Models – Second Choice Only

126

Table C. 16: Experiment 4 Models – Second Choices Only, First Fixation

Table C. 17: Experiment 4 Linear Models – Second Choices Only, Dwell Time Advantage

Table C. 18: Experiment 4 Models – All Choices

127

Table C. 19: Experiment 4 Models – All Choices, First Fixation

Table C. 20: Experiment 4 Linear Models – All Choices, Dwell Time Advantage

Table C. 21: All Studies Combined Models

128

Appendix D: BRMS Code

All Bayesian analyses were completed with the use of the BRMS package for R, which generates and runs RStan code. The brms code for the simple model is as follows:

models <- list() for (i in 1:4)

{

Models[[i]] <- brm(data = choiceData[choiceData$experimentNumber ≤ i, ],

formula = chooseRichSide ~ rateDifference +

(1|subjectNumber) + (0+rateDifference|subjectNumber), family = Bernoulli,

Warmup = 1000, iter = 2000, chains = 4, thin = 1, cores = 4, sample_prior = T)

}

The brms code for the full model is as follows:

RTAccOVModels <- list() for (I in 1:4)

{

RTAccOVModels[[i]] <- brm(data = hoiceData[choicedata$experimentNumber ≤ i, ],

formula = chooseRichSide ~ rateDifference + RTDifference +

trainingAccuracy + overallValue + (1|subjectNumber) +

(0+rateDifference|subjectNumber) + (0+RTDifference|subjectNumber) +

(0+trainingAccuracy|subjectNumber) + (0 + overallValue|subjectNumber),

129

Family = Bernoulli, warmup = 1000, iter = 2000, chains = 4, thin = 1, cores = 4,

sample_prior = T)

}

130

Appendix E: Song List

Song Title Artist Duration (s) Youngblood 5 Seconds of Summer 32.5751701 Jai Ho A R Rahma and The Pussycat Dolls 28.7496825 Thunderstruck ACDC 19.4254875 Dancing Queen Abba 231.849796 Mamma Mia Abba 212.321814 Take a Chance on Me Abba 244.96907 Hello Adele 51.5065533 Rolling in the Deep Adele 19.1491157 Skyfall Adele 28.2586168 Someone Like You Adele 36.5026984 Dream On Aerosmith 31.1279592 No One Alicia Keys 25.5430159 Back to Black Amy Winehouse 22.4175737 Rehab Amy Winehouse 26.0482993 Beauty and the Beast Angela Lansbury 8.79902494 Barbie Girl Aqua 14.8020635 Breathin Ariana Grande 20.7131066 God is a Woman Ariana Grande 13.0003175 No Tears Left to Cry Ariana Grande 31.8869161 Harlem Shake Baauer 17.2563492 Beautiful Bazzi feat Camila Cabello 19.9767347 Meant to Be Bebe Rexha 24.5298866 Stayin Alive Bee Gees 25.4093651 Crazy in Love Beyonce feat Jay Z 19.1963492 Single Ladies Beyonce 19.4481633 Hit Me Baby One More Time Britney Spears 20.3877324 Oops I Did it Again Britney Spears 22.274059 Toxic Britney Spears 14.2372109 Grenade Bruno Mars 32.3115873 Just the Way You Are Bruno Mars 35.0352608 Locked Out of Heaven Bruno Mars 27.4172336 Call Me Maybe Carly Rae Jepsen 16.16 Circle of Life Carmen Twillie 46.3738776

131

My Heart Will Go On Celine Dion 18.4110658 The Way I Am Charlie Puth 19.5570295 No Brainer DJ Khaled 18.0068027 Tequila Dan + Shay 24.7378231 Mission Impossible Danny Elfman 13.0527891 Independent Women Destiny's Child 19.4667574 One Dance Drake feat Wizkid & Kyla 18.9379138 God's Plan Drake 21.7002948 Perfect Ed Sheeran 20.944127 Shape of You Ed Sheeran 19.4805215 Tiny Dancer Elton John 26.1380952 Hound Dog Elvis Presley 17.043288 Jailhouse Rock Elvis Presley 9.87435374 Stuck on You Elvis Presley 7.94918367 Hakuna Matata Ernie Sabella 14.0577098 Like a G6 Far East Movement 23.5178685 Simple Florida Georgia Line 9.80564626 We Are Young Fun 19.6953968 Somebody That I Used to Know Gotye feat Kendra 14.7515873 Rich Girl Gwen Stefani 11.344195 Fancy Iggy Azalea feat Charli XCX 20.0222676 Natural Imagine Dragons 22.2897052 Thunder Imagine Dragons 15.0939002 Whatever it Takes Imagine Dragons 15.3338095 Price Tag Jessie J 11.2503628 I Love Rock 'N Roll Joan Jett & The Blackhearts 11.5288209 All of Me John Legend 14.3409297 Supercalifragilisticexpialidocious Julie Andrews 12.1131066 Love Yourself Justin Bieber 18.2327211 Dark Horse Katy Perry feat Juicy J 15.0371202 California Gurls Katy Perry feat Snoop Dogg 15.3612698 E.T Katy Perry 12.1018594 Firework Katy Perry 15.8294785 Tik Tok Ke$ha 15.9533333 Stronger Kelly Clarkson 16.3861225 Party Rock Anthem LMFAO 14.373288 Thunderclouds LSD feat Sia, Diplo, Labrinth 23.2886395 132

Born This Way Lady Gaga 14.8868934 I Like Me Better Lauv 20.8441723 Royals Lorde 11.2820862 Macarena Los Del Rio 9.1560771 What a Wonderful World Louis Armstrong 9.04605442 Broken Lovelytheband 14.5254195 Despacito Luis Fonsi feat Daddy Yankee 13.0644898 Rude MAGIC! 13.1826984 Macklemore & Ryan Lewis feat Thrift Shop Wanz 9.85895692 Can't Hold Us Macklemore & Ryan Lewis 14.1062358 Like a Virgin Madonna 15.8297506 Material Girl Madonna 13.7229932 Vogue Madonna 17.168322 All I Want for Christmas is You Mariah Carey 15.585737 Uptown Funk Mark Ronson feat Bruno Mars 8.51222222 Girls Like You Maroon 5 feat Cardi B 15.4513605 Moves Like Jagger Maroon 5 feat Christina Aguilera 10.4487302 One More Night Maroon 5 19.923288 Happier Marshmello & Bastille 17.5584127 All About that Base Meghan Trainor 14.5782086 Beat It Michael Jackson 21.6044898 Billie Jean Michael Jackson 17.6768027 Thriller Michael Jackson 19.9903628 Best of Both Worlds Miley Cyrus 17.0846032 Wrecking Ball Miley Cyrus 20.5661905 Guiding Light Mumford & Sons 17.3660318 Bang Bang Nancy Sinatra 16.5089116 Cheerleader OMI 15.8730612 Just Give Me a Reason P!nk feat Nate Ruess 10.3460771 Gangnam Style PSY 15.7542177 High Hopes Panic! At the Disco 11.680907 Happy Pharrell Williams 25.0042404 Timber Pitbull feat Ke$ha 15.54161 Rockstar Post Malone feat 21 Savage 23.8382766 Bohemian Rhapsody Queen 14.2059184 We Will Rock You Queen 10.5982086 133

Game of Thrones Ramin Djawadi 17.1278458 We Found Love Rihanna feat Calvin Harris 16.4827664 Work Rihanna feat Drake 13.4091157 Diamonds Rihanna 24.8893878 Pretty Woman Roy Orbison 19.6005442 Writing's on the Wall Sam Smith 27.3609297 Under the Sea Samuel E Wright 18.3830159 Back to You Selena Gomez 19.872517 Hips Don't Lie Shakira feat Wyclef Jean 9.45099773 Waka Waka Shakira 7.56952381 Whenever, Wherever Shakira 18.0619501 Cheap Thrills Sia feat Sean Paul 20.7080952 So Long, Farewell Sound of Music 8.77761905 Wannabe Spice Girls 13.3650794 Eye of the Tiger Survivor 19.9204308 Blank Space Taylor Swift 20.193356 Delicate Taylor Swift 21.2787075 Look What You Made Me Do Taylor Swift 14.8316553 Shake It Off Taylor Swift 12.0840136 We Are Never Ever Getting Back Together Taylor Swift 12.1316553 Boom Boom Pow The Black Eyed Peas 13.2986848 Imma Be The Black Eyes Peas 10.5247166 Closer The Chainsmokers feat Halsey 21.1246712 Sympathy for the Devil The Rolling Stones 15.0431746 Can't Feel My Face The Weeknd 17.1135374 OMG Usher feat will.i.am 7.56791383 Africa Weezer 24.3469388 I Will Always Love You Whitney Houston 21.7790703 See You Again Wiz Khalifa feat Charlie Puth 23.2489342 The Middle Zedd 18.9163719

134