Computational Modeling of Decisions in Mixed Gambles

Thesis submitted by

Nishad Singhi 2016EE10107

under the guidance of Prof. Sumeet Agarwal Prof. Sumitava Mukherjee

in partial fulfilment of the requirements for the award of the degree of

Bachelor of Technology

Department Of Electrical Engineering INDIAN INSTITUTE OF TECHNOLOGY DELHI July 2020 Contents

ABSTRACT 2

1 INTRODUCTION 3

2 Computational models of decision making 5 2.1 Neurophysiology of Choice [BUZM07] ...... 5 2.2 Drift Diffusion Model ...... 5 2.3 The Leaky Competing Accumulator [UM01] ...... 6

3 Modeling mixed gambles using Drift Diffusion Model 8 3.1 Why do people reject mixed gambles? [ZWB18] ...... 8 3.1.1 Experimental Data ...... 8 3.1.2 Modeling ...... 8 3.1.3 Reported and replicated results ...... 9 3.2 Additional dataset ...... 11 3.2.1 Description of dataset ...... 11 3.2.2 DDM fitting results ...... 13 3.3 Discussion ...... 14

4 Modeling mixed gambles using LCA 15 4.1 LCA for value based decisions [UM04] ...... 15 4.1.1 Fitting Methods ...... 16 4.1.2 Experiments ...... 17 4.1.3 Discussion ...... 21

5 General Discussion 23

A Economics models of decision making 26 A.1 Expected Utility Hypothesis [MCWG+95] ...... 26 A.2 Prospect Theory [KT79] ...... 26 A.2.1 Value Function ...... 26 A.2.2 Weighting Function ...... 26 A.2.3 Evaluation of outcomes ...... 27

B Modeling of Loss Aversion for time 28 B.1 Introduction ...... 28 B.2 Experimental Design ...... 28 B.3 Stimuli ...... 28 B.4 Information given to participants ...... 29 ABSTRACT

An important finding in behavioural economics is that people tend to reject mixed gambles. The predominant explanation for this phenomenon is loss aversion, i.e., decision-makers give more subjective weight to losses as compared to gains. However, other psychological mechanisms such as status-quo bias, loss attention, etc. provide alternate explanations for this finding. While experimental studies have provided some evidence for these hypotheses, how much they affect decision makers during mixed gambles is not clear. Moreover, it is difficult to investigate their effect on the decision process within the framework of Prospect theory – the central theory of behavioural economics – because it posits that people make decisions based on utility alone. In this thesis, I use two models of decision making called the Drift Diffusion Model and the Leaky, Competing, Accumulator model to tease apart the effects of these psychological mechanisms. Using data from two risky-choice experiments, I show that people require less evidence to reject gambles than to accept them, and that this bias is the most important mechanism underlying this behaviour, and that high rates of rejection in mixed gambles should not be understood in terms of loss aversion.

2 Chapter 1

INTRODUCTION

Humans are required to make decisions in hundreds of risky situations every day. From deciding which route to take to work, to investing money in the stock market – we have to deal with risks all the time. For decades, people from various backgrounds and disciplines such as Economics, Psychology, Artificial Intelligence, etc. have tried to study how humans make decisions under uncertainty.

One of the most important findings in behavioural economics is that decision makers tend to reject ‘mixed’ gambles having equal probability of resulting in a gain or a loss [KT79]. For instance, consider the following gamble:

You have to choose between two options, S and R: S: gain Rs. 0 with 100% probability R: gain Rs. 1,00,000 with 50% probability lose Rs. 1,00,000 with 50% probability

Even though the probability of winning Rs. 1,00,000 is the same as that of losing Rs. 1,00,000, most people choose S. This suggests that people tend to reject gambles that may potentially result in a loss. However, the nature of the psychological processes underlying this phenomenon is not properly understood.

Several hypotheses have been proposed to explain this finding. According to expected utility theory (See A.1), this may indicate concavity of the utility function. However, this hypoth- esis cannot explain the high rates of rejection in small gambles, as that would require an unreasonably high degree of concavity in the utility function [Rab00]. Prospect theory (See A.2) [KT79] explains this finding by positing that losses are given more subjective value by decision makers than gains of equal magnitude, i.e., the value function for losses is steeper than gains. This idea – known as loss aversion – is the most widely accepted explanation for the high rates of rejection in mixed gambles.

While loss aversion was originally proposed in the context of decisions under risk, it has been generalized to other settings, and has successfully explained numerous other findings such as Endowment effect [KS84]. The idea that losses and gains have different effects on our psychological state has been extended to other domains such as education, politics, marketing, etc.

However, Prospect theory has several shortcomings. It does not comment upon the time 4 taken by a person to reach a decision. Moreover, while it aims to describe how people actually make decisions (i.e., it is a descriptive model) instead of being a normative model (like expected utility theory), there is no explanation of the psychological processes mentioned in it. It even fails to account for the role of aspects of cognition such as emotion, attention, etc. These issues undermine the role of loss aversion as the mechanism behind rejection of mixed gambles.

In addition to the claims mentioned above, there is increasing evidence suggesting that loss aversion is not a ubiquitous phenomenon – as assumed by Prospect theory – and is context dependent. For example, [MSPS17] showed that losses loom larger than gains only for large magnitudes, and might even be valued less than gains for smaller magnitudes. [EE13] showed that loss aversion is not observed when the option of rejecting a gamble is presented explicitly as opposed to being the status quo.

In addition to loss aversion, other mechanisms explaining the rejection of mixed gambles have been investigated. One of them is Loss Attention, which suggests that losses attract more attention than (but may not necessarily be weighted more than) gains of similar magnitude [YH13] [LSMPH19]. Another possible explanation is that people reject mixed gambles because of ‘psychological inertia’ or a status-quo bias. It is possible that the behavioural finding results from the interplay of these effects, and while there have been some empirical studies about these hypotheses, it is difficult to disentangle their effects from each other as well as from loss aversion using qualitative methods alone. Moreover, as has already been mentioned, economic models such as expected utility theory and prospect theory rely only on the utility of a prospect and cannot accommodate these psychological mechanisms. Hence, computational models that are grounded in psychological principles are required to study these mechanisms quantitatively.

One class of computational models, which is quite attractive for this enterprise, is the family of evidence accumulation models. Essentially, these models assume that people accumulate noisy evidence from stimuli over time to make decisions. These models have been successful in explaining both choice and reaction time data in ‘perceptual decision’ making tasks, so it is possible that the brain uses a similar mechanism in economic decision making tasks. Moreover, these models are capable of incorporating mechanisms such as the status-quo bias or asymmetric attention for losses and gains. Additionally, some of these models have pro- vided elegant explanations for context effects in economic decision making [UM04], [BT93]. Finally, some of these models incorporate known principles of information processing in the brain, increasing their plausibility as models of economic decision making.

In chapter 2, I briefly describe mechanisms of decision making in the brain and how they can inspire models of decision making. In chapters 3 and 4, I employ these models to investigate the mechanisms underlying people’s aversion to mixed gambles. This is followed by a general discussion in chapter 5.

c 2021, Indian Institute of Technology Delhi Chapter 2

Computational models of decision making

2.1 Neurophysiology of Choice [BUZM07]

Let us begin with a common perceptual decision making task (instead of value-based de- cisions that we have considered so far): participants are shown a cloud of identical dots moving on a screen. In every trial, a proportion of the dots are moving in the same di- rection, while the other dots are moving randomly. The task is to identify the direction of prevalent movement.

Sensory neurons (such as those in the MT1 region in the brain) detect particular directions of movement. For instance, a neuron that is tuned to detect objects moving to the right will fire only when an object moving to the right appears in the visual field and not otherwise. The information coded by the these neurons is inherently noisy, so making a decision based on the information obtained at just one instant may cause errors. Instead, information sampled at different instants is used collectively to make a decision.

It has been observed that the firing rates of some neurons in the lateral intraparietal area (LIP) and the frontal eye field (FEF) increases over time. Moreover, the easier the task, the larger is the rate of increase. These observations suggest that neurons in these areas might be integrating the information provided by the sensory neurons, averaging out the noise.

2.2 Drift Diffusion Model

The Drift Diffusion Model is a very influential model that has been able to successfully account for accuracy and reaction-time data from a wide range of behavioural experiments involving decisions between two options. Parameters of the model represent components of information processing, so, studying these parameters (and their variation under differ- ent experimental conditions) can provide insights about the processes underlying decision- making.

I will first describe the model as applied to perceptual decision-making tasks: the model assumes that beginning from a starting point, information about the stimulus is sequen- tially accumulated in a noisy fashion until one of two thresholds is reached (see Figure 2.1). The rate of accumulation is called the drift rate (ν). The drift rate depends on the quality of the information extracted from the stimulus. At each time t, noise from a Gaus- sian distribution having 0 mean is added to the accumulation process. Due to the noise, 2.3 The Leaky Competing Accumulator [UM01] 6

processes having the same drift rate may hit the boundary at different times (giving rise to reaction-time distributions) or may even hit different thresholds (giving rise to different choices) [RM08]. A discrete-time version of the model may be mathematically described as a random-walk

√ yt − yt−1 = ν∆t + ∆tst (2.1)

t ∼ N (0, 1) where s represents the amount of diffusion [FPW+17].

Figure 2.1: A schematic representation of the Drift Diffusion Model [MWR+12]

Significance of the parameters: [RM08] [RH19] • Stronger evidence is linked to a higher drift rate, making the decisions faster. Also, the effect of noise is lower in this case, reducing the probability of errors. • If the decision threshold is large, more evidence is required before a decision is reached. This means that the decisions will be slower. Also, the decisions will be more accurate because a larger number of steps will be able to average out the noise. • The starting point is related to any prior biases that the subject might have. If the starting point is closer to the lower boundary, the model will require less evidence to reach the lower boundary, making the responses corresponding to this boundary faster and more likely to be observed. To reach the upper boundary, a larger amount of evidence will be required, making this response slower and less likely to be observed. If the decision-maker is unbiased, the starting point should be exactly halfway between the two boundaries.

2.3 The Leaky Competing Accumulator [UM01]

The Leaky Competing Accumulator (LCA) model attempts to incorporate several findings from the neurphysiology of decision-making. The Drift Diffusion Model has a single cu- mulative random variable (y in Equation 2.1). However, the LCA model has a separate variable for each possible option, due to which, it can be applied to cases with more than 2 options.

c 2021, Indian Institute of Technology Delhi 2.3 The Leaky Competing Accumulator [UM01] 7

Figure 2.2: Architecture of the LCA model: Arrows denote excitatory connections, lines with filled circles denote inhibitory connections [BUZM07]

Figure 2.2 shows the basic architecture of the LCA model for a decision task involving two

options. Accumulators y1 and y2 correspond to the two options. The input to accumulator i is Ii along with white noise dWi of magnitude c. As soon as any of the accumulators reaches a threshold, decision corresponding to that accumulator is made. These accumulators in- hibit each other by a connection of weight w. Lateral inhibition between the accumulators is incorporated because it introduces relative decision criterion. Such a criterion is desir- able because other models having relative criterion (such as the diffusion models) have been shown to be optimal in the sense that they have the fastest reaction-times for a given accu- racy level. Also, diffusion model has been able to successfully account for a wide variety of experimental data, suggesting the importance of relative decision criteria.

In traditional accumulator models, the accumulation process proceeds indefinitely without any loss of information. Hence, given infinite time, perfect accuracy can be achieved on any task (unless the stimuli are identical). However, in most experiments, accuracy asymptotes to a finite level. To account for this, LCA assumes that the accumulation process is leaky, i.e., the accumulated information decays over time.

Note: Sequential sampling models have been successfully applied to a range of perceptual decision making tasks. However, the nature of value based and perceptual decision making tasks is slightly different. In perceptual decision-making tasks, the stimuli are typically stochastic, and models of perceptual decision making attempt to incorporate the noise in the stimuli (along with neural noise). On the other hand, the stimuli in value-based decision tasks is not stochastic. However, the cognitive representation of various values is likely to be noisy [JBSS20]. Hence, while the nature of noise is different in both cases, the computational problem is similar [MMH+10].

c 2021, Indian Institute of Technology Delhi Chapter 3

Modeling mixed gambles using Drift Diffusion Model

Consider the following gamble: If a coin lands heads, you win $11. If it lands tails, you lose $10. This is an example of a mixed gamble – a gamble that can result in both a gain and a loss. In several experiments, it has been found that people tend to reject such gambles [KT79] (even though the expected utility of the gamble is greater than zero).

3.1 Why do people reject mixed gambles? [ZWB18]

3.1.1 Experimental Data

Participants (n=49) had to play a series of mixed gambles. Each gamble could result in either a gain or a loss with equal probability (see Figure 3.1). The values of gain and loss were selected from the set {10, 20, 30, 40, 50, 60, 70, 80, 90, 100}.

Figure 3.1: Experiment Stimuli: In each trial, the values of possible gain and loss are shown on the screen

3.1.2 Modeling

The expected utility of an equiprobable gamble with a possible gain Gi and a possible loss

Li is

Ui = Gi − λLi

where λ is the loss aversion parameter, which accounts for the unequal weighting of gains

and losses. λ can be estimated using a logistic regression: Ai ∼ α + βGGi − βLLi, where Ai 3.1 Why do people reject mixed gambles? [ZWB18] 9

th is the participant’s binary response to the i gamble, βG and βL are regression coefficients

giving λ = βG/βL, and α is an additive intercept for regression.

According to this regression model, and other economic models of decision-making, the decision made by the participant depends only on the utility of the gamble. However, according to the idea of psychological inertia presented in Section ??, participants may also have a predecisional bias for rejecting mixed gambles. To accommodate this bias, this decision process was modeled using a Drift Diffusion Model. The drift rate for a trial i is assumed to be

νi = α + βGGi − βLLi

The response is made when the accumulated evidence reaches either +θ or −θ. The starting point is represented by γ, and the predecisional bias is zero if γ = 0. If γ > 0, the participants have a bias to accept the gambles, and if γ < 0, they have a bias to reject the gambles.

[ZWB18] used the HDDM package to fit the Drift Diffusion Model to the choice and RT data. 4 chains of 50,000 samples were generated where the first 25,000 samples were discarded as burn-in samples and a thinning of 2 was applied.

3.1.3 Reported and replicated results

I fitted a Drift Diffusion Model on the data collected in this experiment using the HDDM package. However, due to the large amount of time required to generate chains of 50,000 samples each, I generated a chain of 2,500 samples with the first 1,000 samples discarded as burn-in. The trace and low values of auto-correlation of the posterior distribution in Figure 3.2 indicate convergence of the sampling process.

(a) Posterior trace of starting point (b) Autocorrelation of starting point

Figure 3.2: Trace and Autocorrelation of one of the parameters of the model (starting point bias) suggesting that the sampling process has converged.

The reported value for the mean of λ = βG/βL was 2.11 (see Figure 3.3a) and the value I obtained is 2.11 (see Figure 3.3b). Clearly, both values are greater than one, suggesting that the participants were overall loss averse.

The averaged value of γ across all participants was reported to be -0.24 (see Figure 3.4a),

c 2021, Indian Institute of Technology Delhi 3.1 Why do people reject mixed gambles? [ZWB18] 10

(a) Reported (b) Replicated

Figure 3.3: Participant-wise values of βG and βL (a) reported in the paper and (b) obtained by me by fitting the model on the same data. These figures show that βLoss > βGain for almost all participants, which means λ > 1 for most participants. and I computed it to be -0.21 (see Figure 3.4b). This indicates that the participants had a predecisional bias for rejecting the gambles.

(a) Reported (b) Replicated

Figure 3.4: Participant-wise values of γ and α (a) reported in the paper and (b) obtained by me by fitting the model on the same data. These figures show that most participants have a prior tendency to reject gambles (i.e., γ < 1).

As far as choice is concerned, both loss aversion and predecisional bias have an effect of pushing the participants towards rejecting the gambles. It is not clear which of the two plays a more important role in explaining the observations. To disentangle the effects of these two, two restricted models were fitted to the data: • No loss aversion: βG = βL • No predecisional bias: γ = 0

c 2021, Indian Institute of Technology Delhi 3.2 Additional dataset 11

The models were compared among themselves (and with the full model without restrictions) using the deviance information criterion (DIC) [SBCVDL02]. Smaller DIC values indicate better performance. It can be seen from Table 3.1 and Figure 3.5 that the model without loss aversion provides a better fit than the model without predecisional bias.

Table 3.1: DIC values for the three models

Model DICreported DICreplicated Full 16,871 16,869

βG = βL 17,332 17,331 γ = 0 17,979 17,977

(b) Replicated (a) Reported

Figure 3.5: ∆DIC values of the three models (∆DIC = DIC - DICfull).AssmallervaluesofDICindicatebetterfit, itisclearthatamodelwithpredecisionalbiasbutnolossaversionfitsthedatabetterthanamodelwithlossaversionbutnopredecisionalbias.

To further compare the descriptive power of predecisional bias and loss aversion, the rela- tionship of the tendency to accept gambles with these two parameters was analyzed. From Figures 3.6 and 3.7, we can observe that the probability that a participant will accept a gamble is very strongly correlated with the predecisional bias parameter (Pearson correla- tion = 0.91 (reported), 0.907 (replication)) and relatively weakly correlated with the loss aversion parameter (Pearson correlation = -0.25 (reported) and -0.24 (replication)).

3.2 Additional dataset

In order to analyse the generalizability of the results presented in Section 3.1.3, I fitted DDM to model Study 1 (non-adaptive risky choice) of [KK19].

3.2.1 Description of dataset

Participants (n=39) had to complete a series of trials, each of which consisted of a choice between a sure non-negative amount and an equiprobable mixed gamble. The total number of trials per participant was 276, and in 224 of these trials, the value of the sure option

c 2021, Indian Institute of Technology Delhi 3.2 Additional dataset 12

(a) Reported (b) Replicated

Figure 3.6: Participant-wise relation of the probability of accepting the gamble with pre- decisional bias indicating that rejection rates of participants correlate strongly with their individual predecisional bias parameters.

(a) Reported (b) Replicated

Figure 3.7: Participant-wise relation of the probability of accepting the gamble with loss aversion parameter indicating that rejection rates are weakly correlated with individual loss aversion parameters. was $0. The average value of gain in these 224 trials was $7 and the maximum gain was $12. The average value of loss in these trials was $5.9 and the maximum value of loss was $24. To maintain similarity with [ZWB18]’s dataset, I fitted the model only on these 224 trials.

In each trial, subjects (n = 39) chose between a sure amount of money and a 50/50 lottery that included a positive amount and a loss (which in some rounds was equal to $0). Each subject first completed a three-trial practice followed by 276 paid trials. These trials were presented in two blocks of the same 138 choice problems, each presented in random order without any pause between the two blocks. Subjects were endowed with $17 and additionally

c 2021, Indian Institute of Technology Delhi 3.2 Additional dataset 13 earned the outcome of one randomly selected trial (in case of a loss it was subtracted from the endowment).

3.2.2 DDM fitting results

I followed the modeling procedure explained in Section 3.1.2. The mean value of λ = βG/βL was 1.78, which suggests that participants were loss averse. The mean value of γ was -0.28, which indicates that the participants had a predecisional bias for rejecting the gambles.

To further investigate the effects of loss aversion and predecisional bias, two restricted models were also fitted to the data (as done in Section 3.1.3). The DIC values for these models are presented in Table 3.2.

Table 3.2: DIC values for the three models on the second dataset

Model DIC Full 22,582

βG = βL 23,197 γ = 0 23,438

Figure 3.8 shows P(accept) vs. RT adjusted for choice factor for these three models (two restricted and one full model). From Figure 3.8 and Table 3.2, it is clear that the model with predecisional bias but no loss aversion fits the data better than the model with loss aversion but no predecisional bias.

Figure 3.8: P(accept) vs. Adjusted RT for restricted models on additional dataset. It is clear from this figure that a model with predecisional bias (i.e., no loss aversion) is able to fit the data better than a model with loss aversion (i.e., no bias).

c 2021, Indian Institute of Technology Delhi 3.3 Discussion 14

Figure 3.9 shows the participant-level correlations of P(accept) with γ (Pearson’s correlation = 0.8) and λ (Pearson’s correlation = 0.14). It is evident that P(accept) is quite strongly correlated with γ as compared to λ.

Figure 3.9: Participant-wise relation of the probability of accepting the gamble with (a) pre- decisional bias parameter and (b) loss aversion parameter. P(accept) correlates with predecisional bias more strongly as compared to loss aversion.

3.3 Discussion

The results show that while loss aversion does play a role in the rejection of mixed gambles, a predecisional bias for the status-quo is also an important factor. Using both [ZWB18] and [KK19]’s datasets, it was shown that a model containing predecisional bias but no loss aversion could fit the data whereas a model with loss aversion but no predecisional bias could not. Additionally, a participants predecisional bias parameter correlated more strongly with their rejection rates than their loss aversion parameter did. These results indicate that predecisional bias is the primary determinant of this phenomenon. Essentially, the participants have a bias to reject the gamble even before the gamble is presented. While the loss aversion hypothesis and the status-quo bias hypothesis make similar choice predictions, they make different predictions with regard to reaction time distributions. Drift Diffusion model can be employed to harness the information contained in the empirical reaction time (and choice) data.

c 2021, Indian Institute of Technology Delhi Chapter 4

Modeling mixed gambles using LCA

4.1 LCA for value based decisions [UM04]

The LCA model for perceptual decisions was described in section 2.3. In this section, I will describe a variant of the model that can be applied to value based decisions.

Figure 4.1 shows the model for a decision-making task for three options (A1,A2,A3) having two attributes each (E & Q). At every time iteration, the attentional selection mechanism selects one of the two attributes stochastically. The model assumes that when an explicit reference is not provided, the values are judged relative to each other. The input is then preprocessed (Input preprocessing stage in Figure 4.1) in the following manner

I1 = V (d12) + V (d13)

I2 = V (d23) + V (d21)

I3 = V (d31) + V (d32)

where dij is the differential advantage or disadvantage of option i relative to option j for the attribute chosen by the attentional selection mechanism. If a reference is provided, the values are judged relative to that reference. V is the value function described in section A.2.1. The leaky integration process (final stage in Figure 4.1) follows the equation [BUZM07] ! X yi(t + ∆t) = yi(t) + ∆t −kyi − w yj + Ii + I0 + noise j6=i

negative values of y are replaced by zero. I0 is a positive constant without which, the input to accumulators will always be negative due to the loss-averse value function. k is the leak- age parameter, and w is the lateral inhibition weight. This model has the benefit that it can be applied to more than two options as well as to options with more than one attributes. Additionally, it has been able to explain some violations of expected value and preference reversals such as the similarity, attraction, and compromise effects. 4.1 LCA for value based decisions [UM04] 16

Figure 4.1: Variant of LCA model for value based decision making among 3 alternatives having 2 attributes each

4.1.1 Fitting Methods

Metropolis Algorithm [UM01]

For modeling the data from [ZWB18], a closed-form expression of LCA’s predictions isn’t available. Hence, for the fitting process, simulations were performed. Each swap (iteration of fitting process) begins with a set of parameters for which the predictions from the model is compared with the empirical data. At each swap, a new set of parameters is randomly selected from a Gaussian distribution centred on the parameters that have provided the best fit so far. If the cost for the new parameters is lower than the cost for the best-fitting parameters, the set of best parameters is updated with the new parameters. After each 100 swaps, the standard deviation for the Gaussian distribution is reduced by a factor of 0.99 in order to refine the search process.

During each swap, 75 simulations were performed for each combination of potential gain and loss from the set {10, 20, ..., 100}. At each swap, the cost function for each of these combinations was computed using the Chi-squared cost function (computed separately for accept and reject reaction times) 2 N(pi − πi) Σi (4.1) πi

where N is the total number of observations (reaction times) for a given condition (i.e.,

gain-loss pair), pi is the proportion of points (reaction times) generated by the model lying t in the i h bin and πi is the proportion of experimental points in the same bin. The total cost for the swap was computed by adding the costs for each combination. To test the Metropolis algorithm, I used it to find the coefficients of an arbitrarily chosen

c 2021, Indian Institute of Technology Delhi 4.1 LCA for value based decisions [UM04] 17 third degree polynomial function

f(x) = x3 + 4x2 + x − 5 (4.2)

The final values of the parameters were within a distance of 0.01 from the ground-truth values (see Figure 4.2).

Figure 4.2: Variations of the coefficients of the polynomial in Equation 4.2 through the course of the Metropolis Algorithm. Solid blue line shows the value of the coefficients at different swaps of the process. Red dotted line shows the ground truth values.

Look-up tables

Since the fitting process relies on simulations of the process instead of an analytical expres- sion, the time required to complete one simulation is very important. [Eva19] has shown that sampling random numbers from a Gaussian distribution (for the noise) takes up more than 95% of the simulation time. In order to reduce this time, He proposed using look-up tables instead of pseudo-random number generators. He showed that simulations using this method closely match those with standard random number generators, while significantly reducing the simulation time.

4.1.2 Experiments

Note: In the experiments described below, V was replaced with an identity function to minimize the complexity of the model (unless specified otherwise). Instead of carrying out individual level analysis, I grouped the data from all participants in order to reduce variability among participants. Additionally, instead of judging the values w.r.t. each other as mentioned in Section 4.1, the values were judged w.r.t. a fixed reference of 0.

c 2021, Indian Institute of Technology Delhi 4.1 LCA for value based decisions [UM04] 18

Baseline

In order to keep the number of free parameters to a minimum, I started with the assumption that the thresholds for both the accumulators are equal. This assumption is analogous to having no starting-point bias in the drift diffusion model. Figure 4.3 shows the probability to accept gamble as a function of gain/loss of a gamble. It is clear that the model captures the choice data very well. However, Figure 4.4 shows that the model is not able to capture the relationship between P(accept) and reaction-time data. In fact, the trend in the experimental data is increasing, while the model captures a decreasing trend.

(a) Random Initialization (b) After running Metropolis algorithm

Figure 4.3: Probability of accepting gamble as a function of the ratio of possible gain and loss (a) before running Metropolis algorithm (randomly sampled parameters) and (b) after running Metropolis algorithm. The 5 simulations in the Figure were obtained by running the fitting routine 5 times starting from a different set of initial values each time. The model is able to capture the choice data properly and robustly.

Introducing bias

[ZWB18] showed that a bias towards rejecting gambles is essential to properly capture the choice-RT relationship in this dataset. In order to introduce a similar bias in the LCA, I relaxed the assumption of equal thresholds for the two accumulators. To the best of my knowledge, there is no work investigating biased decisions from LCA.

It can be seen from Figure 4.5 that LCA with unequal thresholds can capture both P(accept) vs. choice-factor adjusted RT and P(accept) vs. gain/loss ratio relationships well. The final value for the threshold was 23 for rejecting the gamble and 40 for accepting the gamble. This means that it is easier (less evidence is required) to reject the gamble than to accept it, which is consistent with the results of [ZWB18].

While Figure 4.5b provides evidence that LCA with unequal thresholds is able to capture the joint distribution of RT and choice data quite well, in order to further investigate this point, I plotted Quantile Probability Functions (QPF) of this model and DDM (for reference). QPF

c 2021, Indian Institute of Technology Delhi 4.1 LCA for value based decisions [UM04] 19

Figure 4.4: Probability of accepting gamble vs. choice-factor adjusted RT after 2000 swaps of Metropolis algorithm. The 5 simulations in the Figure were obtained by running the fitting routine 5 times starting from a different set of initial values each time. Clearly, the model is not able to capture the relationship between acceptance rate and reaction times.

(a) P(accept) vs. Gain/Loss (b) P(accept) vs. choice-factor adjusted RT

Figure 4.5: Comparison of the fit of model having unequal thresholds with experimental data. LCA with unequal thresholds is able to fit the data reasonably well. provides a richer description of the RT distribution as compared to P(accept) vs. choice factor adjusted RT graphs.

Figure 4.6 presents the two Quantile Probability functions. The quantiles plotted are 0.1, 0.3, 0.5, 0.7, and 0.9. It is evident that while both LCA and DDM adequately capture the range of RTs for each quantile, none of them is able to fit the data points quite well. Moreover, there is a peak towards right (around the 0.4 mark on the x-axis) for most of the experimental data. DDM is able to capture this peak in some cases whereas LCA shows a peak towards left in most cases. However, these differences seem to be very subtle and inconsistent across quantiles.

c 2021, Indian Institute of Technology Delhi 4.1 LCA for value based decisions [UM04] 20

Figure 4.6: Quantile Probability function. Solid lines represent LCA, dashed lines represent DDM, and the markers represent experimental data. The quantiles plotted are 0.1, 0.3, 0.5, 0.7, 0.9. Both LCA and DDM struggle with capturing the data points exactly, but their quantiles are roughly in the same range as that of the experimental data.

Comparing bias and Loss Aversion

In order to compare the effects of loss aversion and unequal thresholds, I fitted two different models: one having unequal thresholds but equal weight given to gains and losses (i.e., using the identity function as the value function), and the other model with both thresholds forced to be equal but the value function that allows for unequal weighting of losses and gains (henceforth called the ‘loss aversion’ model)

v(x) = x, x ≥ 0, v(x) = λx, x < 0 where λ is a free parameter.

In order to investigate the ‘Loss Attention’ theory [YH13], I also fitted another model in which the probability of focusing on gains and losses was not set = 0.5 and was free (but the thresholds were forced to be equal and the value function was equal to the identity function). Finally, I fitted a baseline with none of these assumptions as a baseline model. The results

c 2021, Indian Institute of Technology Delhi 4.1 LCA for value based decisions [UM04] 21

are presented in Figure 4.7.

Figure 4.7: P(accept) vs. choice factor adjusted RT for constrained LCA models fitted with different assumptions. LCA with different thresholds fits the data better than LCA with loss aversion and LCA with loss attention.

It is clear from Figure 4.7 that a model with unequal thresholds fits the data best. Moreover, the model with loss aversion fails to capture the relationship of choice and RT even qualita- tively. These findings are consistent with the results of [ZWB18]. Moreover, the model with unequal probability of focusing on the two attributes also seems fit the data better than the baseline model as well as the model with loss aversion, which provides some support for the loss attention theory. For the model with loss aversion, the value of λ after fitting is 1.65, which is consistent with the value of λ reported for humans in the literature [GJH07]. For the loss attention model, the value of the probability of focusing on loss is 0.67, which provides support to the idea that people pay more attention to losses as compared to gains [ZWB18].

4.1.3 Discussion

I modeled the experimental data from [ZWB18] using a LCA to assess its validity as a model of economic decision making. Further, I used compared several constrained models to compare several hypotheses that seek to explain why people reject mixed gambles.

LCA is unbiased when the thresholds of all accumulators are equal. It was observed that a model with equal thresholds could capture the choice distribution of the experimental data well, but struggled with the joint distribution of choice and RT data.

c 2021, Indian Institute of Technology Delhi 4.1 LCA for value based decisions [UM04] 22

LCA with unequal thresholds represents an unbiased case of decision making and could capture both choice data as well as the joint distribution of choice and RT data quite well. This provides support to the validity of LCA as a model of economic decision making and illustrates that its performance was comparable to DDM, which has been shown to account for data from value based decision making tasks [MMH+10]. Moreover, the threshold for accepting gambles was larger than the threshold for rejecting them, which implies that more evidence is required to accept gambles. This corresponds to a starting point bias towards rejection in the drift diffusion model. Hence, the results from this modeling study are consistent with the results of [ZWB18].

Further, I compared three cognitive hypotheses that can potentially explain the empirical finding that people have a tendency to reject mixed gambles: • Loss Aversion: This is a valuation bias in which the utility function is steeper for losses as compared to gains. It is widely accepted to be the reason underlying the high rates of rejection of mixed gambles. • Evidence accumulation bias: According to this hypothesis, people require less evidence to reject gambles than to accept them. In a drift diffusion model, this cor- responds to a starting point bias, whereas in the LCA, this corresponds to unequal thresholds for the accumulators. • Loss Attention: This hypothesis states that losses attract more attention as com- pared to gains. The drift diffusion framework cannot accommodate this bias, but it can be incorporated in the LCA by varying the parameter that controls the switching probability.

In order to compare these hypotheses, I fitted three restricted models, each having only one of these mechanisms. It was observed that only the model with evidence accumulation bias was able to capture the data properly. Importantly, the model with loss aversion but no evidence accumulation bias could not capture the data properly, which indicates that loss aversion alone cannot explain the high rates of rejection in mixed gambles, which is the popular view till now. These findings are consistent with the results of [ZWB18].

c 2021, Indian Institute of Technology Delhi Chapter 5

General Discussion

In this study, I attempted to study the cognitive mechanisms behind one of the most im- portant findings in behavioural economics: people tend to reject mixed gambles. This phenomenon is often cited as an evidence of loss aversion, or the idea that people weigh losses more than gains of equal magnitude.

Historically, behavioural economics has employed utility-based models, i.e., models that de- scribe how people make decisions depending on the utility of the prospects at hand. While these models, such as Prospect Theory have been successful in explaining a wide variety of phenomena, they fail to provide a satisfactory account of the psychological processes underly- ing these models. Moreover, they are unable to accommodate aspects of the decision-making process that go beyond utility, such as context, emotion, attention, inertia, etc., which have been shown to robustly and systematically affect people’s choices. In this work, I employ computational models of economic decision making that are grounded in cognition to quan- titatively compare and disentangle the effects of some psychological mechanisms that aim to explain why people dislike mixed gambles. In particular, I investigated the role of loss aversion, status-quo bias, and the loss attention hypothesis in explaining choice and reaction time data from mixed gambles experiments.

I started by successfully replicating the results of [ZWB18] who showed that a predecisional bias favouring rejection of gambles has a larger effect on the fit of DDM as compared to loss aversion. Additionally, using participant-level correlations, they showed that a person’s predecisional bias parameter correlates more strongly with their rejection rate than their loss aversion parameter does. To test whether these results can be generalized, I modeled an additional dataset using similar approach and found that the results from this study were consistent with those of [ZWB18]. These results suggest that a predecisional bias – and not loss aversion – is the most important factor behind high rates of rejection in mixed gambles.

It should be noted that even though the drift diffusion model can successfully capture choice and reaction time data from myriad decision making experiments, it struggles to explain data from time-controlled paradigms of studying decision making. Moreover, it’s neural plausibility is questionable, since it does not explain how neurons in the brain perform this computation. Additionally, it is restricted to decisions between two options. Due to these issues, it is important to investigate models that take inspiration from, and incorporate principles of information processing in the brain, can explain findings from time-controlled paradigms, and can be extended beyond two alternatives. One such model is the Leaky, 24

Competing, Accumulator (LCA) model. Another attractive feature of this model is that it comprises of an attention switching mechanism, which can allow us to understand how losses and gains affect attention.

LCA model for value based decisions was proposed around 15 years ago, but there have been hardly any empirical studies testing its performance on such tasks. Also, there are virtually no studies addressing biased decisions in the LCA framework. Hence, the first step towards using it to compare various mechanisms is to test whether it can capture data from economic decision-making tasks and understand the role and nature of bias in this process.

It was found in my experiments that LCA with equal thresholds was able to capture the choice pattern in [ZWB18]’s data, but failed to capture the joint distribution of choice and reaction times. This finding is consistent with the results from the DDM analysis, since equal thresholds correspond to no bias in the model. I introduced in the LCA, a bias similar to the predecisional bias in the DDM by relaxing the condition that both thresholds in the LCA need to be equal. This adjustment allowed the model to capture both choice as well as choice vs. RT (adjusted for choice factors) data quite well. The threshold for accepting the gambles was higher than the threshold for rejecting the gambles, which suggests that people find it easier (require less evidence) to reject gambles that to accept them, which is consistent with the results from DDM analysis. To compare the effects of loss aversion, unequal thresholds, and unequal attention to losses and gains, I fitted three restricted models. Only the model with unequal thresholds was able to fit the data properly, suggesting that this bias is more important than the other two in explaining experimental data.

While previous studies have provided experimental evidence for a status-quo bias [EE13] [GR18], they could not disentangle its effect from loss aversion, as the predictions of both these mechanisms are towards rejection of gambles. Hence, studies that did not use com- putational models could not conclusively prove the importance of status-quo bias over loss aversion. In this work, using multiple models and datasets, it was demonstrated that a bias in the amount of information required to reject or accept gambles is the primary force un- derlying the phenomenon being discussed. More broadly, this demonstrates the importance and role of computational models in studying psychology.

While results from applying DDM and LCA lead to similar inferences, there are some dif- ferences between the two approaches. Firstly, the drift diffusion model does not have an attention mechanism, which makes it impossible to test the loss attention theory of [YH13] using DDM. But, the LCA does have an attention switching mechanism, and treating the probability of focusing on loss as a free parameter allowed me to test this theory. After fitting a restricted model without unequal thresholds and loss aversion but having unequal attention towards losses and gains, it was found that this model was not able to capture the experimental data, suggesting that the high rates of rejection in mixed gambles is not a product of loss attention alone. It should be noted that the value of P(focus on loss) after

c 2021, Indian Institute of Technology Delhi 25 running the fitting routine was 0.67, which means that people do pay more attention to losses, as suggested by [YH13].

In addition to this, a separate DDM was fitted for each participant whereas a single LCA was fitted for the entire dataset by discarding participant-level information. This was done because the likelihood function for DDM is analytically defined, hence it is computationally less expensive to fit multiple models using hierarchical Bayesian techniques. In the case of LCA, since the likelihood is not analytically defined, noisy simulations need to be run to analyse how well the model fits to the data, which is computationally expensive and would take several days if a separate model is fitted for each participant. This means that participant-level correlations of P(accept) with λ and the parameter controlling evidence accumulation bias could not be computed in the case of LCA. Also, since the likelihood function of LCA is not analytically defined, the information about its goodness-of-fit is noisy. This means that the fit is only approximate. Hence, it might be worthwhile to find other models that retain the attractive features of LCA while also being relatively easier to fit to the data. As far as participant-level correlations with parameter models is concerned, it has been shown that parameter recovery in the case of LCA is quite poor [MTFvM17]. It would be interesting to see if this holds true in the case of parameters that control for loss aversion, loss attention, and unequal thresholds. If these parameters are not recoverable, participant-level correlations would not be reliable.

It is important to note that while a larger threshold for accepting gambles is similar to a predecisional bias towards rejection in the sense that more evidence is required to accept gambles, they are not identical. A predecisional bias is a starting point bias and can be interpreted as a prior affinity towards one of the options, whereas unequal thresholds cannot be interpreted this way. Also, another way to introduce bias in the LCA would be in the form of a starting point bias, i.e., the starting activation (at t = 0) of one of the accumulators may be more than the activation of the other accumulator(s). It needs to be investigated if this bias can account for the data properly.

The results in this study indicate that computational models have a lot of potential and can be extremely helpful, even indispensable, in developing a better understanding of value based decisions. It would be interesting to apply these models to the study of phenomena such as risk aversion, framing effects, etc. They might even be extended to domains other than money, such as time (see B). Just like reaction times provide crucial information about the parameters of the model, additional information such as that obtained from eye-tracking might give further insights into these cognitive processes [Pla20].

c 2021, Indian Institute of Technology Delhi Appendix A

Economics models of decision making

A.1 Expected Utility Hypothesis [MCWG+95]

Expected utility of an alternative: The expected utility of an alternative x is defined as n X E[u(x)] = piu(xi) i=1 th th where xi is the i outcome of the alternative, u(xi) is the valuation of the i outcome, and pi is the probability of occurrence of that outcome. According to the Expected Utility Hypothesis, decision-makers choose the alternative with the highest expected utility.

A.2 Prospect Theory [KT79]

Several studies have shown that Expected Utility Theory fails to account for various em- pirical effects in human behaviour [KT79]. Prospect theory attempts to account for these observations. Two important ways in which Prospect theory differs from Expected Utility Theory are (1) the computation of values and (2) the weighting of outcomes.

A.2.1 Value Function

According to Prospect Theory, the subjective value of an outcome is described by the value function having the following properties • It is defined on deviations from a reference point. • It is generally concave for gains and convex for losses. • It is steeper for losses than for gains.

Figure A.1 shows a hypothetical value function having these properties.

A.2.2 Weighting Function

According to Prospect Theory, each outcome is given a decision weight. This weight π is a function of the probability of the outcome, and has the following properties • It is an increasing function of p. • π(1) = 1 and π(0) = 0. A.2 Prospect Theory [KT79] 27

Figure A.1: A hypothetical value function [KT79]

• Low probabilities are generally over-weighted, i.e., π(p) > p for small p. • Large probabilities are usually under-weighted, i.e., π(p) < p for large p.

Figure A.2 shows a hypothetical weighting function having these properties.

Figure A.2: A hypothetical weighting function [KT79]

A.2.3 Evaluation of outcomes

The formula for computing the Expected Utility of an alternative is:

n X V (x) = π(pi)v(xi) i=1

th where xi is the i outcome of the alternative, π is the probability weighting function, and v is the value function. Decision-makers choose the alternative with the highest V .

c 2021, Indian Institute of Technology Delhi Appendix B

Modeling of Loss Aversion for time

B.1 Introduction

In this study, we want to study the preferences of people over time and their behaviour in risky gambles involving waiting time. We will collect the choice and reaction time data of the participants and use a computational model to study the relative value they give to losses as compared to gains (analogous to the λ parameter in prospect theory of economics) and if they have any prior biases in such tasks.

B.2 Experimental Design

Participants will accept or reject a sequence of 72 gambles, presented in 4 blocks of 18 gambles each. Each gamble has two possible outcomes: an increase in waiting time (in minutes) occurring with a 50% probability and a decrease in waiting time occurring with a 50% probability. Participants will indicate their choices by pressing up or down arrow keys on a keyboard, and the specific key-response associations will alternate across blocks. Choice and reaction time will be recorded. At the end of the experiment, the participants will have to wait in a closed room without any means of entertainment or communication. They will be informed that at the end of the experiment, one of the gambles will be selected at random, and its outcome will affect the duration of the neutral task to be performed at the end. If the participant had rejected that particular gamble, they would have to wait for 25 minutes. If the participant had accepted the gamble, then a coin will be flipped in order to play out the gamble. The duration of the task will be 25 minutes ± outcome of the gamble.

B.3 Stimuli

The possible gain and loss values will be taken from the set 1, 5, 10, 15, 20, 25. For each condition, we can generate 6×6 = 36 unique gambles. By counterbalancing the positions of the gain and loss, we get 72 trials. The presentation format is shown in Figure B.1. There will be a fixation screen between two gambles which will be presented for 1 second. B.4 Information given to participants 29

Figure B.1: Stimuli for the experiment

B.4 Information given to participants

The participants will be informed that they will be playing a sequence of gambles involving time. At the end of the task, they will have to spend some time in a waiting room without any mode of entertainment or communication. The amount of time they will spend in the room depends on their choices in the gambles in the following way: at the end of the task, one gamble will be selected randomly. If the participant had rejected that particular gamble, they would have to wait for 25 minutes. If they had accepted the gamble, it will be played out in front of them and the waiting time would be 25 minutes ± outcome of the gamble. They will be told that playing the gambles will take around 10 minutes, and the waiting time would vary from 0 minutes to 50 minutes depending on their responses.

c 2021, Indian Institute of Technology Delhi Bibliography

[BT93] Jerome R Busemeyer and James T Townsend. Decision field theory: a dynamic-cognitive approach to decision making in an uncertain environment. Psychological review, 100(3):432, 1993.

[BUZM07] Rafal Bogacz, Marius Usher, Jiaxiang Zhang, and James L McClelland. Ex- tending a biologically inspired model of choice: multi-alternatives, nonlinear- ity and value-based multidimensional choice. Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1485):1655–1670, 2007.

[EE13] Eyal Ert and Ido Erev. On the descriptive value of loss aversion in decisions under risk: Six clarifications. Judgment and Decision Making, 8(3):214–235, 2013.

[Eva19] Nathan J Evans. A method, framework, and tutorial for efficiently simulating models of decision-making. Behavior research methods, pages 1–15, 2019.

[FPW+17] Pouyan R Fard, Hame Park, Andrej Warkentin, Stefan J Kiebel, and Sebastian Bitzer. A bayesian reformulation of the extended drift-diffusion model in perceptual decision making. Frontiers in computational neuroscience, 11:29, 2017.

[GJH07] Simon Gächter, Eric J Johnson, and Andreas Herrmann. Individual-level loss aversion in riskless and risky choices. 2007.

[GR18] David Gal and Derek D Rucker. The loss of loss aversion: Will it loom larger than its gain? Journal of Consumer Psychology, 28(3):497–516, 2018.

[JBSS20] Keno Juechems, Jan Balaguer, Bernhard Spitzer, and Christopher Summer- field. Optimal utility and probability functions for agents with finite compu- tational precision. jan 2020.

[KK19] Arkady Konovalov and Ian Krajbich. Revealed strength of preference: In- ference from response times. Judgment and Decision Making, 14(4):381–394, 2019.

[KS84] Jack L Knetsch and John A Sinden. Willingness to pay and compensation demanded: Experimental evidence of an unexpected disparity in measures of value. The Quarterly Journal of Economics, 99(3):507–521, 1984.

[KT79] DANIEL KAHNEMAN and Amos TVERSKY. Prospect theory: An analysis of decision under risk. ECONOMETRICA, 47(2), 1979.

[LSMPH19] Tomás Lejarraga, Michael Schulte-Mecklenbeck, Thorsten Pachur, and Ralph Hertwig. The attention–aversion gap: how allocation of attention relates to loss aversion. Evolution and Human Behavior, 40(5):457–469, 2019.

30 BIBLIOGRAPHY 31

[MCWG+95] Andreu Mas-Colell, Michael Dennis Whinston, Jerry R Green, et al. Microe- conomic theory, volume 1. Oxford university press New York, 1995.

[MMH+10] Mili Milosavljevic, Jonathan Malmaud, Alexander Huth, Christof Koch, and Antonio Rangel. The drift diffusion model can account for the accuracy and reaction time of value-based choices under high and low time pressure. SSRN Electronic Journal, 2010.

[MSPS17] Sumitava Mukherjee, Arvind Sahay, VS Chandrasekhar Pammi, and Narayanan Srinivasan. Is loss-aversion magnitude-dependent? measuring prospective affective judgments regarding gains and losses. Judgment and Decision making, 12(1):81, 2017.

[MTFvM17] Steven Miletić, Brandon M Turner, Birte U Forstmann, and Leendert van Maanen. Parameter recovery for the leaky competing accumulator model. Journal of Mathematical Psychology, 76:25–50, 2017.

[MWR+12] Martijn J Mulder, Eric-Jan Wagenmakers, Roger Ratcliff, Wouter Boekel, and Birte U Forstmann. Bias in the brain: a diffusion model analysis of prior probability and potential payoff. Journal of Neuroscience, 32(7):2335–2343, 2012.

[Pla20] Michael Louis Platta. Decomposing loss aversion from gaze allocation and pupil dilation. PNAS, 117(21):11357, 2020.

[Rab00] Matthew Rabin. Diminishing marginal utility of wealth cannot explain risk aversion. 2000.

[RH19] Ian D Roberts and Cendri A Hutcherson. Affect and decision making: Insights and predictions from computational models. Trends in cognitive sciences, 2019.

[RM08] Roger Ratcliff and Gail McKoon. The diffusion decision model: theory and data for two-choice decision tasks. Neural computation, 20(4):873–922, 2008.

[SBCVDL02] David J Spiegelhalter, Nicola G Best, Bradley P Carlin, and Angelika Van Der Linde. Bayesian measures of model complexity and fit. Journal of the royal statistical society: Series b (statistical methodology), 64(4):583–639, 2002.

[UM01] Marius Usher and James L McClelland. The time course of perceptual choice: the leaky, competing accumulator model. Psychological review, 108(3):550, 2001.

[UM04] Marius Usher and James L McClelland. Loss aversion and inhibition in dy- namical models of multialternative choice. Psychological review, 111(3):757, 2004.

[YH13] Eldad Yechiam and Guy Hochman. Loss-aversion or loss-attention: The im- pact of losses on cognitive performance. Cognitive Psychology, 66(2):212–231, 2013.

c 2021, Indian Institute of Technology Delhi BIBLIOGRAPHY 32

[ZWB18] Wenjia Joyce Zhao, Lukasz Walasek, and Sudeep Bhatia. Why do people reject mixed gambles? 2018.

c 2021, Indian Institute of Technology Delhi