Computational Modeling of Decisions in Mixed Gambles Thesis submitted by Nishad Singhi 2016EE10107 under the guidance of Prof. Sumeet Agarwal Prof. Sumitava Mukherjee in partial fulfilment of the requirements for the award of the degree of Bachelor of Technology Department Of Electrical Engineering INDIAN INSTITUTE OF TECHNOLOGY DELHI July 2020 Contents ABSTRACT 2 1 INTRODUCTION 3 2 Computational models of decision making 5 2.1 Neurophysiology of Choice [BUZM07] . 5 2.2 Drift Diffusion Model . 5 2.3 The Leaky Competing Accumulator [UM01] . 6 3 Modeling mixed gambles using Drift Diffusion Model 8 3.1 Why do people reject mixed gambles? [ZWB18] . 8 3.1.1 Experimental Data . 8 3.1.2 Modeling . 8 3.1.3 Reported and replicated results . 9 3.2 Additional dataset . 11 3.2.1 Description of dataset . 11 3.2.2 DDM fitting results . 13 3.3 Discussion . 14 4 Modeling mixed gambles using LCA 15 4.1 LCA for value based decisions [UM04] . 15 4.1.1 Fitting Methods . 16 4.1.2 Experiments . 17 4.1.3 Discussion . 21 5 General Discussion 23 A Economics models of decision making 26 A.1 Expected Utility Hypothesis [MCWG+95] . 26 A.2 Prospect Theory [KT79] . 26 A.2.1 Value Function . 26 A.2.2 Weighting Function . 26 A.2.3 Evaluation of outcomes . 27 B Modeling of Loss Aversion for time 28 B.1 Introduction . 28 B.2 Experimental Design . 28 B.3 Stimuli . 28 B.4 Information given to participants . 29 ABSTRACT An important finding in behavioural economics is that people tend to reject mixed gambles. The predominant explanation for this phenomenon is loss aversion, i.e., decision-makers give more subjective weight to losses as compared to gains. However, other psychological mechanisms such as status-quo bias, loss attention, etc. provide alternate explanations for this finding. While experimental studies have provided some evidence for these hypotheses, how much they affect decision makers during mixed gambles is not clear. Moreover, it is difficult to investigate their effect on the decision process within the framework of Prospect theory – the central theory of behavioural economics – because it posits that people make decisions based on utility alone. In this thesis, I use two models of decision making called the Drift Diffusion Model and the Leaky, Competing, Accumulator model to tease apart the effects of these psychological mechanisms. Using data from two risky-choice experiments, I show that people require less evidence to reject gambles than to accept them, and that this bias is the most important mechanism underlying this behaviour, and that high rates of rejection in mixed gambles should not be understood in terms of loss aversion. 2 Chapter 1 INTRODUCTION Humans are required to make decisions in hundreds of risky situations every day. From deciding which route to take to work, to investing money in the stock market – we have to deal with risks all the time. For decades, people from various backgrounds and disciplines such as Economics, Psychology, Artificial Intelligence, etc. have tried to study how humans make decisions under uncertainty. One of the most important findings in behavioural economics is that decision makers tend to reject ‘mixed’ gambles having equal probability of resulting in a gain or a loss [KT79]. For instance, consider the following gamble: You have to choose between two options, S and R: S: gain Rs. 0 with 100% probability R: gain Rs. 1,00,000 with 50% probability lose Rs. 1,00,000 with 50% probability Even though the probability of winning Rs. 1,00,000 is the same as that of losing Rs. 1,00,000, most people choose S. This suggests that people tend to reject gambles that may potentially result in a loss. However, the nature of the psychological processes underlying this phenomenon is not properly understood. Several hypotheses have been proposed to explain this finding. According to expected utility theory (See A.1), this may indicate concavity of the utility function. However, this hypoth- esis cannot explain the high rates of rejection in small gambles, as that would require an unreasonably high degree of concavity in the utility function [Rab00]. Prospect theory (See A.2) [KT79] explains this finding by positing that losses are given more subjective value by decision makers than gains of equal magnitude, i.e., the value function for losses is steeper than gains. This idea – known as loss aversion – is the most widely accepted explanation for the high rates of rejection in mixed gambles. While loss aversion was originally proposed in the context of decisions under risk, it has been generalized to other settings, and has successfully explained numerous other findings such as Endowment effect [KS84]. The idea that losses and gains have different effects on our psychological state has been extended to other domains such as education, politics, marketing, etc. However, Prospect theory has several shortcomings. It does not comment upon the time 4 taken by a person to reach a decision. Moreover, while it aims to describe how people actually make decisions (i.e., it is a descriptive model) instead of being a normative model (like expected utility theory), there is no explanation of the psychological processes mentioned in it. It even fails to account for the role of aspects of cognition such as emotion, attention, etc. These issues undermine the role of loss aversion as the mechanism behind rejection of mixed gambles. In addition to the claims mentioned above, there is increasing evidence suggesting that loss aversion is not a ubiquitous phenomenon – as assumed by Prospect theory – and is context dependent. For example, [MSPS17] showed that losses loom larger than gains only for large magnitudes, and might even be valued less than gains for smaller magnitudes. [EE13] showed that loss aversion is not observed when the option of rejecting a gamble is presented explicitly as opposed to being the status quo. In addition to loss aversion, other mechanisms explaining the rejection of mixed gambles have been investigated. One of them is Loss Attention, which suggests that losses attract more attention than (but may not necessarily be weighted more than) gains of similar magnitude [YH13] [LSMPH19]. Another possible explanation is that people reject mixed gambles because of ‘psychological inertia’ or a status-quo bias. It is possible that the behavioural finding results from the interplay of these effects, and while there have been some empirical studies about these hypotheses, it is difficult to disentangle their effects from each other as well as from loss aversion using qualitative methods alone. Moreover, as has already been mentioned, economic models such as expected utility theory and prospect theory rely only on the utility of a prospect and cannot accommodate these psychological mechanisms. Hence, computational models that are grounded in psychological principles are required to study these mechanisms quantitatively. One class of computational models, which is quite attractive for this enterprise, is the family of evidence accumulation models. Essentially, these models assume that people accumulate noisy evidence from stimuli over time to make decisions. These models have been successful in explaining both choice and reaction time data in ‘perceptual decision’ making tasks, so it is possible that the brain uses a similar mechanism in economic decision making tasks. Moreover, these models are capable of incorporating mechanisms such as the status-quo bias or asymmetric attention for losses and gains. Additionally, some of these models have pro- vided elegant explanations for context effects in economic decision making [UM04], [BT93]. Finally, some of these models incorporate known principles of information processing in the brain, increasing their plausibility as models of economic decision making. In chapter 2, I briefly describe mechanisms of decision making in the brain and how they can inspire models of decision making. In chapters 3 and 4, I employ these models to investigate the mechanisms underlying people’s aversion to mixed gambles. This is followed by a general discussion in chapter 5. c 2021, Indian Institute of Technology Delhi Chapter 2 Computational models of decision making 2.1 Neurophysiology of Choice [BUZM07] Let us begin with a common perceptual decision making task (instead of value-based de- cisions that we have considered so far): participants are shown a cloud of identical dots moving on a screen. In every trial, a proportion of the dots are moving in the same di- rection, while the other dots are moving randomly. The task is to identify the direction of prevalent movement. Sensory neurons (such as those in the MT1 region in the brain) detect particular directions of movement. For instance, a neuron that is tuned to detect objects moving to the right will fire only when an object moving to the right appears in the visual field and not otherwise. The information coded by the these neurons is inherently noisy, so making a decision based on the information obtained at just one instant may cause errors. Instead, information sampled at different instants is used collectively to make a decision. It has been observed that the firing rates of some neurons in the lateral intraparietal area (LIP) and the frontal eye field (FEF) increases over time. Moreover, the easier the task, the larger is the rate of increase. These observations suggest that neurons in these areas might be integrating the information provided by the sensory neurons, averaging out the noise. 2.2 Drift Diffusion Model The Drift Diffusion Model is a very influential model that has been able to successfully account for accuracy and reaction-time data from a wide range of behavioural experiments involving decisions between two options. Parameters of the model represent components of information processing, so, studying these parameters (and their variation under differ- ent experimental conditions) can provide insights about the processes underlying decision- making.