The Economics, Neurobiology and Pharmacology of Intertemporal Choice in Humans A thesis submitted for the degree of Doctor of Philosophy Alexander J Pine University College London September 2009 2 Preface I, Alexander J Pine confirm that the work presented in this thesis is my own. Where information has been derived from other sources I confirm that this has been indicated in the thesis. AJ Pine 3 Abstract In intertemporal choice, decision-makers must choose between options whose outcomes occur at different times in the future and are associated with different magnitudes of gain or loss. Previous neuropsychological research on this problem is dominated by a behavioural- economic model which proposes that choice outcome is solely determined by a process of devaluing rewards with time, termed temporal discounting. This thesis investigates the veracity of this assumption by developing a new mathematical model of choice which takes into account another fundamental feature of human preference, namely the non-linearity of the relationship between the utility and magnitude of gains. Using behavioural data, methodologies are developed to demonstrate that this model is superior to previous models in accounting for human intertemporal choices. Specifically, using existing terminologies ‘impulsive’ and ‘self-controlled’ to describe preference in choices between smaller-sooner and larger-later monetary rewards, it is shown that the discounting of increasing magnitudes implied by the law of diminishing marginal utility exerts a significant effect in determining choice outcome. In addition to high rates of temporal discounting, it is shown that impulsivity can be engendered by higher rates of diminishing marginal utility and vice-versa. A neuronal account of this model is delineated using neuroimaging techniques, revealing fundamental properties of the brain’s value systems. It is shown that sub-components of value relating to time and magnitude are evaluated by distinct systems and then integrated to furnish an overall metric of utility used to guide choice – in accordance with utility theory. Finally, the ability of the neurotransmitter dopamine to modulate these features of preference and neurobiological systems is investigated using pharmacological manipulation, where it is shown that enhancing dopamine activity engenders impulsivity. These behavioural and neural findings are shown to offer a compelling account of the pathological impulsivity observed as a feature of disorders associated with aberrant dopamine function. 4 To Opi and Elian 5 Contents Acknowledgements............................................................................................. 10 Abbreviations used in this thesis ........................................................................ 11 Publications ........................................................................................................ 12 Chapter 1. Introduction .................................................................... 13 Overview ............................................................................................................. 13 Introduction to decision theory ............................................................................ 14 Normative decision-making and axioms of utility theory .......................................... 15 Descriptive decision theory ..................................................................................... 18 Intertemporal choice and the discounted utility model ........................................ 20 Temporal discounting: reasons for caring less about the future .............................. 21 Normative models of intertemporal choice: the discounted utility framework ........... 25 Behavioural economics of intertemporal choice: DU anomalies ......................... 28 The sign effect ........................................................................................................ 28 The direction effect: ‘the delay, speed-up asymmetry’ ............................................ 30 The magnitude effect .............................................................................................. 30 Hyperbolic discounting and dynamically inconsistent preferences ..................... 31 Akrasia and weakness of the will ........................................................................ 36 Impulsiveness and self-control ........................................................................... 39 Fractionating impulsivity ......................................................................................... 39 Impulsive choice ..................................................................................................... 42 Methodology of intertemporal choice and the measurement of discount rates ... 45 Indifference point methodology ............................................................................... 45 Differences in animal and human studies ............................................................... 51 Pharmacological and neurochemical studies ...................................................... 55 Dopamine ............................................................................................................... 55 Evidence from Attention-Deficit/Hyperactivity Disorder and monoaminergic stimulants ... 55 Human dopamine manipulation studies of intertemporal choice ....................................... 58 Rodent dopamine manipulation studies ............................................................................ 60 Complicating factors in rodent and human dopamine manipulation studies ..................... 62 In vivo studies of dopamine function during intertemporal choice ..................................... 64 Serotonin ................................................................................................................ 65 6 Evidence from behavioural inhibition and impulsive mood disorders ................................ 65 Human serotonin manipulation studies of intertemporal choice ........................................ 66 Rodent serotonin manipulation studies ............................................................................. 67 In vivo studies of serotonin function during intertemporal choice ...................................... 69 Serotonin-dopamine interactions ....................................................................................... 70 Other pharmacological investigations ..................................................................... 71 Neuroanatomical studies of intertemporal choice ............................................... 72 Animal and lesion studies ....................................................................................... 72 Orbitofrontal cortex ............................................................................................................ 72 Striatum and nucleus accumbens ..................................................................................... 76 Amygdala and other regions .............................................................................................. 78 Human neuroimaging studies of intertemporal choice ............................................. 80 Single or dual systems for evaluating delayed rewards .................................................... 80 Individual differences in discount rates .............................................................................. 84 Dorsal to ventral gradients of discount rates in the striatum ............................................. 85 Self control ......................................................................................................................... 87 Probabilistic versus temporal discounting and the nature of discounting .......................... 87 Neuropsychiatry of impulsive choice .................................................................. 90 ADHD ..................................................................................................................... 91 Adolescent behaviour ............................................................................................. 92 Aggression, suicide, depression and mania ............................................................ 93 Dopamine dysregulation syndrome ......................................................................... 94 Addiction ................................................................................................................. 95 Motivational and visceral states: Hedonic impact of rewards ............................ 100 Real versus hypothetical choices ..................................................................... 101 The problem of non-linear utility and advanced valuation models .................... 104 Organization of work in this thesis .................................................................... 109 Chapter 2. The effects of diminishing marginal utility for gains on intertemporal choice ................................................................. 112 Introduction ....................................................................................................... 112 Methods ............................................................................................................ 121 Procedure and task design ................................................................................... 121 Results .............................................................................................................. 123 Adjusting delay