Neuroeconomics: From The Failures of Expected Utility to the Neurobiology of Choice Paul Glimcher PhD Julius Silver Professor of Neural Science, Economics and Psychology Director, Institute of the Study of Decision Making New York University Blaise Pascal Genius: Expected Value Theory Expected Probability x Value = Value #1 0.5 100 50 #2 1.0 45 45 Pascal's Wager If God Exists If God Doesn't Exist (Prob 3 Value) + (Prob 3 Value) = Exp. Value Believe in God >0 3 `$++ 0 3 0 = ` Do not Believe >0 3 2` ++ $0 3 0 = 2` in God Problem: The Beggar’s Dilemma Daniel Bernoulli Genius: Expected Utility Theory Beggar’s Dilemma 4.3 3.8 Beggar’s Utility (utils) Beggar’s 0 7000 20,000 Rich Man’s Choice Beggar’s Wealth (florins) 6.0056 6.0 5.9969 Rich Man’s Utility (utils) Rich Man’s Problem: 0 losses wins Value ($) Bentham, Pareto, Samuelson 993,000 1,013,000 1,000,000 Jeremy Bentham Vilfredo Pareto Paul Samuelson The Calculus of Utility The Intrinsic Arbitrariness of Utility Ordinal Objective Utility John von Neumann Oskar Morgenstern Genius: Modern Expected Utility Theory 0 < a <1 a $ s = Expected 35til u Utility weight s.w. = prob Subjective 0 Utility (utils) 0 Probability Dollars ($) Critcal Advantages: • Precise • Compact • Normative (people Problem: make sense) Maurice Allais Maurice Allais People Do Not Obey EU all the time 0 < a <1 a $ s = Expected 35til u Utility weight s.w. = prob Subjective 0 Utility (utils) 0 XProbability Dollars ($) Critcal Questons: • How to Predict People? • Are People Dumb? • Why? Amos Tversky Danny Kahneman Prospect Theory Critcal Advantages: • Predictive Critcal Disadvantages: • Bulky • No Why Behavioral Traditional Social-Natural Science Boundary Economics vs. Psychology Behavioral Traditional The Reductive Levels Of Decision Science Samuelson Social-Natural Science Boundary Kahneman The Central Goal of Neuroeconomics (for me) was/is Interdisciplinary Fusion How Can We Combine These? decision value probability variable 3 x 0.5 = 1.5 2 x 0.8 = 1.6 ? 1 x 1 = 1.0 So What Happens When We Vary Value? Platt and Glimcher, 1999 Large Reward Expectation Small Reward Expectation 0 < a <1 a $ s = Expected 35til u Utility weight s.w. = prob Subjective 0 Utility (utils) 0 Probability Dollars ($) 0 < a <1 a $ s = Expected 35til u Utility weight s.w. = prob Subjective 0 Utility (utils) 0 Probability Dollars ($) 4 3 2 1 Percept (expers) 0 100 200 300 400 500 600 700 800 900 1000 Stimulus intensity (e.g., weight) Daniel McFadden Random Utility Theory 4 3 2 Utility 1 0 100 200 300 400 500 600 700 800 900 1000 Value Knutson, Delgado and Elliott Joe Kable Levy and Glimcher Bartra, McGuire and Kable Up to this point: –U"(x) Reservation Preferences U'(x) Based Search ? ? ε RUM Discounting Independence EWA/ Arg Max U(x) Axiom Surprise Learning ECONOMICS Log Stevens' Satisficing Numerosity Power Law Reference Point Behavioral Impulsivity AttentionAttention Stochasticity ? Self-Control Decision Choice Value π(P) RPE PSYCHOLOGY Parietal Transducer Drift Diffusion Efficient Coding Numberline Biophysics Models Hypothesis Neuronal Neuronal Dorso- Orbitofrontal Gain & Tuning Stochasticity Lateral PFC Cortex Winner- Take- Ventral Striatum Dopamine All Networks Medial PFC ? Activation NEUROSCIENCE When? –U"(x) Reservation Preferences U'(x) Based Search ? ? ε RUM Discounting Independence EWA/ Arg Max U(x) Axiom Surprise Learning ECONOMICS Log Stevens' Satisficing Numerosity Power Law Reference Point Behavioral Impulsivity AttentionAttention Stochasticity ? Self-Control Decision Choice Value π(P) RPE PSYCHOLOGY Parietal Transducer Drift Diffusion Efficient Coding Numberline Biophysics Models Hypothesis Neuronal Neuronal Dorso- Orbitofrontal Gain & Tuning Stochasticity Lateral PFC Cortex Winner- Take- Ventral Striatum Dopamine All Networks Medial PFC ? Activation NEUROSCIENCE An Image Horace Barlow An Image Horace Barlow 9 Pixels Probability Pixel is Black: Conditional on Adjacent Pixel Being Black: 0.75 9 Pixels Black = 10spikes While = 0 Spikes 90 Spikes Total 9 Pixels ? Conditional on these 8 pixels being black the ex ante probability of the center being black is ~ 1. So Why Waste Spikes Heeger Normalization Eero Simoncelli and Co. Showed That: For any given pixel to pixel correlation structure there is a set of wi’s such that the minimum number of spikes is used per bit of information wj This is a form of decorrelation These Kinds of Networks Yield Strong ‘Outside the Classical RF Effects’ Is There Evidence of Normalization in Choice-Related Circuits? SACCADE FIX OFF V = 1 CUE TARGET FIX + + V = 0.5 + 500 ms 1000 ms + + V = 2 + + 1 1:1 VRF : ΣVj + 3 2 1:1.5 + TARGET FIX 1:3 + + + 1000 ms 1:3.5 + Randomized target array presentation 0:0.5 + 0:2 + 0:2.5 + 1 1:1 + TAR 3 2 n = 62 (2 monkeys) 1:1.5 + 1:3 + 1:3.5 + Firingrate(norm) 0:0.5 + 0:2 + Time (s) FIX TARGETS 0:2.5 + CUE Comparing Models Across the Dataset Platt & Sugrue, Heeger Heeger & Glimcher Corrado & Reynolds Newsome 37 If There is Normalization, Would it Influence Choice Behavior? The Three Option Problem For Neurons: • Variance Scales with Mean • Variance is the average Squared distance from the mean Variance • Thus as means reduce, discriminablility goes down variance ~1.0-1.5 mean Neuronal Rate Tolhurst, Movshon and Dean, 1983 SACCADE Free choice trials FIX OFF CUE TARGET FIX + + + 500 ms ? 1000 ms V1: 0.156 V2: 0.130 0.143 0.156 0.169 0.182 V3: 0.026 0.104 Target reward magnitude (ml) Monkey D (6916 choices) Monkey B (9201 choices) 1.0 Small V3 Conditional choice (1) Large V3 0 -0.039 0 0.039 -0.039 0 0.039 V1-V2 (ml) Theory Humans If There is Normalization, Would it Influence Choice Behavior? The Three Option Problem The Multi-Option Problem The Curse of Choice var/mean = 1.0 p(A) p(A)/p(B) Observations Choice probability p(B) Simulated activity Additional alternatives 2 2 2 02 0 0 Divisive02 normalization + cortical variability choice-set effect 0 etc. Constructing Variable Size Choice Sets Target pairs Distracters 30 Subjects 6x45 Trials Per 8100 Trials Total Fitted Probabilities From Estimation 0.2 0.1 But How Would A Network Implement Normalization? What Would the Network Dynamics Be Like? 50 Normalization is the Unique Equilibrium State for Networks 51 of this Kind These Networks Have Specific Dynamics 52 Which Actually Are Observed 53 If a Choice Network Used These Dynamics, How Would It Choose? The ESVT “Value Function” Captures: • Reflection Effect • Probability Distortion • Loss Aversion • Endowment Effect Would It Be Normative? John von Neumann Oskar Morgenstern Modern Expected Utility Theory 0 < a <1 a $ s = Expected 35til u Utility weight s.w. = prob Subjective 0 Utility (utils) 0 Probability Dollars ($) Critcal Advantages: • Precise • Compact • Normative (people make sense) Expected Subjective Value Theory Samuelson Social-Natural Science Boundary Pavlov Stevens/Fechner Sherrington Neuroeconomics Aligning and Refining Hard Theories James S. McDonnell Foundation.