John Von Neumann Oskar Morgenstern

Neuroeconomics: From The Failures of Expected Utility to the Neurobiology of Choice

Paul Glimcher PhD Julius Silver Professor of Neural Science, Economics and Psychology Director, Institute of the Study of Decision Making New York University Blaise Pascal

Genius: Expected Value Theory

Expected Probability x Value = Value #1 0.5 100 50

#2 1.0 45 45

Pascal's Wager If God Exists If God Doesn't Exist (Prob 3 Value) + (Prob 3 Value) = Exp. Value

Believe in God >0 3 `$++ 0 3 0 = `

Do not Believe >0 3 2` ++ $0 3 0 = 2` in God

Problem: The Beggar’s Dilemma Daniel Bernoulli

Genius: Expected Utility Theory

Beggar’s Dilemma

4.3 3.8 Beggar’s Utility (utils) Beggar’s

0 7000 20,000 Rich Man’s Choice Beggar’s Wealth (florins) 6.0056 6.0 5.9969 Rich Man’s Utility (utils) Rich Man’s

Problem: 0 losses wins Value ($) Bentham, Pareto, Samuelson 993,000 1,013,000 1,000,000 Jeremy Bentham

Vilfredo Pareto

Paul Samuelson

The Calculus of Utility

The Intrinsic Arbitrariness of Utility Ordinal Objective Utility John von Neumann Oskar Morgenstern

Genius: Modern Expected Utility Theory

0 < a <1 a $ s = Expected 35til u Utility weight s.w. = prob Subjective 0 Utility (utils) 0 Probability Dollars ($)

Critcal Advantages: • Precise • Compact • Normative (people Problem: make sense) Maurice Allais Maurice Allais

People Do Not Obey EU all the time

0 < a <1 a $ s = Expected 35til u Utility weight s.w. = prob Subjective 0 Utility (utils) 0 XProbability Dollars ($)

Critcal Questons: • How to Predict People? • Are People Dumb? • Why? Amos Tversky Danny Kahneman

Prospect Theory

Critcal Advantages: • Predictive

Critcal Disadvantages: • Bulky • No Why Behavioral Traditional

Social-Natural Science Boundary Economics vs. Psychology Behavioral Traditional The Reductive Levels Of Decision Science Samuelson

Social-Natural Science Boundary

Kahneman

The Central Goal of Neuroeconomics (for me) was/is Interdisciplinary Fusion

How Can We Combine These?

decision value probability variable 3 x 0.5 = 1.5

2 x 0.8 = 1.6 ?

1 x 1 = 1.0 So What Happens When We Vary Value?

Platt and Glimcher, 1999 Large Reward Expectation Small Reward Expectation 0 < a <1 a $ s = Expected 35til u Utility weight s.w. = prob Subjective 0 Utility (utils) 0 Probability Dollars ($) 0 < a <1 a $ s = Expected 35til u Utility weight s.w. = prob Subjective 0 Utility (utils) 0 Probability Dollars ($) 4

1 Percept (expers) 0 100 200 300 400 500 600 700 800 900 1000 Stimulus intensity (e.g., weight)

Daniel McFadden Random Utility Theory

2 Utility

0 100 200 300 400 500 600 700 800 900 1000 Value Knutson, Delgado and Elliott Joe Kable

Levy and Glimcher Bartra, McGuire and Kable Up to this point:

–U"(x) Reservation Preferences U'(x) Based Search ?

? ε RUM Discounting

Independence EWA/ Arg Max U(x) Axiom Surprise Learning

ECONOMICS

Log Stevens' Satisficing Numerosity Power Law Reference Point Behavioral Impulsivity AttentionAttention Stochasticity ? Self-Control

Decision Choice Value π(P) RPE

PSYCHOLOGY

Parietal Transducer Drift Diffusion Efficient Coding Numberline Biophysics Models Hypothesis Neuronal Neuronal Dorso- Orbitofrontal Gain & Tuning Stochasticity Lateral PFC Cortex Winner- Take- Ventral Striatum Dopamine All Networks Medial PFC ? Activation

NEUROSCIENCE

When?

–U"(x) Reservation Preferences U'(x) Based Search ?

? ε RUM Discounting

Independence EWA/ Arg Max U(x) Axiom Surprise Learning

ECONOMICS

Log Stevens' Satisficing Numerosity Power Law Reference Point Behavioral Impulsivity AttentionAttention Stochasticity ? Self-Control

Decision Choice Value π(P) RPE

PSYCHOLOGY

NEUROSCIENCE

An Image

Horace Barlow An Image

Horace Barlow 9 Pixels

Probability Pixel is Black: Conditional on Adjacent Pixel Being Black: 0.75 9 Pixels

Black = 10spikes While = 0 Spikes

90 Spikes Total 9 Pixels

Conditional on these 8 pixels being black the ex ante probability of the center being black is ~ 1.

So Why Waste Spikes Heeger Normalization Eero Simoncelli and Co. Showed That:

For any given pixel to pixel correlation structure there is a set of wi’s such that the minimum number of spikes is used per bit of information

This is a form of decorrelation

These Kinds of Networks Yield Strong ‘Outside the Classical RF Effects’ Is There Evidence of Normalization in Choice-Related Circuits? SACCADE FIX OFF V = 1 CUE TARGET FIX + + V = 0.5 + 500 ms 1000 ms + + V = 2 + + 1 1:1 VRF : ΣVj + 3 2

1:1.5 +

TARGET FIX 1:3 + + + 1000 ms 1:3.5 +

Randomized target array presentation 0:0.5 +

0:2 +

0:2.5 + 1 1:1 + TAR 3 2 n = 62 (2 monkeys) 1:1.5 +

1:3 +

1:3.5 + Firingrate(norm) 0:0.5 +

0:2 + Time (s)

FIX TARGETS 0:2.5 + CUE Comparing Models Across the Dataset

Platt & Sugrue, Heeger Heeger & Glimcher Corrado & Reynolds Newsome

37 If There is Normalization, Would it Inﬂuence Choice Behavior?

The Three Option Problem For Neurons:

• Variance Scales with Mean

• Variance is the average Squared distance from the mean

Variance • Thus as means reduce, discriminablility goes down

variance ~1.0-1.5 mean

Neuronal Rate

Tolhurst, Movshon and Dean, 1983

SACCADE Free choice trials FIX OFF CUE TARGET FIX + + + 500 ms ? 1000 ms

V1: 0.156

V2: 0.130 0.143 0.156 0.169 0.182

V3: 0.026 0.104 Target reward magnitude (ml) Monkey D (6916 choices) Monkey B (9201 choices) 1.0

Small V3

Conditional choice (1) Large V3 0 -0.039 0 0.039 -0.039 0 0.039 V1-V2 (ml)

Theory Humans If There is Normalization, Would it Inﬂuence Choice Behavior?

The Three Option Problem

The Multi-Option Problem The Curse of Choice var/mean = 1.0

p(A) p(A)/p(B) Observations Choiceprobability p(B)

Simulated activity Additional alternatives 2 2 2 02 0 0 Divisive02 normalization + cortical variability choice-set effect 0 etc. Constructing Variable Size Choice Sets

Target pairs

Distracters

30 Subjects 6x45 Trials Per 8100 Trials Total Fitted Probabilities From Estimation

0.2 0.1 But How Would A Network Implement Normalization?

What Would the Network Dynamics Be Like? 50 Normalization is the Unique Equilibrium State for Networks 51 of this Kind These Networks Have Specific Dynamics

52 Which Actually Are Observed

53 If a Choice Network Used These Dynamics, How Would It Choose? The ESVT “Value Function”

Captures: • Reﬂection Effect • Probability Distortion • Loss Aversion • Endowment Effect Would It Be Normative?

John von Neumann Oskar Morgenstern

Modern Expected Utility Theory

0 < a <1 a $ s = Expected 35til u Utility weight s.w. = prob Subjective 0 Utility (utils) 0 Probability Dollars ($)

Critcal Advantages: • Precise • Compact • Normative (people make sense) Expected Subjective Value Theory Samuelson

Social-Natural Science Boundary

Pavlov

Stevens/Fechner

Sherrington

Neuroeconomics Aligning and Reﬁning Hard Theories James S. McDonnell Foundation