Journal of Economic Literature 2013, 51(4), 1155–1182 http://dx.doi.org/10.1257/jel.51.4.1155

A Review Essay about Foundations of Neuroeconomic Analysis by Paul Glimcher †

Colin F. Camerer*

Neuroeconomics aims to discover mechanisms of economic decision, and express them mathematically, to predict observed choice. While the contents of neuroeconomic models and evidence are obviously different than in traditional economics, (some of the) goals are identical: to explain and predict choice, the effects of comparative statics, and perhaps make interesting new welfare judgments that are defensible. To this end, Paul Glimcher’s important book carefully describes how economics, psychological, and neural levels of explanation can be linked (a structure which has been successful in visual neuroscience). As Glimcher shows, the neural evidence is quite strong for a process of learning valuations through prediction error, and a simple model of neural valuation and comparison that corresponds to random utility (though subject to normalization, which produces menu effects). There is also rapidly growing evidence for more complicated constructs in behavioral economics, including prospect theory’s account of risky choice, hyperbolic time discounting, level-k models of games, and social preferences corresponding to internal reward based on what happens to other agents. ( JEL D01, D03, Y30)

1. Introduction of economists working extra hard, and inge- niously, to infer unobservables—such as he splashy attention and criticism of beliefs, utilities, thinking costs, and more— Tneuroeconomics is a referendum on from observable choices. Other scientific dis- how curious economists are about how ciplines take an easier route by simply trying the brain makes economic decisions and to measure the unobservables directly as best whether new insights can tweak old models they can, as well as inferring unobservables or inspire new ones. statistically. At one extreme is aggressive disinterest in At the other extreme is a conviction that brain activity. This disinterest is motivated by progress will certainly be made in under- the methodological history and convention­ standing the biological basis of economic choice, that such progress will inform “stan- dard” economic concerns (such as predict- * California Institute of Technology. † Go to http://dx.doi.org/10.1257/jel.51.4.1155 to visit ing choice responses to policy variables), the article page and view author disclosure statement(s). that progress is certainly facilitated by direct

1155 1156 Journal of Economic Literature, Vol. LI (December 2013)

­collaboration and by shared scientific literacy Rustichini (2004) nicely put it, the goal is between economists and neuroscientists, a theory that is mechanistic, mathematical, and that this is an historically good time to and behavioral. Traditional economic mod- start that collaboration. els are both mathematical and behavioral. The extremely skeptical approach fore- Typical neuroscience models are mechanis- casts the time at which neural data is useful tic and behavioral, but rarely mathemati- as “never.” The progressive approach says cal. The shared goal of neuroeconomists, the time is “someday.” Which do you think is whether from neuroscience or economics, is likely to prove correct in your lifetime? to have models with all three properties. This review essay is about the motives, Some neuroeconomists are interested in methods, and tentative early findings in neu- the mechanistic details for their own sake, roeconomics, and where they are between or for practical purposes, such as associating the two extremes. The review part is moti- computational mistakes with mental health vated by the 2011 book Foundations of disorders (e.g., Kishida, King-Casas, and Neuroeconomic Analysis, by Paul Glimcher. Montague 2010). Others, like myself, hope The title is apparently a (perilous?) hom- that understanding the neural mechanisms age to Paul Samuelson’s monumental will actually improve our ability to under- Foundations of Economic Analysis. stand choice in order to better accomplish Glimcher’s book presents a helpfully clear traditional goals in economics—predicting vision of one part of the progressive approach choices, predicting responses to changes in to neuroeconomics. The titled “foundation” prices and other variables, designing insti- that dominates the book is a simple value- tutions that are robust to imperfections and-compare model in which neural circuits in neural implementation, and perhaps in compute random utility for choice objects, understanding more about welfare. then compare them and pick the best one. Please note that all economists who are Glimcher argues that there is actually a interested in neuroscience are perfectly natural correspondence between elements aware that the study of brain activity during of these neural mechanisms and “as if” con- choice is a radical departure from current structs in standard theory (i.e., utilities and standard economic practice. The agnostic maximization). disinterest in mechanistic details is very obvi- The book is not intended to be either an ously a traditional part of modern revealed overall, balanced introduction to a rapidly- preference theory (although not of earlier growing field, nor a comprehensive textbook. economic thought).1 So this essay will first present a broader The key step for seeing whether neu- vision about all the building blocks of neu- roscience can contribute to economics is roeconomics, filling in what readers might distinguishing particular methodological want to know about the many other develop- approaches in economics from the general ments in neuroeconomics that are not cov- ered in the book. Then I will comment, as a book critic, on what is both admirable and 1 It is also noteworthy that many earlier economists imperfect about Glimcher’s book. including Edgeworth, Fisher, and Ramsey were keenly interested in, and wrote fantastically, about “hedonim- 2. Goals and Methods of Neuroeconomics eters” and other then-fictional devices for linking biology and choice (Colander 2007). Thus, the modernized recon- The main goal of neuroeconomics is to struction that “economists were never interested in flesh- and-blood human beings” (Gul and Pesendorfer 2008) is supply a mechanistic account of how eco- wrong (unless Edgeworth et al. are retroactively disbarred nomic choices are made. As Glimcher and from the category “economists” for their speculations). Camerer: A Review Essay about Foundations of Neuroeconomic Analysis 1157 goal of understanding what causes choice. ­contains no biological concepts of any kind” The proper definition of the scope of eco- (Gul and Pesendorfer 2008). nomics is a broad one: our goal is to under- A class of examples that refute the claim stand the causes of choices and their welfare of independence of economics and neuro­ implications. Aggregation across individuals science is the prediction of actual choices and institutions such as firms and govern- from direct recording of neural activity. ments play a special role because those This kind of prediction is useful when past forces produce choice sets, prices, informa- choice data are not likely to predict future tion, and special kinds of constraints (e.g., choices well. Examples include new prod- law). ucts, responses to policy changes that are One way to study the choice process is historically unprecedented (e.g., labor to assume that unobserved preferences are markets upon integration of East and West stable and revealed well by observed past Germany), or when omitted variables in choices. However, there is nothing about the measures of past choices bias estimates. In broader desire to understand and predict these cases, a revealed-preference approach choices that prohibits other attempts to mea- based on inference from past data is subject sure what causes choices by using nonchoice to a variant of the Lucas critique (see Dean data. That is, inference only from revealed 2013). preference is one way to learn about the Some studies have already shown that causes of choices and welfare, but it is not the neural measures during passive viewing of only way. Other ways are certainly conceiv- choice objects can predict, to some extent, able, and might be better in some respects what choices are later made (see Levy et al. (e.g., Bernheim 2009). 2010; Smith et al. forthcoming). Neural Importantly, Glimcher correctly notes subjective value reactions to antismoking that linking economics and neuroscience ads predict whether smokers quit and which first requires “natural kinds” that are under- televised ads draw the most calls to a help stood in their disciplines. Linking them then hotline (Falk, Berkman, and Lieberman requires scientists who understand both 2012; Falk et al. 2011). In fact, neural reac- natural kinds and agree on some of the ways tions to the ads from a small brain-imaged in which the two kinds are related (though sample predicted actual hotline calls (by the empirical details of the relation are likely thousands of callers who see the ads on to be under constant debate and revision). TV) better than subjects’ self-reports of ad Glimcher proposes the term “subjective effectiveness did. What artificial TV-series value” for a neural “kind” and utility—more ideas people choose to forward to others aptly, decision utility—for the associated (a measure of “virality”) are predicted by economic kind. This view has been implicit activity in mentalizing and reward areas in neuroeconomics from the beginning, but (Falk et al. 2013). These examples suggest Glimcher’s persuasive writing creates useful the possibility that standard forecasting of clarity here. future behavior from past choice can be In my view, Glimcher portrays the most improved if omitted variables (neural mea- traditional economic theorists as more flex- sures) are included. ible than their writing suggests. He says that Gul and Pesendorfer “argue, [that] 3. What Neuroeconomics Can Do economic and neuroscientific analyses are independent and must remain indepen- Improved understanding of the link dent as long as the body of economic theory between neural mechanisms and choices is 1158 Journal of Economic Literature, Vol. LI (December 2013) likely to lead to three types of support and blood flow, correspond to values inferred inspiration for areas of economic theory: from choice. Human fMRI and other evi- dence also indicate that this result extends • Neural evidence of utility maximization to somewhat more complicated choices in simple choice; (such as food choices and charitable giv- • Neural evidence distinguishing different ing). However, nothing is yet known about behavioral and rational processes; whether the same types of neural activ- • Neural evidence of other psychological ity compute subjective values for goods or influences. activities that are substantially more complex (health care plans, mates, jobs). And impor- Glimcher’s book concentrates on the first tantly, there is little research on how the kind of evidence. However, the other two neural system accounts for prices or income, types of evidence are likely to grow more time, and attention constraints in the choices rapidly in the years ahead. Therefore, I will made in everyday shopping. describe some of that evidence next. Suppose that the value-and-compare model generates subjective values that 3.1 Neural Evidence of Utility match up with inferred utilities in simple Maximization in Simple Choice cases but not in all cases. What other mecha- The first general contribution of neuro- nisms might be at work and how would they economics will be isolating the neural cir- fit into economics? cuitry of simple choices. In simple cases, this Decades of research with different meth- circuitry is likely to produce stable neural ods and species suggest there are several measures that correspond to some form of neurally distinct sources of subjective valu- utility maximization. That is, it will not be ation or choice. They are sketched in table 1 inaccurate to say that the brain is “comput- (see, Rangel, Camerer, and Montague 2008 ing” utilities (which reliably predict actual and references therein for details). choices), much as the visuomotor system A simple system of some kind exhibits of a furniture mover “computes” the likely prepared or innate preferences. This system weight of a chair in order to prepare muscles responds to “primary reinforcers,” which are for lifting. goods like food, liquids, warmth, and physi- Glimcher’s book is much focused on this cal contact that neonate infants “prefer” at first contribution, except for extensive dis- birth. A striking example is the preference cussion of normalization and relatively for looking at faces. Newborns as young as brief discussions of challenges to utility several minutes old prefer to look longer at maximization. face caricatures than at visually-comparable Note, however, that the decision computa- scrambled faces, and they stare longer when tions that are likely to fall in this category will the faces make eye contact and are lit from be simple: in experiments, they are typically above (as if the face is above them) (Valenza binary choices between two costless rewards et al. 1996; Farroni et al. 2002, 2005). Such that differ on one quality dimension (e.g., innate preferences are likely to be cultur- amount of juice or taste), which are repeated ally universal, and avoiding choices based on many times so that the experienced reward these preferences would be effortful. value is learned. The second, “learned” value system is Many studies strongly indicate that neural the major topic of Glimcher’s book. It is measures of reward value, such as firing rates probably the one about which the most is or blood-oxygen-level dependent (BOLD) known, partly because it seems to be shared Camerer: A Review Essay about Foundations of Neuroeconomic Analysis 1159

Table 1 Hypothesized Economic Effects of Different Neural Choice Systems

Neural choice system Features Interesting economic effects Prepared (innate) Primary reinforcers, faces, imitative “Hard-wired”, universal Habit Overlearned, low effort, Low elasticities in response to outcome stimulus response change ⇾ Conditioned learning Learning u(subjective value) over time; Preference changes over time goal-directed Model-based Goal-directed valuation based on Self-control, framing effects representation; conflicted, “constructed” preferences

­anatomically and functionally across species. consequences have not been experienced. This system uses reinforcement learning This type of valuation is obviously crucial in rules (which are well specified mathemati- many complex novel choices humans face. cally) guided by prediction error to learn When a new Ph.D. is choosing TIAA–CREF subjective values. After sufficient learning plans on their first job, for example, they takes place, the learned subjective values have no history of experience making similar are likely to be the closest neural computa- choices that has enabled learning of values. tion that we would call, in economic terms, a A third system is called “habitual.” It stable preference for a good. associates stimuli to responses (a canonical This learning system is goal-directed: “it example is human motor action, as in learn- depends on a representation of the action- ing to play a sport). Habits, by definition, do outcome contingency (that lever pressing not respond rapidly to changes in the value produces food: the cognitive map) and on the of outcomes that result from responses. A outcome as a desired goal or incentive (that habit is “outcome-insensitive so that it will food is valuable)” (Berridge and O’Doherty persist even after an outcome is no longer 2014, p. 396). valued” (Tricomi, Balleine, and O’Doherty That is, goal-directed valuation attaches 2009). If one could identify when a human value to actions, which are thought (in a choice response was being implemented by sense of mental representation) to achieve the habit system, one could reliably predict goals. Goal-directed valuation is more flex- that the short-run elasticity of a habitual ible because it can learn to shift when a val- response to changes in state variables (out- ued action is devalued, and will shift to a comes, prices, information) would be low. An different action that leads to the valued out- everyday example is “outcome-inappropriate come if the action–outcome contingency is cue-driven behavior (e.g., stepping out of an represented. elevator when the doors open, although it A special kind of goal-directed valuation is has stopped on the wrong floor)” (Tricomi, called “model-based.” In this case, informa- Balleine, and O’Doherty 2009, p. 2229). A tion about consequences of an action can be more dramatic canonical example is highly computed from a “model” or representation, addicted drug users, who overdose when even if the action has not been chosen and the drug purity changes because their dosing 1160 Journal of Economic Literature, Vol. LI (December 2013) choice is habitual and does not respond to Furthermore, even going from two to purity change. three (or a few) choices complicates the Many of the choices that are considered process quite a bit. Working memory could difficult arise when there is conflict between become a constraint on retrieving cached val- choices recommended by different value ues to compare. The presence of low-quality systems. When Bill Clinton was deciding unchosen objects is also thought to affect val- whether to have sexual relations with Monica ues of other objects, in violation of the prin- Lewinsky, a goal-directed system mindful of ciple that objects have menu-independent the chance of being caught and suffering values (Tversky and Simonson 1993). And if impeachment with highly-evolved prepared the objects have different features that are (and learned, in Bill’s case) valuation of sex. not integrated into a single object value, then Score: Legacy 0, Sex 1. the implicit “agenda” in which one object is compared against the set of two others 3.1.1 Major Unmet Challenges (Tversky and Sattath 1979) can affect choice, Going beyond the simple settings in which as has been known for decades. conditioned learning leads to stable prefer- It would be sad if the two-stage back ences presents at least two challenges. First, pocket model only works for highly-learned the basic experiments have no prices or choices among two free goods. As noted ear- income. Monkeys and people choose among lier, when a habit system makes choices it is “free” goods or, in some studies, bid for sin- unlikely, by definition, that many choices are gle goods with foregone juice rewards (Platt all separately valued then compared (e.g., and Glimcher 1999) or with money. As noted the nonhabitual choice will be ignored or earlier, however, the learning system is ideal underprocessed). And when a goal-directed for learning valuation but not for trading off system selects choices that contribute to dif- different goals (getting the most reward, and ferent goals—or the basic systems compete spending the least), which is required for in evaluating them, such as sexual temptation responding to changes in prices and income competing with avoiding a harassment law- constraint. suit—many models other than the two-stage It is easy to choose consistently between value-and-compare are likely to operate. two juices or foods. Measured efficiency Indeed, there is ample evidence that this is losses from inconsistent (nonmaximizing) so, which is briefly addressed in chapter 15. choices between two-good bundles are never 3.2 Neural Evidence Distinguishing higher than a few percent, even for people Different Behavioral and Rational with VMPFC damage (Camille et al. 2011). Processes However, there are many sources of complexity in naturally-occurring human The second general contribution of neuro- choices of much interest to economics. economics is evidence about whether behav- Complexity comes from temptation, risk, ioral economics computations are made by hidden quality, multiple attributes, future the brain. On this count, the coverage in timing, unfamiliarity necessitating goal- Glimcher’s new book is already well behind directed construction of value, and com- the curve, but for a deliberate reason. He petition between prepared, learned, and says, “[J]ust as neoclassical theory defined goal-directed systems. Under these condi- the starting point from which behavioral eco- tions, utility maximization is less plausible nomics begins its explanation, neoclassically (especially when the description of goods anchored models must serve as the start- can affect perceived value). ing point from which neuroeconomics will Camerer: A Review Essay about Foundations of Neuroeconomic Analysis 1161 develop as a broadly interdisciplinary field” found special gain activity in VStr, and special (p. 389). loss activity in amygdala and temporal lobe I disagree. Neuroclassical models are one regions. De Martino, Camerer, and Adolphs starting point for neuroeconomics (a conser- (2010) found that two patients with selec- vative start). However, given rapid progress tive bilateral amygdala lesions exhibited no in behavioral economics, there is no reason loss aversion. De Martino et al. (2006) also to privilege the neoclassical view as a start- found that amygdala activity was associated ing point. One could equally well move with common choices that reflect framing right away to a search for a general theory effects. Sokol-Hessner, Camerer, and Phelps of neural choice architecture, expecting (2013) found that emotionally regulating the to see limiting conditions in which rational reaction to financial loss reduced amygdala choice restrictions hold. Glimcher’s ratio- activity. All these studies indicate an interest- nale for the conservative approach is: “[W]e ing role for emotion in loss aversion. The fact have just begun to explore the strengths of that monkeys seem to exhibit loss-aversion our neural architecture for choice and valu- also indicates that it has a phyologenetically ation. Understanding the weaknesses of that “old” origin that is not unique to humans same system will doubtless follow, although (Chen, Lakshminarayanan, and Santos 2006; that understanding today is in its infancy” Lakshminarayanan, Chen, and Santos 2008). (p. 389). Nonlinear weighting of probabilities: Hsu Glimcher may simply not be aware how et al. (2009) reported evidence that nonlin- much we already understand behaviorally early weighted probabilities of valued out- about the “weaknesses of that same sys- comes are encoded in the striatum. This tem”—that is, the limits that psychologists finding suggests that such weights reflect and behavioral economists have been docu- genuine preference, rather than, say, a mis- menting using lab and field choice data since take in perceiving probabilities (unless the the 1970s. Given this knowledge, accompa- mistake underlies preference). nying neuroscientific evidence could prog- 3.2.2 Time Preference ress very rapidly, and actually is progressing. This section will briefly describe progress The neural architecture of expressed in just two directions (prospect theory and time preference is an area in which neuro- time preference), then mention findings on science is likely to have a large impact. For other behavioral economics topics. example, it is clear that patience in different species, and in the human developmental 3.2.1 Prospect Theory lifecycle, are associated with brain structure The key elements of prospect theory are and development (e.g., Rosati et al. 2007). reference-dependence of outcome evalua- Working memory is also associated with tion, loss-aversion, and nonlinear weighting patience, which suggests an important role of probabilities. for “keeping the future in mind” to make Loss-aversion: Tom et al. (2007) found patient choices. that across subjects, the difference in brain The first wave of neuroeconomic stud- response to potential loss dollar for dollar, ies focused on whether temporal choices relative to potential gain (“neural loss aver- depend on two systems, which separately sion”) was correlated with the degree of loss value a present reward (a bird in the hand) aversion inferred behaviorally from choices and a future imagined reward (two [birds] among gambles. (We call this type of correla- in the bush). One useful two-system tion “neurometric”). Yacubian et al. (2006), model is the model of quasi-hyperbolic β-δ 1162 Journal of Economic Literature, Vol. LI (December 2013) discounting (Laibson 1997). Valuation of a DLPFC activity—a “temporary lesion”— consumption stream at time can be written as, found that such disruption made people 1 ∞ more impatient (Figner et al. 2010), as if V(t) ​​ 1 ​ u(ct) τ u(ct , a = _ − + ∑​ ​​ δ ​ +τ) DLPFC enforces self-control, and disrupt- β τ =0 decomposition( into) a hypervalued initial ing us leads to lapses (and also consistently reward u(c ) (“inflated” by present bias if with McClure et al. 2004, 2007). t 1) and a conventional exponentially-dis- These few studies and new ones that β < counted stream of initial and future utilities. are accumulating rapidly, suggest a view If 1 the first term disappears. in which expressed preference for future β = McClure et al. (2004, 2007) present evi- rewards is determined by neural activ- dence from choices between money and ity associated with cognitive control (in juice suggesting a “meso-limbic dopami- DLPFC) and thinking about future (see β nergic” system (striatum and medial orbi- Peters and Büchel 2010; Carter, Meyer, tofrontal cortex (MOFC)) and a patient and Huettel 2010). This hypothesis should δ system includes dorsolateral prefrontal cor- not be shocking to economists, since it has tex (DLPFC) and parietal cortex. often been proposed in the form of mul- Using a different design with a fixed “pres- tiple-self “planner–doer” models (Thaler ent” reward, Kable and Glimcher (2007) fit and Shefrin 1981; Fudenberg and Levine a unified-value hyperbolic discount func- 2006). However, filling in the neural detail tion u(c ) (1 kt) to brain activity. They will lead to new comparative static predic- t / + conclude that there is only a single system tions about what mental state variables or (encoding u(c ) (1 kt)) since regions exogeneous events change expressed time t / + McClure et al. labeled regions were active preference. β when the delayed reward changed. For example, Hershfield et al. (2011) found Kable and Glimcher (2007) vocally argue evidence that people were more patient for the one-system view. However, other when they interacted with aged versions of kinds of evidence are consistent with some their “future selves” using facial-morphing kind of competition between two (or more) software. It is also well known that prefrontal types of valuation. For example, Luo et al. circuitry, including DLPFC, develops more (2009) find residual activity in valuation slowly than reward and affective circuitry in areas for immediate rewards, even control- adolescence (Casey, Getz, and Galvan 2008). ling for value (as revealed by preference). Given evidence of the role of DLPFC in Sellitto, Ciaramelli, and di Pellegrino (2010) future valuation and self-control immedi- showed that patients with MPFC damage ately suggests why adolescents are behav- are more impatient than matched neurotypi- iorally impulsive. This is important because cal patients. many adolescent decisions have long-run A different paradigm investigates self- consequences (teen pregnancy, schooling control during choices that are all immedi- choice, smoking, and drug use). ate. Hare, Camerer, and Rangel (2009) had 3.2.3 Other Behavioral Economics Topics subjects choose between foods like yogurt and candy bars, which vary in health (long- Social preferences: Behavioral economics run benefit) and taste (short-run benefit). has made substantial progress in replacing a DLPFC activity is associated with success- simple placeholder of self-interested pref- ful self-control—avoiding tasty unhealthy erence with richer models of social pref- snacks—by indirectly controlling value erence for equality, reciprocity, and social encoded in MOFC. A later study disrupting image. There is now ample data about how Camerer: A Review Essay about Foundations of Neuroeconomic Analysis 1163 hormones, genetics, and brain activity dur- the hidden value had higher activity in ing social choice are associated with social amygdala. These findings link computations preference (e.g., Fehr and Camerer 2007; made in behavioral game theory to brain Tricomi, Balleine, and O’Doherty 2009; regions involved in mentalizing “threat” or Rilling and Sanfey 2011). The data over- social fear (amygdala). whelmingly support the view that expressed Keep in mind that traditional equilibrium social preferences are based on internal analyses say nothing at all about individual valuations that are traded off with value for differences. Cognitive hierarchy and level-k one’s own payoffs. So far, there is evidence approaches are one way to do so, and neural consistent with different types of valuation, activity will help distinguish “good” (high- including warm glow (Harbaugh, Mayr, performance) players from weak players. and Burghart 2007), inequality aversion Tentative neuroeconomics data are also (Tricomi, Balleine, and O’Doherty 2009), generally consistent with behavioral econom- and social image (Izuma et al. 2011; Izuma, ics hypotheses in other domains, including: Saito, and Sadato 2010). Strategic thinking: Mentalizing or “the- • Crowding out of intrinsic incentives ory of mind” is the capacity, either unique (Murayama et al. 2010); to humans or most reliably developed in us, • Negative performance response to very to imagine what another organism believes, high incentives (“choking”) (Chib et al. wants, or intends. Obviously such a capac- 2012); ity is presumed by most analyses in game • Overlap in activity during hypothetical theory. and real choice (consistent with overre- Studies are beginning to clearly indicate porting purchase intentions hypotheti- a link between theory of mind regions and cally), and more activity in real choices in game-theoretic computations (see Hampton, in value-computing areas, for consumer Bossaerts, and O’Doherty 2008; Coricelli goods (Kang et al. 2011), and in insula and Nagel 2009; Camerer and Smith 2012). and amygdala for “bads” (eating unpleas- The data also suggest individual differences ant foods) (Kang and Camerer 2013); that are consistent with a cognitive hierar- • Reduced value-related activity when chy or level-k approach to iterated reasoning choice sets are larger than ideal due to (Camerer, Ho, and Chong 2004; Crawford, information processing (“choice over- Costa-Gomes, and Iriberri 2011). load”) (Reutskaja et al. 2011); For example, Bhatt et al. (2010, forth- • Evidence for computation of “foregone coming) studied a bargaining game with payoffs” or fictive learning in decisions one-sided private information. A buyer and games, as in EWA learning in games observed her hidden value, then stated a (Camerer and Ho 1999) (see Thevarajah (cheaptalk) suggested price to a seller. The et al., 2009; Zhu, Mathewson, and Hsu seller inferred whatever she could about 2012); the hidden value from the suggestion, then • Evidence that stock market bubbles stated a take-it-or-leave-it price. Bhatt are associated with enhanced mentaliz- et al. (2010) found that when buyers were ing contributes to stock market bubbles extremely deceptive (level-2), they had (DeMartino et al. 2013) and with het- more activity in DLPFC and in the tem- erogeneous trader response to nucleus poral-parietal junction (TPJ). Sellers who accumbens signals and to insula “early offered prices that reflected suspicion about warning” signals of impending crashes whether buyers revealed information about (Smith et al. 2013). 1164 Journal of Economic Literature, Vol. LI (December 2013)

3.3 Neural Evidence of Other Psychological plainer terms, suppose you regularly inject Processes heroin with K.R. on Avenue C after 2 a.m. bar closing. Then seeing K.R., walking past The most novel contribution of neuro- Avenue C, or noticing that people are leaving economics to economics, per se, may come bars that just closed could induce craving for from supplying evidence about other kinds the drug, as the body adjusts homeostatically of complex psychological processes that have for the expectation of imminent drug use. been understudied in behavioral economics. Laibson (2001) extends the rational choice In these cases, their complexity makes the model to include such cues. axiomatic approach and “as if” testing espe- Emotion: Thorough modeling of emotions cially challenging, and possibly inefficient, has proved elusive in economics (though see compared to supplementing theorizing and Loewenstein 2000; Romer 2000; Loewenstein analysis of field data with information about and O’Donoghue 2007). Some useful exam- mechanisms. Here are two loosely sketched ples treat a particular emotion as a source examples: addiction and emotion. of disutility and look for evidence of a nega- Addiction: This is a topic in which both tive emotion manifested in choice data (e.g., simple extensions of rational choice mod- Costa and Kahn 2007). In psychological game eling and biological details have proved theory (Dufwenberg 2008) and evolution- insightful. Becker and Murphy’s (1988) ary psychology social emotions are thought famous approach boils addiction down to to functionally signal surprises relative to adjacent complementarities between current expectations, and convey an expectation of consumption and a “habit stock” based on future action. For example Sell, Tooby, and past consumption. Their model allows some Cosmides (2009) define as an expression of breathing room for investments to adjust the disappointment in getting less from another habit stock, such as deliberate time in rehab, person’s action than was expected, which both and also allows shocks such as stressful invites an apology and threatens revenge. events (divorce, unemployment) that could Emotion is absent from Glimcher’s book, trigger “self-medicating” substance use. A and indeed, is also absent from much of richer model based on consumption states is decision neuroscience. An economics- Bernheim and Rangel (2004). friendly view is that an emotion is a rapidly- On another front, the neurobiology and processed reaction to good or bad news, felt clinical evidence indicates several interesting by the body and expressed both externally facts about the underlying mechanisms of (e.g., by facial expression, so other people addiction. Rats become biologically addicted, can see) and internally (by the nervous sys- exhibiting tolerance and withdrawal, to all tem and brain circuitry). Adolphs described substances humans do, which implicates the modern view this way: “. . . emotions are a common source in homologous rat and triggered by events of some significance or human brain areas. Many users exhibit poly- relevance to an organism, that they encom- drug use, substituting between drugs (per- pass a coordinated set of changes in brain haps very rationally in the sense of exhibiting and body, and that they appear adaptive in high cross-price elasticity and sensitivity to the sense that they are directed towards cop- full price effects like shifts in policing prior- ing with whatever challenge was posed by ity). A remarkable finding is that drug use the triggering event” (2010). habituates users to create homeostatic “feed- In my view, economics should simply forward” adaptation to environmental cues define an emotion as a distinct primitive con- (like the time and place of habitual use). In cept that can have characteristics of ­utilities, Camerer: A Review Essay about Foundations of Neuroeconomic Analysis 1165

­information, or constraint. For example, How could we describe emotional regu- fear feels unpleasant (people usually spend lation in economic terms? People have the resources to avoid it), conveys noisy informa- capacity to imagine how they would feel and tion about impending threat, and can reduce behave in different mental states (includ- cognitive control and physical action (as in ing emotional ones). Attention to the simu- “paralyzed with fear”). lated state crowds out attention to the true Note also that there is no sharp divide state, which changes behavior and biological between emotion and cognition; they are like expression of emotion. Siamese twins. Top–down cognitive “reap- In fact, this sort of imagination is used rou- praisal” influences emotions, and emotions tinely in life and in economics. One approach influence information processing. to acting is to imagine previous experiences Incorporating emotion into economics is that produce the emotion that is desired. For likely to be substantially aided by affective example, if you imagine how you would feel neuroscience. The neuroscience provides if your child died, you might feel genuine multiple biological measures of emotions, sadness. This doesn’t mean that you “think” which are especially useful because any one your child is dead; it just means you have the measure—such as skin conductance—usu- capacity to do counterfactual reasoning, and ally does not measure both the intensity of that reasoning can produce powerful emo- the emotion (arousal) and its “sign” (positive tions and can change behavior. or negative “valence”). In economics, the ability to imagine what An example is two studies by Sokol- you might do in another state is essentially Hessner et al. (2009) and Sokol-Hessner, assumed in game theory when there are Camerer, and Phelps (2013) on “emotional information asymmetries (e.g., a bidder regulation” (also called “reappraisal”) dur- must imagine what a bidder with a different ing risky choice. Their subjects made a series value than their own will do, in an auction, of choices between certain amounts and unless he or she learned an equilibrium bid- 50 50 gain–loss gambles (designed to esti- ding function over time). So in economic / mate individual parameters of risk-aversion, language, we could translate the psycho- ­loss-aversion, and response noise). They logical concept of emotional regulation were trained to turn on and off a “cognitive into “the use of scarce cognitive resources re-appraisal,” in blocks of 10 trials, which is to self-create alternative states.” Or we can intended to reduce negative emotion when just learn some new vocabulary—regula- taking a risk. tion, or reappraisal. When subjects are “regulating” their nega- Social networks: A small literature is tive emotions, their estimated loss-aversion emerging showing a remarkable link between is lower, and their skin conductance response the extent of social networks, both across and to actually losing (when they do choose a within species. This is interesting as forming gamble) is lower (Sokol-Hessner et al. 2009). network links is a socio-economic choice, and Regulating subjects are changing revealed networks are also institutions which can pass preferences by changing their emotions. information and affect preference. The earli- fMRI analysis showed that DLPFC was est finding is that, across primates and apes active during regulation. Subjects who could (including Homo sapiens), the relative size of regulate their loss response (as inferred from the neocortex is monotonically related to the behavioral loss-aversion) showed reduced size of a typical everyday social group—for activity in the amygdala during loss (and no humans, our extra neocortex implies a group difference in gain). size of about 150 (Dunbar 1992). 1166 Journal of Economic Literature, Vol. LI (December 2013)

More surprisingly, new studies show that is heavily promoted in Glimcher’s book.2 even within species, differences in brain Axiomatic systems summarize all the empiri- volume in amygdala and cortex are associ- cal choice restrictions a general class of mod- ated with personal group size (Dunbar 2012; els should satisfy, without typically specifying Bickart et al. 2011), and with monkey group too precise a functional form. For example, size and dominance ranks (Sallet et al. 2011). expected utility theory axioms allow the util- None of these studies can clearly conclude ity function to have any shape, but require whether brain structure supports large net- valuations to be sums of outcome utilities work interaction, or large networks grow weighted linearly by their probabilities. If brains. The crossspecies evidence suggests the independence axiom is systematically bigger brains subserve bigger networks, but violated, then choices are not described by general brain plasticity (in response to envi- expected utilities. ronment) also suggests that causality goes in The simplest example of how axiomatics both directions. It would be fascinating to could work in neuroscience is reward pre- examine brain structures and activity in peo- diction error (RPE). RPE is the difference ple raised in unusually large or small groups between a received reward and an expecta- (for exogeneous reasons) to help establish tion (an economic “surprise”). RPEs occur causality. during learning, and mathematical adjust- ment to RPE can lead to good long-run learn- ing properties (e.g., Sutton and Barto 1998). 4. Reviewing Foundations of RPEs have generally been measured as Neuroeconomic Analysis linear deviations between reward and expec- As noted, the book is apparently not tations. Caplin et al. (2010) took a step back intended to be a comprehensive discussion of and suggested axioms defining a more gen- all of the methods and discoveries in neuro- eral RPE structure. Define an RPE for out- economics, as a handbook or textbook would come x and expectation (or reference point) strive to be. Instead, it is one part philosophy r as (x, r). An axiomatic definition of an δ of knowledge, one part history of neurosci- RPE is a function (x, r) which is increas- δ ence, one part personal vision of a field, and ing in x, decreasing in r, and which satisfies one part NYU promotional brochure. (x, r) 0 when x r (i.e., the prediction δ = = From the point of view of economists, the error is zero when x is expected and received, book’s coverage and tone reflect two method- regardless of the amount of reward x). ological convictions: first, that simple axiom- Many previous studies found evidence for atic systems which summarize the empirical RPE encoding based on a linear restriction implications (i.e., what axioms must be satis- (x, r) x r, where r(t) is an expectation or δ = − fied) of a class of functional representations reference point that is tracked by reward his- should be central objects of study in neuro- tory (typically with a decay or learning rate). economics, and second, that neuroeconom- Caplin et al. (2010) cite these previous ics can most productively start by exploring findings of linear RPE and conclude that whether neural circuitry implements simple “although generally supportive, these tests rational choice. have all failed to be taken as conclusive proof of the DRPE hypothesis by assuming 4.1 How Useful are Axioms? fixed values for the ‘experienced reward’ of The view that axiomatic models will prove especially helpful in neuroeconomics 2 See also Caplin and Dean (2008), and Dean (2013). Camerer: A Review Essay about Foundations of Neuroeconomic Analysis 1167 different events and using a reinforcement ­positively to reward, negatively to prediction, learning model to construct a time path for and the signal for getting either expected ‘predicted reward’” (p. 927). Basically, their reward is the same. The only difference is planned contribution is to find RPE signals that Abler imposes linearity and Caplin et al. with a less parametric linear restriction. (2010) allow nonlinearity (and fanning out of However, their own analysis assumes fixed the two curves). (cross-trial) values for experienced reward So let’s take stock of what the axioms have (by binning them according to equal out- added here. Axioms suggest that the strongest comes) and assumes a fixed predicted reward test is not what brain areas respond to linear based on stated probabilities, rather than (x, r) x r, but instead what areas indi- δ = − predictions adjusted along a time path. Thus, cate positive and negative responses to x and even though their axiomatic system is more r, respectively, with (5, 5) ( 5, 5) 0. δ = δ − − = general, in principle, than the linear version, Regions responsive to RPE identified by in execution it makes strong assumptions Abler et al. (2006) several years ago appear that reward expectation is constant. to pass these tests, visually. However, Abler The main point here is that both earlier et al. did not actually conduct the prescribed analyses and their analysis must assume some statistical tests; they impose linearized ver- auxiliary hypothesis about stability of reward sions. So they got the right answer, but with- values across trials, and some source of pre- out statistical proof. Put differently, in this dictions. Why is one maintained hypothesis, case the axioms point to more general tests then, more “conclusive proof” than another? about the shapes of BOLD response, but The key difference is simply that the Caplin the linearized results are so compelling the et al. (2010) analysis requires monotonic- contribution is not an empirical surprise, but ity but allows nonlinearity in the response instead is a useful methodological sugges- to reward and prediction, and allows cross- tion about how to do more careful statistical partial effects of reward outcome on predic- testing.4 tion. However, given that linearity is almost Caplin et al. (2010) conclude that, “our always an excellent approximation to a non- work rigorously tests and confirms the linear monotonic function, this is a conceptu- conclusions of previous authors who have ally important distinction but not a practically claimed to have found evidence in favor of important one.3 It is an incarnation of non- the DRPE hypothesis in fMRI data.” I agree parametric versus parametric testing in which that there is an added dimension of “rigor” a parametric restriction is quite reasonable. from statistical testing of more general func- Figure 1 shows the BOLD signal from tional forms (response to x and r rather than Rutledge et al. (2010), with reward predic- linear response to x r), but there is not a − tion on the x-axis. Figure 2 shows a com- big leap in knowledge. parable graph from Abler, who identified Furthermore, there are other neuromet- ROIs using an imposed linear contrast ric dimensions of rigor and confirmation (x, r) x r x – px. The Abler figures strength. For example, as Caplin et al. (2010) δ = − = show strong support for the three general note, there are relatively few dopaminer- properties of DRPE axiomatically identified­ gic neurons in the brain (c. 105). Focusing by Caplin and Dean: signals respond on nucleus accumbens, which is known

3 Abler et al. (2006) also tried a quadratic term using 4 Rutledge et al. (2010) uses the same data as Caplin reward prediction error (RPE) but had no extra success, as et al. (2010) but explores the statistical testing of compari- the linear approximation fit well. sons in multiple regions in a more thorough, powerful way. 1168 Journal of Economic Literature, Vol. LI (December 2013)

0.2 – +$5 –$5

0.1 –

0 – BOLD activity

-0.1 –

-0.2 – – – – – – 0 0.5 1 Probability of winning $5

+$5 –$5 –$5 +$5 –$5 +$5 –$5 +$5

Figure 1. BOLD Activity in Response to Actually Winning or Losing $5, Depending on the Probablility of Winning

Notes: Reward prediction errors should decline monotonically from left to right, be higher for wins ($5) than losses (–$5) fixing probability, and the response to losing for sure (left point for –$5) should be the same as winning for sure (right point for $5). All these properties hold statistically. Sources: Caplin et al. (2010). Reprinted with permission of the Journal of Neurophysiology.

to receive a large density of DA neurons, neurons. Following this strategy, D’Ardenne is then a reasonable choice. An even more et al. (2008) recorded BOLD from the small conclusive approach, in one respect, is to midbrain ventral tegmental area (VTA). The record BOLD from areas that are known VTA was the original locus of recording by from single-unit electrode-based record- that ignited empirical study of prediction ings to have an especially high density of DA error (Schultz, Dayan, and Montague 1997). Camerer: A Review Essay about Foundations of Neuroeconomic Analysis 1169

Expectation of reward 2 – 2 – Expectation: Expectation: left ventral striatum right ventral striatum 1 – 1 –

0 – 0 –

–1 – –1 –

–2 – –2 – Individual values of effects 0% 0% 25% 50% 75% 25% 50% 75% 100% Individual values of effects 100% Probability of reward Probability of reward

Reward outcome/ prediction error 6 – Outcome: 9 – Outcome: left ventral striatum right ventral striatum 4 – 6 – 2 – 3 – 0 – 0 – –2 – –2 – –4 – –6 – –6 –

Individual values of effects –

Individual values of effects –9 0% 0% 25% 50% 75% 25% 50% 75% 100% 100% Probability of reward Win/ No win trials Probability of reward

Figure 2. Bottom panel shows linear contrast of prediction error depending on known reward probability (x-axis) and actual outcome (light win, dark loss). = = Note: Figure reprinted with permission of Neuroimage.

The VTA is small (only 2 voxels of typical for Nash equilibrium­ (until Aumann and 3 x 3 x 3 mm size) so they use a thinner slice Brandenburger 1995). Informational effi- and some improvements in methods—a dif- ciency of stock prices, a hypothesis which ferent kind of rigor—to improve data quality. drove empirical finance for decades starting They find coarse evidence for DRPE in an in 1965 with Gene Fama’s dissertation, did area thought to be richer in DA neurons than not have clear formal basis until Grossman NAcc (figure 3) Their data quality do not and Stiglitz (1980) showed, ironically, that permit a statistically powerful test of the gen- full efficiency was implausible (if informa- eral Caplin et al. (2010) axioms. However, tion is costly to gather). Gilboa et al. (2013) they have the advantage of finding evidence point out other examples. more tightly linked to where DA is actually It is notable that historically, important transmitted. advances in axiomatization of choice models Stepping back, how useful are axioms often come after empirical inspiration. This in neuroeconomics, and in general? First was certainly the case with Glimcher et al.’s note that axioms can illuminate and clarify work on RPE. It is also true in axiomatic work a good idea, but some good ideas also shine on preferences on menus, as illustrated by on their own. In economics, for a long time the many experimental examples described there was no axiomatic (epistemic) basis by Lipman and Pesendorfer (2013), which 1170 Journal of Economic Literature, Vol. LI (December 2013)

A B *

T 0.008 - * 10

0.004 -

–10 Effect size 0 -

Unexpected reward Expected reward –0.004 - Reward expected, not received

NS

Figure 3. BOLD Response in Ventral Tegmental Area (VTA), Fine-Grained Imaging, in Response to Unexpected Reward Surprises

Notes: Results are not as strong in Caplin et al. (2010) and Abler et al. (2006), but image smaller regions of VTA known to be rich in dopamine receptors. Source: D’Ardenne et al. (2008). Reprinted with permission of Science.

motivated axiomatics. Axiomatic systems these lines, Gilboa et al. (2013) propose a are perhaps most useful at a middle stage formal interpretation of axiomatic mod- of the data-theory-data cycle, showing the els as analogies. They note (p. 17) that an underlying ingredients are necessary and axiomatization cannot make a theory more sufficient to account for a phenomenon, true or false. Instead, “axiomatizations (in once the phenomenon is established. In this this case, of utility maximization) point view, neural data might serve as inspiration out to us similarities that are not obvious a for later modeling (Spiegler 2008). It could priori.” be that axiomatization will follow neural evidence, rather than lead it (as in the RPE case). The alternative to an axiomatic or non- problems with nonaxiomatic parametric tests: 1) Tests parametric approach is a workmanlike tin- require a joint hypothesis about RPE and rewards, beliefs and observables (but axiomatic tests do too); 2) parametric kering approach in which surprising effects tests of one theory do not reject another poorly-specified are first generated (often absent any model theory (axiomatic tests don’t either); 3) parametric tests at all), then parametric specifications are may lack power to distinguish different functions (but gen- eral axioms have no distinguishing power at all by construc- proposed and compared, and constant tion, and if distinction is sought it could be achieved, e.g., 5 modification improves the models. Along Hare et al. 2008); and 4) comparative “horse race” tests of specific theories do not give information about goodness- of-fit (that’s wrong; they do). Thus, there is no special guar- 5 In tests of dopaminergic activity as encoding RPE, antee that tests of axioms are necessarily more conclusive Caplin and Dean (2008, p. 26–27) note four potential than fitting specific functions in any of these four ways. Camerer: A Review Essay about Foundations of Neuroeconomic Analysis 1171

Whether axioms or tinkering are bet- According to “[David] Krepsian” method- ter has something to do with scientific risk ology, one starts with an intuition or more tastes. The axiomatic approach is safer, systematic observations suggesting how because choices will satisfy the axioms if any a certain psychological “force” (ambigu- specific function within the class is correct. ity aversion, anxiety, desire for flexibility) Conversely, if the axioms are clearly rejected, might be connected to observed behavior, that could be bad news (less so in the anal- and one looks for a domain of choice objects ogy interpretation). While the rejections (Anscombe–Aumann acts, temporal lotter- may offer guidance about how to weaken ies, menus), such that preferences over this axioms, there may be a natural tendency to domain could elicit this force. either object to the evidence or postpone for The whole point of the modeling exer- a long time the much harder work of find- cise is to identify the “force” by its impact ing weaker systems with plausible testable on choice, without using any direct measure. implications (as happened for many years However, the types of choice data needed to after initial reports of the Allais and Ellsberg test these choice-based models get more and paradoxes).6 more complicated as the psychological force The tinkering approach is a bolder strat- becomes more interesting. For example, in egy, since a more specific functional form is menu-based theories, the data needed are more likely to be rejected. However, if a spe- whether people choose over a wide variety cific functional form passes strong tests, that of larger or smaller choice sets. What field is very good news. And a strong advantage settings supply these data? of working with a specific functional form is Whether the original modelers like it or that more precise predictions can be made not, Krepsian analysis (as Spiegler describes about how much choices respond to changes, it) certainly invites neuroeconomic “direct not simply the likely sign of a response. measure” of the hypothesized psychologi- With this example in mind, it is now useful cal forces because nonexperimental clear to reflect on how promising axiomatic sys- choices over menus might be rarely avail- tems might prove to be in general. able. However, many—perhaps most—of Axiomatic systems are certainly useful in the theorists working in this tradition seem consolidating advancement of economic to think the main value of the exercise is to knowledge. However, the reliance on axi- avoid having to measure anything other than omatic systems to tie observable choice data choices. But any particular axiomatic system to revelation unobservables is also the heart is just one particular way of linking hypoth- of the Gul–Pesendorfer assertion that non- esized forces with axioms and observable choice data are not needed. preferences. It would be an amazing coinci- In my view, axiomatic systems are likely to dence if the axiomatic systems the Krepsian have a more limited role in neuroeconomics theorists produce happen to coincide with than Glimcher believes. To see why, consider how the brain works (unless the axioms are how Spiegler (2013) describes the recipe for proposed with brain knowledge in mind). a particular popular axiomatic approach: There is hope for such coincidences,7 but I predict that the Krepsian program is not likely to produce many of them. 6 Note that it took more than 20–30 years after the Allais paradoxes were pointed out for prospect theory and other formal non-EU systems to emerge. There was a similar lag 7 A promising example is Fudenberg and Levine (2006), from Ellsberg’s 1961 paper on ambiguity-aversion to axi- who clearly think of their short-run vs. long-run self model omatic systems. as having some neural plausibility. 1172 Journal of Economic Literature, Vol. LI (December 2013)

Finally, it is certainly true that the RPE was anticipated, in a sense, by Pareto who axiomatics is a successful application because specifically thought of stable tastes as result- the axioms tell you clearly what to look for in ing from a trial-and-error learning process.9 the data . . . and the data are easily available The extrapolation from this well-docu- from an experiment (possibly even in a field mented process of learned simple valuation setting). to more complex lifelike choices will be much more challenging. In this sense, Foundations 4.2 What is Most Authoritative and documents the current solution to the easi- Convincing est problem, to show economists what part of a good neuroeconomic foundation looks The core of the book, which is most like, but leaves the bigger challenges for the authoritative, is building the empirical case future. for a two-stage model of simple choice: each There are many kinds of evidence and object in a choice set is valued, then the val- nascent theories (some going back decades) ues are compared and the best one is chosen showing when the simple value-then-com- (perhaps with stochastic choice about what’s pare does not seem to apply, including: “best”). The neural evidence is compelling. Neural • Choices among several objects with mul- brain activity of different sorts—neural firing tiple attributes, in which important attri- rates from single-unit recording, and BOLD butes are often used to screen out poor signal from fMRI—is associated across many choices (Tversky 1972); domains with expressed value ratings (on • A shift from decision rules using more numerical scales) and with utilities inferred information to rules using less infor- from decisions. This is an important step mation as choice complexity increases in showing that there are neutrally-derived (Payne, Bettman, and Johnson 1993); numbers that correspond to behaviorally- • Regret theories in which the sign of derived numbers. the comparative value of choices A The evidence for a comparison process and B, (A,B), determines choice (A is ψ is not as clear. (Choice among four or more preferred if (A,B) 0) (Loomes and ψ > options, for example, has rarely been studied Sugden 1987). In this theory there is not in decision neuroscience, to my knowledge). necessarily an initial valuation of choice There is much clear evidence of neurons objects separately; implementing an optimal sequential proba- bilistic likelihood test (Gold and Shadlen 2007), OFC neurons that fire in response and are noisy, learning degrades badly. These conditions to comparative value (Padoa-Schioppa have not been studied in human decision neuroscience. and Assad 2006), and cingulate activity in 9 Pareto (1971) wrote: “A man who buys a certain food response to comparative value (e.g., Hsu for the first time may buy more of it than is necessary to satisfy his tastes, price taken into account. But in a sec- et al. 2005). ond purchase he will correct his error, in part at least, and Utility maximization over two choices thus, little by little, will end up by procuring exactly what works well because reinforcement learning he needs. We will examine this action at the time when he has reached this state. Similarly, if at first he makes a mis- systems do very well under environmental take in his reasoning about what he desires, he will rectify 8 stability with rapid clear feedback. This fact it in repeating the reasoning and will end up by making it completely logical. (Ch. 3, §1)” Bruni and Sugden (2007) give a wonderful description of this point and the ongo- 8 My thesis advisors Einhorn and Hogarth (1978) made ing debates surrounding it during the rise of neoclassical the important point that when feedback lags are not rapid, economics. Camerer: A Review Essay about Foundations of Neuroeconomic Analysis 1173

• Effects of irrelevant alternatives or ­alternative can change the ranges and choice context (also called menu effects) deform normalized subjective values of on individual valuation or comparison of other choice objects. choices (e.g., Soltani, De Martino, and Context-dependent choice has been Camerer 2012); observed in many species besides humans, • Choices from lists (Rubinstein and Salant including honeybees, gray jays (Shafir, Waite, 2006) and satisficing (Caplin, Dean, and and Smith 2002) and slime molds (Latty and Martin 2011) (not all objects are valued). Beekman 2011). The species-generality of these effects suggest that if there is a single explanation of context effects, it is a low- 5. Highs and Lows in Foundations of level sensory normalization, rather than a Neuroeconomic Analysis sophisticated explanation, which does not apply to slime molds (e.g., Kamenica 2008). 5.1 Highlights Privileging a low-level explanation immedi- ately invites a careful look at basic neurosci- Glimcher is a careful and extraordinarily ence, in which normalization is common in clear writer. There are many gems. For sensory systems and is therefore best under- example, Glimcher notes that “[a]t its incep- stood at low levels. tion WARP defined a minimalist esthetic There is a deep, important discussion of that dominates economics to this day. How neural normalization in chapter 10, and its little can you assume in your model and how implications for economics (much based on much can it prove?” (p 57). an important paper by Louie, Grattan, and This compelling passage also shows why Glimcher 2011). The main intuitive point (most) psychologists dislike economic mod- they draw out is that as choice sets grow eling—the models have “too little” in them! larger, normalization means that the (nor- Of course, for economists that is precisely malized) subjective value of the best and the point, and the source of such models’ nearly-best options will fall. This effect will charm and power. create more choice “errors” (or random-util- Range normalization: The simple value- ity-based reversals), lower choice satisfaction and-compare model is most vulnerable in (if satisfaction if driven by relative compari- comparing choices across choice sets. The sons), and possibly choice postponement reason is that increases in the range of stim- (if there is a choice threshold that no single ulus attributes lead to local divisive normal- choice can reach). ization (also called “range adaption”). This The book ends with a nice string of fire- general phenomenon has been well known cracker explosions. for about thirty years, in behavioral eco- Chapter 14 is wonderful. It organizes nomics in the form of context- (or menu-) material around region-by-region evidence dependence (Huber, Payne, and Puto 1982) of how five brain regions are active during and was precisely modeled starting in 1992 aspects of subjective valuation. This is meant for visual cortical neurons (Heeger 1992; to be the best available answer to “Why care Carandini, Heeger, and Movshon 1997); about where?” The general answer is that, and see Padoa-Schioppa (2009) for range- if a region appears to make two different dependent value encoding in OFC. This computations that were not thought to be property can clearly violate independence related in current theory, that finding sug- of irrelevant alternatives (IIA) in choice, gests a need to have a theory linking those since an irrelevant (unchosen, ­low-value) computations. 1174 Journal of Economic Literature, Vol. LI (December 2013)

wealth or welfare, rather than final states.

Chapter 16 collects the formalisms devel- This assumption is compatible with basic prin- oped earlier in one place. Economists who ciples of perception and judgment. Our per- confuse math-free psychology and compu- ceptual apparatus is attuned to the evaluation tational neuroscience should read this chap- of changes or differences rather than to the ter over and over, and assign it to students. evaluation of absolute magnitudes. When we respond to attributes such as brightness, loud- It contains much of the formalism that will ness, or temperature, the past and present con- propel decision neuroscience for the next text of experience defines an adaptation level, 10 years. An action-packed picture accom- or reference point, and stimuli are perceived in panying this grand vision is reproduced in relation to this reference point. Thus, an object figure 4, showing how economic, psycho- at a given temperature may be experienced as hot or cold to the touch depending on the tem- logical, and neuroscientific analyses can be perature to which one has adapted. The same linked from high level (behavioral) to low principle applies to non-sensory attributes level (mechanistic). such as health, prestige, and wealth. The same The last chapter, 17, is a perfect ending to level of wealth, for example, may imply abject the book. The chapter poses six unanswered poverty for one person and great riches for another—depending on their current assets. questions. Every graduate student or new- (p 277) comer to this field should read this list (as well as Huettel 2010). A more ambitious list Thus, Kahneman and Tversky clearly state could be constructed, but Glimcher’s com- that the reference point is “the past and pres- pact six-point plan gives us plenty to do. ent context of experience” and give examples of what that means. It’s true they do not pro- 5.2 Lowlights vide “information” (i.e., data), but there is There are some things to disagree about plenty of inspiration. in the book. The substantive areas undoubt- Glimcher (p. 294) lauds the step forward edly reflect the author’s idiosyncratic beliefs by Koszegi and Rabin (2006), who focus on about intellectual contributions, history, and “recent expectations” about likely outcomes directions (chiefly, discussions of prospect as reference points. However, Kahneman theory). Some other debatable material, in and Tversky clearly anticipated this insight, my view as a long-standing contributor to noting that the status quo is not always the the field, reflects a misunderstanding about natural reference point. They wrote: behavioral economics or simply repeats Although this is probably true for most choice criticisms that have been addressed early problems, there are situations in which gains and often. Chapter 15, titled “Beyond and losses are coded relative to an expectation or aspiration level that differs from the status Neoclassics: Behavioral Neuroeconomics,” is quo. For example, an unexpected tax with- weak. It features an intriguing discussion of drawal from a monthly pay check is experi- hierarchical “editing” (which deeply under- enced as a loss, not as a reduced gain. Similarly, cuts extending the value-compare model an entrepreneur who is weathering a slump beyond 2-choice sets), but no data. with greater success than his competitors may interpret a small loss as a gain, relative to the For example, Glimcher writes that larger loss he had reason to expect. (p 286) “Kahneman and Tversky provided almost no information about how the reference This Kahneman and Tversky passage point was to be determined . . . .” Actually, clearly shined a flashlight in the direc- Kahneman and Tversky wrote: tion of expectations rather than status quo. (Kahneman 1992 also wrote about mixtures An essential feature of the present theory of reference points in a largely-overlooked is that the carriers of value are changes in paper that is insightful). Camerer: A Review Essay about Foundations of Neuroeconomic Analysis 1175

–U (x) Reservation ″ –U (x) Preferences based search ? ′ ? RUM Discounting ε EWA ArgMax Independence U(x) axiom surprise learning

ECONOMICS

Log Stevens’ numerosity Power Law Sacri cing Reference point Behavioral Impulsivity Attention stochasticity ? self-control

Choice Decision value x(P) RPE

PSYCHOLOGY

Parietal Transducer Drift diffusion Ef cient coding numberline biophysics models hypothesis Neuronal Neuronal Dorso- Orbitofrontal gain & tuning stochasticity lateral PFC cortex Winner- Ventral striatum take- medial PFC ? Dopamine all networks activation

NEUROSCIENCE

Figure 4. Schematic Showing Correspondence of Economic (Behavioral), Psychological, and Neuroscientific Concepts

Source: Glimcher (2011). Figure courtesy of the author. 1176 Journal of Economic Literature, Vol. LI (December 2013)

Glimcher goes further in making bold the essence of Samuelson’s book, which is a claims: “What we can see from this neuroeco- pathbreaking mathematical analysis of eco- nomic analysis, however, is that these local nomic systems, not just a useful model of irrationalities [reference-dependencies] arise individual behavior. because evolution is trading off the costs of The book is also packed with the loudest accurate sensory encoding against the costs cheers when the home team (NYU) scores. of irrational decision making.” Remember the famous 1976 Saul Steinberg Of course, the neuroeconomics per se does “New Yorker” cover? It depicted a dis- not show any clear evidence of an evolutionary torted map looking west from 9th avenue tradeoff. Proving that evolutionary selection in Manhattan. The cartoon showed detailed led to a particular kind of neural architecture pedestrians, buildings and cars from the East for tradeoff is extraordinarily difficult. The River up to 10th avenue, then a thin strip of evidence just shows that relative coding is “Jersey” past the wide Hudson River, and a pervasive and absolute coding is not. There modest patch representing the entire rest of is no evidence from the neuroscience, per the United States. se, that the degree of reference-dependence­ This book is akin to a Saul Steinberg car- seen in modern human experiments is a clear toon from a NYU-roeconomic perspective. product of an evolutionary tradeoff. I think Some of the pro–NYU attention is justi- that hypothesis is likely to be true in many fied, but the emphasis is heavy and uneven. respects, but it has not been tested directly; Besides Glimcher’s own work, the work and doing so is an exciting (and difficult) challenge collaborations with Andrew Caplin (NYU) for future work. get special attention and explanation, as does Glimcher gets a little carried away in some an influential mathematical normalization other passages (as enthusiastic authors can of visual cortex responses proposed David do). He says that normalization “may provide Heeger (1992). A normalization of neu- the central tool for fully cardinalizing utility” ral inputs due to Schwartz and Simoncelli (p. 241, fn 10). He also thinks “the baseline (2001) (one of several in the literature) is ush- level of activation measured by the scan- ered onstage as the “Schwartz–Simoncelli” ner—the unique zero point—is the physi- equation. cal instantiation of the reference point…this However, other local NYU contributions finding also provides us with an empirical to neuroeconomics are surprisingly absent. tool for rendering the [expectational] refer- NYU cognitive neuroscientist Elizabeth ence point directly observable” (p. 354). Phelps collaborated with NYU economist Andrew Schotter and others (Delgado et 5.3 Oops: Style and Curious Emphases al. 2008) to show how fMRI activity during There are some notable stylistic features overbidding in auctions suggested a kind of of the choice of what material to emphasize endowment effect from “losing” an auction that I found strange or objectionable. These one was expecting to win. The neural evi- choices are a book writer’s prerogative; and dence suggested a novel prediction about reacting to them is also a reader’s prerogative. how economically identical auctions would First: The title!? The name is presum- lead to different bids, a prediction they ably an homage to Samuelson’s aptly-named confirmed. Using neural evidence to pre- Foundations of Economic Analysis. Naming dict novel, surprising institutional effects is a book after such a seminal classic invites a something economists are keenly interested comparison that an author should not want in, so it is unfortunate that was not discussed readers to make. The homage title also misses at all in the book. Camerer: A Review Essay about Foundations of Neuroeconomic Analysis 1177

Contributions of other neuroeconomists That’s not how it happened (see Kahneman outside of Manhattan below 14th Street and Tversky 1982). are also omitted. Greg Berns, Giorgio He says that Kahneman and Tversky got Coricelli, Mauricio Delgado, Ernst Fehr, reference dependence by “borrowing from Kevin McCabe, Aldo Rustichini, and Paul earlier neoclassical writers” (p. 287). But Zak might be disappointed to see their Markowitz (1952) is the only one Kahneman names missing entirely. Ray Dolan, Read and Tversky cited in their seminal 1979 Montague, and Sam McClure are cited paper, and they noted other empirical stud- once each. Among past presidents of the ies showing reflection of gain-loss risk atti- Society for Neuroeconomics (excluding tudes, but not from neoclassicals. Tversky Paul Glimcher, his former postdoc Michael and Kahneman (1981) further clarify that Platt, and Manhattanites), there are no their inspiration came largely from figure- citations at all of work by Peter Bossaerts, ground perceptual switches familiar from Hauke Heekeren, and Scott Huettel, two visual cognitive science, not from a small for myself, and five for Antonio Rangel. branch of neoclassical thinking. Meanwhile, Oskar Morgenstern is cited six Glimcher also “hastens to point out that times, and neuroskeptic Faruk Gul is cited many features of the ‘Asian disease problem’ nine times. [which demonstrates reflection effects] have There is a lot of Nobel-associated name limited its impact in economic circles,” alleg- checking. Readers are pointed toward edly because it is artificial (“make-believe”), Nobel Laureate Selten’s (1975) paper on low-stakes, possibly conveys information, trembling-hand perfection for an “excellent and is perhaps confusing. All these nuisance description” of random utility stochasticity critiques have been ruled out many times as a series of choice errors. (Selten’s main early (and late, too) in the history of behav- idea, actually, was to use the device of (van- ioral economics experiments. Furthermore, ishing) errors to allow Bayesian updating Laibson and Zeckhauser (1998, table 2) show at non-equilibrium nodes, not especially to from citations that the framing examples introduce stochastic choice or even to model in their 1981 paper, and later papers using choice error in a biologically plausible way.) the same ideas, actually did have substantial We are also told that Nobel Laureate Dan impact in top economics journals. McFadden (1974) developed probabilistic A description of how endowment affects choice modeling after being “struck by this how experiments are run is a pastiche of logic, and how it differed from signal detec- the methods that have actually been used tion theory.” Actually, he cites Thurstone (pp. 105–06), like a Hollywood screen writ- (1927) on psychophysical error and has er’s “composite character.” We are also told nothing to say about signal detection (see (p. 384) that Hertwig et al. (2004) estimated McFadden 2001). a probability weighting function. They did Some of the history and descriptions of not. behavioral economics developments are just The sourcing is missing or sketchy in wrong or curiously reconstructed. Glimcher important passages about how psychology traces a path from responding to the (Nobel and economic theory have interacted his- Laureate) Allais paradox by abandoning the torically. Incomplete sourcing can especially mathematical-axiomatic approach in favor distract people in a rapidly-emerging inter- of the “contemporary ‘heuristics and biases’ disciplinary field for two reasons: fevered approach” (e.g., Tversky and Kahneman criticism may be quickly refuted (or may 1974; Kahneman, Slovic, and Tversky 1982). have been refuted already, but is unknown 1178 Journal of Economic Literature, Vol. LI (December 2013) to critics); and paying attention to the most (13): R549–52. shrill critics can lead to tyranny of a vocal Aumann, Robert, and Adam Brandenburger. 1995. “Epistemic Conditions for Nash Equilibrium.” minority. Here are two examples, of the sort Econometrica 63 (5): 1161–80. that a New Yorker-quality fact checker would Becker, Gary S., and Kevin M. Murphy. 1988. “A The- have noted: ory of Rational Addiction.” Journal of Political Econ- omy 96 (4): 675–700. • “…before Allais’ paradox was discovered, Bernheim, B. Douglas. 2009. “On the Potential of Neu- there was an almost universal convic- roeconomics: A Critical (but Hopeful) Appraisal.” tion that axioms rooted in expected util- American Economic Journal: Microeconomics 1 (2): 1–41. ity theory would successfully describe Bernheim, B. Douglas, and Antonio Rangel. 1994. human choice behavior.” “Addiction and Cue-Triggered Decision Processes.” • “…a group of psychologists and neurobi- American Economic Review 94 (5): 1558–90. Berns, Gregory, Jonathan Chappelow, Milos Cekic, ologists have argued that even the phrase Caroline F. Zink, Giuseppe Pagnoni, and Megan E. ‘neuroeconomics’ is distasteful because Martin-Skurski. 2006. “Neurobiological Substrates of of its obvious ties to what Carlyle (1849) Dread.” Science 312 (5774): 754-58. Berridge, Kent C., and John P. O’Doherty. 2014. “From called ‘the dismal science.’ ” Expected Utility to Decision Utility.” In Neuroeco- nomics: Decision Making and the Brain, Second edi- 6. Conclusion tion, edited by Paul W. Glimcher and Ernst Fehr, 335–54. London and San Diego: Elsevier: Academic Paul Glimcher’s book makes a strong, Press. coherent, empirical argument for the poten- Bhatt, Meghana A., Terry Lohrenz, Colin F. Camerer, tial of neural measures of subjective value and P. Read Montague. 2010. “Neural Signatures of Strategic Types in a Two-Person Bargaining Game.” to match up to, and potentially inform, the Proceedings of the National Academy of Sciences 107 simplest economic concepts and questions. (46): 19720–25. His approach is patient and conservative, in Bhatt, Meghana A., Terry Lohrenz, Colin F. Camerer, and P. Read Montague. Forthcoming. “Distinct Con- the sense that he adheres to a style—rooted tributions of the Amygdala and Parahippocampal in rational choice, moving cautiously, and Gyrus to Suspicion in a Repeated Bargaining Game.” endorsing axiomatic systems as a method to Proceedings of the National Academy of Sciences. Bickart, Kevin C., Christopher I. Wright, Rebecca J. efficiently test families of potential behav- Dautoff, Bradford C. Dickerson, and Lisa Feldman ioral functional forms—which economists Barrett. 2011. “Amygdala Volume and Social Net- will be most comfortable with. I prefer to work Size in Humans.” Nature Neuroscience 14 (2): 163–64. rush ahead using what is known from a few Bruni, Luigino, and Robert Sugden. 2007. “The Road decades of behavioral economics to begin Not Taken: How Psychology Was Removed from looking for neural circuits that adjudicate Economics, and How It Might Be Brought Back.” Economic Journal 117 (516): 146–73. between rational and behavioral models Camerer, Colin F., and Teck-Hua Ho. 1999. “Experi- (often alternative models, which prolifer- ence-Weighted Attraction Learning in Normal Form ate quickly since economists are so good at Games.” Econometrica 67 (4): 827–74. Camerer, Colin F., Teck-Hua Ho, and Juin-Kuan theorizing from sparse data) or which break Chong. 2004. “A Cognitive Hierarchy Model of new ground. Happily, these slow and fast Games.” Quarterly Journal of Economics 119 (3): approaches can both be pursued in parallel, 861–98. Camerer, Colin F., and Alec Smith. 2012. “Cogni- and they are. tive Hierarchies and Emotions in Behavioral Game Theory.” In Oxford Handbook of Thinking and Rea- References soning, edited by Keith J. Holyoak and Robert G. Morrison, 346–63. Oxford and New York: Oxford Abler, Birgit, Henrik Walter, Susanne Erk, Hannes University Press. Kammerer, and Manfred Spitzer. 2006. “Prediction Camille, Nathalie, Cathryn A. Griffiths, Khoi Vo, Les- Error as a Linear Function of Reward Probability is ley K. Fellows, and Joseph W. Kable. 2011. “Ven- Coded in Human Nucleus Accumbens.” NeuroIm- tromedial Frontal Lobe Damage Disrupts Value age 31 (2): 790–95. Maximization in Humans.” Journal of Neuroscience Adolphs, Ralph. 2010. “Emotion.” Current Biology 20 31 (20): 7527–32. Camerer: A Review Essay about Foundations of Neuroeconomic Analysis 1179

Caplin, Andrew, and Mark Dean. 2008. “Axiomatic Decision Making, edited by Philip H. Crowley and Neuroeconomics.” In Neuroeconomics: Decision Thomas R. Zentall, 163–203. Oxford and New York: Making and the Brain, edited by Paul W. Glimcher, Oxford University Press. Colin F. Camerer, Ernst Fehr, and Russell A. Pol- Delgado, Mauricio R., Andrew Schotter, Erkut Y. drack, 21–32. London and San Diego: Elsevier, Aca- Ozbay, and Elizabeth A. Phelps. 2008. “Understand- demic Press. ing Overbidding: Using the Neural Circuitry of Caplin, Andrew, Mark Dean, Paul W. Glimcher, and Reward to Design Economic Auctions.” Science 321 Robb B. Rutledge. 2010. “Measuring Beliefs and (5897): 1849–52. Rewards: A Neuroeconomic Approach.” Quarterly De Martino, Benedetto, Colin F. Camerer, and Ralph Journal of Economics 125 (3): 923–60. Adolphs. 2010. “Amygdala Damage Eliminates Mon- Caplin, Andrew, Mark Dean, and Daniel Martin. 2011. etary Loss Aversion.” Proceedings of the National “Search and Satisficing.” American Economic Review Academy of Sciences 107 (8): 3788–92. 101 (7): 2899–2922. De Martino, Benedetto, Dharshan Kumaran, Ben Sey- Caplin, Andrew, and John Leahy. 2001. “Psychological mour, and Raymond J. Dolan. 2006. “Frames, Biases, Expected Utility Theory and Anticipatory Feelings.” and Rational Decision-Making in the Human Brain.” Quarterly Journal of Economics 116 (1): 55–79. Science 313 (5787): 684–87. Carandini, Matteo, David J. Heeger, and J. Anthony De Martino, Benedetto, John P. O’Doherty, Debajyoti Movshon. 1997. “Linearity and Normalization in Ray, Peter Bossaerts, and Colin F. Camerer. 2013. Simple Cells of the Macaque Primary Visual Cortex.” “In the Mind of the Market: Theory of Mind Biases Journal of Neuroscience 17 (21): 8621–44. Value Computation during Financial Bubbles.” Neu- Carlyle, Thomas. 1849. “Occasional Discourse on the ron 79 (6): 1222–31. Negro Question.” Fraser’s Magazine for Town and Dufwenberg, Martin. 2008. “Psychological Games.” In Country 40. The New Palgrave Dictionary of Economics, Second Carter, R. McKell, Justin R. Meyer, and Scott A. edition, edited by Steven N. Durlauf and Lawrence Huettel. 2010. “Functional Neuroimaging of Inter- E. Blume, 714–18. New York: Palgrave Macmillan. temporal Choice Models: A Review.” Journal of Neu- Dunbar, Robin I. M. 1992. “Neocortex Size as a Con- roscience, Psychology, and Economics 3 (1): 27–45. straint on Group Size in Primates.” Journal of Human Casey, B. J., Sarah Getz, and Adriana Galvan. 2008. Evolution 22 (6): 469–93. “The Adolescent Brain.” Developmental Review 28 Dunbar, Robin I. M. 2012. “The Social Brain Meets (1): 62–77. Neuroimaging.” Trends in Cognitive Sciences 16 (2): Chen, M. Keith, Venkat Lakshminarayanan, and Laurie 101–02. R. Santos. 2006. “How Basic Are Behavioral Biases? Einhorn, Hillel J., and Robin M. Hogarth. 1978. “Con- Evidence from Capuchin Monkey Trading Behav- fidence in Judgment: Persistence of the Illusion of ior.” Journal of Political Economy 114 (3): 517–37. Validity.” Psychological Review 85 (5): 395–416. Chib, Vikram S., Benedetto De Martino, Shinsuke Shi- Ellsberg, Daniel. 1961. “Risk, Ambiguity, and the Sav- mojo, and John P. O’Doherty. 2012. “Neural Mech- age Axioms.” Quarterly Journal of Economics 75 (4): anisms Underlying Paradoxical Performance for 643–69. Monetary Incentives Are Driven By Loss Aversion.” Falk, Emily B., Elliot T. Berkman, and Matthew D. Neuron 74 (3): 582–94. Lieberman. 2012. “From Neural Responses to Popu- Colander, David. 2007. “Edgeworth’s Hedonimeter lation Behavior: Neural Focus Group Predicts Pop- and the Quest to Measure Utility: Retrospectives.” ulation-Level Media Effects.” Psychological Science Journal of Economic Perspectives 21 (2): 215–25. 23 (5): 439–45. Coricelli, Giorgio, and Rosemarie Nagel. 2009. “Neural Falk, Emily B., Elliot T. Berkman, Danielle Whalen, Correlates of Depth of Strategic Reasoning in Medial and Matthew D. Lieberman. 2011. “Neural Activ- Prefrontal Cortex.” Proceedings of the National ity during Health Messaging Predicts Reductions in Academy of Sciences 106 (23): 9163–68. Smoking Above and Beyond Self-Report.” Health Costa, Dora L., and Matthew E. Kahn. 2007. “Desert- Psychology 30 (2): 177–85. ers, Social Norms, and Migration.” Journal of Law Falk, Emily B. Sylvia A. Morelli, B. Locke Welborn, and Economics 50 (2): 323–53. Karl Dambacher, and Matthew D. Lieberman. 2013. Crawford, Vincent P., Miguel A. Costa-Gomes, and “Creating Buzz: The Neural Correlates of Effective Nagore Iriberri. 2013. “Structural Models of Non- Message Propagation.” Psychological Science 24 (7): equilibrium Strategic Thinking: Theory, Evidence, 1234–42. and Applications.” Journal of Economic Literature Fama, Eugene F. 1965. “The Behavior of Stock-Market 51 (1): 5–62. Prices.” Journal of Business 38: 34–105. D’Ardenne, Kimberlee, Samuel M. McClure, Leigh Farroni, Teresa, Gergely Csibra, Francesca Simion, E. Nystrom, and Jonathan D. Cohen. 2008. “BOLD and Mark H. Johnson. 2002. “Eye Contact Detection Responses Reflecting Dopaminergic Signals in in Humans from Birth.” Proceedings of the National the Human Ventral Tegmental Area.” Science 319 Academy of Sciences 99 (14): 9602–05. (5867): 1264–67. Farroni, Teresa, Mark H. Johnson, Enrica Menon, Luisa Dean, Mark. 2013. “What Can Neuroeconomics Tell Us Zulian, Dino Faraguna, and Gergely Csibra. 2005. About Economics (and Vice Versa).” In Comparative­ “Newborns’ Preference for Face-Relevant Stimuli: 1180 Journal of Economic Literature, Vol. LI (December 2013)

Effects of Contrast Polarity.” Proceedings of the Hertwig, Ralph, Greg Barron, Elke U. Weber, and Ido National Academy of Sciences 102 (47): 17245–250. Erev. 2004. “Decisions from Experience and the Fehr, Ernst, and Colin F. Camerer. 2007. “Social Neu- Effect of Rare Events in Risky Choice.” Psychologi- roeconomics: The Neural Circuitry of Social Prefer- cal Science 15 (8): 534–39. ences.” Trends in Cognitive Sciences 11 (10): 419–27. Hertwig, Ralph, Greg Barron, Elke U. Weber, and Ido Figner, Bernd, et al. 2010. “Lateral Prefrontal Cortex Erev. 2005. “The Role of Information Sampling in and Self-Control in Intertemporal Choice.” Nature Risky Choice.” In Information Sampling and Adap- Neuroscience 13 (5): 538–39. tive Cognition, edited by Klaus Fiedler and Peter Fox, Craig R., and Liat Hadar. 2006. “‘Decisions from Juslin, 72–91. Cambridge and New York: Cambridge Experience’ = Sampling Error + Prospect Theory: University Press. Reconsidering Hertwig, Barron, Weber & Erev Hsu, Ming, Meghana Bhatt, Ralph Adolphs, Daniel (2004).” Judgment and Decision Making 1 (2): Tranel, and Colin F. Camerer. 2005. “Neural Systems 159–61. Responding to Degrees of Uncertainty in Human Fudenberg, Drew, and David K. Levine. 2006. “A Decision-Making.” Science 310 (5754): 1680–83. Dual-Self Model of Impulse Control.” American Hsu, Ming, Ian Krajbich, Chen Zhao, and Colin F. Economic Review 96 (5): 1449–76. Camerer. 2009. “Neural Response to Reward Antici- Gilboa, Itzhak, Andrew Postlewaite, Larry Samuel- pation under Risk Is Nonlinear in Probabilities.” son, and David Schmeidler. 2013. “Economic Mod- Journal of Neuroscience 29 (7): 2231–37. els as Analogies.” http://itzhakgilboa.weebly.com/ Huber, Joel, John W. Payne, and Christopher Puto. uploads/8/3/6/3/8363317/gpss_economic_models_ 1982. “Adding Asymmetrically Dominated Alter- as_analogies.pdf. natives: Violations of Regularity and the Similarity Glimcher, Paul W., and Aldo Rustichini. 2004. “Neuro- Hypothesis.” Journal of Consumer Research 9 (1): economics: The Consilience of Brain and Decision.” 90–98. Science 306 (5695): 447–52. Huettel, Scott A. 2010. “Ten Challenges for Decision Gold, Joshua I., and Michael N. Shadlen. 2007. “The Neuroscience.” Frontiers in Neuroscience 4: 171. Neural Basis of Decision Making.” Annual Review of Izuma, Keise, Kenji Matsumoto, Colin F. Camerer, and Neuroscience 30: 535–74. Ralph Adolphs. 2011. “Insensitivity to Social Reputa- Grossman, Sanford J., and Joseph E. Stiglitz. 1980. “On tion in Autism.” Proceedings of the National Acad- the Impossibility of Informationally Efficient Mar- emy of Sciences 108 (42): 17302–07. kets.” American Economic Review 70 (3): 393–408. Izuma, Keise, Daisuke N. Saito, and Norihiro Sadato. Gul, Faruk, and Wolfgang Pesendorfer. 2008. “The 2010. “Processing of the Incentive for Social Case for Mindless Economics.” In The Foundations Approval in the Ventral Striatum during the Chari- of Positive and Normative Economics: A Handbook, table Donation.” Journal of Cognitive Neuroscience edited by Andrew Caplin and Andrew Schotter, 3–42. 22 (4): 621–31. Oxford and New York: Oxford University Press. Jessup, Ryan K., Anthony J. Bishara, and Jerome R. Hampton, Alan N., Peter Bossaerts, and John P. Busemeyer. 2008. “Feedback Produces Divergence O’Doherty. 2008. “Neural Correlates of Mentalizing- from Prospect Theory in Descriptive Choice.” Psy- Related Computations during Strategic Interactions chological Science 19 (10): 1015–22. in Humans.” Proceedings of the National Academy of Kable, Joseph W., and Paul W. Glimcher. 2007. “The Sciences 105 (18): 6741–46. Neural Correlates of Subjective Value during Inter- Harbaugh, William T., Ulrich Mayr, and Daniel R. temporal Choice.” Nature Neuroscience 10 (12): Burghart. 2007. “Neural Responses to Taxation and 1625–33. Voluntary Giving Reveal Motives for Charitable Kahneman, Daniel. 1992. “Reference Points, Anchors, Donations.” Science 316 (5831): 1622–25. Norms, and Mixed Feelings.” Organizational Behav- Hare, Todd A., Colin F. Camerer, and Antonio Rangel. ior and Human Decision Processes 51 (2): 296–312. 2009. “Self-Control in Decision-Making Involves Kahneman, Daniel, Paul Slovic, and Amos Tversky, Modulation of the vmPFC Valuation System.” Sci- eds. 1982. Judgment under Uncertainty: Heuristics ence 324 (5927): 646–48. and Biases. Cambridge and New York: Cambridge Hare, Todd A., John O’Doherty, Colin F. Camerer, University Press. Wolfram Schultz, and Antonio Rangel. 2008. “Dis- Kahneman, Daniel, and Amos Tversky. 1982. “On the sociating the Role of the Orbitofrontal Cortex and Study of Statistical Intuitions.” In Judgment under the Striatum in the Computation of Goal Values and Uncertainty: Heuristics and Biases, edited by Daniel Prediction Errors.” Journal of Neuroscience 28 (22): Kahneman, Paul Slovic, and Amos Tversky, 493–508. 5623–30. Cambridge and New York: Cambridge University Heeger, David J. 1992. “Normalization of Cell Press. Responses in Cat Striate Cortex.” Visual Neurosci- Kamenica, Emir. 2008. “Contextual Inference in ence 9 (2): 181–97. Markets: On the Informational Content of Product Hershfield, Hal E., et al. 2011. “Increasing Saving Lines.” American Economic Review 98 (5): 2127–49. Behavior through Age-Progressed Renderings of the Kang, Min Jeong, and Colin F. Camerer. 2013. “fMRI Future Self.” Journal of Marketing Research (Special Evidence of a Hot–Cold Empathy Gap in Hypotheti- Issue): S23–37. cal and Real Aversive Choices.” Frontiers in Decision Camerer: A Review Essay about Foundations of Neuroeconomic Analysis 1181

Neuroscience 7: 104. ­Evidence of Incentive Bias for Immediate Rewards Kang, Min Jeong, Antonio Rangel, Mickael Camus, Relative to Preference-Matched Delayed Rewards.” and Colin F. Camerer. 2011. “Hypothetical and Real Journal of Neuroscience 29 (47): 14820–27. Choice Differentially Activate Common Valuation Markowitz, H. 1952. “The Utility of Wealth.” Journal of Areas.” Journal of Neuroscience 31 (2): 461–68. Political Economy 60 (2): 151–58. Kishida, Kenneth T., Brooks King-Casas, and P. Read Markowitz, Harry M. 1991. “Foundations of Portfolio Montague. 2010. “Neuroeconomic Approaches to Theory.” Journal of Finance 46 (2): 469–77. Mental Disorders.” Neuron 67 (4): 543–54. McClure, Samuel M., Keith M. Ericson, David I. Laib- Koszegi, Botond, and Matthew Rabin. 2006. “A Model son, George Loewenstein, and Jonathan D. Cohen. of Reference-Dependent Preferences.” Quarterly 2007. “Time Discounting for Primary Rewards.” Journal of Economics 121 (4): 1133–65. Journal of Neuroscience 27 (21): 5796–804. Kreps, David M., and Evan L. Porteus. 1978. “Tempo- McClure, Samuel M., David I. Laibson, George Loew- ral Resolution of Uncertainty and Dynamic Choice enstein, and Jonathan D. Cohen. 2004. “Separate Theory.” Econometrica 46 (1): 185–200. Neural Systems Value Immediate and Delayed Mon- Laibson, David. 1997. “Golden Eggs and Hyperbolic etary Rewards.” Science 306 (5695): 503–07. Discounting.” Quarterly Journal of Economics 112 McFadden, Daniel. 1974. “Conditional Logit Analy- (2): 443–77. sis of Qualitative Choice Behavior.” In Frontiers in Laibson, David. 2001. “A Cue-Theory of Consump- Econometrics, edited by Paul Zarembka, 105–42. tion.” Quarterly Journal of Economics 116 (1): New York: Academic Press. 81–119. McFadden, Daniel. 2001. “Economic Choices.” Ameri- Laibson, David, and Richard Zeckhauser. 1998. “Amos can Economic Review 91 (3): 351–78. Tversky and the Ascent of Behavioral Economics.” Murayama, Kou, Madoka Matsumoto, Keise Izuma, Journal of Risk and Uncertainty 16 (1): 7–47. Kenji Matsumoto. 2010. “Neural Basis of the Under- Lakshminaryanan, Venkat, M. Keith Chen, and Laurie mining Effect of Monetary Reward on Intrinsic R. Santos. 2008. “Endowment Effect in Capuchin Motivation.” Proceedings of the National Academy of Monkeys.” Philosophical Transactions of the Royal Sciences 107 (49): 20911–16. Society B: Biological Sciences 363 (1511): 3837–44. Padoa-Schioppa, Camillo. 2009. “Range-Adapting Rep- Latty, Tanya, and Madeleine Beekman. 2011. “Irra- resentation of Economic Value in the Orbitofrontal tional Decision-Making in an Amoeboid Organism: Cortex.” Journal of Neuroscience 29 (44): 14004–14. Transitivity and Context-Dependent Preferences.” Padoa-Schioppa, Camillo, and John A. Assad. 2006. Proceedings of the Royal Society B: Biological Sci- “Neurons in the Orbitofrontal Cortex Encode Eco- ences 278 (1703): 307–12. nomic Value.” Nature 441 (7090): 223–26. Levy, Ifat, Jason Snell, Amy J. Nelson, Aldo Rustichini, Pareto, Vilfredo. 1971. Manual of Political Economy. and Paul W. Glimcher. 2010. “Neural Representation New York: Kelley, 1909. of Subjective Value under Risk and Ambiguity.” Jour- Payne, John W., James R. Bettman, and Eric J. Johnson. nal of Neurophysiology 103 (2): 1036–47. 1993. The Adaptive Decision Maker. Cambridge and Lipman, Barton L., and Wolfgang Pesendorfer. 2013. New York: Cambridge University Press. “Temptation.” Peters, Jan, and Christian Büchel. 2010. “Episodic In Advances in Economics and Econometrics: Tenth Future Thinking Reduces Reward Delay Discount- World Congress, Volume 1: Economic Theory, edited ing through an Enhancement of Prefrontal-Medio- by Daron Acemoglu, Manuel Arellano, and Eddie temporal Interactions.” Neuron 66 (1): 138–48. Dekel, 243–88. Cambridge and New York: Cam- Platt, Michael L., and Paul W. Glimcher. 1999. “Neural bridge University Press. Correlates of Decision Variables in Parietal Cortex.” Loewenstein, George. 1987. “Anticipation and the Val- Nature 400 (6741): 233–38. uation of Delayed Consumption.” Economic Journal Rangel, Antonio, Colin F. Camerer, and P. Read Mon- 97 (387): 666–84. tague. 2008. “A Framework for Studying the Neu- Loewenstein, George. 2000. “Emotions in Economic robiology of Value-Based Decision Making.” Nature Theory and Economic Behavior.” American Eco- Reviews Neuroscience 9 (7): 545–56. nomic Review 90 (2): 426–32. Reutskaja, Elena, Axel Lindner, Rosemarie Nagel, Rich- Loewenstein, George, and Ted O’Donoghue. 2007. ard Andersen, and Colin F. Camerer. 2011. “Neural “Animal Spirits: Affective and Deliberative Processes Signatures of Choice Overload.” Unpublished. in Economic Behavior.” Unpublished. Rilling, James K., and Alan G. Sanfey. 2011. “The Loomes, Graham, and Robert Sugden. 1987. “Test- Neuroscience of Social Decision-Making.” Annual ing for Regret and Disappointment in Choice under Review of Psychology 62: 23–48. Uncertainty.” Economic Journal 97: 118–29. Romer, Paul M. 2000. “Thinking and Feeling.” Ameri- Louie, Kenway, Lauren E. Grattan, and Paul W. Glim- can Economic Review 90 (2): 439–43. cher. 2011. “Rewards Value-Based Gain Control: Rosati, Alexandra G., Jeffrey R. Stevens, Brian Hare, Divisive Normalization in Parietal Cortex.” Journal and Marc D. Hauser. 2007. “The Evolutionary Ori- of Neuroscience 31 (29): 10627–39. gins of Human Patience: Temporal Preferences in Luo, Shan, George Ainslie, Lisa Giragosian, and Chimpanzees, Bonobos, and Human Adults.” Cur- John R. Monterosso. 2009. “Behavioral and Neural rent Biology 17 (19): 1663–68. 1182 Journal of Economic Literature, Vol. LI (December 2013)

Rubinstein, Ariel, and Yuval Salant. 2006. “A Model of Significance of Neuroeconomics for Economic The- Choice from Lists.” Theoretical Economics 1 (1): 3–17. ory.” Economics and Philosophy 24 (3): 515–21. Rutledge, Robb B., Mark Dean, Andrew Caplin, and Spiegler, Ran. 2013. “Comments on ‘Behavioral’ Deci- Paul W. Glimcher. 2010. “Testing the Reward Pre- sion Theory.” In Advances in Economics and Econo- diction Error Hypothesis with an Axiomatic Model.” metrics: Tenth World Congress, Volume 1: Economic Journal of Neuroscience 30 (40): 13525–36. Theory, edited by Daron Acemoglu, Manuel Arel- Sallet, J., et al. 2011. “Social Network Size Affects lano, and Eddie Dekel, 289–304. Cambridge and Neural Circuits in Macaques.” Science 334 (6056): New York: Cambridge University Press. 697–700. Sutton, Richard S., and Andrew G. Barto. 1998. Rein- Schultz, Wolfram, Peter Dayan, and P. Read Mon- forcement Learning: An Introduction. Cambridge tague. 1997. “A Neural Substrate of Prediction and and London: MIT Press. Reward.” Science 275 (5306): 1593–99. Thaler, Richard H., and H. M. Shefrin. 1981. “An Eco- Schwartz, Odelia, and Eero P. Simoncelli. 2001. “Nat- nomic Theory of Self-Control.” Journal of Political ural Signal Statistics and Sensory Gain Control.” Economy 89 (2): 392–406. Nature Neuroscience 4 (8): 819–25. Thevarajah, Dhushan, Ryan Webb, Christopher Fer- Sell, Aaron, John Tooby, and Leda Cosmides. 2009. rall, and Michael C. Dorris. 2010. “Modeling the “Formidability and the Logic of Human Anger.” Pro- Value of Strategic Actions in the Superior Collicu- ceedings of the National Academy of Sciences 106 lus.” Frontiers in Behavioral Neuroscience 3: 57. (35): 15073–78. Thurstone, Louis Leon. 1927. “A Law of Comparative Sellitto, Manuela, Elisa Ciaramelli, and Giuseppe di Judgment.” Psychological Review 34 (4): 278–86. Pellegrino. 2010. “Myopic Discounting of Future Tom, Sabrina M., Craig R. Fox, Christopher Trepel, Rewards after Medial Orbitofrontal Damage in and Russell A. Poldrack. 2007. “The Neural Basis of Humans.” Journal of Neuroscience 30 (49): 16429–36. Loss Aversion in Decision-Making under Risk.” Sci- Selten, Reinhard. 1975. “Reexamination of the Perfect- ence 315 (5811): 515–18. ness Concept for Equilibrium Points in Extensive Tricomi, Elizabeth, Bernard W. Balleine, and John P. Games.” International Journal of Game Theory 4 (1): O’Doherty. 2009. “A Specific Role for Posterior Dor- 25–55. solateral Striatum in Human Habit Learning.” Euro- Shafir, Sharoni, Tom A. Waite, and Brian H. Smith. pean Journal of Neuroscience 29 (11): 2225–32. 2002. “Context-Dependent Violations of Rational Tversky, Amos. 1972. “Elimination By Aspects: A The- Choice in Honeybees (Apis mellifera) and Gray Jays ory of Choice.” Psychological Review 79 (4): 281–99. (Perisoreus canadensis).” Behavioral Ecology and Tversky, Amos, and Daniel Kahneman. 1974. “Judg- Sociobiology 51 (2): 180–87. ment under Uncertainty: Heuristics and Biases.” Sci- Smith, Alec, B. Douglas Bernheim, Colin F. Camerer, ence 185 (4157): 1124–31. and Antonio Rangel. Forthcoming. “Neural Activ- Tversky, Amos, and Daniel Kahneman. 1981. “The ity Reveals Preferences without Choices.” American Framing of Decisions and the Psychology of Choice.” Economic Journal: Microeconomics. Science 211 (4481): 453–58. Smith, Alec, Terry Lohrenz, Justin King, P. Read Tversky, Amos, and Shmuel Sattath. 1979. “Preference Montague, and Colin F. Camerer. 2013. “Irratio- Trees.” Psychological Review 86 (6): 542–73. nal Exuberance and Neural Warning Signals dur- Tversky, Amos, and Itamar Simonson. 1993. “Context- ing Endogenous Experimental Market Bubbles.” Dependent Preferences.” Management Science 39 Unpublished. (10): 1179–89. Sokol-Hessner, Peter, Colin F. Camerer, and Elizabeth Valenza, Eloisa, Francesca Simion, Viola Macchi Cas- A. Phelps. 2013. “Emotion Regulation Reduces Loss sia, and Carlo Umiltà. 1996. “Face Preference at Aversion and Decreases Amygdala Responses to Birth.” Journal of Experimental Psychology: Human Losses.” Social Cognitive and Affective Neuroscience Perception and Performance 22 (4): 892–903. 8 (3): 341–50. Yacubian, Juliana, Jan Gläscher, Katrin Schroeder, Sokol-Hessner, Peter, Ming Hsu, Nina G. Curley, Mau- Tobias Sommer, Dieter F. Braus, and Christian ricio R. Delgado, Colin F. Camerer, and Elizabeth Büchel. 2006. “Dissociable Systems for Gain- and A. Phelps. 2009. “Thinking Like a Trader Selectively Loss-Related Value Predictions and Errors of Predic- Reduces Individuals’ Loss Aversion.” Proceedings of tion in the Human Brain.” Journal of Neuroscience the National Academy of Sciences 106 (13): 5035–40. 26 (37): 9530–37. Soltani, Alireza, Benedetto De Martino, and Colin F. Zhu, Lusha, Kyle E. Mathewson, and Ming Hsu. 2012. Camerer. 2012. “A Range-Normalization Model of “Dissociable Neural Representations of Reinforce- Context-Dependent Choice: A New Model and Evi- ment and Belief Prediction Errors Underlie Strategic dence.” PloS Computational Biology 8 (7). Learning.” Proceedings of the National Academy of Spiegler, Ran. 2008. “Comments on the Potential Sciences 109 (5): 1419–24.