sign in sign up
<<
Home , Behavioral economics, Cooperation, Neuroeconomics, Paul Glimcher, Prospect theory, Revealed preference

CHAPTER 1

Introduction: A Brief History of Paul W. Glimcher , Colin F. Camerer , , and Russell A. Poldrack

OUTLINE

Neoclassical 1 Two Trends, One Goal 7 Cognitive 5 Summary 11 Setting the Stage for Neuroeconomics 6 References 11

Over the first decade of its existence, neuroeco- nomics has engendered raucous debates of two kinds. First, scholars within each of its parent disciplines have argued over whether this synthetic field offers The birth of economics is often traced to Adam benefits to their particular parent discipline. Second, Smith’s publication of The Wealth of Nations in 1776. scholars within the emerging field itself have argued With this publication began the classical period of eco- over what form neuroeconomics should take. To nomic theory. Smith described a number of phenom- understand these debates, however, a reader must ena critical for understanding choice and understand both the intellectual sources of neuroeco- the aggregation of choices into activity. These nomics and the backgrounds and methods of practic- were, in essence, psychological insights. They were ing neuroeconomists. relatively ad hoc rules that explained how features of Neuroeconomics has its origins in two places; in the environment influenced the behavior of a nation events following the neoclassical economic revolution of consumers and producers. of the 1930s, and in the birth of What followed the classical period was an interval during the 1990s. We therefore begin this brief history during which economic theory became very heterog- with a review of the neoclassical revolution and the enous. A number of competing schools with different birth of cognitive neuroscience. approaches developed. Many of the time

Neuroeconomics: Decision Making and the 1 © 2009, Elsevier Inc. 2 1. INTRODUCTION: A BRIEF HISTORY OF NEUROECONOMICS

(Edgeworth, Ramsey, Fisher) dreamed about tools to about the relative desirability of pairs of objects that infer from physical signals, through a “ hedon- have never been directly compared by the consumer. imeter” for example, but these early neuroeconomists Consider a situation in which a consumer chooses an did not have such tools (Colander, 2008). apple over an orange and then an orange over a peach . One school of thought, due to John Maynard If the assumption of GARP is correct, then this con- Keynes, was that regularities in consumer behavior sumer must not choose a peach over an apple even if could (among other things) provide a basis for fis- this is a behavior we have never observed before. cal policy to manage economic fluctuations. Many The revealed approach thus starts from elements in Keynes’ theory, such as the “ propensity a set of assumptions called axioms which encapsu- to consume” or entrepreneurs ’ “ spirits” that late a theory of some kind (often a very limited one) influence their decisions, were based on in formal language. The theory tells us what a series of psychological concepts. This framework dominated observed choices implies about intermediate variables ’ until the 1960s. such as (and, in more developed versions of Beginning in the 1930s, a group of economists – most the theory, subjective beliefs about random events). The famously, Samuelson, Arrow, and Debreu – began to poetry in the approach (what distinguishes a beautiful investigate the mathematical structure of consumer theory from an ugly one) is embodied in the simplic- choice and behavior in markets (see, for example, ity of the axioms, and the degree to which surprisingly Samuelson, 1938). Rather than simply building mod- simple axioms make sharp predictions about what kind els that incorporated a set of parameters that might, on of choice patterns should and should not be observed. a priori psychological grounds, be predictive of choice Finally, it is critical to note that what the theory pre- behavior, this group of theorists began to investigate dicts is which new choices could possibly follow from what mathematical structure of choices might result an observed set of previous choices (including choices from simple, more “ primitive, ” assumptions on prefer- that respond to policy and other changes in the envi- ences. Many of these models (and the style of modeling ronment, such as responses to changes in , taxes, that followed) had a strong normative flavor, in the or incomes). The theories do not predict intermediate sense that attention was most immediately focused on variables; they use them as tools. What revealed pref- idealized choices and efficient allocation of resources; as erence theories predict is choice. It is the only goal, the opposed to necessarily seeking to describe how people only reason for being, for these theories. choose (as psychologists do) and how markets work. What followed the development of WARP were To better understand this approach, consider what is a series of additional theorems of this type which probably the first and most important of these simple extended the scope of revealed-preference theory to models: the Weak Axiom of (WARP). choices with uncertain outcomes whose likelihoods are WARP was developed in the 1930s by , known (von Neumann and Morgenstern’s expected who founded the revealed preference approach that theory, EU) or subjective (or “ personal, ” in was the heart of the neoclassical revolution. Samuelson Savage’s subjective EU theory), and in which out- proposed that if a consumer making a choice between comes may be spread over time ( an apple and an orange selects an apple, he reveals a theory) (see Chapter 3 for more details). What is most preference for apples. If we assume only that this means interesting about these theories is that they demon- he prefers (preference is here a stable internal prop- strate, amongst other things, that a chooser who obeys erty that economists did not hope to measure directly) these axioms must behave both “ as if ” he has a contin- apples to oranges, what can we say about his future uous utility function that relates the subjective value behavior? Can we say anything at all? of any gain to its objective value and “as if” his actions What Samuelson and later authors showed math- were aimed at maximizing total obtained utility. In ematically was that even simple assumptions about their seminal book von Neumann and Morgenstern binary choices, revealing stable (weak) preferences, also laid the foundations for much of , could have powerful implications. An extension of the which they saw as a special problem in utility theory, WARP axiom called GARP (the “ generalized ” axiom in which outcomes are generated by the choices of of revealed preference, Houthakker, 1950) posits many players (von Neumann and Morgenstern, 1944). that if apples are revealed preferred to oranges, and At the end of this period, neoclassical economics oranges are revealed preferred to peaches, then apples seemed incredibly powerful. Starting with as few as are “indirectly” revealed preferred to peaches (and one and as many as four simple assumptions which similarly for longer chains of indirect revelation). If fully described a new theory the neoclassicists devel- GARP holds for binary choices among pairs of objects, oped a framework for thinking about and predict- then some choices can be used to make predictions ing choice. These theories of would

INTRODUCTION: A BRIEF HISTORY OF NEUROECONOMICS NEOCLASSICAL ECONOMICS 3 later form the basis for the part of the Arrow- “ ambiguity ” (Ellsberg’s term) or “ weight of evidence ” Debreu theory of competitive “ general ” equilibrium, (Keynes ’ term) supporting a judgment of event likeli- a system in which prices and quantities of all hood could influence choices, violating one of Savage’s were determined simultaneously by matching sup- key axioms. The Allais and Ellsberg paradoxes raised ply and demand. This is an important tool because it the possibility that the specific functional forms of EU enables the modeler to anticipate all consequences of and subjective EU implied by simple axioms of pref- a policy change – for example, imposing a luxury tax erence were generally wrong. More importantly, the on yachts might increase crime in a shipbuilding town paradoxes invited mathematical exploration (which because of a rise in there. This sort of only came to fruition in the 1980s) about how weaker analysis is unique to economics, and partly explains systems of axioms might generalize EU and SEU. the broad influence of economics in regulation and The goal of these new theories was to accommodate policy-making. the paradoxical behavior in a way that is both psy- It cannot be emphasized enough how much the chologically plausible and formally sharp (i.e., which revealed-preference view suppressed in the does not predict that any pattern of choices is possible, psychological of preference, because clever and could therefore conceivably be falsified by new axiomatic systems could be used to infer properties paradoxes). of unobservable preference from observable choice One immediate response to this set of observations ( Bruni and Sugden, 2007). Before the neoclassical rev- was to argue that the neoclassical models worked, olution, Pareto noted in 1897 that but only under some limited circumstances – a fact It is an empirical fact that the natural have pro- which many of the neoclassicists were happy to con- gressed only when they have taken secondary principles as cede (for example, Morgenstern said “ the their point of departure, instead of trying to discover the used must be within certain plausible ranges and not essence of things. … Pure political has therefore a go to .01 or even less to .001” ). Surely axioms might great interest in relying as little as possible on the domain of also be violated if the details of the options being . analyzed were too complicated for the chooser to (Quoted in Busino, 1964 : xxiv) understand, or if the chooser was overwhelmed with Later, in the 1950s, wrote an influ- too many choices. Observed violations could then be ential book, The of . seen as a way to map out boundary conditions – a Friedman argued that assumptions underlying a pre- specification of the kinds of problems that lay outside diction about market behavior could be wrong, but the limits of the neoclassical framework’s range of the prediction could be approximately true. For exam- applicability. ple, even if a monopolist seller does not sit down with Another approach was Herbert Simon’s sugges- a piece of paper and figure out what maximizes tion that is computationally bounded, and total , prices might evolve “ as if” such that much could be learned by understanding “ proce- a calculation has been made (perhaps due to selection dural rationality.” As a major contributor to cognitive pressures within or between firms). Friedman’s argu- , Simon clearly had in mind theories of choice ment licensed economists to ignore evidence of when which posited particular procedures, and suggested economic agents violate rational-choice principles that the way forward was to understand choice pro- (evidence that typically comes from that cedures empirically, perhaps in the form of algorithms test the individual choice principles most clearly), a (of which “ always choose the object with the highest prejudice that is still widespread in economics. utility ” is one extreme and computationally demand- What happened next is critical for understand- ing procedure). ing where neuroeconomics arose. In 1953, the French A sweeping and constructive view emerged from designed a series of pair- the work of and wise choices which led to reliable patterns of revealed (1979) in the late 1970s and 1980s, and other psycholo- preference that violated the central “ independence ” gists interested in judgment and decision making axiom of expected utility theory. Allais unveiled his whose intersected with choice theory. What pattern, later called the “,” at a confer- Kahneman, Tversky, and others showed in a series of ence in France at which many participants, includ- remarkable experimental examples was that the range ing Savage, made choices which violated their own of phenomena that fell outside classical expected util- theories during an informal lunch. (Savage allegedly ity theory was even broader than Allais ’ and Ellsberg’s blamed the lunchtime wine.) examples had suggested. A few years after Allais ’ example, Daniel Ellsberg These psychologists studying the foundations (1961) presented a famous paradox suggesting that the of economic choice found many common choice

INTRODUCTION: A BRIEF HISTORY OF NEUROECONOMICS 4 1. INTRODUCTION: A BRIEF HISTORY OF NEUROECONOMICS – typically easily replicated in experiments – guided by the idea that normative theory is a privi- that falsified one or more of the axioms of expected leged starting point. The difference in approaches utility theory and which seemed to conflict with fun- spilled across methodological boundaries too. The damental axioms of choice. For example, some of influence of ideas from roughly their experimental demonstrations showed effects of coincided with a rise in interest among economists “ framing, ” attacking the implicit axiom of “ descrip- such as Charles Plott, Vernon Smith and colleagues in tion invariance ” – the idea that choices among objects conducting carefully controlled experiments on eco- should not depend on how they are described. nomics systems (see, for example, Smith, 1976 ). The These experiments thus led many scholars, particu- experimental economists began with the viewpoint that larly psychologists and economists who had become economic principles should apply everywhere (as interested in decision making through the work of principles in natural and physical sciences are pre- Kahneman and Tversky, to conclude that empirical sumed to); their view was that when theories fail in critiques of the simple axiomatic approaches, in the simple environments, those failures raise doubt about form of counterexamples, could lead to more general whether they are likely to work in more complex envi- axiomatic systems that were more sensibly rooted in ronments. However, the overlap between behavioral principles of psychology. economics and is far from This group of psychologists and economists, who complete. Behavioral economics is based on the pre- began to call themselves behavioral economists , argued sumption that incorporating psychological principles that evidence and ideas from psychology could will improve economic analysis, while experimental improve the model of human behavior inherited economics presumes that incorporating psychological from neoclassical economics. In one useful definition, methods (highly controlled experiments) will improve behavioral economics proposes models of limits on the testing of economic theory. rational calculation, willpower, and self-interest, and In any case, the neoclassical school had a clear the- seeks to codify those limits formally and explore their ory and sharp predictions, but the behavioral econo- empirical implications using mathematical theory, mists continued to falsify elements of that theory with experimental data, and analysis of field data. compelling empirical examples. Neuroeconomics In the realm of risky choice, Kahneman and Tversky emerged from within behavioral and experimental eco- modified expected utility to incorporate a psycho- nomics because behavioral economists often proposed physical idea of reference-dependence – valuation theories that could be thought of as algorithms regard- of outcomes depends on a point of reference, just as ing how information was processed, and the choices sensations of heat depend on previous temperature – that resulted from that information-processing. A nat- along with a regressive non-linear transformation of ural step in testing these theories was simultaneously objective . (Details of are to gather information on the details of both informa- reviewed in Chapter 11.) Another component of the tion processing and associated choices. If information behavioral program was the idea that statistical intui- processing could be hypothesized in terms of neural tions might be guided by , which could be activity, then neural measures could be used (along inferred empirically by observing choice under a broad with coarser measures like eyetracking of information range of circumstances. Heuristics were believed to that choosers attend to) to test theories as simultane- provide a potential basis for a future theory of choice ous restrictions on what information is processed, how ( Gilovich et al. , 2002 ). A third direction is theories of that processing works in the brain, and the choices that social preference – how people value choices when result. Neuroscientific tools provide further predictions those choices impact the values of other people (see in tests with lesion-patient behavior, and transcranial Chapter 15). The goal is eventually to have mathemat- magnetic stimulation (TMS) which should (in theory) ical systems that embody choice heuristics and spe- change choices if TMS disrupts an area that is neces- cific types of social preference which explain empirical sary to producing certain kinds of choices. An impor- facts but also make sharp predictions. Development tant backdrop to this development is that economic of these theories, and tests with both experimen- theorists are extremely clever at inventing multiple tal and field data, are now the frontiers of modern systems of axioms which can explain the same pat- behavioral economics. terns of choices. By definition, choices alone provide a An obvious conflict developed (and continues to limited way to distinguish theories in the face of rapid cause healthy debate) between the behavioral econo- production of alternative theories. Forcing theories to mists, who were attempting to piece together empiri- commit to predictions about underlying neural activ- cally disciplined theories, and the neoclassicists, who ity therefore provides a powerful way to adjudicate were arguing for a simpler global theory, typically among theories.

INTRODUCTION: A BRIEF HISTORY OF NEUROECONOMICS COGNITIVE NEUROSCIENCE 5

COGNITIVE NEUROSCIENCE of these two approaches. Within the domain of neu- rology, models from psychology began to be used to understand the relationship between brain and behav- Like economics, the history of the neuroscientific ior. Although the classes of models that were explored study of behavior also reflects an interaction between were highly heterogeneous and often not very quan- two approaches – in this case, a neurological approach titative, these early steps made it possible to study and a physiological approach. In the standard neuro- mental state, at least in a limited way. Within the phys- logical approach of the last century, human patients or iological tradition, technical advances that led to the experimental with brain lesions were studied development of humane methods made it possible to in a range of behavioral tasks. The behavioral deficits make measurements in awake, behaving animals, also of the subjects were then correlated with their neuro- opening the way to the study of mental state, this time logical injuries and the correlation used to infer func- in animals. tion. The classic example of this is probably the work What followed was a period in which a hetero- of the British neurologist David Ferrier (1878), who geneous group of scholars began to develop models demonstrated that destruction of the precentral gyrus of mental processes and then correlate intermediate of the cortex led to quite precise deficits in move- variables in these models with either physiological ment generation. What marks many of these studies measurements or lesion-induced deficits. However, during the classical period in is that they these scholars faced two very significant problems. often focused on damage to either sensory systems or First, there was a surplus of models. Dozens of related movement control systems. The reason for this should models could often account for the same phenomena, be obvious; the sensory stimuli presented to a subject and it was hard to discriminate between these mod- are easy to control and quantify – they are observables els. Second, there was a paucity of data. Physiological in the economic sense of the word. The same is true experiments are notoriously difficult and slow, and for movements that we instruct a subject to produce. although they yield precise data they do so at an ago- Movements are directly observable and easily quan- nizingly slow rate. Neurological experiments (at least tified. In contrast, mental state is much more elusive. in humans) move more quickly but are less precise, Although there has for centuries been clear evidence because the researcher does not have control over the that neurological damage influences mental state, placement of lesions. relating damage to mental state is difficult specifi- It was the resolution of these two problems, or cally because mental state is not directly observable. attempts to resolve them, that was at the heart of the Indeed, relating mental state to neurological damage cognitive neuroscientific revolution. In describing that requires some kind of theory (often a global one), and revolution, we focus on the study of decision making. it was this theory that was largely absent during the This was by no means a central element in the cogni- classical period in neurology. tive neuroscientific revolution, but it forms the central In contrast to the neurological approach, the physi- piece for understanding the source of neuroeconomics ological approach to the study of the brain involves in the neuroscientific community. correlating direct measurements of biological state, The lack of a clear global theory was first engaged such as the firing of action potentials in neurons, seriously by the importation of signal detection the- changes in blood flow, and changes in neurotrans- ory into the physiological tradition. Signal detection mitters, with events in the outside world. During theory ( Green and Swets, 1966 ) is a normative theory the classical period this more precise set of methodo- of signal categorization broadly used in the study of logical tools was extremely powerful for elucidating human perception. The critical that revo- basic features of nervous function, but was extremely lutionized the physiological study of cognitive phe- limited in its applicability to complex mental states. nomena was the use of this normative theory to relate Initially this limitation arose from a methodological neuronal activity directly to behavior. constraint. Physiological measurements are invasive In the late 1980s, William Newsome and J. Anthony and often destructive. This limits their use in animals Movshon (see, for example, Newsome et al. , 1989 ) and, in the classical period, in anesthetized animals. began work on an effort to relate the activity of The result was an almost complete restriction of phys- neurons in the middle temporal area of visual cor- iological approaches during the classical period to the tex (Area MT) to decisions made by monkeys in the study of sensory encoding in the nervous system. domain of perceptual categorization. In those experi- A number of critical advances during the period ments, thirsty monkeys had to evaluate an ambiguous from the 1960s to the 1980s, however, led to both a visual signal which indicated which of two actions broadening of these approaches and, later, a fusion would yield a fluid reward. What the experiments

INTRODUCTION: A BRIEF HISTORY OF NEUROECONOMICS 6 1. INTRODUCTION: A BRIEF HISTORY OF NEUROECONOMICS demonstrated was that the firing rates of single neu- 1992) simultaneously published the first results using rons in this area, which were hypothesized to encode functional magnetic resonance imaging (fMRI) to the perceptual signal being directly evaluated by the image brain activity non-invasively – a development monkeys in their decision making, could be used to that opened the door for direct imaging of brain activ- predict the patterns of stochastic choice produced by ity while humans engaged in cognitive tasks. This was the animals in response to the noisy sensory signals. a critical event, because it meant that a technique was This was a landmark event in neuroscience, because available for the rapid (if crude) direct measurement it provided the first really clear demonstration of a of neural state in humans. Owing to the wide avail- correlation between neuronal activity and stochastic ability of MRI and the safety of the method, the use choice. Following Newsome’s suggestion, this class of fMRI for functional imaging of human cognitive of correlation came to be known as a psychometric– processes has grown exponentially. Perhaps because neurometric match – the behavioral measurement being of the visually compelling nature of the results, show- referred to as psychometric and the matching neuro- ing brain areas “ lighting up, ” this work became highly nal measurement as neurometric. influential not just in the neuroscientific and psycho- This was also a landmark event in the neural study logical communities but also beyond. The result was of decision making, because it was the first successful that scholars in many disciplines began to consider the attempt to predict decisions from single neuron activ- possibilities of measuring the brain activity of humans ity. However, it was also controversial. Parallel stud- during decision making. The challenge was that there ies in areas believed to control movement generation was no clear theoretical tool for organizing this huge (Glimcher and Sparks, 1992) seemed not to be as easily amount of information. amenable to a signal-detection based analysis (Sparks, 1999 ; Glimcher, 2003 ). This led to a long-lasting debate in the early and mid-1990s regarding whether theories SETTING THE STAGE FOR such as signal detection would prove adequate for the NEUROECONOMICS wholesale study of decision making. The neurological tradition had gained its first glimpses into the effects of brain damage on deci- By the late 1990s, several converging trends had sion making in 1848, in the case of Phineas Gage set the stage for the birth of neuroeconomics. Within ( Macmillan, 2002 ). After his brain was penetrated by a economics and the psychology of judgment and deci- steel rod, Gage exhibited a drastic change in personal- sion making, a critical tension had emerged between ity and decision-making ability. The systematic study the neoclassical/revealed preference school and the of decision-making deficits following brain damage behavioral school. The revealed-preference theorists was initially undertaken, in the 1990s, by Antonio had an elegant axiomatic model of human choice Damasio, Antoine Bechara, and their colleagues (see, which had been revealed to be only crudely predictive for example, Bechara et al. , 1994 ), who began exam- of human behavior, and for which it was easy to pro- ining decision making under in a card-sorting duce counterexamples. Revealed-preference theorists . Their work related damage to frontal cor- responded to this challenge by both tinkering with the tical areas with specific elements of an emotion-based model to improve it and challenging the significance theory of decision making which, though not norma- of many of the existing behavioral economic experi- tive like signal , was widely influential. ments (relying on the Friedman “ F-twist ” – that pre- The interest in decision making that this work sparked dictions based on axioms might be approximately true in the neurological community was particularly oppor- even if the axioms are wrong). tune, because at this time the stage was being set for The behavioral economists, in contrast, responded combining a new kind of physiological measurement to this challenge by looking for alternative mathemati- with behavioral studies in humans. cal theories and different types of data to test those A better understanding of the relation between theories – theories which they saw as being claims mental and neural function in humans awaited the about both computational processes and choices. development of methods to image activ- Their goal was to provide an alternative theoretical ity non-invasively. Early work by Roland, Raichle, and approach for predicting behavior and a methodol- others had used positron emission tomography (PET) ogy for testing those theories. This is an approach that to image the neural correlates to mental function, but requires good theories that predict both choices and this method was limited in its application owing to “ non-choice ” data. The appropriate form for such an the need for radioactive tracers. In 1992, three groups alternative theory has, however, been hotly debated. ( Bandettini et al. , 1992 ; Kwong et al. , 1992 ; Ogawa et al. , One approach to developing such a theory derives

INTRODUCTION: A BRIEF HISTORY OF NEUROECONOMICS TWO TRENDS, ONE GOAL 7 from the great progress economics has made towards introduction to the neuroscientific literature of such understanding the interaction of two systems concepts as expected value and expected utility. in the external world – for example, understanding the interactions of firms and the workers they hire. This pre-existing mathematical facility with two-agent TWO TRENDS, ONE GOAL models aligned naturally with an interest among psy- chologists in what are known as “ dual-process ” mod- els. If, as some behavioral economists have argued, the The birth of neuroeconomics, then, grew from a goal is to minimally complicate the standard models number of related factors that simultaneously influ- from economics, then going from a single agent maxi- enced what were basically two separate communities, mizing a unifying “ utility ” to two independent agents albeit with a significant overlap. A group of behav- (or processes) interacting might be a useful . ioral economists and cognitive psychologists looked This strategy forms one of the principle alternative towards functional brain-imaging as a tool to both theoretical approaches that gave birth to neuroeco- test and develop alternatives to neoclassical/revealed nomics. The appeal of the dual-process model for preference theories (especially when too many theo- economists is that when inefficient choice behaviors ries chased too few data using choices as the only are observed in humans, these can be viewed as the class of data). A group of physiologists and cognitive result of the two (or more) independent agents being looked towards economic theory as locked in a bad equilibrium by their own self-interests. a tool to test and develop algorithmic models of the Of course, other scholars within behavioral economics neural hardware for choice. The result was an interest- have suggested other approaches that also have neu- ing split that persists in neuroeconomics today – and roeconomic implications. A view from evolutionary of which there is evidence in this volume. psychology that may serve as another example is that The result is that the two communities, one pre- encapsulated models execute heuristics that are spe- dominantly (although not exclusively) neuroscientific cially adapted to evolutionarily selected tasks (see, for and the other predominantly (although not exclu- example, Gigerenzer et al., 2000). These models have sively) behavioral economic, thus approached a union something to say about the tradeoff between efficient from two very different directions. Both, however, choice and computational complexity, which might promoted an approach that was controversial within be used to generate hypotheses about brain processes their parent disciplines. Many neurobiologists outside (and cross-species comparisons). the emerging neuroeconomic community argued that Within much of neuroscience, and that fraction the complex normative models of economics would of closely allied with animal be of little value for understanding the behavior of studies of choice, a different tension was simultane- real humans and animals. Many economists, particu- ously being felt as these multiple agent and heuris- larly hardcore neoclassicists, argued that algorithmic- tic models were evolving in behavioral economics. It level studies of decision making were unlikely was clear that both those physiologists interested in to improve the predictive power of the revealed- single neuron studies of decision making and those preference approach. cognitive neuroscientists closely allied to them were Despite these challenges, the actual growth of neur- interested in describing the algorithmic mechanisms oeconomics during the late 1990s and early 2000s was of choice. Their goal was to describe the neurobiologi- explosive. The converging group of like-minded econ- cal hardware that supported choice behavior in situa- omists, neuroscientists, and cognitive psychologists tions ranging from perceptual decision making to the quickly generated a set of meetings and conferences expression of more complicated preferences. What that fostered a growing sense of interdisciplinary they lacked was an overarching theoretical frame- . Probably the first of these interdisci- work for placing their neural measurements into con- plinary interactions was held in 1997 at Carnegie- text. Newsome and his colleagues had argued that the Mellon University, organized by the economists Colin standard mathematical tool for understanding sen- Camerer and . After a hiatus of sory categorization – signal detection theory – could several years this was followed by two meetings in serve that role, but many remained skeptical that this 2001, one held by the Gruter Foundation for Law at approach could be sufficiently generalized. What that their annual meeting in Squaw Valley. At that meeting naturally led to was the suggestion, by Glimcher and the Gruter Foundation chose to focus its workshop on his colleagues, that the neoclassical/revealed pref- the intersection of neuroscience and economics, and erence framework might prove a useful theoretical invited several speakers active at the interface of these tool for neuroscience. What followed was the rapid converging disciplines. The second meeting focused

INTRODUCTION: A BRIEF HISTORY OF NEUROECONOMICS 8 1. INTRODUCTION: A BRIEF HISTORY OF NEUROECONOMICS more directly on what would later become neuroeco- electrical stimulation of the medial forebrain bundle nomics, and was held at . The can be construed as an effort to maximize the animal’s meeting was organized by the Jonathan instant-to-instant utility. In this analysis, then, changes Cohen and the economist Christina Paxson, and is in the desirability of brain-stimulation reward as a often seen as having been the inception of the present- function of stimulation frequency should be formally day Society for Neuroeconomics. At this meeting, interpreted as changes in the utility of stimulus train. economists and neuroscientists met to explicitly dis- Unlike in standard theories of utility, however, Shizgal cuss the growing of these fields and to and Conover proposed that the expected utility of an debate the value of such a convergence. There was, action is perceived by the animal as the expected util- however, no consensus at the meeting that the grow- ity of that action divided by the sum of the expected ing convergence was desirable. utilities of all available actions. This particular for- Nonetheless, the Princeton meeting generated mulation has its root in the work of the psychologist significant , and in 2003 a small invita- Richard Herrnstein, who proposed that many choices tion-only meeting that included nearly all of the reflect this normalization with regard to the value of active researchers in the emerging area was held on other alternatives – a phenomenon he referred to as the Martha’s Vineyard, organized by Greg Berns of Emory matching law . (For more about the matching law, see University. This three-day meeting marked a clear Chapter 30). turning point at which a group of economists, psy- In fact, this equation had been introduced to self- chologists, and neurobiologists began to identify them- stimulation studies five years earlier by Shizgal’s selves as neuroeconomists and to explicitly shape the mentor, C. Randy Gallistel. In the early 1990s, Gallistel convergence between the fields. This led to an open had used Herrnstein’s work to inspire quantitative registration meeting the following year at Kiawah choice-based experiments and analyses of intracra- Island, organized by Baylor College of Medicine’s nial self-stimulation (see Gallistel, 1994 ). Shizgal’s Read Montague. At this meeting a decision was made, extension of this work is critical in the history of neur- by essentially all the central figures in the emerging oeconomics, because he moved away from the largely discipline, to form a society and to turn this recurring descriptive models of Herrnstein towards the norma- meeting into an annual event that would serve as a tive models of economics. What Shizgal’s work did focal point for neuroeconomics internationally. At the not do, however, was fully incorporate the standard meeting, was elected President of the , but rather a more normative version Society. The Society then held its first formal meeting of Herrnstein’s approach. in 2005 at Kiawah Island. In 1999 this set of papers was followed by a paper Against this backdrop of meetings, a series of criti- by Platt and Glimcher (another student of Gallistel’s) cal papers and books was emerging that did even in Nature that argued quite explicitly for a normative more to shape these interactions between scholars in utility-based analysis of choice behavior in monkeys the several disciplines, and to communicate the goals (Platt and Glimcher, 1999). As they put it in that paper: of the emerging neuroeconomic community to the Neurobiologists have begun to focus increasingly on the larger neurobiological and economic communities. study of sensory-motor processing, but many of the models Probably the first neurobiological paper to rest explic- used to describe these processes remain rooted in the classic itly on a normative economic theory was Peter Shizgal reflex … Here we describe a formal economic-mathematical and Kent Conover’s 1996 review, “ On the neural com- approach for the physiological study of the sensory-motor process, or decision making. putation of utility, ” in Current Directions in Psychological Science . This was followed the next year by a related At an experimental level, the paper goes on to dem- paper published by Shizgal in Current Opinion in onstrate that the activity of single neurons in the pos- Neurobiology entitled “ Neural basis of utility estima- terior parietal cortex is a lawful function of both the tion.” The reason that these papers can be viewed as probability and the magnitude of expected rewards. the first in neuroeconomics is because they attempt to This was significant, because standard expected util- describe the neurobiological substrate of a behavioral ity theory predicates choice on lawful functions of choice using a form of normative choice theory derived these same two variables. The paper, however, makes from economics. In these papers, Shizgal analyzed a critical mis-step in its examination of actual choice the results of studies of intracranial self-stimulation behavior. The authors go on to examine a matching- in rats using a type of utility theory related loosely to law type behavior which they interpret in terms of the standard expected utility theory of von Neumann normative expected utility theory. This is problematic, and Morgenstern. The papers argue that the choices because there is no normative standard for the analy- an animal makes regarding whether or not to work for sis of matching-law behaviors. Indeed, in the example

INTRODUCTION: A BRIEF HISTORY OF NEUROECONOMICS TWO TRENDS, ONE GOAL 9 they present in the paper it cannot be proved that the that guided much research in judgment and decision- behavior is predicted by their normative model; if making laboratories throughout the world (a theory anything, the data seem to suggest that the animals’ described in detail in Chapter 11). In the paper, Breiter behave sub-optimally. The result is a mixing of nor- and colleagues manipulated the perceived desirability mative and non-normative approaches that charac- of a particular lottery outcome (in this case, winning terized the early neurobiological work with economic zero dollars) by changing the values of two other pos- approaches. sible lottery outcomes. When winning zero dollars is At the same time that this paper appeared in print, the worst of three possible outcomes, Kahneman and the behavioral economists , George Tversky’s prospect theory predicts that subjects should Lowenstein, and Drazen Prelec began circulating a view it negatively; however, when it is the best of the manuscript in economic circles by the name of Grey three outcomes, then subjects should view it more pos- Matters. In this manuscript the authors also argued itively. The scanning experiment revealed that brain for a neuroeconomic approach, but this time from a activation in the ventral matched these pre- behavioral economic perspective. What these three dicted subjective valuations. economists argued was that the failures of traditional The other paper published that year reflected a col- axiomatic approaches likely reflected neurobiological laboration between the more neoclassically oriented constraints on the algorithmic processes responsible economist Kevin McCabe, his colleague Vernon Smith for decision making. Neurobiological approaches to (who would share the Nobel Prize with Kahneman the study of decision, they argued, might reveal and the following year for his contributions to experimen- define these constraints which cause deviations in tal economics), the econometrician Daniel Houser, behavior from normative theory. and a team that included a psychologist and a bio- What was striking about this argument, in economic medical engineer. Their paper, which appeared in the circles, was that it proposed an algorithmic analysis of Proceedings of the National Academy of Sciences of the the physical mechanism of choice – a possibility that United States of America (McCabe et al. , 2001 ) examined had been explicitly taboo until that time. Prior to the behavior and neural activation while subjects engaged 1990s it had been a completely ubiquitous view in eco- in a strategic game. This also represented the first use nomic circles that models of behavior, like expected of game theory, an economic tool for the study of social utility theory, were “ as if” models – the model was to decision making, in a neurobiological experiment. In be interpreted “ as if” utility were represented inter- this paper, subjects played a game either against nally by the chooser. However, as Samuelson had an anonymous human opponent or against a compu- argued half a century earlier, it was irrelevant whether ter, the details of which are reviewed in Chapter 5 of this was actually the case because the models sought to this volume. Their neurobiological data revealed that link options to choices not to make assertions about the in some subjects the medial is dif- mechanisms by which that process was accomplished. ferentially active under some of the conditions they Camerer and colleagues argued against this view, sug- examined, becoming more active when subjects play gesting that deviations from normative theory should a cooperative strategy that deviates from the standard be embraced as clues to the underlying neurobiologi- normative prediction of play in that game. From these cal basis of choice. In a real sense, then, these econo- data, the authors hypothesized that this non-normative mists turned to neurobiology for exactly the opposite pattern of has its origin in circuits of the reason that the neurobiologists had turned to econom- prefrontal cortex. ics. They embraced neuroscience as a principled alter- The following year, many of these emerging trends native to normative theory. were reviewed in an important special Society for At this point, there was a rush by several research Neuroscience conference issue of the journal Neuron groups to perform an explicitly economic experiment (Volume 36, Issue 2) edited by Jonathan Cohen and that would mate these two disciplines in human choos- Kenneth Blum entitled Reward and Decision . As these ers. Two groups succeeded in this quest in 2001. The editors wrote in the introduction to that issue: first of these papers appeared in the journal Neuron , and Within neuroscience, for example, we are awash with reflected a collaboration between the functional mag- data that in many cases lack a coherent theoretical under- netic resonance imaging pioneer Hans Breiter, Shizgal, standing (a quick trip to the poster floor of the Society for and Kahneman (who would win the Nobel Prize in meeting can be convincing on this point). Economic Sciences for his contribution to behavioral Conversely, in economics, it has become abundantly evident that the pristine assumptions of the “ standard economic economics the following year). This paper (Breiter et al. , model” – that individuals operate as optimal decision mak- 2001) was based on Kahneman and Tversky’s prospect ers in maximizing utility – are in direct violation of even the theory, a non-normative form of expected utility theory most basic facts about human behavior.

INTRODUCTION: A BRIEF HISTORY OF NEUROECONOMICS 10 1. INTRODUCTION: A BRIEF HISTORY OF NEUROECONOMICS

In that issue, although all of the articles are by neu- appeared shortly after this in 2005. Within the eco- robiologists, particular attention is drawn to normative nomic community this issue was hugely influential theories of decision. Of especial interest are articles by and served, to a large degree, to define neuroeconom- Montague and Berns (2002), Schultz (2002) , Dayan and ics. The issue included articles by several economists Balleine (2002) , Gold and Shadlen (2002) , and Glimcher and neuroscientists, including scholars ranging from (2002) , which all point towards the interaction of nor- Gallistel (2005) to Smith ( Houser et al. , 2005 ). mative models and neurobiology. Interestingly, the Another major advance was presented in 2005, this issue draws attention to the ongoing debate regarding one by Michael Kosfeld and his colleagues in Ernst the role of the in reward Fehr’s research group at the University of Zurich processing, and draws upon previous work that had ( Kosfeld et al., 2005). This paper was important because identified normative or near-normative models of it was the first demonstration of a neuropharmacologi- learning that posit a role for dopamine. (This is a sub- cal manipulation that alters behavior in a manner that ject of tremendous importance to neuroeconomists can be interpreted with regard to normative theory. today, and forms the focus of the third section of this In the paper, subjects were asked to play a trust game volume.) What followed was a literal flood of deci- much like the one examined by McCabe and colleagues. sion-making studies in the neuroscientific literature, Fehr’s critical manipulation was to increase brain lev- many of which relied on normative economic theory. els of the neuropeptide (by an intranasal Figure 1.1 documents this flood, plotting the number application of the compound) before the players made of papers published from 1990 to 2006 that list both their decision. What Kosfeld and colleagues found “ brain ” and “ decision making ” as keywords. was that the with oxytocin sent more At the end of this initial period, a set of summary to the trustees in the trust game than investors who reviews began to emerge that served as manifestos received placebo. This increase in trusting behavior for the emerging neuroeconomic discipline. In 2003 occurred despite the fact that investors ’ beliefs about the Glimcher published a book, directed primarily at trustees’ back-transfers remained unchanged. In con- neuroscientists, that reviewed the history of neuro- trast, oxytocin did not affect the trustees ’ behavior – i.e., science and argued that this history was striking in its trustees’ back-transfers remained unchanged – ruling lack of normative models for higher cognitive function out the possibility that the neuropeptide just increases (Glimcher, 2003). Glimcher proposed that economics reciprocity or generosity. However, oxytocin did not could serve as the source for this much needed cause an unspecific increase in the willingness to take normative theory. Shortly thereafter the Camerer, , because in a control experiment – a pure risk Loewenstein, and Prelec paper was published under game – the investors with oxytocin did not behave dif- the title “Neuroeconomics ” (Camerer et al. , 2005 ); this ferently from the subjects with placebo. What was most also served as a manifesto, but from the economic side. interesting about this study from a neuroeconomic Within the economic community a role similar to point of view was the demonstration that the admin- that of the Neuron special issue was played by a spe- istration of this endogenously produced neuropeptide cial issue on neuroeconomics presented by the jour- altered a complex choice behavior of subjects in a very nal Games and Economic Behavior (Volume 52, Issue 2) specific way – it neither affected the trustees’ behavior and edited by the economist , which nor did it affect the investors ’ general willingness to take risks, it only increased the investors’ risk prefer- ence if the risk was constituted by the interaction with 160 another human partner – suggesting a neurobiological 140 basis for a difference between preferences for social and non-social risks. 120 The rise of neuroeconomics has been strongly asso- 100 ciated with the rapid development of non-invasive 80 techniques for human research and single-cell recordings in non-human . One 60 limitation of these technologies is that they produce 40 largely correlative measures of brain activity, making 20 it difficult to examine the causal role of specific brain 0 activations for choice behavior. This limitation can, 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006 2008 however, be overcome with non-invasive methods FIGURE 1.1 The increase in numbers of papers on decision- of brain stimulation, such as transcranial magnetic making studies in the neuroscientific literature, 1990–2006 stimulation (TMS) and transcranial direct current

INTRODUCTION: A BRIEF HISTORY OF NEUROECONOMICS SUMMARY 11 stimulation (tDCS), which enable researchers selec- The chapters that follow should allow readers to tively to modify the neural processing associated draw their own conclusions regarding this growing and with choice behavior. A recent study by Knoch et al. dynamic field. Each of the major threads of contempo- (2006) provides a demonstration of the additional rary research is reviewed in these pages. Although it is neuroeconomic insights generated with these meth- far too soon for there to be consensus in this commu- ods. Previous fMRI results (Sanfey et al. , 2003 ) had nity, the field today is small enough that a single vol- shown that the right and the left dorsolateral prefron- ume can provide a comprehensive review. We therefore tal cortex (DLPFC) are activated when subjects decide invite you, the readers, to estimate for yourselves the about the acceptance or rejection of unfair bargain- future directions that will yield greatest profit. ing offers in the (for a description of this bargaining game, see Chapter 5). This finding raises many points, such as whether both hemispheres References are causally involved in the choice process. Likewise, is DLPFC affecting judgments about the fairness of Allais , M. ( 1953 ). Le comportment de l’homme rationnel devant le bargaining offers, or is it specifically involved in the risqué: Critique des postulates et axioms de l’ecole americaine . implementation of fairness concerns? Knoch and col- Econometrica 21 , 503 – 546 . Bandettini , P.A. , Wong , E.C. , Hinks , R.S. et al . ( 1992 ). Time course leagues disrupted the right and the left DLPFC with EPI of human brain function during task activation . Magn. Res. TMS and found that the disruption of both PFC areas Med. 25 , 390 – 397 . left more abstract judgements of fairness fully intact Bechara , A. , Damasio , H. , Tranel , D. , and Damasio , A. ( 1994 ). (relative to a placebo stimulation), while the disrup- Deciding advantageously before knowing the advantageous tion of the right (but not the left) DLPFC resulted in a strategy . Science 28 , 1293 – 1295 . Breiter , H.C. , Aharon , I. , Kahneman , D. et al . ( 2001 ). Functional large increase in the acceptance of unfair offers. From a imaging of neural responses to expectancy and experience of neuroeconomic viewpoint it is important to know the monetary gains and losses . Neuron 30 , 619 – 639 . dissociations between judgment and choice, because Bruni , L. and Sugden , R. (2007 ). The road not taken: how psychol- choice typically implies that the decision maker must ogy was removed from economics, and how it might be brought bear costs and benefits, while judgment alone is not yet back . Economic J. 117 , 146 – 173 . Busino , G. ( 1964 ). Note bibliographique sur le Cours . In: V. Pareto associated with the bearing of costs and benefits. More (ed.) , Epistolario . Rome : Accademia Nazionale dei Lincei , generally, non-invasive brain stimulation techniques pp. 1165 – 1172 . are likely to play an important role in future neuroeco- Camerer , C. , Loewenstein , G. , and Prelec , D. ( 2005 ). Neuroeconomics: nomic studies because they provide causal knowledge how neuroscience can inform economics . J. Econ. Lit. 4 3 , 9 – 6 4 . and, in combination with imaging tools, make it possi- Colander , D. ( 2007 ). Retrospectives: Edgeworth’s hedonimeter and the quest to measure utility . J. Econ. Persp. 21 , 215 – 225 . ble to isolate whole decision networks that are causally Dayan , P. and Balleine , B.W. ( 2002 ). Reward, motivation, and rein- involved in the generation of choices. forcement learning . Neuron 36 ( 2 ) , 285 – 298 . Ellsberg , D. ( 1961 ). Risk, ambiguity and the savage axioms . Q. J. Econ. 75 , 643 – 669 . Ferrier , D. ( 1878 ). The Localization of Cerebral Disease . New York, NY : SUMMARY G.P. Putnam and Sons . Gallistel , C.R. ( 1994 ). Foraging for brain stimulation: toward a neu- robiology of computation . Cognition 50 , 151 – 170 . Despite these impressive accomplishments, neu- Gallistel , C.R. ( 2005 ). Deconstructing the law of effect . Games Econ. roeconomics is at best a decade old and has yet to Behav. 52 , 410 – 423 . demonstrate a critical role in neuroscience, psychol- Gigerenzer, G., Todd, P.M., and the ABC Research Group. (2000). Simple Heuristics that Make Us Smart . New York, NY: Oxford ogy, or economics. Indeed, scholars within neuroeco- University Press. nomics are still debating whether neuroscientific data Gilovich , T. , Griffin , D. , and Kahneman, D. ( 2002). Heuristics and will provide theory for economists or whether eco- : The Psychology of Intuitive Judgment . New York, NY : nomic theory will provide structure for neuroscience. Cambridge University Press . We hope that both goals will be accomplished, but Glimcher , P. ( 2002 ). Decisions, decisions, decisions: choosing a bio- logical science of dhoice. Neuron 36 , 323 – 332 . the exact form of this contribution is not yet clear. Glimcher , P. ( 2003 ). Decisions, and the Brain: The Science of However, there are also skeptical voices, and the Neuroeconomics . Cambridge, MA : MIT Press. Pareto (1897) and Friedman arguments that econom- Glimcher , P.W. and Sparks , D.L. ( 1992 ). Movement selection in ics is only about choices still lives in the form of fun- advance of action in the superior colliculus. Nature 355 , 542 – 545 . damentalist critique. Gul and Pesendorfer (2008), for Gold , J. and Shadlen , M. ( 2002 ). Banburismus and the brain: decod- ing the relationship between sensory stimuli, decisions, and example, have argued that neuroscientific data and reward . Neuron 36 , 299 – 308 . neuroscientific theories should, in principle, be unwel- Green , D.M. and Swets , J.A. ( 1966 ). Signal Detection Theory and come in economics. Psychophysics . New York, NY : Wiley .

INTRODUCTION: A BRIEF HISTORY OF NEUROECONOMICS 12 1. INTRODUCTION: A BRIEF HISTORY OF NEUROECONOMICS

Gul , F. and Pesendorfer , W. ( 2008 ). The case for mindless econom- Newsome , W.T. , Britten , K.H. , and Movshon , J.A. ( 1989 ). Neuronal ics . In: A. Caplin and A. Schotter (eds), The Foundations of Positive correlates of a perceptual decision . Nature 341 , 52 – 54 . and Normative Economics: A Handbook . Oxford : Oxford University Ogawa , S. , Tank , D.W. , Menon, R. et al . ( 1992 ). Intrinsic signal Press , forthcoming. changes accompanying sensory stimulation: functional brain Houser , D. , Bechara , A. , Keane , M. et al . ( 2005 ). Identifying individ- mapping with magnetic resonance imaging. Proc. Natl Acad. Sci. ual differences: an algorithm with application to Phineas Gage . USA 89 , 5951 – 5955 . Games Econ. Behav. 52 , 373 – 385 . Platt , M.L. and Glimcher , P.W. ( 1999 ). Neural correlates of decision Houthakker , H.S . ( 1950 ). Revealed preference and the utility func- variables in parietal cortex. Nature 400 , 233 – 238 . tion. Economic s 17, 159–174 . Samuelson , P.A. ( 1938 ). A note on the pure theory of consumer Kahneman , D. and Tversky , A. (1979 ). Prospect theory: an analysis behavior . Economia 1 , 6 1 – 7 1 . of decision under risk . Econometrica 47 , 263 – 291 . Sanfey , A.G. , Rilling , J.K. , Aronson , J.A. et al . ( 2003 ). The neural basis Knoch , D. , Pascual-Leone , A., Meyer, K. et al . ( 2006 ). Diminishing of economic decision-making in the Ultimatum Game. Science reciprocal fairness by disrupting the right prefrontal cortex. 300 , 1673 – 1675 . Science 314 , 829 – 832 . Schultz , W. ( 2002 ). Getting formal with dopamine and reward . Kosfeld , M. , Heinrichs , M. , Zak , P.J. et al . ( 2005 ). Oxytocin increases Neuron 36 , 241 – 263 . trust in humans . Nature 435 , 673 – 676 . Shizgal , P. ( 1997 ). Neural basis of utility estimation . Curr. Opin. Kwong , K.K. , Bellieveau , J.W., Chesler , D.A. et al . ( 1992 ). Dynamic Neurobiol. 7 , 198 – 208 . magnetic resonance imaging of human brain activity dur- Shizgal , P. and Conover, K. ( 1996 ). On the neural computation of ing primary sensory stimulation . Proc. Natl Acad. Sci. USA 8 9 , utility . Curr. Direct. Psycholog. Sci. 5 , 3 7 – 4 3 . 5675 – 5679 . Smith , V. ( 1976 ). Experimental economics: induced value theory. Macmillan , M. ( 2002 ). An Odd Kind of Fame: Stories of Phineas Gage . Am. Econ. Rev. 66 , 274 – 279 . Cambridge, MA : MIT Press . Sparks , D.L. ( 1999 ). Conceptual issues related to the role of the McCabe , K. , Houser , D. , Ryan , L. et al . ( 2001 ). A functional imaging superior colliculus in the control of gaze . Curr. Opin. Neurobiol. study of cooperation in two-person reciprocal exchange . Proc. 9 , 698 – 707 . Natl Acad. Sci. USA 98 , 11832 – 11835 . von Neumann , J. and Morgenstern , O. ( 1944 ). Theory of Games and Montague , P.R. and Berns , G.S. ( 2002 ). Neural economics and the Economic Behavior . Princeton, NJ : Princeton University Press. biological substrates of valuation. Neuron 36 , 265 – 284 .

INTRODUCTION: A BRIEF HISTORY OF NEUROECONOMICS