DOCSLIB.ORG

Decision Theory, Reinforcement Learning, and the Brain

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Decision Theory, Reinforcement Learning, and the Brain Cognitive, Affective, & Behavioral Neuroscience 2008, 8 (4), 429-453 doi:10.3758/CABN.8.4.429 CONNE C TIONS BETWEEN COMPUTATIONAL AND NEURO B IOLOGI C AL PERSPE C TIVES ON DE C ISION MAKING Decision theory, reinforcement learning, and the brain PETER DAYAN University College London, London, England AND NATHANIEL D. DAW New York University, New York, New York Decision making is a core competence for animals and humans acting and surviving in environments they only partially comprehend, gaining rewards and punishments for their troubles. Decision-theoretic concepts permeate experiments and computational models in ethology, psychology, and neuroscience. Here, we review a well-known, coherent Bayesian approach to decision making, showing how it unifies issues in Markovian decision problems, signal detection psychophysics, sequential sampling, and optimal exploration and discuss paradigmatic psychological and neural examples of each problem. We discuss computational issues concern- ing what subjects know about their task and how ambitious they are in seeking opti mal solutions; we address algorithmic topics concerning model-based and model-free methods for making choices; and we highlight key aspects of the neural implementation of decision making. The abilities of animals to make predictions about the af- by Cue D and then a smaller shock 20% of the time. The fective nature of their environments and to exert control in converse is true for Cue C. Subjects can, therefore, in gen- order to maximize rewards and minimize threats to homeo- eral expect a large shock when they get Cue A, but this ex- stasis are critical to their longevity. Decision theory is a for- pectation can occasionally be reversed. How can they learn mal framework that allows us to describe and pose quantita- to predict their future shocks? An answer to this question tive questions about optimal and approximately optimal is provided in the Markov Decision Problem section; as behavior in such environments (e.g., Bellman, 1957; Berger, described there, these functions are thought to involve the 1985; Berry & Fristedt, 1985; Bertsekas, 2007; Bertsekas striatum and various neuromodulators. Such predictions & Tsitsiklis, 1996; Gittins, 1989; Glimcher, 2004; Gold & can be useful for guiding decisions that can have deferred Shadlen, 2002, 2007; Green & Swets, 1966; Körding, 2007; consequences; formally, this situation can be character- Mangel & Clark, 1989; McNamara & Houston, 1980; Mon- ized as a Markov decision problem (MDP) as studied in tague, 2006; Puterman, 2005; Sutton & Barto, 1998; Wald, the fields of dynamic programming (Bellman, 1957) and 1947; Yuille & Bülthoff, 1996) and is, therefore, a critical reinforcement learning (Sutton & Barto, 1998). tool for modeling, understanding, and predicting psycho- Figure 1B depicts a decision task that is closely related logical data and their neural underpinnings. to signal detection theory (Green & Swets, 1966) and has Figure 1 illustrates three paradigmatic tasks that have been particularly illuminating about the link between neu- been used to probe this competence. Figure 1A shows a ral activity and percep tion (Britten, Newsome, Shadlen, case of prediction learning (Seymour et al., 2004). Here, Celebrini, & Movshon, 1996; Britten, Shadlen, Newsome, human volunteers are wired up to a device that deliv- & Movshon, 1992; Gold & Shadlen, 2001, 2002, 2007; ers variable strength electric shocks. The delivery of the Shadlen, Britten, Newsome, & Movshon, 1996; Shadlen & shocks is preceded by visual cues (Cue A through Cue D) Newsome, 1996). In the classical version of this task, mon- in a sequence. Cue A occurs on 50% of the trials; it is fol- keys watch a screen that shows moving dots. A proportion lowed by Cue B and then a larger shock 80% of the time or of the dots is moving in one direction; the rest are moving in P. Dayan, [email protected] 429 Copyright 2008 Psychonomic Society, Inc. 430 DAYAN AN D DAW AB C Saccade High Motion Cue A Cue B pain Targets Fixation RF Cue C Cue D Low pain Time Reaction Figure 1. Paradigmatic tasks. (A) Subjects can predict the magnitude of future pain from partially informa- tive visual cues that follow a Markov chain (Seymour et al., 2004; see the Markov Decision Problem section). (B) Monkeys have to report the direction of predominant motion in a random-dot kinematogram by making an eye movement (Britten, Shadlen, Newsome, & Movshon, 1992); see the Signal Detection Theory section. In some experiments, the monkeys have the additional choice of whether to act or collect more information (Gold & Shad- len, 2007); see the Temporal State Uncertainty section. (C) Subjects have to choose between four evolving, noisy bandit machines (whose payments are shown in the insets) and, so, must balance exploration and exploitation (Daw, O’Doherty, Dayan, Seymour, & Dolan, 2006); see the Exploration and Exploitation section. random directions. The monkeys have to report the coher- of the method for solving the problems, and discuss these ent direction by making a suitable eye move ment. By vary- particular cases and their near relatives in some detail. A ing the fraction of the dots that moves coherently (called wealth of problems and solu tions that has arisen in different the coherence), the task can be made easier or harder. The areas of psychology and neurobiology is thereby integrated, visual system of the monkey reports evidence about the and common solution mechanisms are identified. In particu- direc tion of motion; how should the subject use this infor- lar, viewing these problems as differ ent specializations of a mation to make a decision? In some versions of the task, the common task involving both sensory inference and learning monkey can also choose when to emit its response; how can it decide whether to respond or to continue collecting in- formation? These topics are addressed in the Signal Detec- tion Theory and Temporal State Uncertainty sections, along with the roles of two visual cortical areas (MT and lateral c1 x1 c2 x2 intraparietal area [LIP]). The simpler version can be seen as a standard signal detection theory task; the more complex L R L R one has been analyzed by Gold and Shad len (2001, 2007) r (R) r (L) as an optimal-stopping problem. This, in turn, is a form of r1(L) 1 2 r2(R) partially observ able MDP (POMDP) related to the sequen- tial probability ratio test (SPRT; Ratcliff & Rouder, 1998; Shadlen, Hanks, Churchland, Kiani, & Yang, 2007; Smith & Ratcliff, 2004; Wald, 1947). Finally, Figure 1C shows a further decision-theoretic p r (C) p wrinkle in the form of an experiment on the trade-off 31 3 32 between exploration and exploitation (Daw, O’Doherty, Dayan, Seymour, & Dolan, 2006). Here, human subjects C C have to choose between four one-armed bandit machines whose payoffs are changing over time (shown by the curves inside each). The subjects can find out about the current c3 x3 c4 x4 value of a machine only by choosing it and, so, have to balance picking the machine that is currently believed L R L R best against choosing a machine that has not recently r3(L) r3(R) r4(L) r4(R) been sampled, in case its value has increased. Problems of this sort are surprisingly computationally intractable Figure 2. An abstract decision-theoretic task. Subjects normally (Berry & Fristedt, 1985; Gittins, 1989); the section of the start in state x3 or x4, signaled by cues c3 or c4. They have three op- tions (L, R, or C): The former two lead to rewards or punishments present article on Exploration and Exploitation discusses such as r3(L); the latter lead via stochastic transitions (probabili- the issues and approximate solutions, including one that, ties such as p31) to states x1 or x2, which are themselves signaled evidence suggests, implicates the fronto-polar cortex. by cues c1 and c2, and license rewarded or punished choices L and Despite the apparent differences between these tasks, R. Subjects can be (partially) ignorant about their states if the cues are confusable (e.g., if x3 “looks like” x4), and/or about the they actually share some deep underlying commonalities. In rules of the task (the rewards and probabilities). In some cases, this review, we provide a straightforward formal framework the subjects might start in x1 or x2. Different options generate a that shows the links, give a computationally minded view wide family of popular decision-theoretic problems. DECISION THEORY , REINFORCEMEN T LEARNING , AN D T HE BRAIN 431 components gives strong clues as to how sensory systems In terms of knowledge, the subjects might be ignorant and computational mechanisms involved in the sig nal detec- of their precise state in the problem (i.e., which xi they cur- tion tasks—such as areas MT and LIP—are likely to interact rently occupy), and/or the rules of the task (i.e., the transi- with the basal ganglia and neuromodulatory systems that are tion probabilities and rewards contingent on particular ac- implicated in the reinforcement learning tasks. tions in states). If they know both in the standard version of We tie the problems together by inventing a new, slightly the task, which requires L at x3 and x1 and R at x4 and x2, more abstract assignment (shown in Figure 2). Particular with C being costly, and they know that they are at x3, say, specializations of this abstraction are then isomorphic to they should choose L. However, if they know the rules and the tasks associated with Figure 1. The case of Figure 2 is know that they are at either x3 or x4, but do not know which an apparently simple maze-like choice task that we might for sure (perhaps because the cues c3 and c4 are similar or present to animal or human subjects, who have to make identical), it might be worth the cost of choosing C in order decisions (here, choices between actions L, R, and C) in to collect information from c1 and c2 (if these are more dis- order to optimize their outcomes (r).
Load more

Recommended publications
  • Prisoners of Reason Game Theory and Neoliberal Political Economy
    C:/ITOOLS/WMS/CUP-NEW/6549131/WORKINGFOLDER/AMADAE/9781107064034PRE.3D iii [1–28] 11.8.2015 9:57PM Prisoners of Reason Game Theory and Neoliberal Political Economy S. M. AMADAE Massachusetts Institute of Technology C:/ITOOLS/WMS/CUP-NEW/6549131/WORKINGFOLDER/AMADAE/9781107064034PRE.3D iv [1–28] 11.8.2015 9:57PM 32 Avenue of the Americas, New York, ny 10013-2473, usa Cambridge University Press is part of the University of Cambridge. It furthers the University’s mission by disseminating knowledge in the pursuit of education, learning, and research at the highest international levels of excellence. www.cambridge.org Information on this title: www.cambridge.org/9781107671195 © S. M. Amadae 2015 This publication is in copyright. Subject to statutory exception and to the provisions of relevant collective licensing agreements, no reproduction of any part may take place without the written permission of Cambridge University Press. First published 2015 Printed in the United States of America A catalog record for this publication is available from the British Library. Library of Congress Cataloging in Publication Data Amadae, S. M., author. Prisoners of reason : game theory and neoliberal political economy / S.M. Amadae. pages cm Includes bibliographical references and index. isbn 978-1-107-06403-4 (hbk. : alk. paper) – isbn 978-1-107-67119-5 (pbk. : alk. paper) 1. Game theory – Political aspects. 2. International relations. 3. Neoliberalism. 4. Social choice – Political aspects. 5. Political science – Philosophy. I. Title. hb144.a43 2015 320.01′5193 – dc23 2015020954 isbn 978-1-107-06403-4 Hardback isbn 978-1-107-67119-5 Paperback Cambridge University Press has no responsibility for the persistence or accuracy of URLs for external or third-party Internet Web sites referred to in this publication and does not guarantee that any content on such Web sites is, or will remain, accurate or appropriate.
    [Show full text]
  • Statistical Decision Theory: Concepts, Methods and Applications
    Statistical Decision Theory: Concepts, Methods and Applications (Special topics in Probabilistic Graphical Models) FIRST COMPLETE DRAFT November 30, 2003 Supervisor: Professor J. Rosenthal STA4000Y Anjali Mazumder 950116380 Part I: Decision Theory – Concepts and Methods Part I: DECISION THEORY - Concepts and Methods Decision theory as the name would imply is concerned with the process of making decisions. The extension to statistical decision theory includes decision making in the presence of statistical knowledge which provides some information where there is uncertainty. The elements of decision theory are quite logical and even perhaps intuitive. The classical approach to decision theory facilitates the use of sample information in making inferences about the unknown quantities. Other relevant information includes that of the possible consequences which is quantified by loss and the prior information which arises from statistical investigation. The use of Bayesian analysis in statistical decision theory is natural. Their unification provides a foundational framework for building and solving decision problems. The basic ideas of decision theory and of decision theoretic methods lend themselves to a variety of applications and computational and analytic advances. This initial part of the report introduces the basic elements in (statistical) decision theory and reviews some of the basic concepts of both frequentist statistics and Bayesian analysis. This provides a foundational framework for developing the structure of decision problems. The second section presents the main concepts and key methods involved in decision theory. The last section of Part I extends this to statistical decision theory – that is, decision problems with some statistical knowledge about the unknown quantities. This provides a comprehensive overview of the decision theoretic framework.
    [Show full text]
  • Richard Bradley, Decision Theory with a Human Face
    Œconomia History, Methodology, Philosophy 9-1 | 2019 Varia Richard Bradley, Decision Theory with a Human Face Nicolas Gravel Electronic version URL: http://journals.openedition.org/oeconomia/5273 DOI: 10.4000/oeconomia.5273 ISSN: 2269-8450 Publisher Association Œconomia Printed version Date of publication: 1 March 2019 Number of pages: 149-160 ISSN: 2113-5207 Electronic reference Nicolas Gravel, « Richard Bradley, Decision Theory with a Human Face », Œconomia [Online], 9-1 | 2019, Online since 01 March 2019, connection on 29 December 2020. URL : http://journals.openedition.org/ oeconomia/5273 ; DOI : https://doi.org/10.4000/oeconomia.5273 Les contenus d’Œconomia sont mis à disposition selon les termes de la Licence Creative Commons Attribution - Pas d'Utilisation Commerciale - Pas de Modification 4.0 International. | Revue des livres/Book review 149 Comptes rendus / Reviews Richard Bradley, Decision Theory with a Human Face Cambridge: Cambridge University Press, 2017, 335 pages, ISBN 978-110700321-7 Nicolas Gravel∗ The very title of this book, borrowed from Richard Jeffrey’s “Bayesianism with a human face” (Jeffrey, 1983a) is a clear in- dication of its content. Just like its spiritual cousin The Foun- dations of Causal Decision Theory by James M. Joyce(2000), De- cision Theory with a Human Face provides a thoughtful descrip- tion of the current state of development of the Bolker-Jeffrey (BJ) approach to decision-making. While a full-fledged presen- tation of the BJ approach is beyond the scope of this review, it is difficult to appraise the content of Decision Theory with a Hu- man Face without some acquaintance with both the basics of the BJ approach to decision-making and its fitting in the large cor- pus of “conventional” decision theories that have developed in economics, mathematics and psychology since at least the publication of Von von Neumann and Morgenstern(1947).
    [Show full text]
  • 9<HTLDTH=Hebaaa>
    springer.com/booksellers Springer News 11/12/2007 Statistics 49 D. R. Anderson, Colorado State University, Fort J. Franke, W. Härdle, C. M. Hafner U. B. Kjaerulff, Aalborg University, Aalborg, Denmark; Collins, CO, USA A. L. Madsen, HUGIN Expert A/S, Aalborg, Denmark Statistics of Financial Markets Model Based Inference in the An Introduction Bayesian Networks and Life Sciences Influence Diagrams: A Guide A Primer on Evidence to Construction and Analysis Statistics of Financial Markets offers a vivid yet concise introduction to the growing field of statis- The abstract concept of “information” can be tical applications in finance. The reader will learn Probabilistic networks, also known as Bayesian quantified and this has led to many important the basic methods to evaluate option contracts, to networks and influence diagrams, have become advances in the analysis of data in the empirical analyse financial time series, to select portfolios one of the most promising technologies in the area sciences. This text focuses on a science philosophy and manage risks making realistic assumptions of of applied artificial intelligence, offering intui- based on “multiple working hypotheses” and statis- the market behaviour. tive, efficient, and reliable methods for diagnosis, tical models to represent them. The fundamental The focus is both on fundamentals of math- prediction, decision making, classification, trou- science question relates to the empirical evidence ematical finance and financial time series analysis bleshooting, and data mining under uncertainty. for hypotheses in this set - a formal strength of and on applications to given problems of financial Bayesian Networks and Influence Diagrams: A evidence. Kullback-Leibler information is the markets, making the book the ideal basis for Guide to Construction and Analysis provides a information lost when a model is used to approxi- lectures, seminars and crash courses on the topic.
    [Show full text]
  • Theories of International Relations* Ole R. Holsti
    Theories of International Relations* Ole R. Holsti Universities and professional associations usually are organized in ways that tend to separate scholars in adjoining disciplines and perhaps even to promote stereotypes of each other and their scholarly endeavors. The seemingly natural areas of scholarly convergence between diplomatic historians and political scientists who focus on international relations have been underexploited, but there are also some signs that this may be changing. These include recent essays suggesting ways in which the two disciplines can contribute to each other; a number of prizewinning dissertations, later turned into books, by political scientists that effectively combine political science theories and historical materials; collaborative efforts among scholars in the two disciplines; interdisciplinary journals such as International Security that provide an outlet for historians and political scientists with common interests; and creation of a new section, “International History and Politics,” within the American Political Science Association.1 *The author has greatly benefited from helpful comments on earlier versions of this essay by Peter Feaver, Alexander George, Joseph Grieco, Michael Hogan, Kal Holsti, Bob Keohane, Timothy Lomperis, Roy Melbourne, James Rosenau, and Andrew Scott, and also from reading 1 K. J. Holsti, The Dividing Discipline: Hegemony and Diversity in International Theory (London, 1985). This essay is an effort to contribute further to an exchange of ideas between the two disciplines by describing some of the theories, approaches, and "models" political scientists have used in their research on international relations during recent decades. A brief essay cannot do justice to the entire range of theoretical approaches that may be found in the current literature, but perhaps those described here, when combined with citations of some representative works, will provide diplomatic historians with a useful, if sketchy, map showing some of the more prominent landmarks in a neighboring discipline.
    [Show full text]
  • Improving Monetary Policy Models by Christopher A. Sims, Princeton
    Improving Monetary Policy Models by Christopher A. Sims, Princeton University and NBER CEPS Working Paper No. 128 May 2006 IMPROVING MONETARY POLICY MODELS CHRISTOPHER A. SIMS ABSTRACT. If macroeconomic models are to be useful in policy-making, where un- certainty is pervasive, the models must be treated as probability models, whether formally or informally. Use of explicit probability models allows us to learn sys- tematically from past mistakes, to integrate model-based uncertainty with uncertain subjective judgment, and to bind data-bassed forecasting together with theory-based projection of policy effects. Yet in the last few decades policy models at central banks have steadily shed any claims to being believable probability models of the data to which they are fit. Here we describe the current state of policy modeling, suggest some reasons why we have reached this state, and assess some promising directions for future progress. I. WHY DO WE NEED PROBABILITY MODELS? Fifty years ago most economists thought that Tinbergen’s original approach to macro-modeling, which consisted of fitting many equations by single-equation OLS Date: May 26, 2006. °c 2006 by Christopher A. Sims. This material may be reproduced for educational and research purposes so long as the copies are not sold, even to recover costs, the document is not altered, and this copyright notice is included in the copies. Research for this paper was supported by NSF grant SES 0350686 and by Princeton’s Center for Economic Policy Studies. This paper was presented at a December 2-3, 2005 conference at the Board of Governors of the Federal Reserve and may appear in a conference volume or journal special issue.
    [Show full text]
  • Introduction to Decision Theory: Econ 260 Spring, 2016
    Introduction to Decision Theory: Econ 260 Spring, 2016 1/7. Instructor Details: Sumantra Sen Email: [email protected] OH: TBA TA: Ross Askanazi OH of TA: TBA 2/7. Website: courseweb.library.upenn.edu (Canvas Course site) 3/7. Overview: This course will provide an introduction to Decision Theory. The goal of the course is to make the student think rigorously about decision-making in contexts of objective and subjective uncertainty. The heart of the class is delving deeply into the axiomatic tradition of Decision Theory in Economics, and covering the classic theorems of De Finetti, Von Neumann & Morgenstern, and Savage and reevaluating them in light of recent research in behavioral economics and Neuro-economics. Time permitting; we will study other extensions (dynamic models and variants of other static models) as well, with the discussion anchored by comparison and reference to these classic models. There is no required text for the class, as there is no suitable one for undergraduates that covers the material we plan to do. We will reply on lecture notes and references (see below). 4/7. Policies of the Economics Department, including course pre-requisites and academic integrity Issues may be found at: http://economics.sas.upenn.edu/undergraduate-program/course- information/guidelines/policies 5/7. Grades: There will be two non-cumulative in-class midterms (20% each) during the course and one cumulative final (40%). The midterms will be on Feb 11 (will be graded and returned by Feb 18) and April 5th. The final is on May 6th. If exams are missed, documentable reasons of the validity of such actions must be provided within a reasonable period of time and the instructor will have the choice of pro-rating your grade over the missed exam rather than deliver a make-up.
    [Show full text]
  • A Note on States and Acts in Theories of Decision Under Uncertainty
    A Note on States and Acts in Theories of Decision under Uncertainty Burkhard C. Schipper∗ February 19, 2016 Abstract The purpose of this note is to discuss some problems with the notion of states and acts in theories of decision under uncertainty. We argue that any single-person decision problem under uncertainty can be cast in a canonical state space in which states describe the consequence of every action. Casting decision problems in the canonical state space opens up the analysis to the question of how decisions depend on the state space. It also makes transparent that the set of all acts is problematic because it must contain incoherent acts that assign to some state a consequence that is inconsistent with the internal structure of that state. We discuss three ap- proaches for dealing with incoherent acts, each unsatisfactory in that it limits what can be meaningfully revealed about beliefs with choice experiments. Keywords: Decision theory, State spaces, States of nature, Acts, Subjective expected utility, Ambiguity, Ellsberg paradox. JEL-Classifications: D80, D81. 1 States in Theories of Decision under Uncertainty In many theories of decision under uncertainty such as Subjective Expected Utility (Sav- age 1954, Anscombe and Aumann, 1963), Choquet expected utility (Schmeidler, 1989), Maxmin Expected Utility (Gilboa and Schmeidler, 1989) etc., states of nature are used to list possible resolutions of uncertainty. A decision maker is uncertain which action would yield which consequences. A state is \a description of the world so complete that, if true and known, the consequences of every action would be known" (Arrow, 1971, p.
    [Show full text]
  • Decision Theory: a Brief Introduction
    Decision Theory A Brief Introduction 1994-08-19 Minor revisions 2005-08-23 Sven Ove Hansson Department of Philosophy and the History of Technology Royal Institute of Technology (KTH) Stockholm 1 Contents Preface ..........................................................................................................4 1. What is decision theory? ..........................................................................5 1.1 Theoretical questions about decisions .........................................5 1.2 A truly interdisciplinary subject...................................................6 1.3 Normative and descriptive theories..............................................6 1.4 Outline of the following chapters.................................................8 2. Decision processes....................................................................................9 2.1 Condorcet .....................................................................................9 2.2 Modern sequential models ...........................................................9 2.3 Non-sequential models.................................................................10 2.4 The phases of practical decisions – and of decision theory.........12 3. Deciding and valuing................................................................................13 3.1 Relations and numbers .................................................................13 3.2 The comparative value terms .......................................................14 3.3 Completeness ...............................................................................16
    [Show full text]
  • The Role of Models and Probabilities in the Monetary Policy Process
    1017-01 BPEA/Sims 12/30/02 14:48 Page 1 CHRISTOPHER A. SIMS Princeton University The Role of Models and Probabilities in the Monetary Policy Process This is a paper on the way data relate to decisionmaking in central banks. One component of the paper is based on a series of interviews with staff members and a few policy committee members of four central banks: the Swedish Riksbank, the European Central Bank (ECB), the Bank of England, and the U.S. Federal Reserve. These interviews focused on the policy process and sought to determine how forecasts were made, how uncertainty was characterized and handled, and what role formal economic models played in the process at each central bank. In each of the four central banks, “subjective” forecasting, based on data analysis by sectoral “experts,” plays an important role. At the Federal Reserve, a seventeen-year record of model-based forecasts can be com- pared with a longer record of subjective forecasts, and a second compo- nent of this paper is an analysis of these records. Two of the central banks—the Riksbank and the Bank of England— have explicit inflation-targeting policies that require them to set quantita- tive targets for inflation and to publish, several times a year, their forecasts of inflation. A third component of the paper discusses the effects of such a policy regime on the policy process and on the role of models within it. The large models in use in central banks today grew out of a first generation of large models that were thought to be founded on the statisti- cal theory of simultaneous-equations models.
    [Show full text]
  • Nber Working Paper Series Decision Theoretic
    NBER WORKING PAPER SERIES DECISION THEORETIC APPROACHES TO EXPERIMENT DESIGN AND EXTERNAL VALIDITY Abhijit Banerjee Sylvain Chassang Erik Snowberg Working Paper 22167 http://www.nber.org/papers/w22167 NATIONAL BUREAU OF ECONOMIC RESEARCH 1050 Massachusetts Avenue Cambridge, MA 02138 April 2016 We thank Esther Duflo for her leadership on the handbook, and for extensive comments on earlier drafts. Chassang and Snowberg gratefully acknowledge the support of NSF grant SES-1156154. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research. NBER working papers are circulated for discussion and comment purposes. They have not been peer-reviewed or been subject to the review by the NBER Board of Directors that accompanies official NBER publications. © 2016 by Abhijit Banerjee, Sylvain Chassang, and Erik Snowberg. All rights reserved. Short sections of text, not to exceed two paragraphs, may be quoted without explicit permission provided that full credit, including © notice, is given to the source. Decision Theoretic Approaches to Experiment Design and External Validity Abhijit Banerjee, Sylvain Chassang, and Erik Snowberg NBER Working Paper No. 22167 April 2016 JEL No. B23,C9,O1 ABSTRACT A modern, decision-theoretic framework can help clarify important practical questions of experimental design. Building on our recent work, this chapter begins by summarizing our framework for understanding the goals of experimenters, and applying this to re-randomization. We then use this framework to shed light on questions related to experimental registries, pre- analysis plans, and most importantly, external validity. Our framework implies that even when large samples can be collected, external decision-making remains inherently subjective.
    [Show full text]
  • Publics and Markets. What's Wrong with Neoliberalism?
    12 Publics and Markets. What’s Wrong with Neoliberalism? Clive Barnett THEORIZING NEOLIBERALISM AS A egoists, despite much evidence that they POLITICAL PROJECT don’t. Critics of neoliberalism tend to assume that increasingly people do act like this, but Neoliberalism has become a key object of they think that they ought not to. For critics, analysis in human geography in the last this is what’s wrong with neoliberalism. And decade. Although the words ‘neoliberal’ and it is precisely this evaluation that suggests ‘neoliberalism’ have been around for a long that there is something wrong with how while, it is only since the end of the 1990s neoliberalism is theorized in critical human that they have taken on the aura of grand geography. theoretical terms. Neoliberalism emerges as In critical human geography, neoliberal- an object of conceptual and empirical reflec- ism refers in the first instance to a family tion in the process of restoring to view a of ideas associated with the revival of sense of political agency to processes previ- economic liberalism in the mid-twentieth ously dubbed globalization (Hay, 2002). century. This is taken to include the school of This chapter examines the way in which Austrian economics associated with Ludwig neoliberalism is conceptualized in human von Mises, Friedrich von Hayek and Joseph geography. It argues that, in theorizing Schumpeter, characterized by a strong com- neoliberalism as ‘a political project’, critical mitment to methodological individualism, an human geographers have ended up repro- antipathy towards centralized state planning, ducing the same problem that they ascribe commitment to principles of private property, to the ideas they take to be driving forces and a distinctive anti-rationalist epistemo- behind contemporary transformations: they logy; and the so-called Chicago School of reduce the social to a residual effect of more economists, also associated with Hayek but fundamental political-economic rationalities.
    [Show full text]