Towards a Theory of Learning in Games
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Repeated Games
6.254 : Game Theory with Engineering Applications Lecture 15: Repeated Games Asu Ozdaglar MIT April 1, 2010 1 Game Theory: Lecture 15 Introduction Outline Repeated Games (perfect monitoring) The problem of cooperation Finitely-repeated prisoner's dilemma Infinitely-repeated games and cooperation Folk Theorems Reference: Fudenberg and Tirole, Section 5.1. 2 Game Theory: Lecture 15 Introduction Prisoners' Dilemma How to sustain cooperation in the society? Recall the prisoners' dilemma, which is the canonical game for understanding incentives for defecting instead of cooperating. Cooperate Defect Cooperate 1, 1 −1, 2 Defect 2, −1 0, 0 Recall that the strategy profile (D, D) is the unique NE. In fact, D strictly dominates C and thus (D, D) is the dominant equilibrium. In society, we have many situations of this form, but we often observe some amount of cooperation. Why? 3 Game Theory: Lecture 15 Introduction Repeated Games In many strategic situations, players interact repeatedly over time. Perhaps repetition of the same game might foster cooperation. By repeated games, we refer to a situation in which the same stage game (strategic form game) is played at each date for some duration of T periods. Such games are also sometimes called \supergames". We will assume that overall payoff is the sum of discounted payoffs at each stage. Future payoffs are discounted and are thus less valuable (e.g., money and the future is less valuable than money now because of positive interest rates; consumption in the future is less valuable than consumption now because of time preference). We will see in this lecture how repeated play of the same strategic game introduces new (desirable) equilibria by allowing players to condition their actions on the way their opponents played in the previous periods. -
Learning and Equilibrium
Learning and Equilibrium Drew Fudenberg1 and David K. Levine2 1Department of Economics, Harvard University, Cambridge, Massachusetts; email: [email protected] 2Department of Economics, Washington University of St. Louis, St. Louis, Missouri; email: [email protected] Annu. Rev. Econ. 2009. 1:385–419 Key Words First published online as a Review in Advance on nonequilibrium dynamics, bounded rationality, Nash equilibrium, June 11, 2009 self-confirming equilibrium The Annual Review of Economics is online at by 140.247.212.190 on 09/04/09. For personal use only. econ.annualreviews.org Abstract This article’s doi: The theory of learning in games explores how, which, and what 10.1146/annurev.economics.050708.142930 kind of equilibria might arise as a consequence of a long-run non- Annu. Rev. Econ. 2009.1:385-420. Downloaded from arjournals.annualreviews.org Copyright © 2009 by Annual Reviews. equilibrium process of learning, adaptation, and/or imitation. If All rights reserved agents’ strategies are completely observed at the end of each round 1941-1383/09/0904-0385$20.00 (and agents are randomly matched with a series of anonymous opponents), fairly simple rules perform well in terms of the agent’s worst-case payoffs, and also guarantee that any steady state of the system must correspond to an equilibrium. If players do not ob- serve the strategies chosen by their opponents (as in extensive-form games), then learning is consistent with steady states that are not Nash equilibria because players can maintain incorrect beliefs about off-path play. Beliefs can also be incorrect because of cogni- tive limitations and systematic inferential errors. -
Best Experienced Payoff Dynamics and Cooperation in the Centipede Game
Theoretical Economics 14 (2019), 1347–1385 1555-7561/20191347 Best experienced payoff dynamics and cooperation in the centipede game William H. Sandholm Department of Economics, University of Wisconsin Segismundo S. Izquierdo BioEcoUva, Department of Industrial Organization, Universidad de Valladolid Luis R. Izquierdo Department of Civil Engineering, Universidad de Burgos We study population game dynamics under which each revising agent tests each of his strategies a fixed number of times, with each play of each strategy being against a newly drawn opponent, and chooses the strategy whose total payoff was highest. In the centipede game, these best experienced payoff dynamics lead to co- operative play. When strategies are tested once, play at the almost globally stable state is concentrated on the last few nodes of the game, with the proportions of agents playing each strategy being largely independent of the length of the game. Testing strategies many times leads to cyclical play. Keywords. Evolutionary game theory, backward induction, centipede game, computational algebra. JEL classification. C72, C73. 1. Introduction The discrepancy between the conclusions of backward induction reasoning and ob- served behavior in certain canonical extensive form games is a basic puzzle of game the- ory. The centipede game (Rosenthal (1981)), the finitely repeated prisoner’s dilemma, and related examples can be viewed as models of relationships in which each partic- ipant has repeated opportunities to take costly actions that benefit his partner and in which there is a commonly known date at which the interaction will end. Experimen- tal and anecdotal evidence suggests that cooperative behavior may persist until close to William H. -
The Econometric Society European Region Aide Mémoire
The Econometric Society European Region Aide M´emoire March 22, 2021 1 European Standing Committee 2 1.1 Responsibilities . .2 1.2 Membership . .2 1.3 Procedures . .4 2 Econometric Society European Meeting (ESEM) 5 2.1 Timing and Format . .5 2.2 Invited Sessions . .6 2.3 Contributed Sessions . .7 2.4 Other Events . .8 3 European Winter Meeting (EWMES) 9 3.1 Scope of the Meeting . .9 3.2 Timing and Format . 10 3.3 Selection Process . 10 4 Appendices 11 4.1 Appendix A: Members of the Standing Committee . 11 4.2 Appendix B: Winter Meetings (since 2014) and Regional Consultants (2009-2013) . 27 4.3 Appendix C: ESEM Locations . 37 4.4 Appendix D: Programme Chairs ESEM & EEA . 38 4.5 Appendix E: Invited Speakers ESEM . 39 4.6 Appendix F: Winners of the ESEM Awards . 43 4.7 Appendix G: Countries in the Region Europe and Other Areas ........... 44 This Aide M´emoire contains a detailed description of the organisation and procedures of the Econometric Society within the European Region. It complements the Rules and Procedures of the Econometric Society. It is maintained and regularly updated by the Secretary of the European Standing Committee in accordance with the policies and decisions of the Committee. The Econometric Society { European Region { Aide Memoire´ 1 European Standing Committee 1.1 Responsibilities 1. The European Standing Committee is responsible for the organisation of the activities of the Econometric Society within the Region Europe and Other Areas.1 It should undertake the consideration of any activities in the Region that promote interaction among those interested in the objectives of the Society, as they are stated in its Constitution. -
Uniqueness and Symmetry in Bargaining Theories of Justice
Philos Stud DOI 10.1007/s11098-013-0121-y Uniqueness and symmetry in bargaining theories of justice John Thrasher Ó Springer Science+Business Media Dordrecht 2013 Abstract For contractarians, justice is the result of a rational bargain. The goal is to show that the rules of justice are consistent with rationality. The two most important bargaining theories of justice are David Gauthier’s and those that use the Nash’s bargaining solution. I argue that both of these approaches are fatally undermined by their reliance on a symmetry condition. Symmetry is a substantive constraint, not an implication of rationality. I argue that using symmetry to generate uniqueness undermines the goal of bargaining theories of justice. Keywords David Gauthier Á John Nash Á John Harsanyi Á Thomas Schelling Á Bargaining Á Symmetry Throughout the last century and into this one, many philosophers modeled justice as a bargaining problem between rational agents. Even those who did not explicitly use a bargaining problem as their model, most notably Rawls, incorporated many of the concepts and techniques from bargaining theories into their understanding of what a theory of justice should look like. This allowed them to use the powerful tools of game theory to justify their various theories of distributive justice. The debates between partisans of different theories of distributive justice has tended to be over the respective benefits of each particular bargaining solution and whether or not the solution to the bargaining problem matches our pre-theoretical intuitions about justice. There is, however, a more serious problem that has effectively been ignored since economists originally J. -
Slope Takers in Anonymous Markets"
Slope Takers in Anonymous Markets Marek Weretka,y 29th May 2011 Abstract In this paper, we study interactions among large agents trading in anonymous markets. In such markets, traders have no information about other traders’payo¤s, identities or even submitted orders. To trade optimally, each trader estimates his own price impact from the available data on past prices as well as his own trades and acts optimally, given such a simple (residual supply) representation of the residual mar- ket. We show that in a Walrasian auction, which is known to have a multiplicity of Nash equilibria that if all traders independently estimate and re-estimate price im- pacts, market interactions converge to a particular Nash equilibrium. Consequently, anonymity and learning argument jointly provide a natural selection argument for the Nash equilibrium in a Walrasian auction. We then study the properties of such focal equilibrium. JEL clasi…cation: D43, D52, L13, L14 Keywords: Walrasian Auction, Anonymous Thin Markets, Price impacts 1 Introduction Many markets, e.g., markets for …nancial assets are anonymous. In such markets inform- ation of each individual trader is restricted to own trades and commonly observed market price. Traders have no information about other traders’payo¤s, identities or submitted or- ders. The archetypical framework to study anonymous markets, a competitive equilibrium I would like to thank Paul Klemperer, Margaret Meyer, Peyton Young and especially Marzena Rostek for helpful conversations. yUniversity of Wisconsin-Madison, Department of Economics, 1180 Observatory Drive, Madison, WI 53706. E-mail: [email protected]. 1 assumes the following preconditions 1) all traders freely and optimally adjust traded quant- ities to prices; 2) trade occurs in centralized anonymous markets; and 3) individual traders are negligible and hence treat prices parametrically. -
Improving Fictitious Play Reinforcement Learning with Expanding Models
Improving Fictitious Play Reinforcement Learning with Expanding Models Rong-Jun Qin1;2, Jing-Cheng Pang1, Yang Yu1;y 1National Key Laboratory for Novel Software Technology, Nanjing University, China 2Polixir emails: [email protected], [email protected], [email protected]. yTo whom correspondence should be addressed Abstract Fictitious play with reinforcement learning is a general and effective framework for zero- sum games. However, using the current deep neural network models, the implementation of fictitious play faces crucial challenges. Neural network model training employs gradi- ent descent approaches to update all connection weights, and thus is easy to forget the old opponents after training to beat the new opponents. Existing approaches often maintain a pool of historical policy models to avoid the forgetting. However, learning to beat a pool in stochastic games, i.e., a wide distribution over policy models, is either sample-consuming or insufficient to exploit all models with limited amount of samples. In this paper, we pro- pose a learning process with neural fictitious play to alleviate the above issues. We train a single model as our policy model, which consists of sub-models and a selector. Everytime facing a new opponent, the model is expanded by adding a new sub-model, where only the new sub-model is updated instead of the whole model. At the same time, the selector is also updated to mix up the new sub-model with the previous ones at the state-level, so that the model is maintained as a behavior strategy instead of a wide distribution over policy models. -
Approximation Guarantees for Fictitious Play
Approximation Guarantees for Fictitious Play Vincent Conitzer Abstract— Fictitious play is a simple, well-known, and often- significant progress has been made in the computation of used algorithm for playing (and, especially, learning to play) approximate Nash equilibria. It has been shown that for any games. However, in general it does not converge to equilibrium; ǫ, there is an ǫ-approximate equilibrium with support size even when it does, we may not be able to run it to convergence. 2 Still, we may obtain an approximate equilibrium. In this O((log n)/ǫ ) [1], [22], so that we can find approximate paper, we study the approximation properties that fictitious equilibria by searching over all of these supports. More play obtains when it is run for a limited number of rounds. recently, Daskalakis et al. gave a very simple algorithm We show that if both players randomize uniformly over their for finding a 1/2-approximate Nash equilibrium [12]; we actions in the first r rounds of fictitious play, then the result will discuss this algorithm shortly. Feder et al. then showed is an ǫ-equilibrium, where ǫ = (r + 1)/(2r). (Since we are examining only a constant number of pure strategies, we know a lower bound on the size of the supports that must be that ǫ < 1/2 is impossible, due to a result of Feder et al.) We considered to be guaranteed to find an approximate equi- show that this bound is tight in the worst case; however, with an librium [16]; in particular, supports of constant sizes can experiment on random games, we illustrate that fictitious play give at best a 1/2-approximate Nash equilibrium. -
Topologies on Types
Theoretical Economics 1 (2006), 275–309 1555-7561/20060275 Topologies on types EDDIE DEKEL Department of Economics, Northwestern University and School of Economics, Tel Aviv University DREW FUDENBERG Department of Economics, Harvard University STEPHEN MORRIS Department of Economics, Princeton University We define and analyze a “strategic topology” on types in the Harsanyi-Mertens- Zamir universal type space, where two types are close if their strategic behavior is similar in all strategic situations. For a fixed game and action define the dis- tance between a pair of types as the difference between the smallest " for which the action is " interim correlated rationalizable. We define a strategic topology in which a sequence of types converges if and only if this distance tends to zero for any action and game. Thus a sequence of types converges in the strategic topol- ogy if that smallest " does not jump either up or down in the limit. As applied to sequences, the upper-semicontinuity property is equivalent to convergence in the product topology, but the lower-semicontinuity property is a strictly stronger requirement, as shown by the electronic mail game. In the strategic topology, the set of “finite types” (types describable by finite type spaces) is dense but the set of finite common-prior types is not. KEYWORDS. Rationalizability, incomplete information, common knowledge, uni- versal type space, strategic topology. JEL CLASSIFICATION. C70, C72. 1. INTRODUCTION Harsanyi (1967–68) proposed, and Mertens and Zamir (1985) constructed, a universal type space into which (under some technical assumptions) any incomplete information about a strategic situation can be embedded. As a practical matter, applied researchers do not work with that type space but with smaller subsets of the universal type space. -
Implementation Theory*
Chapter 5 IMPLEMENTATION THEORY* ERIC MASKIN Institute for Advanced Study, Princeton, NJ, USA TOMAS SJOSTROM Department of Economics, Pennsylvania State University, University Park, PA, USA Contents Abstract 238 Keywords 238 1. Introduction 239 2. Definitions 245 3. Nash implementation 247 3.1. Definitions 248 3.2. Monotonicity and no veto power 248 3.3. Necessary and sufficient conditions 250 3.4. Weak implementation 254 3.5. Strategy-proofness and rich domains of preferences 254 3.6. Unrestricted domain of strict preferences 256 3.7. Economic environments 257 3.8. Two agent implementation 259 4. Implementation with complete information: further topics 260 4.1. Refinements of Nash equilibrium 260 4.2. Virtual implementation 264 4.3. Mixed strategies 265 4.4. Extensive form mechanisms 267 4.5. Renegotiation 269 4.6. The planner as a player 275 5. Bayesian implementation 276 5.1. Definitions 276 5.2. Closure 277 5.3. Incentive compatibility 278 5.4. Bayesian monotonicity 279 * We are grateful to Sandeep Baliga, Luis Corch6n, Matt Jackson, Byungchae Rhee, Ariel Rubinstein, Ilya Segal, Hannu Vartiainen, Masahiro Watabe, and two referees, for helpful comments. Handbook of Social Choice and Welfare, Volume 1, Edited by K.J Arrow, A.K. Sen and K. Suzumura ( 2002 Elsevier Science B. V All rights reserved 238 E. Maskin and T: Sj'str6m 5.5. Non-parametric, robust and fault tolerant implementation 281 6. Concluding remarks 281 References 282 Abstract The implementation problem is the problem of designing a mechanism (game form) such that the equilibrium outcomes satisfy a criterion of social optimality embodied in a social choice rule. -
Comparative Statics and Perfect Foresight in Infinite Horizon Economies
Digitized by the Internet Archive in 2011 with funding from Boston Library Consortium IVIember Libraries http://www.archive.org/details/comparativestatiOOkeho working paper department of economics Comparative Statics And Perfect Foresight in Infinite Horizon Economies Timothy J. Kehoe David K. Levine* Number 312 December 1982 massachusetts institute of technology 50 memorial drive Cambridge, mass. 02139 Comparative Statics And Perfect Foresight in Infinite Horizon Economies Timothy J. Kehoe David K. Levine* Number 312 December 1982 Department of Economics, M.I.T. and Department of Economics, U.C.L.A., respectively. Abstract This paper considers whether infinite horizon economies have determinate perfect foresight equilibria. ¥e consider stationary pure exchange economies with both finite and infinite numbers of agents. When there is a finite number of infinitely lived agents, we argue that equilibria are generically determinate. This is because the number of equations determining the equilibria is not infinite, but is equal to the number of agents minus one and must determine the marginal utility of income for all but one agent. In an overlapping generations model with infinitely many finitely lived agents, this reasoning breaks down. ¥e ask whether the initial conditions together with the requirement of convergence to a steady state locally determine an equilibrium price path. In this framework there are many economies with isolated equilibria, many with continue of equilibria, and many with no equilibria at all. With two or more goods in every period not only can the price level be indeterminate but relative prices as well. Furthermore, such indeterminacy can occur whether or not there is fiat money in the economy. -
Scenario Analysis Normal Form Game
Overview Economics 3030 I. Introduction to Game Theory Chapter 10 II. Simultaneous-Move, One-Shot Games Game Theory: III. Infinitely Repeated Games Inside Oligopoly IV. Finitely Repeated Games V. Multistage Games 1 2 Normal Form Game A Normal Form Game • A Normal Form Game consists of: n Players Player 2 n Strategies or feasible actions n Payoffs Strategy A B C a 12,11 11,12 14,13 b 11,10 10,11 12,12 Player 1 c 10,15 10,13 13,14 3 4 Normal Form Game: Normal Form Game: Scenario Analysis Scenario Analysis • Suppose 1 thinks 2 will choose “A”. • Then 1 should choose “a”. n Player 1’s best response to “A” is “a”. Player 2 Player 2 Strategy A B C a 12,11 11,12 14,13 Strategy A B C b 11,10 10,11 12,12 a 12,11 11,12 14,13 Player 1 11,10 10,11 12,12 c 10,15 10,13 13,14 b Player 1 c 10,15 10,13 13,14 5 6 1 Normal Form Game: Normal Form Game: Scenario Analysis Scenario Analysis • Suppose 1 thinks 2 will choose “B”. • Then 1 should choose “a”. n Player 1’s best response to “B” is “a”. Player 2 Player 2 Strategy A B C Strategy A B C a 12,11 11,12 14,13 a 12,11 11,12 14,13 b 11,10 10,11 12,12 11,10 10,11 12,12 Player 1 b c 10,15 10,13 13,14 Player 1 c 10,15 10,13 13,14 7 8 Normal Form Game Dominant Strategy • Regardless of whether Player 2 chooses A, B, or C, Scenario Analysis Player 1 is better off choosing “a”! • “a” is Player 1’s Dominant Strategy (i.e., the • Similarly, if 1 thinks 2 will choose C… strategy that results in the highest payoff regardless n Player 1’s best response to “C” is “a”.