Learning and Equilibrium
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Lecture 4 Rationalizability & Nash Equilibrium Road
Lecture 4 Rationalizability & Nash Equilibrium 14.12 Game Theory Muhamet Yildiz Road Map 1. Strategies – completed 2. Quiz 3. Dominance 4. Dominant-strategy equilibrium 5. Rationalizability 6. Nash Equilibrium 1 Strategy A strategy of a player is a complete contingent-plan, determining which action he will take at each information set he is to move (including the information sets that will not be reached according to this strategy). Matching pennies with perfect information 2’s Strategies: HH = Head if 1 plays Head, 1 Head if 1 plays Tail; HT = Head if 1 plays Head, Head Tail Tail if 1 plays Tail; 2 TH = Tail if 1 plays Head, 2 Head if 1 plays Tail; head tail head tail TT = Tail if 1 plays Head, Tail if 1 plays Tail. (-1,1) (1,-1) (1,-1) (-1,1) 2 Matching pennies with perfect information 2 1 HH HT TH TT Head Tail Matching pennies with Imperfect information 1 2 1 Head Tail Head Tail 2 Head (-1,1) (1,-1) head tail head tail Tail (1,-1) (-1,1) (-1,1) (1,-1) (1,-1) (-1,1) 3 A game with nature Left (5, 0) 1 Head 1/2 Right (2, 2) Nature (3, 3) 1/2 Left Tail 2 Right (0, -5) Mixed Strategy Definition: A mixed strategy of a player is a probability distribution over the set of his strategies. Pure strategies: Si = {si1,si2,…,sik} σ → A mixed strategy: i: S [0,1] s.t. σ σ σ i(si1) + i(si2) + … + i(sik) = 1. If the other players play s-i =(s1,…, si-1,si+1,…,sn), then σ the expected utility of playing i is σ σ σ i(si1)ui(si1,s-i) + i(si2)ui(si2,s-i) + … + i(sik)ui(sik,s-i). -
Improving Fictitious Play Reinforcement Learning with Expanding Models
Improving Fictitious Play Reinforcement Learning with Expanding Models Rong-Jun Qin1;2, Jing-Cheng Pang1, Yang Yu1;y 1National Key Laboratory for Novel Software Technology, Nanjing University, China 2Polixir emails: [email protected], [email protected], [email protected]. yTo whom correspondence should be addressed Abstract Fictitious play with reinforcement learning is a general and effective framework for zero- sum games. However, using the current deep neural network models, the implementation of fictitious play faces crucial challenges. Neural network model training employs gradi- ent descent approaches to update all connection weights, and thus is easy to forget the old opponents after training to beat the new opponents. Existing approaches often maintain a pool of historical policy models to avoid the forgetting. However, learning to beat a pool in stochastic games, i.e., a wide distribution over policy models, is either sample-consuming or insufficient to exploit all models with limited amount of samples. In this paper, we pro- pose a learning process with neural fictitious play to alleviate the above issues. We train a single model as our policy model, which consists of sub-models and a selector. Everytime facing a new opponent, the model is expanded by adding a new sub-model, where only the new sub-model is updated instead of the whole model. At the same time, the selector is also updated to mix up the new sub-model with the previous ones at the state-level, so that the model is maintained as a behavior strategy instead of a wide distribution over policy models. -
Lecture Notes
GRADUATE GAME THEORY LECTURE NOTES BY OMER TAMUZ California Institute of Technology 2018 Acknowledgments These lecture notes are partially adapted from Osborne and Rubinstein [29], Maschler, Solan and Zamir [23], lecture notes by Federico Echenique, and slides by Daron Acemoglu and Asu Ozdaglar. I am indebted to Seo Young (Silvia) Kim and Zhuofang Li for their help in finding and correcting many errors. Any comments or suggestions are welcome. 2 Contents 1 Extensive form games with perfect information 7 1.1 Tic-Tac-Toe ........................................ 7 1.2 The Sweet Fifteen Game ................................ 7 1.3 Chess ............................................ 7 1.4 Definition of extensive form games with perfect information ........... 10 1.5 The ultimatum game .................................. 10 1.6 Equilibria ......................................... 11 1.7 The centipede game ................................... 11 1.8 Subgames and subgame perfect equilibria ...................... 13 1.9 The dollar auction .................................... 14 1.10 Backward induction, Kuhn’s Theorem and a proof of Zermelo’s Theorem ... 15 2 Strategic form games 17 2.1 Definition ......................................... 17 2.2 Nash equilibria ...................................... 17 2.3 Classical examples .................................... 17 2.4 Dominated strategies .................................. 22 2.5 Repeated elimination of dominated strategies ................... 22 2.6 Dominant strategies .................................. -
Approximation Guarantees for Fictitious Play
Approximation Guarantees for Fictitious Play Vincent Conitzer Abstract— Fictitious play is a simple, well-known, and often- significant progress has been made in the computation of used algorithm for playing (and, especially, learning to play) approximate Nash equilibria. It has been shown that for any games. However, in general it does not converge to equilibrium; ǫ, there is an ǫ-approximate equilibrium with support size even when it does, we may not be able to run it to convergence. 2 Still, we may obtain an approximate equilibrium. In this O((log n)/ǫ ) [1], [22], so that we can find approximate paper, we study the approximation properties that fictitious equilibria by searching over all of these supports. More play obtains when it is run for a limited number of rounds. recently, Daskalakis et al. gave a very simple algorithm We show that if both players randomize uniformly over their for finding a 1/2-approximate Nash equilibrium [12]; we actions in the first r rounds of fictitious play, then the result will discuss this algorithm shortly. Feder et al. then showed is an ǫ-equilibrium, where ǫ = (r + 1)/(2r). (Since we are examining only a constant number of pure strategies, we know a lower bound on the size of the supports that must be that ǫ < 1/2 is impossible, due to a result of Feder et al.) We considered to be guaranteed to find an approximate equi- show that this bound is tight in the worst case; however, with an librium [16]; in particular, supports of constant sizes can experiment on random games, we illustrate that fictitious play give at best a 1/2-approximate Nash equilibrium. -
Modeling Human Learning in Games
Modeling Human Learning in Games Thesis by Norah Alghamdi In Partial Fulfillment of the Requirements For the Degree of Masters of Science King Abdullah University of Science and Technology Thuwal, Kingdom of Saudi Arabia December, 2020 2 EXAMINATION COMMITTEE PAGE The thesis of Norah Alghamdi is approved by the examination committee Committee Chairperson: Prof. Jeff S. Shamma Committee Members: Prof. Eric Feron, Prof. Meriem T. Laleg 3 ©December, 2020 Norah Alghamdi All Rights Reserved 4 ABSTRACT Modeling Human Learning in Games Norah Alghamdi Human-robot interaction is an important and broad area of study. To achieve success- ful interaction, we have to study human decision making rules. This work investigates human learning rules in games with the presence of intelligent decision makers. Par- ticularly, we analyze human behavior in a congestion game. The game models traffic in a simple scenario where multiple vehicles share two roads. Ten vehicles are con- trolled by the human player, where they decide on how to distribute their vehicles on the two roads. There are hundred simulated players each controlling one vehicle. The game is repeated for many rounds, allowing the players to adapt and formulate a strategy, and after each round, the cost of the roads and visual assistance is shown to the human player. The goal of all players is to minimize the total congestion experienced by the vehicles they control. In order to demonstrate our results, we first built a human player simulator using Fictitious play and Regret Matching algorithms. Then, we showed the passivity property of these algorithms after adjusting the passivity condition to suit discrete time formulation. -
Modeling Strategic Behavior1
Modeling Strategic Behavior1 A Graduate Introduction to Game Theory and Mechanism Design George J. Mailath Department of Economics, University of Pennsylvania Research School of Economics, Australian National University 1September 16, 2020, copyright by George J. Mailath. World Scientific Press published the November 02, 2018 version. Any significant correc- tions or changes are listed at the back. To Loretta Preface These notes are based on my lecture notes for Economics 703, a first-year graduate course that I have been teaching at the Eco- nomics Department, University of Pennsylvania, for many years. It is impossible to understand modern economics without knowl- edge of the basic tools of game theory and mechanism design. My goal in the course (and this book) is to teach those basic tools so that students can understand and appreciate the corpus of modern economic thought, and so contribute to it. A key theme in the course is the interplay between the formal development of the tools and their use in applications. At the same time, extensions of the results that are beyond the course, but im- portant for context are (briefly) discussed. While I provide more background verbally on many of the exam- ples, I assume that students have seen some undergraduate game theory (such as covered in Osborne, 2004, Tadelis, 2013, and Wat- son, 2013). In addition, some exposure to intermediate microeco- nomics and decision making under uncertainty is helpful. Since these are lecture notes for an introductory course, I have not tried to attribute every result or model described. The result is a somewhat random pattern of citations and references. -
Comparing Auction Designs Where Suppliers Have Uncertain Costs and Uncertain Pivotal Status
RAND Journal of Economics Vol. 00, No. 0, Winter 2018 pp. 1–33 Comparing auction designs where suppliers have uncertain costs and uncertain pivotal status ∗ Par¨ Holmberg and ∗∗ Frank A. Wolak We analyze how market design influences bidding in multiunit procurement auctions where suppliers have asymmetric information about production costs. Our analysis is particularly relevant to wholesale electricity markets, because it accounts for the risk that a supplier is pivotal; market demand is larger than the total production capacity of its competitors. With constant marginal costs, expected welfare improves if the auctioneer restricts offers to be flat. We identify circumstances where the competitiveness of market outcomes improves with increased market transparency. We also find that, for buyers, uniform pricing is preferable to discriminatory pricing when producers’ private signals are affiliated. 1. Introduction Multiunit auctions are used to trade commodities, securities, emission permits, and other divisible goods. This article focuses on electricity markets, where producers submit offers before the level of demand and amount of available production capacity are fully known. Due to demand shocks, unexpected outages, transmission-constraints, and intermittent output from renewable energy sources, it often arises that an electricity producer is pivotal, that is, that realized demand is larger than the realized total production capacity of its competitors. A producer that is certain to be pivotal possess a substantial ability to exercise market power because it can withhold output ∗ Research Institute of Industrial Economics (IFN), Associate Researcher of the Energy Policy Research Group (EPRG), University of Cambridge; [email protected]. ∗∗ Program on Energy and Sustainable Development (PESD) and Stanford University; [email protected]. -
Economics 201B Economic Theory (Spring 2021) Strategic Games
Economics 201B Economic Theory (Spring 2021) Strategic Games Topics: terminology and notations (OR 1.7), games and solutions (OR 1.1-1.3), rationality and bounded rationality (OR 1.4-1.6), formalities (OR 2.1), best-response (OR 2.2), Nash equilibrium (OR 2.2), 2 2 examples × (OR 2.3), existence of Nash equilibrium (OR 2.4), mixed strategy Nash equilibrium (OR 3.1, 3.2), strictly competitive games (OR 2.5), evolution- ary stability (OR 3.4), rationalizability (OR 4.1), dominance (OR 4.2, 4.3), trembling hand perfection (OR 12.5). Terminology and notations (OR 1.7) Sets For R, ∈ ≥ ⇐⇒ ≥ for all . and ⇐⇒ ≥ for all and some . ⇐⇒ for all . Preferences is a binary relation on some set of alternatives R. % ⊆ From % we derive two other relations on : — strict performance relation and not  ⇐⇒ % % — indifference relation and ∼ ⇐⇒ % % Utility representation % is said to be — complete if , or . ∀ ∈ % % — transitive if , and then . ∀ ∈ % % % % can be presented by a utility function only if it is complete and transitive (rational). A function : R is a utility function representing if → % ∀ ∈ () () % ⇐⇒ ≥ % is said to be — continuous (preferences cannot jump...) if for any sequence of pairs () with ,and and , . { }∞=1 % → → % — (strictly) quasi-concave if for any the upper counter set ∈ { ∈ : is (strictly) convex. % } These guarantee the existence of continuous well-behaved utility function representation. Profiles Let be a the set of players. — () or simply () is a profile - a collection of values of some variable,∈ one for each player. — () or simply is the list of elements of the profile = ∈ { } − () for all players except . ∈ — ( ) is a list and an element ,whichistheprofile () . -
Interim Correlated Rationalizability
Interim Correlated Rationalizability The Harvard community has made this article openly available. Please share how this access benefits you. Your story matters Citation Dekel, Eddie, Drew Fudenberg, and Stephen Morris. 2007. Interim correlated rationalizability. Theoretical Economics 2, no. 1: 15-40. Published Version http://econtheory.org/ojs/index.php/te/article/view/20070015 Citable link http://nrs.harvard.edu/urn-3:HUL.InstRepos:3196333 Terms of Use This article was downloaded from Harvard University’s DASH repository, and is made available under the terms and conditions applicable to Other Posted Material, as set forth at http:// nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of- use#LAA Theoretical Economics 2 (2007), 15–40 1555-7561/20070015 Interim correlated rationalizability EDDIE DEKEL Department of Economics, Northwestern University, and School of Economics, Tel Aviv University DREW FUDENBERG Department of Economics, Harvard University STEPHEN MORRIS Department of Economics, Princeton University This paper proposes the solution concept of interim correlated rationalizability, and shows that all types that have the same hierarchies of beliefs have the same set of interim-correlated-rationalizable outcomes. This solution concept charac- terizes common certainty of rationality in the universal type space. KEYWORDS. Rationalizability, incomplete information, common certainty, com- mon knowledge, universal type space. JEL CLASSIFICATION. C70, C72. 1. INTRODUCTION Harsanyi (1967–68) proposes solving games of incomplete -
Natural Games
Ntur J A a A A ,,,* , a Department of Biosciences, FI-00014 University of Helsinki, Finland b Institute of Biotechnology, FI-00014 University of Helsinki, Finland c Department of Physics, FI-00014 University of Helsinki, Finland ABSTRACT Behavior in the context of game theory is described as a natural process that follows the 2nd law of thermodynamics. The rate of entropy increase as the payoff function is derived from statistical physics of open systems. The thermodynamic formalism relates everything in terms of energy and describes various ways to consume free energy. This allows us to associate game theoretical models of behavior to physical reality. Ultimately behavior is viewed as a physical process where flows of energy naturally select ways to consume free energy as soon as possible. This natural process is, according to the profound thermodynamic principle, equivalent to entropy increase in the least time. However, the physical portrayal of behavior does not imply determinism. On the contrary, evolutionary equation for open systems reveals that when there are three or more degrees of freedom for behavior, the course of a game is inherently unpredictable in detail because each move affects motives of moves in the future. Eventually, when no moves are found to consume more free energy, the extensive- form game has arrived at a solution concept that satisfies the minimax theorem. The equilibrium is Lyapunov-stable against variation in behavior within strategies but will be perturbed by a new strategy that will draw even more surrounding resources to the game. Entropy as the payoff function also clarifies motives of collaboration and subjective nature of decision making. -
Beyond Truth-Telling: Preference Estimation with Centralized School Choice and College Admissions∗
Beyond Truth-Telling: Preference Estimation with Centralized School Choice and College Admissions∗ Gabrielle Fack Julien Grenet Yinghua He October 2018 (Forthcoming: The American Economic Review) Abstract We propose novel approaches to estimating student preferences with data from matching mechanisms, especially the Gale-Shapley Deferred Acceptance. Even if the mechanism is strategy-proof, assuming that students truthfully rank schools in applications may be restrictive. We show that when students are ranked strictly by some ex-ante known priority index (e.g., test scores), stability is a plausible and weaker assumption, implying that every student is matched with her favorite school/college among those she qualifies for ex post. The methods are illustrated in simulations and applied to school choice in Paris. We discuss when each approach is more appropriate in real-life settings. JEL Codes: C78, D47, D50, D61, I21 Keywords: Gale-Shapley Deferred Acceptance Mechanism, School Choice, College Admissions, Stable Matching, Student Preferences ∗Fack: Universit´eParis-Dauphine, PSL Research University, LEDa and Paris School of Economics, Paris, France (e-mail: [email protected]); Grenet: CNRS and Paris School of Economics, 48 boulevard Jourdan, 75014 Paris, France (e-mail: [email protected]); He: Rice University and Toulouse School of Economics, Baker Hall 246, Department of Economcis MS-22, Houston, TX 77251 (e-mail: [email protected]). For constructive comments, we thank Nikhil Agarwal, Peter Arcidi- acono, Eduardo Azevedo, -
Tacit Collusion in Oligopoly"
Tacit Collusion in Oligopoly Edward J. Green,y Robert C. Marshall,z and Leslie M. Marxx February 12, 2013 Abstract We examine the economics literature on tacit collusion in oligopoly markets and take steps toward clarifying the relation between econo- mists’analysis of tacit collusion and those in the legal literature. We provide examples of when the economic environment might be such that collusive pro…ts can be achieved without communication and, thus, when tacit coordination is su¢ cient to elevate pro…ts versus when com- munication would be required. The authors thank the Human Capital Foundation (http://www.hcfoundation.ru/en/), and especially Andrey Vavilov, for …nancial support. The paper bene…tted from discussions with Roger Blair, Louis Kaplow, Bill Kovacic, Vijay Krishna, Steven Schulenberg, and Danny Sokol. We thank Gustavo Gudiño for valuable research assistance. [email protected], Department of Economics, Penn State University [email protected], Department of Economics, Penn State University [email protected], Fuqua School of Business, Duke University 1 Introduction In this chapter, we examine the economics literature on tacit collusion in oligopoly markets and take steps toward clarifying the relation between tacit collusion in the economics and legal literature. Economists distinguish between tacit and explicit collusion. Lawyers, using a slightly di¤erent vocabulary, distinguish between tacit coordination, tacit agreement, and explicit collusion. In hopes of facilitating clearer communication between economists and lawyers, in this chapter, we attempt to provide a coherent resolution of the vernaculars used in the economics and legal literature regarding collusion.1 Perhaps the easiest place to begin is to de…ne explicit collusion.