Lecture Notes Part 2
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
1 Sequential Games
1 Sequential Games We call games where players take turns moving “sequential games”. Sequential games consist of the same elements as normal form games –there are players, rules, outcomes, and payo¤s. However, sequential games have the added element that history of play is now important as players can make decisions conditional on what other players have done. Thus, if two people are playing a game of Chess the second mover is able to observe the …rst mover’s initial move prior to making his initial move. While it is possible to represent sequential games using the strategic (or matrix) form representation of the game it is more instructive at …rst to represent sequential games using a game tree. In addition to the players, actions, outcomes, and payo¤s, the game tree will provide a history of play or a path of play. A very basic example of a sequential game is the Entrant-Incumbent game. The game is described as follows: Consider a game where there is an entrant and an incumbent. The entrant moves …rst and the incumbent observes the entrant’sdecision. The entrant can choose to either enter the market or remain out of the market. If the entrant remains out of the market then the game ends and the entrant receives a payo¤ of 0 while the incumbent receives a payo¤ of 2. If the entrant chooses to enter the market then the incumbent gets to make a choice. The incumbent chooses between …ghting entry or accommodating entry. If the incumbent …ghts the entrant receives a payo¤ of 3 while the incumbent receives a payo¤ of 1. -
Notes on Sequential and Repeated Games
Notes on sequential and repeated games 1 Sequential Move Games Thus far we have examined games in which players make moves simultaneously (or without observing what the other player has done). Using the normal (strategic) form representation of a game we can identify sets of strategies that are best responses to each other (Nash Equilibria). We now focus on sequential games of complete information. We can still use the normal form representation to identify NE but sequential games are richer than that because some players observe other players’decisions before they take action. The fact that some actions are observable may cause some NE of the normal form representation to be inconsistent with what one might think a player would do. Here’sa simple game between an Entrant and an Incumbent. The Entrant moves …rst and the Incumbent observes the Entrant’s action and then gets to make a choice. The Entrant has to decide whether or not he will enter a market or not. Thus, the Entrant’s two strategies are “Enter” or “Stay Out”. If the Entrant chooses “Stay Out” then the game ends. The payo¤s for the Entrant and Incumbent will be 0 and 2 respectively. If the Entrant chooses “Enter” then the Incumbent gets to choose whether or not he will “Fight”or “Accommodate”entry. If the Incumbent chooses “Fight”then the Entrant receives 3 and the Incumbent receives 1. If the Incumbent chooses “Accommodate”then the Entrant receives 2 and the Incumbent receives 1. This game in normal form is Incumbent Fight if Enter Accommodate if Enter . -
SEQUENTIAL GAMES with PERFECT INFORMATION Example
SEQUENTIAL GAMES WITH PERFECT INFORMATION Example 4.9 (page 105) Consider the sequential game given in Figure 4.9. We want to apply backward induction to the tree. 0 Vertex B is owned by player two, P2. The payoffs for P2 are 1 and 3, with 3 > 1, so the player picks R . Thus, the payoffs at B become (0, 3). 00 Next, vertex C is also owned by P2 with payoffs 1 and 0. Since 1 > 0, P2 picks L , and the payoffs are (4, 1). Player one, P1, owns A; the choice of L gives a payoff of 0 and R gives a payoff of 4; 4 > 0, so P1 chooses R. The final payoffs are (4, 1). 0 00 We claim that this strategy profile, { R } for P1 and { R ,L } is a Nash equilibrium. Notice that the 0 00 strategy profile gives a choice at each vertex. For the strategy { R ,L } fixed for P2, P1 has a maximal payoff by choosing { R }, ( 0 00 0 00 π1(R, { R ,L }) = 4 π1(R, { R ,L }) = 4 ≥ 0 00 π1(L, { R ,L }) = 0. 0 00 In the same way, for the strategy { R } fixed for P1, P2 has a maximal payoff by choosing { R ,L }, ( 00 0 00 π2(R, {∗,L }) = 1 π2(R, { R ,L }) = 1 ≥ 00 π2(R, {∗,R }) = 0, where ∗ means choose either L0 or R0. Since no change of choice by a player can increase that players own payoff, the strategy profile is called a Nash equilibrium. Notice that the above strategy profile is also a Nash equilibrium on each branch of the game tree, mainly starting at either B or starting at C. -
Alpha-Beta Pruning
Alpha-Beta Pruning Carl Felstiner May 9, 2019 Abstract This paper serves as an introduction to the ways computers are built to play games. We implement the basic minimax algorithm and expand on it by finding ways to reduce the portion of the game tree that must be generated to find the best move. We tested our algorithms on ordinary Tic-Tac-Toe, Hex, and 3-D Tic-Tac-Toe. With our algorithms, we were able to find the best opening move in Tic-Tac-Toe by only generating 0.34% of the nodes in the game tree. We also explored some mathematical features of Hex and provided proofs of them. 1 Introduction Building computers to play board games has been a focus for mathematicians ever since computers were invented. The first computer to beat a human opponent in chess was built in 1956, and towards the late 1960s, computers were already beating chess players of low-medium skill.[1] Now, it is generally recognized that computers can beat even the most accomplished grandmaster. Computers build a tree of different possibilities for the game and then work backwards to find the move that will give the computer the best outcome. Al- though computers can evaluate board positions very quickly, in a game like chess where there are over 10120 possible board configurations it is impossible for a computer to search through the entire tree. The challenge for the computer is then to find ways to avoid searching in certain parts of the tree. Humans also look ahead a certain number of moves when playing a game, but experienced players already know of certain theories and strategies that tell them which parts of the tree to look. -
The Extensive Form Representation of a Game
The extensive form representation of a game Nodes, information sets Perfect and imperfect information Addition of random moves of nature (to model uncertainty not related with decisions of other players). Mas-Colell, pp. 221-227, full description in page 227. 1 The Prisoner's dilemma Player 2 confess don't confess Player 1 confess 2.2 10.0 don't confess 0.10 6.6 One extensive form representation of this normal form game is: 2 Player 1 confess don't confess Player 2 Player 2 confessdon't confess confess don't confess 2 10 0 6 payoffs 2 0 10 6 3 Both players decide whether to confess simultaneously (or more exactly, each player decides without any information about the decision of the other player). This is a game of imperfect information: The information set of player 2 contains more than one decision node. 4 The following extensive form represents the same normal form: Player 2 confess don't confess Player 1 Player 1 confessdon't confess confess don't confess 5 Consider now the following extensive form game: Player 1 confess don't confess Player 2 Player 2 confessdon't confess confess don't confess 2 10 6 0 6 payoffs 2 0 10 6 Now player 2 observes the behavior of player 1. This is a game of perfect information. Strategies of player 2: For instance "If player 1 confesses, I…; if he does not confess, then I…" Player 2 has 4 strategies: (c, c), (c, d), (d, c), (d, d) The normal form of this game is: Player 2 c, c c, d d, c d, d Player 1 confess 2.2 2.2 10.0 10.0 don't confess 0.10 6.6 0.10 6.6 There is only one NE, - strategy confess for player 1 and - strategy (c, c) for player 2. -
What Is Local Optimality in Nonconvex-Nonconcave Minimax Optimization?
What is Local Optimality in Nonconvex-Nonconcave Minimax Optimization? Chi Jin Praneeth Netrapalli University of California, Berkeley Microsoft Research, India [email protected] [email protected] Michael I. Jordan University of California, Berkeley [email protected] August 18, 2020 Abstract Minimax optimization has found extensive application in modern machine learning, in settings such as generative adversarial networks (GANs), adversarial training and multi-agent reinforcement learning. As most of these applications involve continuous nonconvex-nonconcave formulations, a very basic question arises—“what is a proper definition of local optima?” Most previous work answers this question using classical notions of equilibria from simultaneous games, where the min-player and the max-player act simultaneously. In contrast, most applications in machine learning, including GANs and adversarial training, correspond to sequential games, where the order of which player acts first is crucial (since minimax is in general not equal to maximin due to the nonconvex-nonconcave nature of the problems). The main contribution of this paper is to propose a proper mathematical definition of local optimality for this sequential setting—local minimax—as well as to present its properties and existence results. Finally, we establish a strong connection to a basic local search algorithm—gradient descent ascent (GDA)—under mild conditions, all stable limit points of GDA are exactly local minimax points up to some degenerate points. 1 Introduction arXiv:1902.00618v3 [cs.LG] 15 Aug 2020 Minimax optimization refers to problems of two agents—one agent tries to minimize the payoff function f : X × Y ! R while the other agent tries to maximize it. -
Combinatorial Game Theory
Combinatorial Game Theory Aaron N. Siegel Graduate Studies MR1EXLIQEXMGW Volume 146 %QIVMGER1EXLIQEXMGEP7SGMIX] Combinatorial Game Theory https://doi.org/10.1090//gsm/146 Combinatorial Game Theory Aaron N. Siegel Graduate Studies in Mathematics Volume 146 American Mathematical Society Providence, Rhode Island EDITORIAL COMMITTEE David Cox (Chair) Daniel S. Freed Rafe Mazzeo Gigliola Staffilani 2010 Mathematics Subject Classification. Primary 91A46. For additional information and updates on this book, visit www.ams.org/bookpages/gsm-146 Library of Congress Cataloging-in-Publication Data Siegel, Aaron N., 1977– Combinatorial game theory / Aaron N. Siegel. pages cm. — (Graduate studies in mathematics ; volume 146) Includes bibliographical references and index. ISBN 978-0-8218-5190-6 (alk. paper) 1. Game theory. 2. Combinatorial analysis. I. Title. QA269.S5735 2013 519.3—dc23 2012043675 Copying and reprinting. Individual readers of this publication, and nonprofit libraries acting for them, are permitted to make fair use of the material, such as to copy a chapter for use in teaching or research. Permission is granted to quote brief passages from this publication in reviews, provided the customary acknowledgment of the source is given. Republication, systematic copying, or multiple reproduction of any material in this publication is permitted only under license from the American Mathematical Society. Requests for such permission should be addressed to the Acquisitions Department, American Mathematical Society, 201 Charles Street, Providence, Rhode Island 02904-2294 USA. Requests can also be made by e-mail to [email protected]. c 2013 by the American Mathematical Society. All rights reserved. The American Mathematical Society retains all rights except those granted to the United States Government. -
Extensive Form Games and Backward Induction
Recap Perfect-Information Extensive-Form Games Extensive Form Games and Backward Induction ISCI 330 Lecture 11 February 13, 2007 Extensive Form Games and Backward Induction ISCI 330 Lecture 11, Slide 1 Recap Perfect-Information Extensive-Form Games Lecture Overview Recap Perfect-Information Extensive-Form Games Extensive Form Games and Backward Induction ISCI 330 Lecture 11, Slide 2 I The extensive form is an alternative representation that makes the temporal structure explicit. I Two variants: I perfect information extensive-form games I imperfect-information extensive-form games Recap Perfect-Information Extensive-Form Games Introduction I The normal form game representation does not incorporate any notion of sequence, or time, of the actions of the players Extensive Form Games and Backward Induction ISCI 330 Lecture 11, Slide 3 I Two variants: I perfect information extensive-form games I imperfect-information extensive-form games Recap Perfect-Information Extensive-Form Games Introduction I The normal form game representation does not incorporate any notion of sequence, or time, of the actions of the players I The extensive form is an alternative representation that makes the temporal structure explicit. Extensive Form Games and Backward Induction ISCI 330 Lecture 11, Slide 3 Recap Perfect-Information Extensive-Form Games Introduction I The normal form game representation does not incorporate any notion of sequence, or time, of the actions of the players I The extensive form is an alternative representation that makes the temporal -
Finding Strategic Game Equivalent of an Extensive Form Game
Finding Strategic Game Equivalent of an Extensive Form Game • In an extensive form game, a strategy for a player should specify what action the player will choose at each information set. That is, a strategy is a complete plan for playing a game for a particular player. • Therefore to find the strategic game equivalent of an extensive form game we should follow these steps: 1. First we need to find all strategies for every player. To do that first we find all information sets for every player (including the singleton information sets). If there are many of them we may label them (such as Player 1’s 1st info. set, Player 1’s 2nd info. set, etc.) 2. A strategy should specify what action the player will take in every information set which belongs to him/her. We find all combinations of actions the player can take at these information sets. For example if a player has 3 information sets where • the first one has 2 actions, • the second one has 2 actions, and • the third one has 3 actions then there will be a total of 2 × 2 × 3 = 12 strategies for this player. 3. Once the strategies are obtained for every player the next step is finding the payoffs. For every strategy profile we find the payoff vector that would be obtained in case these strategies are played in the extensive form game. Example: Let’s find the strategic game equivalent of the following 3-level centipede game. (3,1)r (2,3)r (6,2)r (4,6)r (12,4)r (8,12)r S S S S S S r 1 2 1 2 1 2 (24,8) C C C C C C 1. -
Game Tree Search
CSC384: Introduction to Artificial Intelligence Game Tree Search • Chapter 5.1, 5.2, 5.3, 5.6 cover some of the material we cover here. Section 5.6 has an interesting overview of State-of-the-Art game playing programs. • Section 5.5 extends the ideas to games with uncertainty (We won’t cover that material but it makes for interesting reading). Fahiem Bacchus, CSC384 Introduction to Artificial Intelligence, University of Toronto 1 Generalizing Search Problem • So far: our search problems have assumed agent has complete control of environment • State does not change unless the agent (robot) changes it. • All we need to compute is a single path to a goal state. • Assumption not always reasonable • Stochastic environment (e.g., the weather, traffic accidents). • Other agents whose interests conflict with yours • Search can find a path to a goal state, but the actions might not lead you to the goal as the state can be changed by other agents (nature or other intelligent agents) Fahiem Bacchus, CSC384 Introduction to Artificial Intelligence, University of Toronto 2 Generalizing Search Problem •We need to generalize our view of search to handle state changes that are not in the control of the agent. •One generalization yields game tree search • Agent and some other agents. • The other agents are acting to maximize their profits • this might not have a positive effect on your profits. Fahiem Bacchus, CSC384 Introduction to Artificial Intelligence, University of Toronto 3 General Games •What makes something a game? • There are two (or more) agents making changes to the world (the state) • Each agent has their own interests • e.g., each agent has a different goal; or assigns different costs to different paths/states • Each agent tries to alter the world so as to best benefit itself. -
Revealed Preferences of Individual Players in Sequential Games
Revealed Preferences of Individual Players in Sequential Games Hiroki Nishimura∗ January 14, 2021 Abstract This paper studies rational choice behavior of a player in sequential games of perfect and complete information without an assumption that the other players who join the same games are rational. The model of individually rational choice is defined through a decomposition of the behavioral norm assumed in the subgame perfect equilibria, and we propose a set of axioms on collective choice behavior that characterize the individual rationality obtained as such. As the choice of subgame perfect equilibrium paths is a special case where all players involved in the choice environment are each individually rational, the paper offers testable characterizations of both individual rationality and collective rationality in sequential games. JEL Classification: C72, D70. Keywords: Revealed preference, individual rationality, subgame perfect equilibrium. 1 Introduction A game is a description of strategic interactions among players. The players involved in a game simultaneously or dynamically choose their actions, and their payoffs are determined by a profile of chosen actions. A number of solution concepts for games, such as the Nash equilibrium or the subgame perfect equilibrium, have been developed in the literature in order to study collective choice of actions made by the players. In turn, these solution concepts are widely applied in economic analysis to provide credible predictions for the choice of economic agents who reside in situations approximated by games. ∗Department of Economics, University of California Riverside. Email: [email protected]. I am indebted to Efe A. Ok for his continuous support and encouragement. I would also like to thank Indrajit Ray, anonymous referees, and an associate editor for their thoughtful suggestions. -
Combinatorial Game Theory: an Introduction to Tree Topplers
Georgia Southern University Digital Commons@Georgia Southern Electronic Theses and Dissertations Graduate Studies, Jack N. Averitt College of Fall 2015 Combinatorial Game Theory: An Introduction to Tree Topplers John S. Ryals Jr. Follow this and additional works at: https://digitalcommons.georgiasouthern.edu/etd Part of the Discrete Mathematics and Combinatorics Commons, and the Other Mathematics Commons Recommended Citation Ryals, John S. Jr., "Combinatorial Game Theory: An Introduction to Tree Topplers" (2015). Electronic Theses and Dissertations. 1331. https://digitalcommons.georgiasouthern.edu/etd/1331 This thesis (open access) is brought to you for free and open access by the Graduate Studies, Jack N. Averitt College of at Digital Commons@Georgia Southern. It has been accepted for inclusion in Electronic Theses and Dissertations by an authorized administrator of Digital Commons@Georgia Southern. For more information, please contact [email protected]. COMBINATORIAL GAME THEORY: AN INTRODUCTION TO TREE TOPPLERS by JOHN S. RYALS, JR. (Under the Direction of Hua Wang) ABSTRACT The purpose of this thesis is to introduce a new game, Tree Topplers, into the field of Combinatorial Game Theory. Before covering the actual material, a brief background of Combinatorial Game Theory is presented, including how to assign advantage values to combinatorial games, as well as information on another, related game known as Domineering. Please note that this document contains color images so please keep that in mind when printing. Key Words: combinatorial game theory, tree topplers, domineering, hackenbush 2009 Mathematics Subject Classification: 91A46 COMBINATORIAL GAME THEORY: AN INTRODUCTION TO TREE TOPPLERS by JOHN S. RYALS, JR. B.S. in Applied Mathematics A Thesis Submitted to the Graduate Faculty of Georgia Southern University in Partial Fulfillment of the Requirement for the Degree MASTER OF SCIENCE STATESBORO, GEORGIA 2015 c 2015 JOHN S.