DOCSLIB.ORG

570: Minimax Sample Complexity for Turn-Based Stochastic Game

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
570: Minimax Sample Complexity for Turn-Based Stochastic Game Minimax Sample Complexity for Turn-based Stochastic Game Qiwen Cui1 Lin F. Yang2 1School of Mathematical Sciences, Peking University 2Electrical and Computer Engineering Department, University of California, Los Angeles Abstract guarantees are rather rare due to complex interaction be- tween agents that makes the problem considerably harder than single agent reinforcement learning. This is also known The empirical success of multi-agent reinforce- as non-stationarity in MARL, which means when multi- ment learning is encouraging, while few theoret- ple agents alter their strategies based on samples collected ical guarantees have been revealed. In this work, from previous strategy, the system becomes non-stationary we prove that the plug-in solver approach, proba- for each agent and the improvement can not be guaranteed. bly the most natural reinforcement learning algo- One fundamental question in MBRL is that how to design rithm, achieves minimax sample complexity for efficient algorithms to overcome non-stationarity. turn-based stochastic game (TBSG). Specifically, we perform planning in an empirical TBSG by Two-players turn-based stochastic game (TBSG) is a two- utilizing a ‘simulator’ that allows sampling from agents generalization of Markov decision process (MDP), arbitrary state-action pair. We show that the em- where two agents choose actions in turn and one agent wants pirical Nash equilibrium strategy is an approxi- to maximize the total reward while the other wants to min- mate Nash equilibrium strategy in the true TBSG imize it. As a zero-sum game, TBSG is known to have and give both problem-dependent and problem- Nash equilibrium strategy [Shapley, 1953], which means independent bound. We develop reward perturba- there exists a strategy pair that both agents will not benefit tion techniques to tackle the non-stationarity in from changing its strategy alone, and our target is to find the game and Taylor-expansion-type analysis to the (approximate) Nash equilibrium strategy. Dynamic pro- improve the dependence on approximation error. gramming type algorithms is a basic but powerful approach With these novel techniques, we prove the mini- to solve TBSG. Strategy iteration, a counterpart of policy max sample complexity of turn-based stochastic iteration in MDP, is known to be a polynomial complex- game. ity algorithm to solve TBSG with known transition kernel [Hansen et al., 2013, Jia et al., 2020]. However, these al- gorithms suffer from high computational cost and require full knowledge of the transition dynamic. Reinforcement 1 INTRODUCTION learning is a promising alternative, which has demonstrated its potential in solving sequential decision making problems. Reinforcement learning (RL) [Sutton and Barto, 2018], a However, non-stationarity pose a large obstacle against the framework where agents learn to make sequential decisions convergence of model-free algorithms. To tackle this chal- in an unknown environment, has received tremendous at- lenge, sophisticated algorithms have been proposed [Jia tention. An interesting branch is multi-agent reinforcement et al., 2019, Sidford et al., 2020]. In this work, we focus learning (MARL) that multiple agents exist and they interact on another promising but insufficiently developed method, with the environment as well as the others, which bridges model-based algorithms, where agents learn the model of RL and game theory. In general, each agent attempts to max- the environment and plan in the empirical model. imize its own reward by utilizing the data collected from the environment and also inferring other agents’ strategies. Model-based RL is long perceived to be the cure to the sam- Impressive successes have been achieved in games such ple inefficiency in RL, which is also justified by empirical as backgammon [Tesauro, 1995], Go [Silver et al., 2017] advances [Kaiser et al., 2019, Wang et al., 2019]. However, and strategy games [Ye et al., 2020]. MARL has shown the theoretical understanding of model-based RL is still the potential for superhuman performance, but theoretical far from complete. Recently, a line of research focuses on Accepted for the 37th Conference on Uncertainty in Artificial Intelligence (UAI 2021). analyzing model-based algorithm under generative model TBSG, which is inspired by the technique developed in [Li setting [Azar et al., 2013, Agarwal et al., 2019, Cui and et al., 2020]. The key observation is that if r(s; a) increases, Yang, 2020, Zhang et al., 2020, Li et al., 2020], In this work, Q∗(s; a) increases much faster than Q∗(s; a0) for a0 6= a, we aim to prove that the simplest model-based algorithm, thus the perturbed TBSG enjoys a suboptimality gap with plug-in solver approach, enjoys minimax sample complexity high probability. Different from the usage in the first part, for TBSG by utilizing novel absorbing TBSG and reward here the suboptimality gap is used to ensure the empirical perturbation techniques. Nash equilibrium strategy lies in a finite set so that union bound can be applied. Combining the reward perturbation Specifically, we assume that we have access to a generative technique with the absorbing TBSG technique, we are able model oracle [Kakade et al., 2003], which allows sampling to prove more subtle concentration arguments and finally from arbitrary state-action pair. It is known that exploration show that the empirical Nash equilibrium strategy is an - and exploitation tradeoff is simplified under this setting, i.e., approximate Nash equilibrium strategy in the true TBSG sampling from each state-action pair equally can already with minimax sample complexity O(jSjjAj(1 − γ)−3−2) yield the minimax sample complexity result. With the gen- e for 2 (0; (1 − γ)−1]. In addition, we can recover the erative model, the most intuitive approach is to learn an em- problem-dependent bound by simply setting = ∆. pirical TBSG and then use a planning algorithm to find the empirical Nash equilibrium strategy as the solution, which Recently Zhang et al.[2020] proposes a similar result for separates learning and planning. This kind of algorithm simultaneous stochastic game. However, their result requires is known as the ‘plug-in solver approach’, which means a planning oracle for regularized stochastic game, which arbitrary planning algorithm can be used. We will show is computationally intractable. Our algorithm only requires that this simple plug-in solver approach enjoys minimax planning in a standard turn-based stochastic game, which sample complexity. For MDP, this line of research is studied can be performed efficiently by utilizing strategy iteration in the seminal work by Azar et al.[2013]. In the recent [Hansen et al., 2013] or learning algorithms [Sidford et al., work [Li et al., 2020], they almost completely solves this 2020]. In addition, their sample complexity result N = −3 −2 − 1 problem by proving the minimax complexity with full range Oe(jSjjAj(1 − γ) ) only holds for 2 (0; (1 − γ) 2 ], of error . However, the result for TBSG is still unknown which means N = Oe(jSjjAj(1 − γ)−2). our result fills the and largely due to interaction between agents. In this work, blank in the sample region Oe(jSjjAj(1 − γ)−1) ≤ N ≤ we give the sample complexity upper bounds of TBSG for −2 Oe(jSjjAj(1 − γ) ). Note that the lower bound [Azar et al., both problem-dependent and problem-independent cases. 2013] indicates that N = O(jSjjAj(1−γ)−1) is insufficient Suboptimality gap is a widely studied notion in the bandit to learn a policy. As both parts of our analysis heavily rely theory, which stands for a constant gap between the optimal on the suboptimality gap, we hope our work can provide arm and second optimal arm. This notion has received in- more understanding about this notion and TBSG. creasing focus in MDP and we generalize it to TBSG. To start with, we prove that Oe(jSjjAj(1 − γ)−3∆−2) samples are enough to recover Nash equilibrium strategy accurately − 1 for ∆ 2 (0; (1 − γ) 2 ], where S is the state space, A is the action space, γ is the discount factor and ∆ is the 2 PRELIMINARY suboptimality gap. The key analysis tool is the absorbing TBSG technique that helps to show the empirical optimal Q value is close to true optimal Q value. With the absorbing Turn-based Stochastic Game Turn-based two-player TBSG, the statistical dependence on Pb(s; a) is moved to a zero-sum stochastic game (TBSG) is a generalized version single parameter u, which can be approximated by an cover- of Markov decison process (MDP) which includes two play- ing argument. Therefore, standard concentration arguments ers competing with each other. Player 1 aims to maximize combined with union bound can be applied. Suboptimality the total reward while player 2 aims to minimize it. TBSG is gap plays an critical role in showing the empirical Nash equi- described by the tuple G = (S = Smax [Smin; A; P; r; γ), librium strategy is exactly the same as true Nash equilibrium where Smax is the state space of player 1, Smin is the strategy. state space of player 2, A is the action space of both play- ers, P 2 RjS||A|×|Sj is the transition probability matrix, The main contribution is in the second part, where we give r 2 RjSjjAj is the reward vector and γ is the discount factor. our problem-independent bound which meets the existing In each step, only one player plays an action and a transition lower bound O(jSjjAj(1 − γ)−3−2) [Azar et al., 2013, happens. For instance, if the state s 2 Smax, player 1 needs Sidford et al., 2020]. Note that the problem-dependent re- to select an action a 2 A. After selecting the action, the sult becomes meaningless when the suboptimality gap is 0 state will transit to s 2 Smin according to the distribution sufficiently small and also the gap may not even exist.
Load more

Recommended publications
  • Labsi Working Papers
    UNIVERSITY OF SIENA S.N. O’ H iggins Arturo Palomba Patrizia Sbriglia Second Mover Advantage and Bertrand Dynamic Competition: An Experiment May 2010 LABSI WORKING PAPERS N. 28/2010 SECOND MOVER ADVANTAGE AND BERTRAND DYNAMIC COMPETITION: AN EXPERIMENT § S.N. O’Higgins University of Salerno [email protected] Arturo Palomba University of Naples II [email protected] Patrizia Sbriglia §§ University of Naples II [email protected] Abstract In this paper we provide an experimental test of a dynamic Bertrand duopolistic model, where firms move sequentially and their informational setting varies across different designs. Our experiment is composed of three treatments. In the first treatment, subjects receive information only on the costs and demand parameters and on the price’ choices of their opponent in the market in which they are positioned (matching is fixed); in the second and third treatments, subjects are also informed on the behaviour of players who are not directly operating in their market. Our aim is to study whether the individual behaviour and the process of equilibrium convergence are affected by the specific informational setting adopted. In all treatments we selected students who had previously studied market games and industrial organization, conjecturing that the specific participants’ expertise decreased the chances of imitation in treatment II and III. However, our results prove the opposite: the extra information provided in treatment II and III strongly affects the long run convergence to the market equilibrium. In fact, whilst in the first session, a high proportion of markets converge to the Nash-Bertrand symmetric solution, we observe that a high proportion of markets converge to more collusive outcomes in treatment II and more competitive outcomes in treatment III.
    [Show full text]
  • 1 Sequential Games
    1 Sequential Games We call games where players take turns moving “sequential games”. Sequential games consist of the same elements as normal form games –there are players, rules, outcomes, and payo¤s. However, sequential games have the added element that history of play is now important as players can make decisions conditional on what other players have done. Thus, if two people are playing a game of Chess the second mover is able to observe the …rst mover’s initial move prior to making his initial move. While it is possible to represent sequential games using the strategic (or matrix) form representation of the game it is more instructive at …rst to represent sequential games using a game tree. In addition to the players, actions, outcomes, and payo¤s, the game tree will provide a history of play or a path of play. A very basic example of a sequential game is the Entrant-Incumbent game. The game is described as follows: Consider a game where there is an entrant and an incumbent. The entrant moves …rst and the incumbent observes the entrant’sdecision. The entrant can choose to either enter the market or remain out of the market. If the entrant remains out of the market then the game ends and the entrant receives a payo¤ of 0 while the incumbent receives a payo¤ of 2. If the entrant chooses to enter the market then the incumbent gets to make a choice. The incumbent chooses between …ghting entry or accommodating entry. If the incumbent …ghts the entrant receives a payo¤ of 3 while the incumbent receives a payo¤ of 1.
    [Show full text]
  • Finitely Repeated Games
    Repeated games 1: Finite repetition Universidad Carlos III de Madrid 1 Finitely repeated games • A finitely repeated game is a dynamic game in which a simultaneous game (the stage game) is played finitely many times, and the result of each stage is observed before the next one is played. • Example: Play the prisoners’ dilemma several times. The stage game is the simultaneous prisoners’ dilemma game. 2 Results • If the stage game (the simultaneous game) has only one NE the repeated game has only one SPNE: In the SPNE players’ play the strategies in the NE in each stage. • If the stage game has 2 or more NE, one can find a SPNE where, at some stage, players play a strategy that is not part of a NE of the stage game. 3 The prisoners’ dilemma repeated twice • Two players play the same simultaneous game twice, at ! = 1 and at ! = 2. • After the first time the game is played (after ! = 1) the result is observed before playing the second time. • The payoff in the repeated game is the sum of the payoffs in each stage (! = 1, ! = 2) • Which is the SPNE? Player 2 D C D 1 , 1 5 , 0 Player 1 C 0 , 5 4 , 4 4 The prisoners’ dilemma repeated twice Information sets? Strategies? 1 .1 5 for each player 2" for each player D C E.g.: (C, D, D, C, C) Subgames? 2.1 5 D C D C .2 1.3 1.5 1 1.4 D C D C D C D C 2.2 2.3 2 .4 2.5 D C D C D C D C D C D C D C D C 1+1 1+5 1+0 1+4 5+1 5+5 5+0 5+4 0+1 0+5 0+0 0+4 4+1 4+5 4+0 4+4 1+1 1+0 1+5 1+4 0+1 0+0 0+5 0+4 5+1 5+0 5+5 5+4 4+1 4+0 4+5 4+4 The prisoners’ dilemma repeated twice Let’s find the NE in the subgames.
    [Show full text]
  • Stochastic Game Theory Applications for Power Management in Cognitive Networks
    STOCHASTIC GAME THEORY APPLICATIONS FOR POWER MANAGEMENT IN COGNITIVE NETWORKS A thesis submitted to Kent State University in partial fulfillment of the requirements for the degree of Master of Digital Science by Sham Fung Feb 2014 Thesis written by Sham Fung M.D.S, Kent State University, 2014 Approved by Advisor , Director, School of Digital Science ii TABLE OF CONTENTS LIST OF FIGURES . vi LIST OF TABLES . vii Acknowledgments . viii Dedication . ix 1 BACKGROUND OF COGNITIVE NETWORK . 1 1.1 Motivation and Requirements . 1 1.2 Definition of Cognitive Networks . 2 1.3 Key Elements of Cognitive Networks . 3 1.3.1 Learning and Reasoning . 3 1.3.2 Cognitive Network as a Multi-agent System . 4 2 POWER ALLOCATION IN COGNITIVE NETWORKS . 5 2.1 Overview . 5 2.2 Two Communication Paradigms . 6 2.3 Power Allocation Scheme . 8 2.3.1 Interference Constraints for Primary Users . 8 2.3.2 QoS Constraints for Secondary Users . 9 2.4 Related Works on Power Management in Cognitive Networks . 10 iii 3 GAME THEORY|PRELIMINARIES AND RELATED WORKS . 12 3.1 Non-cooperative Game . 14 3.1.1 Nash Equilibrium . 14 3.1.2 Other Equilibriums . 16 3.2 Cooperative Game . 16 3.2.1 Bargaining Game . 17 3.2.2 Coalition Game . 17 3.2.3 Solution Concept . 18 3.3 Stochastic Game . 19 3.4 Other Types of Games . 20 4 GAME THEORETICAL APPROACH ON POWER ALLOCATION . 23 4.1 Two Fundamental Types of Utility Functions . 23 4.1.1 QoS-Based Game . 23 4.1.2 Linear Pricing Game .
    [Show full text]
  • Alpha-Beta Pruning
    Alpha-Beta Pruning Carl Felstiner May 9, 2019 Abstract This paper serves as an introduction to the ways computers are built to play games. We implement the basic minimax algorithm and expand on it by finding ways to reduce the portion of the game tree that must be generated to find the best move. We tested our algorithms on ordinary Tic-Tac-Toe, Hex, and 3-D Tic-Tac-Toe. With our algorithms, we were able to find the best opening move in Tic-Tac-Toe by only generating 0.34% of the nodes in the game tree. We also explored some mathematical features of Hex and provided proofs of them. 1 Introduction Building computers to play board games has been a focus for mathematicians ever since computers were invented. The first computer to beat a human opponent in chess was built in 1956, and towards the late 1960s, computers were already beating chess players of low-medium skill.[1] Now, it is generally recognized that computers can beat even the most accomplished grandmaster. Computers build a tree of different possibilities for the game and then work backwards to find the move that will give the computer the best outcome. Al- though computers can evaluate board positions very quickly, in a game like chess where there are over 10120 possible board configurations it is impossible for a computer to search through the entire tree. The challenge for the computer is then to find ways to avoid searching in certain parts of the tree. Humans also look ahead a certain number of moves when playing a game, but experienced players already know of certain theories and strategies that tell them which parts of the tree to look.
    [Show full text]
  • Chapter 16 Oligopoly and Game Theory Oligopoly Oligopoly
    Chapter 16 “Game theory is the study of how people Oligopoly behave in strategic situations. By ‘strategic’ we mean a situation in which each person, when deciding what actions to take, must and consider how others might respond to that action.” Game Theory Oligopoly Oligopoly • “Oligopoly is a market structure in which only a few • “Figuring out the environment” when there are sellers offer similar or identical products.” rival firms in your market, means guessing (or • As we saw last time, oligopoly differs from the two ‘ideal’ inferring) what the rivals are doing and then cases, perfect competition and monopoly. choosing a “best response” • In the ‘ideal’ cases, the firm just has to figure out the environment (prices for the perfectly competitive firm, • This means that firms in oligopoly markets are demand curve for the monopolist) and select output to playing a ‘game’ against each other. maximize profits • To understand how they might act, we need to • An oligopolist, on the other hand, also has to figure out the understand how players play games. environment before computing the best output. • This is the role of Game Theory. Some Concepts We Will Use Strategies • Strategies • Strategies are the choices that a player is allowed • Payoffs to make. • Sequential Games •Examples: • Simultaneous Games – In game trees (sequential games), the players choose paths or branches from roots or nodes. • Best Responses – In matrix games players choose rows or columns • Equilibrium – In market games, players choose prices, or quantities, • Dominated strategies or R and D levels. • Dominant Strategies. – In Blackjack, players choose whether to stay or draw.
    [Show full text]
  • Deterministic and Stochastic Prisoner's Dilemma Games: Experiments in Interdependent Security
    NBER TECHNICAL WORKING PAPER SERIES DETERMINISTIC AND STOCHASTIC PRISONER'S DILEMMA GAMES: EXPERIMENTS IN INTERDEPENDENT SECURITY Howard Kunreuther Gabriel Silvasi Eric T. Bradlow Dylan Small Technical Working Paper 341 http://www.nber.org/papers/t0341 NATIONAL BUREAU OF ECONOMIC RESEARCH 1050 Massachusetts Avenue Cambridge, MA 02138 August 2007 We appreciate helpful discussions in designing the experiments and comments on earlier drafts of this paper by Colin Camerer, Vince Crawford, Rachel Croson, Robyn Dawes, Aureo DePaula, Geoff Heal, Charles Holt, David Krantz, Jack Ochs, Al Roth and Christian Schade. We also benefited from helpful comments from participants at the Workshop on Interdependent Security at the University of Pennsylvania (May 31-June 1 2006) and at the Workshop on Innovation and Coordination at Humboldt University (December 18-20, 2006). We thank George Abraham and Usmann Hassan for their help in organizing the data from the experiments. Support from NSF Grant CMS-0527598 and the Wharton Risk Management and Decision Processes Center is gratefully acknowledged. The views expressed herein are those of the author(s) and do not necessarily reflect the views of the National Bureau of Economic Research. © 2007 by Howard Kunreuther, Gabriel Silvasi, Eric T. Bradlow, and Dylan Small. All rights reserved. Short sections of text, not to exceed two paragraphs, may be quoted without explicit permission provided that full credit, including © notice, is given to the source. Deterministic and Stochastic Prisoner's Dilemma Games: Experiments in Interdependent Security Howard Kunreuther, Gabriel Silvasi, Eric T. Bradlow, and Dylan Small NBER Technical Working Paper No. 341 August 2007 JEL No. C11,C12,C22,C23,C73,C91 ABSTRACT This paper examines experiments on interdependent security prisoner's dilemma games with repeated play.
    [Show full text]
  • Game Theory with Translucent Players
    Game Theory with Translucent Players Joseph Y. Halpern and Rafael Pass∗ Cornell University Department of Computer Science Ithaca, NY, 14853, U.S.A. e-mail: fhalpern,[email protected] November 27, 2012 Abstract A traditional assumption in game theory is that players are opaque to one another—if a player changes strategies, then this change in strategies does not affect the choice of other players’ strategies. In many situations this is an unrealistic assumption. We develop a framework for reasoning about games where the players may be translucent to one another; in particular, a player may believe that if she were to change strategies, then the other player would also change strategies. Translucent players may achieve significantly more efficient outcomes than opaque ones. Our main result is a characterization of strategies consistent with appro- priate analogues of common belief of rationality. Common Counterfactual Belief of Rationality (CCBR) holds if (1) everyone is rational, (2) everyone counterfactually believes that everyone else is rational (i.e., all players i be- lieve that everyone else would still be rational even if i were to switch strate- gies), (3) everyone counterfactually believes that everyone else is rational, and counterfactually believes that everyone else is rational, and so on. CCBR characterizes the set of strategies surviving iterated removal of minimax dom- inated strategies: a strategy σi is minimax dominated for i if there exists a 0 0 0 strategy σ for i such that min 0 u (σ ; µ ) > max u (σ ; µ ). i µ−i i i −i µ−i i i −i ∗Halpern is supported in part by NSF grants IIS-0812045, IIS-0911036, and CCF-1214844, by AFOSR grant FA9550-08-1-0266, and by ARO grant W911NF-09-1-0281.
    [Show full text]
  • Stochastic Games with Hidden States∗
    Stochastic Games with Hidden States¤ Yuichi Yamamoto† First Draft: March 29, 2014 This Version: January 14, 2015 Abstract This paper studies infinite-horizon stochastic games in which players ob- serve noisy public information about a hidden state each period. We find that if the game is connected, the limit feasible payoff set exists and is invariant to the initial prior about the state. Building on this invariance result, we pro- vide a recursive characterization of the equilibrium payoff set and establish the folk theorem. We also show that connectedness can be replaced with an even weaker condition, called asymptotic connectedness. Asymptotic con- nectedness is satisfied for generic signal distributions, if the state evolution is irreducible. Journal of Economic Literature Classification Numbers: C72, C73. Keywords: stochastic game, hidden state, connectedness, stochastic self- generation, folk theorem. ¤The author thanks Naoki Aizawa, Drew Fudenberg, Johannes Horner,¨ Atsushi Iwasaki, Michi- hiro Kandori, George Mailath, Takeaki Sunada, and Masatoshi Tsumagari for helpful conversa- tions, and seminar participants at various places. †Department of Economics, University of Pennsylvania. Email: [email protected] 1 1 Introduction When agents have a long-run relationship, underlying economic conditions may change over time. A leading example is a repeated Bertrand competition with stochastic demand shocks. Rotemberg and Saloner (1986) explore optimal col- lusive pricing when random demand shocks are i.i.d. each period. Haltiwanger and Harrington (1991), Kandori (1991), and Bagwell and Staiger (1997) further extend the analysis to the case in which demand fluctuations are cyclic or persis- tent. One of the crucial assumptions of these papers is that demand shocks are publicly observable before firms make their decisions in each period.
    [Show full text]
  • Cooperation Spillovers in Coordination Games*
    Cooperation Spillovers in Coordination Games* Timothy N. Casona, Anya Savikhina, and Roman M. Sheremetab aDepartment of Economics, Krannert School of Management, Purdue University, 403 W. State St., West Lafayette, IN 47906-2056, U.S.A. bArgyros School of Business and Economics, Chapman University, One University Drive, Orange, CA 92866, U.S.A. November 2009 Abstract Motivated by problems of coordination failure observed in weak-link games, we experimentally investigate behavioral spillovers for order-statistic coordination games. Subjects play the minimum- and median-effort coordination games simultaneously and sequentially. The results show the precedent for cooperative behavior spills over from the median game to the minimum game when the games are played sequentially. Moreover, spillover occurs even when group composition changes, although the effect is not as strong. We also find that the precedent for uncooperative behavior does not spill over from the minimum game to the median game. These findings suggest guidelines for increasing cooperative behavior within organizations. JEL Classifications: C72, C91 Keywords: coordination, order-statistic games, experiments, cooperation, minimum game, behavioral spillover Corresponding author: Timothy Cason, [email protected] * We thank Yan Chen, David Cooper, John Duffy, Vai-Lam Mui, seminar participants at Purdue University, and participants at Economic Science Association conferences for helpful comments. Any remaining errors are ours. 1. Introduction Coordination failure is often the reason for the inefficient performance of many groups, ranging from small firms to entire economies. When agents’ actions have strategic interdependence, even when they succeed in coordinating they may be “trapped” in an equilibrium that is objectively inferior to other equilibria. Coordination failure and inefficient coordination has been an important theme across a variety of fields in economics, ranging from development and macroeconomics to mechanism design for overcoming moral hazard in teams.
    [Show full text]
  • The Effective Minimax Value of Asynchronously Repeated Games*
    Int J Game Theory (2003) 32: 431–442 DOI 10.1007/s001820300161 The effective minimax value of asynchronously repeated games* Kiho Yoon Department of Economics, Korea University, Anam-dong, Sungbuk-gu, Seoul, Korea 136-701 (E-mail: [email protected]) Received: October 2001 Abstract. We study the effect of asynchronous choice structure on the possi- bility of cooperation in repeated strategic situations. We model the strategic situations as asynchronously repeated games, and define two notions of effec- tive minimax value. We show that the order of players’ moves generally affects the effective minimax value of the asynchronously repeated game in significant ways, but the order of moves becomes irrelevant when the stage game satisfies the non-equivalent utilities (NEU) condition. We then prove the Folk Theorem that a payoff vector can be supported as a subgame perfect equilibrium out- come with correlation device if and only if it dominates the effective minimax value. These results, in particular, imply both Lagunoff and Matsui’s (1997) result and Yoon (2001)’s result on asynchronously repeated games. Key words: Effective minimax value, folk theorem, asynchronously repeated games 1. Introduction Asynchronous choice structure in repeated strategic situations may affect the possibility of cooperation in significant ways. When players in a repeated game make asynchronous choices, that is, when players cannot always change their actions simultaneously in each period, it is quite plausible that they can coordinate on some particular actions via short-run commitment or inertia to render unfavorable outcomes infeasible. Indeed, Lagunoff and Matsui (1997) showed that, when a pure coordination game is repeated in a way that only *I thank three anonymous referees as well as an associate editor for many helpful comments and suggestions.
    [Show full text]
  • Formation of Coalition Structures As a Non-Cooperative Game
    Formation of coalition structures as a non-cooperative game Dmitry Levando, ∗ Abstract We study coalition structure formation with intra and inter-coalition externalities in the introduced family of nested non-cooperative si- multaneous finite games. A non-cooperative game embeds a coalition structure formation mechanism, and has two outcomes: an alloca- tion of players over coalitions and a payoff for every player. Coalition structures of a game are described by Young diagrams. They serve to enumerate coalition structures and allocations of players over them. For every coalition structure a player has a set of finite strategies. A player chooses a coalition structure and a strategy. A (social) mechanism eliminates conflicts in individual choices and produces final coalition structures. Every final coalition structure is a non-cooperative game. Mixed equilibrium always exists and consists arXiv:2107.00711v1 [cs.GT] 1 Jul 2021 of a mixed strategy profile, payoffs and equilibrium coalition struc- tures. We use a maximum coalition size to parametrize the family ∗Moscow, Russian Federation, independent. No funds, grants, or other support was received. Acknowledge: Nick Baigent, Phillip Bich, Giulio Codognato, Ludovic Julien, Izhak Gilboa, Olga Gorelkina, Mark Kelbert, Anton Komarov, Roger Myerson, Ariel Rubinstein, Shmuel Zamir. Many thanks for advices and for discussion to participants of SIGE-2015, 2016 (an earlier version of the title was “A generalized Nash equilibrium”), CORE 50 Conference, Workshop of the Central European Program in Economic Theory, Udine (2016), Games 2016 Congress, Games and Applications, (2016) Lisbon, Games and Optimization at St Etienne (2016). All possible mistakes are only mine. E-mail for correspondence: dlevando (at) hotmail.ru.
    [Show full text]