An Introduction to Game Theory Bruce Hajek Department of Electrical and Computer Engineering University of Illinois at Urbana-Champaign December 10, 2018 c 2018 by Bruce Hajek All rights reserved. Permission is hereby given to freely print and circulate copies of these notes so long as the notes are left intact and not reproduced for commercial purposes. Email to [email protected], pointing out errors or hard to understand passages or providing comments, is welcome. Contents 1 Introduction to Normal Form Games 3 1.1 Normal form games with finite action sets . .3 1.2 Cournot model of competition . .9 1.3 Correlated equilibria . 11 1.4 On the existence of a Nash equilibrium . 13 1.5 On the uniqueness of a Nash equilibrium . 15 1.6 Two-player zero sum games . 17 1.6.1 Saddle points and the value of two-player zero sum game . 17 1.7 Appendix: Derivatives, extreme values, and convex optimization . 19 1.7.1 Weierstrass extreme value theorem . 19 1.7.2 Derivatives of functions of several variables . 20 1.7.3 Optimality conditions for convex optimization . 21 2 Evolution as a Game 25 2.1 Evolutionarily stable strategies . 25 2.2 Replicator dynamics . 27 3 Dynamics for Repeated Games 33 3.1 Iterated best response . 33 3.2 Potential games . 34 3.3 Fictitious play . 36 3.4 Regularized fictitious play and ode analysis . 39 3.4.1 A bit of technical background . 39 3.4.2 Regularized fictitious play for one player . 40 3.4.3 Regularized fictitious play for two players . 41 3.5 Prediction with Expert Advice . 43 iii CONTENTS v 3.5.1 Deterministic guarantees . 43 3.5.2 Application to games with finite action space and mixed strategies . 46 3.5.3 Hannan consistent strategies in repeated two-player, zero sum games . 49 3.6 Blackwell's approachability theorem . 51 3.7 Online convex programing and a regret bound (skip in fall 2018) . 56 3.7.1 Application to game theory with finite action space . 59 3.8 Appendix: Large deviations, the Azuma-Hoeffding inequality, and stochastic approximation . 59 4 Sequential (Extensive Form) Games 65 4.1 Perfect information extensive form games . 65 4.2 Imperfect information extensive form games . 69 4.2.1 Definition of extensive form games with imperfect information, and total recall . 69 4.2.2 Sequential equilibria { generalizing subgame perfection to games with imperfect infor- mation . 74 4.3 Games with incomplete information . 79 5 Multistage games with observed actions 83 5.1 Extending backward induction algorithm { one stage deviation condition . 83 5.2 Feasibility theorems for repeated games . 87 6 Mechanism design and theory of auctions 91 6.1 Vickrey-Clarke-Groves (VCG) Mechanisms . 92 6.1.1 Appendix: Strict suboptimality of nontruthful bidding for VCG . 96 6.2 Optimal mechanism design (Myerson (1981)) . 96 6.2.1 Appendix: Envelope theorem . 104 7 Introduction to Cooperative Games 107 7.1 The core of a cooperative game with transfer payments . 107 7.2 Markets with transferable utilities . 113 7.3 The Shapley value . 117 vi CONTENTS Preface Game theory lives at the intersection of social science and mathematics, and makes significant appearances in economics, computer science, operations research, and other fields. It describes what happens when multiple players interact, with possibly different objectives and with different information upon which they take actions. It offers models and language to discuss such situations, and in some cases it suggests algorithms for either specifying the actions a player might take, of for computing possible outcomes of a game. Examples of games surround us in everyday life, in engineering design, in business, and politics. Games arise in population dynamics, in which different species of animals interact. The cells of a growing organism compete for resources. Games are at the center of many sports, such as the game between pitcher and batter in baseball. A large distributed resource such as the internet relies on the interaction of thousands of autonomous players for operation and investment. These notes also touch upon mechanism design, which entails the design of a game, usually with the goal of steering the likely outcome of the game in some favorable direction. Most of the notes are concerned with the branch of game theory involving noncooperative games, in which each player has a separate objective, often conflicting with the objectives of other players. Some portion of the course will focus on cooperative game theory, which is typically concerned with the problem of how to to divide wealth, such as revenue, a surplus of goods, or resources, among a set of players in a fair way, given the contributions of the players in generating the wealth. This is the latest version of these notes, written in Fall 2017 and being periodically updated in Fall 2018 in con- junction with the teaching of ECE 586GT Game Theory, at the University of Illinois at Urbana-Champaign. Problem sets and exams with solutions are posted on the course website: https://courses.engr.illinois. edu/ece586gt/fa2017/ and https://courses.engr.illinois.edu/ece586/sp2013/. The author would be grateful for comments, suggestions, and corrections. {Bruce Hajek 1 2 CONTENTS Chapter 1 Introduction to Normal Form Games The material in this section is basic to many books and courses on game theory. See the monograph [13] for an expanded version with applications to a variety of routing problems in networks. 1.1 Normal form games with finite action sets Among the simplest games to describe are those involving the simultaneous actions of two players. Each player selects an action, and then each player receives a reward determined by the pair of actions taken by the two players. A pure strategy for a player is simply one of the possible actions the player could take, whereas a mixed strategy is a probability distribution over the set of pure strategies. There is no theorem that determines the pair of strategies the two players of a given game will select, and no theorem that can determine a probability distribution of joint selections, unless some assumptions are made about the objectives, rationality, and computational capabilities of the players. Instead, the typical outcome of a game theoretic analysis is to produce a set of strategy pairs that are in some sort of equilibrium. The most celebrated notion of equilibrium is due to Nash; a pair of strategies is a Nash equilibrium if whenever one player uses one of the strategies, the strategy for the other player is an optimal response. There are, however, other notions of equilibrium as well. Given these notions of equilibrium we can then investigate some immediate questions, such as: Does a given game have an equilibrium pair? If so, is it unique? How might the players arrive at a given equilibrium pair? Are there any computational obstacles to overcome? How high the payoffs for an equilibrium pair compared to payoffs for other pairs? A two-player normal form game (also called a strategic form game) is specified by an action space for each player, and a payoff function for each player, such that the payoff is a function of the pair of actions taken by the players. If the action space of each player is finite, then the payoff functions can be specified by matrices. The two payoff matrices can be written in a single matrix, with a pair of numbers for each entry, where the first number is the payoff for the first player, who selects a row of the matrix, and the second number is the payoff of the second player, who selects a column of the matrix. In that way, the players select an entry of the matrix. A rich variety of interactions can be modeled with fairly small action spaces. We shall describe some of the most famous examples. Dozens of others are described on the internet. Example 1.1 (Prisoners' dilemma) 3 4 CHAPTER 1. INTRODUCTION TO NORMAL FORM GAMES There are many similar variations of the prisoners' dilemma game, but one instance of it is given by the following assumptions. Suppose there are two players who committed a crime and are being held on suspicion of committing the crime, and are separately questioned by an investigator. Each player has two possible actions during questioning: • cooperate (C) with the other player, by telling the investigator both players are innocent • don't cooperate (D) with the other player, by telling the investigator the two players committed the crime Suppose a player goes free if and only if the other player cooperates, and suppose a player is awarded points according to the following outcomes. A player receives +1 point for not cooperating (D) with the other player by confessing +1 point if player goes free, i.e. if the other player cooperates (C) -1 point if player does not go free, i.e. if the other player doesn't cooperate (D) For example, if both players cooperate then both players receive one point. If the first player cooperates (C) and the second one doesn't (D), then the payoffs of the players are -1, 2, respectively. The payoffs for all four possible pairs of actions are listed in the following matrix form: Player 2 C (cooperate)D Player 1 C (cooperate) 1,1 -1,2 D 2,-1 0,0 What actions do you think rational players would pick? To be definite, let's suppose each player cares only about maximizing his/her own payoff and doesn't care about the payoff of the other player. Some thought shows that action D maximizes the payoff of one player no matter which action the other player selects.