Final Exam Stat155 Game Theory Lecture 27: Final Review “This is an open book exam: you can use any printed or written material, but you cannot use a laptop, tablet, or phone (or any device that can Peter Bartlett communicate). Answer each question in the space provided.” December 6, 2016 1 / 127 2 / 127 Topics 1 Topics 2 Combinatorial games Positions, moves, terminal positions, impartial/partisan, progressively General-sum games, Nash equilibria. bounded Two-player: payoff matrices, dominant strategies, safety strategies. Progressively bounded impartial and partisan games Multiplayer: Utility functions, Nash’s Theorem The sets N and P Congestion games and potential games Theorem: Someone can win Every potential game has a pure Nash equilibrium Examples: Subtraction, Chomp, Nim, Rims, Staircase Nim, Hex Every congestion game is a potential game Zero sum games Other equilibrium concepts Payoff matrices, pure and mixed strategies, safety strategies Evolutionarily stable strategies Von Neumann’s minimax theorem Solving two player zero-sum games Correlated equilibrium Saddle points Price of anarchy Equalizing strategies Braess’s paradox Solving2 2 games × The impact of adding edges Dominated strategies Classes of latencies 2 n and m 2 games × × Pigou networks Principle of indifference Symmetry: Invariance under permutations 3 / 127 4 / 127 Topics 3 Definitions: Combinatorial games Cooperative games Transferable versus nontransferable utility A combinatorial game has: Two-player transferable utility cooperative games Two players, Player I and Player II. Cooperative strategy, threat strategies, disagreement point, final payoff vector A set of positions X . Two-player nontransferable utility cooperative games For each player, a set of legal moves between positions, that is, a set Bargaining problems of ordered pairs, (current position, next position): Nash’s bargaining axioms, the Nash bargaining solution Multi-player transferable utility cooperative games MI , MII X X . Characteristic function ⊂ × Gillies’ core Shapley’s axioms, Shapley’s Theorem Designing games Players alternately choose moves. Voting systems. Play continues until some player cannot move. Voting rules, ranking rules Normal play: the player that cannot move loses the game. Arrow’s impossibility theorem, Gibbard-Satterthwaite Theorem Properties of voting rules Instant runoff voting, Borda count, positional voting rules 5 / 127 6 / 127 Definitions: Combinatorial games Impartial combinatorial games and winning strategies Terminology: An impartial game has the same set of legal moves for both players: MI = MII . Theorem A partisan game has different sets of legal moves for the players. In a progressively bounded impartial A terminal position for a player has no legal move to another position. combinatorial game under normal x is terminal for player I if there is no y X with( x, y) MI . play, X = N P. ∈ ∈ ∪ A combinatorial game is progressively bounded if, for every starting That is, from any initial position, one position x X , there is finite bound on the number of moves before of the players has a winning strategy. 0 ∈ the game ends. (That is, if B(x) denotes the maximum number of Proof: induction on number of moves before the game ends, then B(x) < .) ∞ moves until the end. A winning strategy for a player from position x: a mapping from non-terminal positions to legal moves that is guaranteed to result in a win for that player from that position. 7 / 127 8 / 127 Key Ideas: Progressively bounded impartial games Example: Nim k piles of chips. Remove some (positive) number of chips from some pile. A player wins when they take the last chip. Bouton’s Theorem P: Every move leads to N. A Nim position( x ,..., x ) is in P iff N: Some move leads to P (hence cannot contain terminal positions). 1 k the Nim-sum of its components is0. The Nim-sum of x = (x ,..., x ) is written x x x . 1 k 1 ⊕ 2 ⊕ · · · ⊕ k The binary representation of the Nim-sum is the bitwise sum, in modulo-two arithmetic, of the binary representations of the components of x. Proof: Show that Z := (x ,..., x ): x x = 0 is P: { 1 k 1 ⊕ · · · ⊕ k } Z must lead to Z c ; from Z c , there is a move into Z. 9 / 127 10 / 127 Partisan Games Topics 1 Combinatorial games Positions, moves, terminal positions, impartial/partisan, progressively Recall: bounded Progressively bounded impartial and partisan games An impartial game has the same set of legal moves for both players: The sets N and P MI = MII . Theorem: Someone can win A partisan game has different sets of legal moves for the players. Examples: Subtraction, Chomp, Nim, Rims, Staircase Nim, Hex Zero sum games Theorem Payoff matrices, pure and mixed strategies, safety strategies Consider a progressively bounded partisan Von Neumann’s minimax theorem combinatorial game under normal play, with no ties Solving two player zero-sum games allowed. From any initial position, one of the players Saddle points Equalizing strategies has a winning strategy. Solving2 2 games × Dominated strategies 2 n and m 2 games × × Principle of indifference Symmetry: Invariance under permutations 11 / 127 12 / 127 Two-player zero-sum games Two-player zero-sum games Definitions Player I has m actions, 1, 2,..., m. Definitions Player II has n actions, 1, 2,..., n. A mixed strategy is a probability distribution over actions. m n The payoff matrix A R × represents the payoff to Player I: ∈ A mixed strategy for Player I is a vector a11 a12 a1n ··· x1 a a a 21 22 2n x m A = . ···. 2 m . x = . ∆m := x R : xi 0, xi = 1 . ∈ ∈ ≥ . ( i=1 ) am1 am2 amn X ··· xm If Player I chooses i and Player II chooses j, the payoff to Player I is a and the payoff to Player II is a . ij − ij The sum of the payoff to Player I and the payoff to Player II is0. 13 / 127 14 / 127 Two-player zero-sum games Two-player zero-sum games The expected payoff to Player I when Player I plays mixed strategy A mixed strategy for Player II is a vector x ∆m and Player II plays mixed strategy y ∆n is ∈ ∈ m n y1 n EI x EJ y aIJ = xi aij yj y2 ∼ ∼ n i=1 j=1 y = . ∆n := y R : yi 0, yi = 1 . X X . ∈ ( ∈ ≥ ) Xi=1 = x>Ay yn a a a y 11 12 ··· 1n 1 a21 a22 a2n y2 A pure strategy is a mixed strategy where one entry is1 and the = x1 x2 xm . ···. . ··· . .. others0. (This is a canonical basis vector ei .) a a a y m1 m2 ··· mn n 15 / 127 16 / 127 Two-player zero-sum games Two-player zero-sum games Von Neumann’s Minimax Theorem A safety strategy for Player I is an x ∆ that satisfies m n ∗ m For any two-person zero-sum game with payoff matrix A R × , ∈ ∈ min x∗>Ay = max min x>Ay. max min x>Ay = min max x>Ay. y ∆n x ∆m y ∆n x ∆m y ∆n y ∆n x ∆m ∈ ∈ ∈ ∈ ∈ ∈ ∈ This mixed strategy maximizes the worst case expected gain for Player I. We call the optimal expected payoff the value of the game, Similarly, a safety strategy for Player II is a y ∗ ∆n that satisfies V := max min x>Ay = min max x>Ay. x ∆m y ∆n y ∆n x ∆m ∈ ∈ ∈ ∈ ∈ max x>Ay ∗ = min max x>Ay. x ∆m y ∆n x ∆m LHS: Player I plays x ∆m first, then Player II responds with y ∆n. ∈ ∈ ∈ ∈ ∈ RHS: Player II plays y ∆ first, then Player I responds with x ∆ . ∈ n ∈ m Notice that we should always prefer to play last: This mixed strategy minimizes the worst case expected loss for Player II. max min x>Ay min max x>Ay. x ∆m y ∆n ≤ y ∆n x ∆m ∈ ∈ ∈ ∈ The astonishing part is that it does not help. 17 / 127 18 / 127 Two-player zero-sum games Topics 1 Combinatorial games Von Neumann’s Minimax Theorem Positions, moves, terminal positions, impartial/partisan, progressively m n For any two-person zero-sum game with payoff matrix A R × , ∈ bounded Progressively bounded impartial and partisan games max min x>Ay = min max x>Ay. x ∆m y ∆n y ∆n x ∆m Zero sum games ∈ ∈ ∈ ∈ Payoff matrices, pure and mixed strategies, safety strategies Von Neumann’s minimax theorem Solving two player zero-sum games Safety strategies are optimal strategies: Saddle points For safety strategies x∗ for Player I and y ∗ for Player II, Equalizing strategies Solving2 2 games × min x∗>Ay = max x>Ay ∗ = x∗>Ay ∗ = V . Dominated strategies y ∆n x ∆m 2 n and m 2 games ∈ ∈ × × Principle of indifference Symmetry: Invariance under permutations 19 / 127 20 / 127 Saddle points Saddle points Definition A pair( i ∗, j∗) 1,..., m 1,..., n is a saddle point for a payoff m∈n { } × { } matrix A R × if Theorem ∈ m n If( i ∗, j∗) is a saddle point for a payoff matrix A R × , then max aij = ai j = min ai j . ∈ i ∗ ∗ ∗ j ∗ ei ∗ is an optimal strategy for Player I, ej∗ is an optimal strategy for Player II, and If Player I plays i and Player II plays j , neither player has an ∗ ∗ the value of the game is ai ∗j∗ . incentive to change. Think of these as locally optimal strategies for the players. They are also globally optimal strategies. 21 / 127 22 / 127 2 2 games Dominated pure strategies × How to solve a2 2 game × 1 Check for a saddle point. (Is the max of row mins= min of column maxes?) 2 If there are no saddle points, find equalizing strategies. Definition Equalizing strategies satisfy: A pure strategy ei for Player I is dominated by ei 0 in payoff matrix A if, for all j 1,..., n , ∈ { } x1a11 + (1 x1)a21 = x1a12 + (1 x1)a22, a a .