Repeated Games Games with Incomplete Information

Lecture 25 Repeated Games and Games with Incomplete Information

Jitesh H. Panchal

ME 597: Decision Making for Engineering Systems Design Design Engineering Lab @ Purdue (DELP) School of Mechanical Engineering Purdue University, West Lafayette, IN http://engineering.purdue.edu/delp

November 19, 2019

ME 597: Fall 2019 Lecture 25 1 / 27 Repeated Games Games with Incomplete Information Lecture Outline

1 Repeated Games 1. Finitely Repeated Games 2. Infinitely Repeated Games

2 Games with Incomplete Information

Dutta, P.K. (1999). Strategies and Games: Theory and Practice. Cambridge, MA, The MIT Press. Chapters 14-15. ME 597: Fall 2019 Lecture 25 2 / 27 Repeated Games 1. Finitely Repeated Games Games with Incomplete Information 2. Infinitely Repeated Games Entry Game

Every strategy has three components (e.g., EAT).

Figure: 11.7 on Page 164 (Dutta)

ME 597: Fall 2019 Lecture 25 3 / 27 Repeated Games 1. Finitely Repeated Games Games with Incomplete Information 2. Infinitely Repeated Games Example – Entry Game (contd.)

Strategic form: Coke / Pepsi TA ETT −2, −1 0, −3 ETA −2, −1 1, 2 EAT −3, 1 0, −3 EAA −3, 1 1, 2 OTT 0, 5 0, 5 OTA 0, 5 0, 5 OAT 0, 5 0, 5 OAA 0, 5 0, 5 Pure Strategy Nash equilibria: 1 Pepsi: T ; Coke: OTT , OTA, OAT , or OAA 2 Pepsi: A; Coke: ETA 3 Pepsi: A; Coke: EAA

The only sequentially rational strategy is: Pepsi: A; Coke: ETA

ME 597: Fall 2019 Lecture 25 4 / 27 Repeated Games 1. Finitely Repeated Games Games with Incomplete Information 2. Infinitely Repeated Games Repeated Games

Fundamental Difference (compared to non-repeated games) Players interact not just once but many times. The prospect of “reciprocity” either by way of rewards or punishments, separates a repeated game from one-time interaction.

In every repeated game, there is a component game–called stage game–that is played many times.

The total payoff is the sum of payoffs in each stage.

ME 597: Fall 2019 Lecture 25 5 / 27 Repeated Games 1. Finitely Repeated Games Games with Incomplete Information 2. Infinitely Repeated Games Prisoner’s Dilemma

c = confess n = not confess Higher payoff is better.

1 / 2 c n c 0, 0 7, −2 n −2, 7 5, 5

Nash equilibrium = (c, c)

ME 597: Fall 2019 Lecture 25 6 / 27 Repeated Games 1. Finitely Repeated Games Games with Incomplete Information 2. Infinitely Repeated Games Repeated Game: Definition

Definition (Repeated Game) A repeated game is defined by a stage game G and the number of its repetitions, say T . The stage game G is a game in strategic form: G = {Si , πi ; i = 1,..., N} where Si is player i’s set of strategies and πi is his payoff function [and it depends on (s1, s2,..., sN )].

Finitely repeated game: T is finite Infinitely repeated game: No fixed end

ME 597: Fall 2019 Lecture 25 7 / 27 Repeated Games 1. Finitely Repeated Games Games with Incomplete Information 2. Infinitely Repeated Games Example 1: Once-Repeated Prisoner’s Dilemma

P2 c 0,0 P1 c n 7,-2 c -2,7 n 5,5 c n P2 P2 c 7,-2 c 14,-4 P1 n n c c 5,5 n n 12,3 P1 P2 c -2,7 c n 5,5 n c P1 c -4,14 n n 3,12 n P2 c 5,5 P1 c n 12,3 c 3,12 n n 10,10

Figure: 14.1 on Page 210 (Dutta)

ME 597: Fall 2019 Lecture 25 8 / 27 Repeated Games 1. Finitely Repeated Games Games with Incomplete Information 2. Infinitely Repeated Games Backward Induction in Finitely Repeated Games

P2 c 0,0 P1 c n c n c n P2 P2 c 7,-2 c P1 n n c c n n P1 P2 c -2,7 c n n c P1 c n n n P2 c 5,5 P1 c n c n n

Figure: 14.4 on Page 215 (Dutta)

ME 597: Fall 2019 Lecture 25 9 / 27 Repeated Games 1. Finitely Repeated Games Games with Incomplete Information 2. Infinitely Repeated Games Example 2: Finitely Repeated Modified Prisoner’s Dilemma

c = confess, n = not confess, p = partly confess

Stage game: 1 / 2 c n p c 0, 0 7, −2 3, −1 n −2, 7 5, 5 0, 6 p −1, 3 6, 0 3, 3

This game has two Nash equilibria: (c, c) and (p, p)

ME 597: Fall 2019 Lecture 25 10 / 27 Repeated Games 1. Finitely Repeated Games Games with Incomplete Information 2. Infinitely Repeated Games Example 2: Finitely Repeated Modified Prisoner’s Dilemma

Figure: 14.2 on Page 211 (Dutta)

ME 597: Fall 2019 Lecture 25 11 / 27 Repeated Games 1. Finitely Repeated Games Games with Incomplete Information 2. Infinitely Repeated Games Example 2: Finitely Repeated Modified Prisoner’s Dilemma Subgame Perfect Nash Equilibrium

A subgame Perfect Nash Equilibrium: Start with (n,n) and continue in that way at all stages except the last one, and at that stage play (p,p). Follow this procedure provided neither player deviates from it. In the event of a deviation, play (c,c) from the subsequent stage onward.

Another subgame Perfect Nash Equilibrium: Start with (n,c). If that is in fact played, then play (p,p) in the second stage. Otherwise, play (c,c).

ME 597: Fall 2019 Lecture 25 12 / 27 Repeated Games 1. Finitely Repeated Games Games with Incomplete Information 2. Infinitely Repeated Games Some Conclusions

Consider a finitely repeated game G, T with G = {Si , πi ; i = 1,..., N}. Suppose the stage game G has exactly one Nash equilibrium, say ∗ ∗ ∗ (s1 , s2 ,..., sN ). Then the repeated game has a unique subgame perfect ∗ equilibrium. In this equilibrium, player i plays si at every one of the T stages, regardless of what might have played, by him or any of the others, in any previous stage.

If there is more than one Nash Equilibrium, there is always the possibility of sustaining good behavior in early stages of repeated interaction. This good behavior is sustained by the prospect of better future play than that which follows short-term opportunistic behavior.

ME 597: Fall 2019 Lecture 25 13 / 27 Repeated Games 1. Finitely Repeated Games Games with Incomplete Information 2. Infinitely Repeated Games Discounted Payoffs

Discount factor: Amount by which future payments are discounted to get their present-day equivalent.

For example, if $1 a month from now is equivalent to $0.99 today, then the discount factor, δ is 0.99. Amount by which a payoff two stages from today is discounted = δ2 Amount by which a payoff three stages from today is discounted = δ3 ...

The total discounted payoff for player i is:

2 t πi0 + πi1δ + πi2δ + ··· + πit δ + ...

ME 597: Fall 2019 Lecture 25 14 / 27 Repeated Games 1. Finitely Repeated Games Games with Incomplete Information 2. Infinitely Repeated Games Example 3: Infinitely Repeated Prisoner’s Dilemma

Each time they play the stage game, there is a probability δ that the same players will play the stage game again. There is (1 − δ) probability that the current interaction is the last one. Total expected payoff:

2 t πi0 + δπi1 + δ πi2 + ··· + δ πit + ...

where πit = payoff for player i at period t

ME 597: Fall 2019 Lecture 25 15 / 27 Repeated Games 1. Finitely Repeated Games Games with Incomplete Information 2. Infinitely Repeated Games Example 3: Infinitely Repeated Prisoner’s Dilemma

P2

c P1 c P2 n n c P2

c P1 P1 n n c P2

c n P1

n

Stage 1 Stage t

Figure: 14.3 on Page 212 (Dutta)

ME 597: Fall 2019 Lecture 25 16 / 27 Repeated Games 1. Finitely Repeated Games Games with Incomplete Information 2. Infinitely Repeated Games The Behavior Cycle

Definition (The Behavior Cycle)

A behavior cycle is a repeated cycle of actions; play (n, n) for T1 stages, then (c, c) for T2 stages, followed by (n, c) for T3 stages, and then (c, n) for T4 stages. At the end of these T1 + T2 + T3 + T4 stages, start the cycle again, then yet again, and so on.

The behavior cycle is called individually rational if each player gets strictly positive payoff within a cycle.

ME 597: Fall 2019 Lecture 25 17 / 27 Repeated Games 1. Finitely Repeated Games Games with Incomplete Information 2. Infinitely Repeated Games The Folk Theorem

Theorem (The Folk Theorem) Equilibrium Behavior: Consider any individually rational behavior cycle. Then this cycle is achievable as the play of a subgame perfect equilibrium whenever the discount factor δ is close to 1. Equilibrium Strategy: One strategy that constitutes an equilibrium is the grim trigger; start with the desired behavior cycle and continue with it if the two players do nothing else. If either player deviates to do something else, then play (c, c) forever after.

Implication: Not only are positive payoffs necessary for equilibrium but they are sufficient as well. Every behavior cycle with positive payoffs is an equilibrium for high values of δ.

ME 597: Fall 2019 Lecture 25 18 / 27 Repeated Games Games with Incomplete Information Games with Incomplete Information

Imperfect Information Game A player is unaware of actions taken by other players. However, the player knows who the other players are, what their strategies are, and what their preferences are.

Incomplete Information Game Situations where players do not completely know the other players, their strategies, or their payoff functions.

ME 597: Fall 2019 Lecture 25 19 / 27 Repeated Games Games with Incomplete Information Games with Incomplete Information Example 1: Prisoner’s Dilemma

Player 2’s preferences are unknown to Player 1. He can either be tough or accommodating (prefers n to c). Player 1 does not know which is the relevant matrix, but player 2 does.

Tough Accommodating 1 / 2 c n 1 / 2 c n c 0, 0 7, −2 c 0, −2 7, 0 n −2, 7 5, 5 n −2, 5 5, 7

Dominant strategies: c for both Dominant strategies: c for player 1 players. and n for player 2.

ME 597: Fall 2019 Lecture 25 20 / 27 Repeated Games Games with Incomplete Information Games with Incomplete Information Example 2: Battle of Sexes

Husband is not sure of wife’s preferences. In particular, he does not know if his wife likes him (loving), or if she prefers to go to either event by herself (leaving).

Loving Leaving H/W FO H/W FO F 3, 1 0, 0 F 3, 0 0, 1 O 0, 0 1, 3 O 0, 3 1, 0

Husband does not know which is the relevant matrix, but Wife does.

ME 597: Fall 2019 Lecture 25 21 / 27 Repeated Games Games with Incomplete Information Games with Incomplete Information Example 2: Battle of Sexes

Assumptions 1 The wife knows her preferences, i.e., she knows whether the “loving” game is played or the “leaving” game is played. 2 The husband does not know his wife’s real preferences; he attaches a probability ρ to the fact that her true preferences are given by “loving” and probability (1 − ρ) to “leaving”. 3 The wife knows her husband’s estimate of her preferences, i.e., she knows the value of ρ. ⇐ Common Prior Assumption

ME 597: Fall 2019 Lecture 25 22 / 27 Repeated Games Games with Incomplete Information Games with Incomplete Information Equilibrium Analysis

Equilibrium Analysis 1 Turn the game of incomplete information into a game of imperfect information. 2 Use Nash equilibrium of the imperfect information game as the solution concept. ⇐ Bayesian Nash Equilibrium

ME 597: Fall 2019 Lecture 25 23 / 27 Repeated Games Games with Incomplete Information Games with Incomplete Information Equilibrium Analysis

Figure: 20.1 on Page 315 (Dutta)

ME 597: Fall 2019 Lecture 25 24 / 27 Repeated Games Games with Incomplete Information Games with Incomplete Information Equilibrium Analysis

3 There are two pure strategy Bayes-Nash equilibria whenever ρ ≥ : 4 1 the husband plays F while the wives plays (F, O) 2 the husband plays O and the wives play (O, F)

1 3 If ≤ ρ ≤ , there is only one pure strategy Bayes Nash equilibrium: the 4 4 husband plays F while the wives plays (F, O).

1 If ρ < there is no pure strategy Bayes Nash equilibrium. 4

ME 597: Fall 2019 Lecture 25 25 / 27 Repeated Games Games with Incomplete Information Summary

1 Repeated Games 1. Finitely Repeated Games 2. Infinitely Repeated Games

2 Games with Incomplete Information

ME 597: Fall 2019 Lecture 25 26 / 27 Repeated Games Games with Incomplete Information References

1 Dutta, P.K. (1999). Strategies and Games: Theory and Practice. Cambridge, MA, The MIT Press. Chapters 14-15 for Repeated Games. Chapter 20 for Games with Incomplete Information.

ME 597: Fall 2019 Lecture 25 27 / 27