Extensive-Form Games with Perfect Information Jeffrey Ely April 22, 2015 Jeffrey Ely Extensive-Form Games with Perfect Information Extensive-Form Games With Perfect Information Recall the alternating offers bargaining game. It is an extensive-form game. (Described in terms of sequences of moves.) In this game, each time a player moves, he knows everything that has happened in the past. He can compute exactly how the game will end given any profile of continuation strategies Such a game is said to have perfect information. Jeffrey Ely Extensive-Form Games with Perfect Information Simple Extensive-Form Game With Perfect Information 1b ¨ H L ¨ H R ¨¨ HH 2r¨¨ HH2r AB @ C @ D @ @ r @r r @r 1, 0 2, 3 4, 1 −1, 0 Figure: An extensive game Jeffrey Ely Extensive-Form Games with Perfect Information Game Trees Nodes H Initial node. Terminal Nodes Z ⊂ H. Player correspondence N : H ⇒ f1, ... , ng (N(h) indicates who moves at h. They move simultaneously) Actions Ai (h) 6= Æ at each h where i 2 N(h). I Action profiles can be identified with branches in the tree. Payoffs ui : Z ! R. Jeffrey Ely Extensive-Form Games with Perfect Information Strategies A strategy for player i is a function si specifying si (h) 2 Ai (h) for each h such that i 2 N(h). Jeffrey Ely Extensive-Form Games with Perfect Information Strategic Form Given any profile of strategies s, there is a unique terminal node z(s) that will be reached. We define ui (s) = ui (z(s)). We have thus obtained a strategic-form representation of the game. We can apply strategic-form solution concepts. Jeffrey Ely Extensive-Form Games with Perfect Information Example Rationalizability: I The strategy AD is never a best-reply for player 2. I All other strategies are rationalizable (I think.) Nash equilibrium: I The profile (L, BD) is a Nash equilibrium. I The profile (R, BC ) is a Nash equilibrium. I The profile (R, AC ) is a Nash equilibrium. I There are many mixed equilibria. Iterative elimination of weakly dominated strategies: I Any strategy for 2 that plays either A or D is weakly dominated. I Therefore BC is the only weakly un-dominated strategy for 2. I Next we can eliminate L for 1. I The order of elimination doesn't matter in this example. Jeffrey Ely Extensive-Form Games with Perfect Information Subgames Associated with every non-terminal node h, there is a subgame consisting of all the nodes in the tree that follows h. The game as a whole is a subgame All other subgames are called proper subgames Given a strategy profile s I Denote by sjh the continuation strategy profile in the subgame beginning at h. I Denote by z(sjh) the terminal node reached by s beginning from h. I Denote by ui (sjh), the continuation payoff ui (sjh) = ui (z(sjh)) Jeffrey Ely Extensive-Form Games with Perfect Information Subgame Perfect Nash Equilibrium A strategy profile s is a subgame perfect Nash equilibrium if for every non-terminal node h, the continuation strategy profile sjh is a Nash equilibrium of the subgame that begins at h, i.e. sj ≥ s0 s j ui ( h) ui (( i , −i ) h) s0 for every strategy i . Jeffrey Ely Extensive-Form Games with Perfect Information Example The unique subgame-perfect Nash equilibrium in the example is (R, BC ). Jeffrey Ely Extensive-Form Games with Perfect Information Backward Induction Consider a perfect information game that 1 is finite, 2 has a single player moving at each node (i.e. N(h) is a singleton), 3 and has no indifferences 0 0 z 6= z =) ui (z) 6= ui (z )) Such a game has a unique SPE which can be found by backward induction. (Note that we do not need mixed strategies for existence.) Jeffrey Ely Extensive-Form Games with Perfect Information The Dukes of Earl Example The five Dukes of Earl are scheduled to arrive at the royal palace on each of the first five days of May. Duke One is scheduled to arrive on the first day of May, Duke Two on the second, etc. Each Duke, upon arrival, can either kill the king or support the king. If he kills the king, he becomes the new king and awaits the next Duke's arrival. If he supports the king all subsequent Duke's cancel their visits. A Duke's first priority is to remain alive, and his second priority is to become king. Who is king on May 6? Draw Extensive Form Describe the strategies of each player Identify the subgames Apply Backward Induction. Jeffrey Ely Extensive-Form Games with Perfect Information The Ultimatum Game There is a dollar to be divided between two players. Player 1 moves first and offers a split of the dollar giving x to Player 2 and leaving 1 − x for himself. Player 2 then either accepts or rejects the offer. An accepted offer is implemented, but if the offer is rejected both players get zero. In the unique SPE of this game, Player 1 gets all of the bargaining surplus: He offers x = 0 to Player 2 which is Player 2's outside option. He thus keeps the difference between the total surplus from agreement and the total surplus from disagreement. Issue with continuous action space. Jeffrey Ely Extensive-Form Games with Perfect Information Infinite Games? (Infinite in terms of the tree length.) Obviously there is no backward induction procedure. We can think of backward induction in a different way. The backward induction procedure constructs a strategy for each player that is unimprovable by a one-stage deviation. Jeffrey Ely Extensive-Form Games with Perfect Information One-Stage Deviations Definition s s0 Let i and i be two distinct strategies for player i and let h be a node at s s0 s which i moves. Let h( i , i ) denote the strategy that coincides with i at s0 all nodes except for h where it plays according to i . ( s0(h˜) if h˜ = h s s0 i h( i , i )(h˜) = si (h˜) otherwise Jeffrey Ely Extensive-Form Games with Perfect Information Unimprovable Strategies Fix a strategy profile s−i . Strategy si is unimprovable by a one-stage deviation if for every node h at which i moves, and every alternative s0 strategy i , s s0 s j ≤ sj ui ((h( i , i ), −i ) h) ui ( h) Jeffrey Ely Extensive-Form Games with Perfect Information The One-Stage Deviation Principle Proposition In any finite perfect-information game, if a strategy profile is unimprovable for all players then it is a SPE. Jeffrey Ely Extensive-Form Games with Perfect Information The Marathon Game 1 player game. The player's name is Zeno. H n Z = f1, 2 ...g A(h) = fquit, continueg for all h. I Quitting at h gives payoff −(1 − 1/h). I Continuing at h leads to h + 1. Continuing forever leads to the terminal node ¥ which gives payoff 1. h = 1 is the initial node. Jeffrey Ely Extensive-Form Games with Perfect Information SPE in the Marathon Game There is a unique SPE in which Zeno completes the Marathon. Jeffrey Ely Extensive-Form Games with Perfect Information Unimprovable Strategies The strategy which quits at every node is unimprovable. Jeffrey Ely Extensive-Form Games with Perfect Information Continuity at Infinity Definition A perfect information game is continuous at infinity if for every strategy s s s0 profile −i , player i, history h, pair of strategies i , i , and real number # > 0 there is an integer t such that the strategy s˜ defined by ( si (h˜) if h˜ is no more than t moves past h s˜ (h˜) = i s0 ˜ i (h) otherwise earns a continuation payoff within # of si in the subgame beginning at h, i.e. ui ((s˜i , s−i )jh) > ui (si , s−i jh) − # Jeffrey Ely Extensive-Form Games with Perfect Information Conditions for Continuity at Infinity A finite game is continuous at infinity A game with discounting is continuous at infinity Jeffrey Ely Extensive-Form Games with Perfect Information The One-Stage Deviation Principle Proposition In any perfect information game that is continuous at infinity, a strategy profile that is unimprovable for all players is a subgame perfect equilibrium. (Of course the converse holds as well.) Jeffrey Ely Extensive-Form Games with Perfect Information Proof of OSDP We will show that any strategy profile which is not an SPE is improvable by a one-stage deviation for some player. Suppose s is a strategy profile such that at some subgame h, there is a player i whose continuation strategy is not a best reply, i.e. s0 s j sj ui (( i , −i ) h) > ui ( h) s0 for some i . For any integer t > 0 define the following strategy ( s0(h˜) if h˜ is no more than t moves past h st i ˜i (h˜) = si (h˜) otherwise. By continuity at infinity, for any # there is a t such that st s j s0 s j − # ui ((˜i , −i ) h) > ui (( i , −i ) h) . We take # small enough so that st s j sj ui ((˜i , −i ) h) > ui ( h) Jeffrey Ely Extensive-Form Games with Perfect Information Proof of OSDP Is there any node h˜ which is exactly t moves past h such that s s0 s j sj ui ((h˜( i , i ), −i ) h˜) > ui ( h˜) holds? If yes, then we have found an improvement by a one-stage deviation. If not, then we have st−1 s j ≥ st s j sj ui ((˜i , −i ) h) ui ((˜i , −i ) h) > ui ( h) (to understand the first inequality, note that the strategy profile st−1 s ˜ (˜i , −i ) either reaches a node h which is t moves past h in which case the inequality holds by the previous answer, or it terminates before ever reaching such a node in which case the payoffs are equal.) Jeffrey Ely Extensive-Form Games with Perfect Information Proof of OSDP Continuing backward in this way, we either find a node h˜ which is within t moves of h such that s s0 s j sj ui ((h˜( i , i ), −i ) h˜) > ui ( h˜) i.e.