Recap Perfect-Information Extensive-Form Games Subgame Perfection Backward Induction Introduction I The normal form game representation does not incorporate any notion of sequence, or time, of the actions of the players I The extensive form is an alternative representation that makes the temporal structure explicit. I Two variants: I perfect information extensive-form games I imperfect-information extensive-form games Extensive Form Games and Backward Induction ISCI 330 Lecture 11, Slide 3 Recap Perfect-Information Extensive-Form Games Subgame Perfection Backward Induction Perfect-Information Extensive Form Representation I Represents a finite sequential game as a rooted tree I Each internal node represents a particular player’s ’turn’ I Each branch from this internal node represents a different choice for that player I Each terminal node represents a possible final outcome of the game I Usually written with the basal root on top I Parts of the tree are labeled as follows: I Internal nodes (including the root) are labeled with player identifiers I Branches are labeled with player choices I Terminal nodes are labeled with utility outcomes Extensive Form Games and Backward Induction ISCI 330 Lecture 11, Slide 4 Recap Perfect-Information Extensive-Form Games Subgame Perfection Backward Induction Example: the sharing game q 1 ¨¨HH ¨ H ¨¨ HH ¨ H 0–21–12–0 ¨¨ HH ¨ H ¨ Hqqq 222 ¡A ¡A ¡A ¡ A ¡ A ¡ A ¡ A ¡ A ¡ A yesnoyesnoyesno ¡ A ¡ A ¡ A ¡ A ¡ A ¡ Aqqqqqq (0,2)(0,0)(1,1)(0,0)(2,0)(0,0) Get with a partner and decide on a simple sequential game (e.g. tic-tac-toe) and represent it in extended form Extensive Form Games and Backward Induction ISCI 330 Lecture 11, Slide 5 Recap Perfect-Information Extensive-Form Games Subgame Perfection Backward Induction Pure Strategies I In the sharing game (splitting 2 coins) how many pure strategies does each player have? I player 1: 3; player 2: 8 I Overall, a pure strategy for a player in a perfect-information game is a complete specification of which deterministic action to take at every node belonging to that player. I Can think of a strategy as a complete set of instructions for a proxy who will play for the player in their abscence Extensive Form Games and Backward Induction ISCI 330 Lecture 11, Slide 8 5.1 Perfect-information extensive-form games 109 ¨H1 ¨¨ HH ¨ q H 0–2 ¨ 1–1 H 2–0 ¨¨ HH ¨¨ HH 2 ¨ 2 H 2 ¡A ¡A ¡A ¡ q A ¡ q A ¡ q A no ¡ A yes no ¡ A yes no ¡ A yes ¡ A ¡ A ¡ A ¡ A ¡ A ¡ A (0,0) (2,0) (0,0) (1,1) (0,0) (0,2) q q q q q q Figure 5.1 The Sharing game. Notice that the definition contains a subtlety. An agent’s strategy requires a decision Recapat each choicePerfect-Information node, regardless Extensive-Form of whether Games or not itSubgame is possib Perfectionle to reach thatBackward node given Induction the other choice nodes. In the Sharing game above the situation is straightforward— Pureplayer Strategies 1 has three pure Example strategies, and player 2 has eight (why?). But now consider the game shown in Figure 5.2. 1 A B 2 2 C D E F 1 (3,8) (8,3) (5,5) G H (2,10) (1,0) Figure 5.2 A perfect-information game in extensive form. What are the pure strategies for player 2? InI orderS2 to= define{(C, a E complete); (C, F strategy); (D, for E); this (D, game, F )} each of the players must choose anWhat action are at each the of pure his two strategies choice nodes. forplayer Thus we 1? can enumerate the pure strategies of the players as follows. I S1 = {(B, G); (B, H), (A, G), (A, H)} S1 = (A, G), (A, H), (B, G), (B, H) I This{ is true even though, conditional} on taking A, the choice S = (C, E), (C,F ), (D, E), (D,F ) 2 between{ G and H will never} have to be made ExtensiveIt is Form important Games and to Backward note that Induction we have to include the strategies (A, G) andISCI(A, 330 H Lecture), even 11, Slide 9 though once A is chosen the G-versus-H choice is moot. The definition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfect- information game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5.2 can be converted into the normal form im- age of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are Multi Agent Systems, draft of September 19, 2006 Recap Perfect-Information Extensive-Form Games Subgame Perfection Backward Induction Nash Equilibria Given our new definition of pure strategy, we are able to reuse our old definitions of: I mixed strategies I best response I Nash equilibrium Theorem Every perfect information game in extensive form has a PSNE This is easy to see, since the players move sequentially. Extensive Form Games and Backward Induction ISCI 330 Lecture 11, Slide 10 5.1 Perfect-information extensive-form games 109 ¨H1 ¨¨ HH ¨ q H 0–2 ¨ 1–1 H 2–0 ¨¨ HH ¨¨ HH 2 ¨ 2 H 2 ¡A ¡A ¡A ¡ q A ¡ q A ¡ q A no ¡ A yes no ¡ A yes no ¡ A yes ¡ A ¡ A ¡ A ¡ A ¡ A ¡ A (0,0) (2,0) (0,0) (1,1) (0,0) (0,2) q q q q q q Figure 5.1 The Sharing game. Notice that the definition contains a subtlety. An agent’s strategy requires a decision Recapat each choicePerfect-Information node, regardless Extensive-Form of whether Games or not itSubgame is possib Perfectionle to reach thatBackward node given Induction the other choice nodes. In the Sharing game above the situation is straightforward— Inducedplayer 1 has Normal three pure Form strategies, and player 2 has eight (why?). But now consider the game shown in Figure 5.2. 1 I In fact, the connectionA to the normalB form is even tighter I we can “convert”2 an extensive-form game2 into normal form C D E F 1 CE CF DE DF (3,8) (8,3) (5,5) AG G3, 8 3H, 8 8, 3 8, 3 AH 3, 8 3, 8 8, 3 8, 3 BG (2,10)5, 5 (1,0)2, 10 5, 5 2, 10 BH 5, 5 1, 0 5, 5 1, 0 Figure 5.2 A perfect-information game in extensive form. In order to define a complete strategy for this game, each of the players must choose an actionI this at each illustrates of his two the choice lack nodes. of compactness Thus we can enumerat of thee normal the pure form strategies of the playersI games as follows. aren’t always this small even here we write down 16 payoff pairs instead of 5 S = I(A, G), (A, H), (B, G), (B, H) 1 { } S = (C, E), (C,F ), (D, E), (D,F ) 2 { } ExtensiveIt is Form important Games and to Backward note that Induction we have to include the strategies (A, G) andISCI( 330A, LectureH), even 11, Slide 11 though once A is chosen the G-versus-H choice is moot. The definition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfect- information game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5.2 can be converted into the normal form im- age of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are Multi Agent Systems, draft of September 19, 2006 5.1 Perfect-information extensive-form games 109 ¨H1 ¨¨ HH ¨ q H 0–2 ¨ 1–1 H 2–0 ¨¨ HH ¨¨ HH 2 ¨ 2 H 2 ¡A ¡A ¡A ¡ q A ¡ q A ¡ q A no ¡ A yes no ¡ A yes no ¡ A yes ¡ A ¡ A ¡ A ¡ A ¡ A ¡ A (0,0) (2,0) (0,0) (1,1) (0,0) (0,2) q q q q q q Figure 5.1 The Sharing game. Notice that the definition contains a subtlety. An agent’s strategy requires a decision Recapat each choicePerfect-Information node, regardless Extensive-Form of whether Games or not itSubgame is possib Perfectionle to reach thatBackward node given Induction the other choice nodes. In the Sharing game above the situation is straightforward— Inducedplayer 1 has Normal three pure Form strategies, and player 2 has eight (why?). But now consider the game shown in Figure 5.2. 1 In fact, the connection to the normal form is even tighter I A B I we can “convert” an extensive-form game into normal form 2 2 C D E F CE 1 CF DE DF (3,8) (8,3) (5,5)AG 3, 8 3, 8 8, 3 8, 3 AH G3, 8 3H, 8 8, 3 8, 3 BG 5, 5 2, 10 5, 5 2, 10 BH (2,10)5, 5 (1,0)1, 0 5, 5 1, 0 Figure 5.2 A perfect-information game in extensive form. InI orderwhile to define we can a complete write anystrategy extensive-form for this game, each game of th ase players a NF, must we choose can’t an actiondo at the each reverse. of his two choice nodes. Thus we can enumerate the pure strategies of the players as follows. I e.g., matching pennies cannot be written as a S1 = (A,perfect-information G), (A, H), (B, G), (B, extensive H) form game { } S = (C, E), (C,F ), (D, E), (D,F ) 2 { } ExtensiveIt is Form important Games and to Backward note that Induction we have to include the strategies (A, G) andISCI( 330A, LectureH), even 11, Slide 11 though once A is chosen the G-versus-H choice is moot.