Extensive Form Games and Backward Induction

Home , Backward induction, Perfect information, Sequential game

Recap Perfect-Information Extensive-Form Games Subgame Perfection Backward Induction Introduction

I The normal form game representation does not incorporate any notion of sequence, or time, of the actions of the players

I The extensive form is an alternative representation that makes the temporal structure explicit.

I Two variants:

I perfect information extensive-form games I imperfect-information extensive-form games

Extensive Form Games and Backward Induction ISCI 330 Lecture 11, Slide 3 Recap Perfect-Information Extensive-Form Games Subgame Perfection Backward Induction Perfect-Information Extensive Form Representation

I Represents a ﬁnite sequential game as a rooted tree

I Each internal node represents a particular player’s ’turn’

I Each branch from this internal node represents a diﬀerent choice for that player

I Each terminal node represents a possible ﬁnal outcome of the game

I Usually written with the basal root on top I Parts of the tree are labeled as follows:

I Internal nodes (including the root) are labeled with player identiﬁers

I Branches are labeled with player choices I Terminal nodes are labeled with utility outcomes

Extensive Form Games and Backward Induction ISCI 330 Lecture 11, Slide 4 Recap Perfect-Information Extensive-Form Games Subgame Perfection Backward Induction Example: the sharing game

q 1 ¨¨HH ¨ H ¨¨ HH ¨ H 0–21–12–0 ¨¨ HH ¨ H ¨ Hqqq 222 ¡A ¡A ¡A ¡ A ¡ A ¡ A ¡ A ¡ A ¡ A yesnoyesnoyesno ¡ A ¡ A ¡ A ¡ A ¡ A ¡ Aqqqqqq (0,2)(0,0)(1,1)(0,0)(2,0)(0,0)

Get with a partner and decide on a simple sequential game (e.g. tic-tac-toe) and represent it in extended form

Extensive Form Games and Backward Induction ISCI 330 Lecture 11, Slide 5 Recap Perfect-Information Extensive-Form Games Subgame Perfection Backward Induction Pure Strategies

I In the sharing game (splitting 2 coins) how many pure strategies does each player have?

I player 1: 3; player 2: 8

I Overall, a pure strategy for a player in a perfect-information game is a complete speciﬁcation of which deterministic action to take at every node belonging to that player.

I Can think of a strategy as a complete set of instructions for a proxy who will play for the player in their abscence

Extensive Form Games and Backward Induction ISCI 330 Lecture 11, Slide 8 5.1 Perfect-information extensive-form games 109

¨H1 ¨¨ HH ¨ q H 0–2 ¨ 1–1 H 2–0 ¨¨ HH ¨¨ HH 2 ¨ 2 H 2 ¡A ¡A ¡A ¡ q A ¡ q A ¡ q A no ¡ A yes no ¡ A yes no ¡ A yes ¡ A ¡ A ¡ A ¡ A ¡ A ¡ A (0,0) (2,0) (0,0) (1,1) (0,0) (0,2) q q q q q q Figure 5.1 The Sharing game.

Notice that the deﬁnition contains a subtlety. An agent’s strategy requires a decision Recapat each choicePerfect-Information node, regardless Extensive-Form of whether Games or not itSubgame is possib Perfectionle to reach thatBackward node given Induction the other choice nodes. In the Sharing game above the situation is straightforward— Pureplayer Strategies 1 has three pure Example strategies, and player 2 has eight (why?). But now consider the game shown in Figure 5.2. 1 A B 2 2

C D E F 1 (3,8) (8,3) (5,5) G H

(2,10) (1,0)

Figure 5.2 A perfect-information game in extensive form. What are the pure strategies for player 2?

InI orderS2 to= deﬁne{(C,E a complete); (C,F strategy); (D,E for); this (D,F game,)} each of the players must choose anWhat action are at each the of pure his two strategies choice nodes. forplayer Thus we 1? can enumerate the pure strategies of the players as follows. I S1 = {(B,G); (B,H), (A, G), (A, H)} S1 = (A, G), (A, H), (B, G), (B, H) I This{ is true even though, conditional} on taking A, the choice S = (C, E), (C,F ), (D, E), (D,F ) 2 between{ G and H will never} have to be made ExtensiveIt is Form important Games and to Backward note that Induction we have to include the strategies (A, G) andISCI(A, 330 H Lecture), even 11, Slide 9 though once A is chosen the G-versus-H choice is moot. The deﬁnition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfect- information game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5.2 can be converted into the normal form im- age of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are

Multi Agent Systems, draft of September 19, 2006 Recap Perfect-Information Extensive-Form Games Subgame Perfection Backward Induction Nash Equilibria

Given our new deﬁnition of pure strategy, we are able to reuse our old deﬁnitions of:

I mixed strategies

I best response

I Nash equilibrium Theorem Every perfect information game in extensive form has a PSNE This is easy to see, since the players move sequentially.

Extensive Form Games and Backward Induction ISCI 330 Lecture 11, Slide 10 5.1 Perfect-information extensive-form games 109

¨H1 ¨¨ HH ¨ q H 0–2 ¨ 1–1 H 2–0 ¨¨ HH ¨¨ HH 2 ¨ 2 H 2 ¡A ¡A ¡A ¡ q A ¡ q A ¡ q A no ¡ A yes no ¡ A yes no ¡ A yes ¡ A ¡ A ¡ A ¡ A ¡ A ¡ A (0,0) Recap(2,0) Perfect-Information(0,0) (1,1) Extensive-Form(0,0) Games(0,2) Subgame Perfection Backward Induction q q q q q q InducedFigure Normal 5.1 The Sharing Form game.

Notice that the deﬁnition contains a subtlety. An agent’s strategy requires a decision at each choice node, regardless of whether or not it is possible to reach that node given the other choice nodes. InI theIn Sharing fact, game the above connection the situation to is straightforward— the normal form is even tighter player 1 has three pure strategies, and player 2 has eight (why?). But now consider the game shown in Figure 5.2. I we can “convert” an extensive-form game into normal form 1 A B CE CF DE DF 2 2 AG 3, 8 3, 8 8, 3 8, 3 C D E F AH 3, 8 3, 8 8, 3 8, 3 1 BG 5, 5 2, 10 5, 5 2, 10 (3,8) (8,3) (5,5) G H BH 5, 5 1, 0 5, 5 1, 0

(2,10) (1,0)

Figure 5.2I Athis perfect-information illustrates game the in extensive lack of form. compactness of the normal form

In order to define a complete strategyI games for this aren’t game, each always of the this players small must choose an action at each of his two choiceI nodes.even Thushere we we can write enumerat downe the 16pure payoff strategies pairs instead of 5 of the players as follows. S = (A, G), (A, H), (B, G), (B, H) 1 { } S2 = (C, E), (C,F ), (D, E), (D,F ) { Extensive Form Games and Backward} Induction ISCI 330 Lecture 11, Slide 11 It is important to note that we have to include the strategies (A, G) and (A, H), even though once A is chosen the G-versus-H choice is moot. The definition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfect- information game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5.2 can be converted into the normal form im- age of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are

Multi Agent Systems, draft of September 19, 2006 5.1 Perfect-information extensive-form games 109

(2,10) (1,0)

Figure 5.2I Awhile perfect-information we can game write in extensive any extensive-form form. game as a NF, we can’t In order to deﬁne a completedo strategy the reverse. for this game, each of the players must choose an action at each of his two choiceI nodes.e.g., matchingThus we can enumerat penniese the cannot pure strategies be written as a of the players as follows. perfect-information extensive form game S = (A, G), (A, H), (B, G), (B, H) 1 { } S = (C, E), (C,F ), (D, E), (D,F ) 2 { } It is importantExtensive to note that Form we Games have toand include Backward the Induction strategies (A, G) and (A, H), even ISCI 330 Lecture 11, Slide 11 though once A is chosen the G-versus-H choice is moot. The deﬁnition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfect- information game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5.2 can be converted into the normal form im- age of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are

Multi Agent Systems, draft of September 19, 2006 5.1 Perfect-information extensive-form games 109

¨H1 ¨¨ HH ¨ q H 0–2 ¨ 1–1 H 2–0 ¨¨ HH ¨¨ HH 2 ¨ 2 H 2 ¡A ¡A ¡A ¡ q A ¡ q A ¡ q A no ¡ A yes no ¡ A yes no ¡ A yes ¡ A ¡ A ¡ A ¡ A ¡ A ¡ A (0,0) (2,0) (0,0) (1,1) (0,0) (0,2) q Recap q Perfect-Informationq q Extensive-Formq Games q Subgame Perfection Backward Induction InducedFigure Normal 5.1 The Sharing Form game.

(2,10) (1,0)

Figure 5.2 A perfect-information game in extensive form. I What are the (three) pure-strategy equilibria?

In order to deﬁne a complete strategyI (A, for G this), ( game,C,F each) of the players must choose an action at each of his two choice nodes. Thus we can enumerate the pure strategies I (A, H), (C,F ) of the players as follows. I (B,H), (C,E) S = (A, G), (A, H), (B, G), (B, H) 1 { } S = (C, E), (C,F ), (D, E), (D,F ) 2 { } It is importantExtensive to note that Form we Games have toand include Backward the Induction strategies (A, G) and (A, H), even ISCI 330 Lecture 11, Slide 11 though once A is chosen the G-versus-H choice is moot. The deﬁnition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfect- information game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5.2 can be converted into the normal form im- age of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are

Multi Agent Systems, draft of September 19, 2006