Perfect-Information Extensive-Form Games Subgame Perfection

Recap Perfect-Information Extensive-Form Games Subgame Perfection

Extensive Form Games

Lecture 7

Extensive Form Games Lecture 7, Slide 1 Recap Perfect-Information Extensive-Form Games Subgame Perfection Lecture Overview

1 Recap

2 Perfect-Information Extensive-Form Games

3 Subgame Perfection

Extensive Form Games Lecture 7, Slide 2 Recap Perfect-Information Extensive-Form Games Subgame Perfection Computational Problems in Domination

Identifying strategies dominated by a pure strategy polynomial, straightforward algorithm Identifying strategies dominated by a mixed strategy polynomial, somewhat tricky LP Identifying strategies that survive iterated elimination repeated calls to the above LP Asking whether a strategy survives iterated elimination under all elimination orderings polynomial for strict domination (elimination doesn’t matter) NP-complete otherwise

Extensive Form Games Lecture 7, Slide 3 Recap Perfect-Information Extensive-Form Games Subgame Perfection Rationalizability

Rather than ask what is irrational, ask what is a best response to some beliefs about the opponent assumes opponent is rational assumes opponent knows that you and the others are rational ... Equilibrium strategies are always rationalizable; so are lots of other strategies (but not everything). In two-player games, rationalizable survives iterated removal of strictly dominated strategies.⇔

Extensive Form Games Lecture 7, Slide 4 Recap Perfect-Information Extensive-Form Games Subgame Perfection Formal deﬁnition

Deﬁnition (Correlated equilibrium) Given an n-agent game G = (N, A, u), a correlated equilibrium is a tuple (v, π, σ), where v is a tuple of random variables v = (v1, . . . , vn) with respective domains D = (D1,...,Dn), π is a joint distribution over v, σ = (σ1, . . . , σn) is a vector of mappings σi : Di Ai, and for each agent i and every mapping 0 7→ σ : Di Ai it is the case that i 7→ X π(d)ui (σ1(d1), . . . , σi(di), . . . , σn(dn)) d∈D X 0 π(d)ui σ1(d1), . . . , σ (di), . . . , σn(dn) . ≥ i d∈D

Extensive Form Games Lecture 7, Slide 5 Recap Perfect-Information Extensive-Form Games Subgame Perfection Existence

Theorem For every Nash equilibrium σ∗ there exists a corresponding correlated equilibrium σ.

This is easy to show:

let Di = Ai Q ∗ let π(d) = i∈N σi (di) σi maps each di to the corresponding ai. Thus, correlated equilibria always exist

Extensive Form Games Lecture 7, Slide 6 Recap Perfect-Information Extensive-Form Games Subgame Perfection Remarks

Not every correlated equilibrium is equivalent to a Nash equilibrium thus, correlated equilibrium is a weaker notion than Nash Any convex combination of the payoﬀs achievable under correlated equilibria is itself realizable under a correlated equilibrium start with the Nash equilibria (each of which is a CE) introduce a second randomizing device that selects which CE the agents will play regardless of the probabilities, no agent has incentive to deviate the probabilities can be adjusted to achieve any convex combination of the equilibrium payoﬀs the randomizing devices can be combined

Extensive Form Games Lecture 7, Slide 7 Recap Perfect-Information Extensive-Form Games Subgame Perfection Computing CE

X 0 0 [ui(a) ui(a , a−i)]p(a) 0 i N, ai, a Ai − i ≥ ∀ ∈ ∀ i ∈ a∈A|ai∈a p(a) 0 a A X ≥ ∀ ∈ p(a) = 1 a∈A

variables: p(a); constants: ui(a) we could ﬁnd the social-welfare maximizing CE by adding an objective function X X maximize: p(a) ui(a). a∈A i∈N

Extensive Form Games Lecture 7, Slide 8 Recap Perfect-Information Extensive-Form Games Subgame Perfection Lecture Overview

1 Recap

2 Perfect-Information Extensive-Form Games

3 Subgame Perfection

Extensive Form Games Lecture 7, Slide 9 Recap Perfect-Information Extensive-Form Games Subgame Perfection Introduction

The normal form game representation does not incorporate any notion of sequence, or time, of the actions of the players The extensive form is an alternative representation that makes the temporal structure explicit. Two variants: perfect information extensive-form games imperfect-information extensive-form games

Extensive Form Games Lecture 7, Slide 10 Choice nodes: H Action function: χ : H 2A Player function: ρ : H →N →

Actions: A Choice nodes and labels for these nodes:

Terminal nodes: Z Successor function: σ : H A H Z × → ∪ Utility function: u = (u1, . . . , un); ui : Z R →

Recap Perfect-Information Extensive-Form Games Subgame Perfection Deﬁnition

A (ﬁnite) perfect-information game (in extensive form) is deﬁned by the tuple (N, A, H, Z, χ, ρ, σ, u), where: Players: N is a set of n players

Extensive Form Games Lecture 7, Slide 11 Choice nodes: H Action function: χ : H 2A Player function: ρ : H →N →

Choice nodes and labels for these nodes:

Terminal nodes: Z Successor function: σ : H A H Z × → ∪ Utility function: u = (u1, . . . , un); ui : Z R →

Recap Perfect-Information Extensive-Form Games Subgame Perfection Deﬁnition

A (ﬁnite) perfect-information game (in extensive form) is deﬁned by the tuple (N, A, H, Z, χ, ρ, σ, u), where: Players: N Actions: A is a (single) set of actions

Extensive Form Games Lecture 7, Slide 11 Action function: χ : H 2A Player function: ρ : H →N → Terminal nodes: Z Successor function: σ : H A H Z × → ∪ Utility function: u = (u1, . . . , un); ui : Z R →

Recap Perfect-Information Extensive-Form Games Subgame Perfection Deﬁnition

Extensive Form Games Lecture 7, Slide 11 Player function: ρ : H N → Terminal nodes: Z Successor function: σ : H A H Z × → ∪ Utility function: u = (u1, . . . , un); ui : Z R →

Recap Perfect-Information Extensive-Form Games Subgame Perfection Deﬁnition

Extensive Form Games Lecture 7, Slide 11 Terminal nodes: Z Successor function: σ : H A H Z × → ∪ Utility function: u = (u1, . . . , un); ui : Z R →

Recap Perfect-Information Extensive-Form Games Subgame Perfection Deﬁnition

Extensive Form Games Lecture 7, Slide 11 Successor function: σ : H A H Z × → ∪ Utility function: u = (u1, . . . , un); ui : Z R →

Recap Perfect-Information Extensive-Form Games Subgame Perfection Deﬁnition

Extensive Form Games Lecture 7, Slide 11 Utility function: u = (u1, . . . , un); ui : Z R →

Recap Perfect-Information Extensive-Form Games Subgame Perfection Deﬁnition

A (ﬁnite) perfect-information game (in extensive form) is deﬁned by the tuple (N, A, H, Z, χ, ρ, σ, u), where: Players: N Actions: A Choice nodes and labels for these nodes: Choice nodes: H Action function: χ : H 2A Player function: ρ : H →N → Terminal nodes: Z Successor function: σ : H A H Z maps a choice node and an action to a new choice× node→ or∪ terminal node such that for all h1, h2 H and a1, a2 A, if ∈ ∈ σ(h1, a1) = σ(h2, a2) then h1 = h2 and a1 = a2 The choice nodes form a tree, so we can identify a node with its history.

Extensive Form Games Lecture 7, Slide 11 Recap Perfect-Information Extensive-Form Games Subgame Perfection Deﬁnition

A (ﬁnite) perfect-information game (in extensive form) is deﬁned by the tuple (N, A, H, Z, χ, ρ, σ, u), where: Players: N Actions: A Choice nodes and labels for these nodes: Choice nodes: H Action function: χ : H 2A Player function: ρ : H →N → Terminal nodes: Z Successor function: σ : H A H Z × → ∪ Utility function: u = (u1, . . . , un); ui : Z R is a utility → function for player i on the terminal nodes Z

Extensive Form Games Lecture 7, Slide 11 Play as a fun game, dividing 100 dollar coins. (Play each partner only once.)

Recap Perfect-Information Extensive-Form Games Subgame Perfection Example: the sharing game

1 ¨¨HH ¨ q H ¨¨ HH ¨ H 0–21–12–0 ¨¨ HH ¨¨ HH 222 ¡A ¡A ¡A ¡ A ¡ A ¡ qqq A ¡ A ¡ A ¡ A yesnoyesnoyesno ¡ A ¡ A ¡ A ¡ A ¡ A ¡ A (0,2)(0,0)(1,1)(0,0)(2,0)(0,0) qqqqqq

Extensive Form Games Lecture 7, Slide 12 Recap Perfect-Information Extensive-Form Games Subgame Perfection Example: the sharing game

1 ¨¨HH ¨ q H ¨¨ HH ¨ H 0–21–12–0 ¨¨ HH ¨¨ HH 222 ¡A ¡A ¡A ¡ A ¡ A ¡ qqq A ¡ A ¡ A ¡ A yesnoyesnoyesno ¡ A ¡ A ¡ A ¡ A ¡ A ¡ A (0,2)(0,0)(1,1)(0,0)(2,0)(0,0) qqqqqq Play as a fun game, dividing 100 dollar coins. (Play each partner only once.)

Extensive Form Games Lecture 7, Slide 12 player 1: 3; player 2: 8 Overall, a pure strategy for a player in a perfect-information game is a complete speciﬁcation of which deterministic action to take at every node belonging to that player. Deﬁnition (pure strategies) Let G = (N, A, H, Z, χ, ρ, σ, u) be a perfect-information extensive-form game. Then the pure strategies of player i consist of the cross product χ(h) h∈H,ρ×(h)=i

Recap Perfect-Information Extensive-Form Games Subgame Perfection Pure Strategies

In the sharing game (splitting 2 coins) how many pure strategies does each player have?

Extensive Form Games Lecture 7, Slide 13 Overall, a pure strategy for a player in a perfect-information game is a complete speciﬁcation of which deterministic action to take at every node belonging to that player. Deﬁnition (pure strategies) Let G = (N, A, H, Z, χ, ρ, σ, u) be a perfect-information extensive-form game. Then the pure strategies of player i consist of the cross product χ(h) h∈H,ρ×(h)=i

Recap Perfect-Information Extensive-Form Games Subgame Perfection Pure Strategies

In the sharing game (splitting 2 coins) how many pure strategies does each player have? player 1: 3; player 2: 8

Extensive Form Games Lecture 7, Slide 13 Recap Perfect-Information Extensive-Form Games Subgame Perfection Pure Strategies

In the sharing game (splitting 2 coins) how many pure strategies does each player have? player 1: 3; player 2: 8 Overall, a pure strategy for a player in a perfect-information game is a complete speciﬁcation of which deterministic action to take at every node belonging to that player. Deﬁnition (pure strategies) Let G = (N, A, H, Z, χ, ρ, σ, u) be a perfect-information extensive-form game. Then the pure strategies of player i consist of the cross product χ(h) h∈H,ρ×(h)=i

Extensive Form Games Lecture 7, Slide 13 S2 = (C,E); (C,F ); (D,E); (D,F ) { } What are the pure strategies for player 1? S1 = (B,G); (B,H), (A, G), (A, H) { } This is true even though, conditional on taking A, the choice between G and H will never have to be made

5.1 Perfect-information extensive-form games 109

¨H1 ¨¨ HH ¨ q H 0–2 ¨ 1–1 H 2–0 ¨¨ HH ¨¨ HH 2 ¨ 2 H 2 ¡A ¡A ¡A ¡ q A ¡ q A ¡ q A no ¡ A yes no ¡ A yes no ¡ A yes ¡ A ¡ A ¡ A ¡ A ¡ A ¡ A (0,0) (2,0) (0,0) (1,1) (0,0) (0,2) q q q q q q Figure 5.1 The Sharing game.

Notice that the deﬁnition contains a subtlety. An agent’s strategy requires a decision Recapat each choice node, regardlessPerfect-Information of whether Extensive-Form or not it Games is possib le to reach thatSubgame node given Perfection the other choice nodes. In the Sharing game above the situation is straightforward— Pureplayer Strategies 1 has three pure Example strategies, and player 2 has eight (why?). But now consider the game shown in Figure 5.2. 1 A B 2 2

C D E F 1 (3,8) (8,3) (5,5) G H

(2,10) (1,0)

Figure 5.2 A perfect-information game in extensive form. What are the pure strategies for player 2? In order to deﬁne a complete strategy for this game, each of the players must choose an action at each of his two choice nodes. Thus we can enumerate the pure strategies of the players as follows. S = (A, G), (A, H), (B, G), (B, H) 1 { } S = (C, E), (C,F ), (D, E), (D,F ) 2 { } ExtensiveIt is Form important Games to note that we have to include the strategies (A, G) and (A, HLecture), even 7, Slide 14 though once A is chosen the G-versus-H choice is moot. The deﬁnition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfect- information game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5.2 can be converted into the normal form im- age of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are

Multi Agent Systems, draft of September 19, 2006 What are the pure strategies for player 1? S1 = (B,G); (B,H), (A, G), (A, H) { } This is true even though, conditional on taking A, the choice between G and H will never have to be made

5.1 Perfect-information extensive-form games 109

C D E F 1 (3,8) (8,3) (5,5) G H

(2,10) (1,0)

Figure 5.2 A perfect-information game in extensive form. What are the pure strategies for player 2? In orderS2 to= deﬁne(C,E a complete); (C,F strategy); (D,E for); this (D,F game,) each of the players must choose an action at each{ of his two choice nodes. Thus we can} enumerate the pure strategies of the players as follows. S = (A, G), (A, H), (B, G), (B, H) 1 { } S = (C, E), (C,F ), (D, E), (D,F ) 2 { } ExtensiveIt is Form important Games to note that we have to include the strategies (A, G) and (A, HLecture), even 7, Slide 14 though once A is chosen the G-versus-H choice is moot. The deﬁnition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfect- information game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5.2 can be converted into the normal form im- age of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are

Multi Agent Systems, draft of September 19, 2006 S1 = (B,G); (B,H), (A, G), (A, H) { } This is true even though, conditional on taking A, the choice between G and H will never have to be made

5.1 Perfect-information extensive-form games 109

C D E F 1 (3,8) (8,3) (5,5) G H

(2,10) (1,0)

Figure 5.2 A perfect-information game in extensive form. What are the pure strategies for player 2? In orderS2 to= deﬁne(C,E a complete); (C,F strategy); (D,E for); this (D,F game,) each of the players must choose { } anWhat action are at each the of pure his two strategies choice nodes. forplayer Thus we 1? can enumerate the pure strategies of the players as follows. S = (A, G), (A, H), (B, G), (B, H) 1 { } S = (C, E), (C,F ), (D, E), (D,F ) 2 { } ExtensiveIt is Form important Games to note that we have to include the strategies (A, G) and (A, HLecture), even 7, Slide 14 though once A is chosen the G-versus-H choice is moot. The deﬁnition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfect- information game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5.2 can be converted into the normal form im- age of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are

Multi Agent Systems, draft of September 19, 2006 5.1 Perfect-information extensive-form games 109

C D E F 1 (3,8) (8,3) (5,5) G H

(2,10) (1,0)

Figure 5.2 A perfect-information game in extensive form. What are the pure strategies for player 2? In orderS2 to= deﬁne(C,E a complete); (C,F strategy); (D,E for); this (D,F game,) each of the players must choose { } anWhat action are at each the of pure his two strategies choice nodes. forplayer Thus we 1? can enumerate the pure strategies of the players as follows. S1 = (B,G); (B,H), (A, G), (A, H) S1 = (A, G{ ), (A, H), (B, G), (B, H) } This{ is true even though, conditional} on taking A, the choice S = (C, E), (C,F ), (D, E), (D,F ) 2 between{ G and H will never} have to be made ExtensiveIt is Form important Games to note that we have to include the strategies (A, G) and (A, HLecture), even 7, Slide 14 though once A is chosen the G-versus-H choice is moot. The deﬁnition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfect- information game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5.2 can be converted into the normal form im- age of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are

Multi Agent Systems, draft of September 19, 2006 Recap Perfect-Information Extensive-Form Games Subgame Perfection Nash Equilibria

Given our new deﬁnition of pure strategy, we are able to reuse our old deﬁnitions of: mixed strategies best response Nash equilibrium

Theorem Every perfect information game in extensive form has a PSNE

This is easy to see, since the players move sequentially.

Extensive Form Games Lecture 7, Slide 15 CECFDEDF AG 3, 8 3, 8 8, 3 8, 3 AH 3, 8 3, 8 8, 3 8, 3 BG 5, 5 2, 10 5, 5 2, 10 BH 5, 5 1, 0 5, 5 1, 0

5.1 Perfect-information extensive-form games 109

¨H1 ¨¨ HH ¨ q H 0–2 ¨ 1–1 H 2–0 ¨¨ HH ¨¨ HH 2 ¨ 2 H 2 ¡A ¡A ¡A ¡ q A ¡ q A ¡ q A no yes no yes no yes ¡ RecapA ¡ APerfect-Information¡ Extensive-FormA Games Subgame Perfection ¡ A ¡ A ¡ A ¡ A ¡ A ¡ A (0,0) Induced(2,0) Normal(0,0) (1,1) Form (0,0) (0,2) q q q q q q Figure 5.1 The Sharing game.

Notice that the deﬁnition contains a subtlety. An agent’s strategy requires a decision at each choice node, regardless of whether or not it is possible to reach that node given the other choice nodes. In theIn Sharing fact, game the above connection the situation to is straightforward— the normal form is even tighter player 1 has three pure strategies, and player 2 has eight (why?). But now consider the game shown in Figure 5.2. we can “convert” an extensive-form game into normal form 1 A B 2 2

C D E F 1 (3,8) (8,3) (5,5) G H

(2,10) (1,0)

Figure 5.2 A perfect-information game in extensive form.

In order to deﬁne a complete strategy for this game, each of the players must choose an action at each of his two choice nodes. Thus we can enumerate the pure strategies of the players as follows.

S = (A, GExtensive), (A, H Form), (B, Games G), (B, H) Lecture 7, Slide 16 1 { } S = (C, E), (C,F ), (D, E), (D,F ) 2 { } It is important to note that we have to include the strategies (A, G) and (A, H), even though once A is chosen the G-versus-H choice is moot. The deﬁnition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfect- information game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5.2 can be converted into the normal form im- age of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are

Multi Agent Systems, draft of September 19, 2006 5.1 Perfect-information extensive-form games 109

(2,10) (1,0)

Figure 5.2 A perfect-information game in extensive form.

In order to deﬁne a complete strategy for this game, each of the players must choose an action at each of his two choice nodes. Thus we can enumerate the pure strategies of the players as follows.

Multi Agent Systems, draft of September 19, 2006 5.1 Perfect-information extensive-form games 109

¨H1 ¨¨ HH ¨ q H 0–2 ¨ 1–1 H 2–0 ¨¨ HH ¨¨ HH 2 ¨ 2 H 2 ¡A ¡A ¡A ¡ q A ¡ q A ¡ q A no ¡ A yes no ¡ A yes no ¡ A yes ¡ A ¡ A ¡ A ¡ A ¡ A ¡ A (0,0) Recap(2,0) (0,0) Perfect-Information(1,1) (0,0) Extensive-Form(0,2) Games Subgame Perfection q q q q q q InducedFigure Normal 5.1 The Sharing Form game.

(2,10) (1,0) Figure 5.2 Athis perfect-information illustrates game the in extensive lack of form. compactness of the normal form In order to define a complete strategygames for this aren’t game, each always of the this players small must choose an action at each of his two choice nodes.even Thushere we we can write enumerat downe the 16pure payoff strategies pairs instead of 5 of the players as follows. S = (A, G), (A, H), (B, G), (B, H) 1 { } S2 = (C, E), (C,F ), (D, E), (D,F ) { Extensive Form Games } Lecture 7, Slide 16 It is important to note that we have to include the strategies (A, G) and (A, H), even though once A is chosen the G-versus-H choice is moot. The definition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfect- information game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5.2 can be converted into the normal form im- age of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are

Multi Agent Systems, draft of September 19, 2006 5.1 Perfect-information extensive-form games 109

(2,10) (1,0) Figure 5.2 Awhile perfect-information we can game write in extensive any extensive-form form. game as a NF, we can’t In order to deﬁne a completedo strategy the reverse. for this game, each of the players must choose an action at each of his two choice nodes.e.g., matchingThus we can enumerat penniese the cannot pure strategies be written as a of the players as follows. perfect-information extensive form game S = (A, G), (A, H), (B, G), (B, H) 1 { } S = (C, E), (C,F ), (D, E), (D,F ) 2 { } It is importantExtensive to note that Form we Games have to include the strategies (A, G) and (A, H), even Lecture 7, Slide 16 though once A is chosen the G-versus-H choice is moot. The deﬁnition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfect- information game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5.2 can be converted into the normal form im- age of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are

Multi Agent Systems, draft of September 19, 2006 (A, G), (C,F ) (A, H), (C,F ) (B,H), (C,E)

5.1 Perfect-information extensive-form games 109

¨H1 ¨¨ HH ¨ q H 0–2 ¨ 1–1 H 2–0 ¨¨ HH ¨¨ HH 2 ¨ 2 H 2 ¡A ¡A ¡A ¡ q A ¡ q A ¡ q A no ¡ A yes no ¡ A yes no ¡ A yes ¡ A ¡ A ¡ A ¡ A ¡ A ¡ A (0,0) (2,0) (0,0) (1,1) (0,0) (0,2) q Recap q q Perfect-Informationq q Extensive-Formq Games Subgame Perfection InducedFigure Normal 5.1 The Sharing Form game.

(2,10) (1,0) Figure 5.2 AWhat perfect-information are the game (three) in extensive pure-strategy form. equilibria? In order to deﬁne a complete strategy for this game, each of the players must choose an action at each of his two choice nodes. Thus we can enumerate the pure strategies of the players as follows. S = (A, G), (A, H), (B, G), (B, H) 1 { } S = (C, E), (C,F ), (D, E), (D,F ) 2 { } It is importantExtensive to note that Form we Games have to include the strategies (A, G) and (A, H), even Lecture 7, Slide 16 though once A is chosen the G-versus-H choice is moot. The deﬁnition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfect- information game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5.2 can be converted into the normal form im- age of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are

Multi Agent Systems, draft of September 19, 2006 5.1 Perfect-information extensive-form games 109

(2,10) (1,0) Figure 5.2 AWhat perfect-information are the game (three) in extensive pure-strategy form. equilibria? In order to deﬁne a complete strategy(A, for G this), ( game,C,F each) of the players must choose an action at each of his two choice nodes.(A, H Thus), (C,F we can) enumerate the pure strategies of the players as follows. (B,H), (C,E) S = (A, G), (A, H), (B, G), (B, H) 1 { } S = (C, E), (C,F ), (D, E), (D,F ) 2 { } It is importantExtensive to note that Form we Games have to include the strategies (A, G) and (A, H), even Lecture 7, Slide 16 though once A is chosen the G-versus-H choice is moot. The deﬁnition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfect- information game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5.2 can be converted into the normal form im- age of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are

Multi Agent Systems, draft of September 19, 2006 5.1 Perfect-information extensive-form games 109

Multi Agent Systems, draft of September 19, 2006 Recap Perfect-Information Extensive-Form Games Subgame Perfection Lecture Overview

1 Recap

2 Perfect-Information Extensive-Form Games

3 Subgame Perfection

Extensive Form Games Lecture 7, Slide 17 He does it to threaten player 2, to prevent him from choosing F , and so gets 5 However, this seems like a non-credible threat If player 1 reached his second decision node, would he really follow through and play H?

5.1 Perfect-information extensive-form games 109

¨H1 ¨¨ HH ¨ q H 0–2 ¨ 1–1 H 2–0 ¨¨ HH ¨¨ HH 2 ¨ 2 H 2 ¡A ¡A ¡A ¡ q A ¡ q A ¡ q A no ¡ A yes no ¡ A yes no ¡ A yes ¡ A ¡ A ¡ A ¡ A ¡ A ¡ A (0,0) (2,0) (0,0) (1,1) (0,0) (0,2) q q q q q q Figure 5.1 The Sharing game. Recap Perfect-Information Extensive-Form Games Subgame Perfection Notice that the deﬁnition contains a subtlety. An agent’s strategy requires a decision at each choice node, regardless of whether or not it is possible to reach that node given Subgame Perfectionthe other choice nodes. In the Sharing game above the situation is straightforward— player 1 has three pure strategies, and player 2 has eight (why?). But now consider the game shown in Figure 5.2. 1 A B 2 2

C D E F 1 (3,8) (8,3) (5,5) G H

(2,10) (1,0)

Figure 5.2 A perfect-information game in extensive form. There’sIn something order to deﬁne a complete intuitively strategy for this wrong game, each with of the players the must equilibrium choose an action at each of his two choice nodes. Thus we can enumerate the pure strategies (B,Hof), the(C,E players as) follows. WhyS = would(A, G), ( playerA, H), (B, 1G), ever(B, H) choose to play H if he got to the 1 { } secondS = ( choiceC, E), (C,F node?), (D, E), (D,F ) 2 { } It is importantAfter to all, noteG thatdominates we have to includeH thefor strategies him(A, G) and (A, H), even though once A is chosen the G-versus-H choice is moot. The deﬁnition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfect- information game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5.2 can be converted into the normal form im- age of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are

Multi Agent Systems, draft of September 19, 2006

Extensive Form Games Lecture 7, Slide 18 5.1 Perfect-information extensive-form games 109

C D E F 1 (3,8) (8,3) (5,5) G H

(2,10) (1,0)

Figure 5.2 A perfect-information game in extensive form. There’sIn something order to deﬁne a complete intuitively strategy for this wrong game, each with of the players the must equilibrium choose an action at each of his two choice nodes. Thus we can enumerate the pure strategies (B,Hof), the(C,E players as) follows. WhyS = would(A, G), ( playerA, H), (B, 1G), ever(B, H) choose to play H if he got to the 1 { } secondS = ( choiceC, E), (C,F node?), (D, E), (D,F ) 2 { } It is importantAfter to all, noteG thatdominates we have to includeH thefor strategies him(A, G) and (A, H), even though once A is chosen the G-versus-H choice is moot. HeThe does deﬁnition it ofto best threaten response and playerNash equilibria 2, toin this prevent game are exactly him as from they choosing Fare, andin for normal so gets form games. 5 Indeed, this example illustrates how every perfect- information game can be converted to an equivalent normal form game. For example, the perfect-informationHowever, this game ofseems Figure 5.2 like cana be non-credibleconverted into the normal threat form im- age of theIf game, player shown 1 in reached Figure 5.3. Clearly,his second the strategy decision spaces of the node, two games would are he really follow throughMulti Agent Systems and, play draft ofH September? 19, 2006

Extensive Form Games Lecture 7, Slide 18 Recap Perfect-Information Extensive-Form Games Subgame Perfection Formal Deﬁnition

Deﬁnition (subgame of G rooted at h) The subgame of G rooted at h is the restriction of G to the descendents of H.

Deﬁnition (subgames of G) The set of subgames of G is deﬁned by the subgames of G rooted at each of the nodes in G.

s is a subgame perfect equilibrium of G iﬀ for any subgame G0 of G, the restriction of s to G0 is a Nash equilibrium of G0 Notes: since G is its own subgame, every SPE is a NE. this deﬁnition rules out “non-credible threats”

Extensive Form Games Lecture 7, Slide 19 5.1 Perfect-information extensive-form games 109

¨H1 ¨¨ HH ¨ q H 0–2 ¨ 1–1 H 2–0 ¨¨ HH ¨¨ HH 2 ¨ 2 H 2 ¡A ¡A ¡A ¡ q A ¡ q A ¡ q A no ¡ A yes no ¡ A yes no ¡ A yes ¡ A ¡ A ¡ A ¡ A ¡ A ¡ A Recap (0,0)Perfect-Information(2,0) (0,0) Extensive-Form(1,1) Games(0,0) (0,2) Subgame Perfection q q q q q q Figure 5.1 The Sharing game. Which equilibria are subgame perfect? Notice that the deﬁnition contains a subtlety. An agent’s strategy requires a decision at each choice node, regardless of whether or not it is possible to reach that node given the other choice nodes. In the Sharing game above the situation is straightforward— player 1 has three pure strategies, and player 2 has eight (why?). But now consider the game shown in Figure 5.2. 1 A B 2 2

C D E F 1 (3,8) (8,3) (5,5) G H

(2,10) (1,0)

Figure 5.2 A perfect-information game in extensive form.

In order to deﬁne a complete strategy for this game, each of the players must choose Whichan equilibria action at each of his from two choice the nodes. example Thus we can are enumerat subgamee the pure strategies perfect? of the players as follows. (A, G), (C,F ): S = (A, G), (A, H), (B, G), (B, H) 1 { } (B,HS =),(C,(C,E E), (C,F):), (D, E), (D,F ) 2 { } (A,It is Himportant), (C,F to note) that: we have to include the strategies (A, G) and (A, H), even though once A is chosen the G-versus-H choice is moot. The deﬁnition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfect- information game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5.2 can be converted into the normal form im- age of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are Extensive Form Games Lecture 7, Slide 20 Multi Agent Systems, draft of September 19, 2006 5.1 Perfect-information extensive-form games 109

C D E F 1 (3,8) (8,3) (5,5) G H

(2,10) (1,0)

Figure 5.2 A perfect-information game in extensive form.

In order to deﬁne a complete strategy for this game, each of the players must choose Whichan equilibria action at each of his from two choice the nodes. example Thus we can are enumerat subgamee the pure strategies perfect? of the players as follows. (A, G), (C,F ): is subgame perfect S = (A, G), (A, H), (B, G), (B, H) 1 { } (B,HS =),(C,(C,E E), (C,F):), (D, E), (D,F ) 2 { } (A,It is Himportant), (C,F to note) that: we have to include the strategies (A, G) and (A, H), even though once A is chosen the G-versus-H choice is moot. The deﬁnition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfect- information game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5.2 can be converted into the normal form im- age of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are Extensive Form Games Lecture 7, Slide 20 Multi Agent Systems, draft of September 19, 2006 5.1 Perfect-information extensive-form games 109

¨H1 ¨¨ HH ¨ q H 0–2 ¨ 1–1 H 2–0 ¨¨ HH ¨¨ HH 2 ¨ 2 H 2 ¡A ¡A ¡A ¡ q A ¡ q A ¡ q A no ¡ A yes no ¡ A yes no ¡ A yes ¡ A ¡ A ¡ A ¡ A ¡ A ¡ A (0,0) (2,0) (0,0) (1,1) (0,0) (0,2) Recap Perfect-Informationq q Extensive-Formq q Games q q Subgame Perfection Figure 5.1 The Sharing game. Which equilibria are subgame perfect? Notice that the deﬁnition contains a subtlety. An agent’s strategy requires a decision at each choice node, regardless of whether or not it is possible to reach that node given the other choice nodes. In the Sharing game above the situation is straightforward— player 1 has three pure strategies, and player 2 has eight (why?). But now consider the game shown in Figure 5.2. 1 A B 2 2

C D E F 1 (3,8) (8,3) (5,5) G H

(2,10) (1,0)

Figure 5.2 A perfect-information game in extensive form.

In order to deﬁne a complete strategy for this game, each of the players must choose Whichan equilibria action at each of his from two choice the nodes. example Thus we can are enumerat subgamee the pure strategies perfect? of the players as follows. (A, G), (C,F ): is subgame perfect S = (A, G), (A, H), (B, G), (B, H) 1 { } (B,HS =),(C,(C,E E), (C,F):), (D,B,H E), (D,F) is) an non-credible threat; not subgame 2 { } perfectIt is important to note that we have to include the strategies (A, G) and (A, H), even (A,though H once), (AC,Fis chosen): the G-versus-H choice is moot. The deﬁnition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfect- information game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5.2 can be converted into the normal form im- age of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are Extensive Form Games Lecture 7, Slide 20 Multi Agent Systems, draft of September 19, 2006 5.1 Perfect-information extensive-form games 109

¨H1 ¨¨ HH ¨ q H 0–2 ¨ 1–1 H 2–0 ¨¨ HH ¨¨ HH 2 ¨ 2 H 2 ¡A ¡A ¡A ¡ q A ¡ q A ¡ q A no ¡ A yes no ¡ A yes no ¡ A yes ¡ A ¡ A ¡ A ¡ A ¡ A ¡ A (0,0) (2,0) (0,0) (1,1) (0,0) (0,2) q q q q q q Recap Perfect-Information Extensive-Form Games Subgame Perfection Figure 5.1 The Sharing game.

Which equilibriaNotice that are the deﬁnition subgame contains a subtlety. perfect? An agent’s strategy requires a decision at each choice node, regardless of whether or not it is possible to reach that node given the other choice nodes. In the Sharing game above the situation is straightforward— player 1 has three pure strategies, and player 2 has eight (why?). But now consider the game shown in Figure 5.2. 1 A B 2 2

C D E F 1 (3,8) (8,3) (5,5) G H

(2,10) (1,0)

Figure 5.2 A perfect-information game in extensive form.

In order to define a complete strategy for this game, each of the players must choose Whichan equilibria action at each of his from two choice the nodes. example Thus we can are enumerat subgamee the pure strategies perfect? of the players as follows. (A, G), (C,F ): is subgame perfect S = (A, G), (A, H), (B, G), (B, H) 1 { } (B,HS =),(C,(C,E E), (C,F):), (D,B,H E), (D,F) is) an non-credible threat; not subgame 2 { } perfectIt is important to note that we have to include the strategies (A, G) and (A, H), even (A,though H once), (AC,Fis chosen): the(A,G-versus- H) Hischoice also is moot. non-credible, even though H is The definition of best response and Nash equilibria in this game are exactly as they “off-path”are in for normal form games. Indeed, this example illustrates how every perfect- information game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5.2 can be converted into the normal form im- age of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are

Extensive Form Games Multi Agent Systems, draft of September 19, 2006 Lecture 7, Slide 20