Extensive-Form Games with Perfect Information

Jeﬀrey Ely

April 22, 2015

Jeﬀrey Ely Extensive-Form Games with Perfect Information Extensive-Form Games With Perfect Information

Recall the alternating oﬀers bargaining game. It is an extensive-form game. (Described in terms of sequences of moves.) In this game, each time a player moves, he knows everything that has happened in the past. He can compute exactly how the game will end given any proﬁle of continuation strategies Such a game is said to have perfect information.

Jeﬀrey Ely Extensive-Form Games with Perfect Information Simple Extensive-Form Game With Perfect Information

1b ¨ H L ¨ H R ¨¨ HH 2¨r ¨ HH2r AB @ C @ D @ @ r @r r @r 1, 0 2, 3 4, 1 −1, 0 Figure: An extensive game

Jeﬀrey Ely Extensive-Form Games with Perfect Information Game Trees

Nodes H Initial node. Terminal Nodes Z ⊂ H. Player correspondence N : H ⇒ {1, ... , n} (N(h) indicates who moves at h. They move simultaneously)

Actions Ai (h) ̸= ∅ at each h where i ∈ N(h). ▶ Action proﬁles can be identiﬁed with branches in the tree.

Payoﬀs ui : Z → R.

Jeﬀrey Ely Extensive-Form Games with Perfect Information Strategies

A strategy for player i is a function σi specifying σi (h) ∈ Ai (h) for each h such that i ∈ N(h).

Jeﬀrey Ely Extensive-Form Games with Perfect Information Strategic Form

Given any proﬁle of strategies σ, there is a unique terminal node z(σ) that will be reached.

We deﬁne ui (σ) = ui (z(σ)). We have thus obtained a strategic-form representation of the game. We can apply strategic-form solution concepts.

Jeﬀrey Ely Extensive-Form Games with Perfect Information Example

Rationalizability: ▶ The strategy AD is never a best-reply for player 2. ▶ All other strategies are rationalizable (I think.) Nash equilibrium: ▶ The profile (L, BD) is a Nash equilibrium. ▶ The profile (R, BC ) is a Nash equilibrium. ▶ The profile (R, AC ) is a Nash equilibrium. ▶ There are many mixed equilibria. Iterative elimination of weakly dominated strategies: ▶ Any strategy for 2 that plays either A or D is weakly dominated. ▶ Therefore BC is the only weakly un-dominated strategy for 2. ▶ Next we can eliminate L for 1. ▶ The order of elimination doesn’t matter in this example.

Jeﬀrey Ely Extensive-Form Games with Perfect Information Subgames

Associated with every non-terminal node h, there is a subgame consisting of all the nodes in the tree that follows h. The game as a whole is a subgame All other subgames are called proper subgames Given a strategy profile σ ▶ Denote by σ|h the continuation strategy profile in the subgame beginning at h. ▶ Denote by z(σ|h) the terminal node reached by σ beginning from h. ▶ Denote by ui (σ|h), the continuation payoff

ui (σ|h) = ui (z(σ|h))

Jeﬀrey Ely Extensive-Form Games with Perfect Information Subgame Perfect Nash Equilibrium

A strategy proﬁle σ is a subgame perfect Nash equilibrium if for every non-terminal node h, the continuation strategy proﬁle σ|h is a Nash equilibrium of the subgame that begins at h, i.e.

σ| ≥ σ′ σ | ui ( h) ui (( i , −i ) h) σ′ for every strategy i .

Jeﬀrey Ely Extensive-Form Games with Perfect Information Example

The unique subgame-perfect Nash equilibrium in the example is (R, BC ).

Jeﬀrey Ely Extensive-Form Games with Perfect Information Backward Induction

Consider a perfect information game that 1 is ﬁnite, 2 has a single player moving at each node (i.e. N(h) is a singleton), 3 and has no indiﬀerences

′ ′ z ̸= z =⇒ ui (z) ̸= ui (z ))

Such a game has a unique SPE which can be found by backward induction. (Note that we do not need mixed strategies for existence.)

Jeﬀrey Ely Extensive-Form Games with Perfect Information The Dukes of Earl

Example The five Dukes of Earl are scheduled to arrive at the royal palace on each of the first five days of May. Duke One is scheduled to arrive on the first day of May, Duke Two on the second, etc. Each Duke, upon arrival, can either kill the king or support the king. If he kills the king, he becomes the new king and awaits the next Duke’s arrival. If he supports the king all subsequent Duke’s cancel their visits. A Duke’s first priority is to remain alive, and his second priority is to become king. Who is king on May 6?

Draw Extensive Form Describe the strategies of each player Identify the subgames Apply Backward Induction.

Jeﬀrey Ely Extensive-Form Games with Perfect Information The Ultimatum Game

There is a dollar to be divided between two players. Player 1 moves first and offers a split of the dollar giving x to Player 2 and leaving 1 − x for himself. Player 2 then either accepts or rejects the offer. An accepted offer is implemented, but if the offer is rejected both players get zero. In the unique SPE of this game, Player 1 gets all of the bargaining surplus:

He oﬀers x = 0 to Player 2 which is Player 2’s outside option. He thus keeps the diﬀerence between the total surplus from agreement and the total surplus from disagreement. Issue with continuous action space.

Jeﬀrey Ely Extensive-Form Games with Perfect Information Inﬁnite Games?

(Inﬁnite in terms of the tree length.) Obviously there is no backward induction procedure. We can think of backward induction in a diﬀerent way. The backward induction procedure constructs a strategy for each player that is unimprovable by a one-stage deviation.

Jeﬀrey Ely Extensive-Form Games with Perfect Information One-Stage Deviations

Deﬁnition σ σ′ Let i and i be two distinct strategies for player i and let h be a node at σ σ′ σ which i moves. Let h( i , i ) denote the strategy that coincides with i at σ′ all nodes except for h where it plays according to i . { σ′(h˜) if h˜ = h σ σ′ i h( i , i )(h˜) = σi (h˜) otherwise

Jeﬀrey Ely Extensive-Form Games with Perfect Information Unimprovable Strategies

Fix a strategy proﬁle σ−i . Strategy σi is unimprovable by a one-stage deviation if for every node h at which i moves, and every alternative σ′ strategy i , σ σ′ σ | ≤ σ| ui ((h( i , i ), −i ) h) ui ( h)

Jeﬀrey Ely Extensive-Form Games with Perfect Information The One-Stage Deviation Principle

Proposition In any ﬁnite perfect-information game, if a strategy proﬁle is unimprovable for all players then it is a SPE.

Jeﬀrey Ely Extensive-Form Games with Perfect Information The Marathon Game

1 player game. The player’s name is Zeno. H \ Z = {1, 2 ...} A(h) = {quit, continue} for all h. ▶ Quitting at h gives payoﬀ −(1 − 1/h). ▶ Continuing at h leads to h + 1. Continuing forever leads to the terminal node ∞ which gives payoﬀ 1. h = 1 is the initial node.

Jeﬀrey Ely Extensive-Form Games with Perfect Information SPE in the Marathon Game

There is a unique SPE in which Zeno completes the Marathon.

Jeﬀrey Ely Extensive-Form Games with Perfect Information Unimprovable Strategies

The strategy which quits at every node is unimprovable.

Jeﬀrey Ely Extensive-Form Games with Perfect Information Continuity at Inﬁnity

Definition A perfect information game is continuous at infinity if for every strategy σ σ σ′ profile −i , player i, history h, pair of strategies i , i , and real number ε > 0 there is an integer t such that the strategy σ˜ defined by { σi (h˜) if h˜ is no more than t moves past h σ˜ (h˜) = i σ′ ˜ i (h) otherwise

earns a continuation payoﬀ within ε of σi in the subgame beginning at h, i.e. ui ((σ˜i , σ−i )|h) > ui (σi , σ−i |h) − ε

Jeﬀrey Ely Extensive-Form Games with Perfect Information Conditions for Continuity at Inﬁnity

A finite game is continuous at infinity A game with discounting is continuous at infinity

Jeﬀrey Ely Extensive-Form Games with Perfect Information The One-Stage Deviation Principle

Proposition In any perfect information game that is continuous at inﬁnity, a strategy proﬁle that is unimprovable for all players is a subgame perfect equilibrium.

(Of course the converse holds as well.)

Jeffrey Ely Extensive-Form Games with Perfect Information Proof of OSDP We will show that any strategy profile which is not an SPE is improvable by a one-stage deviation for some player. Suppose σ is a strategy profile such that at some subgame h, there is a player i whose continuation strategy is not a best reply, i.e. σ′ σ | σ| ui (( i , −i ) h) > ui ( h) σ′ for some i . For any integer t > 0 define the following strategy { σ′(h˜) if h˜ is no more than t moves past h σt i ˜i (h˜) = σi (h˜) otherwise. By continuity at infinity, for any ε there is a t such that σt σ | σ′ σ | − ε ui ((˜i , −i ) h) > ui (( i , −i ) h) . We take ε small enough so that σt σ | σ| ui ((˜i , −i ) h) > ui ( h)

Jeﬀrey Ely Extensive-Form Games with Perfect Information Proof of OSDP

Is there any node h˜ which is exactly t moves past h such that

σ σ′ σ | σ| ui ((h˜( i , i ), −i ) h˜) > ui ( h˜)

holds? If yes, then we have found an improvement by a one-stage deviation. If not, then we have

σt−1 σ | ≥ σt σ | σ| ui ((˜i , −i ) h) ui ((˜i , −i ) h) > ui ( h) (to understand the first inequality, note that the strategy profile σt−1 σ ˜ (˜i , −i ) either reaches a node h which is t moves past h in which case the inequality holds by the previous answer, or it terminates before ever reaching such a node in which case the payoffs are equal.)

Jeﬀrey Ely Extensive-Form Games with Perfect Information Proof of OSDP

Continuing backward in this way, we either ﬁnd a node h˜ which is within t moves of h such that

σ σ′ σ | σ| ui ((h˜( i , i ), −i ) h˜) > ui ( h˜)

i.e. we ﬁnd an improvement by a one-stage deviation, or if we never do, we obtain

σ0 σ | ≥ σ1 σ | ≥ σ| ui ((˜i , −i ) h) ui ((˜i , −i ) h) ... > ui ( h) σ0 σ but ˜i is a one-stage deviation from i and so in this case we are done as well.

Jeﬀrey Ely Extensive-Form Games with Perfect Information The Rubinstein Bargaining Game

Two players take turns proposing a division of a unit surplus.. Player 1 is the first to make a proposal. Player 2 accepts or rejects. When an offer is accepted the game ends. When an offer is rejected, the game continues and rejecting player makes a proposal. If the tth proposal gives x to player 1 and 1 − x to player 2 (0 ≤ x ≤ 1) and it is accepted, the payoffs are δt−1x for player 1 and δt−1(1 − x) for player 2, where δ ∈ (0, 1) is the discount factor.

Jeﬀrey Ely Extensive-Form Games with Perfect Information Continuity at Inﬁnity

The discounting in the payoﬀs ensures that the game is continuous at inﬁnity so that we can apply the OSDP

Jeﬀrey Ely Extensive-Form Games with Perfect Information SPE of the Rubinstein game

The following constitutes a SPE of the Rubinstein bargaining game δ The proposer offers to give R = 1+δ to the opponent and keep (1 − R) for himself. The responder accepts all offers that give at least R and rejects all offers that give less. (Write this out as a properly specified extensive-form game strategy.) Note that R = δ(1 − R). Thus, in parallel with the ultimatum game, the offering player is extracting the full surplus associated with agreement today rather than agreement tomorrow.

Jeffrey Ely Extensive-Form Games with Perfect Information We show that this strategy profile is unimprovable. Suppose that a responder is offered y < R in some period t. The strategy profile dictates that he should reject. If he does so, then in the next period there will be agreement and he will get δ(1 − R). He would not deviate and accept y because

y < R = δ(1 − R)

Suppose that a responder is offered y which is at least R. He would not deviate and reject y due to the same equality. If the proposer follows the strategy, he gets a payoff of (1 − R). Consider a deviation by the proposer. ▶ An offer more than R will be accepted and this is worse than offering R which will also be accepted. ▶ If he offers less than R, it will be rejected and in the next period an agreement will be reach giving him δR. Since R = δ(1 − R) we have δR = δ2(1 − R) < (1 − R)so this is not an improvement either.

Jeﬀrey Ely Extensive-Form Games with Perfect Information