Subgame Perfect Equilibrium

Ichiro Obara

UCLA

February 20, 2012

Obara (UCLA) SPE February 20, 2012 1 / 29 Backward Induction

Backward Induction

Chain Store Game:

5, 1

Out C 2, 2 A

IN CS F 0, 0

Obara (UCLA) SPE February 20, 2012 2 / 29 Backward Induction

Backward Induction

There are two NE: (Out, F ), (In, A) One may argue that (Out, F ) is not a reasonable NE in the chain store game:

I If C chose “In”, then A would be the best response for CS.

I If C expects this, then C would choose “In”.

Obara (UCLA) SPE February 20, 2012 3 / 29 Backward Induction

Backward Induction

This logic can be generalized to general ﬁnite horizon extensive games with perfect information. Backward induction is the following procedure.

I Let L < ∞ be the maximum length of all histories.

I Find all nonterminal histories of L − 1 length and assign an optimal action there. Eliminate unreached L-length terminal histories and regard other L-length terminal histories as L − 1-length terminal histories.

I Find all nonterminal histories of L − 2 length and assign an optimal action there. Eliminate unreached L − 1-length terminal histories and regard other L − 1-length terminal histories as L − 2-length terminal histories. Obara (UCLA) SPE February 20, 2012 4 / 29

I ... Backward Induction

Backward induction selects (In, A) in the chain-store game. Let’s apply BI to Stackelberg game with linear inverse demand function P(Q) = 12 − Q and linear cost 3qi .

I Given ﬁrm 1’s production q1, ﬁrm 2 solves

∗ 9−q1 maxq2∈<+ (12 − q1 − q2)q2 − 3q2. So q2 (q1) = 2 .

I After eliminating ﬁrm 2’s non-optimal response, ﬁrm 1’s problem is

9−q1 ∗ maxq1∈<+ (12 − q1 − 2 )q1 − 3q1. So q1 = 4.5. ∗ ∗ I Hence we derive the strategy proﬁle (q1 , q2 (·)) and the outcome proﬁle (4.5, 2.25) from BI.

Note that the level of ﬁrm 1’s production is diﬀerent from its NE production level (= 3).

Obara (UCLA) SPE February 20, 2012 5 / 29 Subgame Perfect Equilibrium Subgame Perfect Equilibrium

Subgame Perfect Equilibrium

We introduce a new notion of equilibrium for extensive games with perfect information to formalize and generalize the intuition behind BI.

Obara (UCLA) SPE February 20, 2012 6 / 29 Subgame Perfect Equilibrium Subgame Perfect Equilibrium

At any history, the “remaining game” can be regarded as an extensive game on its own. It is called a subgame after the history. Subgame The subgame of the extensive game with perfect information

(N, H, P, (Vi )) that follows h ∈ H/Z is the extensive game

(N, H|h, P|h, (Vi |h)) that satisﬁes the following conditions (with (h, ∅) = h).

0 0 I h ∈ H|h ⇔ (h, h ) ∈ H.

0 0 0 I P|h(h ) = P(h, h ) for any h ∈ H|h.

0 0 0 I Vi |h(h ) = Vi (h, h ) for any terminal history h ∈ Z|h ⊂ H|h.

One example of subgame is the original game itself.

Obara (UCLA) SPE February 20, 2012 7 / 29 Subgame Perfect Equilibrium Subgame Perfect Equilibrium

Note that a strategy speciﬁes a strategy for any subgame.

For any strategy si ∈ Si , denote the continuation strategy after h ∈ H/Z by si |h, which is a strategy for (N, H|h, P|h, Vi |h) that satisﬁes:

0 0 0 0 I si |h(h ) := si (h, h ) for any h such that (h, h ) ∈ H/Z.

Let Si |h be the set of all strategies for player i for (N, H|h, P|h, Vi |h).

Obara (UCLA) SPE February 20, 2012 8 / 29 Subgame Perfect Equilibrium Subgame Perfect Equilibrium

A strategy proﬁle is a subgame perfect equilibrium if after any nonterminal history it constitutes a Nash equilibrium. Subgame Perfect Equilibrium

For a ﬁnite extensive game with perfect information (N, H, P, (Vi )), a strategy proﬁle s∗ ∈ S is a subgame perfect equilibrium if

∗ ∗ Vi |h(O(s |h)) ≥ Vi |h(O(si , s−i |h)) for any si ∈ Si |h, any i ∈ N and any h ∈ H/Z.

Note that every SPE is a NE.

Obara (UCLA) SPE February 20, 2012 9 / 29 Subgame Perfect Equilibrium One-Shot Deviation Principle

One-Shot Deviation Principle

To verify that some strategy proﬁle is a SPE, we need to check a very large number of incentive constraints.

It turns out that we can focus on a subset of incentive constraints for ﬁnite horizon extensive games with perfect information.

0 A strategy si ∈ Si |h at h ∈ H/Z for i = P(h) is called one-shot 0 deviation from si if si |h and si prescribes a diﬀerent action only at 0 0 0 0 the initial history (i.e. si (∅) 6= si (h) and si (h ) = si (h, h ) for any h0 6= ∅ such that (h, h0) ∈ H/Z).

Obara (UCLA) SPE February 20, 2012 10 / 29 Subgame Perfect Equilibrium One-Shot Deviation Principle

For ﬁnite extensive games with perfect information, we just need to check the incentive constraints with respect to one-shot deviations. One-Shot Deviation Principle

For a ﬁnite extensive game with perfect information (N, H, P, (Vi )), a strategy proﬁle s∗ ∈ S is a subgame perfect equilibrium if and only if

∗ ∗ Vi |h(O(s |h)) ≥ Vi |h(O(si , s−i |h))

∗ for any one shot deviation si ∈ Si |h from si |h for i = P(h) at any h ∈ H/Z.

The proof is omitted as we prove a more general version of this result.

Obara (UCLA) SPE February 20, 2012 11 / 29 Subgame Perfect Equilibrium One-Shot Deviation Principle

Comments: For any ﬁnite horizon extensive game with perfect information (ex. Chess),

I the set of subgame perfect equilibria is exactly the set of strategy proﬁles that can be found by BI.

I there always exists a subgame perfect equilibrium.

I there exists the unique subgame perfect equilibrium for generic games 0 0 (i.e. Vi (z) 6= Vi (z ) for any z 6= z for any i ∈ N).

Obara (UCLA) SPE February 20, 2012 12 / 29 Subgame Perfect Equilibrium One-Shot Deviation Principle

Continuity at Inﬁnity

In general, one-shot deviation principle does not hold for inﬁnite horizon games.

We say that a game satisfies continuity at infinity if payoffs in very far future are not important. Continuity at Infinity

(N, H, P, (Vi )) satisﬁes continuity at inﬁnity if for any > 0 there exists T such that 0 Vi (z) − Vi (z ) <

for any z, z0 ∈ Z with the same initial T -length histories and for any i ∈ N.

Obara (UCLA) SPE February 20, 2012 13 / 29 Subgame Perfect Equilibrium One-Shot Deviation Principle

Comments:

Every finite horizon extensive game satisfies continuity at infinity.

Consider infinite horizon games where player i’s payoff takes the following form: ∞ X t t Vi (z) = δ gi (a ) t=0 1 2 for z = (a , a , ...., ). Suppose that gi is bounded. Then this infinite horizon extensive game satisfies continuity at infinity.

Obara (UCLA) SPE February 20, 2012 14 / 29 Subgame Perfect Equilibrium One-Shot Deviation Principle

One-Shot Deviation Principle

Suppose that (N, H, P, (Vi )) satisfies continuity at infinity. Then a strategy profile s∗ ∈ S is a subgame perfect equilibrium if and only if

∗ ∗ Vi |h(O(s |h)) ≥ Vi |h(O(si , s−i |h))

∗ for any one shot deviation si ∈ Si |h from si |h for i = P(h) at any h ∈ H/Z.

Obara (UCLA) SPE February 20, 2012 15 / 29 Subgame Perfect Equilibrium One-Shot Deviation Principle

Proof. The “only if” part follows from the deﬁnition. For the “if” part:

I If there is no one-shot profitable deviation, then there is no finite-period profitable deviation.

I If there is no finite-period profitable deviation, then there cannot be any infinite-period profitable deviation because of continuity at infinity.

Obara (UCLA) SPE February 20, 2012 16 / 29 Subgame Perfect Equilibrium Examples

Example: Chain-Store game

Suppose that the chain store plays the chain-store game sequentially with K potential competitors in K diﬀerent cities.

In market k, competitor Ck chooses either “In” or “Out” given the histories in the previous k − 1 markets. Then CS chooses whether to accommodate or fight. The payoff of CS is the sum of its payoffs in all K markets.

The unique subgame perfect equilibrium is that every competitor always enters and the chain store always accommodates.

There are many other Nash equilibria.

Obara (UCLA) SPE February 20, 2012 17 / 29 Subgame Perfect Equilibrium Examples

Example: Centipede game

Consider the following game with two players. What is the subgame perfect equilibrium?

1 2 1 2 1 8,6 U U U U U

D D D D D

2,0 1,2 4,1 3,4 6,3

Obara (UCLA) SPE February 20, 2012 18 / 29 Subgame Perfect Equilibrium Interpretations of Strategies

Interpretation of Strategy

In the following game, the unique SPE is that player 1’s strategy is Ba and player 2’s strategy is C.

A B 2

2,0 C D 1

0,0 a b

0,1 -1,-2

Why does player 1 need to choose between a and b when playing B?

Obara (UCLA) SPE February 20, 2012 19 / 29 Subgame Perfect Equilibrium Interpretations of Strategies

Player 1 cannot determine player 2’s behavior without specifying his behavior after (A, C).

Thus one interpretation of a strategy oﬀ the equilibrium path is that they are just beliefs: player 1 believes that player 2 believes that player 1 plays a after (A, C) in this case.

Another interpretation of a strategy oﬀ the equilibrium path is that it is an actual plan in noisy environments: (A, C) sometimes happen because people always make mistakes or some unexpected things always happen.

Obara (UCLA) SPE February 20, 2012 20 / 29 Subgame Perfect Equilibrium Extensions

Extensive Games with Exogenous Uncertainty

It is easy to incorporate random public signal into an extensive game.

Extensive game with perfect information and chance moves

(N, H, P, fc , (Vi )) is an extensive game with perfect info + exogenous randomization. S I P maps nonterminal histories to N {c}. Let Hc be the set of histories such that P(h) = c.

I For any h ∈ Hc , fc (h) ∈ ∆(A(h)) assigns an action a ∈ A(h) with

probability fc (a|h).

Obara (UCLA) SPE February 20, 2012 21 / 29 Subgame Perfect Equilibrium Extensions

Extensive Games with Simultaneous Moves

Another extension is to allow for simultaneous moves.

Extensive game with perfect information and simultaneous moves

(N, H, P, (Vi )) is an extensive game with perfect info where players may move simultaneously.

I History H is a sequence of action proﬁles.

I P maps nonterminal histories to a subset of N. Q I A(h) is a product set: A(h) = i∈P(h) Ai (h).

Obara (UCLA) SPE February 20, 2012 22 / 29 Subgame Perfect Equilibrium Further Examples

Example: Bank Runs

Two investors with deposit D > 0 each at some bank, invested in some project.

There are two periods. Each investor decides whether to withdraw her deposit in each period. If any investor withdraws her deposit in the ﬁrst period, then the project is terminated and the game ends.

The payoﬀs of the investors are as follows (assume 0.5D < r < D < R):

I each investor receives r if both investors withdraw in the 1st period.

I if one investor withdraws and the other decides not to withdraw in the 1st period, then this investor secures D and the other investor receives the rest (= 2r − D).

I if both investors do not withdraw in the ﬁrst period, then the game moves to the 2nd period and the project matures to generate 2R.

I each investor receives R if both investors withdraw or do not withdraw in the 2nd period.

I if one investor withdraws and the other decides not to in the 2nd period, then this investor receives all the proﬁts except for the other investor’s original investment D, Obara (UCLA) SPE February 20, 2012 23 / 29 which is left for the other investor. Subgame Perfect Equilibrium Further Examples

This game can be represented as follows.

WD N

WD N WD N

1 r, r D, 2r-D 2r-D, D WD N

WD N WD N

R, R 2R-D, D D, 2R-D R, R

Find all SPE.

Obara (UCLA) SPE February 20, 2012 24 / 29 Subgame Perfect Equilibrium Further Examples

Location Model with Quadratic Transportation Cost

Two restaurants A, B are opening their restaurant on [0, 1] on which customers are uniformly distributed. A customer i ∈ [0, 1] pays transportation cost t (x − i)2 to go to any restaurant located at x ∈ [0, 1] and enjoys utility V by consuming one meal . The game proceeds as follows.

I In the ﬁrst period, restaurants choose locations xA, xB simultaneously

I In the second period, restaurants choose prices pA, pB simultaneously. Customer i buys one meal from restaurant A if and only if 2 n 2 o V − pA − t (xA − i) ≥ max V − pB − t (xB − i) , 0 . The same for restaurant B. SupposeObara (UCLA) that restaurants maximizeSPE their expected revenues.February 20, 2012 25 / 29 Subgame Perfect Equilibrium Further Examples

Under a regulation that the price must be p, this is essentially one-period location game. The equilibrium would be

(xA, xB ) = (0.5, 0.5) if V is large enough.

What is the subgame perfect equilibrium of this two period game?

Obara (UCLA) SPE February 20, 2012 26 / 29 Subgame Perfect Equilibrium Further Examples

War of Attrition

Two players are ﬁghting with each other.

In period t = 0, 1, 2, ..., the players decide whether to ﬁght or concede simultaneously. The game ends as soon as any player concedes.

The cost of fighting is c > 0. When player i fights and player −i concedes in period t, player i receives a reward V > 0 in period t without incurring cost c. Player i’s payoff is PT −1 t I − t=0 δ c if i concedes in period T . PT −1 t T I − t=0 δ c + δ V if i fights and −i concedes in period T . Is there any stationary subgame perfect equilibrium?

Obara (UCLA) SPE February 20, 2012 27 / 29 SPE and IEWDS

SPE and IEWDS

Write down an extensive game as a strategic game.

Notice that BI for extensive games with perfect information corresponds to a particular way of eliminating weakly dominated strategies.

Obara (UCLA) SPE February 20, 2012 28 / 29 SPE and IEWDS

Burning Money

Here is an interesting example.

I Players play the following game.

B C

B 3,1 0,0

C 0,0 1,3

I Player 1 can burn $1 publicly before playing this game.

Write down an extensive game. Find all SPE.

Write down a reduced form strategic game. Apply IEWDS. What do you Obara (UCLA) SPE February 20, 2012 29 / 29 ﬁnd?