Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection

Week 7: Games in Extensive Form

Dr Daniel Sgroi Reading: 1. Osborne chapter 5 (and also 6, 7); 2. Snyder & Nicholson, pp. 255–260. With thanks to Peter J. Hammond.

EC202, University of Warwick, Term 2 1 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Sequential BoS, I

1 BS XX 2 XXX2 BS @ b BS @ r @@ r @@ 2, 1 0, 0 0, 0 1, 2 r r r r Consider the BoS game, with the key difference that player 1 who prefers (B, B) to (S, S) moves first, and player 2 observes player 1’s move before replying. This difference affects the normal or strategic form, because player 2 now has four instead of two strategies, as follows: 1. B always (which we denote by b); 2. S always (which we denote by s); 3. B if B and S if S (which we denote by m for match); 4. S if B and B if S (which we denote by x for cross).

EC202, University of Warwick, Term 2 2 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Sequential BoS, II

Here is the normal form of sequential BoS, with (multiple) best responses starred as usual.

P2 b s m x B 2∗ 0 2∗ 0∗ 1∗ 0 1∗ 0 ∗ ∗ P1 S 0 1 1 0 0 2∗ 2∗ 0

There are three pure strategy Nash equilibria, and many mixed strategy equilibria (for example, B with any mixture of b and m).

EC202, University of Warwick, Term 2 3 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Sequential BoS, III

P2 b s m x B 2∗ 0 2∗ 0∗ 1∗ 0 1∗ 0 ∗ ∗ P1 S 0 1 1 0 0 2∗ 2∗ 0

Player P2’s strategy x is strictly dominated by m, while both b and s are weakly dominated by m. If we eliminate all weakly dominated strategies once, then eliminate 1’s strictly dominated strategy, only (B, m) remains. This is a refinement of the set of Nash equilibria.

EC202, University of Warwick, Term 2 4 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Sequential BoS, IV

An alternative way to justify P2 choosing strategy m considers the two different subgames that arise after player 1 has chosen either B or S. In each of these player 2 can also choose B or S: after B, player 2’s best response is B; after S, player 2’s best response is S. Thus player 2 matches by choosing m in the full game. Player 1’s best response is B. In effect, player 1’s move anticipates player 2’s response. The result is player 1’s favoured outcome, so moving first confers an advantage in this game. Player 1 has a “first-mover advantage”.

EC202, University of Warwick, Term 2 5 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Sequential Cornot, I Consider the Cournot duopoly model where firms have zero cost, and inverse demand is p = max{0, 100 − q}, with q = q1 + q2.

Firm i’s best response, taking the other firm’s qj as fixed, 1 solves maxqi (100 − qi − qj )qi , so qi = 50 − 2 qj . After allowing for the possibility of a corner solution, we derive 1 the best response (BR) function bi (qj ) = max{0, 50 − 2 qj }.

As in sequential BoS, suppose that before firm 2 chooses q2, it will have observed firm 1’s choice of q1. Firm 2’s strategy space becomes the set of all response functions q1 7→ r2(q1) from R+ to R+. A strategy for firm 2 that weakly dominates all others is to choose r2(·) to be the best response function b2(·). When making its move, firm 2 not only has the deterministic belief, but the certain knowledge that q1 has been played.

EC202, University of Warwick, Term 2 6 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Sequential Cornot, II

Assuming common knowledge of rationality, what should firm 1 do?

A deterministic belief about q2 would be na¨ıve, since firm 1 knows that a rational firm 2 1 will choose b2(q1) = max{0, 50 − 2 q1} in response to its choice of q1. This, in turn, means that a rational firm 1 would replace the “fixed” q2 in its profit function with firm 2’s best response b2(q1) to 1’s choice of q1.

That is, firm 1 now chooses q1 to solve

max[100 − q1 − b2(q1)] q1. q1

EC202, University of Warwick, Term 2 7 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Stackelberg, I

Firm 1 chooses q1 to solve

max[100 − q1 − b2(q1)] q1, q1

1 where b2(q1) = max{0, 50 − 2 q1}. In the interior case, the maximand 1 1 is [100 − q1 − (50 − 2 q1)] q1 = (50 − 2 q1) q1, with derivative 50 − q1, which is ≷ 0 as q1 ≶ 50.

So, anticipating firm 2’s response b2(q1) to q1, firm 1’s optimal strategy is q1 = 50.

After this firm 2 responds by choosing q2 = b2(50) = 25.

The price is 100 − q1 − q2 = 100 − 75 = 25. This is the Stackelberg equilibrium.

EC202, University of Warwick, Term 2 8 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Stackelberg, II

Given q1 = 50, q2 = 25, and p = 25, the resulting profits (revenues) are π1 = 25 · 50 = 1, 250 and π2 = 25 · 25 = 625. Recall that in the original Cournot equilibrium (with simultaneous moves) 1 the quantities were q1 = q2 = 100/3 = 33 3 , 1 the price was p = 100/3 = 33 3 , 1 and the profits were π1 = π2 = 10, 000/9 = 1, 111 9 . So firm 1 has a first-mover advantage, relative to both firm 2 and the Cournot equilibrium. Total profits, however, are lower.

EC202, University of Warwick, Term 2 9 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Extensive Form Games

One obvious drawback (sometimes an advantage) of the normal form of a game is its failure to capture time. A player’s strategy describes one complete plan for the whole game. Payoff functions represent how players value the consequences of strategy profiles. But how is the order of moves captured? A game in extensive form captures sequential strategic situations when there is a well defined order of moves. Our analyses of sequential BoS and sequential Cournot suggest that the order of moves can affect what we would regard as a reasonable outcome of the game. It illustrates a solution concept that captures an important idea of sequential rationality.

EC202, University of Warwick, Term 2 10 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Players and Payoffs

The extensive form formally captures situations where: 1. players may move sequentially; 2. what players know, when it is their turn to move, may depend on players’ previous choices.

Like the normal or strategic form, any extensive form game has two key ingredients in its description: (EF1) The set of players, N

(EF2) The players’ payoffs (ui (·))i∈N as functions of everybody’s actions.

EC202, University of Warwick, Term 2 11 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Move Order and Feasible Sets To allow sequential play to be represented, we extend the notion of pure strategy by specifying two separate parts: first, as before, what players can do; second, when (and with what information) they can do it. In the Stackelberg example, as in the Cournot game, players can choose any quantity they like. But we also needed to specify that player 1 moves first, and only then does player 2 move (after observing 1’s prior choice). Thus, in general we need two extra components in order to model allowable move sequences: (EF3) The order of moves. (EF4) What choices players have when they do move (their feasible sets).

EC202, University of Warwick, Term 2 12 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Players’ Knowledge

Simultaneous moves in the normal form represent the situation where players know nothing about other players’ prior moves. In the extensive form, it is not the chronological order of play that matters, but what players know when they choose. In both the Cournot and Stackelberg models of duopoly, firm 1 may indeed move some time before firm 2. But if firm 2 must choose without observing firm 1’s choice, then the Cournot model of simultaneous choice still applies. In contrast, if firm 1’s choice is revealed before firm 2 moves, then we should use the sequential Stackelberg model. In general, we need to specify how information and knowledge change over time: (EF5) Players’ information when making their moves.

EC202, University of Warwick, Term 2 13 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Chance Moves

Exogenous random events vastly enrich the strategic decision problems we can describe. Example Suppose firm A embarks on a research and development (R&D) project that may succeed or fail. Competing firm B can adapt its strategy to the outcome of firm A’s R&D project, by waiting to see if it succeeds or not. We may think of the outcome of the R&D project as random — either success or failure with fixed probabilities p and 1 − p respectively.

EC202, University of Warwick, Term 2 14 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Time Line

The time line of this scenario is as follows: 1. firm A chooses its R&D project; 2. the project succeeds with probability p and fails with probability 1 − p; 3. after observing the outcome of firm A’s project, firm B will make its own choice.

EC202, University of Warwick, Term 2 15 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Probabilities of Chance Moves

We will generally regard uncertainty outcomes as determined by “chance moves” or moves of nature. We think of nature (or chance) as a capricious player with a fixed random strategy, but no strategic objective. The probabilities attached to different chance moves are treated as exogenous — i.e., as not chosen by any player, but fixed, like the strategy sets and payoff functions. We represent chance/nature as: (EF6) Probabilities over exogenous events. Finally, to use the tools and concepts we have already learned, we add the final familiar requirement: (EF7) The structure of the extensive form game, represented by (EF1)–(EF6) above, is common knowledge.

EC202, University of Warwick, Term 2 16 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Game Tree Representation

Components (EF1)–(EF7) capture all we need to represent a sequential game. But a formal notation will be required. For this, we introduce the idea of a game tree as a figurative way of representing the unfolding of an extensive form game.

EC202, University of Warwick, Term 2 17 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection A Biped or Trust Game

1 2 - - 1, 1 E C r r r N ? D ?

0, 0 −1, 2 r r Example Player 1 first chooses whether to enter (E) the game, or not to enter (N). The latter gives both players a payoff of zero. If player 1 plays E, player 2 can choose cooperate (C), giving both players a payoff of 1. Or player 2 can defect (D) and get a payoff of 2, while leaving player 1 with a payoff of −1.

EC202, University of Warwick, Term 2 18 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Trees

Definition A tree T = hX ,

2. There is a unique initial node x0 that precedes every other node, but has none preceding it.

An interpretation is that, apart from the initial node x0, every node x in the tree can be reached as a result of specific actions chosen at all the nodes x0 < x that precede it.

EC202, University of Warwick, Term 2 19 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Terminal Nodes and Game Trees

Let Z ⊆ X denote the set of terminal nodes, defined as those that precede no others. Each branch of a tree will have a unique terminal node unless that branch has infinitely many nodes. Definition A game tree is a tree T supplemented by a set of players i ∈ N and their payoff functions ui : Z → R defined on the terminal nodes of T . The order of play is given by a function i : X \ Z → N that identifies a player i(x) at each non-terminal node x ∈ X \ Z. Let A(x) denote the feasible set of actions that are possible for player i(x) at node x.

EC202, University of Warwick, Term 2 20 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Sequential BoS . . .

1 OF XX 2 XXX2 o @ f b o @ f r @@ r @@ 2, 1 0, 0 0, 0 1, 2 r r r r Consider again sequential BoS, with O and o representing opera, and with F and f representing football. One equilibrium strategy profile is (F , ff ), resulting in payoffs of (1, 2).

EC202, University of Warwick, Term 2 21 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection . . . Has Some Incredible Nash Equilibria

The equilibrium (F , ff ) is supported by player 1’s belief that deviating to play O would lead to payoff 0 because player 2 will play f at x1. This “threat” to choose f off the equilibrium path supports player 1’s action on the equilibrium path. But this “threat” is incredible because for player 2 to carry it out would be irrational given the payoffs of the game. So does the Nash equilibrium (F , ff ) make a reasonable prediction of player 1’s rational choice? It does meet the definition of Nash equilibrium: strategy ff is a best response to F , and vice versa. But ff commits player 2 to f after 1’s choice of O, even though o would be better.

EC202, University of Warwick, Term 2 22 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Refined Equilibria in the Sequential BoS, I

The normal form representation treats all strategic choices as “once-and-for-all” commitments. Yet we expect rational players to play best responses to their beliefs wherever they are called on to move. In the normal form it is hard to analyse how credibility limits what can be accepted as “rational behavior”. But this can be done in the extensive form.

EC202, University of Warwick, Term 2 23 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Refined Equilibria in the Sequential BoS, II

Two of the three Nash equilibria in this game — namely (O, oo) and (F , ff ) — commit player 2 to a strategy that, though a best response to 1’s equilibrium strategy, would not be best were player 1 to deviate from the equilibrium path. (“Counterfactual” analysis.) The new structure we are about to set up will result in more refined predictions for dynamic games. It will rule out equilibria that involve such incredible threats. In particular, it will admit only (O, of ) as the unique “refined” Nash equilibrium.

EC202, University of Warwick, Term 2 24 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Sequential Rationality

Equilibria (O, oo) and (F , ff ) in sequential BoS require an implausible reaction by player 2 when 1 departs from the presumed equilibrium. We now insist that every player use strategies that are optimal at every information set in the game tree, even those not reached along the equilibrium path. Definition Given any player i and any strategy profile s−i ∈ S−i , the strategy si is sequentially rational if and only if what it prescribes at each information set h ∈ Hi is a best response to s−i . In sequential BoS, sequential rationality requires: if player 1 plays O, then player 2 should play o; if player 1 plays F , then player 2 should play f .

EC202, University of Warwick, Term 2 25 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Sequential Rationality in Dynamic BoS

Now move back to initial node x0, where player 1 has to choose between O and F . Player 2’s sequential rationality should lead player 1 to expect that playing O will result in payoffs (2, 1), whereas playing F will result in payoffs (1, 2). So sequential rationality implies player 1 chooses O. Sequential rationality makes the unique prediction that the path of play is O followed by o. Furthermore, the process predicts what would happen if one or more players deviate from the path of play: if player 1 chooses F instead of O, then 2 will choose f . Conclusion The particular Nash equilibrium (O, of ) is the only strategy pair (and the only one of the three Nash equilibria) that meets the sequential rationality requirement.

EC202, University of Warwick, Term 2 26 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Backward Induction

The procedure used in dynamic BoS to find sequentially rational Nash equilibria (or SRNE) (in pure strategies) starts at terminal nodes. It is a backward induction procedure that steps back node by node, all the way to the initial node, where it ends. At each node, the player to move responds best to the other players’ anticipated actions at all later nodes.

EC202, University of Warwick, Term 2 27 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection The Centipede (or Quadruped) Game

1 2 1 2 - - - - 3, 3 C c1 C c2 r 1 rr 2 r r N1 ? n1 ? N2 ? n2 ?

1, 1 0, 3 2, 2 1, 4 r r r r

The game goes from left to right as follows:

First, player 1 can stop the game at once by choosing N1, or continue by choosing C1.

Second, player 2 faces a choice between n1 and c1. If player 2 chooses c1, the next move is player 1’s, who chooses between N and C. If C2 is chosen, player 2 concludes the game by choosing n2 or c2 a second time.

EC202, University of Warwick, Term 2 28 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection What the Players Would Like

1 2 1 2 - - - - 3, 3 C c1 C c2 r 1 rr 2 r r N1 ? n1 ? N2 ? n2 ?

1, 1 0, 3 2, 2 1, 4 r r r r

If the players could reach a binding agreement, they would surely like to continue at each move, to reach the payoffs of (3, 3), the highest possible total. However, backward induction indicates that this contradicts sequential rationality.

EC202, University of Warwick, Term 2 29 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Appying Backward Induction . . .

1 2 1 2 - - - - 3, 3 C c1 C c2 r 1 rr 2 r r N1 ? n1 ? N2 ? n2 ?

1, 1 0, 3 2, 2 1, 4 r r r r

At the last information set player 2 will choose n2 rather than c2, to get 4 instead of 3. Anticipating this one step earlier, player 1 will choose N2 to get 2 instead of 1.

Anticipating this, player 2 will choose n1 on his first move. The same argument goes all the way back to the initial node, where player 1 will choose N1. So the backward induction payoffs are (1, 1).

EC202, University of Warwick, Term 2 30 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection The Curse of Rationality

In the quadruped game, every time a player continues, the sum of their payoffs goes up by 1. We can continue increasing the payoffs like this with (2, 5), (4, 4), (3, 6), etc. after (3, 3). The payoffs late in the game become extremely large. Backward induction still unravels any form of cooperation, leading to the unique sequentially rational payoffs (1, 1). This is the “curse of rationality”.

EC202, University of Warwick, Term 2 31 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection Proper Subgames: Definition

Definition Let Γ be an extensive-form game with X as its set of nodes. A (proper) subgame G of Γ starts at some single node xG , and includes all nodes belonging to information sets that include one or more successors of xG in Γ. The subgame G is itself an extensive-form game tree with its information sets and payoffs inherited from Γ. Definition Let Γ be an n-player extensive form game. ∗ ∗ ∗ ∗ A behavioral strategy profile σ = (σ1, σ2, . . . , σn) is a subgame-perfect Nash equilibrium (or SPE) if for every proper subgame G of Γ, the restriction of σ to G is a Nash equilibrium in G.

EC202, University of Warwick, Term 2 32 of 33 Battle of the Sexes Stackelberg Essential Ingredients Game Trees Backward Induction Subgame Perfection On and off the Equilibrium Path

Subgame perfection imposes sequential rationality on Nash equilibrium by requiring best responses not only on the equilibrium path (by definition), but off the equilibrium path as well. It does so by requiring the relevant restriction of the equilibrium strategy profile σ∗ to be a Nash equilibrium in every proper subgame, including any subgame not reached on an equilibrium path. By its definition, every SPE is a Nash equilibrium. But not every Nash equilibria is an SPE, implying that SPE refines the set of Nash equilibria, and thus predicts behaviour more precisely.

EC202, University of Warwick, Term 2 33 of 33