SEQUENTIAL GAMES WITH PERFECT INFORMATION Example 4.9 (page 105) Consider the sequential game given in Figure 4.9. We want to apply backward induction to the tree. 0 Vertex B is owned by player two, P2. The payoffs for P2 are 1 and 3, with 3 > 1, so the player picks R . Thus, the payoffs at B become (0, 3). 00 Next, vertex C is also owned by P2 with payoffs 1 and 0. Since 1 > 0, P2 picks L , and the payoffs are (4, 1). Player one, P1, owns A; the choice of L gives a payoff of 0 and R gives a payoff of 4; 4 > 0, so P1 chooses R. The final payoffs are (4, 1). 0 00 We claim that this strategy profile, { R } for P1 and { R ,L } is a Nash equilibrium. Notice that the 0 00 strategy profile gives a choice at each vertex. For the strategy { R ,L } fixed for P2, P1 has a maximal payoff by choosing { R }, ( 0 00 0 00 π1(R, { R ,L }) = 4 π1(R, { R ,L }) = 4 ≥ 0 00 π1(L, { R ,L }) = 0. 0 00 In the same way, for the strategy { R } fixed for P1, P2 has a maximal payoff by choosing { R ,L }, ( 00 0 00 π2(R, {∗,L }) = 1 π2(R, { R ,L }) = 1 ≥ 00 π2(R, {∗,R }) = 0, where ∗ means choose either L0 or R0. Since no change of choice by a player can increase that players own payoff, the strategy profile is called a Nash equilibrium. Notice that the above strategy profile is also a Nash equilibrium on each branch of the game tree, mainly starting at either B or starting at C. Such a strategy profile is called subgame perfect. A solution found by backward induction is alway subgame perfect. There is another Nash equilibrium which is not subgame perfect. Assume P1 picks { R }. P2 should pick L00 and R0. We do not derive this strategy profile, but merely check that it satisfies the conditions necessary to be a Nash equilibrium. 0 00 0 What if P2 instead picks L and L . The choice L is “irrational” in terms of a choice at B. We claim 0 00 0 00 that { R } for P1 and { L ,L } is still a Nash equilibrium. For the strategy { L ,L } fixed for P2, P1 has a maximal payoff by choosing { R }, ( 0 00 0 00 π1(R, { L ,L }) = 4 π1(R, { L ,L }) = 4 ≥ 0 00 π1(L, { L ,L }) = 2. 0 00 In the same way, for the strategy { R } fixed for P1, P2 has a maximal payoff by choosing { L ,L }, ( 00 0 00 π2(R, {∗,L }) = 1 π2(R, { L ,L }) = 1 ≥ 00 π2(R, {∗,R }) = 0. Thus, neither player has an incentive to change their own strategy. This strategy is not subgame perfect 0 0 because starting at vertex B, P2 has an incentive to change from L to R . This did not affect the payoffs above because the path resulting from fixing P1’s strategy cannot reach vertex B. In summary, this example has two Nash equilibria. The first, found by backward induction, is subgame perfect. The second is not subgame perfect. 1 2 SEQUENTIAL GAMES WITH PERFECT INFORMATION Definition (page 96) A tree T is an n-player game tree provided that there are n players P1,... Pn and a. each nonterminal node of the tree is owned by either (i) exactly one player who needs to make a choice of an edge going out of the node, or (ii) “nature” with a probability on the edges going out of the node, b. each terminal node is not owned by any player but has a payoff vector, (p1(v), . , pn(v)), with a payoff for each player. In the above example, the two players know exactly which node they are at when they make a choice, and the choices at each node is made independently. Such a game is said to have perfect information.A sequential game with perfect information, or an extensive form game with perfect information, is a game tree where the players can make at each node independently other the choices at other nodes. We treat the case of a game with perfect information first, and then return to the case of “imperfect information”. Definition (page 99) Let V be the set of all the nodes in a game tree T , and let Nj be the set of all the nodes owned by Pj.A strategy for Pj is a choice of an outgoing edge at each v ∈ Nj, i.e., a function sj : Nj → V such that sj(v) is a child of v and (v, sj(v)) is an edge in the graph going out from v. Thus, a strategy can also be thought of as a function from Nj into the edges E such that (v, sj(v)) is an edge going out from node v. Let Sj be the set of tall the possible strategies for Pj.A strategy profile, (s1, . sn) is a strategy for each player, i.e., a choice of an edge at each node owned by each player. The set of all strategy profiles is the set S1 × · × Sn. Given the set of strategy profiles with the resulting payoffs expresses the game in strategic form. A strategy profile (s1, . sn) determines a path from the root to a terminal node. The payoff for Pj is the payoff at the terminal node determined by this path and is denoted by uj(s1, . sn). Definition (page 104) ∗ ∗ A strategy profile (s1, s2) for two-person sequential game with perfect information is called a Nash equilib- rium provided that ∗ ∗ ∗ u1(s1, s2) = max u1(s1, s2) and s1∈S1 ∗ ∗ ∗ u2(s1, s2) = max u2(s1, s2). s2∈S2 ∗ In this definition, the maximum of u1(s1, s2) corresponds to keeping all the choices of the second player fixed and varying the choices of edges going out of all the vertices owned by P1. Similarly, for the maximum ∗ of u2(s1, s2) by P2. Thus, a Nash equilibrium is not just a path in the game tree, but is a choice at each vertex owned by a player. The definition of a Nash equilibrium for an n-person sequential game is defined in a similar manner. Definition (page 116) A subgame of a game is formed by taking a branch of the game tree, using the same payoffs as the original tree and the nodes are owned by the same players. Definition (page 118) ∗ ∗ A strategy profile (s1, . , sn) of a strategic game is called a subgame perfect (Nash) equilibrium provided that it is a Nash equilibrium for each subgame and also for the whole game. Theorem 4.10 (Kuhn) (page 106) Every sequential game with perfect information has a subgame perfect Nash equilibrium. SEQUENTIAL GAMES WITH PERFECT INFORMATION 3 Idea of the proof: Backward induction constructs a subgame perfect Nash equilibrium. Take a node v1 that goes only to terminal nodes. This node v1 belongs to some player Pj. Take the maximum of uj over the terminal nodes ∗ which are children of v1. This defines sj (v1). Replace the tree with a game with v1 as a terminal node and ∗ the payoff at v1 the original payoff at sj (v1). Continue by induction. Example: Dueling In this duel, there are two players. The players alternate turns. If it is P1 turn, then the player can either fire (F) or wait (W) and take a step toward P2. At this stage the players are some distance d apart. If P1 fires, then there is a probability p1(d) that the player will hit the other player and a probability 1 − p1(d) that the player will miss. If the player hits the other player, then the payoff is 1 for P1 and 0 for P2. If P1 misses, then P2 can wait until they are close enough to hit, so the payoff is 1 for P2 and 0 for P1. Thus, the payoffs for the choice to fire by P1 is p1(d) · 1(1 − p1(d)) · 0 = p1(d) for P1 and p1(d) · 0(1 − p1(d)) · 1 = 1 − p1(d) for P2, or a payoff vector of (p1(d), 1 − p1(d)). In the same manner, if it is the turn for P2, this player can wait and take a step toward P1 or fire. If P2 fires, then there is probability of p2(d) that the player will hit P1, so the payoff P2 is p2(d) and 1 − p2(d) for P2, or a payoff vector of (1 − p2(d), p2(d)). (p1(d2n), 1 − p1(d2n)) (p1(d2n−2), 1 − p1(d2n−2)) (p1(d2), 1 − p1(d2)) F F F W (P2) W W (P2) (P1) (P1) (P1) F F (1 − p2n−1(d2), p2(d2n−1)) (0, 1) This game can be solved by backward induction. Assume that players start at a distance d2n = 2nδ apart. After 2n − 1 steps of size δ, the distance is d1 = δ. Thus, the game tree is as given in the following figure with 2n vertices that correspond to distances d2n = 2nδ,..., d1 = δ. At distance δ, assume that P2 is certain to hit and p2(δ) = 1, so P2 is certain to fire. Assume we know that P2 fires at distance d2j−1 = (2j − 1)δ. At this vertex the payoff vector is (1 − p2(d2j−1), p2(d2j−1))). At the previous vertex at distance d2j = (2j)δ, P1 can either fire or wait.