Outline for today

Stat155 Game Theory Lecture 16: Evolutionary game theory Evolutionarily stable strategies ESS and Nash equilibria ESMS: evolutionarily stable against mixed strategies Peter Bartlett Multiplayer ESS

October 20, 2016

1 / 21 2 / 21 Nash equilibrium Evolutionarily stable strategies

Definition A strategy profile x = (x ,..., x ) ∆ ∆ is a Nash equilibrium There is a population of individuals. 1∗ k∗ ∈ S1 × · · · × Sk for utility functions u ,..., u if, for each player j 1,..., k , There is a game played between pairs of individuals. 1 k ∈ { } Each individual has a pure strategy encoded in its genes. max uj (xj , x∗ j ) = uj (xj∗, x∗ j ). xj ∆S − − ∈ j The two players are randomly chosen individuals. A higher payoff gives higher reproductive success. If the players play these mixed strategies xj∗, nobody has an incentive to unilaterally deviate: each player’s mixed strategy is a best response This can push the population towards stable mixed strategies. to the other players’ mixed strategies.

3 / 21 4 / 21 Evolutionarily stable strategies Hawks and Doves

Consider a two-player game with payoff matrices A, B. Payoff Suppose that it is symmetric (A = B ). > Fix v = 2, c = 2. Consider a strategy x. Think of x as the proportion of each pure strategy in the population. H D H (-1,-1) (2,0) If x is invaded by a small population of mutants z = x, will x survive? 6 D (0,2) (1,1)

x’s utility: x>A (z + (1 )x) = x>Az + (1 )x>Ax − − x = (1/2, 1/2) is a Nash equilibrium. z’s utility: z>A (z + (1 )x) = z>Az + (1 )z>Ax − − Is it an ESS? Definition Consider a mutant pure strategy z. A strategy x ∆ is an evolutionarily stable strategy (ESS) if, for any For z = (1, 0) (that is, H), z Az = 1 < 1/2 = x Az. ∈ n > > pure strategy z = x, − − 6 For z = (0, 1) (that is, D), z>Az = 1 < 3/2 = x>Az. 1 z Ax x Ax (x, x) is a Nash equilibrium. > ≤ > ←− So x is an ESS. 2 If z>Ax = x>Ax then z>Az < x>Az.

5 / 21 6 / 21 Rock-Paper-Scissors Outline

Payoff R P S R 0 -1 1 P 1 0 -1 Evolutionarily stable strategies S -1 1 0 ESS and Nash equilibria ESMS: evolutionarily stable against mixed strategies Multiplayer ESS x = (1/3, 1/3, 1/3) is a Nash equilibrium. Is it an ESS?

For a mutant pure strategy z, z>Ax = 0 = z>Az. So x is not an ESS. Cycles can occur, with the population shifting between strategies.

7 / 21 8 / 21 ESS and Nash equilibria ESS and Nash equilibria

Theorem Every ESS is a Nash equilibrium. Definition A strategy profile x = (x ,..., x ) ∆ ∆ is a strict Nash This follows from the definition: 1∗ k∗ ∈ S1 × · · · × Sk equilibrium for utility functions u1,..., uk if, for each player Definition j 1,..., k , for all xj ∆Sj that is different from xj∗, A strategy x ∆ is an evolutionarily stable strategy (ESS) if, for any ∈ { } ∈ ∈ n pure strategy z = x, uj (xj , x∗ j ) < uj (xj∗, x∗ j ). 6 − − 1 z Ax x Ax > ≤ > 2 If z>Ax = x>Ax then z>Az < x>Az. A Nash equilibrium has uj (xj , x∗ j ) uj (xj∗, x∗ j ). − ≤ − Proof: If every pure strategy z satisfies z>Ax x>Ax, then every By the principle of indifference, only a pure Nash equilibrium can be a ≤ mixed strategy z (mixture of pure strategies) must obey the same strict Nash equilibrium. inequality. Hence,( x, x) is a Nash equilibrium. The converse is true for strict Nash equilibria.

9 / 21 10 / 21 ESS and Nash equilibria Outline

Theorem Every strict Nash equilibrium is an ESS.

Proof: A strict Nash equilibrium has z Ax < x Ax for z = x, so both Evolutionarily stable strategies > > 6 conditions defining an ESS are satisfied. ESS and Nash equilibria ESMS: evolutionarily stable against mixed strategies Multiplayer ESS Definition A strategy x ∆ is an evolutionarily stable strategy (ESS) if, for any ∈ n pure strategy z = x, 6 1 z Ax x Ax > ≤ > 2 If z>Ax = x>Ax then z>Az < x>Az.

11 / 21 12 / 21 ESMS: evolutionarily stable against mixed strategies ESMS: evolutionarily stable against mixed strategies

Proof Pure versus mixed When there are only two pure strategies, the space of mixed strategies,

An ESS is a Nash equilibrium( x , x ) satisfying, for all ei = x , if ∆2, is one-dimensional. So insisting that a small perturbation in the ∗ ∗ 6 ∗ direction of one or other pure strategies leads to a strict decrease in utility ei>Ax = x>Ax , then ei>Aei < x>Aei . ∗ ∗ ∗ ∗ is equivalent to insisting that a perturbation in the direction of any What about invasion by a mixed strategy? mixture of those pure strategies. For instance, if q > p, Say that a symmetric strategy( x , x ) is evolutionarily stable against ∗ ∗ mixed strategies (ESMS) if it is a Nash equilibrium and, for all mixed p q (1 ) +  strategies z = x , if z>Ax = x>Ax , then z>Az < x>Az. − 1 p 1 q 6 ∗ ∗ ∗ ∗ ∗  −   −  (ESS is sometimes defined this way, e.g., Leyton-Brown and Shoham) p + (q p) = − Clearly, every ESMS strategy is an ESS. 1 p + (1 q (1 p))  − − − −  q p p +  1−p (1 p) Theorem = q −p − 1 p +  1−p (0 (1 p))! For a two-player2 2 symmetric game, every ESS is ESMS. − − − − × q p p q p 1 = 1  − + 1  − . − 1 p 1 p − 1 p 0  −   −   −    13 / 21 14 / 21 ESMS: evolutionarily stable against mixed strategies Outline

ESS might not be ESMS

Consider a game with three actions, with an ESS at e1:

111 Evolutionarily stable strategies A = 10 20 .   ESS and Nash equilibria 1 200 ESMS: evolutionarily stable against mixed strategies   Multiplayer ESS 1 (e , e ) is a (pure) Nash equilibrium: e Ae = 1 1 = e Ae . 1 1 1> 1 ≥ j> 1 2 e1 is an ESS: ej>Ae1 = e1>Ae1 = 1, and ej>Aej = 0 < 1 = e1>Aej .

3 e1 is not ESMS: For x = (1/3, 1/3/1/3)>, x>Ax = 5 > 1 = e1>Ax. The text gives an example of a mixed ESS that is not ESMS.

15 / 21 16 / 21 Multiplayer evolutionarily stable strategies Multiplayer evolutionarily stable strategies

Consider a symmetric multiplayer game (that is, unchanged by relabeling the players). Definition Suppose that a symmetric mixed strategy x is invaded by a small (Suppose, for simplicity, that the utility for player i depends on si and on population of mutants z: x is replaced by (1 )x + z. the set of strategies played by the other players, but is invariant to a − permutation of the other players’ strategies.) Will the mix x survive? A strategy x ∆ is an evolutionarily stable strategy (ESS) if, for any ∈ n pure strategy z = x, 6 1 u1(z, x 1) u1(x, x 1) x is a Nash equilibrium. x’s utility: u1(x, z + (1 )x, z + (1 )x) − ≤ − ←− − − 2 If u1(z, x 1) = u1(x, x 1) then for all j = 1, 2 − − 6 = u1(x, z, x) + u1(x, x, z) + (1 2)u1(x, x, x) + O( ) u1(z, z, x 1, j ) < u1(x, z, x 1, j ). − − − − − z’s utility: u (z, z, x) + u (z, x, z) + (1 2)u (z, x, x) + O(2). 1 1 − 1

17 / 21 18 / 21 Example: Polluting Factories Example: Polluting Factories

ESS? (p, p, p) with p = Pr(purify).

1 Is u1(purify, purify, x) < u1(x, x, purify)? i.e.,1 > p2 + p(1 p) + 3(1 p)2? Equivalent to p >−(3 √3)/3−. − 2 Is u1(pollute, pollute, x) < u1(x, x, pollute)? (Karlin and Peres, 2016) (Karlin and Peres, 2016) 3 > p2 + (3 + 4)p(1 p) + 3(1 p)2? Equivalent to p > 1/−3. − Symmetric Nash equilibria (Karlin and Peres, 2016) (p, p, p) with p = Pr(purify): Equilibrium at p = (3 + √3)/6 is an ESS. p = (3 + √3)/6 0.79. The other equilibria are not. p = (3 √3)/6 ≈ 0.21. p = 0. − ≈

19 / 21 20 / 21 Outline

Evolutionarily stable strategies ESS and Nash equilibria ESMS: evolutionarily stable against mixed strategies Multiplayer ESS

21 / 21