Introduction to Game Theory

3a. More on Normal-Form Games

Dana Nau University of Maryland

Nau: Game Theory 1 More Solution Concepts

 Last time, we talked about several solution concepts  Pareto optimality  Nash equilibrium  Maximin and Minimax  Dominance  Rationalizability  We’ll continue with several more  Trembling-hand perfect equilibrium  ε-Nash equilibrium  Rationalizability  Evolutionarily stable strategies

Nau: Game Theory 2 Trembling-Hand Perfect Equilibrium

 A solution concept that’s stricter than Nash equilibrium

 “Trembling hand”: Requires that the equilibrium be robust against slight errors or “trembles” by the agents  I.e., small perturbations of their strategies

 Recall: A fully mixed strategy assigns every action a non-0 probability

 Let S = (s1, …, sn) be a mixed strategy profile for a game G  S is a (trembling hand) perfect equilibrium if there is a sequence of fully mixed-strategy profiles S0, S1, …, that has the following properties: k  lim k→∞ S = S k k k k k  for each S = (s1 , …, si , …, sn ), every strategy si is a best response to k the strategies S−i  The details are complicated, and I won’t discuss them

Nau: Game Theory 3 ε-Nash Equilibrium  Another solution concept  Reflects the idea that agents might not change strategies if the gain would be very small

 Let ε > 0. A strategy profile S = (s1, . . . , sn ) is an ε-Nash equilibrium if, for every agent i and for all strategies siʹ ≠ si,

ui (si , S−i ) ≥ ui (siʹ, S−i ) − ε  ε-Nash equilibria always exist  Every Nash equilibrium is surrounded by a region of ε-Nash equilibria for any ε > 0  This concept can be computationally useful  Algorithms to identify ε-Nash equilibria need consider only a finite set of mixed-strategy profiles (not the whole continuous space)  Because of finite precision, computers generally find only ε-Nash equilibria, where ε is roughly the machine precision

Nau: Game Theory 4 Problems with ε-Nash Equilibrium

 For every Nash equilibrium, there are ε-Nash equilibria that approximate it, but the converse isn’t true  There are ε-Nash equilibria that aren’t close to any Nash equilibrium  Example: the game at right has just one Nash equilibrium: (D, R)  We can use strategy elimination to get it: • D dominates U for agent 1 • On removing U, R dominates L for agent 2  (D, R) is also an ε-Nash equilibrium  But there’s another ε-Nash equilibrium: (U, L)  In this equilibrium, neither agent’s payoff is within ε of the agent’s payoff in a Nash equilibrium  Problem:  In the ε-Nash equilibrium (U, L), agent 1 can’t gain more than ε by deviating  But if agent 1 deviates, agent 2 can gain more than ε by best-responding to agent 1’s deviation

Nau: Game Theory 5 Problems with ε-Nash Equilibrium

 Some ε-Nash equilibria are very unlikely to arise  Agent 1 might not care about a gain of ε/2, but might reason as follows: • Agent 2 may expect agent 1 to to play D since it dominates U • So agent 2 is likely to play R • If agent 2 plays R, agent 1 does much better by playing D rather than U

 In general, ε-approximation is much messier in games than in optimization problems

Nau: Game Theory 6 Rationalizability

 A strategy is rationalizable if a perfectly rational agent could justifiably play it against perfectly rational opponents

 The formal definition is complicated  Informally, a strategy for agent i is rationalizable if it’s a best response to some beliefs that agent i could have about the strategies that the other agents will take  But agent i’s beliefs must take into account i’s knowledge of the rationality of the others. This incorporates • the other agents’ knowledge of i’s rationality, • their knowledge of i’s knowledge of their rationality, • and so on ad infinitum  A rationalizable strategy profile is a strategy profile that consists only of rationalizable strategies

Nau: Game Theory 7 Heads Tails Example Heads 1,–1 –1, 1 Matching Pennies  Agent 1’s pure strategy Heads is rationalizable Tails –1, 1 1,–1  Let’s look at the chain of beliefs

 For agent 1, Heads is a best response to agent 2’s pure strategy Heads, …  … and believing that 2 would also play Heads is consistent with 2’s rationality, for the following reasons  2 could believe that 1 would play Tails, to which 2’s best response is Heads; …  … and it would be rational for 2 to believe that 1 would play Tails, for the following reasons: • 2 could believe that 1 believed that 2 would play Tails, to which Tails is a best response; …

Nau: Game Theory 8 Strategies that aren’t rationalizable

Prisoner’s Dilemma  Strategy C isn’t rationalizable for agent 1 3, 3 0, 5  It isn’t a best response to any of agent 2’s strategies 5, 0 1, 1 The 3x3 game we used earlier  M is not a rationalizable strategy for agent 1  It is a best response to one of agent 2’s strategies, namely R  But there’s no belief that agent 2 could have about agent 1’s strategy for which R would be a best response

Nau: Game Theory 9 Comments

 The formal definition of rationalizability is complicated because of the infinite regress  But we can say some intuitive things about rationalizable strategies  Nash equilibrium strategies are always rationalizable  So the set of rationalizable strategies (and strategy profiles) is always nonempty  In two-player games, rationalizable strategies are simply those that survive the iterated elimination of strictly dominated strategies  In n-agent games, this isn’t so  Rather, rationalizable strategies are those that survive iterative removal of strategies that are never a best response to any strategy profile by the other agents  Example: the p-beauty contest

Nau: Game Theory 10 The p-Beauty Contest

 At the start of my first class, I asked you to do the following:  Choose a number in the range from 0 to 100  Write it on a piece of paper, along with your name  In a few minutes, I’ll ask you to pass your papers to the front of the room  After class, I’ll compute the average of all of the numbers  The winner(s) will be whoever chose a number that’s closest to 2/3 of the average  I’ll announce the results in a subsequent class

 This game is famous among economists and game theorists  It’s called the p-beauty contest  I used p = 2/3

Nau: Game Theory 11 The p-Beauty Contest

 Recall that in n-player games,  Rationalizable strategies are those that survive iterative removal of strategies that are never a best response to any strategy profile by the other agents  In the p-beauty contest, consider the strategy profile in which everyone else chooses 100  Every number in the interval [0,100) is a best response  Thus every number in the interval [0,100) is rationalizable

Nau: Game Theory 12 Nash Equilibrium for the p-Beauty Contest

 Iteratively eliminate dominated strategies  All numbers ≤ 100 => 2/3(average) < 67 => any strategy that includes numbers ≥ 67 isn’t a best response to any strategy profile, so eliminate it  The remaining strategies only include numbers < 67 => for every rationalizable strategy profile, 2/3(average) < 45 => any strategy that includes numbers ≥ 45 isn’t a best response to any strategy profile, so eliminate it  Rationalizable strategies only include numbers < 45 => for every rationalizable strategy profile, 2/3(average) < 30

. . .  The only strategy profile that survives elimination of dominated strategies:  Everybody chooses 0  Therefore this is the unique Nash equilibrium

Nau: Game Theory 13 p-Beauty Contest Results

 (2/3)(average) = 21  winner = Giovanni

Nau: Game Theory 14 Another Example of p-Beauty Contest Results

 Average = 32.93  2/3 of the average = 21.95  Winner: anonymous xx

Nau: Game Theory 15 We aren’t rational

 Most of you didn’t play Nash equilibrium strategies

 We aren’t game-theoretically rational agents  Huge literature on behavioral economics going back to about 1979  Many cases where humans (or aggregations of humans) tend to make different decisions than the game-theoretically optimal ones  Daniel Kahneman received the 2002 Nobel Prize in Economics for his work on that topic

Nau: Game Theory 16 Choosing “Irrational” Strategies

 Why choose a non-equilibrium strategy?  Limitations in reasoning ability • Didn’t calculate the Nash equilibrium correctly • Don’t know how to calculate it • Don’t even know the concept  Hidden payoffs • Other things may be more important than winning › Want to be helpful › Want to see what happens › Want to create mischief  Agent modeling (next slide)

Nau: Game Theory 17 Agent Modeling

 A Nash equilibrium strategy is best for you if the other agents also use their Nash equilibrium strategies

 In many cases, the other agents won’t use Nash equilibrium strategies  If you can forecast their actions accurately, you may be able to do much better than the Nash equilibrium strategy

 I’ll say more about this in Session 9  Incomplete-information games

Nau: Game Theory 18 Evolutionarily Stable Strategies

 An evolutionarily stable strategy (ESS) is a mixed strategy that’s “resistant to invasion” by new strategies  This concept comes from evolutionary biology  Consider how various species’ relative “fitness” causes their proportions of the population to grow or shrink

 For us, an organism’s fitness = its expected payoff from interacting with a random member of the population

 An organism’s strategy = anything that might affect its fitness • size, aggressiveness, sensory abilities, intelligence, …  Suppose a small population of “invaders” playing a different strategy is added to a population  The original strategy is an ESS if it gets a higher payoff against the mixture of the new and old strategies than the invaders do

Nau: Game Theory 19 r r' Evolutionary Stability r a b  Let G be a symmetric 2-player game  Recall that the matrix shows u(r,r') = payoff for r against r' r' c d  A strategy r' invades a strategy r at level x if fraction x of the population uses r' and fraction (1–x) of the population uses r  fitness(r) = expected payoff for r against a random member of the population = (1–x)a + xb  Similarly, fitness(r') = (1–x)c + xd  r is evolutionarily stable against r' if there is an ε > 0 such that for every x < ε, fitness(r) > fitness(r')  i.e., (1–x)a + xb > (1–x)c + xd  As x → 0, (1–x)a + xb → a and (1–x)c + xd → c  For sufficiently small x, the inequality holds if either a > c, or a = c and b > d  Thus r is evolutionarily stable against r' iff one of the following holds:  a > c  a = c and b > d

Nau: Game Theory 20 r r' Evolutionary Stability r a b  More generally  We’ll use a mixed strategy s to represent a population r' c d that is composed of several different species  We’ll talk about s’s evolutionary stability against all other mixed strategies  s is evolutionarily stable iff for every mixed strategy s' ≠ s, one of the following holds: • u(s,s) > u(s',s) • u(s,s) = u(s',s) and u(s,s') > u(s',s')  s is weakly evolutionarily stable iff for every mixed strategy sʹ ≠ s, one of the following holds: • u(s,s) > u(s',s) • u(s,s) = u(s',s) and u(s,s') > u(s',s')  Includes cases where the original strategy and invading strategy have the same fitness, so the population with the invading strategy neither grows nor shrinks

Nau: Game Theory 21 Example

 The Hawk-Dove game  2 animals contend for a piece of food  The animals are chosen at random from the entire population • Each animal may be either a hawk (H) or a dove (D)  The prize is worth 6 to each  Fighting costs each 5  When a hawk meets a dove, the hawk gets the prize without a fight: payoffs 6, 0  When 2 doves meet, they split the prize without a fight: payoff 3, 3  When 2 hawks meet, they fight (–5 for each), each with a 50% chance of getting the prize ((0.5)(6) = 3): payoffs –2,–2  It’s easy to show that this game has a unique Nash equilibrium (s, s), where s = (3/5, 2/5)  i.e., 60% hawks, 40% doves Nau: Game Theory 22 Example

 To confirm that s is also an ESS, show that, for all sʹ ≠ s,

u1(s, sʹ) = u1(sʹ, s) and u1(s, sʹ) > u1(sʹ, sʹ)

 u1(s,sʹ) = u1(sʹ,s) is true of any mixed strategy equilibrium with full support

 To show u1(s,sʹ) > u1(sʹ,sʹ), find the sʹ that minimizes

f (sʹ) = u1(s,sʹ) − u1(sʹ,sʹ)  s = “play H with probability 3/5, D with probability 2/5”  sʹ = “play H with probability p, D with probability 1–p”

 u1(s,s') = (3/5)[–2p + 6(1–p)] + (2/5)[0p + 3(1–p)]

 u1(s',s') = p[–2p + 6(1–p)] + (1–p)[0p + 3(1–p)]

 so f (sʹ) = u1(s,sʹ) − u1(sʹ,sʹ) is quadratic in p

 Set d f(s')/d p = 0, solve for p => p = 3/5 • So the unique minimum occurs when sʹ = s

Nau: Game Theory 23 Evolutionary Stability and Nash Equilibria

Theorem. Let G be a symmetric 2-player game, and s be a mixed strategy. If s is an evolutionarily stable strategy, then (s, s) is a Nash equilibrium of G.

Proof. By definition, an ESS s must satisfy u(s,s) ≥ u(sʹ,s), i.e., s is a best response to itself, so it must be a Nash equilibrium. Theorem. Let G be a symmetric 2-player game, and s be a mixed strategy. If (s,s) is a strict Nash equilibrium of G, then s is an evolutionarily stable strategy.

Proof. If (s,s) is a strict Nash equilibrium, then u(s,s) > u(sʹ,s).  This satisfies the first of the two alternative criteria of an ESS

Nau: Game Theory 24 Summary

 We’ve discussed several more solution concepts  trembling-hand perfect equilibria  epsilon-Nash equilibria  rationalizability • the p-Beauty Contest  evolutionarily stable strategies • Hawk-Dove game

Nau: Game Theory 25