Maxmin and Minmax

Home , Best response, Matching pennies, Nash equilibrium, Outcome (game theory)

Recap Computing Mixed NE Fun Game Maxmin and Minmax

Lecture 5

Computing Nash Equilibrium; Maxmin Lecture 5, Slide 1 Recap Computing Mixed NE Fun Game Maxmin and Minmax Lecture Overview

1 Recap

2 Computing Mixed Nash Equilibria

3 Fun Game

4 Maxmin and Minmax

Computing Nash Equilibrium; Maxmin Lecture 5, Slide 2 Recap Computing Mixed NE Fun Game Maxmin and Minmax Pareto Optimality

Idea: sometimes, one outcome o is at least as good for every agent as another outcome o0, and there is some agent who strictly prefers o to o0 in this case, it seems reasonable to say that o is better than o0 we say that o Pareto-dominates o0.

An outcome o∗ is Pareto-optimal if there is no other outcome that Pareto-dominates it. a game can have more than one Pareto-optimal outcome every game has at least one Pareto-optimal outcome

Computing Nash Equilibrium; Maxmin Lecture 5, Slide 3 Recap Computing Mixed NE Fun Game Maxmin and Minmax Best Response

If you knew what everyone else was going to do, it would be easy to pick your own action

Let a−i = a1, . . . , ai−1, ai+1, . . . , an . h i now a = (a−i, ai)

∗ Best response: ai BR(a−i) iﬀ ∗ ∈ ai Ai, ui(a , a−i) ui(ai, a−i) ∀ ∈ i ≥

Computing Nash Equilibrium; Maxmin Lecture 5, Slide 4 Recap Computing Mixed NE Fun Game Maxmin and Minmax Nash Equilibrium

Now we return to the setting where no agent knows anything about what the others will do

Idea: look for stable action proﬁles.

a = a1, . . . , an is a (“pure strategy”) Nash equilibrium iﬀ h i i, ai BR(a−i). ∀ ∈

Computing Nash Equilibrium; Maxmin Lecture 5, Slide 5 Recap Computing Mixed NE Fun Game Maxmin and Minmax Mixed Strategies

It would be a pretty bad idea to play any deterministic strategy in matching pennies Idea: confuse the opponent by playing randomly

Deﬁne a strategy si for agent i as any probability distribution over the actions Ai. pure strategy: only one action is played with positive probability mixed strategy: more than one action is played with positive probability these actions are called the support of the mixed strategy

Let the set of all strategies for i be Si

Let the set of all strategy proﬁlesbe S = S1 ... Sn. × ×

Computing Nash Equilibrium; Maxmin Lecture 5, Slide 6 Recap Computing Mixed NE Fun Game Maxmin and Minmax Utility under Mixed Strategies

What is your payoﬀ if all the players follow mixed strategy proﬁle s S? ∈ We can’t just read this number from the game matrix anymore: we won’t always end up in the same cell Instead, use the idea of expected utility from decision theory: X ui(s) = ui(a)P r(a s) | a∈A Y P r(a s) = sj(aj) | j∈N

Computing Nash Equilibrium; Maxmin Lecture 5, Slide 7 Recap Computing Mixed NE Fun Game Maxmin and Minmax Best Response and Nash Equilibrium

Our deﬁnitions of best response and Nash equilibrium generalize from actions to strategies. Best response: ∗ ∗ s BR(s− ) iﬀ s S , u (s , s− ) u (s , s− ) i ∈ i ∀ i ∈ i i i i ≥ i i i

Nash equilibrium:

s = s , . . . , s is a Nash equilibrium iﬀ i, s BR(s− ) h 1 ni ∀ i ∈ i

Every ﬁnite game has a Nash equilibrium! [Nash, 1950] e.g., matching pennies: both players play heads/tails 50%/50%

Computing Nash Equilibrium; Maxmin Lecture 5, Slide 8 Recap Computing Mixed NE Fun Game Maxmin and Minmax Lecture Overview

1 Recap

2 Computing Mixed Nash Equilibria

3 Fun Game

4 Maxmin and Minmax

Computing Nash Equilibrium; Maxmin Lecture 5, Slide 9 60 3 Competition and Coordination: Normal form games

Rock Paper Scissors

Rock 0 1 1 Recap Computing Mixed NE − Fun Game Maxmin and Minmax

Paper 1 0 1 Computing Mixed Nash Equilibria: Battle− of the Sexes Scissors 1 1 0 −

Figure 3.6 Rock, Paper, Scissors game.

B F

B 2, 1 0, 0

F 0, 0 1, 2

Figure 3.7 Battle of the Sexes game. It’s hard in general to compute Nash equilibria, but it’s easy 3.2.2 Strategies in normal-form games when you can guess the support We have so far deﬁned the actions available to each player in a game, but not yet his Forset of BoS,strategies let’s, or hislook available foran choices. equilibrium Certainly one where kind of strategy all actions is to select are pure strategy a single action and play it; we call such a strategy a pure strategy, and we will use partthe notation of the we supporthave already developed for actions to represent it. There is, however, another, less obvious type of strategy; a player can choose to randomize over the set of available actions according to some probability distribution; such a strategy is called mixed strategy a mixed strategy. Although it may not be immediately obvious why a player should introduce randomness into his choice of action, in fact in a multi-agent setting the role of mixed strategies is critical. We will return to this when we discuss solution concepts Computing Nashfor Equilibrium; games in Maxmin the next section. Lecture 5, Slide 10 We deﬁne a mixed strategy for a normal form game as follows.

Definition 3.2.4 Let (N, (A1,...,An), O, µ, u) be a normal form game, and for any set X let Π(X) be the set of all probability distributions over X. Then the set of mixed mixed strategy strategies for player i is Si = Π(Ai). The set of mixed strategy profiles is simply the profiles Cartesian product of the individual mixed strategy sets, S S . 1 × · · · × n

By si(ai) we denote the probability that an action ai will be played under mixed strategy si. The subset of actions that are assigned positive probability by the mixed strategy si is called the support of si.

Deﬁnition 3.2.5 The support of a mixed strategy si for a player i is the set of pure strategies a s (a ) > 0 . { i| i i } c Shoham and Leyton-Brown, 2006

u1(B) = u1(F ) 2p + 0(1 p) = 0p + 1(1 p) − − 1 p = 3

60 3 Competition and Coordination: Normal form games

Rock Paper Scissors

Rock 0 1 1 −

Paper 1 0 1 − Recap Computing Mixed NE Fun Game Maxmin and Minmax Scissors 1 1 0 Computing Mixed Nash Equilibria:− Battle of the Sexes Figure 3.6 Rock, Paper, Scissors game.

B F

B 2, 1 0, 0

F 0, 0 1, 2

Figure 3.7 Battle of the Sexes game. Let player 2 play B with p, F with 1 p. − 3.2.2If Strategies player 1in normal-formbest-responds games with a mixed strategy, player 2 must We have so far defined the actions available to each player in a game, but not yet his makeset of strategies him indifferent, or his available between choices. CertainlyF and oneB kind(why?) of strategy is to select pure strategy a single action and play it; we call such a strategy a pure strategy, and we will use the notation we have already developed for actions to represent it. There is, however, another, less obvious type of strategy; a player can choose to randomize over the set of available actions according to some probability distribution; such a strategy is called mixed strategy a mixed strategy. Although it may not be immediately obvious why a player should introduce randomness into his choice of action, in fact in a multi-agent setting the role of mixed strategies is critical. We will return to this when we discuss solution concepts for games in the next section. We define a mixed strategy for a normal form game as follows.

Computing NashDefinition Equilibrium; 3.2.4 MaxminLet (N, (A1,...,An), O, µ, u) be a normal form game, and for anyLecture 5, Slide 10 set X let Π(X) be the set of all probability distributions over X. Then the set of mixed mixed strategy strategies for player i is Si = Π(Ai). The set of mixed strategy profiles is simply the profiles Cartesian product of the individual mixed strategy sets, S S . 1 × · · · × n

Deﬁnition 3.2.5 The support of a mixed strategy si for a player i is the set of pure strategies a s (a ) > 0 . { i| i i } c Shoham and Leyton-Brown, 2006

60 3 Competition and Coordination: Normal form games

Rock Paper Scissors

Rock 0 1 1 −

Paper 1 0 1 − Recap Computing Mixed NE Fun Game Maxmin and Minmax Scissors 1 1 0 Computing Mixed Nash Equilibria:− Battle of the Sexes Figure 3.6 Rock, Paper, Scissors game.

B F

B 2, 1 0, 0

F 0, 0 1, 2

Figure 3.7 Battle of the Sexes game. Let player 2 play B with p, F with 1 p. − 3.2.2If Strategies player 1in normal-formbest-responds games with a mixed strategy, player 2 must We have so far defined the actions available to each player in a game, but not yet his makeset of strategies him indifferent, or his available between choices. CertainlyF and oneB kind(why?) of strategy is to select pure strategy a single action and play it; we call such a strategy a pure strategy, and we will use the notation we have already developedu1(B for) actions = u1 to(F repres) ent it. There is, however, another, less obvious type of strategy; a player can choose to randomize over the set of available actions according2p + to some0(1 probabilityp) = distribut0p + 1(1ion; suchp a) strategy is called mixed strategy a mixed strategy. Although it may not− be immediately obvious− why a player should 1 introduce randomness into his choice of action,p = in fact in a multi-agent setting the role of mixed strategies is critical. We will return to this3 when we discuss solution concepts for games in the next section. We define a mixed strategy for a normal form game as follows.

Deﬁnition 3.2.5 The support of a mixed strategy si for a player i is the set of pure strategies a s (a ) > 0 . { i| i i } c Shoham and Leyton-Brown, 2006

Let player 1 play B with q, F with 1 q. − u2(B) = u2(F ) q + 0(1 q) = 0q + 2(1 q) − − 2 q = 3 2 1 1 2 Thus the strategies ( 3 , 3 ), ( 3 , 3 ) are a Nash equilibrium.

60 3 Competition and Coordination: Normal form games

Rock Paper Scissors

Rock 0 1 1 −

Paper 1 0 1 − Recap Computing Mixed NE Fun Game Maxmin and Minmax Scissors 1 1 0 Computing Mixed Nash Equilibria:− Battle of the Sexes Figure 3.6 Rock, Paper, Scissors game.

B F

B 2, 1 0, 0

F 0, 0 1, 2

Figure 3.7 Battle of the Sexes game. Likewise, player 1 must randomize to make player 2 3.2.2indifferent. Strategies in normal-form games Why is player 1 willing to randomize? We have so far defined the actions available to each player in a game, but not yet his set of strategies, or his available choices. Certainly one kind of strategy is to select pure strategy a single action and play it; we call such a strategy a pure strategy, and we will use the notation we have already developed for actions to represent it. There is, however, another, less obvious type of strategy; a player can choose to randomize over the set of available actions according to some probability distribution; such a strategy is called mixed strategy a mixed strategy. Although it may not be immediately obvious why a player should introduce randomness into his choice of action, in fact in a multi-agent setting the role of mixed strategies is critical. We will return to this when we discuss solution concepts for games in the next section. We define a mixed strategy for a normal form game as follows.

Deﬁnition 3.2.5 The support of a mixed strategy si for a player i is the set of pure strategies a s (a ) > 0 . { i| i i } c Shoham and Leyton-Brown, 2006

60 3 Competition and Coordination: Normal form games

Rock Paper Scissors

Rock 0 1 1 −

Paper 1 0 1 − Recap Computing Mixed NE Fun Game Maxmin and Minmax Scissors 1 1 0 Computing Mixed Nash Equilibria:− Battle of the Sexes Figure 3.6 Rock, Paper, Scissors game.

B F

B 2, 1 0, 0

F 0, 0 1, 2

Figure 3.7 Battle of the Sexes game. Likewise, player 1 must randomize to make player 2 3.2.2indifferent. Strategies in normal-form games Why is player 1 willing to randomize? We have so far defined the actions available to each player in a game, but not yet his Letset of playerstrategies 1, or play his availableB with choices.q, F Certainlywith 1 one kindq. of strategy is to select pure strategy a single action and play it; we call such a strategy a pure− strategy, and we will use the notation we have already developedu2(B for)= actionsu2( toF repres) ent it. There is, however, another, less obvious type of strategy; a player can choose to randomize over the set of available actions accordingq + to 0(1 some probabilityq) = 0 distributq + 2(1ion; suchq a) strategy is called − − mixed strategy a mixed strategy. Although it may not be immediately2 obvious why a player should introduce randomness into his choice of action,q = in fact in a multi-agent setting the role of mixed strategies is critical. We will return to this3 when we discuss solution concepts for games in the next section. We define a mixed strategy for2 a1 normal1 form2 game as follows. Thus the strategies ( 3 , 3 ), ( 3 , 3 ) are a Nash equilibrium.

Deﬁnition 3.2.5 The support of a mixed strategy si for a player i is the set of pure strategies a s (a ) > 0 . { i| i i } c Shoham and Leyton-Brown, 2006

Recap Computing Mixed NE Fun Game Maxmin and Minmax Interpreting Mixed Strategy Equilibria

What does it mean to play a mixed strategy? Randomize to confuse your opponent consider the matching pennies example Players randomize when they are uncertain about the other’s action consider battle of the sexes Mixed strategies are a concise description of what might happen in repeated play: count of pure strategies in the limit Mixed strategies describe population dynamics: 2 agents chosen from a population, all having deterministic strategies. MS is the probability of getting an agent who will play one PS or another.

Computing Nash Equilibrium; Maxmin Lecture 5, Slide 11 Recap Computing Mixed NE Fun Game Maxmin and Minmax Lecture Overview

1 Recap

2 Computing Mixed Nash Equilibria

3 Fun Game

4 Maxmin and Minmax

Computing Nash Equilibrium; Maxmin Lecture 5, Slide 12 What does row player do in equilibrium of this game? row player randomizes 50-50 all the time that’s what it takes to make column player indifferent What happens when people play this game? with payoff of 320, row player goes up essentially all the time with payoff of 44, row player goes down essentially all the time