Recap Mixed Strategies Fun Game Maxmin and Minmax Mixed Strategies; Maxmin CPSC 532A Lecture 4 January 28, 2008 Mixed Strategies; Maxmin CPSC 532A Lecture 4, Slide 1 Recap Mixed Strategies Fun Game Maxmin and Minmax Lecture Overview 1 Recap 2 Mixed Strategies 3 Fun Game 4 Maxmin and Minmax Mixed Strategies; Maxmin CPSC 532A Lecture 4, Slide 2 Recap Mixed Strategies Fun Game Maxmin and Minmax Example games We saw a variety of example games: Zero-sum: matching pennies Pure cooperation: coordination General-sum: battle of the sexes; prisoner's dilemma Mixed Strategies; Maxmin CPSC 532A Lecture 4, Slide 3 Recap Mixed Strategies Fun Game Maxmin and Minmax Pareto Optimality Sometimes, one outcome o is at least as good for every agent as another outcome o0, and there is some agent who strictly prefers o to o0 in this case, it seems reasonable to say that o is better than o0 we say that o Pareto-dominates o0. An outcome o∗ is Pareto-optimal if there is no other outcome that Pareto-dominates it. Mixed Strategies; Maxmin CPSC 532A Lecture 4, Slide 4 Recap Mixed Strategies Fun Game Maxmin and Minmax Best Response, Nash equilibrium If you knew what everyone else was going to do, it would be easy to pick your own action Let a−i = a1; : : : ; ai−1; ai+1; : : : ; an . h i ∗ Best response: ai BR(a−i) iff ∗ 2 ai Ai; ui(a ; a−i) ui(ai; a−i) 8 2 i ≥ Nash equilibrium: stable action profiles. a = a1; : : : ; an is a (\pure strategy") Nash equilibrium iff h i i; ai BR(a−i). 8 2 Mixed Strategies; Maxmin CPSC 532A Lecture 4, Slide 5 Recap Mixed Strategies Fun Game Maxmin and Minmax Lecture Overview 1 Recap 2 Mixed Strategies 3 Fun Game 4 Maxmin and Minmax Mixed Strategies; Maxmin CPSC 532A Lecture 4, Slide 6 Recap Mixed Strategies Fun Game Maxmin and Minmax Mixed Strategies It would be a pretty bad idea to play any deterministic strategy in matching pennies Idea: confuse the opponent by playing randomly Define a strategy si for agent i as any probability distribution over the actions Ai. pure strategy: only one action is played with positive probability mixed strategy: more than one action is played with positive probability these actions are called the support of the mixed strategy Let the set of all strategies for i be Si Let the set of all strategy profilesbe S = S1 ::: Sn. × × Mixed Strategies; Maxmin CPSC 532A Lecture 4, Slide 7 Instead, use the idea of expected utility from decision theory: X ui(s) = ui(a)P r(a s) j a2A Y P r(a s) = sj(aj) j j2N Recap Mixed Strategies Fun Game Maxmin and Minmax Utility under Mixed Strategies What is your payoff if all the players follow mixed strategy profile s S? 2 We can't just read this number from the game matrix anymore: we won't always end up in the same cell Mixed Strategies; Maxmin CPSC 532A Lecture 4, Slide 8 Recap Mixed Strategies Fun Game Maxmin and Minmax Utility under Mixed Strategies What is your payoff if all the players follow mixed strategy profile s S? 2 We can't just read this number from the game matrix anymore: we won't always end up in the same cell Instead, use the idea of expected utility from decision theory: X ui(s) = ui(a)P r(a s) j a2A Y P r(a s) = sj(aj) j j2N Mixed Strategies; Maxmin CPSC 532A Lecture 4, Slide 8 Nash equilibrium: s = s ; : : : ; s is a Nash equilibrium iff i; s BR(s− ) h 1 ni 8 i 2 i Every finite game has a Nash equilibrium! [Nash, 1950] e.g., matching pennies: both players play heads/tails 50%/50% Recap Mixed Strategies Fun Game Maxmin and Minmax Best Response and Nash Equilibrium Our definitions of best response and Nash equilibrium generalize from actions to strategies. Best response: ∗ ∗ s BR(s− ) iff s S ; u (s ; s− ) u (s ; s− ) i 2 i 8 i 2 i i i i ≥ i i i Mixed Strategies; Maxmin CPSC 532A Lecture 4, Slide 9 Every finite game has a Nash equilibrium! [Nash, 1950] e.g., matching pennies: both players play heads/tails 50%/50% Recap Mixed Strategies Fun Game Maxmin and Minmax Best Response and Nash Equilibrium Our definitions of best response and Nash equilibrium generalize from actions to strategies. Best response: ∗ ∗ s BR(s− ) iff s S ; u (s ; s− ) u (s ; s− ) i 2 i 8 i 2 i i i i ≥ i i i Nash equilibrium: s = s ; : : : ; s is a Nash equilibrium iff i; s BR(s− ) h 1 ni 8 i 2 i Mixed Strategies; Maxmin CPSC 532A Lecture 4, Slide 9 Recap Mixed Strategies Fun Game Maxmin and Minmax Best Response and Nash Equilibrium Our definitions of best response and Nash equilibrium generalize from actions to strategies. Best response: ∗ ∗ s BR(s− ) iff s S ; u (s ; s− ) u (s ; s− ) i 2 i 8 i 2 i i i i ≥ i i i Nash equilibrium: s = s ; : : : ; s is a Nash equilibrium iff i; s BR(s− ) h 1 ni 8 i 2 i Every finite game has a Nash equilibrium! [Nash, 1950] e.g., matching pennies: both players play heads/tails 50%/50% Mixed Strategies; Maxmin CPSC 532A Lecture 4, Slide 9 60 3 Competition and Coordination: Normal form games Rock Paper Scissors Rock 0 1 1 Recap Mixed Strategies − Fun Game Maxmin and Minmax Paper 1 0 1 Computing Mixed Nash Equilibria: Battle− of the Sexes Scissors 1 1 0 − Figure 3.6 Rock, Paper, Scissors game. B F B 2, 1 0, 0 F 0, 0 1, 2 Figure 3.7 Battle of the Sexes game. It's hard in general to compute Nash equilibria, but it's easy 3.2.2 Strategies in normal-form games when you can guess the support We have so far defined the actions available to each player in a game, but not yet his Forset of BoS,strategies let's, or hislook available foran choices. equilibrium Certainly one where kind of strategy all actions is to select are pure strategy a single action and play it; we call such a strategy a pure strategy, and we will use partthe notation of the we supporthave already developed for actions to represent it. There is, however, another, less obvious type of strategy; a player can choose to randomize over the set of available actions according to some probability distribution; such a strategy is called mixed strategy a mixed strategy. Although it may not be immediately obvious why a player should introduce randomness into his choice of action, in fact in a multi-agent setting the role of mixed strategies is critical. We will return to this when we discuss solution concepts Mixed Strategies;for Maxmin games in the next section. CPSC 532A Lecture 4, Slide 10 We define a mixed strategy for a normal form game as follows. Definition 3.2.4 Let (N, (A1,...,An), O, µ, u) be a normal form game, and for any set X let Π(X) be the set of all probability distributions over X. Then the set of mixed mixed strategy strategies for player i is Si = Π(Ai). The set of mixed strategy profiles is simply the profiles Cartesian product of the individual mixed strategy sets, S S . 1 × · · · × n By si(ai) we denote the probability that an action ai will be played under mixed strategy si. The subset of actions that are assigned positive probability by the mixed strategy si is called the support of si. Definition 3.2.5 The support of a mixed strategy si for a player i is the set of pure strategies a s (a ) > 0 . { i| i i } c Shoham and Leyton-Brown, 2006 u1(B) = u1(F ) 2p + 0(1 p) = 0p + 1(1 p) − − 1 p = 3 60 3 Competition and Coordination: Normal form games Rock Paper Scissors Rock 0 1 1 Recap Mixed Strategies − Fun Game Maxmin and Minmax Paper 1 0 1 Computing Mixed Nash Equilibria: Battle− of the Sexes Scissors 1 1 0 − Figure 3.6 Rock, Paper, Scissors game. B F B 2, 1 0, 0 F 0, 0 1, 2 Let player 2 play B with p, F with 1 p. Figure 3.7 Battle of the Sexes game.− If player 1 best-responds with a mixed strategy, player 2 must make him indifferent between F and B (why?) 3.2.2 Strategies in normal-form games We have so far defined the actions available to each player in a game, but not yet his set of strategies, or his available choices. Certainly one kind of strategy is to select pure strategy a single action and play it; we call such a strategy a pure strategy, and we will use the notation we have already developed for actions to represent it. There is, however, another, less obvious type of strategy; a player can choose to randomize over the set of available actions according to some probability distribution; such a strategy is called mixed strategy a mixed strategy. Although it may not be immediately obvious why a player should introduce randomness into his choice of action, in fact in a multi-agent setting the role of mixed strategies is critical. We will return to this when we discuss solution concepts Mixed Strategies;for Maxmin games in the next section. CPSC 532A Lecture 4, Slide 10 We define a mixed strategy for a normal form game as follows.