Section Notes 6 Game Theory Applied Math 121 Week of March 22, 2010 Goals for the week • be comfortable with the elements of game theory. • understand the difference between pure and mixed strategies. • be able to identify a player’s best reponse. • understand the concept of a Nash equilibrium and know how to find Nash equilibria. Contents 1 Let’s play a game: a burglary gone bad... 2 2 Two player games, Payoff Matrices, Pure Strategies 3 3 Best Responses and Dominated Strategies 5 4 Pure Nash Equilibrium 7 5 MiniMax Technique for finding Mixed Nash Equilibria in Zero-Sum Games 8 1 1 Let’s play a game: a burglary gone bad... Exercise 1 Imagine the following scenario: two suspects, A and B, are arrested by the police. The police believes suspects A and B have committed a burglary, but they have insufficient evidence for a conviction. Thus they will offer the prisoners the following deal: if one testifies for the prosecution against the other and the other remains silent, the betrayer goes free and the silent accomplice receives the full 10-year sentence. If both remain silent, both prisoners are sentenced to only one year in jail for a minor charge (e.g. trespassing). If each betrays the other, each receives a five-year sentence. Each prisoner must make the choice of whether to betray the other or to remain silent. 1. What should the two prisoners do when they are held in the same room and can communicate with each other and can make a decision on what to do together? 2. What should each prisoner individually do if they are kept separate, i.e. the prisoners cannot talk to each other? 3. Give an explanation for why this game is called “The Prisoner’s Dilemma” (in the second case when the players are kept separate). End Exercise 1 By completing the above exercises, you should now: • have some intuition about a non-zero sum game • be able to identify how players would look at their rewards. 2 Two player games, Payoff Matrices, Pure Strategies So far in this course we have used optimization methods (linear programming up until now) to maximize the objective function of an individual assuming that all the decision making power is in the hands of that individual. However, the situation changes drastically when 2 or more players are competing with each other. Generally, different players have conflicting interests, i.e. they want to maximize different objective functions (imagine two car companies, imagine two presidential candidates, . ). In these cases, we can use game theory to analyze such situations, reason about 2 what will probably happen, and maximize our personal utility given the actions of all the other players around. 2.1 Review A two-player game is defined by the following elements: • A set of player 1 and 2. • A set of actions {a1, ..., an}, {b1, ..., bm} for each player 1,2. These actions are also called pure strategies. • A payoff matrix defining the utility for each combination of strategies played by each player. The entry aij specifies a tuple (π1(i, j), π2(i, j)) for the case when player 1 chooses action i and player 2 chooses action j. The tuple (π1(i, j), π2(i, j)) denotes that player 1 gets utility π1(i, j) and player 2 gets utility π2(i, j). • If all entries aij of the payoff matrix is such that payoffs for players 1 and 2 add up to 0, that is, π1(i, j) + π2(i, j) = 0 ∀i, j, then we call such a game a two-player zero sum game. Sometimes, we will then simply omit the payoffs for player 2 when writing down the payoff matrix for a zero sum game, because they are simply the negative of the payoffs for player 1. 2.2 Practice Exercise 2 Consider the following TF-Student Game: Every week, a TF of AM121 has to choose whether he puts a lot of effort, little effort, or no effort at all into preparing for the section. The students, on the other hand must decide whether they attend the section or not (for simplicity, let’s model the students as one player here). Obviously, preparing the section is costly for the TF, but if a lot of students come and he is well-prepared, he gets a very high reward. On the other, if he is badly-prepared and his students attend section he will be embarrassed. The students’ payoff, of course, depends on how well the TF is prepared. And obviously, if the students don’t attend section, they get 0 payoff. 1. List all pure strategies for both players of the game. 2. Write down a payoff matrix with payoffs that you think represent the TF-Student game most appropriately. 3. What do you think will be the outcome of the game when played? End Exercise 2 3 By completing the above exercises, you should now: • understand when game theory is used • be comfortable with the elements of game theory. 3 Best Responses and Dominated Strategies 3.1 Review We need a few more solution concepts: 1. Best Response: When the strategy of player 2 is given, player 1 has a best response strategy to the strategy by player 2. Remember, in the Prisoner’s Dilemma game, the best response of player 1 was always to testify, independent of what player 2’s strategy was. In general, when player 2’s strategy j is fixed, the best response of player 1 is found by examining all payoff entries a1j, ..., anj that correspond to player 2’s choice and selecting the strategy that maximizes player 1’s payoff among these choices. 2. Dominated Strategy: Strategy i for player 1 dominates strategy i0 if for all choices of 0 0 j by player 2 the payoff for player 1 π1(i, j) ≥ π1(i , j) and π1(i, j) > π1(i , j) for some j ∈ {1, ..., m}. We say that strategy i0 is dominated by strategy i for player 1. 3. Iterated Elimination of Dominated Strategies: No player will ever choose a dominated strategy. Thus, those can be eliminated from the set of pure strategies. This process can be iterated, i.e. after one strategy has been eliminated (by either player) any remaining dominated strategies can be eliminated (by either player) and so on. 4 3.2 Practice Exercise 3 For this exercise, consider the following game: Player 2 Left Center Right Up (3,0) (0,1) (0,2) Player 1 Middle (0,0) (2,2) (1,-1) Down (0,0) (3,1) (0,0) 1. What is the best response of player 1 when player 2 plays Right? What is the payoff he gets? 2. What is the best response of player 2 when player 1 plays Middle? What is the payoff he gets? 3. Apply iterated elimination of dominated strategies to this game. 4. What are the strategy profiles (tuples of strategies) that remain? What are the corresponding payoffs for both players? End Exercise 3 By completing the above exercises, you should now: • be able to identify a player’s best reponse and dominated strategies. • know how to iteratively remove dominated strategies. 4 Pure Nash Equilibrium 4.1 Review Probably the most important definition in game theory: 1. In a two-player game, with players 1 and 2, strategies (x, y) are a Nash equilibrium (NE) 0 0 0 0 if π1(x, y) ≥ π1(x , y) for all x 6= x, and if π2(x, y) ≥ π2(x, y ) for all y 6= y. In other words, (x, y) is a NE if x is a best response to y and if y is a best response to x. 5 2. We denote a NE where the players are only allowed to play pure strategies (i.e. they have to choose a single strategy from the finite set of strategies) as pure Nash equilibrium. Pn 3. Mixed Strategies: Let x = (x1, .., xn), xi ≥ 0, i=1 xi = 1 denote a mixed strategy of row player; the probability xi denotes the probability with which the row player plays each strategy i ∈ {1, ..., n} 4. A Nash equilibrium where one or more players play a mixed strategy is called a mixed Nash equilibrium. 4.2 Practice Exercise 4 1. Find all pure Nash equilibria of the Prioner’s Dilemma game. 2. In the previous exercise we solved the following game Player 2 Left Middle Right Up (3,0) (0,1) (0,2) Player 1 Middle (0,0) (2,2) (1,-1) Down (0,0) (3,1) (0,0) Find all pure NE’s of this game. 3. Remember the TF-Student game: Students attend section don’t attend section much effort (10,10) (-10,0) TF little effort (5,5) (-5,0) no effort (-5,-5) (0,0) Find all pure NE’s of this game. What do you think will happen if this game will be played? End Exercise 4 6 By completing the above exercises, you should now: • understand when a Nash equilibrium exists and how to identify it. • understand the difference between pure and mixed strategies. 5 MiniMax Technique for finding Mixed Nash Equilibria in Zero- Sum Games 5.1 Mixed Nash Equilibria Pure Nash equilibria can be found by simply analyzing the payoff table and checking whether a pair of strategies constitutes best responses to each other. Finding mixed strategy Nash equilibria is somewhat harder. We will restrict ourselves to mixed Nash equilibria for two-player zero-sum games here, because only for them can we use particularly nice techniques.