GAME THEORY a Game Represents a Competitive Or Conflicting Situation

MATH2901 Operations Research I Game Theory p.1

A game represents a competitive or conflicting situation between two or more players. Each player has a number of choices, called moves (or pure strategies). A player selects his moves without any knowledge of the moves chosen by the other players. The simultaneous choices of all players lead to the respective payoffs of the game. If the sum of the payoffs to all players is zero, then it is called a zero-sum game. A game played by n persons is called an n-person game. We shall consider only two-person zero-sum games, which are also called matrix games because the gain of one player signifies an equal loss to the other that it suffices to express the outcomes (as a result of the selection of moves) in terms of the payoffs to one player (player I) in a matrix.

A strategy for a given player (I or II) is a plan that specifies which of the available choices he should made with what probabilities. Hence a strategy for player I is the P P specification of xi(xi ≥ 0, xi = 1) and for player II, yj(yj ≥ 0, yj = 1). A strategy is generally a mixed strategy to distinguish it from a pure strategy in which xi (or yj) is 1 for some i (or j) and all other xk (or yk) being 0, k 6= i, j. Furthermore, we explicitly adopt the “rationality assumptions” about the players, who are then assumed to be “intelligent” players that act so as to maximize his expected payoff (or minimize his expected loss).

Game theory deals with the determination of the “optimal” strategies for each player. In view of the conflicting nature and lack of information about the specific strategies selected by each player, optimality is based on a rather conservative criterion, namely, each player selects his strategy (mixed or pure) which guarantees a payoff that can never be worsened by the selections of his opponent. This criterion is known as the “minimax criterion” (or “maximin criterion”).

§1. Stable game

Consider the following (two-person zero-sum) game matrix which represents the payoﬀ to player I. II 1 2 3 4 Row minimum 1 8 2 9 5 2 I 2 6 5∗ 7 8 5* 3 7 3 −4 7 -4 Column maximum 8 5∗ 9 8

1 MATH2901 Operations Research I Game Theory p.2

Player I, by playing his ﬁrst (pure) strategy guarantees a gain of at least 2 = Min {8, 2, 9, 5}. Similarly the second strategy guarantees at least 5 = Min {6, 5, 7, 8}, and the third, −4 = Min {7, 3, −4, 7}. Thus the “row minimum” is the value guaranteed I for each pure strategy. Player I, if he selects strategy 2, is maximizing his minimum (guaranteed) gain. Hence this selection is called the maximin strategy and the corresponding gain (= 5) is called the maximin (or lower) value of the game. A completely analogous considera- tion for player II indicates that he will be interested in the “column maximum” and that he seeks to minimize the column minimum by using the minimax strategy leading to a minimax (or upper) value of the game.

The selections made by I and II are based on the so-called maximin (minimax) criterion. The criterion expresses a conservative attitude which guarantees the best of the worst results. For any matrix game, the minimax (upper) value is greater than or equal to the maximin (lower) value. In the case when equality holds, that is, minimax value = maximin value, the corresponding pure strategies are called the “optimal” strategies and the game is said to be a stable game, which possesses a saddle point that equals to this common value called the value of the game. [Exercise: Prove that the saddle point is highest in its column and lowest in its row.]

§2. Unstable game

In general, the value of the game must satisfy the inequality,

maximin value ≤ value of the game ≤ minmax value .

The existance of a saddle point immediately yields the optimal pure strategies for the game. However, there are unstable games where such a saddle point does not exist. Consider the very simple game of coin-matching. Each player selects either a head (H) or a tail (T ). If the outcomes match (i.e. H,H or T,T ), I wins $1 form II, otherwise II wins $1 from I. The matrix of the game is II HT Row minimum

IH +1 −1 −1 Maximum = −1 value T −1 +1 −1

Column maximum +1 +1

Minimax value = +1 MATH2901 Operations Research I Game Theory p.3

Optimal solution to such games requires each player to use a mixture of pure strategies, or mixed strategies. [Exercise: Prove the inequality Maximin ≤ value of game ≤ Minimax.]

§3. Mixed strategies

Each player, instead of selecting pure strategies only, may play all his strategies according to a predetermined set of ratios. Let xi, i = 1, 2, ··· , m and yj, j = 1, 2, ··· , n be the row and column ratios representing the relative frequencies by which I and II, respectively, select their pure strategies. Then

Xm Xn xi ≥ 0 , yj ≥ 0 and xi = yj = 1 . i=1 j=1

II y1 y2 ··· yn

x1 a11 a12 ··· a1n . . . . I . . . .

xm am1 am2 ··· amn

We can think of xi and yj as probabilities (generated by some random mechanism) by which I and II select their ith and jth pure strategies, respectively.

The solution of the mixed strategy problem is based also on the minimax criterion.

The only difference is that I selects the ratios xi (instead of the pure strategies i) which maximize the minimum expected payoff in a column, while II selects the ratios yj (instead of pure strategies j) which minimize the maximum expected payoff in a row. Mathematically, P player I selects xi(xi ≥ 0, xi = 1) which will yield ( ) · Xm Xm Xm ¸ Max Min ai1xi , ai2xi, ··· , ainxn , xi i=1 i=1 i=1 P and player II selects yj(yj ≥ 0, yj = 1) which will yield    · Xn Xn Xn ¸ Min Max a1jyj , a2jyj, ··· , amjyj . yj   j=1 j=1 j=1

These values are referred to as the maximin and minimax expected payoﬀs, respectively. As in the pure strategies case, the relationship,

Maximin expected payoﬀ ≤ Minimax expected payoﬀ , MATH2901 Operations Research I Game Theory p.4

∗ ∗ holds in general. When xi and yi correspond to the optimal solution, the above relation holds in equality sense and the resulting expected values become equal to optimal expected value of the game. (This is known as the Minimax Theorem in Game Theory, which we ∗ ∗ shall see the justiﬁcation using duality of linear programming later.) Now if xi and yj are the optimal solutions, then each payoﬀ element aij will be associated with the probability ∗ ∗ ∗ (xi , yj ). Thus, if v denotes the optimal expected value of the game, then

Xm Xn ∗ ∗ ∗ v = aij xi yj . i=1 j=1

§4. (2 × 2) unstable game

Consider the following (2 × 2) game in which we assume there is no saddle point.

II y1 y2 = 1 − y1

x1 a11 a12 I x2 = 1 − x1 a21 a22

Player I’s expected payoﬀs corresponding to the pure strategies of II are given by

II’s pure strategy I’s expected payoﬀ

1 x1a11 + x2a21 = (a11 − a21)x1 + a21

2 x1a12 + x2a22 = (a12 − a22)x1 + a22

∗ ∗ Since the optimal x1 and x2 have been chosen to make I’s mixture of moves optimal against ∗ any of the two possible II’s moves, the two expected payoﬀs for I must be equal if x1 and ∗ x2 are optimal. Hence we have

∗ ∗ (a11 − a21)x1 + a21 = (a12 − a22)x1 + a22 ,

∗ ∗ ∗ from which x1 (and x2 = 1 − x1) can be determined. A similar analysis for II leads to

∗ ∗ (a11 − a12)y1 + a12 = (a21 − a22)y1 + a22 ,

∗ ∗ ∗ from which y1 (and y2 = 1 − y1 ) can be determined. MATH2901 Operations Research I Game Theory p.5

§5. Graphical solution of (2 × n) and (m × 2) games

Consider a game in which one of the player (say I) has avaiable to him only two pure strategies. This is then a (2 × n) game as follows:

y1 y2 ··· yn

x1 a11 a12 ··· a1n I x2 = 1 − x1 a21 a22 ··· a2n

It is assumed that the game does not have a saddle point. As before, the expected payoﬀs to I corresponding to II’s pure strategies are

1 (a11 − a21)x1 + a21

2 (a12 − a22)x1 + a22 . . . .

n (a1n − a2n)x1 + a2n

According to the minimax criterion, player I should select the value of x1 so as to maximize his minimum expected payoﬀs. This may be done by plotting the straight lines of expected payoﬀs as functions of x1. Typically, we have

Each line is numbered according to its corresponding II’s pure strategy. The lower envelope of these lines give the minimum expected payoﬀ as a function of x1:

Lower envelope (x1) = Min[(a11 − a21)x1 + a21, ··· , (a1n − a2n)x1 + a2n] . MATH2901 Operations Research I Game Theory p.6

The highest point in this lower envelope then gives the maximum of the minimum expected ∗ payoﬀ and hence the optimal value of x1(= x1), with optimal value of the game:

∗ ∗ The optimal yj for II can be obtained by observing that yj have been chosen to make II’s mixture of moves optimal against any of the possible strategies of I. Hence Xn ∗ ∗© ∗ ª v = yj (a1j − a2j)x1 + a2j . j=1

We claim that all lines {(a1j − a2j)x1 + a2j} that do not pass through the maximin point ∗ P ∗ ∗ must have their corresponding yj = 0. (Why? Hint: yj = 1, yj ≥ 0.) Because the maximin point is determined by the intersection of two straight lines (if more than two, can be any two with opposite slopes), we have the important result that any (2 × n) game is basically equivalent to a (2 × 2) game because only two of the n moves are “effective”. For notational simplicity, assume the first two pure strategies of II are effective. Now, the expected payoffs (loss) to II corresponding to I’s pure strategies are

1 (a11 − a12)y1 + a12

2 (a21 − a22)y1 + a22

Typically, the situation is as follows:

Of course, algebraically, this intersection can also be obtained by solving the two linear equations for y1. The (m × 2) games are treated similarly as in the (2 × n) games except that II’s ∗ optimal strategies yj , j = 1, 2, are first determined using the minimax criterion. This MATH2901 Operations Research I Game Theory p.7 automatically determines the two effective strategies for I, from which the application of ∗ maximin criterion determines the optimal xi , i = 1, 2, ··· , m. (Note the close connection of having only two (positive) effective strategies with the number of positive entries for a BFS in an linear program.) [Exercise: Give the (similar) analysis of an (m × 2) game.]

§6. Numerical example

∗ ∗ [Exercise: Verify that the solutions to the following (2 × 4) game, are: x1 = x2 = 1/2; ∗ ∗ ∗ ∗ ∗ y1 = y4 = 0, y2 = y3 = 1/2; v = 5/2.] II 1 2 3 4 1 2 2 3 −1 I 2 4 3 2 6

Remark: It is sometimes possible to reduce the size of the game matrix by using the “dominance property”. This occurs (as in the numerical example above) when one or more of the pure strategies of either player can be deleted because they are inferior to at least one of the remaining, or to a weighted average of two or more other pure strategies.

[Exercise: Verify that in the above, strategy 1 of II is dominated by strategy 2; and that in the following game matrix, the third strategy of I is dominated by a weighted average of I’s ﬁrst and second pure strategies. What are the weights?]

II 1 2 3 1 5 0 2 I 2 −1 8 6 3 1 2 3

Upon deletion of I’s pure strategy 3 (domainated), we have reduced the game to a (2 × 3), or eﬀectively a (2 × 2) game. [Exercise: Solve this game by the graphical procedure.] MATH2901 Operations Research I Game Theory p.8

§7. (m × n) games and linear programming

For any (m × n) game, player I selects his optimal mixed strategies which yield ( ) · Xm Xm ¸ Max Min ai1xi, ··· , ainxi xi i=1 i=1

Pm subject to the constraints xi = 1, xi ≥ 0, i = 1, 2, ··· , m. This problem can be put i=1 into a linear program as follows: £ P P ¤ Let v = Min ai1xi, ··· , ainxi , then we have i i

Max v Xm subject to aijxi ≥ v , j = 1, 2, ··· , n i=1 Xm xi = 1 i=1 xi ≥ 0 , i = 1, 2, ··· , m .

Clearly, v represents the value of the game in this case. Dividing all constraints by v. [Exercise: What modiﬁcation to the original game matrix may be necessary and legitimate to ensure v > 0?] Deﬁning Xi = xi/v, i = 1, 2, ··· , m, we get, since max v yields the same Pm Pm solutions xi as min 1/v, Min 1/v = Min xi/v = Min Xi. Finally, we have i=1 i=1

Xm Min X0 = Xi i=1 Xm subject to aijXi ≥ 1 i=1 xi ≥ 0, i = 1, 2, ··· , m .

∗ ∗ After the optimal Xi are obtianed using the simplex method (say), the original xi easily follow.

Play II’s problem, on the other hand, is given by    · Xn Xn ¸ Min Max aij yj, ··· , amjyj yj   j=1 j=1

Pn subject to the constraints yj = 1 , yj ≥ 0 , j = 1, 2, ··· , n. Putting in a linear j=1 MATH2901 Operations Research I Game Theory p.9 programming form gives [Exercise: Verify!], Xn Max Y0 = Yj j=1 Xn subject to aij Yj ≤ 1 j=1

Yj ≥ 0, i = 1, 2, ··· , n . where Y0 ≡ 1/v, Yj ≡ yj/v, j = 1, 2, ··· , n. Notice that I’s and II’s problems together is actually a dual pair! Duality theorem then leads to an easy conclusion that the optimal solution of one problem will automatically yield the optimal solution to the other one, and Minimax expected payoff = Maximin expected payoff. (In fact, Dantzig states that when non Neumann, father of game theory, was first introduced to the simplex method of linear programming in 1947 immediately recognized this relationship and further pinpointed and stressed the concept of duality in linear programming!)