Equilibria, Fixed Points, and Computational Complexity – Nevanlinna Prize Lecture
Total Page：16
File Type：pdf, Size：1020Kb
P. I. C. M. – 2018 Rio de Janeiro, Vol. 1 (147–210) EQUILIBRIA, FIXED POINTS, AND COMPUTATIONAL COMPLEXITY – NEVANLINNA PRIZE LECTURE C D Abstract The concept of equilibrium, in its various forms, has played a central role in the development of Game Theory and Economics. The mathematical properties and computational complexity of equilibria are also intimately related to mathemat- ical programming, online learning, and fixed point theory. More recently, equi- librium computation has been proposed as a means to learn generative models of high-dimensional distributions. In this paper, we review fundamental results on minimax equilibrium and its re- lationship to mathematical programming and online learning. We then turn to Nash equilibrium, reviewing some of our work on its computational intractability. This intractability is of an unusual kind. While computing Nash equilibrium does belong to the well-known complexity class NP, Nash’s theorem—that Nash equilibrium is guaranteed to exist—makes it unlikely that the problem is NP-complete. We show instead that it is as hard computationally as computing Brouwer fixed points, in a precise technical sense, giving rise to the complexity class PPAD, a subclass of total search problems in NP that we will define. The intractability of Nash equilibrium makes it unlikely to always arise in practice, so we study special circumstances where it becomes tractable. We also discuss modern applications of equilibrium computation, presenting recent progress and several open problems in the train- ing of Generative Adversarial Networks. Finally, we take a broader look at the complexity of total search problems in NP, discussing their intimate relationship to fundamental problems in a range of fields, including Combinatorics, Discrete and Continuous Optimization, Social Choice Theory, Economics, and Cryptography. We overview recent work and present a host of open problems. 1 Introduction: 90 years of the minimax theorem Humans strategize when making decisions. Sometimes, they optimize their decisions against a stationary, stochastically changing, or completely unpredictable environment. Optimizing a decision in such a situation falls in the realm of Optimization Theory. Other times, a decision is pit against those made by others who are also thinking strate- gically. Understanding the outcome of the interaction among strategic decision-makers falls in the realm of Game Theory, which studies situations of conflict, competition or cooperation as these may arise in everyday life, game-playing, politics, international MSC2010: primary 68Q17; secondary 05C60. 147 148 CONSTANTINOS DASKALAKIS relations, and many other multi-agent settings, such as interactions among species (Os- borne and Rubinstein [1994]). A founding stone in the development of Game Theory was von Neumann’s minimax theorem (von Neumann [1928]), whose statement is as simple as it is remarkable: if X Rn and Y Rm are compact and convex subsets of Euclidean space, and f (x; y) is a continuous function that is convex in x X and concave in y Y, then 2 2 (1) min max f (x; y) = max min f (x; y): x X y Y y Y x X 2 2 2 2 To see how (1) relates to Game Theory, let us imagine two players, “Min” and “Max,” engaging in a strategic interaction. Min can choose any strategy in X, Max can choose any strategy in Y, and f (x; y) is the payment of Min to Max when they choose strate- gies x and y respectively. Such interaction is called a two-player zero-sum game, as the sum of players’ payoffs is zero for any choice of strategies. Let us now offer two interpretations of (1): • First, it confirms an unusual fact. For some scalar value v, if for all strategies y of Max, there exists a strategy x of Min such that f (x; y) v, that is Min pays Ä no more than v to Max, then there exists a universal strategy x for Min such that for all strategies y of Max it holds that f (x; y) v. Ä • In particular, if x is the optimal solution to the left hand side of (1), y is the optimal solution to the right hand side of (1), and v is the optimal value of both sides, then if Min plays x and Max plays y then Min pays v to Max and neither Min can decrease this payment nor can Max increase this payment by unilaterally changing their strategies. Such a pair of strategies is called a minimax equilibrium, and von Neumann’s theorem confirms that it always exists! One of the simplest two-player zero-sum games that were ever played is rock-paper- scissors, which every kid must have played in the school yard, or at least they used to before electronic games took over. In this game, both players have the same three actions, “rock,” “paper,” and “scissors,” to choose from. Their actions are compared, and each player is rewarded or charged, or, if they choose the same action, the players tie. Figure 1 assigns numerical payoffs to the different outcomes of the game. In this tabular representation, the rows represent the actions available to Min, the columns represent the actions available to Max, and every square of the table is the payment of Min to Max for a pair of actions. For example, if Min chooses “rock” and Max chooses “scissors,” then Min wins a dollar and Max loses a dollar, so the payment of Min to Max is 1, as the corresponding square of the table confirms. To map rock- paper-scissors to the setting of the minimax theorem, let us take both X and Y be the simplex of distributions over rock; paper; scissors , and let Z be the matrix of Figure 1 f g so that given a pair of distributions x X and y Y, the expected payment of Min 2 2 to Max is f (x; y) = xTZy. The minimax equilibrium of the game thus defined is 1 1 1 (x; y), where x = y = ( 3 ; 3 ; 3 ), which confirms the experience of anyone who was challenged by a random student in a school yard to play this game. Under the uniform strategy, both players receive an expected payoff of 0, and no player can improve their payoff by unilaterally changing their strategy. As such, it is reasonable to expect a EQUILIBRIA, FIXED POINTS, AND COMPUTATIONAL COMPLEXITY 149 rock paper scissors rock 0 1 1 paper 1 0 1 scissors 1 1 0 Figure 1: Rock–paper–scissors population of students to converge to the uniform strategy. After all, rock-paper-scissors is so symmetric that no other behavior would make sense. The question that arises is how reasonable it is to expect minimax equilibrium to arise in more complex games (X; Y; f ), and how computationally burdensome it is to compute this equilibrium. Indeed, these two questions are intimately related as, if it is too burdensome computationally to compute a minimax equilibrium, then it is also unlikely that strategic agents will be able to compute it. After all, their cognitive abilities are also computationally bounded. Shedding light on these important questions, a range of fundamental results have been shown over the past ninety years establishing an intimate relationship between the minimax theorem, mathematical programming, and online learning. Noting that all results we are about to discuss have straightforward analogs in the general case, we will remain consistent with the historical development of these ideas, discussing these n m results in the special case where X = ∆n R and Y = ∆m R are probability simplices, and f (x; y) = xTAy, where A is some n m real matrix. In this case, (1) becomes: (2) min max xTAy = max min xTAy: x ∆n y ∆m y ∆m x ∆n 2 2 2 2 1.1 Minimax Equilibrium, Linear Programming, and Online Learning. It was not too long after the proof of the minimax theorem that Dantzig met von Neumann to show him the Simplex algorithm for linear programming. According to Dantzig [1982], von Neumann quickly stood up and sketched the mathematical theory of linear pro- gramming duality. It is not hard to see that (2) follows from strong linear programming duality. Indeed, one can write simple linear programs to solve both sides of (2), hence, as we came to know some thirty years after the Simplex algorithm, a minimax equilib- rium can be computed in time polynomial in the description of matrix A (Khachiyan [1979]). Interestingly, it was conjectured by Dantzig [1951] and recently proven by Adler [2013] that the minimax theorem and strong linear programming duality are in some sense equivalent: it is not just true that a minimax equilibrium can be computed by solving linear programs, but that any linear program can be solved by finding an equilibrium! As the connections between the minimax theorem and linear programming were sink- ing in, researchers proposed simple dynamics for solving min-max problems by having the Min and Max players of (2) run simple learning procedures in tandem. An early pro- cedure, proposed by G. W. Brown [1951] and shown to converge by Robinson [1951], 150 CONSTANTINOS DASKALAKIS was fictitious play: in every round of this procedure, each player plays a best response to the history of strategies played by her opponent thus far. This simple procedure con- verges to a minimax equilibrium in the following “average sense:” if (xt ; yt )t is the trajectory of strategies traversed by the Min and Max players using fictitious play, then 1 the average payment t P t f (x ; y ) of Min to Max converges to the optimum of (2) as t , albeit this convergenceÄ may be exponentially slow in the number of actions, ! 1 n + m, available to the players, as shown in recent work with Pan (Daskalakis and Pan [2014]).