Intrinsic Robustness of the Price of Anarchy Tim Roughgarden Department of Computer Science Stanford University 353 Serra Mall, Stanford, CA 94305 [email protected] ABSTRACT ble to eliminate in many situations, with large networks be- The price of anarchy, defined as the ratio of the worst-case ing one obvious example. The past ten years have provided objective function value of a Nash equilibrium of a game and an encouraging counterpoint to this widespread equilibrium that of an optimal outcome, quantifies the inefficiency of self- inefficiency: in a number of interesting application domains, ish behavior. Remarkably good bounds on this measure have decentralized optimization by competing individuals prov- been proved in a wide range of application domains. How- ably approximates the optimal outcome. ever, such bounds are meaningful only if a game’s partici- A rigorous guarantee of this type requires a formal behav- pants successfully reach a Nash equilibrium. This drawback ioral model, in order to define“the outcome of self-interested motivates inefficiency bounds that apply more generally to behavior”. The majority of previous research studies pure- weaker notions of equilibria, such as mixed Nash equilibria, strategy Nash equilibria, defined as follows. Each player i correlated equilibria, or to sequences of outcomes generated selects a strategy si from a set Si (like a path in a network). by natural experimentation strategies, such as simultaneous The cost Ci(s) incurred by a player i in a game is a function regret-minimization. of the entire vector s of players’ chosen strategies, which We prove a general and fundamental connection between is called a strategy profile or an outcome. By definition, a the price of anarchy and its seemingly more general relatives. strategy profile s of a game is a pure Nash equilibrium if no First, we identify a“canonical sufficient condition”for an up- player can decrease its cost via a unilateral deviation: ′ per bound on the price of anarchy of pure Nash equilibria, Ci(s) ≤ Ci(si, s−i) (1) which we call a smoothness argument. Second, we prove an ′ “extension theorem”: every bound on the price of anarchy for every i and si ∈ Si, where s−i denotes the strategies that is derived via a smoothness argument extends automat- chosen by the players other than i. These concepts can ically, with no quantitative degradation in the bound, to be defined equally well via payoff-maximization rather than mixed Nash equilibria, correlated equilibria, and the aver- cost-minimization; see also Example 2.5. age objective function value of every no-regret sequence of The price of anarchy (POA) measures the suboptimality joint repeated play. Third, we prove that in routing games, caused by self-interested behavior. Given a game, a notion smoothness arguments are “complete” in a proof-theoretic of an “equilibrium” (such as pure Nash equilibria), and an sense: despite their automatic generality, they are guaran- objective function (such as the sum of players’ costs), the teed to produce an optimal worst-case upper bound on the POA of the game is defined as the ratio between the largest price of anarchy. cost of an equilibrium and the cost of an optimal outcome. An upper bound on the POA has an attractive worst-case flavor: it applies to every possible equilibrium and obviates 1. INTRODUCTION the need to predict a single outcome of selfish behavior. Re- Every student of game theory learns early and often that markably good bounds on the POA have been proved in equilibria are inefficient — self-interested behavior by au- a wide range of models; see [17, Chapters 17–21] and the tonomous decision-makers with competing objectives gener- references therein. ally leads to an outcome that could be improved upon given dictatorial control over everyone’s actions. Such inefficiency 1.1 The Need For More Robust Bounds is ubiquitous in real-world situations and arises for many A good bound on the price of anarchy of a game is not different reasons: congestion externalities, network effects, enough to conclude that self-interested behavior is relatively mis-coordination, and so on. It can also be costly or infeasi- benign. Such a bound is meaningful only if a game’s par- ticipants successfully reach an equilibrium. For pure Nash The original version of this paper was published in the Pro- equilibria, however, there are a number of reasons why this ceedings of the 41st Annual ACM Symposium on Theory of might not occur: perhaps the players fail to coordinate on Computing, May 2009. one of multiple equilibria; or they are playing a game in Permission to make digital or hard copies of all or part of this work for which computing a pure Nash equilibrium is a computation- personal or classroom use is granted without fee provided that copies are ally intractable problem [8]; or, even more fundamentally, a not made or distributed for profit or commercial advantage and that copies game in which pure Nash equilibria do not exist. These cri- bear this notice and the full citation on the first page. To copy otherwise, to tiques motivate worst-case performance bounds that apply republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. to as wide a range of outcomes as possible, and under min- Copyright 2012 ACM 0001-0782/08/0X00 ...$5.00. imal assumptions about how players play and coordinate in degradation in the bound, to all of the more general equilibrium concepts pictured in Figure 1. (C) We prove that routing games, with cost functions re- No Regret CorEq MNE PNE stricted to some arbitrary set, are “tight” in the fol- lowing sense: smoothness arguments, despite their au- (CCE) tomatic generality, are guaranteed to produce optimal worst-case upper bounds on the POA, even for the set of pure Nash equilibria. Thus, in these classes of games, the worst-case POA is the same for each of the equilibrium concepts of Figure 1. Figure 1: Generalizations of pure Nash equilibria. “PNE” stands for pure Nash equilibria; “MNE” for mixed Nash equilibria; “CorEq” for correlated equi- 2. SMOOTH GAMES libria; and “No Regret (CCE)” for coarse correlated equilibria, which are the empirical distributions cor- 2.1 Definitions responding to repeated joint play in which every By a cost-minimization game, we mean a game — play- player has no (external) regret. ers, strategies, and cost functions — together with the joint k cost objective function C(s) = i=1 Ci(s). Essentially, a “smooth game” is a cost-minimization game that admits a POA bound of a canonical type (aP “smoothness argument”). a game. We give the formal definition and then explain how to inter- This paper presents a general theory of “robust” bounds pret it. on the price of anarchy. We focus on the hierarchy of fun- damental equilibrium concepts that is displayed in Figure 1; Definition 2.1 (Smooth Games) A cost-minimization game the full version [22] discusses additional generalizations of is (λ, µ)-smooth if for every two outcomes s and s∗, pure Nash equilibria, including approximate equilibria and k outcome sequences generated by best-response dynamics. ∗ ∗ Ci(s , s−i) ≤ λ · C(s )+ µ · C(s). (2) We formally define the equilibrium concepts of Figure 1 — i i=1 mixed Nash equilibria, correlated equilibria, and coarse cor- X related equilibria — in Section 3.1, but mention next some Roughly, smoothness controls the cost of a set of“one-dimensional of their important properties. perturbations” of an outcome, as a function of both the ini- ∗ Enlarging the set of equilibria weakens the behavioral and tial outcome s and the perturbations s . technical assumptions necessary to justify equilibrium anal- We claim that if a game is (λ, µ)-smooth, with λ> 0 and ysis. First, while there are games with no pure Nash equilib- µ < 1, then each of its pure Nash equilibria s has cost at ∗ ria — “Matching Pennies” being a simple example — every most λ/(1 − µ) times that of an optimal solution s . In (finite) game has at least one mixed Nash equilibrium [16]. proof, we derive As a result, the “non-existence critique” for pure Nash equi- k libria does not apply to any of the more general concepts in C(s) = Ci(s) (3) Figure 1. Second, while computing a mixed Nash equilib- i=1 X rium is in general a computationally intractable problem [4, k ∗ 7], computing a correlated equilibrium is not (see, e.g., [17, ≤ Ci(si , s−i) (4) Chapter 2]). Thus, the “intractability critique” for pure and i=1 X mixed Nash equilibria does not apply to the two largest sets ≤ λ · C(s∗)+ µ · C(s), (5) of Figure 1. More importantly, these two sets are “easily learnable”: when a game is played repeatedly over time, where (3) follows from the definition of the objective func- there are natural classes of learning dynamics — processes tion; inequality (4) follows from the Nash equilibrium condi- by which a player chooses its strategy for the next time step, tion (1), applied once to each player i with the hypothetical ∗ as a function only of its own payoffs and the history of play deviation si ; and inequality (5) follows from the defining — that are guaranteed to converge quickly to these sets of condition (2) of a smooth game. Rearranging terms yields equilibria (see [17, Chapter 4]). the claimed bound. Definition 2.1 is sufficient for the last line of this three-line 1.2 Overview proof (3)–(5), but it requires more than what is needed: it Our contributions can be divided into three parts.