The Price of Anarchy Tim Roughgarden UC Berkeley + Stanford University also featuring some slides contributed by Christos Papadimitriou (UC Berkeley) The Price of Anarchy OR, a rendezvouz of computer science and game theory Tim Roughgarden UC Berkeley + Stanford University also featuring some slides contributed by Christos Papadimitriou (UC Berkeley) The Price of Anarchy OR, a rendezvouz of computer science and game theory OR, how free food can lead to a thesis topic Tim Roughgarden UC Berkeley + Stanford University also featuring some slides contributed by Christos Papadimitriou (UC Berkeley) (Very) Brief Overview • basic object of study: noncooperative games • widespread phenomenon: selfish (or “rational”) behavior is inefficient – could improve natural objective functions by dictating behavior •price of anarchy-- “competitive analysis for noncooperative games” – when does noncooperative behavior lead to an approximately optimal outcome? Example: Load-Balancing • setup: 4 jobs self-schedule on 2 machines – each wants a lightly loaded machine • with coordination, can achieve the following “good” outcome (makespan = 5) 1 2 4 3 m1 m2 Example: Load-Balancing • if they cannot be centrally scheduled? Example: Load-Balancing • if they cannot be centrally scheduled? • e.g., the following outcome is “stable” – no job has incentive to switch machines 2 1 4 3 m1 m2 Example: Braess’s Paradox • setup: traffic (e.g., many cars or packets) pick s-t paths to minimize travel time • travel time increases with congestion c(x)=x c(x)=1 s t c(x)=1 c(x)=x Example: Braess’s Paradox • setup: traffic (e.g., many cars or packets) pick s-t paths to minimize travel time • travel time increases with congestion c(x)=x c(x)=1 ½ of traffic s t c(x)=1 c(x)=x ½ of traffic • at “equilibrium”: travel time = 3/2 for all Braess’s Paradox Initial Network: ½ ½ x1 s ½ ½ t 1 x Travel time = 1.5 Braess’s Paradox Initial Network: Augmented Network: ½ ½ ½ ½ x1x1 0 s ½ ½ t s ½ ½ t 1 x 1 x Travel time = 1.5 Now what? Braess’s Paradox Initial Network: Augmented Network: ½ ½ x1 x 1 s 0 t s ½ ½ t 1 x 1 x Travel time = 1.5 Travel time = 2 Braess’s Paradox Initial Network: Augmented Network: ½ ½ x1 x 1 s 0 t s ½ ½ t 1 x 1 x Travel time = 1.5 Travel time = 2 All traffic worse off! [Braess 68] • (with thanks to Leonard Schulman) The Price of Anarchy • both examples: selfish behavior inefficient – e.g., equilibria need not minimize avg/max cost vs. vs. The Price of Anarchy • both examples: selfish behavior inefficient – e.g., equilibria need not minimize avg/max cost vs. vs. • price of anarchy: worst-case ratio between “social cost” of equilibrium and of optimum – w.r.t. a game + a definition of social cost Agenda of Tutorial • small price of anarchy: – selfish behavior benign; little benefit of centralized control over “laissez-faire” • our focus: bounding the price of anarchy--- when, what, how, why? – want techniques that cut across applications Caveats • price of anarchy = active subfield bordering theoretical CS, game theory – we will only scratch the surface – will focus on the two most well-studied models – even for these models, will only discuss results that I can describe simply – see also bibliography to appear on my Web page • will not have many proofs or all the details – 3 hours only seems long to audience! Road Map: First Half • Part I: Basic Notions – games, strategies, Nash equilibria, the price of anarchy, examples next • Part II: The KP Model (load-balancing) – upper and lower bounds on the price of anarchy – extensions and open questions Road Map: Second Half • Part III: Selfish Routing – the potential function technique – tight bounds on the price of anarchy – extensions and open questions • Part IV: Other cool models: a brief tour – resource allocation, submodular, facility location, and network design games Part I: Basic Notions • finite (normal-form) games • strategies • Nash equilibria • nonatomic games + their equilibria – models large population of players •price of anarchy – pessimistic vs. optimistic Games In a (finite, normal-form) game there is: • a finite set of players • for each player, a finite set of strategies • for each strategy profile (each player picks a strategy), a payoff to each player Games In a (finite, normal-form) game there is: • a finite set of players • for each player, a finite set of strategies • for each strategy profile (each player picks a strategy), a payoff to each player Row 12 Column 12 player: 1 -2 -1 player: 1 -2 -1 2 -1 -2 2 -1 -2 Pure vs. Mixed Strategies Pure strategy: player picks single strategy Mixed strategy: player picks probability distribution over its strategies Pure vs. Mixed Strategies Pure strategy: player picks single strategy Mixed strategy: player picks probability distribution over its strategies • allowing randomization has pros and cons – permits universal existence of equilibria – seems to model some real behavior • e.g., bluffing in poker – still, conceptually problematic • how often to people really randomize? Example: Load-Balancing Example: 2 jobs scheduled on 2 machines • players = jobs; strategies = machines • payoff to job = -1 × #jobs on its machine j2 j1 j2 vs. j1 Example: Load-Balancing Example: 2 jobs scheduled on 2 machines • players = jobs; strategies = machines • payoff to job = -1 × #jobs on its machine j2 j1 j2 vs. j1 m1 m2 m1 m2 m1 -2 -1 m1 -2 -1 Job 1: Job 2: m2 -1 -2 m2 -1 -2 Nash Equilibria Defn: a set of mixed strategies (one per player) is a Nash equilibrium if no player has a unilateral incentive to change its strategy. – no player can increase expected payoff w/some other strategy, if strategies of others are fixed Nash Equilibria Defn: a set of mixed strategies (one per player) is a Nash equilibrium if no player has a unilateral incentive to change its strategy. – no player can increase expected payoff w/some other strategy, if strategies of others are fixed Fact 1: pure-strategy Nash eq need not exist. – “matching pennies” Fact 2: mixed-strategy Nash eq always exist. – Nash’s theorem (1950) Example: Load-Balancing Back to: two-machine, two-job example. Two pure-strategy Nash eq, with jobs on different machines. j1 j2 j2 j1 m1 m2 m1 m2 Example: Load-Balancing Back to: two-machine, two-job example. Two pure-strategy Nash eq, with jobs on different machines. j1 j2 j2 j1 m1 m2 m1 m2 One mixed-strategy Nash eq: j1 50% 50% j2 50% 50% m1 m2 Nonatomic Games • nonatomic game = model large population with infinitely many players • sounds scary, but can increase analytical tractability – (continuous math a good thing) • could define a payoff per strategy profile – but we will only care what fraction of population picks each strategy; we will only model this Nonatomic Games (con'd) Ingredients (abstract and in selfish routing): • finite # of player types [(si,ti) pairs] • population sizes [amt of traffic per pair] • finite strategy sets [(si,ti) paths] • for each distribution (fraction of population using each strategy), payoffs corresponding to each strategy – given traffic pattern, cost of each path Equilibria in Nonatomic Games • like Nash eq, though note that individual deviations do not affect payoffs – strategies used by a player type have equal cost x 1 x 1 s 0 t vs. s 0 t 1 x 1 x – cf., load-balancing: Equilibria in Nonatomic Games • like Nash eq, though note that individual deviations do not affect payoffs – strategies used by a player type have equal cost x 1 x 1 s 0 t vs. s 0 t 1 x 1 x – cf., load-balancing: • existence in general: [Schmeidler 73], etc. • WLOG, can focus on pure-strategy eq The Price of Anarchy Defn: price of obj fn value of a Nash eq = sup anarchy optimal obj fn value ⇒ with respect to a game + an objective fn ⇒ supremum is over the set of equilibria ⇒ price of anarchy of a collection C of games = take supremum over games in C Comparison to Other Notions • approximation ratio = quantifies cost of not having exponential computing time • competitive ratio = quantifies cost of not knowing the future •price of anarchy= quantifies cost of not being able to dictate behavior – also called coordination ratio [Koutsoupias/Papadimitriou 99] Optimistic Price of Anarchy • optimistic price of anarchy: compare best Nash equilibrium with the social optimum • interpretation: what is achievable with “single-shot” centralized control – e.g., for network design (later) • can often upper bound the optimistic p.o.a. but not the pessimistic p.o.a. Road Map: First Half • Part I: Basic Notions – games, strategies, Nash equilibria, the price of anarchy, examples next • Part II: The KP Model (load-balancing) – upper and lower bounds on the price of anarchy – extensions and open questions The KP Model: Overview • formal definition of model • special case: uniform machine speeds • non-uniform speeds • extensions + open problems The KP Model •nplayers/jobs, each with weight wj •mstrategies/machines, each with speed si • outcomes: assignment of jobs to machines The KP Model •nplayers/jobs, each with weight wj •mstrategies/machines, each with speed si • outcomes: assignment of jobs to machines • J(i) = jobs on machine i 2 • L(i) = load of i = 1 Σ 1/si j in J(i) wj 4 • [Koutsoupias/ 3 Papadimitriou 99] m1 m2 Costs with Mixed Strategies i • mixed strategy pj = Pr[job j on machine i] • expected load of machine i: i EL(i) = 1/si Σj wj pj • Cost of i from job j’s perspective: i Cj(i) = 1/si [(Σk not j wk pk ) + wj] j1 50% 50% j2 50% 50% m1 m2 What is a Nash equilibrium? i • no job j has incentive to change its pj ‘s • equivalently: for each job j, i pj > 0 ⇒ Cj(i) = min all machines