American Economic Association
Uncoupled Dynamics Do Not Lead to Nash Equilibrium Author(s): Sergiu Hart and Andreu Mas-Colell Source: The American Economic Review, Vol. 93, No. 5 (Dec., 2003), pp. 1830-1836 Published by: American Economic Association Stable URL: http://www.jstor.org/stable/3132156 . Accessed: 13/01/2011 08:10
Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at . http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content in the JSTOR archive only for your personal, non-commercial use.
Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at . http://www.jstor.org/action/showPublisher?publisherCode=aea. .
Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed page of such transmission.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected].
American Economic Association is collaborating with JSTOR to digitize, preserve and extend access to The American Economic Review.
http://www.jstor.org UncoupledDynamics Do Not Lead to Nash Equilibrium
By SERGIUHART AND ANDREU MAS-COLELL*
It is notoriously difficult to formulate sensi- to an "informational"requirement-that the dy- ble adaptive dynamics that guarantee conver- namics be uncoupled. gence to Nash equilibrium. In fact, short of variants of exhaustive search (deterministicor I. The Model stochastic), there are no general results; of course, there are many important, interesting The setting is that of games in strategic (or and well-studied particularcases. See the books normal)form. Such a game F is given by a finite of J6rgen W. Weibull (1995), FernandoVega- set of players N, and, for each player i E N, a Redondo (1996), Larry Samuelson (1997), strategy set S' (not necessarily finite) and a Drew Fudenbergand David K. Levine (1998), payoff function' u1:HEN Sj -> R. Josef Hofbauer and Karl Sigmund (1998), H. We examine differential dynamical systems Peyton Young (1998), and the discussion in defined on a convex domain X, which will be Section IV below. either HieN Si or2 IieN A(S'), and are of the Here we provide a simple answerto the ques- form tion: Why is that so? Our answer is that the lack of a generalresult is an intrinsicconsequence of )(t) = F(x(t); r), the naturalrequirement that dynamics of play be "uncoupled"among the players, that is, the ad- or x = F(x; F) for short. We also write this as justment of a player's strategy does not depend xi = Fi(x; F) for all i, where x = (xi)iEN and3 on the payoff functions (or utility functions) of F = (Fi)iEN. the other players (it may depend on the other From now on we keep N and (S1)ieN fixed, players' strategies, as well as on the payoff and identify a game F with its N-tuple of payoff function of the player himself). This is a basic functions (ui)ieN, and a family of games with a informational condition for dynamics of the set 'U of such N-tuples; the dynamics are thus "adaptive"or "behavioral"type. It is importantto emphasize that, unlike the (1) xi = F'(x; (uJ)jEN) for all i E N. existing literature(see Section IV), we make no "rationality"assumptions: our dynamics are not We consider families of games U where every best-reply dynamics, or better-reply,or payoff- game F E U has a single Nash equilibrium improving, or monotonic, and so on. What we x(r). Such families are the most likely to allow show is that the impossibility result is due only for well-behaved dynamics. For example, the dynamic x = x(r) - x will guaranteeconver- gence to the Nash equilibriumstarting from any * Hart: Center for Rationality and InteractiveDecision Theory, Department of Mathematics, and Department of Economics, The Hebrew University of Jerusalem,Feldman Building, Givat-Ram, 91904 Jerusalem, Israel (e-mail: URL: [email protected]; (http://www.ma.huji.ac.il/'hart)); 1 Mas-Colell: of Economics and Business, Uni- R denotesthe realline. Department 2 versitat Pompeu Fabra, Ramon Trias Fargas 25-27, 08005 We write A(A) for the set of probabilitymeasures Barcelona, Spain (e-mail: [email protected]). The researchis overA. partially supported by grants of the Israel Academy of 3 Fora well-studiedexample (see for instanceHofbauer Sciences and Humanities, the Spanish Ministry of Educa- andSigmund, 1998), consider the classof "fictitiousplay"- tion, the Generalitatde Catalunya, and the EU-TMR Re- like dynamics:the strategyq'(t) playedby i at time t is Robert J. Yaacov some sort of to the of the other search Network. We thank Aumann, "goodreply" past play - Vincent Josef Piero La i.e., to thetime average xJ(t) of qj(T)for T t; then Bergman, Crawford, Hofbauer, playersj, - Mura, Eric Maskin, Motty Perry, Alexander Vasin, Bob (afterrescaling the time axis) x1 = q' x- Gi(x;F) - x' Wilson, and the referees for their useful comments. Fi(x; F).
1830 VOL. 93 NO. 5 HARTAND MAS-COLELL:UNCOUPLED DYNAMICS 1831 initial condition.4Note, however, that in this dy- THEOREM 1: Let U be a family of games namicX dependson jx(F),which, in turn,depends containing a neighborhood of the game Fo. on all the componentsof the game F, in particular Then every uncoupled dynamic for U is not on uJ forj 0 i. This motivatesour next definition. Nash-convergent. We call a dynamical system F(x; F) (defined for r in a family of games U) uncoupled if, for Thus an arbitrarilysmall neighborhoodof Fo every player i E N, the function F' does not is sufficient for the impossibility result (of depend on uJ for ]j i; i.e., course, nonexistence for a family U implies nonexistence for any larger family U' D U). (2) x' = F'(x; ui) for all i E N II. An Examplewith a Continuumof Strategies [comparewith (1)]. Thus the change in playeri's strategycan be a functionof the currentN-tuple of Take N = {1, 2) and S1 = S2 = D, where strategiesx and i's payoff function u' only.5 In D := {z = (z1, Z2)E R2 : \z11? 1} is the unit other words, if the payoff functionof player i is disk. Let ) : D -> D be a continuous function identicalin two games in the family, then at each that satisfies: x his strategyx' will change in the same way.6 If, given a family U with the single-Nash- * )(z) = 2z for z in a neighborhoodof 0; and equilibriumproperty, the dynamical system al- * +(+(z)) : z for all z : 0. ways converges to the unique Nash equilibrium of the for r E u1-i.e., if Such a function exists; for game any game clearly - instance, F(x(F); F) = 0 and limt,,x(t) = x(F) for any let us put )(z) = 2z for all lIzll 13, define 4 solution x(t) (with any initial condition)-then on the circle lizil= 1 to be a rotation by, say, we will call F a Nash-convergent dynamic for r/4, and interpolate linearly on rays between U. To facilitate the analysis, we always restrict |lzl = 1/3and ||Izl = 1. ourselves to C1 functions F with the additional Define the game Fo with payoff functions uo propertythat at the (unique) rest point x(F) the and u2 given by8 Jacobian matrix J of F( ; F) is hyperbolic and (asymptotically) stable-i.e., all eigenvalues of uo(xi, xi) := -|' - (i)ll2 for all x, xi E D. J have negative real parts. We will show that: Fo has a unique Nash equilibrium9x = (0, 0). We embed Fo in the family Uo consisting of There exist no uncoupled dynamics which all games F (ul, u2) where, for each i = 1, 2, guarantee Nash convergence. we have ui(xi, x) = -x - (xJ)ll2, with : D -> D a continuousfunction, such that the Indeed,in the next two sections we presenttwo equatione((j(x')) = xi has a unique solution xi. simplefamilies of games(each game having a single Then x = (x1, x2) is the unique Nash equilib- Nash equilibrium),for which uncouplednessand rium of the game'0 F. Nash convergenceare mutuallyincompatible. We will now prove that every uncoupled More precisely, in each of the two cases we dynamic for 'U, is not Nash-convergent. This exhibit a game Fo and show that:7 proof contains the essence of our argument,and
4 8 - The same applies to various generalizedNewton meth- We use j := 3 i throughoutthis section. In the game ods and each i wants to choose xi so as to 5 fixed-point-convergentdynamics. ro, player match as closely It may depend on the function u'(-), not just on the as possible a function of the other player's choice, namely, currentpayoffs u'(x). 4<(x). 6 9 What the other players do (i.e., x-i) is much easier to x is a pure Nash equilibriumif and only if x1 = (r2) observe than why they do it (i.e., their utility functions u-i). and x2 = (1), or xi = 4(4(xi)) for i = 1, 2. There are no 7 = An E-neighborhoodof a game Fr (u)iEN consists of mixed-strategy equilibria since the best reply of i to any all games F = (u')i, satisfying lu'(s) - uo(s)I < e for all mixed strategy of j is always unique and pure. s E i,EN Si and all i E N. 10 Moreover x is a strict equilibrium. 1832 THEAMERICAN ECONOMIC REVIEW DECEMBER2003 the technicalmodifications needed for obtaining Therefore the 4 x 4 Jacobian matrix J of the Theorem 1 are relegatedto the Appendix. Let F system (f, f2) at x = (0, 0) is thus be, by contradiction, a dynamic for the family 'Uo which is uncoupled and Nash-con- r i1 -2/' vergent. The dynamic can thus be written:x' = J=- -22 j2 ] F'(x', xj; u') for i = 1, 2. The following key lemma uses uncoupled- LEMMA 3: If the eigenvalues of J1 and J2 ness repeatedly. have negative real parts, then J has at least one eigenvalue with positive real part. LEMMA 2: Assume that y' is the unique ui- best-reply of i to a given yJ, i.e., ul(y1,yO) > PROOF: ui(x', y) for all x' f y'. Then F'(yi, y; u') = 0, The coefficient a3 of A in the characteristic and the eigenvalues of the 2 X 2 Jacobian polynomial det(J - A) of J equals the negative matrix11J = (F[k(y', y; u')/xl)k,l=1,2 have of the sum of the four 3 X 3 principalminors; negative real parts. a straightforwardcomputation shows that
PROOF: a3 = 3 det(J')trace(J2)+ 3 det(J2)trace(J'). Let rF be the game (u', uj) with u(xi, xi) := -||xI - yj12 (i.e., ~j is the constant function But det(Ji) > 0 and trace(J') < 0 (since the ~J(z) -- y); then (y', yj) is its unique Nash eigenvalues of J' have negative real parts), so equilibrium,and thus Fi(yi, yJ; u') = 0. Apply that a3 < 0. this to playerj, to get Fx(x',yJ; uJ) = 0 for all x' Let A1, A2, A3, A4 be the eigenvalues of J. (since yJ is the unique uJ-best-replyto any x1). Then Hence aF{(x', yi; uJ)lax = 0 for k, I = 1, 2. The 4 X 4 Jacobianmatrix J of F( *, *; 1l) at (y', A1A2A3 + AlA2A4 + A1A3A4 + A2A3A4 y') is thereforeof the form = -a3 > 0 j- i K] J 0 L from which it follows that at least one Ar must have positive real part. O The eigenvalues of J-which all have negative real partsby assumption-consist of the eigen- This shows that the unique Nash equilibrium values of J' together with the eigenvalues of L, x = (0, 0) is unstable for F( ; Fo)-a contra- and the result follows. O diction which establishes our claim. For a suggestive illustration,12see Figure Putf'(x) := F(x; uo); Lemma 2 implies that 1, whichis drawnfor x in a neighborhoodof (0, 0) = the eigenvalues of the 2 X 2 Jacobian matrix where (xi) 2x. In the region 1||1x/2< Ilx1l < J := (Wfk(0, O)/a)k,l=1,2 have negative real 211x211the dynamic leads "away"from (0, 0) (the parts. Again by Lemma 2, f'(4(xJ), xJ) = 0 for arrowsshow that,for x' fixed, the dynamicon x all xj, and thereforein particularf'(2xJ, xJ) = 0 must convergeto x1 = 2xi-see Lemma 2). for all xJ in a neighborhoodof 0. Differentiating and then evaluating at x = (0, 0) gives III. An Examplewith FinitelyMany Strategies
2af (0, o)/Iax + af/(o, o)/ax = o If the games have a finite number of strate- gies (i.e., if the S' are finite), then the state space for all k,1 = 1, 2. for the dynamics is the space of N-tuples of mixed strategies HiEN A(S').
= 12 1 Subscriptsdenote coordinates:xi = (x'i, x2) and F' The actualdynamic is 4-dimensionaland may be quite (F', F2). complex. VOL.93 NO. 5 HARTAND MAS-COLELL:UNCOUPLED DYNAMICS 1833
11x211 Nash-convergentdynamic 'U. Forthe game Fo (o)i= 1,2,3 we denotef(x, x2, x3) := F(x, x2, x3; uo);let J be the 3 X 3 Jacobianmatrix off at xo. For any y' (close to 1/2), the unique equilib- rium of the game F1 = (u2, u2,u3) in L/owith u3 given by a3 y /(1 - ) is (yl, 1/2, 1/2), and so Fl(y1, 1/2, 1/2; F1) = 0. This holds therefore also for Fr since the dynamic is uncoupled: f'(y', 1/2, 1/2) = 0 for all yl close to 1/2. Hence afl(xo)l/x' = 0. The same applies to the other two players, and we conclude that the diagonal-and thus the trace-of the Jacobian matrix J vanishes. Together with hyperbolicity [in fact, det(J) : 0 suffices here], this implies the existence of an eigenvalue with positive real part,'5 thus establishing our contradiction- which proves Theorem 1 in this case. We put on record that the uncoupledness of FIGURE 1. THE DYNAMIC FOR THE GAME Fo OF SECTION II the additional structureon J. AROUND dynamic implies (0, 0) Indeed, we have fl(x', 1/2, x3) = 0 for all x1 and x3 close to 1/2 [since (xl, 1/2, x3) is the Consider a family 'Uo of three-playergames unique Nash equilibrium when a' = 1-as in where each player has two strategies, and the Fo-and a2 = x3/(1 - x3), a3 = xl/(1 - xl)]. payoffs are: Therefore fl(xo)/ x3 = 0 too, and so J is of the form
0, 0, 0 a, 1,0 0,a2, a, , 1 -0 c 0- 1,0, a3 0, , a3 1, a2,0 0,0,0 J= 0 0 d e 0 0 where all the a1are close to 1 (say, 1 - E < a1 < for some reall6 c, d, e. 1 + s for some small s > 0), and, as usual, We conclude by observing that the specific- player 1 chooses the row, player 2 the column ities of the example have played very little role and player 3 the matrix.13 Let Fr be the game in the discussion. In particular,the propertythat with a1 = 1 for all i; this game has been intro- the trace of the Jacobianmatrix is null, or thatfi duced by James Jordan(1993). vanishes over a linear subspaceof co-dimension Denote by xl, x2, x3 E [0, 1] the probability 1, which is determinedfrom the payoff function of the top row, left column, and left matrix, of player i only, will be true for any uncoupled respectively. For every game in the family 'Uo dynamics at the equilibrium of a game with a there is a unique Nash equilibrium:14 x (rI) = completely mixed Nash equilibrium-provided, ai-1/(ai-l + 1). In particular, for Jordan's of course, that the game is embedded in an game xo - x(F) = (1/2, 1/2, 1/2). appropriatefamily of games. Let F be, by way of contradiction,an uncoupled
13 15 Each player i wants to mismatchthe next player i + 1, Indeed: otherwise the real parts of all eigenvalues are regardless of what player i - 1 does. (Of course, i + 1 is 0. The dimensionbeing odd implies thatthere must be a real always taken modulo 3.) eigenvalue. Therefore0 is an eigenvalue-and the determi- 14 In equilibrium:if i plays pure, then i - 1 plays pure, nant vanishes. so all players play pure-but there is no pure equilibrium; 16 If cde : 0 there is an eigenvalue with positive part, - 1 if i plays completely mixed, then a'(1 x+ l) = xi+ and if cde = 0 then 0 is the only eigenvalue. 1834 THE AMERICANECONOMIC REVIEW DECEMBER2003
IV. Discussion Hofbauer(1995), Foster and Young (1998, 2001), and Hofbauer and Sigmund (1998, (a) There exist uncoupled dynamics converg- Theorem 8.6.1). All these dynamics are ing to correlated equilibria17-see Dean necessarily uncoupled (since a player's Foster and Rakesh V. Vohra (1997), Fuden- "good reply" to x(t) depends only on his berg and Levine (1999), Hart and Mas- own payoff function). Ourresult shows that Colell (2000),18 and Hart and Mas-Colell what underlies such impossibility results is (2003). It is thus interestingthat Nash equi- not necessarily the rationality-typeassump- librium, a notion that does not predicate tions on the behavior of the players-but coordinatedbehavior, cannot be guaranteed rather the informational requirement of to be reached in an uncoupled way, while uncoupledness. correlated equilibrium, a notion based on (e) In a two-population evolutionary context, coordination,can.19 AlexanderVasin (1999) shows that dynam- (b) In a general economic equilibrium frame- ics that depend only on the vector of pay- work, the parallel of Nash equilibrium is offs of each pure strategy against the Walrasian (competitive) equilibrium. It is currentstate-a special class of uncoupled again well known that there are no dynam- dynamics-cannot be Nash-convergent. ics that guaranteethe general convergence (f) There exist uncoupled dynamics that are of pricesto equilibriumprices if the dynamic guaranteedto converge to (the set of) Nash has to satisfy natural uncoupledness-like equilibriafor specialfamilies of games, like conditions, for example, the nondepen- two-person zero-sum games, two-person dence of the adjustmentof the price of one potentialgames, dominance-solvablegames, commodityon the conditions of the markets and others;20 for some recent work see for other commodities (see Donald G. Saari Hofbauerand William H. Sandholm(2002) and Carl P. Simon, 1978). and Hart and Mas-Colell (2003). (c) In a mechanism-design framework, the (g) There exist uncoupled dynamics that are counterpartof the uncouplednesscondition most of the time close to Nash equilibria, is Leonid Hurwicz's "privacy-preserving" but are not Nash-convergent (they exit in- or "decentralized"condition-see Hurwicz finitely often any neighborhood of Nash (1986). equilibria);see Foster and Young (2002). (d) There are various results in the literature, (h) Sufficient epistemic conditions for Nash startingwith Lloyd S. Shapley (1964, Sec. equilibrium-see Robert J. Aumann and 5), showing that certainclasses of dynamics Adam Brandenburger(1995, Preliminary cannot be Nash-convergent.These dynam- Observation,p. 1161)-are for each player ics assume that the players adjust to the i to know the N-tuple of strategiesx and his current state x(t) in a way that is, roughly own payoff function ui. But that is precisely speaking, payoff-improving; this includes the informationa player uses in an uncou- fictitious play, best-reply dynamics, better- pled dynamic-which we have shown not reply dynamics, monotonic dynamics, ad- to yield Nash equilibrium.This points out justment dynamics, replicator dynamics, the difference between the static and the and so on; see Vincent P. Crawford(1985), dynamic frameworks:converging to equi- Jordan (1993), Andrea Gaunesdorfer and libriumis a more stringentrequirement than being in equilibrium. (i) By their nature,differential equations allow to be conditioned on the 17 Of course, these dynamics are defined on the appro- strategies only priate state space of joint distributions A(IiN Si), i.e., limited informationof the past capturedby probabilityvectors on N-tuples of (pure) strategies. 18 In fact, the notion of "decoupling"appears in Section 20 4 (i) there. A special family of games may be thoughtof as giving 19 Cum grano salis this may be called the "Coordination informationon the other players' payoff function (e.g., in ConservationLaw": there must be some coordinationeither two-person zero-sum games and potential games, u' of one in the equilibriumconcept or in the dynamic. player determines uj of the other player). VOL.93 NO. 5 HARTAND MAS-COLELL:UNCOUPLED DYNAMICS 1835
the state variable. It may thus be of interest REFERENCES to investigate the topic of this paperin more general settings. Aumann, Robert J. and Brandenburger,Adam. "Epistemic Conditions for Nash Equilib- APPENDIX rium."Econometrica, September 1995, 63(5), pp. 1161-80. We show here how to modify the argumentof Crawford,Vincent P. "LearningBehavior and Section II in order to prove Theorem 1. Con- Mixed-Strategy Nash Equilibrium."Journal sider a family of games U' that is a neighbor- of Economic Behavior and Organization, hood of Fo, and thus is certain to contain only March 1985, 6(1), pp. 69-78. those games in 'U1 that are close to Fr. The Foster, Dean and Vohra, Rakesh V. "Calibrated proof of Lemma 2 uses payoff functions of the Learning and Correlated Equilibrium." form uJ(xi, x) = -Ixj - J112that do not depend Games and Economic Behavior, October on the other player's strategy x' (i.e., e is the 1997, 21(1), pp. 40-55. constant function e(z) y'). Since the proof in Foster, Dean and Young, H. Peyton. "On the Section II needs the result of Lemma 2 only for Non-Convergence of Fictitious Play in y' in a neighborhoodof 0, we will replace the Coordination Games." Games and Eco- above constantfunction e with a functionthat is nomic Behavior, October 1998, 25(1), pp. constant in a neighborhoodof the origin and is 79-96. close to >. . "Onthe Impossibilityof Predictingthe We will thus construct for each a E D with Behavior of Rational Agents." Proceedings Ilall< E a function Ca: D -- D such that: (1) of the National Academyof Sciences, October I||la - (l||1 CE for some constant C > 0; (2) 2001, 98(22), pp. 12848-53. 4la(Z) = a for all l||z|| 2E; (3) )(P(1a(Z)) = Z if . "Learning, Hypothesis Testing, and and only if z = (a) = 2a; and (4) qib(a(Z)) = Nash Equilibrium."Mimeo, Johns Hopkins z if and only if z =,fb(a) = b. The games University, 2002. correspondingto ((, iiy) and to (
Hurwicz,Leonid. "Incentive Aspects of Decentral- Games," in M. Dresher, L. S. Shapley, and ization,"in K. J. Arrowand M. D. Intriligator, A. W. Tucker,eds., Advances in game theory eds., Handbook of mathematicaleconomics, (Annals of MathematicsStudies 52). Prince- Vol. II. Amsterdam:North-Holland, 1986, Ch. ton, NJ: PrincetonUniversity Press, 1964, pp. 28, pp. 1441-82. 1-28. Jordan, James. "Three Problems in Learning Vasin, Alexander."On Stability of Mixed Equi- Mixed-StrategyNash Equilibria."Games and libria."Nonlinear Analysis, December 1999, Economic Behavior, July 1993, 5(3), pp. 38(6), pp. 793-802. 368-86. Vega-Redondo,Fernando. Evolution, games, and Saari, Donald G. and Simon, Carl P. "Effective economic behavior. New York: Oxford Uni- Price Mechanisms."Econometrica, Septem- versity Press, 1996. ber 1978, 46(5), pp. 1097-125. Weibull, Jorgen W. Evolutionary game theory. Samuelson, Larry. Evolutionary games and Cambridge,MA: MIT Press, 1995. equilibriumselection. Cambridge,MA: MIT Young,H. Peyton.Individual strategy and social Press, 1997. structure. Princeton, NJ: Princeton Univer- Shapley,Lloyd S. "Some Topics in Two-Person sity Press, 1998.