MASSACHUSETTS INSTITUTE Exchangeable Equilibria OF TECHNOLOGY by JUN 17 2011 Noah D. Stein LIBRARIES B.S., Electrical & Computer Engineering, Cornell University, 2005 S.M., Electrical Engineering & Computer Science, MIT, 2007 ARCHIVES Submitted to the Department of Electrical Engineering & Computer Science in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Electrical Engineering & Computer Science at the Massachusetts Institute of Technology June 2011 @ 2011 Massachusetts Institute of Technology. All rights reserved. Signature of Author: Department of Electrical Engineering & Computer Science May 20, 2011 Certified by: Asuman Ozdaglar Associate Professor of Electrical Engineering & Computer Science Class of 1943 Career Development Professor Thesis Co-SuDervisor Certified by: Pablo A. Parrilo Professor of Electrical Engineering & Computer Science Fireccanica Career Development Professor Thesis Co-Supervisor Accepted by: 2 - A 'ie A. Kolodziejski Professor of Electrical Engineering Chair, Committee for Graduate Students Exchangeable Equilibria by Noah D. Stein Submitted to the Department of Electrical Engineering and Computer Science on May 20, 2011 in partial fulfillment of the requirements for the degree of Doctor of Philosophy Abstract The main contribution of this thesis is a new solution concept for symmetric games (of complete information in strategic form), the exchangeable equilibrium. This is an intermediate notion between symmetric Nash and symmetric correlated equi- librium. While a variety of weaker solution concepts than correlated equilibrium and a variety of refinements of Nash equilibrium are known, there is little previous work on "interpolating" between Nash and correlated equilibrium. Several game-theoretic interpretations suggest that exchangeable equilibria are natural objects to study. Moreover, these show that the notion of symmetric correlated equilibrium is too weak and exchangeable equilibrium is a more natural analog of correlated equilibrium for symmetric games. The geometric properties of exchangeable equilibria are a mix of those of Nash and correlated equilibria. The set of exchangeable equilibria is convex, compact, and semi-algebraic, but not necessarily a polytope. A variety of examples illustrate how it relates to the Nash and correlated equilibria. The same ideas which lead to the notion of exchangeable equilibria can be used to construct tighter convex relaxations of the symmetric Nash equilibria as well as convex relaxations of the set of all Nash equilibria in asymmetric games. These have similar mathematical properties to the exchangeable equilibria. An example game reveals an algebraic obstruction to computing exact ex- changeable equilibria, but these can be approximated to any degree of accuracy in polynomial time. On the other hand, optimizing a linear function over the exchangeable equilibria is NP-hard. There are practical linear and semidefinite programming heuristics for both problems. A secondary contribution of this thesis is the computation of extreme points of the set of correlated equilibria in a simple family of games. These examples illus- trate that in finite games there can be factorially many more extreme correlated equilibria than extreme Nash equilibria, so enumerating extreme correlated equi- libria is not an effective method for enumerating extreme Nash equilibria. In the case of games with a continuum of strategies and polynomial utilities, the exam- ples illustrate that while the set of Nash equilibria has a known finite-dimensional description in terms of moments, the set of correlated equilibria admits no such finite-dimensional characterization. Thesis Co-Supervisor: Asuman Ozdaglar Title: Associate Professor of Electrical Engineering & Computer Science Class of 1943 Career Development Professor Thesis Co-Supervisor: Pablo A. Parrilo Title: Professor of Electrical Engineering & Computer Science Finmeccanica Career Development Professor Dedicated to Marianne Cavanaugh, who once said she would be slightly disappointed if I never wrote this. Thank you for everything. Contents Acknowledgments 11 1 Introduction 15 1.1 Overview. ........................... 18 1.2 Previous work ................ .. .. .. 21 1.2.1 Leading towards exchangeable equilibria ..... 21 1.2.2 Literature related to extreme correlated equilibria 23 2 Background 27 2.1 Game theory ....... ............................. 27 2.1.1 Games and equilibria . ....... ........ ..... 28 2.1.2 The Hart-Schmeidler argument ..... ...... .... 35 2.1.3 Groups acting on games .... ..... ...... .... 37 2.2 Exchangeability . ............ ............ ... 42 2.3 Tensors ....................... ......... 47 2.4 Complete positivity ................. ......... 49 2.5 Semidefinite relaxations ....................... 52 2.5.1 Conic programming .... ........... ....... 52 2.5.2 Linear, semidefinite, and completely positive programming 2.5.3 Polynomial nonnegativity and sums of squares ..... .. 2.5.4 Double nonnegativity ........ ............. 3 Symmetric Exchangeable Equilibria 63 3.1 Generalized exchangeable distributions .......... 63 3.2 Definition and properties .................. 65 3.3 Exam ples .......................... 71 3.4 Convex relaxations of Nash equilibria ........... 85 4 Interpretations of Symmetric Exchangeable Equilibria 91 8 CONTENTS 4.1 Hidden variable interpretation .. .. .. .. .. 91 4.2 Unknown opponent interpretation .. .. .. .. 94 4.3 Many player interpretation . .. .. .. .. .. 96 4.4 Sealed envelope implementation . .. .. .. .. 100 5 Higher Order Exchangeable Equilibria 101 5.1 Refinement of the sealed envelope implementation .... 101 5.2 Powers of games . .. .. .. .. .. .. .. .. .. 106 5.3 Order k exchangeable equilibria .. .. .. .. .. .. 111 5.4 Order oo exchangeable equilibria .. .. .. .. .. .. 113 5.5 Nash equilibria from higher order exchangeable equilibria 114 5.5.1 The player-transitive case .. .. .. .. .. .. 114 5.5.2 Arbitrary symmetry groups . .. .. .. .. .. 116 6 Asymmetric Exchangeable Equilibria 119 6.1 Partial exchangeability .. .. .. .. .. .. .. .. 119 6.2 Defining asymmetric exchangeable equilibria .. .. .. 123 6.3 Convex relaxations of Nash equilibria . .. .. .. .. 132 7 Computation of Symmetric Exchangeable Equilibria 137 7.1 Computational complexity .. .. .. .. .. .. .. .. .. .. 137 7.1.1 Background .. .. .. .. .. ... .. .. .. ...... 138 7.1.2 The Ellipsoid Against Hope algorithm .. .. .. .. .. 140 7.1.3 Paradox . .. ... .. .. .. .. .. ... .. .... .. 142 7.1.4 Resolution .. ... .. .. .. .. .. ... .. .. .. .. 142 7.1.5 Approximate Ellipsoid Against Hope . .. .. ...... 146 7.1.6 Run time .. ... .. .. .. .. .. ... .. .. .... 150 7.1.7 Hardness of optimizing over exchangeable equilibria 150 7.2 Linear and semidefinite relaxations . .. .. .. .. ... 151 8 Structure of Extreme Correlated Equilibria 155 8.1 Background . .. .. .. .. .. .. .. .. .... .... .... 156 8.1.1 Extreme equilibria in finite games . .... .... .... 156 8.1.2 Ergodic theory . .. .. .. .. .. .. ... ... .. .. 159 8.2 Description of the examples . .. .. .. .. ... .. ... .. 160 8.3 Extreme Nash equilibria . .. .. .. ... ... .. ... 161 8.4 Extreme correlated equilibria . .. .. .. ... ... .... .. 163 9 Future Directions 173 9.1 Open questions .. .. .. .. .. .. .. .. .. .. .. 174 CONTENTS 9 9.1.1 Higher order exchangeable equilibria . ....... .... 174 9.1.2 Finitely-supported correlated equilibria in polynomial games 174 9.1.3 The correlated equilibrium conundrum ....... .... 175 9.1.4 Rational exchangeable equilibria .... ....... .... 176 9.1.5 Symmetric identical interest games ... ........ .. 177 9.1.6 Further computational questions ....... ....... 177 9.1.7 Applications of exchangeable equilibria .. .... .... 178 9.1.8 Structuralist game theory ... ..... ...... .... 178 References 183 Notation 189 Acknowledgements My deepest thanks go to my thesis advisors, Asuman Ozdaglar and Pablo Parrilo. Each has provided a constant source of ideas and research directions to pursue. Through some sort of resonance, constructive interference, or positive feedback, the rate at which these suggestions come at least quadruples whenever they are in the same room. At first I found this daunting, but they taught me a great deal quickly and I became comfortable with this process faster than I expected. Knowing how stubborn I can be, they have managed to maintain this endless enthusiasm while almost never pushing me, except on a few occasions when they could sense I really needed it. While Asu and Pablo were always interested in whatever I was working on, one of these occasions came when they were worried that I was not interested enough in my own work. At this point the push came in the form of a no-strings-attached assignment to explore and find a project which I thought was "great," not merely "good enough." In the end this gamble paid off and I am much prouder of the final product than I would have been. This principle of never trying to pigeonhole me into a particular area extended to the other aspects of my graduate studies as well. Though Asu and Pablo suggested enough research-relevant coursework to fill many more Ph.D.'s (to match their endless stream of project ideas), they did not complain when I instead chose to study less-obviously relevant subjects such as algebraic topology. It made my life as a student much simpler and less stressful that this steadfast intellectual support was always backed up financially 100%. I did not once have to worry whether my funding would come through. Their