Evolutionary games on graphs Gy¨orgy Szab´o a G´abor F´ath b aResearch Institute for Technical Physics and Materials Science, P.O. Box 49, H-1525 Budapest, Hungary bResearch Institute for Solid State Physics and Optics, P.O. Box 49, H-1525 Budapest, Hungary Abstract Game theory is one of the key paradigms behind many scientific disciplines from biology to behavioral sciences to economics. In its evolutionary form and especially when the interacting agents are linked in a specific social network the underlying solution concepts and methods are very similar to those applied in non-equilibrium statistical physics. This review gives a tutorial-type overview of the field for physicists. The first four sections introduce the necessary background in classical and evolutionary game theory from the basic definitions to the most important results. The fifth section surveys the topological complications implied by non-mean-field-type social network structures in general. The next three sections discuss in detail the dynamic behavior of three prominent classes of models: the Prisoner’s Dilemma, the Rock-Scissors-Paper game, and Competing Associations. The major theme of the review is in what sense and how the graph structure of interactions can modify and enrich the picture of long term behavioral patterns emerging in evolutionary games. Key words: Game theory, graphs, networks, evolution PACS: 02.50.Le, 89.65.-s, 87.23.Kg, 05.65.+b, 87.23.Ge Contents 1 Introduction 2 2 Rational game theory 7 2.1 Games, payoffs, strategies 8 2.2 Nash equilibrium and social dilemmas 9 2.3 Potential and zero-sum games 10 2.4 NEs in two-player matrix games 11 2.5 Multi-player games 14 2.6 Repeated games 14 3 Evolutionary games: population dynamics 18 3.1 Population games 18 3.2 Evolutionary stability 20 3.3 Replicator dynamics 24 3.4 Other game dynamics 30 arXiv:cond-mat/0607344v3 [cond-mat.stat-mech] 24 Sep 2007 4 Evolutionary games: agent-based dynamics 32 4.1 Synchronized update 33 4.2 Random sequential update 34 4.3 Microscopic update rules 35 4.4 From micro to macro dynamics in population games 39 4.5 Potential games and the kinetic Ising model 41 4.6 Stochastic stability 43 Email addresses: [email protected] (Gy¨orgy Szab´o), [email protected] (G´abor F´ath). Preprint submitted to Elsevier 4 February 2008 5 The structure of social graphs 45 5.1 Lattices 46 5.2 Small worlds 47 5.3 Scale-free graphs 49 5.4 Evolving networks 50 6 Prisoner’s Dilemma 51 6.1 Axelrod’s Tournaments 52 6.2 Emergence of cooperation for stochastic reactive strategies 53 6.3 Mean-field solutions 57 6.4 Prisoner’s Dilemma in finite populations 58 6.5 Spatial Prisoner’s Dilemma with synchronized update 59 6.6 Spatial Prisoner’s Dilemma with random sequential update 63 6.7 Two-strategy Prisoner’s Dilemma game on a chain 69 6.8 Prisoner’s Dilemma on social networks 71 6.9 Prisoner’s Dilemma on evolving networks 73 6.10 Spatial Prisoner’s Dilemma with three strategies 76 6.11 Tag-based models 79 7 Rock-Scissors-Paper games 81 7.1 Simulations on the square lattice 82 7.2 Mean-field approximation 84 7.3 Pair approximation and beyond 85 7.4 One-dimensional models 86 7.5 Global oscillations on some structures 88 7.6 Rotating spiral arms 91 7.7 Cyclic dominance with different invasion rates 94 7.8 Cyclic dominance for Q> 3 states 95 8 Competing Associations 100 8.1 A four-species cyclic predator-prey model with local mixing 100 8.2 Defensive alliances 102 8.3 Cyclic dominance between associations in a six-species predator-prey model 103 9 Conclusions and outlook 106 A Games 107 A.1 Battle of the Sexes 107 A.2 Chicken game 108 A.3 Coordination game 108 A.4 Hawk-Dove game 108 A.5 Matching Pennies 109 A.6 Minority game 109 A.7 Prisoner’s Dilemma 109 A.8 Public Good game 110 A.9 Quantum Penny Flip 110 A.10 Rock-Scissors-Paper game 111 A.11 Snowdrift game 111 A.12 Stag Hunt game 111 A.13 Ultimatum game 112 B Strategies 112 B.1 Pavlovian strategies 112 B.2 Tit-for-Tat 112 B.3 Win stay lose shift 113 B.4 Stochastic reactive strategies 113 C Generalized mean-field approximations 113 1. Introduction Game theory is the unifying paradigm behind many scientific disciplines. It is a set of analytical tools and solution concepts, which provide explanatory and predicting power in interactive decision situations, when the aims, goals and preferences of the participating players are potentially in conflict. It has successful applications in such diverse fields as evolutionary biology and psychology, computer science and operations research, political science and military strategy, cultural anthropology, ethics and moral philosophy, and economics. The cohesive force of the theory stems from its formal mathematical structure which allows the 2 practitioners to abstract away the common strategic essence of the actual biological, social or economic situation. Game theory creates a unified framework of abstract models and metaphors, together with a consistent methodology, in which these problems can be recast and analyzed. The appearance of game theory as an accepted physics research agenda is a relatively late event. It required the mutual reinforcing of two important factors: the opening of physics, especially statistical physics, towards new interdisciplinary research directions, and the sufficient maturity of game theory itself in the sense that it had started to tackle into complexity problems, where the competence and background experience of the physics community could become a valuable asset. Two new disciplines, socio- and econophysics were born, and the already existing field of biological physics got a new impetus with the clear mission to utilize the theoretical machinery of physics for making progress in questions whose investigation were traditionally connected to the social sciences, economics, or biology, and were formulated to a large extent using classical and evolutionary game theory (Sigmund, 1993; Ball, 2004; Nowak, 2006a). The purpose of this review is to present the fruits of this interdisciplinary collaboration in one specifically important area, namely in the case when non-cooperative games are played by agents whose connectivity pattern (social network) is characterized by a nontrivial graph structure. The birth of game theory is usually dated to the seminal book of von Neumann and Morgenstern (1944). This book was indeed the first comprehensive treatise with a wide enough scope. Note however, that as for most scientific theories, game theory also had forerunners much earlier on. The French economist Augustin Cournot solved a quantity choice problem under duopoly using some restricted version of the Nash equi- librium concept as early as 1838. His theory was generalized to price rivalry in 1883 by Joseph Bertrand. Cooperative game theory concepts appeared already in 1881 in a work of Ysidro Edgeworth. The concept of a mixed strategy and the minimax solution for two person games were developed originally by Emile Borel. The first theorem of game theory was proven in 1913 by E. Zermelo about the strict determinacy in chess. A particularly detailed account of early (and later) history of game theory is Paul Walker’s chronology of game theory available on the Web (Walker, 1995) or William Poundstone’s book (Poundstone, 1992). You can also consult Gambarelli and Owen (2004). A very important milestone in the theory is John Nash’s invention of a strategic equilibrium concept for non-cooperative games (Nash, 1950). The Nash equilibrium of a game is a profile of strategies such that no player has a unilateral incentive to deviate from this by choosing another strategy. In other words, in a Nash equilibrium the strategies form “best responses” to each other. The Nash equilibrium can be considered as an extension of von Neumann’s minimax solution for non-zero-sum games. Beside defining the equilibrium concept, Nash also gave a proof of its existence under rather general assumptions. The Nash equilibrium, and its later refinements, constitute the “solution” of the game, i.e., our best prediction for the outcome in the given non-cooperative decision situation. One of the most intriguing aspects of the Nash equilibrium is that it is not necessarily efficient in terms of the aggregate social welfare. There are many model situations, like the Prisoners Dilemma or the Tragedy of the Commons, where the Nash equilibrium could be obviously amended by a central planner. Without such supreme control, however, such efficient outcomes are made unstable by the individual incentives of the players. The only stable solution is the Nash equilibrium which is inefficient. One of the most important tasks of game theory is to provide guidelines (normative insight) how to resolve such social dilemmas, and provide an explanation how microscopic (individual) agent-agent interactions without a central planner may still generate a (spontaneous) aggregate cooperation towards a more efficient outcome in many real-life situations. Classical (rational) game theory is based upon a number of severe assumptions about the structure of a game. Some of these assumptions were systematically released during the history of the theory in order to push further its limits. Game theory assumes that agents (players) have well defined and consistent goals and preferences which can be described by a utility function. The utility is the measure of satisfaction the player derives from a certain outcome of the game, and the player’s goal is to maximize her utility. Maximization (or minimization) principles abound in science. It is, however, worth enlightening a very important point here: the maximization problem of game theory differs from that of physics.