Public Goods Games on Adaptive Coevolutionary Networks Elgar Pichler1, A) and Avi M

Public Goods Games on Adaptive Coevolutionary Networks Elgar Pichler1, a) and Avi M. Shapiro2, b) 1)Department of Chemistry and Chemical Biology, Northeastern University, Boston, MA 02115 2)Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA 02138 Productive societies feature high levels of cooperation and strong connections between individuals. Public Goods Games (PGGs) are frequently used to study the development of social connections and cooperative behavior in model societies. In such games, contributions to the public good are made only by cooperators, while all players, including defectors, can reap public goods benefits. Classic results of game theory show that mutual defection, as opposed to cooperation, is the Nash Equilibrium of PGGs in well-mixed populations, where each player interacts with all others. In this paper, we explore the coevolutionary dynamics of a low information public goods game on a network without spatial constraints in which players adapt to their environment in order to increase individual payoffs. Players adapt by changing their strategies, either to cooperate or to defect, and by altering their social connections. We find that even if players do not know other players’ strategies and connectivity, cooperation can arise and persist despite large short-term fluctuations. Keywords: cooperation, complex networks, evolutionary game theory, public goods game

Evolutionary models of cooperation have been ac- tial for the overall functioning of a society. Successful so- tively studied by biologists, physicists, economists cieties tend to have high levels of cooperation and social and mathematicians. Early game theoretic mod- cohesion16,29 while the opposite is true of less functional els involving two players, such as the Prisoner’s societies.16 It has been shown that the interaction pattern Dilemma game, have been generalized to multi- between individuals or the topological structure of social player games where players, or agents, interact groups plays a significant role in the evolution of cooper- with all others as cooperators or defectors. It- ative behavior, although conclusions vary. Whether net- erative games allow players to change strategies, work heterogeneity favors cooperation is unclear. For often with knowledge of neighbor strategies and prisoner dilemma games, regular graphs support coop- payoffs. Subsequently, spatial structure has been eration the most while scale-free graphs do not support incorporated into multiplayer games by arranging cooperation at all,13 however it is hypothesized that this players on simple lattices defining local interac- may depend on the specific rules of games. Indeed, scale- tion partners. Since social connections in applica- free graphs are shown to support cooperation in similar tions are often not limited by space, complex net- games.23–25 In most studies, including these, the network works have become the arena for numerous games structure is static throughout the evolutionary game, un- investigating the emergence and stability of coop- like in real societies. eration. Games on networks with diverse topolo- To study the emergence of cooperation in a more gen- gies have illuminated the importance of network eral and realistic setting, we consider a game theoretic structure on dynamics, yet most models assume a model of a group of interacting individuals, a repeated static network topology. We study Public Goods public goods game (PGG) involving mobile players with Games on evolving complex networks to model two different behavioral strategies: cooperation and de- the realistic setting where social structure varies fection. A PGG generalizes the Prisoner’s Dilemma to along with individual strategy changes. We find more than two players and allows for greater diversity in parameter regimes where cooperation is favored system evolution.2,3,9,25,27 and players become highly connected, all without In a PGG, cooperators contribute a certain amount knowing the strategies of neighboring players. to the public good and defectors do not, yet all players receive an equal share of the sum of contributions, mul- arXiv:1609.05542v1 [physics.soc-ph] 18 Sep 2016 tiplied by a synergy factor. If every player cooperates, I. INTRODUCTION then everyone benefits, and if every player defects, then no one benefits. Clearly, a single player is better off when many or all other game participants cooperate. However, The emergence of social cohesion, cooperation and con- an individual who defects still receives the group bene- nectivity between individuals, are general features essen- fits without any cost. This so-called free-rider problem,10 leads to a situation in which no one cooperates. Still, cooperation persists in real societies. a)Electronic mail: [email protected] In a well-mixed population, where every player in- b)Author to whom correspondence should be addressed. Electronic teracts with every other player, the Nash equilibrium mail: [email protected] of PGGs is total defection, leading to the tragedy of 2 the commons,10 in which public resources are exhausted. induce widespread cooperation such as those mentioned Spatially restricted versions of PGGs have been stud- in SectionI. ied, typically on simple lattices with nearest neighbor Cooperators earn a lower payoff than defectors since 22 interactions, in order to demonstrate how cooperation Equations (1) clearly imply πC < πD. If all n players are can flourish in more complicated yet realistic systems. defectors, their payoffs are zero. If one player is instead Motivated by applications to peer-to-peer, mobile, and a cooperator, its payoff decreases and becomes negative vehicular networks, we generalize such studies by allow- if r < n. Hence, an individual defector has no incen- ing varying connections between arbitrary players on ar- tive to unilaterally switch to become a cooperator. The bitrary nodes in a non-planar network topology. Allowing Nash Equilibrium in a PGG is thus total defection. In for the network topology to coevolve with player strategy repeated games, as we consider here, players may change has been studied using the Prisoner’s Dilemma game5,7,30 strategies after each round based on their payoffs. In and various other symmetric two-player games.20,21 a well-mixed population with homogeneous social struc- Models of strategy and network evolution often follow a ture, when players seek to maximize their payoffs, the random satisfying dynamic7,26 reminiscent of biological dynamics lead to total defection, as in the single round (genetic) evolution, or an imitation dynamic14,30 remi- game. niscent of social evolution. In both cases, a player’s de- To investigate a mechanism which promotes coopera- cisions to change strategies or connections is dependent tion, one may relax the well-mixed assumption and in- on knowledge of neighbors’ strategies and payoffs, a high clude a spatial interaction structure so that individuals information requirement which often does lead to high only play with a subset of the population. Spatial games levels of cooperation. However, even among knowledge- often take place on simple lattices.17 To generalize such able humans, experiments have shown individuals often local interactions, this paper focuses on a PGG on a dy- do not always imitate the best neighbor strategy.28 In namic, complex network described in SectionIIB. order to create a minimal model, we consider a low information setting in which player decisions depend only on individual payoffs. Despite lacking mechanisms such as B. PGG on a complex network punishment6,11 or reputation effects,6 which are known to promote cooperation, we observe both high levels of On a network or graph consisting of nodes and edges, cooperation and connectivity for a range of model pa- a separate game is played in a neighborhood centered on rameters while the adaptive network develops a highly each player. A neighborhood is composed of a central peaked degree distribution. player and all neighbors, the players connected to the central player by a single edge. A player’s total payoff is accumulated from each game played. So a highly con- II. PUBLIC GOODS GAME nected player can earn payoffs from many games. For example, in Figure1, a player at node 1 has three neighbors and thus collects a total payoff from four separate A. PGG in well-mixed populations games (only two are shown). A player without any connected neighbors does not play any games. In a PGG, cooperators contribute a certain amount to the public good and defectors do not. All players then receive an equal share of the sum of all contributions multiplied by a synergy factor, r 1, accounting for non-zero-sum benefits of cooperation.≥ Given n mutu- PGG ally interacting players including nC cooperators, we can 2 2 write the payoffs for cooperators and defectors,

nC nC 1 πC = rc c, πD = rc , (1) n − n PGG where c is the fixed cost of cooperation per game. In com- 1 mon formulations, b = rc is the benefit of cooperation so that synergy, r = b/c, is the benefit-cost ratio.13,16,18,19 Without loss of generality, we take c = 1. Other variations of PGGs have a fixed cost per individual, with FIG. 1. Example network of public goods games an equal fraction contributed to each game the individual plays. This variation is only quantitatively distinct In this network setting, each player can be connected in the heterogeneous populations discussed below in Sec- to any other, so the number of players in a single game is tionIIB, yet is often more supportive of cooperation than only bounded by the total network size. In contrast, on the one considered here.25,27 Our modeling decisions have a 2D regular square lattice, players are only connected to been guided by the avoidance of common mechanisms to nearest neighbors (either at most 4 for a von Neumann 3 or 8 for a Moore neighborhood22). Similarly, location where H is the Heaviside step function. The quantity (and thus, neighbor) changes for a player are usually lim- 1 µ describes a fractional memory length. Our choice, ited to spaces within a certain distance in such cellular µ−= 0.01, implies a long but finite memory. The noise automata. With such restrictions, an unsatisfied player term ηi(t) is drawn from a Gaussian distribution with may not be able to change its location or connections to mean zero and standard deviation 0.1. other players sufficiently to reflect its level of dissatisfac- tion. Players make separate decisions about strategy and B. System Evolution connectivity and each decision creates its own dilemma. Profit-maximizing players should all choose the Nash At each time step, an unsatisfied player (si < 0) is equilibrium strategy of defection (if r < n), which leads allowed to make any or all of three updates: change to zero payoff for all players, clearly the worst possible strategy, add an edge to a random player (excluding self- state in terms of total wealth. This is the classic social edges and multiple edges), or remove an edge to a ran- dilemma which also exists in the well-mixed case. On a dom neighbor. The outcomes of each possible update are network, it is also advantageous to participate in a large based on three independent trials with probability number of games with many cooperators. However, this runs the risk of being exploited by many defectors. This pi(t) = tanh( si /k)H( si). (6) is the secondary dilemma due to the irregular distribu- | | − tion of edges and node strategies throughout the dynamic The scaling factor, k = 8, as in Ref. 22, and ensures a network. In SectionIII, we introduce repeated PGG up- range of probabilities so that very unsatisfied players are date and decision rules which overcome these dilemmas. more likely to change their state than slightly unsatisfied players. The Heaviside function ensures that players with non-negative satisfaction do not change their strategies or III. COEVOLUTIONARY DYNAMICS neighbors. While this probability function is continuous in satisfaction, it changes quickly as satisfaction crosses The following dynamical rules closely follow Roca et zero and is thus very sensitive. A player’s satisfaction al.22 The behavior of the players is driven by the individ- and change probability depend strongly on the current ual player’s satisfaction. Based on the level of satisfac- network topology. To allow each player to change neigh- tion, a player may change strategy and/or connectivity bors and/or strategy based on the same network state, to other players as detailed in the following sections. updating is performed synchronously.

A. State variables IV. RESULTS

At time step t, player i may change strategy and/or A. Initial Conditions neighbors (network plasticity) with independent probabilities based on its satisfaction, To test whether cooperation can develop in a non- cooperative environment, we assume all players are ini- si(t) = πi(t) ai(t) + ηi(t), (2) tially defectors and allow cooperation to emerge stochas- − tically due to the noise in Equation (2). Our main results where πi is the net payoff from all games played by player are simulated using an initial network with no edges. Re- i, ηi is a Gaussian noise, and ai is aspiration, defined markably, a variety of other initial networks, including small world and scale-free networks, result in very simi- ai(t) = α πi,max(t) + (1 α) πi,min(t) lar long time behavior. − = πi,min(t) + α πi,max(t) πi,min(t) , (3) While we have conducted simulations using various − network sizes and parameters, we present results on which depends on greediness 0 α 1, another impor- PGGs consisting of 1000 players and focus on the effects tant control parameter. Aspiration≤ is≤ a convex combina- of the two parameters synergy, r, and greediness, α on tion of functions πi,max/min which reflect (finite) memory cooperation and network structure. of past max/min payoffs. A habituation effect is modeled by defining the update equations B. PGG Dynamics/Evolution πi,max(t + 1) = πi(t) + (1 µ)(πi,max(t) πi(t)) − − H πi, (t) πi(t) , 1. Single simulation max − (4) The model evolution is captured by the variables co- πi,min(t + 1) = πi(t) (1 µ)(πi(t) πi,min(t)) − − − (5) operation, the cooperator fraction, instability, the frac- H πi(t) πi, (t) , − min tion of players changing strategies or neighbors, and ag- 4 glomeration, the average node degree.1 Time series for cooperator fraction these variables are shown in Figure2, where r = 4 and 1.0 no time averaging α = 0.65. To observe overall trends through the stochas- 0.5 ticity, time series show data averaged over a moving window of 100 time steps. 0.0

1.0 moving average 0.5 over 100 steps 0.0 1.0 30 0 10000 20000 cooperation t instability

FIG. 3. Large amplitude ﬂuctations in cooperation time se- 0.5 agglomeration 15 ries from simulation shown in Figure2. (upper) Short but large variations in cooperator fraction are present in the raw instability

cooperation data. (lower) Averaging over a moving window of 100 time agglomeration steps reveals overall trend while smoothing high frequency oscillations. 0.0 0 0 10000 20000 oscillatory cooperator time series while the lower is av- t eraged over a sliding window of 100 time steps, as in Figure2. The amplitude of oscillations in the raw data FIG. 2. Typical simulation run that favors cooperation. The reveal a high level of strategy synchronization between network has 1000 players, initially all defectors with zero players even though players make independent decisions. edges. Game parameters are synergy, r = 4, and greediness, These ﬂuctuations and resulting synchronization are due α = 0.65. Time series are averaged over a moving window of to the habituation eﬀect in satisfaction modeled in equa- 100 time steps. tions (2), (3), (4), and (5). The evolution of satisfaction is shown in Figure4 without window averaging. Due

20 We observe several characteristic periods of behavior. Initially, many edges are added and sustained, increasing 0 average node degree rapidly from zero while there is a short lived spike in cooperation. Second, agglomeration 20 increases slowly while cooperation remains low but with − growing ﬂuctuation. Third, cooperation jumps up and

agglomeration follows. Lastly, cooperation remains high average satisfaction 40 while agglomeration grows slowly. − 0 10000 20000 t Stochasticity in the system arises from the aspiration noise in Equation (2) and causes large amplitude cooperation ﬂuctuations shown in Figure3 and AppendixA. FIG. 4. Network averaged satisfaction time series. The pos- The upper plot in Figure3 shows the original highly itive spikes and subsequent exponential decay cause the oscillations in cooperation in Figure3. Time series is from the same simulation as used in Figures2 and3 and is shown without moving window averaging.

to Equations (3), (4), and (5), unless recent payoffs are higher than πi,max, satisfaction will decrease toward zero. At late times, after the onset of high cooperation, we do observe positive average satisfaction decaying to zero, then a large number of strategy changes toward defection dropping satisfaction far below zero. This induces strate- 1 While the clustering coefficient might be a more appropriate mea- gies to revert back toward cooperation and satisfaction sure on a network, results are qualitatively similar to those of the resets at a positive value. simpler measure average degree. PGG evolution only occurs when satisfaction decreases 5 below zero, i.e. when aspiration nears payoff. This is an 2. Multiple simulations imposed feature of the model, described by the equations in SectionIII, yet, as we have seen, has interesting con- sequences. For long times, when high cooperation and Several simulations for each parameter pair reveal con- agglomeration have been reached, the network is still sub- sistently similar behavior, although shifted in time; see ject to intermittent cooperation crashes which occur with Figure5. This is one example of the system’s strong some regularity. The time scale of such crashes should depend on µ and α. Here we make that dependency ex- cooperation plicit under some mild assumptions and show that higher 1.0 µ and α increase the frequency of cooperation collapses. 0.5 Consider a satisfied player i whose neighbors and their 0.0 strategies are not changing at a given time step t0. Then, for some time later, the player’s payoff is constant, instability πi(t) πi(t0) for t0 t t1. Assume also the strict 1.0 ≈ ≤ ≤ bounds πi,min(t) < πi(t) < πi,max(t). The relaxation of the maximum and minimum payoffs toward the current 0.5 (constant) payoff occurs at the same rate as the relax- 0.0 ation of satisfaction to the size of the noise. Given the agglomeration stated assumptions on payoff, it is shown in AppendixB 30 that 15

µ(t t0) si(t) Ki(α)e− − + ηi(t), t t t , (7) 0 ≈ 0 ≤ ≤ 1 0 10000 20000 t where FIG. 5. Stochasticity in onset of cooperation. Multiple simu- Ki(α) = [πi(t0) πi,min(t0)] lations for the parameters in Figure2 show similar behavior − (8) but shifted in time due to the stochasticity of the system. α [πi, (t ) πi, (t )] − max 0 − min 0 is a linearly decreasing function of α. sensitivity to small perturbations due to random edge attachment and the sharp difference in the probability of The behavior of satisfaction shown in Equation7 leads state changes for satisfactions near zero. to several important conclusions regarding the effects of the memory and greediness parameters. Recall, the faster Despite strong time dependence of cooperation, long satisfaction decays to zero, the sooner a collapse in coop- time averages show, similarly to the 2D lattice setting in eration occurs. For short memory (high µ), satisfaction Ref. 22, the existence of parameter regions which result decreases toward zero faster, increasing the frequency of in both high and low cooperativity after sufficient rounds cooperation collapse. of PGGs. However, Figure6 shows that the transition between these regions is sharper in the network setting. If µ is fixed, then what controls the duration of a period of quasi-stability in cooperation, as seen in the upper plot This phase diagram shows greediness encourages co- in Figure3, is the initial amplitude of the satisfaction de- operation, especially for low synergy where the cost to cay, Ki(α). A lower amplitude reduces the time required cooperate is highest. When synergy is at least 10, high for a player to become dissatisfied and shortens the inter- cooperation results for all levels of greediness. Remark- val between huge collapses in cooperation, making such ably, given Equations (1), the highest cooperation levels collapses more frequent. Three factors contribute to a do not occur at highest synergy. Low instability through- lower amplitude. Clearly, and most importantly, the ex- out most of phase space shows the cooperation levels are pression for Ki(α) shows the amplitude decreases linearly nearly stationary at the end of simulations. with greediness. Additionally, if the payoff is close to its minimum or if the difference between the maximum and minimum payoffs is large, the amplitude is also di- minished. These variations and the random noise are responsible for the irregular timing of cooperation collapses C. Network Topology even for a fixed α and µ. Notice that synergy, r, does not appear explicitly in In order to study the sharp divide between cooperator Equation (7), but actually scales all of the payoffs in and defector dominated regions in phase space seen in Ki(α) due to Equation (1). In other words, higher syn- Figure6, we investigate the long time network degree ergy should increase satisfaction, as expected. distribution and time dependent edge placement. 6

cooperation instability agglomeration 1.0

0.8 0.8

0.6 0.6 α 0.4 0.4

0.2 0.2

0.0 0.0 5 10 15 20 5 10 15 20 5 10 15 20 r r r

FIG. 6. Phase diagrams depicting the fraction of cooperators, fraction of strategy and/or edge changers, and scaled average node degree (agglomeration) of PGGs as functions of synergy, r, and greediness, α. Values are computed for PGGs with 1000 players averaged over the last 100 of 20,000 time steps.

1. Degree distribution over time. In particular, we would like to determine whether cooperators are located at nonrandom nodes in The network topology evolves during the simulation the graph and how their statistics diﬀer from defectors. and a player can have between 0 and n 1 edges. As We begin by considering the dynamics of edges be- players add connections to others, their node− degree in- ing added to each type of player for the case r = 4 and creases. The long term degree distribution is surprisingly α = 0.65, parameters near the sharp transition in Fig- narrow and peaked for parameters which exhibit high co- ure6. The edge dynamics in Figure8 show growth in operation, such as for r = 4, α = 0.65 shown in Figure7. edges involving cooperators but not to the extent that Note the absence of high degree nodes or hubs. This dis- cooperators are accumulating edges preferentially. While tribution is suggestive of an Erd˝os-R´enyi random graph.8 In fact, this PGG could be used as a network growth 12000 mechanism. C-C edges 10000 C-D edges 100 D-D edges 8000 r = 3, α = 0.65 r = 4, α = 0.65

1 6000 10− edges 4000

degree 2 10− 2000 distribution 0 3 0 5000 10000 15000 20000 10− 100 101 102 t degree FIG. 8. Stacked bar plot for edges between two cooperators (C-C), a cooperator and a defector (C-D), and two defectors FIG. 7. Degree distribution for a 1000 node network at t = (D-D). Note the resemblance of the total edges curve to the 20000. The r = 3 (r = 4) distribution corresponds to a low agglomeration plot from the same simulation in Figure2. r = (high) cooperation long term state. 4 and α = 0.65.

the total number of edges increases, the distribution of edge types shows a strong and meaningful relationship 2. Edge dynamics to the number of cooperators, which is also changing. In fact, the number of C-C, C-D, and D-D edges varies with While full network topology as shown by degree distri- cooperation as would be expected if edges were placed bution is clearly quite diﬀerent for parameters exhibiting randomly. high and low cooperation, it is valuable to look closer at First, consider a complete graph KN with N = NC + the connections of cooperators and defectors separately ND nodes of cooperators and defectors. Then we can 7 count the number of edges connecting any two nodes, 104

M, two cooperators, MCC, a cooperator and a defector, slope 2.02 ≈ 3 MCD, and two defectors, MDD, as follows 10 N 1 M = = N(N 1), 102 2 2 − C-C edges 1 NC 1 10 MCC = = NC(NC 1), 2 2 − (9) 0 MCD = NCND = NC(N NC), 10 − 100 101 102 103 ND 1 MDD = = ND(ND 1). 2 2 − cooperators (a) In an Erd˝os-R´enyi random graph with N nodes and edge probability p (0, 1), the edge counts would all be mul- 4 ∈ 10 tiplied by p on average. In other words, the scaling of slope 1.97 edges versus nodes would be the same as in Equations (9). ≈ 103 The log-log plots of PGG graph data in Figure 9a show

MCC and MDD are quadratic in NC and ND, respectively. 102 Likewise, Figure 10 shows MCD is linear in the product

NCND. In the plots, points darken as time advances and a restricted data range is chosen centered at the onset D-D edges 101 of high cooperation, near t = 15000. Indeed, we observe in the PGG, long time edge scaling consistent with an 100 Erd˝os-R´enyi random graph. 100 101 102 103 defectors

3. Cooperator clustering (b) FIG. 9. Edge counts between players of like strategy. The With edges suggesting the network is evolving into a passage of time is indicated by the darkening of points with random graph, it is not clear whether cooperators and data taken from the simulation used in Figure2. The data defectors occupy completely random positions within the range plotted is centered at the onset of high cooperation, network. We would like to determine whether strategy near t = 15000. r = 4 and α = 0.65. statistics for a player’s immediate neighbors differ sig- nificantly from statistics of the whole network. Given a network, we can simply calculate if cooperators have a higher fraction of cooperating neighbors, on average, 104 slope 1.01 (nC 1)/(nC 1+nD) , than the fraction of cooperators ≈ h − − i 3 in the whole network, (NC 1)/(NC 1 + ND). If this is 10 the case, then cooperators− are showing− evidence of clustering. (Note that we subtract one in the above ratios to 102 omit a central cooperator from the counts.) The difference between these two ratios is shown for various greed- C-D edges 101 iness and synergy values in the heatmap in Figure 11. One simulation for each parameter pair has been used 100 rather than an ensemble average. Positive values indi- 0 1 2 3 4 5 6 cate neighborhoods having more cooperators than would 10 10 10 10 10 10 10 be expected if they were uniformly distributed over the cooperators * defectors network. Figure 11 shows that there are parameter regions in FIG. 10. Edge counts between cooperators and defectors. which some local cooperator clustering is present. These Parameters are as in Figure9. regions resemble the regions of high cooperation in the leftmost plot in Figure6. In other words, when cooperation flourishes, cooperators are taking advantage of network reciprocity.13,16,18 However, the magnitude of this effect is very small, since the values in Figure 11 are gen- the values in the heatmap would decrease by approxi- erally very close to zero. To get a sense of scale, imagine mately 0.010, an amount greater than the magnitude of that 1% of the players were to become cooperators; then most of the current values. 8

0.9 0.060 Observing the temporal development of system state 0.8 0.045 variables like payoff, satisfaction, instability, agglomera- 0.7 0.030 tion, and cooperation, we can distinguish several char- 0.6 0.015 acteristic periods; see Figure2. In each of those peri- 0.5 α 0.000 0.4 ods, state variables can fluctuate greatly over short time 0.015 0.3 − spans. In order to get a clearer picture of overall system 0.2 0.030 − trends, we primarily study moving average time series as 0.1 0.045 − in Figure3. 0.0 0.060 Starting from a network without player connections, 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 − consisting only of defectors, we find an initial period of r low cooperation and agglomeration undergoes a phase transition into a late period of higher cooperation and FIG. 11. Deviation of neighborhood cooperator fraction from full network cooperator fraction, h(n − 1)/(n − 1 + n )i − agglomeration for a wide regime of parameters. Both pe- C C D riods begin rapidly and are marked by a high degree of (NC − 1)/(NC − 1 + ND). Parameters are as in Figure9. instability. In the later period, while the moving average number of cooperators is quite large, short-term fluctuations occur where the number drops to nearly zero. This V. SUMMARY indicates strong global synchronization of player strategies made possible through the long-range interactions The behavior of a spatially constrained, low informa- on the network. Temporal variations of state variables tion (no exchange of strategy, position, or connectivity) of this magnitude are nonexistent for the constrained 2D repeated PGG was previously analyzed by Roca et al.22 lattice system. with a cellular automaton model on a 2D square lattice. The decision rule responsible for evolution in the model In such models, players can only interact with their near- is based on player satisfaction. Satisfied players do not est neighbors (8 in the case of a Moore neighborhood on change strategies or connections while unsatisfied play- a planar square lattice). The most important finding was ers may change with nonzero probability. However, an that cooperativity arises and persists in such systems for unsatisfied player may break a connection to a satisfied certain parameter regions. player so being satisfied does not guarantee stasis, even In this paper, we have investigated the behavior of the short-term. The exponential decay of satisfaction due same low information PGG system with spatial restric- to habituation triggers the extreme fluctuations in pay- tions removed using a network model. Here, players in- off and cooperator fraction as well as network rearrange- teract with others without regard to any distance met- ments as recorded by variables instability and agglom- ric. Single game sizes can become much larger and are eration. When the same habituation model is used on only bounded by the total number of players in this spa- a 2D lattice, the effect of decaying satisfaction is signifi- tially unconstrained model. We find that high cooper- cantly more localized due to players being limited to eight ation regimes exist for a wide range of parameters (see neighbors. Figure6) as they did for the constrained system. Despite dramatic temporal dynamics, characteriza- Intuitively, it makes sense that cooperative regions tion of typical long-term states is possible. While a would be more stable and persist in the cellular automa- steady state on an individual player level has not been ton model once they develop, since such clusters are spa- reached by the end of simulations after 20,000 time steps, tially segregated from defectors, but it is surprising that heatmaps of the cooperator fraction and average player cooperation survives even in the spatially unconstrained connections vary only slightly over the last several thou- model given that cooperators are always exposed to ex- sand steps; see Figures6 and 13. ploitation by invading defectors. We describe the PGG by a network model of interact- Our simulations show that cooperative behavior is ing players. This description makes it easy to describe the dominant strategy for certain parameter regions, re- the characteristics of our particular network in a generic gardless of initial conditions. In particular we observe way. In particular, we are able to study the evolution the emergence of cooperation even when we start with of cooperation along with the growth of connections or a highly disadvantageous initial strategy distribution, edges across the network. where all players start out as defectors. We investigate the influence of different initial net- While the behavior of a network PGG shares quali- work topologies on final state topology and cooperation. tative similarities with the cellular automaton PGG, we Specifically, we compare starting from networks with no also observe marked differences: parameter regions for connections to Barábasi-Albert networks generated by cooperativity differ, relaxation times tend to be much preferential attachment.1,4 Initial player connections do longer for the network model, and state variable fluctu- not persist for very long and PGG cooperation dynamics ations (e.g., for instability, agglomeration, and cooper- are fairly universal after a transient period of network re- ation) are much more pronounced in the network PGG configuration. In all cases we find long time development where single game sizes are larger on average. towards random regular networks. 9

The total number of edges grows during the simula- Appendix B: Habituation rate derivation tions even though adding and removing edges at a certain time occurs with equal probability. Evidently, adding In SectionIV, we state that for short periods of time, a edges increases satisfaction more than removing edges. player’s satisfaction decays exponentially at rate µ. Here Indeed, a connection to a defector may actually increase we show that explicitly by deriving Equation (7). As a player’s net payoff if that defector is connected to suf- before, assume πi(t) πi(t ) and πi, (t) < πi(t) < ≈ 0 min ficiently many cooperators. πi,max(t) for t0 t t1. Then, from Equation (4) Edges are chosen for addition or removal at random ≤ ≤ and we observe the accumulation of edges among play- πi,max(t+1) = πi(t) + (1 µ)(πi,max(t) πi(t)) ers of both types tends toward the expectation for the − − πi,max(t+1) πi(t0) given number of cooperators and defectors; See Figures9 − (1 µ)[πi, (t) πi(t )] and 10. This is further illustrated by the full network de- ≈ − max − 0 2 gree distribution which becomes more narrow and peaked (1 µ) [πi,max(t 1) πi(t0)] ≈ − − − as time increases; see Figure7. Cooperation is robust to 3 . (1 µ) [πi,max(t 2) πi(t0)] the random rewiring in the model and does not require ≈. − − − t+1 t0 the presence of high degree hubs to be maintained. No- (1 µ) − [πi,max(t0) πi(t0)] , tably, the long-term network topology is quite homoge- ≈ − − neous with no scale-free statistics, even when initialized where we’ve used πi(t) πi(t0). So, replacing t + 1 by t, as a Barábasi-Albert network. This result is especially we find ≈ surprising in view of the fact that a fixed fully-connected t t0 network, homogeneous in every way, does not support πi,max(t) (1 µ) − [πi,max(t0) πi(t0)] + πi(t0). ≈ − − cooperation, yet our cooperative PGG networks evolve toward such a network. Similarly, by Equation (5) We suggest that our network PGG model is remi- t t0 niscent of and can be used to describe several real life πi,min(t) (1 µ) − [πi,min(t0) πi(t0)] + πi(t0). systems in which public goods are managed and ex- ≈ − − changed like in open peer-to-peer networks, source code We easily compute the aspiration and satisfaction from development groups, wikipedia/collaborative online work Equations (3) and (2), respectively: groups, crowd-source problem solving teams, insurance t t0 ai(t) πi(t ) (1 µ) − [πi(t ) πi, (t )] and loan markets. It is important to notice that those ≈ 0 − − 0 − min 0 t t0 systems might still be very fluid and undergo frequent + α(1 µ) − [πi, (t ) πi, (t )] − max 0 − min 0 and big changes on an individual level even if they oper- t t0 ate in an overall cooperative regime. = πi(t0) (1 µ) − [πi(t0) πi,min(t0)] − − − Future research will explore the influence of more re- + α[πi,max(t0) πi,min(t0)] , − alistic satisfaction measures, like the ones discussed in 12 t t0 Kahneman (prospect theory). It might be possible to si(t) (1 µ) − [πi(t0) πi,min(t0)] ≈ − − decouple single player behavior by allowing for limited α [πi,max(t0) πi,min(t0)] + ηi(t). options of changes at a certain time step (new connec- − − tion formation, or severance of existing connections, or t t0 Since (1 µ) − = exp ((t t0) log(1 µ)), satisfaction strategy changes at a single time step instead of allowing decays exponentially− at a rate− log(1 −µ) µ for 0 < all at the same time). Similarly, asynchronous (vs. syn- µ 1 as given by Equation (7)| in Section− |IV ≈. chronous) updating might alleviate some synchronization effects and should be investigated. Appendix C: Cooperation phase diagrams over time

Appendix A: Time series oscillations The phase diagram in Figure6 reveals the states of systems at t = 20000. Since simulations for various parameters reach their characteristic long term states at diﬀerent Cooperation, instability, and agglomeration time series times, it is interesting to observe the phase diagram de- are shown in Figure2 with a moving average over 100 velop over time, as in Figure 13. Note in particular the time steps to illuminate long time trends. In Figure 12, similarity between t = 10000 and t = 20000. we plot the original data revealing the high frequency oscillations discussed in SectionIV and the relationship between cooperation and instability. In Figure 12a, notice that instability involves mainly strategy changes toward cooperation yet in Figure 12b, over a longer time frame, instability coincides with strategy changes as well as increasing agglomeration. 10

30 30 cooperation 1.0 1.0 instability

agglomeration 15 agglomeration 15 0.5 0.5 instability instability cooperation cooperation

agglomeration cooperation agglomeration instability 0.0 0 0.0 0 14750 14800 14850 14900 14950 15000 16000 17000 18000 19000 20000 t t (a) (b)

FIG. 12. Time series data without windowed averaging. Note the diﬀering time axis scales in 12a and 12b. Parameters are as in Figure2.

1.0 1.0 1.0

0.8 0.8 0.8 0.8 0.8 0.8

0.6 0.6 0.6 0.6 0.6 0.6 α α α 0.4 0.4 0.4 0.4 0.4 0.4

0.2 0.2 0.2 0.2 0.2 0.2

0.0 0.0 0.0 0.0 0.0 0.0 5 10 15 20 5 10 15 20 5 10 15 20 r r r

(a) t = 100 (b) t = 250 (c) t = 500

1.0 1.0 1.0

0.8 0.8 0.8 0.8 0.8 0.8

0.6 0.6 0.6 0.6 0.6 0.6 α α α 0.4 0.4 0.4 0.4 0.4 0.4 0.2 0.2 0.2 0.2 0.2 0.2 0.0 0.0 0.0 0.0 0.0 0.0 5 10 15 20 5 10 15 20 5 10 15 20 r r r (d) t = 1000 (e) t = 10000 (f) t = 20000

FIG. 13. Phase diagram of cooperation at diﬀerent simulation time steps. Parameters are as in Figure6. 11

1R. Albert and A.-L. Barabási.Statistical mechanics of complex 17M. A. Nowak, S. Bonhoeffer, and R. M. May. Spatial games networks. Reviews of Modern Physics, 74:47–97, Jan 2002. and the maintenance of cooperation. Proceedings of the National 2M. Archetti and I. Scheuring. Review: Game theory of public Academy of Sciences, 91(11):4877–4881, 1994. goods in one-shot social dilemmas without assortment. J Theor 18H. Ohtsuki, C. Hauert, E. Lieberman, and M. A. Nowak. A Biol, 299:9–20, Apr 2012. simple rule for the evolution of cooperation on graphs and social 3R. Axelrod and W. D. Hamilton. The evolution of cooperation. networks. Nature, 441(7092):502–505, 05 2006. Science, 211(4489):1390–1396, Mar 1981. 19H. Ohtsuki and M. A. Nowak. The replicator equation on graphs. 4A.-L. Barabásiand R. Albert. Emergence of scaling in random Journal of Theoretical Biology, 243(1):86–97, 2006. networks. Science, 286(5439):509–512, 1999. 20J. M. Pacheco, A. Traulsen, and M. A. Nowak. Active linking in 5C. Biely, K. Dragosits, and S. Thurner. The prisoner’s dilemma evolutionary games. Journal of Theoretical Biology, 243(3):437 on co-evolving networks under perfect rationality. Physica D: – 443, 2006. Nonlinear Phenomena, 228(1):40–48, 2007. 21J. M. Pacheco, A. Traulsen, and M. A. Nowak. Coevolution 6H. Brandt, C. Hauert, and K. Sigmund. Punishment and repu- of strategy and structure in complex networks with dynamical tation in spatial public goods games. Proceedings of the Royal linking. Phys Rev Lett, 97(25):258103, Dec 2006. Society of London B: Biological Sciences, 270(1519):1099–1104, 22C. P. Roca and D. Helbing. Emergence of social cohesion in a 2003. model society of greedy, mobile individuals. Proc Natl Acad Sci 7H. Ebel and S. Bornholdt. Evolutionary games and the emer- USA, 108(28):11370–11374, Jul 2011. gence of complex networks. eprint arXiv:cond-mat/0211666, 23F. C. Santos, J. M. Pacheco, and T. Lenaerts. Evolutionary (CAU-THP-02-28), November 2002. dynamics of social dilemmas in structured heterogeneous popu- 8P. Erd˝osand A. Rényi. On the evolution of random graphs. Publ. lations. Proc Natl Acad Sci USA, 103(9):3490–3494, Feb 2006. Math. Inst. Hung. Acad. Sci, 5(17-61):43, 1960. 24F. C. Santos, J. F. Rodrigues, and J. M. Pacheco. Graph topology 9J. Gomez-Gardenes, M. Romance, R. Criado, D. Vilone, and plays a determinant role in the evolution of cooperation. Proc R A. Sanchez. Evolutionary games defined at the network Soc B, 273(1582):51–55, January 2006. mesoscale: The public goods game. Chaos: An Interdisciplinary 25F. C. Santos, M. D. Santos, and J. M. Pacheco. Social diversity Journal of Nonlinear Science, 21(1):016113, 2011. promotes the emergence of cooperation in public goods games. 10G. Hardin. The tragedy of the commons. Science, Nature, 454(7201):213–216, Jul 2008. 162(3859):1243–1248, Dec 1968. 26H. A. Simon. A behavioral model of rational choice. The Quar- 11D. Helbing, A. Szolnoki, M. Perc, and G. Szabó.Punish, but not terly Journal of Economics, pages 99–118, 1955. too hard: how costly punishment spreads in the spatial public 27G. Szabóand G. Fáth.Evolutionary games on graphs. Physics goods game. New Journal of Physics, 12(8):083005, 2010. Reports, 446(4-6):97–216, 2007. 12 D. Kahneman. Thinking, Fast and Slow. Farrar, Strous and 28A. Traulsen, D. Semmann, R. D. Sommerfeld, H.-J. Krambeck, Giroux / New York, 2011. and M. Milinski. Human strategy updating in evolutionary 13 T. Konno. A condition for cooperation in a game on complex games. Proc Natl Acad Sci USA, 107(7):2962–2966, Feb 2010. networks. Journal of Theoretical Biology, 269(1):224 – 233, 2011. 29J. Y. Wakano, M. A. Nowak, and C. Hauert. Spatial dynamics of 14 M. Kuperman and G. Abramson. Small world effect in an epi- ecological public goods. Proc Natl Acad Sci USA, 106(19):7910– demiological model. Physical Review Letters, 86(13):2909–2912, 7914, May 2009. March 2001. 30M. G. Zimmermann, V. M. Egu´ıluz,and M. San Miguel. Coevo- 15 While the clustering coefficient might be a more appropriate mea- lution of dynamical states and interactions in dynamic networks. sure on a network, results are qualitatively similar to those of the Physical Review E, 69(6):065102, 2004. simpler measure average degree. 16M. A. Nowak. Five rules for the evolution of cooperation. Science, 314(5805):1560–1563, Dec 2006.