CHAOS 28, 103105 (2018)
Evolutionary dynamics in the public goods games with switching between punishment and exclusion Linjie Liu,1 Shengxian Wang,1 Xiaojie Chen,1,a) and Matjaž Perc2,b) 1School of Mathematical Sciences, University of Electronic Science and Technology of China, Chengdu 611731, China 2Faculty of Natural Sciences and Mathematics, University of Maribor, Koroška cesta 160, SI-2000 Maribor, Slovenia; Complexity Science Hub Vienna, Josefstädterstraße 39, A-1080 Vienna, Austria (Received 8 August 2018; accepted 23 September 2018; published online 9 October 2018) Pro-social punishment and exclusion are common means to elevate the level of cooperation among unrelated individuals. Indeed, it is worth pointing out that the combined use of these two strategies is quite common across human societies. However, it is still not known how a combined strategy where punishment and exclusion are switched can promote cooperation from the theoretical perspective. In this paper, we thus propose two different switching strategies, namely, peer switching that is based on peer punishment and peer exclusion, and pool switching that is based on pool punishment and pool exclusion. Individuals adopting the switching strategy will punish defectors when their numbers are below a threshold and exclude them otherwise. We study how the two switching strategies influence the evolutionary dynamics in the public goods game. We show that an intermediate value of the threshold leads to a stable coexistence of cooperators, defectors, and players adopting the switching strategy in a well-mixed population, and this regardless of whether the pool-based or the peer-based switching strategy is introduced. Moreover, we show that the pure exclusion strategy alone is able to evoke a limit cycle attractor in the evolutionary dynamics, such that cooperation can coexist with other strategies. Published by AIP Publishing. https://doi.org/10.1063/1.5051422
Large-scale cooperation among unrelated individuals dis- I. INTRODUCTION tinguishes humans markedly from other animals, and it is indeed crucial for our evolutionary success. Coopera- The emergence of altruistic behavior among unrelated tion is remarkable because it is costly for the individual individuals has been a puzzling phenomenon, since such 1Ð12 that cooperates, but it is beneficial for the society as a behavior is usually costly to perform but benefits others. whole. As such, to cooperate is in contradiction with the A number of game theoretic researches over the past Darwinistic principle of maximizing personal fitness, and decades have provided numerous answers to this ques- it is therefore challenged by defection. Individuals are thus tion, such as indirect reciprocity, reputation, reward, and torn between what is best for them and what is best for the punishment.13Ð18,20,21 Among them, scholars pay more atten- society—the blueprint of a social dilemma. The theoretical tion to the role of pro-social punishment played in promoting framework for studying this fascinating aspect of our biol- public cooperation.19,22Ð24 Theoretical and experimental stud- ogy is evolutionary game theory, with the most commonly ies have indicated that pro-social punishment can reduce the used games being the prisoner’s dilemma and the public expected payoff of self-interested individuals since they need goods game. The latter two games describe the essence to pay an extra fine.25Ð27 However, such action is also costly; of the problem succinctly for pairwise and group inter- it results in the second-order free-riders dilemma, in which actions, respectively. The resolution of social dilemmas individuals may prefer to benefit from punishment but do not toward the pro-social outcome has received ample atten- contribute the related costs.28Ð35 tion in the recent past. In our paper, we extend the scope of Social exclusion has recently drawn more attention since the classic public goods game with cooperators and defec- it can perform better than costly punishment for maintain- tors to account for a new third type of strategy, namely, the ing cooperation.36Ð41 It is thought that social exclusion is switching strategy that either punishes or excludes defec- still an advantageous strategy even facing with a large num- tors depending on their numbers. Our research reveals ber of free-riders, due to the fact that excluding defectors fascinatingly different evolutionary dynamics, including from sharing benefit can decrease the number of beneficiaries. the stable coexistence of three strategies and limit cycles, Recently, such exclusion strategy has been studied from an which enables cooperators to survive where otherwise they 36 would perish. These results have important implications evolutionary perspective by Sasaki and Uchida. The results for the better understanding of cooperation, and we also show that social exclusion strategy can overcome two difficul- hope they will be inspiring for economic experiments in ties of costly punishment: first, rare punishers cannot defeat the future. a large area of free-riders; second, punishers will be elim- inated by natural selection in the presence of second-order 40 a)[email protected] free-riders. Subsequently, Liu et al. studied the competi- b)[email protected] tion between pro-social exclusion and punishment in finite
1054-1500/2018/28(10)/103105/13/$30.00 28, 103105-1 Published by AIP Publishing. 103105-2 Liu et al. Chaos 28, 103105 (2018) populations and claimed that social exclusion can always do of punishment strategies can be chosen based on whether better than punishment. the number of defectors in the group exceeds the tolerance It is worth pointing out that in most previous stud- threshold. Concretely, if the number of free-riders in the group ies, the evolution of these two incentive strategies has been exceeds a given threshold T, those players with the switch- explored in a manner in which the two strategies work ing strategy will exclude free-riders, otherwise punish them. independently.36,40 But in the realistic world, what is pretty Thus, the levels of tolerance threshold are determined by the widespread is the combined use of these sanctioning strate- number of defectors in the group, namely, T = 0, ..., N.In gies. It is still unclear how such a combined strategy can particular, T = 0 means that all free-riders will be excluded promote cooperation from the theoretical perspective. Con- from sharing public goods once there exist strategy-switching sidering the different roles of pro-social punishment and players, while T = N means that all defectors will be pun- exclusion played in raising cooperation, it is interesting to ished. Here, we introduce two switching strategies, namely, investigate how to jointly use the two strategies for the pro- peer-based switching and pool-based switching. It is nec- motion of cooperation. Notice that a specific example in essary to point out that regardless of peer-based switching the realistic world is the problem of environmental pollution strategy or pool-based switching strategy, strategy switching control. If the number of enterprises discharging pollutants individuals need to bear a monitoring cost τ for detecting illegally exceeds a given threshold, the Environmental Pro- the number of defectors in the group before they exclude or tection Agency (EPA) will force these enterprises to suspend punish free-riders. operations. Otherwise, the EPA will impose fines on them.42,43 In this paper, we thereby propose a switching strategy 1. Peer-based switching strategy with which individuals can either punish or exclude free-rider in the public goods game (PGG) depending on the number of Here, we consider peer-based switching strategists who defectors in the group. Specifically, if the number of defec- not only contribute to the common pool but also monitor the tors in the group is above a certain threshold, individuals number of free-riders in the group. If the level of defection who contribute to the monitoring organization will behave exceeds a certain level in the group, they become exclud- as excluders; otherwise, they act as punishers. We consider ers who prevent defectors collecting benefit from the public these assumptions and then construct a model based on the goods sharing at a cost cE on every defector in the group, β one in our previous work.44 Furthermore, in our model, we otherwise they change to be punishers who impose a fine γ 45 further consider that strategy-switching individuals need to on each free-rider at a cost for themselves. Then, the pay a monitoring cost, which is ignored in Ref. 44, but it is payoffs of pure cooperators (C), pure defectors (D), and peer- more reasonable for real-world systems. This is because that based switching strategists (IE) obtained from the group can monitoring the whole population and knowing the number of be, respectively, given by free-riders in the population require a certain monitoring cost. ⎧ ⎨ rc − c,ifN = 0andN ≥ T; We find that a middle threshold can make the system con- IE D π = (1) verge to a stable coexistence state of cooperators, defectors, C ⎩ rc(NC + NI + 1) E − c, otherwise, and strategy switching players no matter whether pool-based N switching strategy or peer-based switching strategy is used. In ⎧ addition, we prove that when pure exclusion strategy is con- ⎨ 0, if NI = 0andND ≥ T; sidered into public goods games, the population system can E πD = ( + ) (2) exhibit a limit cycle where cooperative strategy can coexist ⎩ rc NC NIE − NI β, otherwise, with other types of strategies. N E
II. MODEL AND METHOD ⎧ ⎨ rc − c − NDcE − τ,ifND ≥ T; A. Public goods game π = IE ⎩ rc(NC + NI + 1) E − c − N γ − τ, otherwise, We consider an infinite well-mixed of individuals who N D play the public goods game. In a group of N individuals, each (3) player has the opportunity to cooperate by contributing to the where N , N ,andN denote the numbers of cooperators, common pool with a cost c to itself, or act as defectors by con- C D IE defectors, and peer-based switching strategists among the tributing nothing. Then, the sum of all contributions in each other N − 1 players, respectively. group is multiplied by an enhancement factor r (1 < r < N) and equally distributed among all the N individuals. Thus, if all individuals choose to cooperate, the group can yield 2. Pool-based switching strategy − the maximum benefit rc c for each player. If all choose to Different from peer-based switching strategy, pool-based defect, the group can get nothing. switching strategists resort to the institution of monitoring which can choose to exclude defectors or punish them by B. Switching strategy of pro-social punishment and giving a corresponding fine B. The costs of exclusion and pun- exclusion ishment are δ and G, respectively. Accordingly, the payoffs We introduce a switching strategy to depict the combined of cooperators, defectors, and pool-based switching strategists use of pro-social punishment and exclusion. Different forms (IF) obtained from the group can be, respectively, written as 103105-3 Liu et al. Chaos 28, 103105 (2018) follows: average payoff of the whole population. And we have
⎧ − − − = ≥ N−1 N NC 1 ⎨ rc c,ifNIF 0andND T; N − 1 N − NC − 1 Pi = πC = ( + + ) (4) ⎩ rc NC NIF 1 = = NC ND − c, otherwise, NC 0 ND 0 N NC ND N−NC−ND−1 x y z πi,(8) ⎧ where i = C, D,orI. ⎨ 0, if N = 0andN ≥ T; IF D To better characterize the evolutionary dynamics of the πD = ( + ) (5) ⎩ rc NC NIF population for different switching thresholds, we present our − N B, otherwise, N IF results regarding the switching threshold in the form of bifur- cation diagrams in Sec. III. In addition, we provide detailed ⎧ theoretical analysis when the switching threshold is set to N or − − δ − τ ≥ ⎨ rc c ,ifND T; 0. Unless otherwise specified, theoretical analyses in special π = IF ⎩ rc(NC + NI + 1) conditions are presented in Appendixes AÐD, respectively. F − c − G − τ, otherwise, N (6) III. RESULTS A. Evolutionary dynamics in the population with where N , N ,andN denote the numbers of cooperators, C D IF peer-based switching strategy defectors, and pool-based switching strategists among the N − 1 players, respectively. We first present the results of evolutionary dynamics in the population with peer-based switching strategy for different values of T,asshowninFig.1. Clearly, if level of toler- C. Replicator dynamics ance is strong enough (T = N), players will opt for defection In a well-mixed population, the fraction of C, D,and (more detailed theoretical analysis is shown in Appendix A 1). I(IE or IF) players can be denoted by x, y,andz, respectively. Besides, an unstable equilibrium point can appear on edge Thus, x, y, z ≥ 0andx + y + z = 1. Consequently, the strat- IED [see Figs. 1(a) and 1(b)]. It needs to stress that a spe- 46Ð52 egy evolution can be studied by using replicator equations cific stable point appears on the edge IED when we decrease ⎧ threshold value slightly [see Fig. 1(b)]. For an intermediate ⎪ ˙ = ( − ¯ ) = ⎨⎪ x x PC P , threshold (T 3), a new dynamical characteristic appears, ¯ that is, an interior stable equilibrium point displays the sim- y˙ = y(PD − P), (7) ⎪ plex S3. With decreasing T, the interior stable point moves ⎩ ¯ z˙ = z(PI − P), along a straight line approximately parallel to the edge CD [see Figs. 1(c)Ð1(e)]. If T decreases still further to T = 0, where PC, PD,andPI represent the expected payoffs of C, D, peer-based switching strategist will always use the exclu- ¯ and I, respectively, and P = xPC + yPD + zPI represents the sion action. Interestingly, defectors will be excluded by peer
FIG. 1. Effects of peer-based switching strategy on cooperation for different switching thresholds T. The triangles represent the state space S3 ={(x, y, z) : x, y, z ≥ 0, and x + y + z = 1},wherex, y,andz are the frequencies of C, D,andIE, respectively. Filled circles represent stable fixed points, whereas open circles represent unstable fixed points. The threshold values are (a) T = 5, (b) T = 4, (c) T = 3, (d) T = 2, (e) T = 1, and (f) T = 0, respectively. Other parameters: N = 5, r = 3, c = 1, β = γ = 0.4, τ = 0.1, and cE = 0.4. 103105-4 Liu et al. Chaos 28, 103105 (2018)
FIG. 2. Time evolution of the fractions of three strategies C (black dash line), D (red dot line), and IE (blue solid line) for different switching threshold values T. Initial conditions: (x, y, z) = (0.1, 0.8, 0.1). Parameters: N = 5, r = 3, c = 1, β = γ = 0.4, τ = 0.1, and cE = 0.4. The threshold values are (a) T = 5, (b) T = 4, (c) T = 3, (d) T = 2, (e) T = 1, and (f) T = 0, respectively. excluders, defectors dominate cooperators, and cooperators [see Figs. 2(a)Ð2(d)]. However, it is worth noting that when invade peer excluders, forming a heteroclinic cycle on the T = 1, pure cooperators have higher fitness than defectors boundary of simplex S3. In order to judge the stability of this [see Fig. 2(e)]. In particular, a periodic oscillation occurs heteroclinic cycle, we calculate the eigenvalues of the Jaco- when peer exclusion is performed [see Fig. 2(f)], which is bian matrix of the three vertex equilibrium points as follows: corresponding to the stable limit cycle shown in Fig. 1(f). ⎧ ⎪ λ− =−(rc − c − τ), λ+ = τ, ⎪ IE IE ⎨⎪ B. Evolutionary dynamics in the population with λ− =−τ λ+ = − rc C , C c , pool-based switching strategy ⎪ N (9) ⎪ ⎩⎪ − rc + We next illustrate how the introduction of pool-based λ = − c, λ = rc − c − τ − (N − 1)cE. D N D switching strategy influences the cooperation level for dif- − = λ λ− ferent T, as shown in Fig. 3.WhenT N, pool-based λ =− IE λ =− C λ = Then, we define that IE λ+ , C λ+ ,and D switching strategists will always act as pool punishers. In IE C λ− this case, no interior equilibrium point appears in the sim- − D λ = λ λ λ = rc−c−τ > λ+ , and we have IE C D rc−c−τ−(N−1)c 1. < D E plex S3 since PIF PC, while an unstable equilibrium point 53 = N(c+δ+τ)−rc Thus, the heteroclinic cycle is asymptotically stable (see exists on edge IFD with z N(N−1)B (for further details, Appendix B 3 for theoretical analysis). Besides, a stable see Appendix C 1). The system only has one stable point D, limit cycle exists in the interior of the simplex S3 (see which means that defectors will dominate the whole popu- Appendix B 4 for theoretical analysis). Frequencies of three lation. However, this phase portraits will be changed when strategies oscillate, but the interior equilibrium point is unsta- we reduce the tolerance threshold T.WhenT = 4, an inte- ble [see Fig. 1(f), and more detailed theoretical analysis is rior stable equilibrium point is present in the simplex S3; thus, presented in Appendix B 2]. cooperators, defectors, and pool-based switching strategists In order to shed light on the details behind the results pre- can coexist steadily in the population [see Fig. 3(b)]. Fur- sented in Fig. 1, we depict the frequency of the mentioned thermore, with decreasing T, the interior stable equilibrium three strategies as a function of time for different switch- point moves toward full cooperation state [see Fig. 3(b)Ð3(e)]. ing thresholds in Fig. 2. It can be observed that although As a special case of pool-based switching strategy, T = 0 pure defectors can always occupy the highest proportion means that pool-based switching strategists will always act as of the population for T ≥ 2, the evolutionary advantage pool excluders. The unique stable coexistent equilibrium point of defectors is weakened gradually with decreasing T changes to a center surrounding by periodic closed orbits. 103105-5 Liu et al. Chaos 28, 103105 (2018)
FIG. 3. Effects of pool-based switching strategy on cooperation for different switching threshold T. Parameters: N = 5, r = 3, c = 1, B = 0.4, G = δ = 0.4, and τ = 0.1. The threshold values are (a) T = 5, (b) T = 4, (c) T = 3, (d) T = 2, (e) T = 1, and (f) T = 0, respectively. ¯ The reason is that pool excluders invade defectors, cooper- By substituting x = ε(1 − z) and P = x(PC − PD) + ¯ ators invade pool excluders, and defectors invade cooperators (1 − z)(PD − PF) + PF into z˙ = z(PF − P),wehavez˙ = [see Fig. 3(f)]. To analyze the dynamics in the interior of S3, z[x(PD − PC) − (1 − z)(PD − PF)]. Thus, we have we introduce a new variable ε = x , which represents the x+y ⎧ fraction of cooperators among members who do not contribute ( − ) ⎨⎪ N−1 rc N 1 to the exclusion pool. This yields ε˙ =−ε(1 − ε) (1 − z) − rc + c , N ⎩⎪ ε˙ =−ε(1 − ε)(PD − PC). z˙ = z(1 − z)[rc − c − δ − τ − ε(rc − c)].
IF IF
IF
IF IF
IF
FIG. 4. Time evolution of the fractions of three strategies C (black dash line), D (red dot line), and IF (blue solid line) for different switching threshold values T. Initial conditions: (x, y, z) = (0.1, 0.8, 0.1). Parameters: N = 5, r = 3, c = 1, B = 0.4, G = δ = 0.4, τ = 0.1, and cE = 0.4. The threshold values are (a) T = 5, (b) T = 4, (c) T = 3, (d) T = 2, (e) T = 1, and (f) T = 0, respectively. 103105-6 Liu et al. Chaos 28, 103105 (2018)
Through detailed theoretical analysis, we can prove that we investigate the evolutionary dynamics among cooperators, the system is a conservative Hamiltonian system (see defectors, and switching strategists in an infinite population. Appendix D 3 for details). The oscillation is caused by the mutual restriction among To further explain the results presented in Fig. 3,we these three strategists, that is, defectors will be excluded by continue by showing the time evolution of the frequencies peer excluders, defectors defeat cooperators, and cooperators of these three strategies for different values of T in Fig. 4. invade peer excluders. When T = N, defection strategy can be dominant, which is We have also investigated the evolutionary dynamics of irrelevant to the initial state [see Fig. 4(a)]. Furthermore, by pool-based switching strategy and revealed that in addition to decreasing the tolerance level, however, we can observe the the two special cases (T = 0andT = N correspond to the fraction of cooperators increases [see Figs. 4(b)Ð4(e)]. Of par- case of pool exclusion and pool punishment, respectively) ticular note is that pure cooperation strategy can become the mentioned above, the intermediate switching threshold val- most advantageous strategy when T is set to an intermediate ues can guarantee the stable coexistence state of cooperators, value, e.g., T = 2. As T is decreased further, the advantage of defectors, and pool-based switching strategists. It needs to be cooperators is further enhanced [see Figs. 4(d) and 4(e)]. Par- pointed out that the result is still valid even though the cost ticularly, for T = 0, the frequencies of C, D,andIF display of exclusion exceeds four times than given in Fig. 3.Fur- periodic oscillations, which correspond to the limit cycle in thermore, the decrease of threshold value will weaken the Fig. 3(f). advantage of defectors in evolution. Particularly, if switching threshold is zero, the result shows that there can be isolated periodic orbits, and hence, cooperators, defectors, and pool IV. DISCUSSION excluders can coexist by forming limit cycles [see Fig. 3(f)]. Thus far, many previous theoretical works have revealed Finally, we have to note that the switching strategy pro- that pro-social punishment and exclusion strategies can both posed in this work can better induce the stable coexistence maintain sufficiently high levels of public cooperation no mat- state for cooperation compared with pure punishment or pure ter whether these two strategies are implemented separately exclusion strategies. Besides, such switching strategy can or jointly.36,38,40 However, few studies have thus far consid- flexibly manage public goods in different environments, such ered the combined use of these two strategies despite it is as punishing exiguous free-riders in favorable surroundings particularly common in our real society. In this paper, we or excluding massive free-riders in extremely unfavorable have introduced the switching strategy with which individu- environments. Thus, the presently discussed strategy offers als can either punish or exclude free-riders in the public goods a simple, but still effective, way on how we can better pro- game (PGG), depending on the number of defectors in the mote the stable coexistence of different strategies including group. Based on the evolutionary game theoretical models, cooperation. we have studied the evolutionary dynamics in the well-mixed population with two different switching forms, by focusing APPENDIX A: EVOLUTIONARY DYNAMICS IN THE particularly on the role of switching threshold played in the POPULATION WITH PEER-BASED SWITCHING evolutionary dynamics of cooperation. STRATEGY FOR T = N We have shown that a stable coexistence state among cooperators, defectors, and peer-based switching strategists 1. Peer punishment can appear when an intermediate switching threshold is used. We first study the replicator dynamics for defectors (D), In addition, the reduction of switching threshold can enhance cooperators (C), and peer punishers (W). This corresponds to the level of public cooperation. It is necessary to point out that the special case for peer-based switching strategy with T = N. the special cases in our model (T = N and T = 0 correspond We denote by x, y,andz the frequencies of C, D, and W, to peer punishment and peer exclusion, respectively) have respectively. Thus, x, y, z ≥ 0andx + y + z = 1. The evolu- been investigated in a recent work,36 which demonstrated that tionary fate of the population can be modeled by the replicator social exclusion strategy can not only avert free-riders, but equations, given as also prevent second-order free-riders from invading, which ⎧ ⎪ ˙ = ( − ¯ ) solves the two substantial difficulties of peer punishment. ⎨⎪ x x PC P , Interestingly, we find that the introduction of an observation ¯ y˙ = y(PD − P), (A1) cost will completely change the evolutionary results of these ⎪ ⎩ ¯ two strategies in our model. Concretely, the coexistence of z˙ = z(PW − P), cooperators and peer punishers will disappear, and defection becomes a global stability strategy [see Fig. 1(a)]. Or, the where PC, PD,andPW denote the expected payoffs of these ¯ = + + periodic oscillations among the three strategies replace the three strategies and P xPC yPD zPW describes the coexistence of cooperators and peer excluders [see Fig. 1(f)]. average payoff of the entire population. Although similar shapes of oscillations have been presented Accordingly, the expected payoffs of these three strate- in previous works54,55 where oscillations are caused by the gies can be, respectively, given by feedback between players’ payoffs and their local interac- rc rc P = (N − 1)(x + z) + − c, tion topology or the cyclic dominance of the three species, C N N the mechanisms for the oscillations between our work and rc P = (N − 1)(x + z) − (N − 1)zβ, the two studies mentioned above are different. In our work, D N 103105-7 Liu et al. Chaos 28, 103105 (2018)
rc rc P = (N − 1)(x + z) + − c − (N − 1)yγ − τ. Then, we have the following conclusion. W N N Theorem 1. For 1 < r < N, only the fixed point (0, 1, 0) is Since PW < PC, there is no interior fixed point. Then, we stable, and the other equilibria (0, 0, 1), (1, 0, 0), and 0, 1 − investigate the dynamics on each edge of the simplex S3. = ˙ = ( − ) c+(N−1)γ +τ− rc c+(N−1)γ +τ− rc On the edge C-D, we have z 0, resulting in y y 1 y N , N are unstable. ( − ) = ( − )( − rc )> (N−1)(β+γ) (N−1)(β+γ) PD PC y 1 y c N 0. Thus, the direction of the dynamics goes from C to D. On the edge D-W, since x = 0 Proof. (1) For (x, y, z) = (0, 0, 1), the Jacobian is and y + z = 1, we have z˙ = z(1 − z)(P − P ). Here, we W D τ assume that 0 < c + τ − rc + (N − 1)γ < (N − 1)(γ + β); 0 N J = rc ,(A4) +( − )γ +τ− rc 0 c + τ − − (N − 1)β = = c N 1 N N thus, solving PW PD results in z (N−1)(β+γ) ,which means that there exists a boundary fixed point on the edge thus the fixed point is unstable since τ>0. D-W. On the edge C-W, since y = 0andx + z = 1, we have (2) For (x, y, z) = (1, 0, 0), the Jacobian is x˙ = x(1 − x)(PC − PW ) = x(1 − x)τ > 0; thus, the direction −τ −(c + τ − rc ) of the dynamics goes from W to C. J = N ,(A5) c − rc Therefore, there are four equilibria, namely, three ver- 0 N ( ) = ( ) ( ) ( ) tex fixed points [ x, y, z 0, 0, 1 , 1, 0, 0 ,and 0, 1, 0 ]and thus the fixed point is unstable since 1 − r > 0. c+(N−1)γ +τ− rc N the boundary fixed point (x, y, z) = 0, 1 − N , ( ) = ( ) (N−1)(β+γ) (3) For x, y, z 0, 1, 0 , the Jacobian is c+(N−1)γ +τ− rc N rc ( − )(β+γ) in the simplex S3. − c N 1 = N 0 J −τ − ( − )γ rc − − τ − ( − )γ ,(A6) N 1 N c N 1 2. The stabilities of equilibria rc − < thus the fixed point is stable since N c 0. +( − )γ +τ− rc +( − )γ +τ− rc Here, we define ( ) = − c N 1 N c N 1 N (4) For x, y, z 0, 1 (N−1)(β+γ) , (N−1)(β+γ) , f (x, y) = x[(1 − x)(PC − PW ) − y(PD − PW )], the Jacobian is
g(x, y) = y[(1 − y)(P − P ) − x(P − P )]. a 0 D W C W J = 11 ,(A7) a a Then, the Jacobian matrix of the equation system is 21 22 ⎡ ⎤ ∂f (x, y) ∂f (x, y) where a11 = (N − 1)yγ + τ, a21 = y(1 − y)β(N − 1) − ⎢ ∂ ∂ ⎥ (N − 1)y2γ − τy,anda = y(1 − y)(N − 1)(β + γ), thus = ⎣ x y ⎦ 22 J ∂g(x, y) ∂g(x, y) ,(A2)the fixed point is unstable since y(1 − y)(N − 1)(β + γ)>0 ∂x ∂y and (N − 1)yγ + τ>0. where ⎧ APPENDIX B: EVOLUTIONARY DYNAMICS IN THE ⎪ ∂f (x, y) ⎪ = [(1 − x)(P − P ) − y(P − P )] POPULATION WITH PEER-BASED SWITCHING ⎪ ∂ C W D W ⎪ x STRATEGY FOR T = 0 ⎪ ∂ ⎪ + −( − ) + ( − ) ( − ) ⎪ x PC PW 1 x PC PW 1. Peer exclusion ⎪ ∂x ⎪ ⎪ ∂ We now consider another special case for peer-based ⎪ − ( − ) = ⎪ y PD PW , switching strategy with T 0. Thus, the replicator equations ⎪ ∂x ⎪ are written as ⎪ ∂ ( ) ∂ ⎧ ⎪ f x, y = ( − ) ( − ) − ( − ) ⎪ x˙ = x(P − P¯ ), ⎪ x 1 x PC PW PD PW ⎨⎪ C ⎪ ∂y ∂y ⎪ y˙ = y(P − P¯ ), (B1) ⎪ ∂ ⎪ D ⎪ ⎩⎪ ⎪ −y (PD − PW ) , ˙ = ( − ¯ ) ⎨ ∂y z z PE P ,
⎪ ∂g(x, y) ∂ ¯ = + + ⎪ = y (1 − y) (P − P ) − (P − P ) where P xPC yPD zPE represents the average payoff of ⎪ ∂ ∂ D W C W ⎪ x x the entire population. ⎪ ⎪ ∂ We assume that exclusion never fails. In this condition, ⎪ −x (P − P ) , ⎪ ∂ C W we can formalize the expected payoffs as follows: ⎪ x ⎪ ⎪ ∂g(x, y) − rc(N − 1)y ⎪ = ( − )( − ) − ( − ) P = rc − c − (1 − z)N 1 ,(B2) ⎪ [ 1 y PD PW x PC PW ] C ( − ) ⎪ ∂y N 1 z ⎪ ⎪ ∂ ⎪ + −( − ) + ( − ) ( − ) ⎪ y PD PW 1 y PD PW rc(N − 1)x ⎪ ∂y = ( − )N−1 ⎪ PD 1 z ,(B3) ⎪ N(1 − z) ⎪ ∂ ⎩ −x (P − P ) ∂ C W . y = − − ( − ) − τ (A3) PE rc c N 1 ycE .(B4) 103105-8 Liu et al. Chaos 28, 103105 (2018)
Remark 1. Because of z = 1 − x − y, the system (B1) thus the fixed point is unstable since τ>0. becomes (2) For (x, y, z) = (1, 0, 0), the Jacobian is x˙ = x[(1 − x)(PC − PE) − y(PD − PE)], (B5) ⎡ ⎤ y˙ = y[(1 − y)(PD − PE) − x(PC − PE)], rc −τ −(c + τ − ) ⎣ ⎦ where J = rc N ,(B7) 0 c − rc(N − 1)y N P − P = (N − 1)yc + τ − (1 − z)N−1 , C E E N(1 − z) rc(N − 1)x P − P = ( − z)N−1 thus the fixed point is a unstable since 1 − r > 0. D E 1 ( − ) N N 1 z (3) For (x, y, z) = (0, 1, 0), the Jacobian is − rc + c + (N − 1)ycE + τ.
N(r−1) 1 Theorem 2. For [ ] N−1 (N − 1)c < rc − c − τ
∂f ∗ ∗ ∂g ∗ ∗ ∂f ∗ ∗ ∂g −1 Then, we define that p1 = (x , y ) (x , y ) − (x , y ) Let T = M ,thenwehave ∂x ∂y ∂y ∂x ( ∗ ∗) = ∂f ( ∗ ∗) + ∂g ( ∗ ∗) x , y and q1 ∂ x , y ∂ x , y . We know that − 00 x y TAT 1 = . (B13) 0 rc − c ∂ ∂ N p = x∗y∗(1 − x∗ − y∗) (P − P ) (P − P ) 1 ∂x C E ∂y D E Using variable substitution, we have
∂ ∂ j x 01 x z − ( − ) ( − ) = T = = . (B14) PC PE PD PE i z z x ∂y ∂x 10 rc(N − 1)3 Therefore, the system (B10) can be rewritten as = x∗y∗(1 − x∗ − y∗) (x∗ + y∗)N−2 ⎧ N ⎪ ˙ ⎪ j = j(1 − j) rc − c − (N − 1)(1 − j − i)cE ∗ ∗ N−2 rc ⎪ × (x + y ) − 1 ⎪ N ⎪ ( − ) ⎪ N−1 rc N 1 i < ⎪ −τ − (1 − j) 0 ⎪ N(1 − j) ⎪ ⎪ ( − ) and ⎪ N−1 rc N 1 ⎨⎪ − ji rc − c − (1 − j) , ∂ ∂ N q = x∗( − x∗) (P − P ) + y∗( − y∗) (P − P ) 1 1 ∂ C E 1 ∂ D E ⎪ rc(N − 1) x y ⎪ ˙ = ( − ) − − ( − )N−1 ⎪ i i 1 i rc c 1 j ∂ ∂ ⎪ N − x∗y∗ (P − P ) + (P − P ) ⎪ ∂ D E ∂ C E ⎪ x y ⎪ − − − ( − )( − − ) − τ ⎪ ji rc c N 1 1 j i cE = ( − ) ∗( − ∗ − ∗) ⎪ N 1 cEy 1 y x ⎪ ⎪ rc(N − 1)i > 0. ⎩⎪ −(1 − j)N−1 . N(1 − j) We thus conclude that the Jacobian matrix has a posi- Using the center manifold theorem, we have that j = h(i) is a tive eigenvalue. Therefore, the interior equilibrium point is center manifold for the above system. Then, the dynamics on unstable. the center manifold can be described by
Theorem 4. For rc − c − τ − (N − 1)c = 0, the equilib- ( − ) E ˙ N−1 rc N 1 ( ) i = i(1 − i) rc − c − (1 − h(i)) rium point 0, 1, 0 is stable. N
Proof. Because y = 1 − x − z, the dynamic equations − h(i)i rc − c − (N − 1)(1 − h(i) − i)c − τ (B5) become E ˙ = ( − )( − ) − ( − ) − rc(N − 1)i x x[ 1 x PC PD z PE PD ], −(1 − h(i))N 1 . (B15) (B10) N(1 − h(i)) z˙ = z[(1 − z)(PE − PD) − x(PC − PD)], We start to try h(i) = O(i2); thus, the system (B15) can be where expressed as
− rc(N − 1) ( − ) − = − − ( − )N 1 rc N 1 3 PC PD rc c 1 z , ˙i = i(1 − i) rc − c − + O(|i| ). (B16) N N − = − − ( − ) − τ PE PD rc c N 1 ycE ( ) = ( − ) − − rc(N−1) By defining m i i 1 i [rc c N ], we have ( − ) ( − ) N−1 rc N 1 x ( ) = ( − ) − − rc N 1 ( )< − (1 − z) . m i 1 2i [rc c N ]. Since m 0 0, thus we N(1 − z) can judge that i = 0 is asymptotically stable. Therefore, we 103105-10 Liu et al. Chaos 28, 103105 (2018) can know the fixed point [i, h(i)] = (0, 0) is also stable for the domain 0 containing the interior fixed point. The trajectories system (B10). Accordingly, the fixed point (0, 1, 0) is stable on the boundary of the closed domain 0 are all diverged out- as well.56Ð58 ward. Thus, a ring domain is formed between the boundaries of the closed domains and 0, and the trajectories on the 3. Heteroclinic cycle boundary of the inner and outer ring will go to the interior of In this subsection, we show that there is a stable het- the domain. Accordingly, we prove that there exists a stable limit cycle in the closed domain .56 eroclinic cycle on the boundary of the simplex CDIE [see Fig. 1(f)]. When rc − c − τ − (N − 1)c > 0andr < N,we E APPENDIX C: EVOLUTIONARY DYNAMICS IN THE know that the three vertex equilibrium points (C, D,andI ) E POPULATION WITH POOL-BASED SWITCHING are all saddle nodes, and the heteroclinic trajectories can dis- STRATEGY FOR T = N play on the three edges (CD, DIE,andIEC). All of these guarantee the existence of the heteroclinic cycle on the bound- 1. Pool punishment ary S3. In the following, we will prove that the heteroclinic Next, we analyze the replicator dynamics for public cycle is asymptotically stable. goods game with pool punishment, which corresponds to the According to Theorem 3, we can get the eigenval- special case for pool-based switching strategy with T = N. ues of the Jacobian matrix of the three vertex equi- Accordingly, there are three strategists, namely, coopera- librium points, namely, λ− =−(rc − c − τ), λ+ = τ, λ− = IE IE C tors, defectors, and pool punishers, respectively. Thus, the −τ λ+ = − rc λ− = rc − λ+ = − − τ − , C c N , D N c,and D rc c − replicator equations can be written as λ λ− ⎧ ( − ) λ =− IE λ =− C ˙ = ( − ¯ ) N 1 cE. Then, we define that IE λ+ , C λ+ ,and ⎪ x x PC P , IE C ⎨ − λ − −τ ¯ λ =− D λ = λ λ λ = rc c > y˙ = y(PD − P), (C1) D λ+ , and we have IE C D rc−c−τ−(N−1)c 1. ⎪ D E ⎩⎪ 53 ¯ Thus, the heteroclinic cycle is asymptotically stable. z˙ = z(PV − P), where P¯ = xP + yP + zP is the average payoff of the 4. Limit cycle C D V whole population. Then, the expected payoffs of these three Next, we prove that a stable limit cycle can exist in the strategies can be, respectively, given by simplex CDIE [see Fig. 1(f)]. According to the above theo- rc rc P = (N − 1)(x + z) + − c,(C2) retical analysis, we know that the interior fixed point of the C N N system is unstable. Now, we set a closed domain that con- rc tains the interior fixed point on the phase plane of the system PD = (N − 1)(x + z) − B(N − 1)z,(C3) (B5). Here, we take a straight line l = x + y − b = 0, where b N is an undetermined constant. In order to determine the direc- rc rc P = (N − 1)(x + z) + − c − G − τ,(C4) tion of the trajectory of the system, we solve the derivative of V N N = + − the equation l x y b.Thus,wehave where (N − 1)(x + z) denotes the expected number of con- ∂l ∂x ∂y tributors among the N − 1 co-players, and B(N − 1)z repre- = + = x(1 − x − y) ∂ ∂ ∂ sents the expected fine on a defector. t t t ( − ) N−1 rc N 1 y Theorem 5. For r < N, the system (C1) has four fixed points, × (N − 1)ycE + τ − (x + y) N(x + y) namely, (x, y, z) = (0, 0, 1), (1, 0, 0), (0, 1, 0), and
− N(c+G+τ)−rc N(c+G+τ)−rc rc(N − 1)x 0, 1 ( − ) , ( − ) . + y(1 − x − y) (x + y)N−1 − rc + c N N 1 B N N 1 B N(x + y) Proof. By solving the system equations (C1), we can know that there are three vertex equilibrium points, namely, + (N − 1)yc + τ E (x, y, z) = (0, 0, 1), (0, 1, 0),and(1, 0, 0). There is no interior fixed point since PV < PC. Then, = (1 − x − y)[(N − 1)ycE(x + y) + (x + y)τ − (rc − c)y] we study the dynamics on each edge of simplex S3.On = ( − ) ( − ) + τ − ( − ) 1 b [ N 1 ycEb b rc c y]. (B17) the edge V-D, y + z = 1 results in z˙ = z(1 − z)(PV − PD) = rc ∂ z(1 − z)[ − c − G − τ + B(N − 1)z]; thus, there is an equi- Thus, when b is sufficiently large, we know that l < 0. In this N ∂t z = N(c+G+τ)−rc < N(c+G+τ)−rc < case, the straight line l = x + y − b = 0, x = 0, and y = 0are librium N(N−1)B for 0 N(N−1)B 1. On the + = ˙ = ( − )( − ) = enclosed in a closed domain that points to the interior of the edge C-V, we have x z 1andx x 1 x PC PV ( − )( + τ) > boundary. x 1 x G 0; thus, the direction of the dynamics goes ∗ ∗ from V to C. On the edge C-D, D can defeat C, as presented On the straight line x = x , we select a point (x , y1), ∗ in Appendix A 1. where y1 is slightly larger than y , and the trajectory that passes through (x∗, y ) surrounds the interior fixed point 1 2. The stabilities of equilibria (x∗, y∗), then intersects with the straight line x = x∗ at another ∗ ∗ point (x , y2) which meets y2 > y . Since the interior fixed Theorem 6. In the conditions of Theorem 5, the fixed point ∗ ∗ point (x , y ) is unstable, we have y2 > y1. The trajectory from (0, 1, 0) is stable, while the others (0, 0, 1), (1, 0, 0), and ( ∗ ) ( ∗ ) ( − N(c+G+τ)−rc N(c+G+τ)−rc ) x , y1 to x , y2 and the line segment y1y2 form a closed 0, 1 N(N−1)B , N(N−1)B are unstable. 103105-11 Liu et al. Chaos 28, 103105 (2018)
Proof. (1) For (x, y, z) = (0, 0, 1), the Jacobian is Thus, when δ + τ
In this subsection, we study the evolutionary dynamics of pool exclusion, which corresponds to the special case for δ + τ 0 J(0, 0, 1) = ,(D5) pool-base switching strategy with T = 0. Then, the replicator 0 −rc + c + δ + τ equations become ⎧ ⎪ ˙ = ( − ¯ ) ⎨⎪ x x PC P , ¯ y˙ = y(PD − P), (D1) thus the fixed point is unstable since δ + τ>0. ⎪ ⎩ ¯ (2) For (x, y, z) = (1, 0, 0), the Jacobian is z˙ = z(PF − P), where PC, PD,andPF are the expected payoffs of cooperators, defectors, and pool excluders, respectively. We also assume that exclusion never fails. In this condi- −δ − τ −(c + δ + τ − rc ) ( ) = N tion, we give the expected payoffs as follows: J 1, 0, 0 − rc ,(D6) 0 c N rc(N − 1)y P = rc − c − (1 − z)N−1 ,(D2) C N(1 − z)
− rc x thus the fixed point is unstable since 1 − r > 0. P = (1 − z)N 1 (N − 1) ,(D3) N D N 1 − z (3) For (x, y, z) = (0, 1, 0), the Jacobian is
PF = rc − c − δ − τ.(D4) Theorem 7. For r < N and δ + τ
Jacobian matrix are written as 2 − < = ⎧ We have q2 4p2 0andq2 0, therefore the eigenvalues ∂ ∂ ⎪ f ∗∗ ∗∗ ∗∗ ∗∗ of the Jacobian matrix are pure imaginary. The dynamics anal- ⎪ (x , y ) = x (1 − x ) (PC − PF) ⎪ ∂ ∂ ysis of the interior of S3 and the stability of interior fixed point ⎪ x x ⎪ can be found in Appendix D 3. ⎪ ∂ ⎪ ∗∗ ⎪ −y (PD − PF) , ⎪ ∂x ⎪ 3. The Hamiltonian system ⎪ ∂ ∂ ⎪ f ∗∗ ∗∗ ∗∗ ∗∗ ⎪ (x , y ) = x (1 − x ) (PC − PF) To analyze the dynamics in the interior of S3, we intro- ⎪ ∂y ∂y ε = x ⎪ duce a new variable x+y , which represents the fraction ⎪ ∂ ⎪ ∗∗ of cooperators among members who do not contribute to the ⎨⎪ −y (PD − PF) , ∂y exclusion pool. This yields
⎪ ∂ ∂ xy ⎪ g ∗∗ ∗∗ ∗∗ ∗∗ ε˙ = (P − P ) ⎪ (x y ) = y ( − y ) (P − P ) 2 C D ⎪ ∂ , 1 ∂ D F (x + y) ⎪ x x ⎪ ⎪ ∂ =−ε(1 − ε)(PD − PC).(D8) ⎪ ∗∗ ⎪ −x (PC − PF) , ⎪ ∂x = ε( − ) ¯ = ( − ) + ( − ) ⎪ By substituting x 1 z and P x PC PD 1 z ⎪ ∂ ∂ (P − P ) + P into z˙ = z(P − P¯ ),wehavez˙ = ⎪ g ∗∗ ∗∗ ∗∗ ∗∗ D F F F ⎪ (x , y ) = y (1 − y ) (PD − PF) ( − ) − ( − )( − ) ⎪ ∂y ∂y z[x PD PC 1 z PD PF ]. Thus, we have ⎪ ⎧ ⎪ ⎪ rc(N − 1) ⎪ ∗∗ ∂ ⎨ ε˙ =−ε( − ε) ( − )N−1 − + ⎩⎪ −x (P − P ) , 1 1 z rc c , ∂y C F N (D9) ⎩⎪ where z˙ = z(1 − z)[rc − c − δ − τ − ε(rc − c)]. ⎧ ∗∗ ∂ ∗∗ ∗∗ − rcy (N − 1)(N − 2) By dividing the right-hand side of Eq. (D9) by the function ⎪ (P − P ) =−(x + y )N 3 ⎪ ∂ C F , ε( − ε) ( − ) ⎪ x N 1 z 1 z , we further have ⎪ ⎧ ⎪ ∂ ∗∗ ∗∗ − rc(N − 1) ( − ) ⎪ ( − ) =−( + )N 3 ⎪ 1 N− rc N 1 ⎪ PC PF x y ⎪ ε˙ =− ( − z) 1 − rc + c ⎪ ∂y N ⎨ ( − ) 1 , ⎪ z 1 z N ⎨⎪ ∗∗ ∗∗ [(N − 1)y + x ], ⎪ 1 ⎩ z˙ = [rc − c − δ − τ − ε(rc − c)]. ⎪ ∂ rc(N − 1) ε(1 − ε) ⎪ ( − ) = ( ∗∗ + ∗∗)N−3 ⎪ PD PF x y (D10) ⎪ ∂x N ⎪ Let us introduce H(ε, z) = M (z) + L(ε),whereM (z) and ⎪ ( − ) ∗∗ + ∗∗ ⎪ [ N 1 x y ], L(ε) are primitives of z˙ and ε˙, which are, respectively, given ⎪ ⎪ ∂ rcx∗∗(N − 1)(N − 2) as ⎩ (P − P ) = (x∗∗ + y∗∗)N−3 ∂ D F . y N M (z) = (−δ − τ)log(1 − ε) − (rc − c − δ − τ)log(ε), = ∂f ( ∗∗ ∗∗) ∂g ( ∗∗ ∗∗) − ∂f Then, we define that p2 ∂x x , y ∂y x , y ∂y L(ε) = (rc − c)[log(1 − z) − log(z)] ∗∗ ∗∗ ∂g ∗∗ ∗∗ ∂f ∗∗ ∗∗ ∂g ∗∗ ∗∗ (x , y ) (x , y ) and q2 = (x , y ) + (x , y ). − ∂x ∂x ∂y N2 N − 2 zk Thus, we know that + (−1)k + log(z). k k ∂ ∂ k=1 p = x∗∗y∗∗(1 − x∗∗ − y∗∗) (P − P ) (P − P ) 2 ∂x C F ∂y D F Then, we obtain the Hamiltonian system as ⎧ ∂ ∂ ⎪ ∂H − (P − P ) (P − P ) ⎨⎪ ε˙ = , ∂y C F ∂x D F ∂z ⎪ (D11) 2 2( − )3 ⎪ ∂H ∗∗ ∗∗ ∗∗ ∗∗ ∗∗ ∗∗ 2N−4 r c N 1 ⎩ z˙ = = x y (1 − x − y )(x + y ) ∂ε . N 2 > 0, Thus, the system is conservative, and all constant level sets of H are closed curves. and ∂ ∂ Acknowledgments q = x∗∗(1 − x∗∗) (P − P ) + y∗∗(1 − y∗∗) (P − P ) 2 ∂x C F ∂y D F This research was supported by the National Natural Sci- ∗∗ ∗∗ ∂ ∂ ence Foundation of China (NNSFC) (Grant No. 61503062) − x y (P − P ) + (P − P ) ∂y D F ∂y C F and by the Slovenian Research Agency (Grant Nos. J1-7009 rc(N − 1)(N − 2) and P5-0027). = x∗∗y∗∗(x∗∗ + y∗∗)N−3 [(y∗∗ − x∗∗) N 1R. Axelrod, Am. Pol. Sci. Rev. 75, 306Ð318 (1981). 2 ∗∗ ∗∗ O. Durán and R. Mulet, Physica D 208, 257Ð265 (2005). + (1 − y ) − (1 − x )] 3J. Tanimoto, Phys. Rev. E 76, 041130 (2007). 4 = F. C. Santos, M. D. Santos, and J. M. Pacheco, Nature 454, 213Ð216 0. (2008). 103105-13 Liu et al. Chaos 28, 103105 (2018)
5M. A. Nowak, A. Sasaki, C. Taylor, and D. Fudenberg, Nature 428, 646 32X. Chen, A. Szolnoki, and M. Perc, Phys.Rev.E92, 012819 (2015). (2004). 33R. Boyd, H. Gintis, and S. Bowles, Science 328, 617Ð620 (2010). 6Y. Zhang, X. Chen, A. Liu, and C. Sun, Appl. Math. Comput. 321, 641Ð653 34T. Sasaki, S. Uchida, and X. Chen, Sci. Rep. 5, 8917 (2015). (2018). 35A. Szolnoki and M. Perc, New J. Phys. 18, 083021 (2016). 7J. Tanimoto, Physica D 200, 1Ð24 (2005). 36T. Sasaki and S. Uchida, Proc. R. Soc. B 280, 20122498 (2013). 8W. X. Wang, Y. C. Lai, and D. Armbruster, Chaos 21, 033112 (2011). 37K. Li, R. Cong, T. Wu, and L. Wang, Phys.Rev.E91, 042810 (2015). 9X. Chen and L. Wang, Phys.Rev.E77, 017103 (2008). 38A. Szolnoki and X. Chen, Phys.Rev.E95, 052316 (2017). 10M. Perc and A. Szolnoki, Phys.Rev.E77, 011904 (2008). 39K. Li, R. Cong, and L. Wang, EPL 114, 58001 (2016). 11M. Perc and A. Szolnoki, “Coevolutionary games - A mini review,” 40L. Liu, X. Chen, and A. Szolnoki, Sci. Rep. 7, 46634 (2017). BioSystems 99, 109Ð125 (2010). 41X. Sui, B. Wu, and L. Wang, EPL 121, 38003 (2018). 12M. Perc, J. J. Jordan, D. G. Rand, Z. Wang, S. Boccaletti, and A. Szolnoki, 42E. Helland, Rev. Econom. Statist. 80, 141Ð153 (1998). Phys. Rep. 687, 1Ð51 (2017). 43J. P. Shimshack and M. B. Ward, J. Environ. Econ. Manage. 50, 519Ð540 13M. A. Nowak, Science 314, 1560Ð1563 (2006). (2005). 14F. Fu, C. Hauert, M. A. Nowak, and L. Wang, Phys. Rev. E 78, 026117 44L. Liu, X. Chen, and S. Wang, in Chinese Automation Congress (CAC) (2008). (IEEE, 2017), pp. 3961Ð3966. 15D. J. De Quervain, U. Fischbacher, V. Treyer, and M. Schellhammer, 45K. Sigmund, H. De Silva, A. Traulsen, and C. Hauert, Nature 466, 861Ð863 Science 305, 1254 (2004). (2010). 16X. Chen, T. Gross, and U. Dieckmann, J. Theor. Biol. 335, 13Ð21 (2013). 46P. Schuster and K. Sigmund, J. Theor. Biol. 100, 533Ð538 (1983). 17A. Szolnoki and M. Perc, EPL 92, 38003 (2010). 47J. Hofbauer and K. Sigmund, “Evolutionary game dynamics,” Bull. Am. 18A. Szolnoki and M. Perc, New J. Phys. 14, 093016 (2012). Math. Soc. 40, 479Ð519 (2003). 19M. Perc and A. Szolnoki, New J. Phys. 14, 043013 (2012). 48C. Hauert, M. S. De, J. Hofbauer, and K. Sigmund, J. Theor. Biol. 218, 20Z. Wang, A. Szolnoki, and M. Perc, Sci. Rep. 3, 1183 (2013). 187Ð194 (2002). 21X. Chen, T. Sasaki, and M. Perc, Sci. Rep. 5, 17050 (2015). 49T. Sasaki and T. Unemi, J. Theor. Biol. 287, 109Ð114 (2011). 22A. Szolnoki, G. Szabó, and M. Perc, Phys.Rev.E83, 036101 (2011). 50J. Tanimoto, Fundamentals of Evolutionary Game Theory and Its Applica- 23X. Chen, T. Sasaki, Å. Brännström, and U. Dieckmann, J. R. Soc. Interface tions (Springer, Tokyo, 2015). 12, 20140935 (2014). 51Q. Wang, N. He, and X. Chen, Appl. Math. Comput. 328, 162Ð170 (2018). 24X. Chen, A. Szolnoki, and M. Perc, New J. Phys. 16, 083016 (2014). 52R. H. Rand, M Yazhbin, and D. G. Rand, Commun. Nonlinear Sci. Numer. 25D. G. Rand, A. Dreber, T. Ellingsen, D. Fudenberg, and M. A. Nowak, Simul. 16, 3887Ð3895 (2011). Science 325, 1272Ð1275 (2009). 53J. Park, Appl. Math. Comput. 320, 425Ð436 (2018). 26Q.¬ Gürerk, B. Irlenbusch, and B. Rockenbach, Science 312, 108Ð111 54G. Zschaler, A. Traulsen, and T. Gross, New J. Phys. 12, 093015 (2010). (2006). 55T. Reichenbach, M. Mobilia, and E. Frey, Phys. Rev. E 74, 051907 (2006). 27E. Fehr and S. Gächter, Nature 415, 137Ð140 (2002). 56H. K. Khalil, Noninear Systems (Prentice-Hall, Englewood Cliffs, NJ, 28Z. Wang, C. Y. Xia, S. Meloni, C. S. Zhou, and Y. Moreno, Sci. Rep. 3, 1996). 3055 (2013). 57J. Carr, Applications of Center Manifold Theory (Springer-Verlag, New 29H.-X. Yang and X. Chen, Appl. Math. Comput. 316, 460Ð466 (2018). York, 1981). 30A. Szolnoki and M. Perc, Phys.Rev.X7, 041027 (2017). 58B. D. Hassard, N. D. Kazarinoff, and Y. H. Wan, Theory and Applications 31M. Perc and A. Szolnoki, Sci. Rep. 5, 11027 (2015). of Hopf Bifurcation (Cambridge University Press, Cambridge, UK, 1981).