Replicator Dynamics with Diffusion on Multiplex Networks

PHYSICAL REVIEW E 94, 022301 (2016)

Replicator dynamics with diffusion on multiplex networks

R. J. Requejo* and A. D´ıaz-Guilera Departament de F´ısica Fonamental, Universitat de Barcelona, Mart´ı i Franques 1, 08028 Barcelona, Spain (Received 6 May 2015; revised manuscript received 7 December 2015; published 1 August 2016) In this study we present an extension of the dynamics of diffusion in multiplex graphs, which makes the equations compatible with the replicator equation with mutations. We derive an exact formula for the diffusion term, which shows that, while diffusion is linear for numbers of agents, it is necessary to account for nonlinear terms when working with fractions of individuals. We also derive the transition probabilities that give rise to such macroscopic behavior, completing the bottom-up description. Finally, it is shown that the usual assumption of constant population sizes induces a hidden selective pressure due to the diffusive dynamics, which favors the increase of fast diffusing strategies.

DOI: 10.1103/PhysRevE.94.022301

I. INTRODUCTION more, an extension of the replicator dynamics to regular networks has been developed in a situation where each node During the last decade agent-based modeling has increased is an agent [32], and a second extension of the replicator its importance as a powerful tool to model situations in which dynamics has been developed to represent a two-dimensional the complexity of the interactions of many-agent systems world where the agents diffuse [33]. However, many real-world makes it impossible, or at least very difficult, to make analytic situations cannot be modeled as a simple network or a two- predictions of the dynamical behavior of the system [1,2]. dimensional space, and need the introduction of several kinds Furthermore, the agent-based models complement the an- of links to represent different properties of the system [34], as alytic approach allowing for an exploration of the coarse- in air transportation networks, where each airline represents a grained dynamics and the connection between microscale different network [35]. and macroscale behavior [3–6]. Such agent-based modeling Let us focus on a cultural evolutionary framework for the is of special importance in evolutionary game theoretical remainder of the paper. In such a framework the individual studies [7,8], which aim to capture the intricacies of biological, traits being selected are behavioral traits, called strategies. social, and economical systems, where the nonlinearity and The main aim of this paper is to develop the mathematical feedbacks of the systems cannot be easily foreseen [9–12]. tools that allow us to model diffusion of strategies in multiplex In the middle of such framework [13], and connected with networks in a compatible way with the selection dynamics the microevolutionary dynamics [3–6], stands the replicator described by the replicator equation (with or without muta- equation [14,15] tions), irrespective of the microscopic dynamics that give rise α α α (x˙ )rep = x (f − f¯). (1) to the replicator equation (see Appendix A). Let us provide a simple example to illustrate the situation: imagine several The replicator equation, which was introduced shortly cities that are connected by bus, train, and plane, each kind after the foundation of the evolutionary game theoretical of connection with a different network structure. Individuals framework [7,16,17], has been extensively used during the last in each city are interacting between them, both directly or α = α decades to model the evolution of the fractions x n /N of indirectly: they have different jobs and different incomes, traits of type α = 1,...,L in large well-mixed populations α which determines, at least partially, their choices on how to with frequency dependent selection, i.e., when the fitness f travel, and they also have information about the transportation (reproductive potential or capacity) of the agents traits and ways. Based on such interactions and information, they may ¯ = α α the mean population fitness f α f x depend on the decide to travel in one or another transport, which determines population composition. Such traits may represent different their individual strategy. As the different transportation ways phenotypes and genotypes in biological settings, or different determine different networks structures, as well as the travel behavioral strategies in a cultural evolutionary framework. speeds are different, a multilayer network is necessary to The replicator equation is on the core of the framework of represent the full transportation system; hence, as a first evolution, linked to the quasispecies equation [13], the game approach to such simplified situation, we need to extend the dynamical equation [3,4], adaptive dynamics [13], Lotka- replicator dynamics (describing strategy changes) to diffusive Volterra dynamics [18,19], and the Price equation [20–23]. individuals in this kind of multilayer. During the last years the physics community has analyzed how We may think that introducing diffusion on multiplex such equation is affected by physical entities, as the interplay models of evolution is a straightforward task, as the diffusive between different temporal scales of interaction and selec- process has been studied on detail on such networks [36,37]. tion [24,25], the self-organized homeostatic regulation [26] However, it poses a challenge: as the replicator dynamics triggered by constraints on resource availability [9,10,12,27], describe the evolution of fractions of individuals, the diffusive or the existence of network structures [25,28–31]. Further- models need to be rewritten in a compatible way, accounting for the constraint on the addition of the fractions to one, as well as for the conservation of the number of agents. We *[email protected] develop such extension in this paper, showing that working

2470-0045/2016/94(2)/022301(10) 022301-1 ©2016 American Physical Society R. J. REQUEJO AND A. DIAZ-GUILERA´ PHYSICAL REVIEW E 94, 022301 (2016) with fractions introduces dependencies, which are not present Each site i of a multiplex may be regarded as one for the diffusion of the numbers of agents. These new structural entity, and the different layers α represent different dependencies can only be taken into account introducing a interconnection structures between these entities. Structural nonlinear term, which has to be added to the linear one, in order entities (sites) may represent specific locations (cities), and to represent the general dynamics of diffusion of fractions of the different connectivity of each transportation system or individuals in the multiplex. Furthermore, we discuss some mobility pattern would define the different layers through situations in which the linear scenario can be recovered, as which the agents move, while the interlayer connections when population sizes or population size ratios between sites refer to the possibility to reach another transport (layer) are constant, and show that in such situations hidden selective from a specific location (for simplicity we will assume that pressures act on the system, even when they do not appear αβ = 1). Individuals changing from one layer to another can explicitly on the equations. be represented as changing their strategy due to selection (due The paper is outlined as follows. We start in Sec. II by to prices, availability, or other competitive reasons) or mutation defining multiplex networks and showing the problem to (random trial) in an evolutionary framework. overcome when working with fractions. After that, in Sec. III In order to describe the state of the entire multiplex system, we derive the diffusion term compatible with the replicator we may define a set of vectors {ni }, one per site i, each one dynamics and infer the transition probabilities that give rise to with L (number of layers) components. From an evolutionary it. Then, in Sec. IV we discuss some situations in which the game theoretical point of view the components of such vectors α α linear dynamics are recovered (and the analytic calculations represent the number of agents ni in a given position i of simplified), and show that some of this situations carry attached the multiplex, and hence ni is the population composition at the appearance of hidden selective pressures. Finally, in Sec. V i. Then, the evolution of the state of node iα may be assumed α { } α αβ we discuss the results. as given by a functional Fi [ ni (t) ,aij , ], which depends, respectively, on the state of the system, and on the intralayer and interlayer connectivities. II. DIFFUSION IN THE MULTIPLEX: THE PROBLEM As the state of the system is instantaneously defined by a OF WORKING WITH FRACTIONS set of quantities ni for each site i, we may write the dynamics In a general multilayer network [34] each node in one layer as a set of coupled differential equations, one per component can be connected to any node in any other layer. Hence, the (layer), αβ connectivity may be given by a tensor with four indices, Mij , dnα(t) whose entries are 1 if nodes iα—node i in layer α—and j β are i = α α α α αβ β Fi ni (t),aij nj (t), ni (t) , (3) connected and zero otherwise. In multiplex networks [36–40], dt however, the set of nodes i = 1,...,N is the same in all layers α = ,...,L where the terms on which the functional depends are, re- 1 , and the connectivity of each layer is given by a α matrix Aα ={aα } (see Fig. 1). Furthermore, only connections spectively, the state of i , the state of connected nodes in ij the same layer (intralayer neighborhood of iα), and the state among one node iα and its counterpart in another layer iβ are of equivalent nodes connected through interlayer connections allowed. This interlayer connectivity is the same for all nodes (interlayer neighborhood of iα). and is given by the interlayer connectivity matrix αβ [37]. In order to approach a replicator-equation-like functional Hence, multiplex networks have a connectivity defined by α = [see Eq. (1)], we have to normalize ni with respect to Ni nα, the number of agents (population size) on site i, which Mαα = aα Mαβ = αβ Mαβ = 0(2)α i ij ij ii ij is only site dependent. The normalized quantity is the fraction α = α α = i = j α = β xi ni /Ni , which must fulfill the restriction α xi 1, and with and . α = hence its derivative satisfies α x˙i 0. Note that, whenever the agents diffuse from j β to iβ (or in the opposite direction), they are modifying the value of the fraction in iα through the modification of the population size Ni . Hence, in order to account for the dynamics in a position iα it is no longer enough to take into account the direct neighborhood of iα, given by its connectivity, but it is necessary to take into account the entire neighborhood of i, introducing an extra dependence. The functional in terms of fractions is thus

dxα(t) i = F α x (t),aα xα(t),αβ xβ (t),aβ xβ (t) , (4) dt i i ij j i ij j

where the extra dependencies are given by the first term FIG. 1. Example of multiplex network formed by three layers between brackets, which now accounts for the entire state of i, β (blue, gray, and red). The set of nodes is the same in all layers and and the last term, which accounts for the neighborhood of i . the interlayer links connect them in a one to one basis. For clarity, As we show in the following, such extra dependencies require the interlayer connection between the first and third layer has been a modification of the diffusion term, which will no longer be omitted. linear, but include a nonlinear term.

022301-2 REPLICATOR DYNAMICS WITH DIFFUSION ON . . . PHYSICAL REVIEW E 94, 022301 (2016)

III. REPLICATOR DYNAMICS WITH DIFFUSION behavior of the system and the latter (diffusion term) for IN THE MULTIPLEX stochastic effects. Note that the stochastic effects disappear in the thermodynamic limit Ni →∞, or whenever Ni In this section we derive a diffusion term in the multiplex + − (T α + T α), i.e., when the transition rates are small compatible with the replicator dynamics, i.e., describing the j i|j i|j compared to the population size. evolution of the fractions of agents xα at site i with diffusive i Whenever we introduce the transition probabilities Eq. (5) pattern given by layer α, and keeping the conservation into the drift term in Eq. (8) expressed for numbers of of the total number of agents in the multiplex, N = N i i individuals, the deterministic diffusive dynamics for the where N is constant, as well as the constraint xα = 1. We α i number of agents are will assume that diffusion and evolution are uncoupled, and hence the diffusion term can simply be added to the dynamics. n˙α = Dα aα nα − nα =−Dα Lα nα, (9) As the diffusion of agents has been already studied, let us i diff ij j i ij j j j start trying to derive the equations for diffusion of the fractions from those for the number of agents, and discuss its use in a well-known equation for the diffusion of particles on a α ={ α }={ α − α } game theoretical studies. If the agents are diffusing across a network, where the tensor L Lij δij ki aij is the network structure with adjacency matrix of layer α given by graph Laplacian of the corresponding layer α (with δαβ the the elements aα = 1 for connected nodes and 0 otherwise, the α ij Kronecker’s δ) and ki is the degree of site i in layer α. transition probabilities determining the microscopic dynamics In order to derive the equation for the diffusion of fractions of diffusion of (numbers of) individuals i given j in layer α instead of numbers, it is important to note that, if we are differentiate the fraction xα = nα/N, we get i i +α α α α α α α = = α Ti|j diff D aij nj D aij Nj xj n˙ N˙ x˙α = i − xα i (10) −α = α α α = α α α i N i N Ti|j diff D aij ni D aij Ni xi , (5) i i α α where D is the diffusion coefficient of layer α and and hence a nonlinear term that depends on xi appears. Since we are trying to find the exact description of the diffusive +α = α → α + α → α − Ti|j T ni ni 1 nj nj 1 (6) process which is compatible with the replicator equation for is the transition rate of increase (decrease, by changing all mobile agents in a multiplex, we can follow this approach: signs) of the number of agents in iα in one unit triggered by first, construct a compatible macroscopic equation, and then the movement of one agent in j α. Note that, as the process infer the transition probabilities that give rise to it. In order to of diffusion implies a redistribution of agents, the aggregated do this we may introduce Eq. (9) into Eq. (10) [note that the latter is a replicator equation of the form of Eq. (A1)], and use number of agents Ni + Nj is preserved in each diffusive event + − β α = α n = Ni , obtaining a term of the form, among i and j, and hence the constraint (Ti|j )diff (Tj|i )diff β i holds, which also ensures the global conservation of agents α α α α α β β β α → x˙ =−D ρij L x + x D ρij L x , in the entire multiplex system by linking the processes ni i diff ij j i ij j α + α → α − j β j ni 1 and nj nj 1. In the following, for simplicity, we will write the transition rates in a simplified form, explicitly (11) stating the process to which it refers in the focal variables and = the terms involved, but not the possible linked processes, which where ρij Nj /Ni is a population size dependence between will be pointed out when they are relevant for the discussion; in neighboring sites. As it can be observed, in addition to a linear +α = α → α + | α term, the first one, a second term appears. This term ensures this way, Eq. (6)wouldbe(Ti|j )diff T [ni ni 1 nj ]. The microscopic dynamics can be connected with the that the normalization of the fractions is the proper one, i.e., macroscopic behavior of the system by expanding a Fokker- fractions always add up to one, as shown in Figs. 2(a), 2(b) for a Planck equation and truncating high-order terms (this happens system of two layers and two nodes in each layer. Note that, as naturally for N 1 when working with fractions, see Ap- previously foreseen in Sec. II, this term implies a dependence pendix A for the one-dimensional derivation), which results in of the dynamics at iα on positions j β to which it is not directly the Langevin equation, connected, but represents the neighborhood of iβ nodes. The diffusion term (11) may be rewritten in a more compact α = + η˙i e ξs, (7) form where ξ is uncorrelated Gaussian noise, η is either the number α =− β β αβ − α β − β x˙i diff D xj ρij δ xi ki δij aij . (12) of individuals or the fraction of individuals, and the drift and β j diffusion terms (do not confuse the latter with the diffusive dynamics studied in this section) are respectively Note that, if one extracts the node degree from the second parenthesis—the network Laplacian term—then it becomes +α −α β β β T | + T | k (δ − a /k ), which is similar to the first parenthesis term, = +α − −α = j i j i j i ij ij i e Ti|j Ti|j ,s , (8) αβ − α Ni ρij (δ xi ), but with constant values instead of variables. In j this way the latter term may be interpreted as a population- where the transition probabilities have to be written in terms composition-dependent Laplacian, which accounts for the of numbers or fractions depending on the choice in Eq. (7). instantaneous heterogeneity in population sizes and fractions The first term in Eq. (8) (drift) accounts for the deterministic of individuals in the network.

022301-3 R. J. REQUEJO AND A. DIAZ-GUILERA´ PHYSICAL REVIEW E 94, 022301 (2016)

(a) (b) 1 α 1

γ i x

x α 0.8 x 0.8 β x β 0.6 x 0.6

0.4 0.4

0.2 0.2

0 0 Fraction of individuals, 0 50 100 150 200 250 0 20 40 60 80 100

0.6 0.6

0.4 0.4

0.2 0.2

0 0 Fraction of individuals, 0 50 100 150 200 250 0 50 100 150 200

Time, t Time, t

FIG. 2. Intralayer diffusive dynamics of a system consisting of two layers with diffusion coefficients Dα = 10−1 and Dβ = 10−2, and two sites, i = 1, 2. Initially there are N individuals in nodes 1α and 2β , which are allowed to diffuse within its own layer (note that N may take any value in the situation depicted, provided that it is the same in both layers, as the fractional character of ρ makes it vanish); no interlayer α = process is acting on the system. In order to fulfill the constraint α xi 1, filled and empty circles, as well as filled and empty triangles, have to add up to one. (a), (b) dynamics of Eq. (11), with (a) ρij calculated exactly analytically and (b) setting ρij = 1; (b), (c) dynamics of Eq. (17) = with (c) ρij calculated exactly analytically and (d) ρij 1; Cases (c) and (d) do not keep the normalization of the fractions. The dynamics γ = given in (a) and (b) maintain the restriction γ xi 1 due to the quadratic term in Eq. (11). However, while case (a) describes accurately the expected dynamics, (b) shows an asymmetry in the final state, which is not expected due to pure intralayer diffusion, but the result of an induced evolutionary pressure due to the different diffusive velocities and the restriction ρij = 1.

α Now, it is possible to infer the transition probabilities of the of individuals xi , which give rise to Eq. (12), resulting in microscopic process from the emergent macroscopic dynamics [Eq. (12)], which may be used in Markovian analysis [12]. The +α|α = −α|α = α α α − α Ti|i 0 Ti|i D ki xi 1 xi macroscopic deterministic dynamics emerge from the drift +α|α = α α α − α −α|α = term in Eq. (8), which by extrapolation to two dimensions Ti|j D ρij aij xj 1 xi Ti|j 0 results in +α|β β β α β −α|β T | = D k x x T | = 0 α +α|β −α|β i i i i i i i x˙ = T | − T | , (13) i i j i j +α|β = −α|β = β β α β β j Ti|j 0 Ti|j D ρij aij xi xj , (15) where where we have omitted the subscript “diff” for simplicity. +α|β α α α β Note that the transition probabilities depend on both the T | = T x → x + x x (14) i j i i i j layers (α) and the sites (i), and that the symmetry, which is the transition rate of increase (decrease, by changing all is present for mutation transition probabilities, which makes signs) of the fraction of agents in iα given the fraction of them keep local population sizes constant (see Appendix B), is β β −α|β = +β|α agents in j (see Appendix C). Note that xj mayalsovaryin broken in general settings, T T . Such asymmetry is the same process, but such variation is speciﬁed in its related however expected, as diffusion implicitly needs variable local transition rate, and not explicitly written here for simplicity. population sizes (although the total number of particles in Then, we can compare Eq. (12) and Eq. (13) (see Ap- the multiplex is conserved), and assuming constant population pendix C for the derivation), and infer the bidimensional sizes locally may have unexpected effects, as shown in the next transition probabilities describing the variation of the fractions section.

022301-4 REPLICATOR DYNAMICS WITH DIFFUSION ON . . . PHYSICAL REVIEW E 94, 022301 (2016)

The transition probabilities in Eq. (15), together with The case in which ρij is forced to be equal to a certain those that give rise to the replicator equation with mutations value in Eq. (11) may happen only if the conservation of (Appendixes A, B) complete the microscopic description of particles due to the diffusive process does no longer hold. In the evolutionary process for agents in a multiplex. Let us such case, which may be due to a fast (compared to diffusion) finally write down the deterministic replicator dynamics (see and neutral evolutionary process acting on the system, the Appendix A) with additive mutations (see Appendix B) and difference in diffusive velocities transforms into different diffusion in a multiplex: diffusive pressures, which induce different selective pressures while the system decays to the equilibrium. α = α α − ¯ + β βα − α αβ x˙i xi fi fi xi qi xi qi The different selective pressures are induced by the fact β that fast diffusing individuals are increasing their frequency in a site compared to slower diffusing ones, and then the − Dβ xβ ρ δαβ − xα kβ δ − aβ , (16) j ij i s ij ij increased fraction is fixated by the neutral selective process, β j which only renormalizes the population size without altering α α ¯ = the proportions. This is similar to a Wright-Fisher process, where fi is the fitness of individuals in position i , fi α in which the population reproduces during the reproduction α fi is the mean fitness of individuals in i across layers αβ period according to their fitness and then, keeping the and qi is the mutation rate, which accounts for transitions of agents between α and β layers in site i. For the case of proportions of individuals, the population size is renormalized mutations coupled to reproduction substitute the first two terms to its initial value. In our case, however, the variation in the in the previous equation by the so called replicator-mutator fractions is due to the diffusion of the agents and the continuous equation [Eq. (B7)]. renormalization of the population size is due to the neutral Remarkably, the nonlinear effects due to diffusion are evolutionary process, thus favoring the fast diffusive strategy, α as shown in Fig. 2(b). all contained in the xi in the first parenthesis of the latter term in Eq. (16). There are, however, some situations in which the nonlinearity disappears, and in which the analytic B. Slow population size change and quasineutral selection calculations can be simplified. Let us discuss them, as well as The equation describing the evolution of the fraction of their implications, with special emphasis on the appearance of particles present at each point may in principle be simplified hidden selective pressures. whenever, starting from a situation near the equilibrium, the variation of the number of agents is so slow that it can be IV. RECOVERING LINEAR DIFFUSION AND neglected, N˙i /Ni → 0. This is equivalent to assuming that α ≈ α SIMPLIFYING THE ANALYTICS x˙i n˙i /Ni , which results in

Some situations allow us to recover linear diffusion, as well N x˙α = Dα aα j xα − xα =−Dα ρ Lα xα. as to simplify the analytic calculations. Here, we analyze three i diff ij j i ij ij j Ni of such scenarios. The first one refers to a situation in which j j population size ratios are all forced to be constant; note that this (17) includes as a subcase having constant populations, situation when the second term in Eq. (10) disappears. Remarkably, However, Eq. (17) does not generally keep the proper nor- β = in this situation a hidden selective pressure appears favoring malization for the fractions β xi 1, as it can be easily the increase of fast diffusing strategies. The second scenario proven with a simple example. Let us assume a multiplex refers to slow population change, a situation that approximates network formed by identical networks with identical diffusion the constant population sizes scenario. The last subsection coefficients (for two-dimensional spatial networks, this equals explores the situation in which the population ratio can be the Fisher-Kolmogorov reaction-diffusion scenario for gene expressed as some analytic function. In that case, time scales wave-front propagation [44–46]). In this case, which is separation allows us to write some formulas, which simplify analogous to a monolayer network, there is no dependence the calculations. of the adjacency matrix terms and diffusion coefficients on the layer index α, which now serves only to identify the different strategies present in each node. Hence, by summing A. Constant population ratios (or sizes) induce hidden Eq. (17) over layers (or strategies) and noting that xβ = 1, selective pressure β i we obtain the condition x˙β = (D/N ) a (N − N ) = Let us first study the case in which some mechanism makes β i i j ij j i (D/Ni ) aij Nj − ki Ni , which is only equal to zero, i.e., ρij = Nj /Ni → c, where c is a constant; for simplicity, we j = satisfies the dynamical constraint, whenever will assume c 1, which includes the usual assumption in evolutionary game theoretical studies of constant population j aij Nj sizes [3–5,41–43]). This case is shown in Fig. 2(b), where Ni = or Ni = Nj . (18) k the restriction on the addition of fractions of individuals to i one during the dynamical evolution of the system is fulfilled. These two restrictions are equivalent to requiring that = However, the final state is not a symmetric one in which there j Lij Nj 0, which could be implemented or engineered are N/2 individuals in each node, as it happens without the in the system, but it is not aprioriexpected to happen restriction on the quotient of population sizes. Why does this as a self-organizing feature. Furthermore, simulations using happen? a system with two nodes and two layers confirm that the

022301-5 R. J. REQUEJO AND A. DIAZ-GUILERA´ PHYSICAL REVIEW E 94, 022301 (2016) normalization is not fulfilled using Eq. (17), as shown in is influencing the present state. Hence, Eq. (22)isamemory Figs. 2(c), 2(d). Hence, the construction of a microscopic kernel of the past states of the system. model that describes the macroscopic diffusive dynamics of As the memory kernel has an exponential form, although fractions of individuals cannot be done by simply rewriting the entire history is contained in it, the influence of past the transition probabilities in Eq. (5) in terms of fractions. states decays very fast with time compared to present states. The case of slow population change is equivalent to This can be easily proved noting that, given two time = assuming that the second term in Eq. (11) vanishes lapses beginning at t 0 and ending at t1 and t2 >t1,the t 0 1 (f¯ −f¯ )dt β β memory kernels are ρ (t ) = ρ e 0 j i and ρ (t ) = β → ij 1 ij ij 2 D ρij L x 0 (19) t ij j t1 2 ¯ − ¯ 0 (f¯ −f¯ )dt (fj fi )dt β j ρ e 0 j i e t1 . The quotient between them is ij t2 ¯ − ¯ t (fj fi )dt and is hence only slightly influencing the dynamics described ρij (t2)/ρij (t1) = e 1 , which is a memory kernel of the by the first term. In such case, the diffusion term approaches time lapse between t1 and t2 and does not take into account Eq. (17). Note however that, if we assume that the term above is the time lapse between 0 and t1. Hence, we may always make strictly zero, we recover the case of strictly constant population t2 = t1 + dt to express the quotient as an instantaneous integral sizes, and hence hidden selective pressures may appear, as of the fitness differences, which allows for computation of the explained in the previous subsection. dynamics without keeping track of the entire history of the From an evolutionary perspective, small perturbations system. introduced in the neutral selection limit satisfy the conditions The opposite limit to slow diffusion is the fast diffusion leading to the slow population size change approximation limit: In this case we may assume that, after a short transient, whenever diffusion is slow. If we define the quasineutral the population size variations fulfill N˙i /N˙j = κij (t) = 0,∞; α → selection limit as represented by fi 0, then the population then the approximation size at each site varies slowly, given that N˙ = N f¯ → 0 and i i i ρ = (f¯/f¯ )κ (t) (23) Dγ → 0. The quasineutral selection limit is important for two ij i j ij reasons: first, it allows for analytic calculations to be carried can be used. Furthermore, in the fast diffusion limit we may out in a similar way to the weak selection limit [47,48] and, assume that the population size variations due to replication second, this limit approaches the neutral theory of molecular and death may be negligible compared to diffusion, as evolution proposed by Kimura [49]. However, care should be in the previous subsection; in such case the conditions in taken when approaching this limit, as explained above. Eq. (18)—equal populations across all sites or a local mean field describing the population sizes—may be prone to happen, C. Time-scales separation although again, this may introduce hidden selective pressures.

Let us finish exploring the situations in which ρij can be written as a function of the fractions of individuals (or their V. DISCUSSION AND CONCLUSIONS fitness, which are determined by such fractions once the payoff When dealing with replicator dynamics the equations are matrix is known) and time due to time-scales separation. written in terms of the fractions of strategies in the population Whenever we are able to write the population size depen- in every node, although diffusion is usually described in dence of the diffusion term as terms of the populations themselves. We have solved this = γ arbitrariness by writing the complete diffusion equations ρij ρij xk ,t (20) for the fractions, but this makes the diffusion process have the entire replicator dynamics in the multiplex, including nonlinear contributions. On the one hand, it introduces the replication and deaths, mutations, and diffusion, can be written ratio of population sizes, which is clearly state dependent. On as a function of the state of the system, given by the fractions of the other hand, it generates an additional quadratic term in the individuals in each node, and of the structural terms (diffusion population fractions. coefficients and Laplacian of the multiplex). The extra nonlinear diffusive term in Eq. (11) takes into In the cases in which it is not possible to write an explicit account the state of the vicinity of the focal node and its dependence for ρij as above, there are at least two situations in equivalent nodes in all layers (extended neighborhood), and which time-scales separation allows for approximations that not only its direct neighbors (directly connected nodes). This take such form. The first one is whenever diffusion is slow is relevant for the calculation of the evolution of the fractions of and most of the population size change is due to replication individuals at each site, and contrasts with the usual diffusion and death. In this case it is easy to prove that the differential term, which only takes into account the state of directly equation connected nodes within each layer. We have finally explored the recovery of the linear scenario, ρ˙ij ≈ (f¯j − f¯i )ρij (21) finding some situations in which the analytics are simplified. governs the evolution of population sizes. Note that the One of such scenarios happens whenever population sizes solution of this equation is an exponential integral, are constant across sites (due to environmental saturation,

0 t (f¯ −f¯ )dt for instance). In that case the nonlinearity disappears, but ρ (t) = ρ e 0 j i , (22) ij ij the diffusion process induces an extra evolutionary pressure which implies that the system has memory. More precisely, acting while the system is out of equilibrium. This has been the entire history of the difference of mean population ﬁtness argued to happen because a hidden selective process induces a differences, which depends on the population compositions, constant renormalization of the population size, which favors

022301-6 REPLICATOR DYNAMICS WITH DIFFUSION ON . . . PHYSICAL REVIEW E 94, 022301 (2016)

+ α − α the increase of fast diffusing strategies. This latter result of individuals by one individual will be T (ni ) and T (ni ) suggests that, depending on the network architecture, the respectively, where induced evolutionary selective pressures may work so as to T + nα = T nα → nα + create extra gradients of selection acting while the system is out i i i 1 (A4) of equilibrium, which may induce, depending on the multiplex (similarly for T − with a sign change). With this notation, it is architecture, an unexpectedly complex phenomenology. possible to write the master equation α + − α P ni ,t 1 P ni ,t ACKNOWLEDGMENTS = P nα − 1,t − 1 T + nα − 1 + P nα + 1,t − 1 The authors acknowledge discussions about the ideas i i i × − α + − α − − α + + α expressed in the paper with N. E. Kouvaris and J. Camacho, T ni 1 P ni ,t 1 T ni T ni , and the commentaries of the anonymous referees. This work (A5) was supported by the LASAGNE (Contract No. 318132) and α MULTIPLEX (Contract No. 317532) EU projects. The authors which describes the evolution of ni . Now, let us assume that = α acknowledges financial support from Generalitat de Catalunya the total number of agents in site i is Ni α ni 1. We do (Grant No. 2014SGR608) and Spanish MINECO (Grant No. not require it to be infinite, but just large enough, so that we FIS2012-38266). can make a continuous approach without neglecting finite-size fluctuations. In this case it is possible to define the rescaled variables xα = nα/N , τ = t/N and ρ(xα,τ) = N P (nα,t). APPENDIX A: MICROSCOPIC LOCAL DERIVATION i i i i i i i For simplicity, let us assume that there are only two strategies OF THE REPLICATOR EQUATION α = present in the population, and the constraint α xi 1; we The replicator equation in a well-mixed population [no can then expand the master equation in a one-dimensional network structure, Eq. (1)] can be derived—for large popu- Taylor expansion for Ni 1 (the derivation is similar for the lation sizes accepting smooth derivatives—by differentiating three- [42] and n-strategies cases [43] by using a multivariate the fraction of individuals xα, Taylor expansion), giving rise to

α α α ˙ d d d(x )rep d(n /N)rep α n˙ N α =− α α = = x − (A1) ρ xi ,t α e xi ρ xi ,t dt dt nα N dt dxi 1 d2 and assuming that the fitness of the individuals corresponds to + s2 xα ρ xα,t . (A6) the instantaneous per-capita growth rate of each strategy due 2 α 2 i i dxi to replication and deaths [41], As the previous equation has the form of a Fokker-Planck n˙α f α(x) = , (A2) equation, it is possible to transform it into the Langevin nα equation where x ={xα} is the state vector of the population, and α = α + α x˙i e xi s xi ξ, (A7) the mean fitness of the aggregated population fulfills f¯ = xαf α = N/N˙ . If the population size is constant, N˙ = 0, where ξ is uncorrelated Gaussian noise and the drift and α diffusion terms are then the per-capita growth rate is proportional to the difference between trait and mean population fitness, α = + α − − α e xi T xi T xi ,

n˙α (A8) f α(x) − f¯(x) = , (A3) T + xα + T − xα nα s xα = i i , i N being hence the replicator equation applicable for constant and i variable populations with slightly different fitness definitions. respectively accounting for the deterministic behavior and the As it will be proven in the following, the replicator dynamics stochastic effects. also emerge as the macroscopic description of some micro- Note that whenever the transition probabilities can be scopic dynamics. written as Let us start the derivation of the replicator equation by + − T xα = xαR xα ,T xα = xαD xα , (A9) assuming that there is a microscopic process, which can be i i i i i i α described by some transition probabilities between states, and the drift term e(xi ) looks like a replicator equation. This that such states are well defined. As we will assume that is indeed the only term acting in the thermodynamic limit individuals of different types diffuse through different network N →∞, and whenever the size of large populations does not α α + α architectures, each of such networks being part of a multiplex increase or decrease too fast, Ni xi [R(xi ) D(xi )]. In structure, let us introduce now the related notation: As before, this cases the Langevin equation simplifies to each agent type will be labeled by the superscript α of the x˙α = xα · R xα − D xα (A10) layer to which it belongs (related with its strategy), and the i i i i α α subscript i will refer to a site in such layer. and the terms R(xi ) and D(xi ) act as the replication and α α − ¯ The probability for node i to be at time t in a state death components of the fitness difference fi fi [compare α α with ni individuals of α type will be denoted as P (ni ,t), Eqs. (A10) and (1)]. Hence, any process in which the transition and the probability of increasing or decreasing such number probabilities can be factorized as shown in Eq. (A9), can be

022301-7 R. J. REQUEJO AND A. DIAZ-GUILERA´ PHYSICAL REVIEW E 94, 022301 (2016) written as a replicatorlike system with fitness obtained as the multiplicative factor, which relates to the temporal scale of the solution of the equation dynamics. Whenever mutations happen as a strategy change at any α − ¯ = α − α fi fi R xi D xi . (A11) point during the lifetime of the individuals, they can be This property has a particularly useful value: by using the latter introduced as additive terms [4]oftheform equation and decomposing fitness in payoff components, it is +α|β β βα −α|β +β|α (T )mut = x q , (T )mut = (T )mut (B4) possible to combine several processes, as virus spread (linear models) and cultural reproduction (frequency dependence, to the transition rates in Eq. (B3), where the coupled mu- αβ = αβ αβ usually nonlinear), into a unified evolutionary framework. tations may be eliminated by setting qi δ (with δ the Kronecker’s δ). The introduction of the additive term in APPENDIX B: MICROSCOPIC LOCAL UPDATING RULES the transition rates gives rise to the extra additive term in AND MUTATIONS the replicator dynamics α = β βα − α αβ Whenever there is an arbitrary number of strategies in the x˙i mut xi qi xi qi . (B5) population, the transition rates in Eq. (A9) can be decomposed β in additive terms in a way in which each term relates to the This term describes the effect of random mutations or contribution of each of the strategies. In this way Eq. (A7) can equivalently of random exploration of strategies [50]in be generalized to the evolutionary process. For the case of equal symmetric αβ βα +α|β −α|β = = x˙α = T − T , (B1) mutations, i.e., qi qi μ, between all strategies this i i i term simplifies to β α α + | = − α β x˙i mut μ 1 Lxi , (B6) where Ti is the transition rate of increase of the fractions of individuals in i,α corresponding to the increase in one unit where L is the number of strategies [51]. due to the action of agents in β, Let us finally recall that, when mutations are coupled to the reproductive dynamics as in Eq. (B3), then Eq. (B1) gives rise +α|β = α → α + β Ti T ni ni 1 ni , (B2) to the replicator-mutator equation, and decreases them by one unit for the minus sign (note that α β β βα α x˙ = x f q − x f¯i . (B7) this may imply a variation in the local population size N ). The i rep,mut i i i i i β exact choice of the transition rates depends on the microscopic dynamics. These transition rates may be written as [42], The diffusion term in Eq. (12) could also be added to this equation to represent situations where only newborns mutate. +α|β = γ β +γ |β γα −α|β = +β|α Ti xi xi gi qi ,T T (B3) γ APPENDIX C: DERIVATION OF THE TRANSITION whenever the transitions depend on a big number of random PROBABILITIES FOR THE NONLINEAR DIFFUSION interactions between agents in the same site (local well-mixing TERM assumption) in which one individual replaces another (defined The microscopic dynamics that give rise to the nonlinear by the second condition) and there are mutations between γ equation describing the dynamics of diffusion for fractions of γα ±γ |β and α individuals at a rate qi . Factor g contains the individuals [Eq. (12)] is defined by the transition probabilities information about how the microscopic updating rule acts: it determining the increase or decrease of the number of indi- states the exact mechanism by which one strategy increases or viduals, as shown in Eq. (13). In order to infer such transition decreases due to the action of another. probabilities, which complete the bottom-up description, we Two probabilistic microscopic dynamics are usually inves- can start expanding Eq. (12)as tigated, which give rise to the replicator dynamics (assuming γα = γα α =− β β αβ β − αβ β − β α qi δ ). The first one is the modified Moran process, x˙i diff D xj ρij δ ki δij δ aij ki xi δij where one randomly chosen individual is assumed to die, β j and another individual, chosen according to a probability + β α proportional to its fitness, reproduces. This process is defined aij xi (C1) +α|β = α ¯ −α|β = +β|α by gi fi /fi and gi gi . The second process Then, we can split the double sum on the four contributions is proportional imitation, where one individual compares its corresponding to the terms arising from iα, j α, iβ and j β strategy to another one, both chosen at random in the mean field (assuming that α = β and i = j), obtaining limit, and changes its strategy with a probability increasing α =− α α α − α + α α α − α linearly with the difference of the payoffs between them. This x˙i diff D ki xi 1 xi D aij ρij xj 1 xi +α|β = + α − β j=i process is defined by gi (1/2)[1 (fi fi )/ fs,max] − | + | α β β α β β β β and g = g , where f s,max is the maximum fitness + β α − β α i i D ki xi xi D aij ρij xi xj , difference and keeps the proper normalization. In both β=α j=i β=α − | + | cases the symmetry condition g α β = g β α ensures that i i (C2) the evolutionary process maintains a constant population, and the dynamics result in the replicator equation, up to a respectively.

022301-8 REPLICATOR DYNAMICS WITH DIFFUSION ON . . . PHYSICAL REVIEW E 94, 022301 (2016)

By performing the same kind of split in Eq. (13) we find −α|α = and the transition rates are Ti|j 0 and α α + | n n + 1 +α|α −α|α +α|α −α|α α α i i α α α α α α = − + − T | = T → x = D a ρij x 1 − x . x˙i Ti|i Ti|i Ti|j Ti|j i j j ij j i Ni Ni + 1 j=i + | − | + | − | (C5) + T α β − T α β + T α β − T α β . i|i i|i i|j i|j The limits, as before, can be easily proven to behave in the β=α j=i β=α correct way. (C3) Then, for the influence of iβ on iα (β = α), we have to note β that agents diffusing away from i decrease the denominator α = α γ β Then, since the terms relate to the influence of different of the fraction xi ni / γ ni , since they are decreasing ni , γ α α nodes k on the focal node i , we can compare term by and hence the fraction xi increases. Therefore, the transition −α|β = term both equations above, finding that each term in the first rates are Ti|i 0 and equation corresponds to the substraction of a pair of terms in α α + | n n the second. α β = i → i β = β β α β Ti|i T xi D ki xi xi . (C6) In order to finally find the right transition terms, we need Ni Ni − 1 to take into account the physical constraints in the system. Let Finally, individuals diffusing away from j β (j = i and β = us analyze them one by one. α) are increasing the number of individuals in iβ , and hence First, we can focus on the first term on the above equations. increasing the denominator in the fraction of xα (opposite as This term relates to the influence of iα on its own dynamics. i α in the previous case). The influence is then so as to decrease Since the agents in i are diffusing away from that position, the α +α|β = +α|α = the fraction of individuals at i and hence Ti|j 0 and influence is necessarily negative, and hence Ti|i 0, and α α − | n n α β = i → i β = β β α β Ti|j T xj D aij ρij xi xj . (C7) α α + − | n n − 1 Ni Ni 1 α α = i → i α = α α α − α Ti|i T xi D ki xi 1 xi , Ni Ni − 1 From the transition probabilities above we can derive the (C4) diffusion term corresponding to the stochastic effects in the Langevin equation [extrapolating s to two dimensions in Eq. (8)], which is | α where xi denotes a dependence, not implying that such term β β α + − α αβ β + β is constant (indeed, it varies in the process). Note also that β j D xj ρij xi 1 2xi δ aij ki δij α = s = . whenever xi 0, the transition rate is zero, as expected due to N α = i the absence of agents diffusing away. Furthermore, if xi 1, the transition rate is again zero, as expected due to the fact (C8) that agents diffusing away decrease the number of agents, but This term can be added to Eq. (12)—multiplied by white leave the fraction unchanged. Gaussian noise—in order to account for the stochastic effects α Now, let us focus on the influence of j on the dynamics of introduced by the diffusive process in situations in which the α α i (j = i). Since agents diffusing away from j are increasing thermodynamic limit cannot be assumed, or population sizes α the number of agents in i , the influence is necessarily positive, vary fast.

[1] E. Bonabeau, Proc. Nat. Acad. Sci. USA 99, 7280 (2002). [10] R. J. Requejo and J. Camacho, Phys.Rev.Lett.108, 038701 [2] V. Grimm, E. Revilla, U. Berger, F. Jeltsch, W. M. Mooij, (2012). S. F. Railsback, H.-H. Thulke, J. Weiner, T. Wiegand, and [11] D. Vilone, A. Robledo, and A. Sanchez,´ Phys. Rev. Lett. 107, D. L. DeAngelis, Science 310, 987 (2005). 038101 (2011). [3] D. Helbing, Theory and Decision 40, 149 (1996). [12] R. J. Requejo, J. Camacho, J. A. Cuesta, and A. Arenas, Phys. [4] D. Helbing, in Game Theory, Experience, Rationality, edited by Rev. E 86, 026105 (2012). W. Leinfellner and E. Kohler¨ (Kluwer Academic, Dordrecht, [13] K. M. Page and M. A. Nowak, J. Theor. Biol. 219, 93 (2002). 1998), pp. 211–214. [14] P. D. Taylor and L. B. Jonker, Math. Biosc. 40, 145 (1978). [5] A. Traulsen, J. C. Claussen, and C. Hauert, Phys.Rev.Lett.95, [15] J. Hofbauer, P. Schuster, and K. Sigmund, J. Theor. Biol. 81, 238701 (2005). 609 (1979). [6] A. Traulsen, J. C. Claussen, and C. Hauert, Phys. Rev. E 74, [16] J. Maynard Smith and G. R. Price, Nature (London) 246, 15 011901 (2006). (1973). [7] J. Maynard Smith, Evolution and the Theory of Games (Cam- [17] J. Maynard Smith, J. Theo. Biol. 47, 209 (1974). bridge University Press, Cambridge, 1982). [18] J. Hofbauer, Nonlinear Anal. Theor. Meth. Appl. 5, 1003 (1981). [8] K. Sigmund, The Calculus of Selﬁshness (Princeton University [19] J. Hofbauer and K. Sigmund, Evolutionary Games and Popula- Press, Princeton, 2010). tion Dynamics (Cambridge University Press, Cambridge, 1998). [9] R. J. Requejo and J. Camacho, J. Theor. Biol. 272, 35 (2011). [20] G. R. Price, Nature (London) 227, 520 (1970).

022301-9 R. J. REQUEJO AND A. DIAZ-GUILERA´ PHYSICAL REVIEW E 94, 022301 (2016)

[21] G. R. Price, Ann. Hum. Genet. 35, 485 (1972). [37] A. Sole-Ribalta,´ M. De Domenico, N. E. Kouvaris, A. D´ıaz- [22] G. R. Price, Ann. Hum. Genet. 36, 129 (1972). Guilera, S. Gomez,´ and A. Arenas, Phys. Rev. E 88, 032807 [23] A. Traulsen, Evolution 64, 316 (2010). (2013). [24] C. P. Roca, J. A. Cuesta, and A. Sanchez,´ Phys.Rev.Lett.97, [38] M. DeDomenico, A. Sole-Ribalta, E. Cozzo, M. Kivela, Y. 158701 (2006). Moreno, M. A. Porter, S. Gomez, and A. Arenas, Phys. Rev. [25] C. P. Roca, J. A. Cuesta, and A. Sanchez,´ Phys. Life Rev. 6, 208 X 3, 041022 (2013). (2009). [39] C. Granell, S. Gomez,´ and A. Arenas, Phys. Rev. Lett. 111, [26] J. Lovelock and L. Margulis, TELLUS 26, 2 (1974). 128701 (2013). [27] R. J. Requejo and J. Camacho, Phys.Rev.E85, 066112 [40] N. E. Kouvaris, S. Hata, and A. D´ıaz-Guilera, Sci. Rep. 5, 10840 (2012). (2015). [28] M. A. Nowak and R. M. May, Nature (London) 359, 826 [41] G. SzaboandG.F´ ath,´ Phys. Rep. 446, 97 (2007). (1992). [42] A. J. Bladon, T. Galla, and A. J. McKane, Phys. Rev. E 81, [29] C. P. Roca, J. A. Cuesta, and A. Sanchez,´ Phys.Rev.E80, 066122 (2010). 046106 (2009). [43] A. Traulsen, J. C. Claussen, and C. Hauert, Phys. Rev. E 85, [30] F. C. Santos and J. M. Pacheco, Phys.Rev.Lett.95, 098104 041901 (2012). (2005). [44] R. A. Fisher, Ann. Eugenics 7, 355 (1937). [31] F. C. Santos, M. D. Santos, and J. M. Pacheco, Nature (London) [45] A. N. Kolmogorov, I. G. Petrovsky, and N. S. Piskunov, 454, 213 (2008). Bull. Univ. Moscow, Ser. Int. A 1, 1 (1937) [reprinted in [32] H. Ohtsuki and M. Nowak, J. Theor. Biol. 243, 86 (2006). V. M. Tikhomirov (ed.), Selected Works of A. M. Kolmogorov [33] J. Y. Wakano, M. A. Nowak, and C. Hauert, Proc. Nat. Acad. (Kluwer, Amsterdam, 1991)]. Sci. USA 106, 7910 (2009). [46] J. Fort and V. Mendez,` Rep. Prog. Phys. 65, 895 (2002). [34] M. Kivela, A. Arenas, M. Barthelemy, J. P. Gleeson, Y. Moreno, [47] M. A. Nowak, A. Sasaki, C. Taylor, and D. Fudenberg, Nature and M. Porter, J. Complex Networks 2, 203 (2014). (London) 428, 646 (2004). [35] A. Cardillo, J. Gomez-Garde´ nes,˜ M. Zanin, M. Romance, [48] A. Traulsen and M. A. Nowak, PLoS ONE 2, e270 (2007). D. Papo, F. del Pozo, and S. Boccaletti, Sci. Rep. 3, 1344 [49] M. Kimura, Nature (London) 217, 624 (1968). (2013). [50] A. Traulsen, C. Hauert, H. De Silva, M. Nowak, and K. Sigmund, [36] S. Gomez,´ A. D´ıaz-Guilera, J. Gomez-Garde´ nes,˜ C. J. PNAS 106, 709 (2009). Perez-Vicente,´ Y. Moreno, and A. Arenas, Phys. Rev. Lett. 110, [51] A. Arenas, J. Camacho, J. A. Cuesta, and R. J. Requejo, J. Theor. 028701 (2013). Biol. 279, 113 (2011).

022301-10