Multiple strategies in structured populations Corina E. Tarnitaa,b,c,1,2, Nicholas Wagea,b,1, and Martin A. Nowaka,b,d aProgram for Evolutionary Dynamics, bDepartment of Mathematics, dDepartment of Organismic and Evolutionary Biology, and cHarvard Society of Fellows, Harvard University, Cambridge, MA 02138 Edited* by Clifford H. Taubes, Harvard University, Cambridge, MA, and approved December 20, 2010 (received for review October 28, 2010) Many specific models have been proposed to study evolutionary For weak selection, all strategies have roughly the same av- game dynamics in structured populations, but most analytical erage frequency, 1/n, in the stationary distribution. A strategy is results so far describe the competition of only two strategies. Here favored by selection, if its average frequency is >1/n. Otherwise it we derive a general result that holds for any number of strategies, is opposed by selection. Our main result is the following: Given for a large class of population structures under weak selection. We some mild assumptions (specified in SI Text), strategy k is fa- show that for the purpose of strategy selection any evolutionary vored by selection if process can be characterized by two key parameters that are fi coef cients in a linear inequality containing the payoff values. ðσ1akk þ akÃ− aÃk − σ1aÃÃÞþσ2ðakÃ− aÞ > 0: [1] These structural coefficients, σ1 and σ2, depend on the particular process that is being studied, but not on the number of strategies, a ¼ð =nÞ∑n a Here Ãà 1 i¼1 ii is the average payoff when both individ- n, or the payoff matrix. For calculating these structural coefficients a ¼ð =nÞ∑n a uals use the same strategy, kà 1 i¼1 ki is the average pay- one has to investigate games with three strategies, but more are k a ¼ð =nÞ∑n a off of strategy , Ãk 1 i¼1 ik is the average payoff when not needed. Therefore, n = 3 is the general case. Our main result k a ¼ð =n2Þ∑n ∑n a playing against strategy , and 1 i¼1 j¼1 ij is the aver- has a geometric interpretation: Strategy selection is determined by age payoff in the population. The parameters σ and σ are struc- fi 1 2 the sum of two terms, the rst one describing competition on the tural coefficients that need to be calculated for the specific edges of the simplex and the second one in the center. Our for- evolutionary process that is investigated. These parameters de- mula includes all known weak selection criteria of evolutionary pend on the population structure, the update rule, and the muta- fi games as special cases. As a speci c example we calculate games tion rate, but they do not depend on the number of strategies or on on sets and explore the synergistic interaction between direct rec- the entries of the payoff matrix. iprocity and spatial selection. We show that for certain parameter How can we interpret this result? Let xi denote the frequency of values both repetition and space are needed to promote evolution strategy i. The configuration of the population (just in terms of of cooperation. frequencies of strategies) is given by a point in the simplex Sn, fi ∑n x ¼ which is de ned by i¼1 i 1. The vertices of the simplex cor- volutionary games arise whenever the fitness of individuals respond to population states where only one strategy is present. Eis not constant, but depends on the relative abundance of The edges of the simplex correspond to states where two strategies strategies in the population (1–7). Evolutionary game theory is are present. In the interior of the simplex all strategies are present. a general theoretical framework that can be used to study many Inequality [1] is the sum of two terms, both of which are linear in biological problems including host–parasite interactions, eco- the payoff values. The first term, σ1akk þ akà − aÃk − σ1aÃÃ, systems, animal behavior, social evolution, and human language describes competition on the edges of the simplex that include (8–18). The traditional approach of evolutionary game theory k A fi strategy (Fig. 1 ). In particular, it is an average over all pairwise uses deterministic dynamics describing in nitely large, well- comparisons between strategy k and each other strategy, weighted mixed populations. More recently the framework was expanded fi σ σ ða − aÞ fi by the structural coef cient, 1. The second term, 2 kà , to deal with stochastic dynamics, nite population size, and evaluates the competition between strategy k and all other strat- structured populations (19–32). egies in the center of the simplex, where all strategies have the Here we consider a mutation–selection process acting in same frequency, 1/n (Fig. 1B). a population of finite size. The population structure determines Therefore, the surprising implication of our main result (Eq. who interacts with whom to accumulate payoff and who com- 1) is that strategy selection (in a mutation–selection process in petes with whom for reproduction. Individuals adopt one of n strategies. The payoff for an interaction between any two strat- a structured population) is simply the sum of two competition terms, one that is evaluated on the edges of the simplex and the egies is given by the n × n payoff matrix A =[aij]. The rate of reproduction is proportional to payoff: Individuals that accu- other one in the center of the simplex. The simplicity of this mulate higher payoff are more likely to reproduce. Reproduction result is surprising because an evolutionary process in a struc- is subject to symmetric mutation: With probability 1 − u the tured population has a very large number of possible states; to offspring inherits the strategy of the parent, but with probability describe a particular state it is not enough to list the frequencies u a random strategy is chosen. Our process leads to a stationary of strategies but one also has to specify the population structure. distribution characterizing the mutation–selection equilibrium. Further intuition for our main result is provided by the concept Important questions are the following: What is the average fre- of risk dominance. The classical notion of risk dominance for quency of a strategy in the stationary distribution? Which strat- a game with two strategies in a well-mixed population is as fol- egies are more abundant than others? lows: Strategy i is risk dominant over strategy j if aii + aij > aji + ajj. To make progress, we consider the limit of weak selection. One way to obtain this limit is as follows: The rate of re- production of each individual is proportional to 1 + w Payoff, Author contributions: C.E.T., N.W., and M.A.N. designed research, performed research, where w is a constant that measures the intensity of selection; the contributed new reagents/analytic tools, and wrote the paper. limit of weak selection is then given by w → 0. Weak selection is The authors declare no conflict of interest. not an unnatural situation; it can arise in different ways: i) Payoff *This Direct Submission article had a prearranged editor. differences are small, ii) strategies are similar, and iii) individuals 1C.E.T. and N.W. contributed equally to this work. are confused about payoffs when updating their strategies. In 2To whom correspondence should be addressed. E-mail: [email protected]. such situations, the particular game makes only a small contri- This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10. bution to the overall reproductive success of an individual. 1073/pnas.1016008108/-/DCSupplemental. 2334–2337 | PNAS | February 8, 2011 | vol. 108 | no. 6 www.pnas.org/cgi/doi/10.1073/pnas.1016008108 Downloaded by guest on September 27, 2021 ABcalculate the competition of multiple strategies in a structured population for weak selection but any mutation rate, then all we have to do is to calculate two parameters, σ1 and σ2. This cal- culation can be done for a very simple payoff matrix and n =3 strategies. Once σ1 and σ2 are known, they can be applied to any payoff matrix and any number of strategies. For n = 2 strategies, inequality [1] leads to (a11 − a22)(2σ1 + σ2) +(a12 − a21)(2 + σ2) > 0. If 2 + σ2 ≠ 0, we obtain the well-known condition σa11 + a12 > a21 + σa22 with σ =(2σ1 + σ2)/(2 + σ2). Many σ-values have been calculated characterizing evolutionary Fig. 1. Our main result has a simple geometric interpretation, which is il- games with two strategies in structured populations (31). lustrated here for the case of n = 3 strategies. (A) The first term of inequality For a large, well-mixed population we know that σ1 = 1 and B 1 describes competition on the edges of the simplex. ( ) The second term of σ2 = μ, where μ = Nu is the product of population size and inequality 1 describes competition in the center of the simplex. In general, mutation rate (30). Therefore, if the mutation rate is low, μ → 0, the selective criterion for strategy 1 is the sum of the two terms. then the evolutionary success of a strategy is determined by av- erage pairwise risk dominance, akk þ akà − aÃk − aÃÃ. If the mu- tation rate is high, μ → ∞, then the evolutionary success depends If i and j are engaged in a coordination game, given by aii > aji and on risk dominance, akà − a: ajj > aij, then the risk-dominant strategy has the bigger basin of For any population structure, we can show that low mutation, attraction. In a structured population the risk-dominance condi- μ → σ → fi σa a > a σa σ 0, implies 2 0.