Proc. Natl. Acad. Sci. USA Vol. 92, pp. 3596-3600, April 1995 Evolution Types of evolutionary stability and the problem of cooperation (evolution of cooperation/evolutionary game theory/evolutionarily stable strategies/iterated prisoner's dilemma/tit for tat) JONATHAN BENDORt AND PIOTR SWISTAKt tGraduate School of Business, Stanford University, Stanford, CA 94305-5015; and *Department of Government and Politics, University of Maryland, College Park, MD 20742-8221 Communicated by Robert Axelrod, University of Michigan, Ann Arbor, MI, November 14, 1994 ABSTRACT The evolutionary stability ofcooperation is a choices, to cooperate or defect; the same payoff matrix de- problem of fundamental importance for the biological and scribes the outcomes of these choices in all periods. The players social sciences. Different claims have been made about this observe each others' choices. A pure strategy in a repeated issue: whereas Axelrod and Hamilton's [Axelrod, R. & Ham- game between two players specifies an action for one player in ilton, W. (1981) Science 211, 1390-1398] widely recognized a current period as a function of the two players' past sequence conclusion is that cooperative rules such as "tit for tat" are of moves. A mixed strategy is a probability distribution on a set evolutionarily stable strategies in the iterated prisoner's of pure strategies. In dynamical analyses, it is assumed that dilemma (IPD), Boyd and Lorberbaum [Boyd, R. & Lorber- strategies that obtain higher payoffs reproduce at a higher rate. baum, J. (1987) Nature (London) 327, 58-59] have claimed (See ref. 7 for a discussion of the standard model and of its that no pure strategy is evolutionarily stable in this game. modifications.) Here we explain why these claims are not contradictory by In the one-shot PD when one player cooperates while the showing in what sense strategies in the IPD can and cannot be other defects, denote the payoff to the defector as T and the stable and by creating a conceptual framework that yields the cooperator as S; their payoff is R when both cooperate and P type of evolutionary stability attainable in the IPD and in when both defect. The PD structure of the game is given by the repeated games in general. Having established the relevant assumption that T > R > P > S. In the repeated version we concept of stability, we report theorems on some basic prop- further assume that 2R > T + S. The essence of the dilemma erties of strategies that are stable in this sense. We first show in a one-shot PD is that defection yields a higher payoff that the IPD has "too many" such strategies, so that being regardless of the opponent's action (T > R and P > S), yet stable does not discriminate among behavioral rules. Stable mutual defection is less rewarding than mutual cooperation (P strategies differ, however, on a property that is crucial for < R). Thus, in a one-shot PD, defection is the unique their evolutionary survival-the size of the invasion they can equilibrium strategy. Likewise perpetual defection is the resist. This property can be interpreted as a strategy's evo- unique best strategy when the game is iterated but the future lutionary robustness. Conditionally cooperative strategies does not matter "too much"; i.e., when w, the probability of such as tit for tat are the most robust. Cooperative behavior playing one more round, is low. The emergence and stability supported by these strategies is the most robust evolutionary of cooperation become meaningful issues only when w is equilibrium: the easiest to attain, and the hardest to disrupt. sufficiently large. Evolutionary Stability as Defined by Payoff Conditions: In the last decade, one of the most famous theoretical prob- Conceptual Issues lems of the biological and social sciences-how humans and other animals can cooperate despite temptations to defect- In games where w is sufficiently large, the conventional wisdom has attracted much interest. The issue has been approached via had been that conditionally cooperative strategies such as TFIl a new framework, evolutionary game theory, which has pro- are evolutionarily stable. This point was first made by Axelrod vided one of the most interesting ways of thinking about this and Hamilton (4) in their celebrated article in Science. The problem. However, different claims have been made about the claim-a deductive one-was backed by a proof showing that evolutionary stability of cooperation, and these solutions seem for all strategies j, V(TF[, TFT) 2 V(j,TFT), where V(i, j) to contradict each other. Several well-known though appar- denotes strategy i's payoffwhen it plays strategyj. (We use this ently incongruent results have been reported in many re- framing of the result, since this payoff condition was later nowned books (1-3) and journals (4-8). Clearly, those who adopted by Axelrod to define collective stability; see Eq. 1.) claim tit for tat (TFT) to be evolutionarily stable and those who This, however, does not imply that TFT meets Maynard claim it is not cannot both be right-if,.of course, they are using Smith's condition (see Eq. 2) for an evolutionarily stable the same concept of stability. Our first objective, then, is to strategy. clarify these seemingly contradictory claims by showing in what Indeed, Axelrod later (2, 6) distinguished collective stability sense strategies are and are not stable in the iterated prisoner's from evolutionary stability, defining strategy i as collectively dilemma (IPD) and other repeated games. Our second objec- stable if for all strategies j, tive is to report theorems, within the clarified conceptual framework, that establish some basic properties of strategies V(i, i) V(j, i). [1] which are stable in the IPD. The problem of cooperation among unrelated animals is On the other hand, by Maynard Smith's condition, strategy i is typically (1, 2, 4-6, 9-11) modeled as a population of indi- evolutionarily stable if, for all strategies j 0 i, viduals engaged in random pairwise prisoner's dilemmas V(i, i) 2 i), and if V(i, i) = V(j, i), (PDs). The players have a constant probability, w, of meeting V(j, in the next period (0 < w < 1). In each period there are two then V(i,j) > V(j, j). [2] The publication costs of this article were defrayed in part by page charge Abbreviations: PD, prisoner's dilemma; IPD, iterated PD; TFT, tit for payment. This article must therefore be hereby marked "advertisement" in tat; TF2T, tit for two tats; STFT, suspicious TFT; PFR, proportional accordance with 18 U.S.C. §1734 solely to indicate this fact. fitness rule. 3596 Downloaded by guest on October 1, 2021 Evolution: Bendor and Swistak Proc. Natl. Acad. Sci. USA 92 (1995) 3597 In fact, while Axelrod turned to distinguishing collective from three strategies-and a sufficiently high w, of course-TF2T evolutionary stability, others kept referring to TFT as evolu- has the highest fitness. tionarily stable in the standard sense of condition 2. Perhaps Yet what Boyd and Lorberbaum (5) have shown is not the most conspicuous example here is John Maynard Smith. In directly related to condition 1, 2, or 3. What they have proved his seminal work (1), Maynard Smith reported the Axelrod- is that any pure strategy, no matter how common in the Hamilton finding about TFT (ref. 1, pp. 167-168) and, having population, can be outscored by some strategy in an invading reproduced their proof, concluded that it shows that TFT is an group of mutants. (The assumption of heterogeneous mutants evolutionarily stable strategy. The idea that cooperation can be is essential for Boyd and Lorberbaum's result. If there is only made evolutionarily stable via reciprocity has been further one type of mutant strategy in the population, then TFT is disseminated by the exceptionally influential books of Axelrod indeed unbeatable for sufficiently large w.) Hence Boyd and (2) and Dawkins (3). Lorberbaum refer to an evolutionarily stable strategy as one Meanwhile, however, researchers started pointing out some which, once sufficiently frequent, is unbeatable. In general, problems. One problem related to condition 2, for instance, is however, satisfying Maynard Smith's condition 2 does not that neither TFT nor any other pure strategy in the IPD can imply unbeatability. This difference between Boyd and Lor- satisfy it. Selten and Hammerstein (12) pointed this out about berbaum's notion of stability and that implied by condition 2 TFT: if a neutral mutant strategy, such as tit for two tats becomes clear when we note that a strategy which is stable in (TF2T), invades a population of TFTs, all strategies will Boyd and Lorberbaum's sense-one that, when sufficiently cooperate with all; hence, all will score the same in all games. common, is maximally fit-can be equivalently described by This, of course, violates condition 2, since this condition's the following condition 4. (The easy proof is left to the reader.) second part requires strict inequality. And this problem is not We will say that strategy i meets condition 4 if for all strategies created by some arcane construction: every nice strategy-one J, that never defects first-is a neutral mutant of TFT. While Maynard Smith's condition 2 cannot be met by any V(i, i) 2 V(j, i), and if V(i, i) = V(j, i), pure strategy in the IPD, the following weaker version of condition 2 can. We will say that i meets condition 3 if for all then for allj*, V(i,j*) 2 V(j,j*). [4] strategies j, It is now easy to see how conditions 1-4 relate to one V(i, i) 2 V(j, i), and if V(i, i) = V(j, i), another.