Magazine R503

ecology. If prey is abundant, predators cost of helping an unrelated Primer will increase for a while. But this individual? Suppose that the benefit increase reduces the abundance of (to the recipient) is b and the cost (to prey, and eventually leads to a the donor) is c, with c < b. If an Evolutionary game decrease of the predators. individual is equally likely to be theory potential recipient or donor in a Hawks and doves given encounter, then a population Karl Sigmund* and Intraspecific fights provide a first of cooperators would earn, on † example of changes in a population average, b – c per interaction, and be Martin A. Nowak that are dependent on the frequency better off than a population of of a trait. Assume that there are two defectors earning 0. But no matter Ever since Darwin read Malthus, the behaviourally distinct morphs: how the population is composed, an theory of evolution has benefited ‘hawks’ escalate a fight until the individual would always increase its from the interaction of ecology with injury of one contestant settles the economics. Evolutionary game theory outcome, ‘doves’ stick to some belongs to this tradition: it merges conventional display (a pushing The hawk–dove game population ecology with game theory. match, for instance, where injuries Game theory originally addressed are practically excluded) and take In the hawk–dove game, G is the gain in problems confronted by decision flight if the adversary escalates. If fitness resulting from winning the contest, and C is the cost in fitness due makers with diverging interests (for most contestants are doves, the hawk to an injury (see Figure 1). instance, firms competing for a morph will spread; but if most market). The ‘players’ have to choose contestants are hawks, escalating a Figure 1 between strategies whose payoff conflict will lead with probability ½ depends on their rivals’ strategies. to injury (see green box and Figure 1). Type of adversary This interdependence leads to a Even this oversimplified example Hawk Dove mutual ‘outguessing’, as with chess shows that heavily armed species (she thinks that I think that she (where the risk of injury is large) are (G-C)/2 G thinks…). There usually is no particularly prone to conventional Hawk solution that is unconditionally displays (see Figure 2).

wins payoff 0 G/2 optimal, no matter what the A wide variety of behavioural Dove Contestant who co-players are doing. traits — and purely morphological or Current Biology In the context of evolutionary physiological characters, like the biology, the two basic notions of height of trees or the length of The payoff matrix for the hawk–dove game. game theory, namely strategy and antlers — are submitted to payoff, have to be re-interpreted. A frequency-dependent selection and strategy is not a deliberate course of are amenable to game analysis. Such No morph is unconditionally better than the other. If the object of the fight is action, but an inheritable trait; payoff traits may influence conflicts of not worth the injury, then the dove morph is Darwinian fitness (average interest between two individuals, for will spread. Hawks can spread only if reproductive success). The ‘players’ instance, territorial disputes (between their frequency is below G/C. If G is less are members of a population, all neighbours), the length of the than C, a mixed population of hawks and doves will evolve. It is conceivable that competing for a larger share of weaning period (between parents and some phenotype plays a mixed strategy, descendants. offspring), or the division of parental escalating only with a certain frequency. If several variants of a trait occur investment (between male and Such a mixed strategy escalating with in a population, then natural selection female). But frequency-dependent probability G/C is evolutionarily stable; no mutant with a different propensity to leads to an increase in the frequency selection also occurs without escalate can invade. If there exists an of those variants with higher fitness. antagonistic encounters. The sex asymmetry between the two contestants If the success of a trait does not ratio is an example of this (if it is (larger versus smaller, for instance, or depend on its frequency, this will biased towards males, it pays to owner versus intruder), this will alter the game. In such cases a conditional eventually lead to the fixation of the produce daughters, and vice versa). strategy will emerge which uses the optimal variant. But if the success of a asymmetry as a cue, for instance the trait is frequency-dependent, then its The prisoner’s dilemma so-called ‘bourgeois strategy’: if owner, increase may lead to a composition of The evolution of cooperation escalate; if intruder, display. Some the population where other variants through reciprocation is a particularly intraspecific conflicts use a long assessment phase to detect an do better; this can be analysed by extensive chapter of evolutionary asymmetry in, for instance, size, by means means of game theory. This is similar game theory. Why should a selfish of a pushing match or a parallel walk. to what happens in population gene, or ‘fitness maximiser’, bear the R504 Current Biology, Vol 9 No 14

fitness by refusing help, and hence Figure 2 further round is sufficiently high, we would not see cooperation. then the presence of even a small Game theorists have number of so-called ‘retaliators’ is encapsulated this tug-of-war enough to favour cooperation. The between common good and selfish best known example of such a interest in the so-called prisoner’s retaliatory strategy is ‘tit-for-tat’. A dilemma game. In this game, a player tit-for-tat player cooperates in the has two possible strategies C (to first round and from then on always cooperate) and D (to defect). Two C repeats whatever the co-player did in players will get a reward R which is the previous round. higher than the punishment P After promoting the emergence A clash between male red deer. Only rarely obtained by two D players. But a D does the fight escalate beyond a pushing of cooperators, retaliatory strategies player exploiting a C player obtains a match. (Photograph provided by Tim are often superseded by more payoff T (temptation to defect) Clutton-Brock, Department of Zoology, tolerant strategies, for instance which is higher than R, and this University of Cambridge, UK.) ‘generous tit-for-tat’ or ‘win–stay, leaves the C player with the sucker’s lose–shift’. A generous tit-for-tat payoff S, which is lower than P. So, reproduces inside a bacterium. In player cooperates after an opponent’s because T > R > P > S, a rational this instance, one phage variant defection with a certain probability, player will always play D, which is produces less of the intracellular whereas a win–stay, lose–shift player the better move no matter what the products needed for replication than repeats the previous move after co-player is doing. the other, and thus may be said to receiving a high payoff in a round, Many species engage in defect. The payoff values can be but otherwise switches to the other interactions which seem to be of the measured with some precision: they move. Both generous tit-for-tat and prisoner’s dilemma type. Vampire satisfy the rank ordering required for win–stay, lose–shift players return to bats feed each other, monkeys the prisoner’s dilemma. cooperation after an erroneous engage in allogrooming, birds and There are several ways in which defection, whereas tit-for-tat players vervet monkeys utter alarm calls, the prisoner’s dilemma can be do not. A great many theoretical guppies and stickleback cooperate in overcome. With the phages, selection results and computer simulations predator inspection, hermaphroditic for a particular trait-group operates show that under very general sea bass alternate as egg-spenders whenever the virus population is so conditions, populations of defectors and lions engage in cooperative small that most bacterial hosts are can be invaded by small groups of hunting or joint territorial defense. invaded by only one virus. A phage stern retaliators, who pave the way But often, attempts to specify the will then most likely interact only for more tolerant populations, which, payoff values of these behavioural with members of its own clone, and in turn, can eventually ‘soften up’ to interactions lead to doubts as to this is when cooperators are better such a degree that defectors may whether one is seeing a bona fide off than defectors. More generally, take over again. prisoner’s dilemma. It is difficult to any form of associative interaction The best examples of reciprocal measure the fitness of fish darting in favours cooperation. Such association strategies may be found in human and out of shoals or of monkeys can be due to kinship, to partner societies. But here, reciprocation is hiding in the bush. It may be that choice, to ostracism of defectors or often indirect. An act of assistance is some of these are simply instances of simply to spatial structure (or returned, not by the recipient, but by by-product mutualism, in which both ‘population viscosity’). a third party. Such indirect players are best served by cooperating reciprocation can be based on and none is tempted to defect, or an Repeated interactions score-keeping by ‘discriminate instance of the so-called snowdrift Among higher organisms, interactions altruists’ (who help only those game, in which the best reply to the of the prisoner’s dilemma type are individuals that have not refused co-player’s C is a D, but the best reply probably repeated between the same help too often). Withholding help to a D is a C. For instance, if two two players — called the iterated can be costly, as it reduces the drivers are caught with their cars in a prisoner’s dilemma. Depending on discriminators’ chance of being snowdrift and one defects by not their past experience, players can helped. Simple models show that shovelling, the other is still better off break up partnerships, or vary their indiscriminate altruists can therefore by digging, rather than defecting. amount of cooperation. But even spread at the expense of discriminate Recently, however, a striking without these options, the strategy of altruists; once their frequency example of a prisoner’s dilemma type always defecting is not invariably the exceeds a certain threshold, of interaction has been uncovered for best option in the iterated prisoner’s defectors can invade. Defectors will an RNA phage — a virus that dilemma. If the probability of a reduce the proportion of Magazine R505

indiscriminate altruists, however, and inheritable male mating strategies. ‘like begets like’, so that more therefore sap the basis of their own Type A forms no lasting bonds but successful types can also produce success. Discriminate altruists will looks for sneaky matings; type B lives less successful variants and vice then be able to eliminate them again. monogamously and closely guards the versa. The complications caused by Other models based on female; and type C guards a harem of Mendelian segregation, pleiotropy evolutionary games have been used several females, of course less closely. and sexual recombination can, in to analyse the emergence of moral With still more strategies, the principle, be integrated to model systems, of division of labour or even dynamics can become ever more frequency-dependent selection of proto-languages in primitive complex, and the composition of the acting within the gene pool, but this societies. In particular, it has been population need not converge to an often results in intractable dynamics. shown that highly diverse social equilibrium: the frequencies of the But arguments from evolutionary norms can be stably sustained by strategies keep oscillating in a regular game theory can fail if genetic punishment; in many cases, or chaotic fashion. constraints come into play. (If the deviating behaviour (for instance, the optimal genotype can only be refusal to punish dissenters) cannot Long-term evolution realized by heterozygotes, for spread. In this sense, evolutionary We have so far discussed the instance, then it can never reach game theory returns to some basic evolution of frequencies of traits fixation in the population.) topics of classical game theory. under selection alone. But, in Theoretical studies tend to show, addition, mutation will introduce however, that the genetic system is Population dynamics new variants from time to time. This flexible enough to overcome such The latest advances in evolutionary proceeds at another time scale. Game constraints. In the absence of game theory are being made by theory can be used to analyse both specific information on the means of population dynamics. This short-term and long-term evolution. genotype–phenotype map, transfer from population ecology In order to deal with the latter, it is evolutionary game theory often relies on the assumption that often advantageous to assume that provides a reasonably efficient tool successful traits spread. If there are the transient effects following a for understanding adaptation only two possible morphs A and B, random mutation have settled down without getting bogged down by for instance, then essentially only before the next mutation occurs. As complications at the genotype level. three scenarios are possible. In long as the population remains Here again, evolutionary game scenario 1, A can invade B and B can monomorphic, this leads to a trait dynamics follow the traces of invade A. This leads to the substitution sequence: selection Darwin — who, it will be noted, coexistence of both types in stable settles the fate of a mutant (fixation read Malthus, not Mendel. proportions as, for instance, if A are or elimination) before the next hawks and B are doves. In scenario 2, mutation comes along. Key references A can invade B but B cannot invade Along with the two time scales Axelrod R: The Complexity of Cooperation. Princeton: Princeton University Press; 1998. A. In this case, the dominant strategy come two notions of stability: Dugatkin LA: Cooperation Among Animals: an A will always outcompete B. This internal stability against Evolutionary Perspective. Oxford: Oxford University Press; 1997. happens with the prisoner’s dilemma, perturbations in the frequencies Hofbauer J, Sigmund K: Evolutionary Games and if A players are the defectors and within the resident population, and Population Dynamics. Cambridge: B players cooperate. In scenario 3, no external stability against invasion Cambridge University Press; 1998. Maynard Smith J: Evolution and the Theory of type can invade the other. This is a attempts by any mutant morph. For a Games. Cambridge: Cambridge University bistable situation; whoever exceeds a monomorphic population, the latter Press; 1992. certain (possibly quite small) notion corresponds to an Milinski M: Tit-for-tat in sticklebacks and the evolution of cooperation. Nature 1987, threshold will outcompete the other. evolutionary stable strategy: no 325:434-435. This happens with the iterated conceivable alternative strategy can Nowak MA, May RM: Evolutionary games and spatial chaos. Nature 1992, 359:826-829. prisoner’s dilemma if A is a tit-for-tat invade. But adaptation need not Nowak MA, Sigmund K: Evolution of indirect player and B always defects. converge to an evolutionarily stable reciprocity by image scoring. Nature 1998, As soon as three morphs A, B and strategy: the outcome can be a 393:573-577. Sigmund K: Games of Life. London: Penguin C compete, the situation becomes so-called ‘Red Queen’ type of Books; 1995. considerably more complex, because dynamics, leading for instance to Trivers R: The evolution of reciprocal altruism. cycles can occur: A dominates B, B potentially endless ‘arms races’, to Quarterly Rev Biol 1971, 46:35-57. Turner PE, Chao L: Prisoner’s dilemma in an dominates C, and C dominates A. ‘runaway selection’ or to cyclic RNA virus. Nature 1999, 398:441-443. This apparently bizarre situation chases in trait space. holds for the lizard Uta stantibus, Evolutionary game theory deals Addresses: *Institute of Mathematics, University of Vienna, and IIASA, Laxenburg, where three morphs (with different with phenotypes. But with sexual Austria. †Institute for Advanced Study, throat colors) correspond to replication, it cannot be assumed that Princeton, New Jersey, USA.