Toward a Dynamics of Social Behavior

J. Social Biol. Struct. 1985 8,255 277

Towards a dynamicsof socialbehaviour: Strategicand geneticmodels for the evolution of animal conflicts

Peter Schuster* and Karl Sigmundl rlnstitut für Theoretische Chemie uncl Strahlenchemie undllnstitut für Mathematik tler Universität Wien, A-1090 Wien and IIASA, Laxenburg' Austria

persistenceof behavioural traits in animal societiesrather than optimality of reproductive successis described in terms of the game-theoreticnotion of evolutionarily stable strategy.Game dynamics provides a suitable framework to accomodatefor the dynamic urp..i, of permanenceand uninvadability, to model the evolution of phenotypes with frequency äependent fitness and to relate the strategic models of sociobiology to the mechanismsol inheritance in population genetics.

Introduction Classical ethology provided numerous investigations of animal populations exhibiting 'altruistic' social phenomena based on behaviour. In this non-antropomorphic context, an act performed by an amimal is called altruistic if it increasesthe fitnessof some other animul at the cost of its own. Such behaviour was often explained by notions such as 'benefit 'group selection' and of the species'(for example, seeWynne-Edwards, 1962)' But while there is no logical inconsistency in viewing groups or speciesas units of selection.it seemswell establishedthat Darwinian evolution acts much more commonly through selection of the level of individual organisms or eventually genes'The explanation of many altruistic traits in terms of individual selectionhas, therefore,to be viewed as a major successof sociobiology. Roughly speaking,there are three approacheswhich proved to be fruitful. They may be designatedas genetic.economic and strategic,and attributed to Hamilton (1964)' Trivers (1972) and Maynard Smith (1974), respectively,although the principles go back to Haldane and Fisher, and arguably to Darwin himself. (l) The geneticexplanation is basedon the notion of kin selection.A genecomplex programming altruistic acts which benefit relativesmay well spread,since it occurs,with a certain probability, within the relatives whose reproductive successis increased. (2) The economic explanation relies on the notion of reciprocal altruism. If the altruistic act is likely to be returned at some later occasion, then the behaviour may become established,especially so if there is individual recognition between members of the population preventing'cheaters'.

0l40 l750/85/030255+23 $03.00/0 .O 1985 Academic PressInc. (London) Limited 256 P. St'huster& K. Sismund

(3) The strategicexplanation, finally, usesthe panoply of game theory to show how the fitnessof an individual may dependon what the other onesare doing. An act which in certainsituations would be calledaltruistic need not, in other situations,decrease the reproductivesuccess of the organismperforming it. In this paper.we shalldeal exclusivelywith the third approachstressing its dynamical aspects.It must, however, be stated at the outset that the game-dynamicapproach described here constitutes only a fraction of the game-theoretical analysis of animal behaviour. Our discussion is accordingly biased. The theory of games covers many applicationsin biologicalevolution. Some of them do not easilyfit into the mould of ordinary differentialequations. For a balancedsurvey of thesetopics. we may refer to a recentbook by Maynard Smith (1982). Our paper consistsof two parts. The first one discusses,within the static context of gametheory. the crucialnotion of evolutionarilystable strategy (ESS), which centreson aspectsof permanencerather than optimality. The secondpart usesgame dynamicsto obtain broadernotions of permanence,to model the evolution of behaviouralstrategies and to relate them to geneticmechanisms. Beforewe start our discussionof strategicmodels in sociobiologywe haveto comment on the evolutionaryaspects of this approach.The actual mathematicalmodels usedin sociobiologyas well as those of population geneticsdiscuss the spreadingof mutant genesin populations.Thecauses of mutationsare consideredas externalfactors and not as part of the systemunder consideratiom.A proper descriptionof the evolutionary process,however. is completeonly if it includesthe mechanismof mutant formation. At present.such complete theories of evolution exist for polynucleotidesin laboratory systems(for example,see Eigen & Schuster,1979; Biebricher et al.l982; Küppers, 1983) and eventually for simple viruses(Weissmann , 1914).The replication of bacteria follows a much more complicatedmechanism. Already ten enzymesor more are required for DNA copying. Although not all the mechanisticdetails of local mutations and DNA rearrangementsin bacteriaare known yet, the basicfeatures of bacterialevolution are establishedin principle and we can expect a complete molecular theory to become availablein the not too distant future. In caseof higher.multicellular organisms we encounternew sourcesof complications which obscurethe relation betweenmutations and their manifestationin phenotypic properties.The major problemconcerns the unfolding of the genotype:higher organisms obtain final form and shapethrough morphogenesis.This is a complicateddynamical processwhich is not completelyunderstood at presentand to which the genescontribute markers and other regulatory signalsfor intercellularcommunication apart from the ordinary dutiesin cell metabolismand cell division. Any completetheory of evolution of higher organismshence has to deal explicitelywith the influenceof mutations on morphogenesis,which is far outside the bounds of presentpossibilities. Behavioural traits are exceedinglycomplex phenotypicfeatures which apparentlycannot be traced 'mutant down to singlegenes. The notion of genes'leading to changesin behaviourhas to be understoodas a metaphor. Mutations actually operatewithin complicatednet- works of geneactions which are convertedinto the propertiesof the phenotypeby a highly complex transformationduring morphogenesis.

Evolutionarily stable strategies 'evolutionarily The term stablestrategy' was introduced by Maynard Smith and Price (1913) to describecertain seneticallv determined traits of animal behaviour which are 257 Dynamics of sociul behaYiour

evolutionarystability in uninvadableby mutants. The basic idea was to describesuch was subsequentlyused termsof the mathematicframework of gametheory. This method betweenspecies' and can to analysea wide varietyof biologicalconflicts. both within and be righily consideredto be a cornerstoneof theoreticalbiology.

Game theorYand bblog.v first publishedin Ever sincevon Neumann and Morgenstern's(1953) classical treatise of all kinds of 1944.the methods of game theory have been applied to the modelling and plants' conflictsin human societies.The applicationsto conflictswithin animals using game however. date back to the last ten years.There were a few forerunners one describedhere' theoreticnotions in biology. but in a context quite differentfrom the 'games'were neglected With the benefitof hindsilht it seemsastonishing that biological military or economicones. Indeed. there are at least for so long in favour of political. 'natural' their two reasonswhich make human conflictsmore difficult to analyzethan counterpartsin biologY: sinceit is hard to evaluatemoney, (l ) The payoff in human systemsis often doubtful. In the evolutionarily social piestige,health and other factors on a singleutility scale. quantity. Although weightedcontests in biology, Darwinian fitnessis the ony relevant and empirical this notron presentsconsiderable di{iculties for theoretical analysis success' measurementit providesin principle,at least,a scalarestimate of reproductive 'rationality game theoristsat (2) The axiom' frequently setsempirical and theoretical such an assumption' odds. The investigationof animal behaviour is unhindered by the simplest Biologicalconflicts tend to be fairly straightforward.compared with all but lunacy' etc' h,.,manconflicts of any real interest.The possibilitiesfor conspiracy,fraud. are greatly reduced strategyIn It is interestingto note. however,that the notion of evolutionarily stable contestsis quite closeto that of Nash equilibrlum strategyfor noncooperative biological 'Characterizations gamesbetween rational players (cf. the sectionon of evolutionary is interpretedas !,ubl. r,.u,"gies(ESS)'below). In Parker& Hammerstein(1984), this 'quasi-ratioÄlity' of selection.In our view,the convergenceof the two conceptsis rather 'blind 'rational to adaptative due to the fäct that both selection'and decision' lead solutions.After all, rational behaviouritself is a product of selectron.

Strategie s and Pheno t vPes entailsa shift The shift in the applicationof gametheory from human to animal conflicts 'strategy' 'payoff. 'Payoff, as we have in the meaningof the two basicnotions of and or-in subtler said.now correspondsto Darwinian fitness,i.e. the number of offspring generation' an situations-the number of copiesof genestransmitted to the next [For ( 'Strategy'.in the elaborateanalysis of the notion of fitnesswe refer to Dawkins l9S2)'] of moves' This context of chess.of war games,suggests a nicely calculatedsequence trait of fighting aspectof plotting and schemingwas replacedby the notion of an innate this viewpoint was biauioui (Maynard Smith & Price. 1973).It was soon realizedthat of any clash useful.not only in the analysisof aggressiveencounters. but in the modelling the length of the ol interests:in the parent offspringconflict, for example,concerning in parental weaningperiod. or in the male female conflict about the respectiveshare programmed way investment('I'flvers. 1g72).A strategy,thus, becamea genetically fraction of of behaviour in parrwisecontests. But such contestsform only a small 'subgames'of the strugglefor life. Differencesin resourceallocation. size of the litter' way. In suchcases' disoersalrate etc..affecittre fitness of an individual in quite another 258 P. Schuster& K. Sigmund. the successis determined,not through playing againsta particular opponent,or a series 'playing of opponents,but through the field' (seeMaynard Smith, 1982). A strategy,then, is any behavioral phenotype, or may be any phenotype at all. Indeed, game theory has beensuccessfully applied to desertshrubs in order to discussthe relative meritsof catchingrain water by spreadinghoizontal roots near the surface,or of taping ground water by sendinga vertical root deep below: the successobviously dependson what the neighboursare doing (seeRiechert & Hammerstein,1983). From thereit is only a small step to the protectivecolouring of moths or the familiar featuresof Mendelian 'strategy', peas.Thus, any phenotypemay be viewedas a as long as we wish Io analyze it by game theoreticalmethods or, to put it less subjectively,as long as its fitnessis frequencydependent, (which meansthat it dependson the distribution of phenotypesin the population considered). 'mixed On the other hand not everystrategy is a phenotype.In particular,a strategy' consisting in playing differentstrategies with preassignedprobabilities-need not be realizedas phenotype.In order to avoid confusion,we shall thereforekeep the distinction betweenphenotype and strategy,even if this implies some modifications in the standardterminology of ESS theory.

Optimum phenotl'pesand genetic constraints It is tempting.and often useful.to view selectionas an optimization process.In element- ary situations,as for examplewith asexuallyand independentlyreplicating units, this is 'Asexual indeed the case(see the sectionon inheritance:optimization and the gamedynamical equation' below). 'optimum', The term of course,has meaning only with respectto a givenenvironment. There is no reason to expect that under continuously changing conditions, selection would lead to an optimum responseto the instantaneoussituation. Furthermore,since the environment includesall other membersof the population, adaptation to an environment will change this environment. It is easy to exhibit correspondingpopulation geneticalmodels where the frequencydependent fitness will not, on the average!increase. This is actually the main theme of this paper. But evenifeach phenotypehad constantfitness, one could not expect,in general,that the phenotype with highest fitness would prevail. In particular, if this optimum phenotypecorresponds to a heterozygousgenotype, as may easilybe the case,then at leastone half of its offspring will be suboptimum. It is true that accordingto Fisher's fundamental theorem. the average fitness will never decrease.and even increase, provided the composition of the gene pool changesat all. However. the maximum averagefitness of the population may be much lower than the fitnessof the optimum 'trapped' phenotype.Furthermore, a population may be in a statewhere a local.but not global maximum is attained. Moreover. the fundamental theorem rests upon the assumptionthat fitnesscorresponds to the probability of survival from zygote to adult stageand that this probability dependson a singlegenetic locus. For many-loci models involving recombination, and for fecundity models where reproductive successis a function of the mating pair, the averagefitness will not increase,in general(for example, seeEwens. 1979).

Sel.fin tt,ra.s t ttnI opt inti:a t ion Apart from geneticconstraints, there may also be strategicobstacles to an optimization of the'good of the species',i.e. of the meanfitness of a population.Selection acts through differential reproductive successof individuals, which may be quite opposed to the Dynamics of social behaviour 259 welfare of the group. Even in human gamesallowing forethought and deliberation, the self-interestof two players may in some caseslead without failure to an outcome which is disastrousfor both. 'prisoners' This is best shown by the well known paradigm of the dilemma'' Two prisonerskept in separatecells are askedto confessa joint crime. If both confess,both will remain in jail for sevenyears. If only one confesses,he will instantly be freed, while the other one getssentenced to ten years.Ifnone ofthem confesses,both will be detained for one year.This latter solution is optimum but it will not be obtained.Each prisoner, indeed. will confess:it is the better option, no matter what the other one is going to decide.As a result, both will have to spend sevenyears in jail. [For a recent application of the prisoners' dilemma to the problem of cooperative behaviour in biology see Axelrod and Hamilton (1981).1 The prisoners' dilemma is a game between two opponents. Similar results hold for gameswhere an individual is'playing the field'. The best known example,in this respect, 'tragedy is Harding's of the commons', where the common meadow is ruined by overexploitation, each villager trying to increasehis payoff. [We refer to Masters (1982) 'prisoners' who baseshis sociobiologicaldiscussion of political institutions on the 'tragedy dilemma' and the of the commons'.]

Ev olut ionarily st ab le p henoty pe s If adaption leadsto an equilibrium at all, it will be characterizedby stability rather than optimality. An evolutionarily stable phenotype, in Maynard Smith's definition, is a phenotype with the property that if all members of the population share it, no mutant phenotype could invade the population under the influence of natural selection. One possibleway of formalizingthis is the following one (Maynard Smith, 1982).Let us denote by W(l,"r) the fitness of an individual /-phenotype in a population of J-phenotypesand by pl + (1 - p)J the mixed population where p is the frequency of .I-phenotypesand 1 - p that of ,I-phenotypes.A population of /-phenotypes will be evolutionarily stableif, whenevera small amount of deviant "I-phenotypesis introduced, the old phenotype 1 fares better than the newcomers,I. This means that for all phenotypes J * I, w(J,il + (l - {11 < w(1, e"I* (l - s)/), (1) for all e > 0 which are sufficientlysmall. It must be mentioned that this definition has to be modified if the population is small. In such a case,a deviant individual would change the composition of the population by a notable amount, and not just by some small e. This situation has been analysedby Riley (1979).In our context,we shallalways assume large population sizes,thus avoiding this problem and, moreover, the effectsof sampling errors. By letting e - 0 we obtain from inequality (l) that W(J,I) < W(1,D, Q)

for all .I, which means that no individual fares better against a population of /-phenotypes,than the 7-phenotypeitself. The converseis not true in general:(2) does not imply (l)

Evolut ionar ilY st ab le state s Now we restrict our attention to those biological gameswhich satisfy the following two assumDtions: 260 P. Schuster& K. Sipmund

(l) They consistin one or repeatedpairwise contests, rather than in'playing the field'. This implies that the payoff lI/'is an affine function of its secondvariable. i.e. that for any three phenotypesE, I and./ and any p e (0. l):

w(E,pJ + (l - p)1) : pw(E, J) + (l - p)w(E. I) (3) Indeed the probability that an individual f-phenotype matches up against a J- phenotype,in the mixed populationpJ + (l * p)1.is just p; and its payoff. in this case. is W(E. "/). Here. we have tacitly assumedrandom matching of opponents.In some situations----e.g. gamesagainst relatives-this will be not valid. [For a discussionof this casewe refer to Hines and Maynard Smith (1979).] (2) There are only finitely many phenotypesE, , . . . , E, This assumption may look innocuous. But there are situationswhere it is quite misleading,even as an approxi- 'war mation. As an example,we mention the so called of attrition', a contestwhere the prize goesto the player who investsmore time-or energy-than his opponent.The set of strategiesconsists of a continuum of waiting times and its discretizationmay be as ill-advisedas the replacement,say, of the bell shapedprobability densityof body sizes by a finitesubdivision. The lossof generalityentailed by thesetwo assumptionsis compensatedin a way by an increasein mathematicalconvenience. We may now describethe game in normal form, with the help of a payoff matrix. By ,t, we denote the frequency of the phenotype E, in the population. The point x: (,t,....,n,)describesthestateofthepopulation.Sincether,sarefrequencies,xlies on the unit simplexS,: {x: (,tr,...,,r,):-r:,) 0, Lr,: 1l.By o,,we denotethe averagepayoff, i. e. the expectedchange in fitness,for an d-individual matchedagainst an,{-opponent. The n x n-maftix A : (a,,)is called the payoffmatrix. The average payoff for the phenotypesd is

(Ax), : a,rt, -l ... ttinXn: u,*.t*. (4) I I k If x and y describetwo populations,then the averagepayoff for a y-member opposed to an x-member is (5) .y.Ar:,r1(Ax), +...+,)',(Ax), ll.r.o*,.t,. and in particular, the averagepayoff for contestswithin the x-population is II \pes1X1. (6)

Thereis an alternativeinterpretation of distributionsof strategies.In gametheory, E, 'pure to A, are viewedas strategies'.They correspondto the cornerse, of the unit simplex 3,. By e,we denotethe lth unit vector:all componentsare 0 exceptthe ith one which is 'mixed L We may interpret now a point x e S, as a strategy'which consistsin playing A,with probability.r,(i : 1,...,n). The expressiony'Ax is the payoffof a y-strategist playing againstan x-strategist.In the sequel,we shall make useof both interpretations. Pointsof ,S,will correspond,according to the context,either to strategiesor to statesof the population. Under our assumptions,the phenotyped is evolutionarily stableilT e,'A,(te,+ (l - 6)ei)< e,'A(t:e,* (l - e)e,), (7) for all .7 * i and all e > 0 which are sufficiently small. Dvnamicsof socialbehaviour 261

One may extendthis definition,just as oneextends the notion of a pure strategyto that all of a mixed one. A strategy (or state)p € S, is evolutionarily stable (or an ESS) iff for J€S,withy * p. . - - (8) J A(sJ + (l e)p) < p' A(ty + (l e)p), for all e > 0 which are sufficientlysmall. i.e smaller than some appropriateä(y)'

Characterizationsof evolutionarill'-stable strategies ( ESS) Equation(8) may be written as 'Av) (l - e)(p'A.p - y'^p) + E(p'Lv - v > O' (9)

This implies that p is an ESS iff the following two conditions are satisfied: (l) equilibrium condition y' ^p < p'Ap for all .1'€ S,; (l0) (2) stability condition : A! < p' Ly' if -y€ S,, y + p and y' ^p p' Ap, then !' (l l)

This is the original definition from Maynard Smith (1914).The equilibrium condition statesthat the strategypis a bestreply against itself. It correspondsto the notion of Nash equilibriumfamiliar to game theorists.In Dawkin's (1982)terms'an ESS "' can be crudely encapsulatedas a strategythat is successfulwhen competing with copiesof itsell' This property alone. however,does not guaranteeuninvadability, since it permits that another itrategyy is an alternativebest reply top. In facts,if p is'properly mixed', i'e' in if p, > 0 for all i, then any J € S, is a best reply. The stability condition statesthat such a case,p fares better, against y, than y against itself. The same argument leads to the following characterization: a phenotype d is evolutionarily stable iff ( I ) equilibrium condition ai12 a1,i for all A; (r2) (2) stability condition if eii ari for k + i, then a1, ) app' (13) A third equivalent definition of ESS,probably the most useful for computations' is the following one (Hofbauer et al.,1979):p e S, is an ESS iff p' Ax > x'Ax (14)

for all x + p in some neighbourhood U of p in S,. There are examplesof gamesadmitting no ESS,or severalones. If thereis an ESSin (14) the interior of S,, however,then it is the only ESS.Indeed, in sucha caseexpression must be valid for all x # P. 'hawk-dove' The game and mixed strategies 'hawk Let us now consider the famous dove' game, a model devisedby Maynard Smith and price (1973)to explainthe prevalenceof ritual fighting in innerspecificconflicts' We do not attempt to repeat here the considerable amount of work done on this model (Maynard Smith, 1912; 1914; 1982;Zeeman, 1979:Schuster et al., l98l), but describeit in its most rudimentary form in order to elaborate the distinction between mixed strategiesand mixed populations. 262 P. Schuster& K. Sismund

We shallassume that thereare two phenotypes.E, ('hawk') escalatesuntil flight of the opponentor injury decidesthe outcome.E ('dove') is only preparedto fight in a ritual way, without risking or inflicting injury. The prize of the contestcorresponds to a gain *. G in fitness.while an injury reducesfitness by IC I The effort expandedin a ritual contest costs I,El.A natural and self-explantoryorder relation among theseparameters is G>O>E>C, 'hawk'has Then a,, is (G + C)12----each the samechance to win or to get injured; a, is 'doves' 'dove' (G + E)12since only one of the can win, a,, is G and a^ is 0-the retreats 'hawk'. when it meets a Furthermore we restrict our discussionto the caselCl > G. Thus le+ G1 A:l- | L''g#) 'doves' Clearly.no phenotypeis evolutionarilystable. A population of can be invaded 'hawks' by and vice versa. 'hawks' The unique ESS of the population p : (pt, pr) is a mixture of p' and 'doves',with Pz:1-P' E-G (15) v1 E+C

This follows from the fact that (l6)

is positive for all x : (xr, xr) with xr * pr. Note that this state is not optimal. The averagefitness - x'Ax !(C 1 E 2Ex,+ :E + C;x,) is largestfor x, : El(C + -E).A numericalexample is shown in Fig. l. What happensif we introducea third phenotypeär, which plays the mixed strategy correspondingto the ESS: p, et + p2e21(For simplicity, we set -E : 0). The payoff matrix now is

G+C ('- G(G+C) 2c G(G+c) A 9 (17) 22C G(G + C) G(C - G) G(G + C) 2C 2C 2C

Conditions (12) and (13) show that E, is evolutionarily stable and cannot be replacedby neither E1 nor E, phenotype. The corresponding state e3,however, is not an ESS,and neitheris p,e1 * pre., in this extendedgame: condition (11) is not satisfied

* We measureall parameters on the same value scale.Hence, gains like G are positive and losseslike C or Ä negative quantities. 263 Dynamics of social behavtour

I I \l i\ t\ t. ! t\ lö ---r-----)---t--r\ \ \ I ro I o I Ol -l o I o I I ä-z I I -3 I I I -4 I I I -5 I I ol i o.s I I

-mox' ESS : rI Zt \8' g' '8' 8', xl are the t. The'hax'k-dove'game G:4' C:14' E:1' är andA2 lrequenc'v Fig.',zä,rrrlrnt :nÄ.tr' ,doves,, total pa1,oflsp; and v.hite@ is the averagepa1,off for the ' 'hawks' - 'dotes' p'ipilation consistingoJ' 11 andx' : | x1

'Some protectedagainst the mixture and (cf. alsothe section examples').Indeed, e3 is not vice versa. 'mixed' to mixed strategies we may consider other phenotypes corresponding (i'e' I p')' Such pheno- -rrer + -)r"e,which are not the ESS in the hawk-dove game "x, be invaded,either by'hawks" typesare nät evolutionarilystable: the population could .doves'. not invade E| All this is easy or by Furthermore,such'mixed' strategies could to check. with severalphenotypes continuing along theselines, one could analyzepopulations 'hawk' 'dove' strategies'There is no corresponding to different mixtures of the and the reason,however,tostopatafinitenumberofsuchphenotypes.Inthemostnaturalset phenotypes competing with each up, all possible mixed sirategieswould correspond to shall not pursue this other, but we pref'erto stick to a finite set of phenotypes here and (1983)' questionany further, referinginstead, to Zeeman,(1981) and Akin' Howplausible,anyway,arephenotypescorrespondingtomixedstrategies?Thisisa His con- which Maynard Smith (1982)devotes several chapters' delicateproblem to 'playing when the field" e'g' clusion is roughly that such phenotypesare most important .sex in the pairwise contests in the ratio' game,but probably of rather minor importance we are considering here. which are evolution- On the other hand, does one find mixed statesof the population are of course extremely arily stable?Again, this is not an easy matter. Mixed states of severalphenotypes for common, but we are looking for strategicequilibria, consisting is obviously difficult to which the fiequency depen-dentfitness values are equal. This in Hamilton's, (1979) verify. The saf-estempirical evidence,at present,seems to be found investigation of figwasPs. 264 P. Schtt:;ter& K. Sigmund

A st'tnnte Ir t( co n te.\ ts So far we have consideredcontests which are symmetric. in the sensethat both opponentsplay the samerole. Many conflictsare asymmetrichowever: within the same specieswe have conflictsbetween males and females.between young and old. between owners and intruders of a habitat. and so on. The methods apply equally well to interspecificconflicts. e.g. betweenpredators and preys. If we assumeagain that the contestsare pairwise and the numbersof phenotypesfinite. we areled to bimatrix games. Thus let Är.. . .. d, denotethe phenotypesof populationX. and F ,.. .F,, thoseof population )'. By .v,we denotethe frequencyolE-,. and by l', that of {. Hence.x is a point in S,,anclJ'in S,,. If an d-individualis matchcdagainst an {-individual. the payoffwill be a,, for the former and h,,for the latter. Thus the game is describedby two payoff matricesA : (a,,)and B : (ä,i).If the population X is in the state.r and the population I in the state -y. then x'Ay and -l'' Bx are the respectiveaverage payoffs. If the phenotypeä, is stablein contestsagainst the populationJ, then it must do as well as all competingphenotypes. Thus e,' Ay ) er' Al for k : 1... -.n. But what if equaltity holds?There is nothing. then. to prevent Ä^ from invading. Thus, in contrast to the 'best symrnetriccase. we cannot allow several replys'. This leads to the lollowing definition.A pair of phenotypes(8. {) is said to be evolutionarilystable iff eki

and hr,

Similarly, a pair of states(or strategies)(p. g) with p e $, and q € S-, is said to be evolutionarlilystable if p'Aq>x'Aq forall -r€S,. x*P (20)

and

q'Bp > y'Bp for arll J' € ,S,,,.)' + q (2l) It is easyto seethat sucha pair (p. q) must consistof pure strategies.Thus. in contrast to the symmetriccase. a mixed strategycan neverbe evolutionarilystable. This has been shown for a considerablywider classof asymmetricgames by Selten(1980). It also reflectson symmetric contestssince it is often quite possiblethat a small, seemingly irrelevantdifference can break the symmetrybetween the opponentsand transform the onginallysymmetric contest into an asymmetricone (Maynard Smith, 1976;Hammer- stein,1979). This could explainwhy mixed strategiesare rather rare in pairwisecontests. On the other hand. if the population X interacts,not only with the other population X, but also with itself, then mixed ESSsbecome possibleagain (Taylor. 1919:Schuster et ul..l98la,h) 'cot'ness Ita.shequilibria untl the philanderittg'game A pair of strategies(p, q) (with p e S,, { € .S-) is called a Nash equilibrium pair if p. Aq > x' ^q. Vx e S,, (22) q'Bp > J' Bp. Vl e S,,,. Nash equilibria play an import role in classicalgame theory, sincefor rational players thereis no reasonto depart from the strategiesp and q. as long as their opponentsticks Dy,'nctmicsof social behaviour 265

games.Nash equilibria are not to it. we shall presentlysee, however. that in biological invasion proof. 'coyness-philandering'game by Indeed,let us consideranother famous example.the an offspringincreases the Dawkins (1976).Let us supposethat the successfulraising of will be entirely borne by the fitnessof both parentsUy C. fne parentalinvestment lCl strategyto counter femaleif the male deserts.otherwise, it is sharedequally. The female period.which costs maledesertion ls'coyness', i.e. the insistenceupon a long engagement all parameterson the samevalue scale and have ]El to both partners.Again we measure population X' namely E' G > 0: C. E < 0. There are two phenotypesin the male population I'' (.philandering')and ä, ('faithful') and two phenotypesin the female namely F, ('coy') and F ('fast'). The payoff matricesare I o GI t-o c.i.4 A fo*f*u".;l'B:1".. o*t_l "o' population is an ESS No pair of phenotypesis evolutionarilystable. and no stateof the mixed strategles (we have or.,lyto check the pure states).There existsa unique pair of p and q in Nash equilibrium, given bY EC 4r (2s) Pt E+C+G. ztr+cl

Game dYnamics present dynamical Evolution is dynamrc, and hence any evolutionary model must may well aspects.For the study of equilibria and persistence,however. such aspects straight appli- remain implicit, as they do in the game theoreticalapproach. Indeed, a geneticsmay be quite cation of the inventory of dynamicalsystems used in popullation phenotyptc off the point, in ...tuin situations.In particular. ESS theory is essentially part of socio- rather than genotypic.It is, incidentally,quite interestingthat a major humanity' into the biology-a sciencewhich reputedly deliverseverything, including rather than genetlc cluchesof selfishgenes- {erives its interestfrom taking a strategic point of view. about the The presentstate of knowledgedoes not allow anything definiteto be said to geneticmechanism behind a giv-n behaviouraltrait; and to tie sucha trait arbitrarily machinery could iome hypothetical genotyp. n1uywell confuse things. The Mendelian P. St'huster& K. Sismund override the frequency-dependentregulation at the strategiclevel which one wants to investigate.To avoid suchinterference, it will thereforebe quite appropriateto assume asexualreproduction at leastas a first approximation. Most game-theoreticalmodels in biology concernsexual populations, of course;but this may eventuallylead to irrelevant complications based not on a necessaryconstraint but on speculationsabout the inheritanceof behaviour.

Aserual inheritance:optinti:ation and the game-dlnamical equation Let us considera population with r phenotypes.The point x(t) e S, denotesthe stateof 'like the population at time t. Asexual reproduction means begetslike'. The better the phenotype{ is adapted.the higher its rate of relativeincrease

I I d't' ri -x, dl

Ifthe fitnessofthe phenotypeE, is given by a constantL,, then the averagefitness ofa populationinstatexisgivenby

(26) restrictedto the invariant set S. will describethe evolution of the distribution of phenotypesin the population. This equation plays an important role in the theory of Eigen and Schuster(1979) on prebiotic evolution. The population, in this case,consists of n types of selfreplicating macromolecules,RNA or DNA, in a flow reactor.It can easilybe shownthat the average 'fitness', i.e. the mean reproduction rate. increasesmonotonically. Indeed, the time 'fitness' derivative of O is just the variance of the in the molecular population. The molecular types with lessthan averagefitness will be eliminated, and the averagefitness therebyincreased. In the limit, only the moleculeswith the highestfitness will remain. In populationswith asexualreplication and frequencyindependent fitness, selection is a global optimization process. 'error-free' The aboveresults are valid for replication.In more detailedstudies (Eigen, l97l; Thompson & McBride, 1974;Jones et al., 1976;'Swetina and Schuster,1982) mutationswere taken into account.In this case.the averagefitness

O(x) r.Ax accordingto equation (6). To be correct,(Ax), and x . Ax are not to be viewedas fitness but as increasein fitnessresulting from the conflict. The differential equation (27) given below, however, is invariant to the addition of constants to the columns of A and hence remains unchanged when we replace differential fitness by total fitness. Dynamics of social behaviour 267

'game The equation of dYnamics -- i, : x,{(Ax),- o(x)}' i 1,"',n, (27) restricted to the invariant set S, (the unit simplex) describes the evolution of the distribution of phenotypes(Taylor & Jonker, 1978). In writing down a differential of equation for gamedynamics we made two implicit assumptions:( 1) completemixing we mean in this generationsu"a 121infinitely large population size.By infinitely large context so large that fluctuations can be neglected. Equation (27) plays a central role for many models of selectionin fields as diverse as ecology (see Schuster& prebiotic-Sigmr,nd, evolution, population geneticsand mathematical 1983).Its usefulnessin sociobiologyis a new facet of its widespreadapplic- ability.

Some eramPles In a special case,namely when rz',: a;rholds for all i andj' the averagefitness o will alwayi increase.Games whose payoff matrix satisfiesthis condition are called partner- ship games:both playersalways share the outcomefairly.Indeed, it can easilybe checked that the rate of increaseof O corresponds,again, to the variation of the fitnessin the population. It is, of course,no surprisethat in partnershipgames, in contrast to the 'prisonersdilemma' and the like, an optimization principle holds. 'hawk - - The ESS of the dove' game,p : Pr€r + (l p')e, with Pr: (E G)l dove' (E + C) as describedby equation (15) is globally stable.In the extended'hawk game given by Expression(17) with the phenotypeE correspondingto this ESS,there existsa line F of fixed points through er and p (seeFig' 2)' All orbits converge to 4 remaining on the constant level curves of the function P'x, o'. (28) QG) x.,r,

There is an argument (Hofbauer, private communication) that random drift will cause the stateto approach e.,,leading to fixation of the phenotypeE.Indeed the constant level curves of-q ut" concave. If the state of the population has approached some fixed the differential equation point 4 on F, and if a random perturbation sendsit away, then will lead it back to some point q' on -F.The probability I"haI q' is between 4 and e, is will drive slightlylarger than I /2 (seeFig. 2). In this way, a sequenceof smallfluctuations the statetowards ej. This, however,depends upon the assumptionof fluctuationswith radial symmetry. 268 P. Schuster& K. Sigmund

Fig. 2. Strutegie.se, uul e- torrespondto'luvk' urul dore' plrcnttlt'pt'.t,e. lo u phcruttrpe plu.tingu ni.rt'tl strute,s,.t,u('ting tt.s'hunk' nith prohuhilil.t'1t, tutd us'dovc'tith prohubilit.r p.(rharcpluntl p.uregit'enh)'tlte ESSp : (ptet + p.e.)o/ thc'huwkdote'gume).The linc lront el to p (onsistso/ li-redpoint.s. Rundont drilnnight leud the .\tuteof thc population, throu.qhflttctuutittns ulong tlte line. tlttser to e.

We recall from equation (17) that phenotype E. is not evolutionarily stable. A morc strictins examole for such a situation is obtaincd with r0 [n A (2e) I']' : Lt I il (seeFig. 3) Ä, is stableagainst invasion by E, alone.or by E, alone.but not if both phenotypesinvadesimultaneously.

€t €z

Fig. -l . The gunte-d.tnunitul equuliott (.:7) \'ith nnlri.r givan ht (29) Dvnamicsof socialhehuviour 269

€1

Fig.4. The game-th'namicalequation (27) tt'ith mdtri't giv('nbt G0)

Another interesting example due to Zeeman (1981) is given by the matrix u -ol Io A 0 s (30) l-3 I L-l 3 ol

In this case, equation (27) admits two asymptotically stable equilibria, namely er: (1,0,0) andm: (1i3,li3, 1/3)(seeFig.4).Theformerpointiseasilycheckedto be an ESS. Hence the latter one, which lies in the interior of S,, cannot be an ESS'

Permanenceand uninvadlhilit.v in terms of dynamical svstems As mentionedbefore, the orbits of a game-dynamicalequation (27) neednot converge to an ESS.They may convergeto an equilibrium which is asymptoticallystable but not an evolutionarilystable straregy in the senseof game theory.or they may settleto a persistentoscillatory regime, or exhibit chaotic behaviour.In the two latter casesit can be shown that the time averagesof the phenotypefrequencies converge to an equilibrium value(see Schuster et a1.,1981a). Of course.one cannot hope to measuresuch time averagesdirectly, since they would haveto includea largenumber of generations.If the population is subdividedinto many suchpopulations (or demes)which oscillateout of phasewith eachother. then the mean value (at a given instant) of the phenotypefrequencies of the different demesgives a plausibleestimate of their time-averagesand henceof the quilibrium. The game dynamical approach suggeststwo eventualities: (l) the existenceof equilibria which, although not'evolutionarily stable',are neverthe- lessrelevant and persistentfeatures of the model. eitherbecause they are asymptotically stableand henceperturbation-proof, or becausethey correspondto the time averagesof regular or irregular oscillations,and (Zi ttre existenie of asymptotic regimeswhich, although not static. are nevertheless robust featuresof the sYstem. 270 P. Schuster& K. Sigmund

Whether suchphenomena occur in situationsof biologicalinterest is still open.but the possibilityshold be kePt in mind. In any caseit is not dilhcult to describe,within the framework of populationdynamics' the notions of permanenceand uninvadabilityin a non-staticway which generalizesthe concepr of evolutionary stability. Roughly speaking.the n phenotypesE' to E, are pernlanentif thereis someminimum level4 > 0 suchthat, if initially all phenotypesare present.i.e. if .v,(0) > 0 for all i. then after sometime their frequenciesr'(t) will be larger itrun q. Thesefrequencies could oscillateor converge:the only relevantproperty, in this context.is that random fluctuationswhich are small and occur rarelyare not ableto wipe out some of the PhenotYPes. The societycan be termed unint,adahle,with respectto some lurther phenotypesd'*, to E,_r, if it is permanentand if all initial conditions with sullicientlylow frequencies .r,,*,i0jto.t,,,*(0) leadto the ultimatevanishing of thesephenotypes' Permanence and uninvadability,in this sense.mean protectednessagainst disturbance from within and from without. We refer to Schusterand Sigmund (1984)for a more detaileddiscussion of thesenotions and their applicability to models of biological evolution.

D.rttuntit.s Ior as.l'mntclrit' gamcs Let us turn now to asymmetriccontests. i.e. to gamesdescribed by two payoffmatrices 'Asymmetric A and B (seethe sectionon contests'above). The sameargument which led to the game-dynamicalequation (27) now yields the differentialequation : - x'Al]. i : 1""'n' -ii x,[(AY), (31) - : t, : l,[(Bx); r'Br]. i 1."'.m, describingthe evolution of the population statesx(l) e S,'and y(t) e S,' (seeSchuster et al.,1981b). If one of the populations contains only one phenotype,equation (31) reducesto equation (26) and henceto an optimization problem. In the generalcase, selection leads usually to the fixation of one genotypein each population. This reflectsthe fact that there exist no mixed ESS for asymmetricgames. In the casen : nlj however.it may also happenthat the frequenciesoscillate periodic- 'coyness ally. This is the case, for example. in the philandering' game of Dawkins 'Nash - (Schuster& Sigmund,l98l; seealso the section on the equilibriaand thecoyness philandering game' above). Again, the time averagesconverge to the game-theoretic equilibrium. which is not evolutionarily stable.

D i:;cretegame dYnumics Differenceequations are the appropriatetool to study infinite populationswith distinct generations.Thus. they apply to situations in which premise (l) for the validity of equation(27) is not fulfilled.Often, blendingof generationis preventedby the action of someinternal or externalpacemakers for the reproductivecycle. An obviousexample for the latter caseis the periodicity of seasons. The most straight-forward candidate for discrete-gamedynamics of symmetric contestsis the equation (32)

Here r' : (ri...... rj) denotesthe state of the population in the next generation.It is somewhatdisturbing that this classof differenceequations, in contrastto the differential Dvnamics of .gocialhehaviour 271

above,does not seemto fit ESS theory very well. Indeed,equation equationsdiscussed 'hypercycle' (3)) behavesrather badly in severalcases. We consider as an example the gamewithn:3givenby

A : r,0 (33) l0 jl Ll 0

Ir has rn : (l/3, l/3, l/3) as an ESS. The fixed point rr is not asymptoticallystable, however.This is surprising,since the continuous analogueof expression(32), namely r ,ii : (x .Ar) x,[(Ar), - x'Ax] (34) has the sametrajectories as the game-dynamicalequation (27). For asymmetricgames. the differenceequation ri : (r'Ay) ',t,(A.y)', (35) '1(Bx), t; : (Y'Bx) correspondsto the differentialequation ',t,[(A-Y)' ii : (x'Ay) - r'AY]. (36) '-r1[(Bx), j'i : (-r 'Bx) - l''Bx], which needno longerbe equivalentto equation(3 1). In particular,the inner fixedpoint (p, q) of the'coyness-philandering'gameis now asymptoticallystable (Hofbauer, see Vuy"u.a Smith, 1982,appendix J). This may be viewedeither as a misleadingtrick of the dynamics-after all, (p. 4) is not an ESS-or as a remarkable vindication of Dawkins earlyclaim (1916)that the strategiesdo convergeto (p, q)- Of course,there are no empirical dates to decidewhich of the two equations(31) or (36) is more correct. An interestingapproach has beenproposed by Esheland Akin (1983).It consistsin - assumingthat the sign of ,t, (or. in the discretecase, of "ti x,) is that of the difference (31) (36).The (Al ), - .r . AJ, without writing down an explicitequation like equation or dynamics,then, is only incompletelyspecified. but should reflectthe basictraits of the model; surely,the last word on differenceequations, differential equations and ESShas not been said yet. 'like So far, we have acceptedthe simplificationthat begetslike'. More sophisticated discussionshave to take account of the Mendelian mechanismof inheritance.

Serual models Although the strategicand the geneticpoint of view are to some degreedisjoint, it is neverthelessof interestto check their compatibility, even if this requestsassumptions which are quite hypothetical. It is obvious that a geneticconstraint can prevent the population from attaining an ESS. In particular, an evolutionarily stable phenotype which is only realized by a heterozygotegenotype can neverbecome fixed. Sucha caseof'overdominance' leads to different predictions of game theory and population genetics.So far only one locus modelshave been studied (Maynard Smith, l98l; Eshel,1982; Hines, 1980;Bomze er a/.. 1983).In this caseit was always overdominancewhich led to divergencesfrom ESS results.In many-locusmodels one may expectother geneticconstraints as well. 272 P. Schuster& K. Sismund

Let us consider first the frequencyindependent case. We assumethat there are k alleles Ar to Ar in the genepool. Their frequenciesare denotedby p,to p*. The fitnessof the genotypeA,A, is given by some constant w, which correspondsto the probability of survivalfrom the zygoteto the adult stage.Introducing the laws of Mendelian genetics we obtain the classicalselection equation of Fisher for the evolution of the gene distribution

ii: P,{twPl,-O(P)}. l:1.....k. (37) 'viability The differential equation is invariant on S*, W : (w,i) is the matrix' and O(p) : p 'Wp representsthe averagefitness in a randomly mating population with the genedistribution p : (p,,...,pJ.This equationis a specialcase of equation(27). Precisely,it correspondsformally to the (game)dynamical version of a partnershipgame sincew, : nii. Accordingly,the averagefitness O increasesmonotonically. Selection can be visualizedas optimization of the mean reproductivesuccess. There is neverthelessan important differencewith respect to the nature of the optimization processin equation (37) and in the asexualcase described by equation (26). fn the latter caseoptimization was global on S,.Here, selectiondoes not lead in generalto a global optimum of O. Consider for example the particularly simple case r? : 2. The averagefitness is of the form

O (h',, - 2lr',1* rr,,r,)p]f 2(wr, - tt,.r)p,I w.2. In caselr' > tl''r2and n,r, ) D'12,O has a minimum on the interval 0 < p, < 1 and hence,there are two optima coincidingwith the pure statespr : 0 and pr : 1. Depend- ing on the initial conditions either alleleA, or alleleA, will be selected. We turn now to the frequency dependentcase and consider equations which combine the game theoreticalmodel with popultion genetics.It will be appropriateto carry out the analysison two levels,the phenotypicand the genotypicone. We shall assumethat the population consistsof r phenotypesE, to En and that each genotypeA,A, corresponds to one of those phenotypes,or possibly to a probability distribution p(ry) : Q,(D,...,p,(n), whereprQ) is the frequencyof A,A,-individuals of phenotypeE*. The converse,however, need not be true: as it happensin the caseof dominance,two or more genotypesmay give rise to the samephenotype. The payoff is given as before by the n x n-matrix A. Now we have to specify how its is relatedto reproductivesuccess. This is a rather delicatepoint which cannot be decided without a closerinspection of the situationto be modelled.Two alternativesimple cases are: (l) The payoffis independentof sex,and the number of offspring of a given couple is proportional to the product of parentalpayoffs. This situation correspondsto fights which are not sex-specific,like contestsfor food. Maynard Smith (1981)considered such a model. (2) The expression of the genotype is sex-specificalthough the relevant genes are carried by both sexes.The number of offspring of a given couple again is proportional to contributionsfrom both parents.The parent who carriesthe silentgenes now contri- butes a constant factor. Such a model is appropriate for fights within the male population, e.g. contests for females,breeding grounds or other ressources.A detailed discussionof this approach can be found in Hofbauer et al. (1982). In any caseone is led to fecundity models.To set up the corresponding(discrete or continuous)equations for the time evolution of genotypeor phenotypefrequencies is immediate, but the analysis of the resulting systemsis in general difficult. D I'namic.sof' soL'ialbe hat' tour 273

lAol ?

Fig.5 A tliagramdest,rihing a typit:alevolution of genott'pesAA. Aa and aa(correspond- irigtothecornerse,,e.,er)il'ttriirprortur'tilesr.r('('errdependsondffirentiullbcunditv.The ,t:it, u./th, pnpulatiotitont,ergn, ,,;r.i. quitklt'to u'Hurdt' llefuberg'parahola, and evoltes P thr1ughfriqiencv-ctependeni seletr'irri utn,ig the parahola tov'rtrd.s an equitibrium [fbr rletuils.see Hofbauer et al. (1982)l

In some cases.however. the investigationis simplified by the fact that a Hardy- Weinbergequilibrium gets established: in case(l)if the modelis discrete(nonoverlap- ping generations)and in case(2) ifit is continuous(generations blending into eachother. ,.. äg 5). Thus rhe frequencyof genotypeA,A, is givenby the product of the frequencies oithe correspondinggenes A, and A,. (More generally.it often happensthat the mating system-random or assortative-leai1sto relations betweenthe genotypefre- qu..,.i..j. This allows to reducethe problem to the time evolution of genefrequencies' The correspondingequations are similar to equation (37). exceptthat the l1'/rare now not increasein general' polynomials- in p, to p^. The averagefitness will An analysisof two-strategygames (see Maynard Smith. l98l; Hofbauet et al., 1982 Eshel.1982) shows, essentially. that if thegenetic constraints allow the ESS to be realized is at all. then it will asymptoticallybe reachedindeed. If, for example.one of the alleles dominant,then the outcomepredicted by the simplegame theoretic considerations of the 'The'hawk sectionon dove' gameand mixed strategies'above will also be obtainedby the geneticmodel. (It is worth mentioningin this contextthat Bürger. 1983.using a model of Sheppard.1965, has shown that dominancemay be established.if the selection pressueis sufficientlyhigh, by the action of a secondarygene locus. Thus the genetic mechanism itself may evolve towards a suppressionof the genotypic obstaclesto phenotypicadaptation). One may similarlyinvestigate genetic models lor asymmetriccontests. In Bomzee'l a/. (19S3);the'coyness philandering' model is discussedat somelength. We briefly sketch it. as an illustration. Let us assumethat two allelesA, and Ar determine the male behaviour,and that P,,, P, and Pr are the probabilitesthat the male genotypesA'A'. A,A, and A,A", respectively,are faithful. Then. if p denotesthe frequencyof A'.

6 : p2{Po* P.-2Pt)+2p(P,- P)+ P) 274 P. Schuster& K. Sismund is the frequencyof faithful maleswithin the male population. Similarly,let the allelesB' and B, determine female behaviour and let Qo, Q, and Q, be the probabilities that the femalegenotypes Br Br, Br B, and BrB,, respectively.are fast.Then, if 4 is the frequency of 8,, P : q2(Qo+ Q. - 2Q,)+ 2q(Q,- Q) + 2, 'payoffs' is the frequency of fast females,It is easy, now, to compute the a) and a2 for 'payoffs' faithful and philandering males, respectively,as function of B and the a, and a, for fast and coy females,respectively, as function of ä. If one assumesthat the payoff correspondsto fertility, one can check that Hardy Weinberg relations hold. Simple computations lead to the differential equations - - - - i Lp(l p)lp(Po * P, 2P,) + P, Prl(a, ar) s : Lq(r- q)lq(Qo* Q, - 2Q) + Q, - Q.l(a,- a,). The strategic component-i.e. the terms depending on the payoff matrices-reduce to the factors at - azand a, - ar. This facilitates the analysisof the genefrequencies. We 'overdominance'. shallonly describethe interestingcase of If P0 - P, and P. - P,have 'state-space' the samesign, as well as Qo - Q, and Q, - Q,, then the orbit in the (p, q) will be periodic (see Fig. 6). There are four possible oscillatory regimes,depending on initial conditons, but the time averagesof the strategieswill quickly convergeto the equilibrium valuesobtained in Equation (25) by simplegame theoreticconsiderations. 'microscopic' This may well be a typical situation:complicated features on the level of 'macroscopic' genefrequencies yield a simple result on the level of phenotypes.(For a detailed discussion see also Bomze et al., 1983). Asymmetric conflicts between two differentspecies can also be modelledin similar ways. The first paper in this direction, which apparently found only little attention in the literature, seems to be due to o o O o oo.sr

Fig. 6. A possibledynamicsfor the'coyns.s5*phfionderirg'game.The genefrequences p and q governing male and.fbmale behaviour oscillate endlesslv, but the corresponding payofls converge,in the time average,to the game theoretic equilibrium (25) (see Bomze et al., 1983,t'or details) Dynamics of .soe'ialbehaviour 275

Stewart(1971) and investigatesdimorphism in two coevolvingpopulations of predators 'game' and prey. His betweencats which can stalk or watch and mice which can run or freeze.is similarin structureto the'coyness-philandering'game.In its emphasison game theoretic aspectsit is a remarkable forerunner of ESS theory: it also offers an analysis of dynamical modelsbased on sexualand asexualgenetics, which fully agreeswith the Nash equilibrium solution obtained from strategicconsiderations alone. Resultsof this type stronglysuggest that one may confidentlystick to the phenotypic level.as long as there is no solid candidatefor the underlyinggenotypic mechanism. Concludingremarks The considerationsconcerning the role of intrinsic constraints on the evolutionary optimization processcan be subsumedwith the help of Fig. 7. The mean reproductive successin populations of independently and asexually reproducing individuals is opti- mized though evolution. This will be modified by two mechanisms: (a) The rulesof Mendelian geneticsrestrict the dynamicsof selectiononto the Hardy - Weinberg surfaceor onto a'near Hardy Weinberg'submanifold (seeEwens, 1979). Other distributionsof genotypesare evolutionarilyunstable for geneticreasons. (b) Direct interactionsof replicatingelements lead to constraintson the optimization process.The resultis a displacementof the evolutionarilystable distribution from that which is characterizedby the maximal reproductive success. In populationsin which both constraintsare in operation,the geneticand the strategic one, we observea superpositionof both effects.In the sexualmodels considered so far, many cases(e.g. dominance)lead to situationswhich are indistinguishablefrom the correspondingasexual ones. This justifies a posteriori the various static or dynamic asexualmodels. Of course the situations analysedso far are characterizedby oversimplification. One could introduce further complicationswithout end, but often without gaining lurther insights. Basically,the gamedynamical models incorporate features of highly developedfields, namely population geneticsand game theory. A further emphasison geneticaspects

Genetrcconstroinl

Asexuol Sexuol replrcot ion replicofion c> >o; o- o C I o! Q Globoloptimizotion Restrictionto H-W surfoce ; of@ Optimizotionof Ö o o c

.c o >:: C 9öe Restrictionto H-W surfoce (-)o Y -! n No optimizotionof @ Nooptimizotion of 0 PöR3 [! - o

Fig. 7. Constraintson optimization. Thegenetic mechanism of se.rualreproduction leads to relationsbetween gene and genotype.frequencies (e.g. the Hardy-Weinberg equation).This reduces the state space of genot-Vpe.frequencies to a subset of lower dimension. FrequencT' dependentJitness parameters preclude optimization through strategic constraints 276 P. Schuster& K. Sismund leads to intricate frequencydependent population genetics.and a stressingof game theoreticaspects to the subtleconstructions of extensivegames and subgames(see Selten. 1983).In its presentstate, game dynamics is a compromisebetween these two directions, trading elaboratesophistication in one or the other direction for broadrangedcompati- bility betweenboth aspects.It aims at acquiring an intuitive insight into biological conflicts.without too many technicaldetails. It seemsvery difficult to derive from field data valid estimatesfor the parameters involvedin the payoffmatrix. Someremarkable results have been obtained, however [we refer to Maynard Smith (1982)and Riechertand Hammerstein(1983) for surveysl. Moreover. it has beenshown in severalcases that the dynamicsconsist of a few types only. which arevalid for wide rangesof the parameters.Thus, diversesexual and asexual 'hawk-dove' modelsof variationsof the conflict lead to a very small number of possible outcomes(see Schuster et at., 1981a.ä). It seemspossible, in suchcases, to relate(at least qrnlitatively) empirical data with some of the few theoreticallypossible cases. Finally, we mention that the assumptionof geneticaldetermination of behaviourcan be relaxed. In particular. the effect of learning can be incorporated into the game 'developmentally theoreticalmodels [for example.see Harley (1981)and the notion of stablestrategy'DSS by Dawkins (1980)1.It seemspossible. therefore, that the dynamical approachsketched in this paper can be adaptedto the modellingof complex behaviour in highly developedsocial structures(for example.see Wilson. 1975.Masters. 1983).

Acknowledgements 'Fonds The work has been supportedfinancially by the Austrian zur Förderung der WissenschaftlichenForschung' Project P 4506and P 5286.We would like to thank Prof. John Maynard Smith and Dr Peter Hammersteinfor fruitful discussions.

References Akin.E. (1983).Can. J. Math.36.374. Arneodo.A.. Coullct,P. & Tresscr.C. (1980).Phl's. Lett.79.259. Axelrod.B. & Hamilton,W. W. (1981).Sclence,r 2ll. 1390. Biebricher.C. K.. Diekmann.D. & Luce.R. (1982).J. math.Biol. 154'629. Bomze.I.. Schuster.P. & Sigmund.K. (1983).J. theor.Biol. l0l. 19. Bürger.R. (1983).J. math.Biol.16.269. Dawkins.R. (1976).The Selfish Gere. Oxford: Oxford University Press. Deutsch'. Das egoisrische Gen.Berlin: Springcr-Verlag. Dawkins.R. (1982).The Ertended Phenottpe. Oxford: Freeman. Eigen.M. (1971).Die Naturv,i.ssenscha,/'ten58,465. Eigen.M. & Schuster.P. (1979).The Hypercyclc'.A Principleof'Natural Selfbrganizatiorr. Berlin: Springer-Verlag. Eshel.I. (1982).Theor. Pop. 8io1.22.204. Eshel.I. & Akin. E. (1983).J. math.Biol. 18. 123. Ewens.W. J. (1979).Mathernatital Population Genetics. Berlin: Springer-Verlag. Hamilton,W. D. (1964).J. theor.Biol.7.l. Hamilton.W. D. (1979).In: (M. S. Blum& N. A. Blum.Eds) Competition in Insects.New York: AcademicPress. Hammerstein,P. (1979).Anim. Behav.29, 193. Harley.C. B. (1981).J. theor.Biol.89.6ll. Hines,W. G. S.(1980). J. theor.Biol.87,379. Hines.W. G. S. & MaynardSmith. J. (1979).J. theor.Biol.79. 10. Hofbauer,J. (1981).Nonlinear Analysis. TMA 5, 1003. Hofbauer,J., Schuster,P. & Sigmund,K. (1979).J. tlrcor.Biol.8l,609. Dlnamic's oJ'soc'ial behaviour 277

(1980). J' appl Math' 38' 382' Holbauer. J.. Schuster.P.. Sigmund. K. & Wolff. R. SIAM 155' Hofbauer. J.. Schuster.P. & Sigmund.K' (1981) J' rnath' Biol' 11' ' Hofbauer. J.. Schuster,P. & Sigmund.K (1982) Biol' Cvbern' 43' 51 Jones.B. L. (1978).J. math.Biol. 6. 169' ton.r. S. L. . Enns.R. & Ragnckar.S (1976) Bull' math' Biol'38'15 iäfp..r. B. O. (19g3). Moleiular Theorl. of' Et,olution, Berlin: Springer-Verlag. Maiie.s, R. D. (1982):J. sot'ialbktl. Struct'5' 439 Press. M;;;".ä Smith. J. (t9.72).On Evolution.Etinburgh: Edinburgh University Maynard Smith. J. (1914)-J. theor.Bbl' 47.209' Maynard Smith,J. (1976).Am. Sci.64.4l' Maynard Smith,J. (1981).Am. Nut.177.1015 . cambridge: Cambridgeuniversity Press. vuynu.o smith. J. (lgszi. rrotutionart,Game lh.eor1" Maynard Smith, J. & Price.G. (1973).Nature 246' l5 and Economic Behoviour' Neumann.. J. von & Morgensie.n,ö. ( t ssl) ' Theort' of Gumes PrincetonN.J. P. (1984).Evolution parker, G. A., Hammersrein.P., in Greenwood,P.. Slatkin, M. & Marvey. University Press' Essaysto the honour tt John Ma1'nurd snltft. Cambridge: cambridge 14,3'7'l Riechert.S. & Hammerstein.P. (1983).Ann. Rev.Ecol. s)'stemarics. Riley, J. (1916).J. thettr.Biol.76' 109. Schuster.P. & Sigmund.K. (1981).Anim Behav'29. 186' Schuster.P.,Sigmund.K..Hofbauer.J.&Wolff.R.(l98la):Biol.C).bern.40.1' 40' 9' Schuster.P.. Silmund, K.. Hofbauer' J. & Wolff, R (1981Ö):Biol' Cyhern' Schuster.P. & Swetina.J. (1982).Biophl's Chem' 16.329 Schustcr,P. & Sigmund.K. (1983) J. theor' Biol 100' 533' Phenomenaand Chaotic Schuster.- P. & Sigmuno,i.'trssal. In: (P. Schuster.Ed.). stochastit' Bi;ehayiourin Cimpler S,rs/ens, SynergeticsVot.21. Berlin: Springer-Verlag. Selten,R. (1980).-r. theor.Biol' 84' 93. Selten.R. (1983).Math. soc. Scr.5, 269' i6.ppu.O. p. Nf. (f qSSt.Natural Selectionand Hereditt'.London: Hutchinson. Stewart,F. M. (1971).Theor' Pop Biol-2,493 Taylor, P. (1979).J. Appl Prttb. 16"76. taytor. P. & Jonker. L' (1978).Math Biosci'21, 121 Thtmpson, C. & McBridc, J. (1974).Math Biosci' 21' l2'7' descentof man' London: Trivers. R. L. (1972). tn: iCaäpUeil. Ed') Sexual selectionand the Heinemann. Weissmann,C. (1974).FEBS Lett.40' 10 Harvard University Wilson, E. D. (1975). Socittbiologt'-TheNev' Svnthesis.Cambridge' MA: Press. Behaviour. Edinburgh: Wynne-Edwards. V. C. (1962). Animal Dispersion in Relation to Sot'ial Oliver and Boyd. Notesin Mathematics819' Zeeman.E. C. (1979).ln: GlobalTheor-v of D.'namicals1'stems, Lecture Berlin: Springer-Verlag. Zeeman.E. C. (1981).J. theor'Biol.89' 249'