Replicators and the Evolution of Diversity

ELSEVIER Physica D 75 (1994) 275-291

Multilevel evolution: replicators and the evolution of diversity

P. Hogeweg

Bioinformatica, University of Utrecht, Padualaan 8, 3584 CH, Utrecht, The Netherlands

Abstract

In this paper we report on an initial attempt at studying evolution as a multilevel process, in which interactions of different space/time scales mutually determine the evolutionary potential. Side effects of processes on one scale may form a substrate for evolution at another scale. We study the generation and maintenance of diversity in a multilevel replicator system and focus on the role of local information processing and pattern formation. We show how the conflicting requirements of the generation and maintenance of diversity are met in different ways in completely mixed (stirred) and poorly mixed (diffusive) systems. In the former diversity is maintained primarily by non-invadability, and operates at the lowest level; in the latter case diversity is maintained by locally contained invasions which are subsequently eliminated by higher level processes.

1. Introduction coevolution. Coevolutionary processes are most often modelled as (one level) dynamical systems, Four lines of reasoning converged on the rather than optimization systems, and the attrac- research reported in this paper. tors are characterized. Although the important insights in evolution (1) Evolution as a multilevel process. so far have been obtained by virtue of squeezing the multiple levels into one level, we think the Evolution is most often studied as a one level time has come to start exploring evolution as a process involving informatic entities which can multiple-level process in which patterns gener- mutate in various ways and are selected by the ated at one level may determine the evolutionary environment through differential replication or dynamics and the evolutionary potential at other decay (“survival of the fittest”). In this frame- levels. Indeed we think that such a multiple-level work evolution is studied most often as an approach is a prerequisite for studying the cru- optimization process. Indeed, Genetic Algo- cial questions of evolution, i.e. how novel struc- rithms exploit this paradigm successfully as a tures and functions can arise. In the above method for optimization. Realising that most of mentioned one level models either the coding the relevant environment of biotic systems may structure and the target are fixed (optimization consist of other biotic systems, i.e. systems which models), or the variables and interactions are may change their characteristics at a similar fixed (dynamical systems). By modelling evolu- time-scale, this paradigm is extended to include tion as a multiple-level process the ‘side effects’

of one level can modify the targets, the coding [17,18] which studies the coevolution and coding structure and/or the interaction structure at structure of entities which evolve the coding of other levels [lO,ll]. the replicative process itself, and the (spatial) Extension of one-level models to multiple game dynamics studied by Lindgren [14,25] levels can be done starting from the optimization which studies the coevolution of interaction viewpoint, starting from the coevolutionary ‘strategies’ in relation to the spatial structures viewpoint, or from both. generated by these strategies under different parameter regimes. In this paper we combine 1.1. Evolution of the coding structure in coevolution and coding the other way around, optimization models i.e. we study a system in which the coding structure of the higher level entities is modelled By recognising that the ‘targets’ of evolution in terms of a coevolutionary process of (self (i.e. continued existence) are such that they can replicating) molecules. The higher level entities be realised as effectively in many different ways, optimize a very general optimization criterion, we can study which ‘solutions’ are preferred by i.e. the diversity of the underlying coevolution- the dynamics of the evolutionary process. In ary system. We show that generation and mainte- other words we can study how the choice be- nance of diversity in the system is achieved in tween ‘neutral peaks’ in the fitness landscape is very different ways dependent on pattern forma- not a ‘neutral choice’, and how such non-neutral tion or the absence thereof in the lower level neutral choices affect the coding structure and coevolutionary system. thence the future evolutionary dynamics. In this vein we have studied the generation of multiple (2) The cell as the environment for an evolv- or single coding under respectively high and low ing genome mutation rates [ll] and generation of patterns in RNA sequences by evolutionary dynamics lead- Possibly even more important than interacting ing to relatively smooth or rugged parts of the with other biotic systems, a biotic system has to RNA secondary structure landscape [ 121. deal with itself in its most direct environment. More specifically, the genome interacts with its 1.2. Evolution of new levels of selection in own products (i.e. the cell), rather than with the coevolutionary models environment, and the interaction of the cell with the environment just ensures continued existence By embedding coevolutionary systems in of the system. Any new ingredient of the system space, the interaction between the selfreplicating in the first place should be sufficiently compat- units may form spatial structures. The dynamics ible with what is already there. Moreover genes of these spatial structures may determine the are a passive form of information storage; only evolutionary fate of the selfreplicating units transcribed genes play any role in the system. which generate them and therewith the low level Thus the cellular environment should not only be definition of the (co)evolutionary system [l-4]. undisrupted by a new gene, it should also acti- In other words the ‘emergent properties’ of a vate it. Gene regulation has been modelled with (one-level) coevolutionary system may generate Boolean networks by Kauffman [13] and the a new level of selection. amenability of such networks to optimization, and coevolution has been studied by him in 1.3. Combining coevolution and coding models in which these processes are quenched into a powerful and extremely simple representa- Interesting combinations of these two ap- tion. proaches include the Tierra system of Ray In this paper we intend to focus on the internal P. Hogeweg I Physica D 75 (1994) 275-291 277

aspects of evolution by including explicit in- Embedding the coevolution in a spatial con- tracellular dynamics in an evolutionary system text does alleviate some of the difficulties occur- which takes a ‘solipsistic’ view of a cell. What- ring with the well-mixed systems: temporal fhc- ever a cell ‘is’ or ‘does’ is fine as long as it can tuations are stabilized, noncyclic interaction continue doing it. However the external environ- structures are stable, and the system can be ment is represented in the form of an optimi- resistant to parasites through the generation of zation criterion: the more ‘information’ the cell higher levels of selection. Nevertheless infor- carries the more it might be able to “do”. Thus mation accumulation appears to be limited also we maximize diversity of the system. in these systems, and in some cases we have shown that evolutionary processes tend to lead (3) Generation and maintenance of diversity the system to an area of parameter space where, in selfreplicating systems e.g., they are not resistant to parasites [l- 5,8,16]. Maintenance of diversity in non-self- Selfreplicating systems which interact through replicating systems is less of an issue. Fontana [7] competition for a common resource tend to therefore initially constrains his system excluding exclude each other. In an ecological context this selfreplication of his primary building blocks. In phenomenon is called “competitive exclusion”, fact ecological systems as well as evolutionary in evolutionary context “survival of the fittest”. systems and also other selfreplicating systems Although the former term has definitely a more such as the immune system, are not purely negative connotation than the latter, the mainte- selfreplicating systems: influx from the ‘external’ nance of diversity is a crucial question in models environment (other areas, seed banks, bone of evolution as well. This is in particular the case marrow/thymus), or by mutations occurs. In- because the amount of information which can be deed such an influx tends to stabilize the system, stored in one replicator is limited by the exist- as is shown by Stadler et al. [21,22] in replicator ence of a so called ‘error threshold’ (or ‘in- systems including mutations. formation threshold’): only by reducing the mu- In this paper we study the generation and tation rate can more information be included in maintenance of diversity as mediated by higher one replicator. However, the reduction of the level selection and influx. The higher level pro- mutation rate is presumably only possible by cesses maximising diversity select interaction more sophisticated replication mechanisms (e.g. topologies and can control the influx of the proofreading), which require more information ‘environment’ (in our system the ‘genome’). to be stored in the genome. Therefore an important open question in (prebiotic) evolution is (4) De-coupling of dynamics and information how the information threshold can be crossed storage and how information accumulation can occur. Eigen and Schuster’s classical proposal to cross Complex information processing can occur in the information threshold by coevolution of dynamical systems when there are entities catalysing each other’s selfreplication, is (1) stable configurations which provide for in- hampered by the difficulty of maintaining high formation storage; diversity in selfreplicative systems. In well-mixed (2) moving configurations (for information systems only relatively small cyclic interaction transfer) structures can persist, and these structures are (3) nonlinearity (decision making) (e.g. [24]). vulnerable to ‘parasites’, i.e. molecules which Clearly these requirements are met in some receive but do not give catalysis to other mole- dynamical systems, i.e. those systems which in cules. Such parasites are likely to arise in mutat- CA are called type IV systems, and occur near ing systems. the onset of chaos. Interesting as this notion is, 278 P. Hogeweg I Physica D 75 (1994) 27-F-291 biological systems may store information in other as a dynamical system of coevolving entities and ways, e.g. by ‘using’ long lived molecules. DNA as optimization process, using genetic algo- can be considered such a long lived analogue of rithms. The optimization with genetic algorithms RNA. Likewise memory cells in the immune may be considered as a convenient and powerful system may be used for immunity, rather than way to investigate the properties of the low-level network configurations. Information storage in dynamical system [20], or as an extension of the the form of long lived molecules decouples biotic system under investigation, i.e. to a biotic information storage to a certain extent from the systems consisting of ‘cells’. parameters of the dynamical system. The CAs harbour a collection of replicators In this paper we study the potential of this which are semi-isolated from the environment. type of information storage, which takes the Each cell (automaton) of the CA can contain just form of the control of information influx in a one of these replicators, or is empty. In this replicator system. baseline study the structure of the replicators is not considered and their interactions are as- signed randomly. The building blocks of the 2. The model multilevel evolution system replicators are not considered; the space limitation introduced automatically by the CA formal- We investigate the role of information storage, ism is assumed to represent any limitation which local information processing and pattern forma- might occur. There is a fixed set of possible tion for the potential to evolve species-rich replicators in the universe; in most simulations catalytic networks of selfreplicating units. For shown there are 160. This set is apriori fixed to this purpose we embed random catalytic net- facilitate the comparison of different selection works of selfreplicating units in a spatially ex- regimes. The replicators can catalyse the replica- tended system, i.e. in a Cellular Automaton tion of other replicators. These interactions are (CA), add a second layer to the CA containing fixed at the beginning of the simulation. We long lived counterparts of these replicators for study the effects of varying the interaction fre- information storage, and embed the CA in quency. compartments and study a population of these The CA’s transitions are defined in 4 processes. under selection for ‘diversity’ (= number of (1) Catalysis: the amount of catalysis each species per ‘compartment’). replicator (or its long lived counterpart in the Thus we have a 4 level evolutionary system: genome layer) gets, is calculated as the sum of (1) the individual low level replicators, which are catalytic interactions it has with its 8 neighbours represented as the local states of the cellular in the replicator layer. automata, (2) the meso-scale patterns generated (2) Replication and Decay: if a CA cell in the in the CA by the dynamics of the catalytic replicator layer is empty (i.e. is in state = 0) its network, which (co)determine the fate of the next state is equal to the state of its neighbour replicators as discussed above, and (3) the popu- which gets maximum catalysis; its neigh- lation level which causes only the most diverse bourhood consists of the 8 neighbouring cells in CA’s to survive and replicate, and therefore both CA layers. If none of the neighbours gets codetermines the evolutionary fate of mesoscale catalysis the cell remains empty. If the cell is in patterns as well as the replicators and the in- any other state (i.e. contains a replicator) it has a formation storage thereof. In addition there is a certain (species specific) probability to decay and ‘genome’ in which information is stored. This become empty. four level evolutionary system uses both (3) Mixing: replicators move either by diffu- paradigms for studying biological evolution, i.e. sion (i.e. they move in a random walk fashion to P. Hogeweg I Physica D 75 (1994) 275-291 279

neighbouring cells, using the diffusion algorithm simply adding a randomly chosen replicator-ana- of [23]) or they move by complete stirring: each logue to a position in the genome layer. This replicator is moved to an arbitrary location in the position is chosen as follows: a random position CA, i.e. we model the mean field situation. (x, y) is selected; when no position in a (49-) Replicators can diffuse out of the CA (i.e. the neighbourhood of X, y contains a replicator-ana- CA has zero boundary conditions). We compare logue, a new one is added at position x, y (i.e. the evolutionary consequences of these two gene duplication + mutation takes place); other- different mixing regimes. wise the new replicator-analogue replaces an (4) Somatic mutations: in addition ‘somatic existing one in this neighbourhood (i.e. mutation mutations’ can be added to the CA’s transition without duplication takes place); if moreover rule by randomly changing replicator-types. We position X, y contains a replicator-analogue, it is studied high and low mutation rates (50 or 1 deleted (genedeletion). This procedure ensures mutations/CA-generation) as source of nonselec- that genome size is limited: once the genome tive information and/or disruption of the system. layer becomes densely populated, replacements A second layer of the CA is passive and and deletions predominate. However in the represents the ‘genome’, i.e. contains long-lived initial stages there is a strong bias to an increase counterparts of the replicators. In analogy with of information storage. In this way we make the the situation for DNA and RNA, the long lived build-up of information relatively ‘easy’. The size elements of the genome layer can be transcribed of the neighbourhood (49) is an arbitrary param- into the upper layer of the CA forming the eter of the model, chosen such that the genome corresponding replicator. The reverse process is will build up to about 120 elements during our not taken into account in the experiments de- runs. scribed here. Transcription takes place through Replication of the entire CA occurs period- catalysis by replicators, analogous to the catalysis ically. The CA’s with the highest number of of replicators among themselves, i.e. the non- replicator-types are preferentially selected for zero elements in the genome layer get catalysis replication, those with the smallest number are by the replicators in the neighbouring positions replaced. During replication crossover can occur: in the replicator layer, and if they get most in that case two cells fuse, by combining part of catalysis of any of the neighbours of an empty one CA with the complementary part of the position in the replicator layer, they are tran- other. thus combining both replicators and scribed into this layer. Genome elements do not genomes. catalyse replication or transcription. They thus act as a passive information store, activated only by their collective transcription products. The 3. Local vs. global competition same network of catalysis is used of replication and transcription. The dynamics of replicator systems are well At initialization the first layer of the CA studied in terms of differential equations contains a number of replicators and each re- [6,9,19,21,22]. We focus here on replicator sys- plicator type has just one representative in the tems with only positive interactions (sometimes genome layer; they are randomly distributed called Hypercycles). In contrast to differential over this layer, which is empty elsewhere. New equation models we study replicator systems genome elements are added during replication of under discrete, finite conditions and with local the entire CA (see below), by a process ana- rather than global competition. Local competi- logous to gene duplication with mutation, or tion arises because only the replicators surround- mutation alone. This process is implemented by ing an empty position compete for its occupation 280 P. Hogeweg I Physica D 75 (1994) 275-291

(and do so in a nonlinear way) whereas in the able, as is the number of replicators active over a usual ODE models global resource competition period of time rather than simultaneously. The is generally assumed (e.g. chemostat condition). shape and stability of the attractor attained Global resource competition ensures that compe- under selection at the population level was tition is local in terms of interactions of re- assessed by continuing its dynamics for 5000 plicators: only those being catalysed by the same steps in isolation, measuring the mean number of other replicators compete directly, and those replicators present over time as well as the total getting the largest (net) catalysis will ultimately number and by visual inspection of space-time survive. In contrast local resource competition plots (see plates). Another important observable makes competition in interaction space a global is the topology of the attained networks (see figs. affair. For example giving catalysis gives now a 1, 2). Indeed these other measures are more greater disadvantage to the replicator doing so informative about the various evolution than to the others because it is in its own local scenarios than is the fitness criterion itself. surroundings that it generates a stronger com- petitor. One of the consequences is that in contrast to differential equation models, systems with parasites (i.e. replicators which receive but 5. Initial observations and default parameter do not give catalysis) can be very stable ([1,2]; settings see results). Local interactions lead to spatial pattern formation. Thus pattern formation and Initial observations led to the default settings local interactions are most often observed to- of the parameters. In this section we discuss gether. In this paper we differentiate the two by these shortly. introducing in a local interaction system two Initialization. We initiate the population in each different mixing regimes, i.e. local diffusive CA with 30 replicators (and their long lived mixing (i.e. each replicator can move into neigh- analogues) drawn randomly from the 40 lowest bouring patches in between every interaction numbered replicators. Thus we ensure a moder- step) and complete mixing, i.e. in between ate diversity of the initial population, as well as interaction steps all replicators are moved to an the necessity to incorporate information not arbitrary patch in the system. It turns out that available initially into the system. These random- both types of mixing markedly affect the strate- ly initiated CAs most often degenerate to very gies for information generation and conservation simple systems (2-3 replicators) or die out com- open to such systems and/or preferentially ex- pletely. This is in particular the case for lower ploited by them. connectivities and for the well-mixed systems: in the latter often only a few CAs ‘survive’ to the first replication. Nevertheless the population 4. Observables of the system level optimization is able to generate diverse networks eventually. The primary observable of this system is of Population size. Initially a population size of 36 course the value of the fitness criterion attained was used, i.e. the maximum convenient popula- under the various selection regimes. The fitness tion size. It appeared that halving this size did criterion, i.e. the number of replicators simulta- not affect the results provided the initial popula- neously present in the upper layer of the CA, is, tion contained some ‘surviving’ members. We however, but one of the ways in which the therefore used a population size of 18 through- diversity of the system can be measured. The out the reported runs. Where the initial popula- size of the genome is another interesting observ- tion did not survive until the first CA replication P. Hogeweg I Physica D 7.5 (1994) 275-291 281 a few other initiations were executed, before finally settled on the deterministic rule explained deciding that the system was not ‘viable’. above in order to avoid unnecessary variability. CA replication rate. Population level optimi- The detailed properties of the system do depend zation is more successful when replication of the on the particular choice of the replication rule CAs happens frequently, i.e. before it has settled and the size of the catalysis neighbourhood. in its attractor. Fusion of CAs (i.e. the crossover Nevertheless, our observations gave us confi- operator) is more effective during transients than dence that the major conclusions do not crucially when stable attractors are reached. In the latter depend on this choice. case most often the result is just that one CA size. The size of the CA codetermines the attractor out-competes the other one. At present type of patterns which can persist. If the scale of we have not analysed this interesting phenom- a pattern is big it might not fit into a small space. enon further, but just used the observation to This is the case, for example, for spiral patterns choose the default CA replication rate of once in formed by cyclic catalytic interactions among a 125 time steps; replication rate of once in 250 large number of species [l]. The limitation of the time steps appeared to be as efficient, but size of the system to 100 x 100 cells may indeed requires of course more simulation time. prevent the formation of interesting larger scale Population level selection. The dynamics of the patterns, but this may also be the case in the system introduces much temporal variation in spatial confines of cells. Nevertheless, visual the value of the optimization criterion, i.e. observation in our systems did not reveal many number of species ‘active’ in the system at the large scale patterns if larger automata were used. time of the CA replication. Therefore strong We did our comparative runs on a size of 100 X selection could be used without constraining the 100, and limited diffusion to one step per genera- search space of the genetic algorithm too much. tion. For display purposes, Plates I and II show Therefore both “reproduction of the fittest” as experiments on a CA of size 160 x 160: the well as “non-survival of the non-fittest” were equivalent runs on 100 x 100 revealed similar used. Thus, CAs to be killed were selected patterns. uniformly on the basis of (Max N - N)3; and Duration of observations. Population size is only CAs to reproduce were selected uniformly on slowly increasing after t = 25 000; stability still the basis of N” (N = number of species>50 X does increase in the spatial systems. Longer runs present in the CA; Max N = maximum of N in were sometimes used to check stability trends, the population). but the main comparisons were done after 25 000 Decay rate and strength of catalysis. In order to steps (= 200 CA generations). avoid ‘ties’ and very slow drift of competing replicators, variation in decay rate and catalysis strength were set to moderate levels, i.e. decay 6. Requirements and strategies for generating rate is drawn uniformly between .15 and .25, and and maintaining diversity catalysis between 80 and 120. It turns out that the average decay and catalysis in the final In order to generate and maintain a large networks are close to the average value, showing number of replicators in the network, partially that fast decay and low catalysis can also be contradictory requirements should be met and ‘advantageous’ for replicators (compare [2]). different strategies can be used to meet them. Replicator reproduction. Several rules for re- Under different circumstances (i.e. mutation plicator reproduction were tested, including sto- rates, and connectivities) the difficulty of meet- chastic rules and rules in which non-catalysed ing these requirements varies. It turns out that replication was allowed with low probability. We the two mixing regimes differ significantly in 282 P. Hogeweg I Physica D 75 (1994) 275-291 their ability to meet the requirements and there- information: the spatial heterogeneity generates fore tend to apply different strategies under the a large number of different ‘opportunities’ for same circumstances. The opposing requirements invasion. At the same time pattern formation for generating and maintaining a high informa- can increase the possibilities for the elimination tion content (high diversity) are of replicators which did invade successfully: Incorporation of information: new entities waves can ‘push’ information out of the system. should be able to ‘invade’ in order to be able to However, unlike a high degree of connectedness, ‘test’ their suitability for inclusion in the reper- pattern formation does not prevent harmfulness toire. itself. Elimination of information: those entities which are ‘harmful’ for maintaining the diversity should be eliminated from the system. 7. Strategies selected Preservation of information: entities should be available whenever ‘needed’. Active replicators There, are three types of attractors which are not only needed for their direct contribution maintain diversity far above the average diversity to diversity, but also for the preservation of attained by an arbitrary network from arbitrary other replicators, for the incorporation of new initial conditions. These are: replicators and the elimination of harmful ones. Under high somatic mutation rates preserva- (1) ‘Invariant attractors’. tion of information is automatically achieved: all These attractors preserve (almost) all infor- information is present all the time in the form of mation in the form of replicators which are mutants (because the average lifetime of represent continuously in the system in fairly large plicators is 5 time steps, 50 mutations per time numbers. Thus the total number of replicators step maintains a store of 250 replicators indepen- occurring in the system is equal to the number of dent of network interactions). The crucial issue replicators present at any time. In our experi- under these circumstances is to be able to elimi- ments such networks consist of a maximum of nate those replicators which can out-compete all about 10 replicators in a number of interlocked others and therefore ‘kill’ the network. Con- cycles of length 3-4, and usually have one or trariwize, under lower mutation rates preserva- more ‘coupled parasites’ on this ‘core’. tion of information is the crucial issue, because An example. A typical example of such net- competitive exclusion tends to limit the amount works is shown in Fig. la; its dynamics are of information drastically. The ‘genome’ allows shown in Fig. lb. The lower panel contains the selective information conservation. However this core species which have relatively low densities information contributes to diversity only if it is and small fluctuations and the upper panel dis- activated and is able to invade. A high degree of plays the parasites. The attractor shown remains connectivity increases the possibilities for infor- invariant notwithstanding the presence of a mation to be incorporated in the network: there genome which contains counterparts of other are more ‘entry points’. If information can enter replicators: these replicators cannot invade; al- into the system easily, preservation as well as though they may be catalysed by the members of elimination of information becomes less of an the network they are always out-competed by issue: there are no really harmful replicators them. However, not for all species is the attrac- because inclusion of others is not preventable by tor uninvadable: ‘harmful’ species are excluded anyone and preservation is not necessary from the genome through population level selec- because there are enough new recruits. Pattern tion; exposing the attractor to an influx of 50 formation also increases the incorporatablity of random replicators per time step causes it to P. Hogeweg I Physica D 75 (1994) 27%291 283

parasites

species of 3 and 4 cycle

(b) O.O

Fig. 1. Attractor of well-mixed system with connectivity = 41160. (a) The selected subnet, consisting of a core of an interconnected 314 cycle and several parasitic chains. (b) Timk plot of densities: Upper panel; parasites; Lower panel 314 cycle.

collapse rapidly to 2 species only. Indeed the to the remaining, now stronger, parasites and is genome contains only 46 different ‘genes’ after killed, which in its turn kills the parasites as well. 25000 generations while the average influx of General properties and circumstances in which genes is over one hundred during this period. they are formed. Other, similar attractors are The attractor shown is also not self-sustaining: globally uninvadable; they are strongly selected the coupled parasite number 37 (upper panel) is in systems with high rates of somatic mutations. crucial for the maintenance of the attractor but is Globally uninvadable attractors occur also in not preserved by the dynamics in this (finite) genomic systems without somatic mutations. The system. It is provided by the genome whenever globally uninvadable attractors accumulate up to needed. If the influx is blocked the system de- twice as many genes in the same period as the stabilizes and all species die out within 2 000 time ones which have to battle harmful genes. These steps. Without no. 37, its ‘mother’ parasite accumulated genes are, however, not expressed eventually out-competes species 30 thus breaking unless the attractor is destabilized. These attrac- the 4 cycle; the remaining 3 cycle is not resistant tors are very stable in isolation. Fusion (recombi- 284 P. Hogeweg I Physica D 75 (1994) 275-291

nation) of two of these attractors which are out selection can lead the population back towards of phase may cause however destabilization. fixation of uninvadable attractors. These occa- Such destabilization will cause the expression of sional, most often harmful ‘excursions’ lead to hitherto unused genes which were accumulated the build-up of the larger uninvadable attractors only because they were ‘harmless’ under normal from the smaller ones (2-4) species which are circumstances. Invasion of these gene products initially selected. (i.e. new replicators) may lead the system back Well mixed systems with a genome and with a to the attractor, or may cause extinction after a connectivity of up to 7 catalytic links per node long transient. Notwithstanding the long tran- almost always evolve such an ‘invariant attractor’ sient relative to the mean life span (i.e. a few of 7-10 species under selection for diversity (see thousand vs. ca. 500 time steps), population level Fig. 2). Likewise well-mixed systems with high

total # species mean # species

mixed mixed -- 1

20 60

diftusive diffusive 100 T-----

20 60 t

40 10

0 0

Fig. 2. Diversity of attractors for different mixing regimes and different connectivities. Diversity is measured in isolated CA’s during a period of 5 000 timesteps, after a selection period of 25 000 timesteps in which the population of CA’s reproduce and fuse (crossover). The lefthand panels show the total number of species present in the period of 5 000 timesteps, the righthand panels the average number of species present simultaneously (the number of species present simultaneously is the selection criterion). Connectivities range from (on average) 3 to 8 connections per species (x-axis). 3 populations (of 18 CA’s) are included for each connectivity. The attractors of the well-mixed systems (upper panels) with connectivities upto 7 are of the “invariant” type: the total number of species and the average number of species are typically similar, although some are unstable which results in a larger total number of species and a smaller average number of species (i.e. they die out after a transient). For connectivity X “percolation” occurs with a large total number of species, but an average number of species equal to that of lower connectivities. The attractors of the diffusive systems (lower panels) have a permanent core which is successively invaded by different species: the total number of species is larger than the average number of species. The average number of species in the stable attractors are similar for all connectivities (but for higher connectivities a larger proportion of the population of CA’s is stable), the total number of species increases with increasing connectivity. Percolation occurs at c = 8: the absence of a permanent core is not reflected in either the total number of species or the average number of species. P. Hogeweg I Physica D 75 (1994) 275-291 285

rates of somatic mutations can attain such in- 2-cycle, i.e. 45-122 (red and yellow) can invade variant attractors which are globally uninvadable (these species are catalysed by 12 and 18, respec- for all mutants, or nearly enough so for popula- tively). This 2-cycle can however easily be in- tion level selection to eliminate harmful mutants vaded and extinguished by a variety of parasitic from the population. Well-mixed systems with species, which eventually also come to a dead- low mutation rates are generally not viable when end themselves. The dominant 2 cycle invades initiated with a small number of species (as is our the open area again, etc. default): the incoming mutants kill them. Nei- Color plate 2 and fig. 3b show the network as ther is a low mutation rate (one mutation per CA evolved by the same universe, and the same step) sufficient for preserving enough informa- initial conditions, but without somatic mutation tion for maintaining stabilizing second parasites and with information conservation in the in such attractors. In the pattern formation genome. A quite different attractor is selected by systems these invariant attractors are not found. the system, which is not resistant to a high influx Their higher invadability prevents such an ‘isolis- of arbitrary species by somatic mutation, but tic’ strategy and their potential for dynamically which can maintain itself with a genome. With- containing invaded species makes it unprofitable. out genome the system persists in impoverished Only for very low connectivities evolved attrac- form: the 10 ‘core’ species (i.e. those permanent- tors of diffusive systems resemble the attractors ly present in the system) remain (colour plate described in this section. 2b.) The core consists of 2 domains: one dominated by a 2-cycle; the other consists of a (2) Invasions and the preservation of an in- number of interlocked 3-cycles and parasites. variant ‘core’. The latter domain is ‘fed’ by the first domain: a In evolved diffusive systems only a minority of parasite of the first domain is a member of a the species which do occur during a 5 000 time 3-cycle of the second domain. The second do- step period are present continuously during that main is eventually killed by waves of its parasitic period (see Fig. 2). Most species just appear species. The empty space is then reoccupied by once in a while. There is however a core of the 2-cycle, and the process continues as before. species which dynamically maintains itself. This Thus a fairly regularly large scale space time core of species allows the invasion of potentially pattern is generated. very long chains (networks) of other species. It is clear from the picture that the space-time This ‘parasitic’ network is not stable, and will behaviour of the full system is very much shaped eventually die out and free the space it occupied; by the permanent core species: the genome only subsequently the core species recolonise this provides some additional transient species which area. make detours or shortcuts between the phases, Examples. Color plate 1 shows such an attrac- and thereby make a more irregular pattern. tor for a diffusive system without a genome but a Color plate 2a is generated by initially mixing the high rate of somatic mutations (50 per time step) evolved pattern. This disturbance killed species ensuring species preservation. Panel 1 shows a 16, i.e. the catalytic connection from domain 1 to space-time plot, panel 2 a species profile, panel 3 domain 2 so that the domains remain connected the number of species present over time and the only through a common parasite. The system last panel, four snapshots of the CA. The cata- then becomes dominated by domain 1 which is lytic network of the species present is shown in regularly locally extinguished by its parasite, but Fig. 3a. The attractor is dominated by the 2-cycle rapidly re-invades the barren area left when the of species 12-18 (coloured grey). This 2 cycle is parasite dies out. Domain 2 persists as well, and the ‘sink’ in which the stirred counterpart of this can occasionally out-compete domain 1 when the system falls. Due to pattern formation another latter is ‘suffering’ from a parasite which does 286 P. Hogeweg I Physica D 7S (1994) 275-291

I Ii6 60

143

136

Fig. 3. Selected subnets in diffusive system with connectivity = 41160. The encircled species are permanently present. All three are subnets of the same work. (a) Subnet selected in system with high somatic mutation rate and without a genome. (b) Subnet selected in system with genome without somatic mutations. (c) For comparison: subnet selected in the same system under global mixing. not affect the former. The surprising stability of but with slow return dynamics. This type of systems consisting of subnets interacting through ‘solution’ is found mainly in diffusive systems a common parasite was also noted by [3]. with a genome, and occasionally in diffusive General properties and circumstances in which systems without a genome and with a high rate they are formed. In these systems with a perma- of somatic mutations. In order to be generated nent core which tolerate and contain parasitic an influx of information and selection at the invasions, the problem of maintaining a large population level are necessary in the early (instantaneous) diversity is ‘solved’ by selecting stages. Once formed they are stable in absence an attractor with a large domain of attraction, of population level selection; the core is even P. Hogeweg I Physica D 75 (1994) 275-291 287

Color plate 1. Attractor of subnet selected in diffusive system with high rate of somatic mutations (Fig. 3a). (a) (lefthand panel): space-time plot of crosssection of CA; vertical axis: time (downwards); horizontal axis: space. (b) (2nd panel): species profile through time. (c) (3rd panel): number of species present through time. (d) (righthand panel): 4 snapshots of CA taken at time of their lowest edge.

maintained without information influx from the recombination (fusion)) which are not long term genome or through somatic mutations. Although stable and are eliminated by population level the populations converge to such a self maintain- selection. These non-stable variants decrease the ing attractor, they do contain a large number of mean number of species per CA; therefore the variants (created by genomic mutation or by mean diversity (whether measured at any one 288 P. Hogeweg I Physica D 75 (1994) 275-291

Color plate 2. Attractor of subnet selected in diffusive system with genome (Fig. 3b.) The figure shows 3 space/timeplots star ting in the same initial state (i.e. the state of one CA after 25 000 timesteps of selection). (c) (righthand panel) shows the space/t patter n in the full system, with genomic influx. (b) (middle panel) shows the space/time pattern in the system when the geno influx is blocked (only permanent species survive). (a) (lefthand panel) shows the space/timeplot in the system when gena lmic influx is blocked and the initial spatial pattern is randomized. Only an improvized form survives in which the 2 subnets are tonne cted via a common parasite as species 16 is not present (see Fig. 3b).

time , or over a period of 5 000 time steps) is not exceeds the repertoire of the invariant syste :ms mucl 1 larger in these attractors than in the case (see fig. 2). of tlhe invariant attractors occurring in well- Diffusive systems without a genome and I ow mixe d systems although the repertoire used in rates of somatic mutations were not observed I to both the stable and the unstable individuals far form selfmaintaining systems of this type: tl w P. Hogeweg I Physica D 75 (1994) 275-291 289 still rely on population level selection for their potential for selecting specific network maintenance after the observation period and topologies: e.g. the distribution of minimal cycle the mean diversity in these populations does not lengths is determined by connectivity of the exceed 10 species. Containment of invasions by entire network of 160 species. By selecting spatial localization is, of course, not possible in subsets of species the profile of cycle lengths can well-mixed systems. Nevertheless a limited form be modified but only to a limited extent. The of allowing and containing invasions was ob- transitions between attractors with and without a served in well-mixed systems as discussed above. permanent core occurs for connectivities be- In both cases the containment of invasions de- tween 6 and 9, i.e. where the majority of species pends on both the availability of enough in- attains a minimal cycle length of 2-3, instead of formation to do so (i.e. by secondary ‘harmless’ 3-4 or longer as is the case for lower connec- invasions) and on the avoidance of some types/ tivities. The interaction topologies of the in- magnitudes of invasions (by selection at the variant attractors of well mixed systems always genomic level). consists of interconnected cycles of length 3-4, with some short parasitic chains attached. The (3) Percolation and long transients. permanent core of the invadable attractors of the High connectivities with and without pattern diffusive systems are dominated by 2-cycles, formation allow the system to wander all over notwithstanding the scarcity of 2-cycles in the the 160 dimensional state space if information overall network for the (relatively low) connec- preservation is ensured, i.e. under high mutation tivities in which they are found. Notwithstanding rates or when enough information has accumu- these opposite ‘choices’ of the well-mixed and lated in the genome. Thus the total amount of the diffusive system, stability of both appears to information is accessed over time in both the depend on the absence of a too strong (self stirred and in the diffusive system. In these reinforcing) parasitic ‘shadow’. In the well mixed systems we have crossed a ‘percolation thres- percolating systems species occurring simulta- hold’. New mutants can always enter the system, neously form parasitic chains but most often do and so diversity is maintained. In the diffusive not form cycles: indeed cycles are not expected system up to more than 30 types of molecules are for subsets of ca. 10 species for the connectivities present simultaneously in more than 50 copies in used (maximal 10/160). In the diffusive percolat- a 100 X 100 field, whereas in a completely mixed ing systems diversity is large (up to 30) and system not more than 12 molecules are present cycles do occur, but none of them is resistant to at any one time. Dynamically information is not invasions. preserved in either case: none of the species is In future research we will use pattern match- present all of the time. At a high somatic ing between ‘genotype’ or ‘phenotype’ (e.g. mutation rate percolation-like behaviour also secondary structure) as the basis of interaction occurs at relatively low connectivities: in those between replicators. This enables us to extend cases however the trajectory is eventually the number of species, without necessarily trapped in a state which is not invadable any- changing the profile of cycle length drastically; more, e.g. dies out. e.g. a new species ‘similar’ to an existing one will have similar catalytic connections and thus add connectivity without altering cycle lengths. 8. Cycle lengths in fixed random networks

The experiments reported in this paper are 9. Discussion and conclusions intended as a baseline for future research. The closed universe of random interactions limit its One may object to our experiments that our 290 P. Hogeweg I Physica D 7.5 (1994) 275-291 systems ‘do not do’ anything with the infor- taining high diversity according to our fitness mation which they maintain. This is indeed the criterion as well as according the total number of case. However, I think that maintenance of species present, its inability to focus the system information may be the limiting factor. Infor- on some permanent information seems to render mation which can be preserved will, because of it a less attractive solution for evolutionary its preservation, be ‘selected’ as substrate for the systems. Whether selection for permanent in- maintenance of other information, and thus formation can attain species rich attractors with a obtain, secondarily, a ‘function’. Because biotic preserved ‘core’ in the circumstances which now systems do not have any apriori goals they can selects percolation is an open question. simply do what they are able to do. Intermediate between invariance and isolation We have shown that maximalization of the on the one hand and percolation on the other number of species in replicator systems can be hand, the ‘solution’ of maintaining diversity achieved in three different ways: through contained invasions while preserving a (1) invariance and isolation; permanent autocatalytic core seems the most (2) permanence and contained invasions; powerful strategy found. It is selected always in (3) percolation and long transients. diffusive systems with a genome and low to Diversity by invariance and isolation is selected intermediate degree of connectivity and some- in well-mixed systems and, once formed, de- times in diffusive systems without a genome but pends for its maintenance often only weakly on with relatively high mutation rates. In these influx and higher (population) level processes; systems the existence of spatial diversity permits indeed its stability is mainly hampered by popu- and constrains temporal diversity and vice versa. lation level recombination. Only a small reper- For assessing the persistence of such systems toire of species is expressed. The concept invadability criteria, and related concepts, are ‘evolutionary stable strategy (ESS)’ [15] refers to useless (compare [4]). This ‘strategy’ employs all population level processes. The isolistic selection levels (when available): genomic, re- ‘evolutionary stable strategy’ operating through plicator network, meso-scale patterns and CA regulation and intra-compartment competition, population. It generates as side-effects of max- found here, appears to be an alternative which imising our fitness criterion (i.e. a large number deserves to be used as search image in the study of different replicators present at any one time), of biotic evolution. This ‘strategy’ may be re- a large repertoire of species over longer periods, sponsible for the occurrence of species which and processes reminiscent of adaptability, ex- have maintained their characteristic over exces- citability and signal transduction: i.e. properties sively long periods of time (‘living fossils’). which are often taken as (co)-‘defining’ living Interestingly these often occur in circumstances systems. where we would expect relatively high mutation rates. as we have shown high mutation rates favour this type of attractors in well mixed low Acknowledgements connected systems. Long transients and percolation occur in sys- I thank Maarten Boerlijst for his contributions tems with high connectivities and/or under high to understanding evolutionary consequences of nonselective influx. Even if percolation leads pattern generation in replicator systems. Mike eventually into a sink, population level selection Dale reviewed the manuscript and gave useful may maintain relatively high diversity in such suggestions. The conceptual framework leading systems due to the long transients. Although to this research is shaped by my long term percolation indeed ‘solves’ the problem of main- collaboration with Ben Hesper. P. Hogeweg I Physica D 75 (1994) 275-291 291

References [12] M.A. Huynen and P. Hogeweg, Pattern generation in Molecular evolution; exploitation of the variation in RNA landscapes, Mol. Ecol. submitted; in: Thesis [ll M.C. Boerlijst and P. Hogeweg, Spiral wave structure in Utrecht University (1993). prebiotic evolution: hypercycles stable against parasites, [13] S.A. Kauffmann, The Origins of Order; selforganization Physica D 48 (1991) 17-28. and Selection in Evolution (Oxford Univ. Press, Ox- PI M.C. Boerlijst and P. Hogeweg, Selfstructuring and ford, 1993). selection, in: Artificial Life II, C.G. Langton, C. Taylor, [14] C. Lindgren, in: Artificial Life II, C.G. Langton, C. J.D. Farmer and S. Rasmussen, eds. (1992) pp. 255- Taylor, J.D. Farmer and S. Rasmussen, eds. (Addison 276. Wesley, Reading, MA, 1991) pp. 295-312. ]31 M.C. Boerlijst and P. Hogeweg. Evolutionary conse- [ 151 J. Maynard Smith, Evolution and the Theory of Games quences of spatial pattern formation, in: Spatio-tempo- (Cambridge Univ. Press, Cambridge, 1982), 224~~. ral organization in nonequilibrium systems, S.C. Muller [ 161 M. Nowak and R.M. May (1993) The spatial dilemmas and T. Plesser, eds. contributions to the Dortmunder of evolution, lnt J. of Bifurcation and Chaos 3 (1993) Dynamische Systemen Woche, June 1992 (Project Ver- 35-78. lag, 1992) pp. 26-28. [17] T.S. Ray, An approach to the synthesis of life, in: (41 M.C. Boerlijst, M.E. Lamers and P. Hogeweg, Artificial Life II, C.G. Langton, C. Taylor, J.D. Farmer Evolutionary consequences of spiral waves in a host and S. Rasmussen, eds. (Addison Wesley, Reading, parasitoid system, Proc. R. Sot London Ser. B 253 MA, 1991) pp. 371-408. (1993) 15-18. [18] T.S. Ray, Evolution, complexity, entropy and artificial ]51 H.N. Comins, M.P. Hassel and R.M. May, The spatial reality, Physica D 75 (1994) 239, these proceedings. dynamics of host parasite systems, J. Anim. Ecol. 61 [19] P. Schuster and K. Sigmund, Replicator dynamics, J. (1992) 735-748. Theor, Biol. 100 (1983) (Reading, MA, 1991) 533-538. ]61 M. Eigen and P. Schuster, The hypercycle: A principle (201 K. Sims, Poster 1st European conference on Artificial of selforganization (Springer, Berlin, 1979). Life (Ecal I), Paris (1991). ]71 W. Fontana, Algorithmic chemistry, in: Artificial Life [21] P.F. Stadler and P. Schuster, Mutation in autocatalytic II, C.G. Langton, C. Taylor, J.D. Farmer and S. reaction networks, an analysis based on perturbation Rasmussen, eds. (Addison Wesley, Reading, MA, 1991) theory, J. Math. Biol. (1992) pp. 579-631. pp. 159-209. [22] P.F. Stadler, W. Fontana and J.H. Miller, Random ]81 M.P. Hassel, H.N. Comins and R.M. May, Spatial catalytic reaction networks, Physica D 63 (1993) 378- structure and chaos in insect population dynamics, 392. Nature 353 (1991) 255-258. [23] T. Toffoli and N. Margolus, Cellular Automata ma- ]91 J. Hofbauer and K. Sigmund, The theory of evolution chines: a new environment for modelling (MIT Press, and dynamical systems (Cambridge Univ. Press, Cam- Cambridge, MA, 1987). bridge, 1988). [24] S. Wolfram, Universality and complexity in cellular P. Hogeweg and B. Hesper, Evolution as pattern ]lOl automata. Physica D 10 (1984) pp. 15-57. processing: TODO as a substrate for evolution, in: [25] C. Lindgren, Evolutionary dynamics of spatial games, From Animals to Animats, J.A. Meyer and S.A. Wil- Physica D 75 (1994) 292, these proceedings. son, eds. (MIT Press, Cambridge, MA, 1991). [ill P. Hogeweg and B. Hesper, Evolutionary dynamics and the coding structure of sequences: Multiple coding as a consequence of high mutation rates, Comput. Chem 16 (1992) 171-182.