Netcrawling - Optimal Evolutionary Search with Neutral Networks

Centre for the Study of Evolution Centre for Computational Neuroscience and Robotics School of Cognitive and Computing Sciences University of Sussex Brighton BN1 9QH, UK March 14, 2001

Abstract- Several studies have demonstrated that in the selectively neutral mutation [19, 5] initiated a debate amongst presence of a high degree of selective neutrality, in par- biologists which continues to this day. More recently, re- ticular on fitness landscapes featuring neutral networks, search into the structure of RNA secondary structure folding evolution is qualitatively different from that on the more landscapes [8, 10, 14, 24] led to the concept of neutral net- common model of rugged/correlated fitness landscapes of- works. These are connected networks of genotypes which ten (implicitly) assumed by GA researchers. We charac- map to the same phenotype, where two genotypes are “con- terise evolutionary dynamics on fitness landscapes with nected” if they differ by one (or possibly a few) point muta- neutral networks and argue that, if a certain correlation- tions. It is found that the dynamics of populations of geno- like statistical property holds, the most efficient strat- types evolving on fitness landscapes featuring neutral net- egy for evolutionary search is not population-based, but works differ qualitatively from population dynamics on more rather a population-of-one netcrawler - a variety of hill- conventional rugged/correlated landscapes as might be en- climber. We derive quantitative estimates for expected countered in either the biological or artificial evolution liter- waiting times to discovery of fitter genotypes and discuss ature. Although the structures of fitness landscapes arising in implications for evolutionary algorithm design, including the context of artificial evolution are ill-understood, a cursory a proposal for an adaptive variant of the netcrawler. examination of the types of genotype to phenotype mappings deployed in many real-world applications of artificial evolution (e.g. evolution of neural network controllers in robotics, 1 Introduction on-chip electronic circuit evolution, etc.) suggests that we might expect a substantial degree of neutral mutation. Work In the GA community at large there is a widely-held percep- here at Sussex suggests that this neutrality may well take the tion of fitness landscapes as being characteristically rugged or form of neutral networks as envisaged by RNA researchers. semi-correlated, a perception that has been reinforced by the In this paper we argue that, given large-scale neutrality, study of “toy” problems and highly artificial multi-peaked test the traditional view of evolutionary dynamics may be largely functions. This, coupled with historical factors within the de- irrelevant to the GA practitioner concerned with real-world velopmentof GA’s as a search technique(such as the Building problems. In Section 2 we characterise evolutionary search Block Hypothesis), have led to what might be characterised on fitness landscapes featuring neutral networks and address as the “Big Bang” view of evolutionary search (see Section what is for the GA practitioner perhaps the most pertinent as- 2.1). At Sussex University there is a tradition of the applica- pect of evolutionary search: how long can we expect to wait tion of evolutionary techniques to decidedly real-world prob- to see improvements in fitness and how should we design and lems, such as on-chip hardware evolution [12, 27, 21, 29] tune our search algorithms so as to effect the most efficient and evolutionary robotics [4, 11, 15, 16, 26], from which has search? In Section 3 these issues are analysed quantitatively emerged a markedly different view of the nature of artificial for a class of fitness landscapes with neutral networks satisfy- evolutionary fitness landscapes and their concomitant evolu- ing a statistical property that we term -correlation. We show tionary dynamics. In particular the presence and significance that there is an optimal mutation scheme for such landscapes of neutral evolution has come under the research spotlight. and conjecture that there is too an optimal evolutionary search The major impetus for this research has come from evolu- algorithm, the netcrawler. Section 4 presents a summary of tionary biology; ironically, GA research has traditionally re- results. mained somewhat isolated from its biologically-inspired origins. The work of the population biologist Motoo Kimura on 1 2 Evolutionary Dynamics on Neutral Networks discovered, or...

¯ ... all genotypes on the current highest neutral network 2.1 Overview are lost due to sampling noise - this phenomenon is related to the concept of the error threshold [23]. Fig. 2.1 captures many of the salient features of evolutionary

¯ If a higher fitness genotype is discovered it will survive dynamics on a fitness landscape featuring neutral networks1 and drift to fixation with a certain probability [5]. as identified in [14, 13, 30, 2, 3]:

¯ During the transient period when a higher fitness genotype is fixating, the population becomes strongly converged genetically (this phenomenon is variously known in the literature as “hitch-hiking” or the “founder effect”), as the higher fitness portal genotype and its selectively neutral mutants are strongly selected at the expense of the old population. Now a more “traditional” view might impute a somewhat different interpretation to Fig. 2.1. It might be assumed that epochs correspond to episodes during which the population fitness is entrapped in the vicinity of a local fitness optimum, while transitions to higher fitness levels signify discovery of higher local peaks. Broadly, the traditional view might be characterised thus: recombination assembles fitness-enhancing mean “building blocks” present in the population into higher fit- best ness genotypes (the Building Block Hypothesis); mutation is time merely a “background operator” to prevent total loss of ge- Figure 2.1: Typical evolutionary dynamics on a fitness land- netic diversity. This process continues as long as there is scape featuring neutral networks. sufficient genetic diversity in the population for recombination to work with. Once genetic diversity has waned (in- In summary: evitably so, due to the combined effects of selection pressure

¯ Evolution proceeds by fitness epochs, during which the and finite-population stochastic sampling) the population is mean fitness of the population persists close to the fit- deemed “converged” and no further fitness improvements are ness of the fittest genotype(s) currently represented in likely. the population. Thus it is deemed necessary to initiate the GA with a (usu- ally large) randomly generated population - that is, with a ¯ Transitions to higher fitness epochs are preceded by the discovery of higher fitness genotype(s) than currently “Big Bang” of genetic diversity for recombination to work reside in the population. upon. This perception goes some way to explaining the ob- session of much GA research with “premature convergence”

¯ Transitions may be down as well as up; the current and the multitudinous schemes prevalent in the literature for ﬁttest genotype(s) may be lost. the avoidance thereof. In this author’s view there are several

¯ The discovery of a higher ﬁtness genotype does not serious ﬂaws to this picture: necessarily initiate a new epoch; the new genotypemay

¯ There is scant evidence that real-world artiﬁcial evo- be quickly lost before it can become established in the lution ﬁtness landscapes conform to the rugged (non- population. This may be repeated several times. neutral) picture often implicitly assumed.

¯ If a higherﬁtness genotype does initiate a ﬁtness epoch,

¯ It is not clear whether “building blocks” in the conven- there is a transition period, brief compared to a typical tionally understood sense will necessarily exist. epoch duration2, during which the population mean fitness climbs to the new epoch level. ¯ Even if there are building blocks, serious doubt must be cast on the Building Block Hypothesis - see for in- Through the work of various researchers a consistent expla- stance [9]. Basically, due to the comparatively brief nation has emerged for the features presented above: takeover period of any genotype with a fitness advan- ¯ During a fitness epoch the population is localised in se- tage (see Fig. 2.1) and the effects of hitch-hiking, mul- quence space, somewhat like a classical quasi-species tiple building blocks are never likely to be simultane- [7]. The fittest genotypes reside on a neutral network, ously present in a population. along which they diffuse [14, 6], until either...

¯ ... a portal [30] to a higher ﬁtness neutral network is

1The landscape in Fig. 2.1 is an NKp landscape [2, 3], the GA standard ﬁtness-proportional roulette-wheel selection, with a ﬁxed population size. 2Indeed so brief as to be virtually indiscernible on the time-scale of Fig. 2.1.

These points lead us to question the role of recombination. the number of copies of genotype in the population. The

For if recombination is not assisting us by assembling build- population size is given by g . We deﬁne an evo-

g ¾Ë

Ò´µ ing blocks what is its purpose? Might it help us ﬁnd build- lutionary process on Ë to be a stochastic process [17]

ing blocks? In this author’s opinion there is no good reason with state space the set of all populations on Ë. In this pa- to think so; at best recombination will function as a macro- per all evolutionary processes will be assumed to satisfy the

mutation operator - a “leap in the dark” across the ﬁtness following: landscape. Indeed, it might be remarked, in biological evolu- ¯ The time parameter is discrete.

tion the role of recombination is almost certainly not the as- ¯ Population size is ﬁxed. sembly of putative building blocks [22], although other mech- anisms such as error repair [20] are considered plausible. ¯ The process is Markovian [17].

This is not to imply that there is no role for recombina- ¯ At each time step a number of genotypes are se- tion; indeed, GA practitioners commonly report a significant lected (independently and possibly with replacement) improvement in search efficacy with recombination. Never- for replication. A copy of each selected genotype then theless, in this paper we reverse received wisdom and view mutates independently and is added to the population. mutation as the driving force behind evolutionary search. If A number of genotypes are then eliminated from the recombination has a role to play we view it as secondary (and resultant (larger) population, so as to maintain a fixed obscure!) and thus exclude it from our current analysis. population size. An evolutionary process is elitist if the fittest genotypes in the 2.2 Statistical Dynamics population are never eliminated en masse.

Given a population Ò, following our statistical mechanics

Our analysis of evolutionary dynamics on ﬁtness landscapes analogy, we can deﬁne the corresponding “coarse-grained”

= ´ µ

with neutral networks follows the model developed by van X

¾ Æ

population to be the vector ½ where

Nimwegen et. al. [30], termed “statistical dynamics” by g

i represents the number of genotypes in our

g ¾ i

obvious analogy with classical statistical mechanics. Here

population that are on the neutral network i . Population size

the ﬁtness landscape is coarse-grained by decomposition into

is given by i . We shall also refer to any vector

i=½

neutral networks. For populations of genotypes evolving on X of the above form as a “population” - it should be clear

the landscape a maximum entropy approximation is made re- from the context and notation to which manner of population

´µ

garding the distribution of genotypes within the constituent we refer. Likewise, given an evolutionary process Ò , we

´µ

neutral networks, thereby reducing the state-space of popu- derive a corresponding stochastic process X . It must be

´µ lations to a manageable number of “macroscopic” variables. stressed that the process X will in general not be Marko- As in statistical mechanics, it is difﬁcult to predict how well vian - the transition probabilities from one population to the this model will approximate the actual dynamics of evolving next will generally depend on the distributions of genotypes

populations - it is thus judicious to test all theoretical predic-

within the individual neutral networks i . We shall, however,

´µ tions of such models against computer simulations. approximate the evolutionary process X with a Markov All genotypes are taken to be binary, haploid sequences process; to do so we make a maximum entropy approximation of ﬁxed sequence length . The space of all such sequences, that, roughly speaking, given a genotype from our population

with the graph structure induced by the adjacency of se-

in i , that genotype is treated as if it had been chosen uni-

quences connected by a single point-mutation (i.e. bit ﬂip)

formly and at random from i . We are ultimately, however,

´µ deﬁnes the sequence space , an -dimensional binary hy- modeling the underlying “real” evolutionary processes Ò ,

percube. A ﬁtness landscape is deﬁned to be a mapping from so all Monte Carlo simulations used to test results should

´µ

the sequence space to the set of real numbers (so ﬁtness is model the full stochastic process Ò rather than the coarse-

´µ deterministically associated with genotype; i.e. there is no grained approximation X .

“noise” on fitness evaluation). Given a fitness landscape Ä we define two genotypes to be connected iff there is a sequence 2.3 Mutation Modes of fitness-preserving point-mutations taking one genotype to the other; this is evidently an equivalence relation and thus All evolutionary processes considered in this paper operate induces a partitioning of the sequence space. The neutral via selection (on the basis of fitness) and mutation, in the networks are defined to be the equivalence classes of this par- sense that the Markov transition probabilities depend only on

titioning; i.e. the maximal connected subsets. We label the the ﬁtnesses of sequences and the probabilities that one se-

i i

neutral networks i , where has ﬁtness ; indices quence mutates to another. Mutation is assumed independent

run from ½ to , where is the number of neutral networks. of both genotype and locus; i.e. the probability that a point-

By convention the i ’s are listed in order of ascending ﬁtness. mutation occursat a given locus fora given genotypedoes not

For simplicity, in this paper we assume that neutral network depend on the speciﬁc genotype nor on the locus under con-

¾ Æ

ﬁtnesses are strictly increasing; i.e. ½ . sideration. In all that follows subscripts run from

We now define a (finite) population on a sequence space ¼ to . We shall consider a mutation mode to be defined by

Ë Ò ´ : ¾ Ëµ = ¼ ½ ¾

g « « to be a sequence g where represents a probability distribution , where

is the probability that in the event of mutation of a genotype so that ij represents the probability that a sequence in

exactly (randomly selected) loci undergo point-mutations. mutates to i under the given mutation mode. Thus e.g. for

We deﬁne the per-sequence mutation rate « ; i.e. Poisson mutation we have: «

the expected number of point-mutations per sequence. Exam-

´Á Ñµ

ples of mutation modes include:

= Å ´µ = Å (6)

Poisson mutation: Here each locus ﬂips independently with

¢ where Á is the identity matrix, while for constant mu- the same probability . The expected number of ﬂips per se-

tation:

quence is given by . In the long sequence length

! ½

limit , the mutation distribution approaches a Poisson

= Å ´µ = Ñ

distribution; i.e.: Å (7)

« (1) ! 2.4 Epochal Dynamics

Constant mutation: Here exactly loci undergo point- We will now make more precise what we mean by an mutation; i.e.: “epoch”. Referring to Fig. 2.1, during an epoch (i.e. dur-

ing periods when transients associated with losing the current

= Æ « « (2)

neutral network or moving to a higher network have subsided)

´µ

To analyse our evolutionary processes we will want to know the evolutionary process X is, as a Markov process, “al-

the probability ij that an (arbitrary) sequence in which most” stationary [17]; roughly speaking, the probability of

undergoespoint-mutation at an (arbitrary) locus ends up in i ﬁnding the population in a given state does not vary over time.

(note the order of indices). This reﬂects our coarse-grained In [30] an evolutionary process during such an episode is de-

approach; in reality the probability of mutation to i will dif- scribed as metastable. We shall thus say that the evolutionary

X ´µ ´µ ¼ ´µ = ¼

j i

fer accordingas to which sequence in we choose to mutate. process is in epoch if Ò and for

ij We then adopt the maximum entropy approximation that (i.e. Ò is the highest-ﬁtness neutral network currently

also reﬂects with sufﬁcient accuracy the probability that, if represented in the population) and the process is metastable

a sequence chosen arbitrarily from a population is in j , that as described above. As an approximation we may consider

X ´µ sequence ends up in i after point-mutation at an arbitrary lo- an evolutionary process during epoch as a (station-

cus. At this level of analysis the (single-locus) mutation ma- ary) Markov process in its own right.

Ñ ´ µ trix ij contains all the structural information about

our landscape that we require. Note that Ñ is a stochastic

matrix; i.e. each column sums to ½. We also deﬁne the neu- 2.5 Waiting Time to Portal Discovery

i ii trality of the neutral network i to be ; i.e. the

probability that an arbitrary point-mutation of a sequence in We attempt to provide an approximate answer to the follow-

´µ

ing question: given that an evolving population X is in

i i leaves that sequence in . Since mutation will in general involve more than a sin- the -th epoch, what is the expected waiting time until a por-

gle point-mutation we will also want to know the probabil- tal genotype (i.e. one of higher ﬁtness than currently resides

«µ

´ in the population) is discovered? The ﬁrst issue is how we

ity that an (arbitrary) sequence in j which undergoes

ij should measure time. Given that for real-world GA’s the most

point-mutations at exactly (arbitrary) loci ends up in i .

time-intensive aspect of the process is likely to be fitness eval- We consider the -locus mutation as a sequence of point- uation, it makes sense to measure time to portal discovery in mutations (taken in some arbitrary order). We have, condi- terms of the number of fitness evaluations. It is furthermore tioning on the first point-mutation:

supposed that we may store the ﬁtnesses of all genotypes cur- Æ

rently in the population; i.e. it is only necessary to evalu-

´« ½µ ´«µ

k j (3) ij

ik ate ﬁtness when a new genotype appears in the population. =½

k Since the only generator of new genotypes is mutation, we

´«µ

= ½ ¾ Ñ

for . In matrix notation, setting to be thus associate ﬁtness evaluation with the occurrence of mu-

´«µ

«µ ´« ½µ

´ tation. Note that time thus deﬁned may not equate to a time

Ñ = Ñ ¡ Ñ the matrix , we have , so that:

ij step of the evolutionary process. When confusion might arise

«µ «

´ we shall make clear which time we are talking about.

= Ñ Ñ (matrixpower) (4)

Now from Section 2.4 we deduce that, given the assumed

= ¼ ½ ¾ ´ µ for . Now given a mutation mode « we metastability of the process during epoch , the probability

introduce the (general) mutation matrix:

·½

of discovery of a portal to Ò will be approximately the Ä

same at each time step of the process; we write this (per-time

Å = Ñ

(5) step) portal discovery probability as Ò . The distribution of

«=¼

the waiting time Ò (i.e. the first passage time) to discovery of a portal during epoch is thus approximately geometric fitness. Thus there is a small but non-zero probability that a and the mean first passage time measured in time steps of the point-mutation from any neutral network leads to a (probably evolutionary process is given by: small) increase in fitness, while the probability of a larger fit-

ness increase is of a smaller order of magnitude. Note that

´ µ = E -correlation is a rather stringent condition - it is certainly not

Ò (8) Ò to be claimed as a property that might generally be assumed

of fitness landscapes arising in the context of artificial (or in- Now in all processes considered in this paper, number of fitness evaluations per time-step of the evolutionary process deed natural) evolution. We remark, however, that if there is is constant. The expected waiting time in fitness evaluations no correlation then no search technique is likely to be more is thus given by: efficient than random search; this is a form of “no free lunch theorem” [32]. We are thus obliged to assume some correla-

tion. Note also that correlation and neutrality are (in a sense

E ´ µ =

Ò (9)

Ò that may be made quite precise) statistically independent [2, 3, 25]; that is, we should not assume that neutrality im- We have, however, disregarded an important issue: in Sec- plies, nor precludes, correlation. Furthermore, it is reasonable tion 2.1 we noted that, if not elitist, an evolutionary process

to suppose that the higherup (in ﬁtness) ourlandscape we are,

Ò·½ in epoch may well “lose” Ò before a portal to is dis-

the more rare are those point-mutations taking us higher still; covered. How then can Ò be well-defined? We side-step the and that point-mutationsleading to large fitness increases will issue in this paper; the evolutionary process which will most be rarer than those (already rare) point-mutations leading to concern us is, in fact, elitist and we may at worst have to sup- small fitness increases. We should be aware, however, that pose that if our algorithm is not elitist, then the mutation rate it is quite possible that even if there is a reasonable degree is low enough that the probability of losing the current fittest of correlation there may still exist sub-optimal neutral net- network before finding a portal is small. We refer the reader works for which no fitness-increasing point-mutations exist; to [30] for more detailed analysis of this topic. such networks may be thought of as the neutral analogues

of isolated sub-optimal ﬁtness peaks in the standard theory ¯

3 -Correlated Landscapes of rugged landscapes [31, 18]. Although -correlation effectively rules out the existence of sub-optimal neutral networks We now make some structural assumptions about our fitness there is some evidence to suggest that such networks may landscape Ä. Specifically, we assume that the probability of not necessarily be common; studies of RNA folding land- a point-mutation taking a sequence to a higher fitness neu- scapes in particular have demonstrated various percolation- tral network is very small compared to the probability of it like properties of neutral networks [8, 10, 13, 24] which sug- being neutral or of reducing fitness. We shall actually go fur- gest that almost every network approaches to within a few ther than this and assume that the only non-negligible fitness- point-mutations of almost every other network. increasing point-mutations are those to the next-highest net- For the remainder of this section we assume that our land- work. More precisely:

scape satisﬁes the -correlation property (10) and we neglect

Ó ´µ ´ µ

terms of . Now, given a mutation mode « as in Sec-

Deﬁnition: We say that a ﬁtness landscapes is -correlated

tion 2.3 let us deﬁne Ò to be the probability that an (arbi-

¼ ½

iff there exists an with such that the point-

Ò·½ trary) individual sequence in Ò discovers a portal to

mutation matrix takes the form:

= Å

Ò·½;Ò

under mutation; i.e. Ò where is given by (5).

Ó ´½µ

From (10), we can calculate that to in :

« «

½ ¾

Ò Ò·½

¾ ¿

Ò Ò·½

Ñ =

(10)

Ò (11)

Ò·½;Ò

. .

« ½

. .

Ó ´µ

Ò·½

. . Ò

½ Æ Æ so we ﬁnd e.g. that for Poisson mutation in the long sequence

length limit:

¼ = ½ ¾ ½ ¤ i

for some i with for .

´½ µ

Ò·½

e e

Ò Ò·½

Here £ represents the transition probabilities from higher to

´µ =

Ò Ò

´½ µ

i =

lower ﬁtness networks, while is the probability of back-

Ò Ò·½

i·½ i

mutation from i to under point-mutation. The and

£ terms are not taken to be . With -correlation we work

´½µ Ó ´µ throughout to Ó in ; i.e. we neglect all terms. as a function of per-sequence mutation rate , while for con- This property is related to the degree of (genotype-ﬁtness) stant mutation:

correlation present in our landscape; i.e. the degree to which

´µ =

Ò (12)

·½;Ò sequences nearby in sequence space are likely to be of similar Ò 3.1 Optimal Search “proof” supplied is by no means rigorous. Firstly, we deﬁne the netcrawler evolutionary process: We are now in a position to state the principal results of this paper:

Deﬁnition: The netcrawler process operates as follows: ½ population size = . At each time step the current (single)

Proposition 1: On an -correlated ﬁtness landscape, of all genotype replicates and the copy mutates according to the possible mutation modes, that yielding the maximum value

mutation mode. If the mutant offspring is less ﬁt than its for Ò is given by constant mutation with per-sequence mu-

parent it is eliminated; otherwise the parent is eliminated. ¤

tation rate: = (nearest integer to) We note that this algorithm is almost identical to the Random

Mutation Hill Climber (RMHC) presented in [9], the only

ÐÒ ´ ÐÒ µ ÐÒ ´ ÐÒ µ

Ò Ò·½

(13)

Ò Ò·½

ÐÒ

ÐÒ difference being that the RMHC only ever ﬂips one (random)

Ò Ò·½

bit at each step. We avoid the term “hill-climber” to empha-

Ò Ò·½

ÐÒ Ò sise that, in the presence of neutral networks, the netcrawler

spends most of its time not climbing hills, but rather perform-

´ µ

Proof: For a general mutation mode « , we have

ing neutral walks [13]. It is elitist, the number of ﬁtness eval-

= «

Ò . We need to ﬁnd the maximum

Ò·½;Ò

«=¼

½ = Ò

uations per time step is and we always have Ò so

¼ ½ Ä

value for Ò as a function of under the con-

that the expected waiting time on Ò is just .

= ½ ¼ ½ 8

« Ò

straints « and . Now being

«=¼

linear in the « ’s describes a hyper-plane in -space; we Proposition 2: On an -correlated ﬁtness landscape, given

have to find the maximum “height” of this hyper-plane over a mutation mode, the most efficient (fixed-population) evolu- the simplex described by the constraints on the « ’s. It is tionary process is the netcrawler.

clear that (barring any “degeneracies” among the coefﬁcients «

) the maximum must lie above a “corner” of the

Ò·½;Ò

Informal Proof: Suppose a population of ﬁxed size evolv-

simplex; i.e. a point where all the « ’s are zero except for ing according to some evolutionary process under a particu-

= = ½ one, say, for which ; i.e. the mutation mode is

lar mutation mode is in epoch . From (10) it follows that

´µ

constant with per-sequence mutation rate . From (12) Ò

Ó ´µ

·½;i

for Ò will be and hence negligible. There

is given by (11) with ; the result follows by differenti- is thus no point in maintaining genotypes in the population

´µ

ating Ò with respect to , setting the resulting expression

on i for . Recalling our deﬁnition of an evolutionary

to zero and solving for . Note that the degenerate cases arise process (Section 2.2) our process should thus, at the elimi- «

where two (or more) of the coefﬁcients coincide;

·½;Ò Ò nation stage, eliminate all genotypes which have mutated to

in these cases the of (13) still yields a (now non-unique)

i for . At the start of each time step, then, the entire

maximum for Ò .

population will be on Ò .

Now let Ò be as before the probability that (under the op-

Fig. 3.1 plots the optimal mutation rate of Proposition 1 as

erant mutation mode) a single genotype mutates from Ò to

Ò·½

a function of Ò .

·½

Ò . The probability that (during the mutation phase) some

½ ´½ µ

·½ Ò Ò mutant ﬁnds a portal to Ò is thus

where is the number of replicants; equality occurs when the probabilities of the individual mutants ﬁnding a portal are µ independent. Selection for replication should thus be without replacement; for if a genotype is selected more than once 8 its mutant offspring will be genetically correlated and their 7 probabilities of ﬁnding a portal thus not independent. 6

Our evolutionary process during epoch now looks 5 like the following: at each time step a certain number of 4

3 the genotypes in our population is selected for replica- 1 2 tion/mutation. Suppose ﬁrstly that no portal is discovered.

0.8

1 0.6 The mutants that fall off Ò are eliminated. The remaining 0 0.2 0.4 ν 0.4 n+1 population consists of the original population plus those mu- ν 0.6 0.2 n 0.8 0 1 tants that have stayed on Ò . Nowifouraimis tokeepgenetic correlations to a minimum, then we don’t wish to leave (correlated) parent-offspring pairs in the population. On the other

hand, we don’t wish to eliminate a stay-on mutant offspring

Ò·½

Figure 3.1: The of Proposition 1 plotted against Ò while retaining its un-mutated parent, since we would thus

end up “anchoring” our search to the limited region of Ò The next proposition is of a more conjectural nature - the neighbouring our current population; in effect we would like 4 Conclusions

to maximise drift of our population on . We should thus re- tain stay-on mutants and eliminate their un-mutated parents. Fitness landscapes for many real-world artiﬁcial evolution

Effectively, our evolutionary process now comprises problems are likely to feature large-scale neutrality. We independent netcrawlers, some selection of which are “ac- have argued that on such landscapes the conventional view tivated” at each time step. But if during some time-step a of evolutionary search associated with rugged non-neutral

landscapes is misleading and inappropriate. Instead we have ·½

mutant discovers a portal to Ò then we must initiate a new

epoch. Since the remaining genotypes now have a neg- characterised evolutionary dynamics with neutral networks in

terms of drift along networks punctuated by transitions be- ·¾ ligible chance of ﬁnding a portal to Ò all members of the population must become copies of the newly discovered tween networks.We have also argued that the role of recom- portal genotype. But this again introduces genetic correla- binationis obscureand may amount to little more than macro-

tions between members of the population. To minimise this mutation. Given this scenario, the perceived problem of pre-

= ½ effect we need only choose our population size to be ; mature convergence disappears and search for high-fitness indeed there is no loss in search efficiency by reducing the genotypes is dominated by waiting for drift and mutation to population size, since the netcrawlers are independent. Thus find portals to higher neutral networks. the most efficient evolutionary process for the given mutation We introduced the statistical property of -correlation to

mode comprises a single netcrawler. ¤ describe landscapes with neutral networks for which higher networks are “accessible” only from the current network. For To test our propositions extensive simulations (not pre- such landscapes we have shown (Proposition 1) that there sented here) were carried out on Royal Road landscapes is an optimal mutation mode and mutation rate and conjec-

[9, 30], which are indeed -correlated. As predicted by tured (Proposition 2) that there is also an optimal evolution- Proposition 2, the netcrawler consistently outperformed var- ary search strategy which is not population-based but rather ious population-based GA’s without recombination (includ- a form of hill-climber which we have dubbed the netcrawler. ing ﬁtness-proportional roulette-wheel selection and several We have also proposed an adaptive variant of the netcrawler rank-based tournament algorithms) for Poisson and constant which gathers statistical information about the landscape as it mutation modes. Waiting times to portal discovery were pre- proceeds and uses this information to self-optimise. It should dicted with reasonable accuracy and in particular the opti- be remarked in this context that a major motivation for the re- mum (constant) mutation rate was correctly predicted by (13) search presented in this paper was a series of experiments by of Proposition 1. Thompson and Layzell in on-chip electronic circuit design by evolutionary methods [28], during which an algorithm almost identical to our netcrawler was used with some success. The 3.2 The Adaptive Netcrawler mutation rate deployed in these experiments, albeit chosen Propositions 1 and 2 of Section 3.1 suggest the following on heuristic grounds, in fact turns out to be the optimum pre- adaptive netcrawler for landscapeswhich we know, or at least dicted in this paper given the (estimated) neutrality inherent

suspect, of being the -correlated: we deploy a netcrawler in the problem. with the constant mutation mode. We would then like to Finally, we remark that work in progress by the author be able to ﬁnd the optimal per-sequence mutation rate (13) suggests that the principal results presented in this paper may

for the current ﬁttest neutral network , say. This involves still obtain under considerably less stringent assumptionsthan

knowledge of the neutrality of and also the neutrality , -correlation. ¼

say, of the next highest network , say. Now we can estimate “on the ﬂy”: suppose that is our current per-sequence

mutation rate. Then during the course of the progress of our Acknowledgements netcrawler on , we simply log the fraction of mutations,

The author would like to thank Inman Harvey, Adrian

say, which leave us on . The neutrality of is then ap- = ½ Thompson and Nick Jakobi for helpful s.

proximately . However, we have no way of knowing

¼ ¼ ¼

the neutrality of . A reasonable guess might be that

is close to . Indeed, examination of Ò shows that if

Bibliography

is not too different from , the value for Ò obtained from

(12) by setting is not too different to the actual op- [1] Back, T., Hoffmeister, F., and Schwefel, H.-P. (1991). A sur-

timum Ò . According to (13) then, we should re-adjust our vey of evolution strategies. In Belew, L. B. and Booker, R. K.,

½ ÐÒ ÐÒ mutation rate to . Curiously this im- editors, Proc. 4th Int. Conf. on Genetic Algorithms, pages 2–9.

plies that, whatever the neutrality of , we should ultimately Morgan Kaufmann, San Diego, CA.

= ½ ¼¿6788 see a fraction of neutral mutations if our netcrawler is optimised by the above scheme. This result [2] Barnett, L. (1997). Tangled webs: Evolutionary dy- echoes Rechenberg’s “1/5 success rule” for Evolution Strate- namics on fitness landscapes with neutrality. Mas- gies [1], to which our netcrawler bears some resemblance. ter’s thesis, COGS, University of Sussex. Avail- able for download from ftp://ftp.cogs.susx.ac.uk/ [16] Jakobi, N. and Quinn, M. (1998). Some problems and a few so- pub/users/inmanh/lionelb/FullDiss.ps.gz. lutions for open-ended evolutionary robotics. In Husbands, P. and Meyer, J.-A., editors, Proceedings of Evorob98. Springer Verlag. [3] Barnett, L. (1998). Ruggedness and neutrality - the NKp family of fitness landscapes. In Adami, C., Belew, R. K., Kitano, H., [17] Karlin, S. and Taylor, H. M. (1975). A first course in Stochastic and Taylor, C., editors, ALIFE VI, Proceedings of the Sixth In- Processes (2nd ed.). Academic Press (NY). ternational Conference on Artificial Life, pages 18–27. The MIT [18] Kauffman, S. A. (1993). The Origins of Order - Orginization Press. and Selection in Evolution. Oxford University Press, New York.

[4] Cliff, D., Husbands, P., and Harvey, I. (1993). Evolving visu- [19] Kimura, M. (1983). The Neutral Theory of Molecular Evolu- ally guided robots. In J.-A. Meyer, H. Roitblat, S. W., editor, tion. Cambridge University Press. From Animals to Animats 2: Proc. of the Second Intl. Conf. on Simulation of Adaptive Behavior, (SAB92), pages 374–383. MIT [20] Kondrashov, A. S. (1982). Selection against harmful mutations Press/Bradford Books, Cambridge MA. in large sexual and asexual populations. Genet. Res., 40:325–332.

[5] Crow, J. F. and Kimura, M. (1970). An Introduction to Popula- [21] Layzell, P. (1998). The evolvable motherboard. Cognitive Sci- tion Genetics Theory. Harper and Row, New York. ence Research Paper 479, University of Sussex, School of Cog- nitive and Computing Sciences. [6] Derrida, B. and Peliti, L. (1991). Evolution in a flat fitness landscape. Bull. Math. Biol., 53(3):355–382. [22] Maynard Smith, J. (1978). The Evolution of Sex. Cambridge University Press, Cambridge, UK. [7] Eigen, M., McCaskill, J., and Schuster, P. (1989). The molecu- [23] Nowak, M. and Schuster, P. (1989). Error thresholds of repli- lar quasispecies. Adv. Chem. Phys., 75:149–263. cation in finite populations: Mutation frequencies and the onset [8] Fontana, W., Stadler, P. F., Bornberg-Bauer, E. G., Griesmacher, of Muller’s ratchet. J. Theor. Biol., 137:375–395. T., Hofacker, I. L., Tacker, M., Tarazona, P., Weinberger, E. D., [24] Reidys, C., Stadler, P. F., and Schuster, P. (1997). Generic and Schuster, P. (1993). RNA folding and combinatory land- properties of combinatory maps: Neutral networks of RNA sec- scapes. Phys. Rev. E, 47(3):2083–2099. ondary structures. Bull. Math. Biol., 59(2):339–397.

[9] Forrest, S. and Mitchell, M. (1993). Relative building block fit- [25] Reidys, C. M. and Stadler, P. F. (1998). Neutrality in fitness ness and the building block hypothesis. In Whitely, D., editor, landscapes. working paper 98-10-089, Santa Fe Institute. Foundations of Genetic Algorithms 2. Morgan Kaufmann, San Mateo, CA. [26] Smith, T. M. C., Philippides, A., and O’Shea, M. (2001). Not measuring evolvability: Initial exploration of an evolutionary [10] Forst, C. V., Reidys, C., and Weber, J. (1995). Evolutionary dy- robotics search space. In Proceedings of the 2001 Congress on namics and optimization: Neutral networks as model-landscapes Evolutionary Computation: CEC2001 (submitted). IEEE, Korea. for RNA secondary-structure folding-landscapes. Lecture notes in Artificial Intelligence, 929: Advances in Artificial Life. [27] Thompson, A. (1996). Silicon evolution. In Koza, J. R., Gold- berg, D. E., Fogel, D. B., and Riolo, R. L., editors, Genetic Pro- [11] Harvey, I. (1994). Evolutionary robotics and saga: the case gramming 1996: Proc. 1st Annual Conf. (GP96), pages 444–452. for hill crawling and tournament selection. In Langton, C., edi- Cambridge, MA: MIT Press. tor, Artificial Life III, Santa Fe Institute Studies in the Sciences of Complexity, Proc., volume XVI, pages 299–326. Addison Wes- [28] Thompson, A. and Layzell, P. (2000). Evolution of robustness ley. in an electronics design. In Miller, J., Thompson, A., Thomson, P., and Fogarty, T., editors, Proc. 3rd Int. Conf. on Evolvable Sys- [12] Harvey, I. and Thompson, A. (1996). Through the labyrinth tems (ICES2000): From biology to hardware, volume 1801 of evolution finds a way: A silicon ridge. In Proc. 1st Internatl. LNCS, pages 218–228. Springer-Verlag. Conf. Evol. Sys.: From Biology to Hardware (ICES 96). Springer [29] Thompson, A., Layzell, P., and Zebulum, R. S. (1999). Ex- Verlag. plorations in design space: Unconventional electronics design [13] Huynen, M. A. (1996). Exploring phenotype space through through artificial evolution. IEEE Trans. Evol. Comp., 3(3):167– neutral evolution. J. Mol. Evol., 63:63–78. 196. [30] van Nimwegen, E., Crutchfield, J. P., and Mitchell, M. (1998). [14] Huynen, M. A., Stadler, P. F., and Fontana, W. (1996). Smooth- Statistical dynamics of the royal road genetic algorithm. working ness within ruggedness: The role of neutrality in adaptation. paper 97-04-35, Santa Fe Institute. Proc. Natl. Acad. Sci. (USA), 93:397–401. [31] Weinberger, E. D. (1990). Correlated and uncorrelated fitness [15] Jakobi, N., Husbands, P., and Harvey, I. (1995). Noise and landscapes and how to tell the difference. Biol. Cybern., 63:325– the reality gap: The use of simulation in evolutionary robotics. 336. In Moran, F., Moreno, A., Merelo, J. J., and Chacon, P., editors, Proc. 3rd European Conference on Artificial Life, Springer- [32] Wolpert, D. H. and Macready, W. G. (1995). No free lunch Verlag, Lecture Notes in Artificial Intelligence 929, volume XVI, theorems for search. technical report SFI-TR-95-02-010, Santa pages 704–720. Springer Verlag. Fe Institute.