Dynamics Computation and the Edge of Chaos

A ReExamination



Melanie Mitchell James PCrutcheld and Peter T Hrab er

Santa Fe Institute Working Pap er

To app ear in G Cowan D Pines and D Melzner editors Integrative Themes Santa

Fe Institute Stuides in the Sciences of Complexity Pro ceedings Volume Reading MA

AddisonWesley

Abstract

In this pap er we review previous work and present new work concerning the relationship b e

tween dynamical systems theory and computation In particular we review work by Langton

and Packard on the relationship b etween dynamical b ehavior and computational

capability in cellular automata CAs We present results from an exp eriment similar to

the one describ ed byPackard whichwas cited as evidence for the hyp othesis that rules

capable of p erforming complex computations are most likely to b e found at a phase tran

sition b etween ordered and chaotic b ehavioral regimes for CAs the edge of chaos Our

exp eriment pro duced very dierent results from the original exp eriment and we suggest that

the interpretation of the original results is not correct Weconcludeby discussing general

issues related to dynamics computation and the edge of chaos in cellular automata

Intro duction

A central goal of the sciences of complex systems is to understand the laws and mechanisms

by which complicated coherent global b ehavior can emerge from the collective activities of

relatively simple lo cally interacting comp onents Given the diversity of systems falling into

this broad class the discovery of any commonalities or universal laws underlying such sys

tems will require very general theoretical frameworks Twosuch frameworks are dynamical

systems theory and the theory of computation These have indep endently provided p ower

ful to ols for understanding and describing common prop erties of a wide range of complex

systems

Dynamical systems theory has develop ed as one of the main alternatives to analytic

closedform exact solutions of complex systems Typically a system is considered to b e

solved when one can write down a nite set of nite expressions that can b e used to

predict the state of the system at time tgiven the state of the system at some initial

time t Using existing mathematical metho ds such solutions are generally not p ossible for



most complex systems of interest The central contribution of dynamical systems theory to

Santa Fe Institute Old Pecos Trail Suite A Santa Fe New Mexico USA

Email mmsantafeedu pthsantafeedu



Physics Department University of California Berkeley CA USA

Email chaosgo jirab erkeleyedu

mo dern science is that exact solutions are not necessary for understanding and analyzing a

nonlinear pro cess Instead of deriving exact single solutions or opting for coarse statistical

descriptions the emphasis of dynamical systems theory is on describing the geometrical and

top ological structure of ensembles of solutions In other words dynamical systems theory

gives a geometric view of a pro cesss structural elements such as attractors basins and

separatrices It is thus distinguished from a purely probabilistic approach such as statistical

mechanics in which geometric structures are not considered Dynamical systems theory

also addresses the question of what structures are generic that is what b ehavior typ es are

typical across the sp ectrum of complex systems

In contrast to fo cusing on how geometric structures are constrained in a state space

computation theory fo cuses on how basic informationpro cessing elementsstorage logical

gates stacks queues pro duction rules and the likecan b e combined to eect a given

informationpro cessing task As such computation theory is a theory of organization and

the functionality supp orted by organization When adapted to analyze complex systems

it provides a framework for describing b ehaviors as computations of varying structure For

example if the global mapping from initial to nal states is considered as a computation then

the question is what function is b eing computed by the global dynamics Another range of

examples concern limitations imp osed by the equations of motion on information pro cessing

can a given complex system b e designed to emulate a universal Turing machine In contrast

to this sort of engineering question one is also interested in the intrinsic computational

capability of a given complex system that is what informationpro cessing structures are

intrinsic in its b ehavior

Dynamical systems theory and computation theory have historically b een applied in

dep endently but there have b een some eorts to understand the relationship b etween the

twothat is the relationship b etween a systems ability for information pro cessing and other

measures of the systems dynamical b ehavior

Relationships Between Dynamical Systems Theory and Computation Theory

Computation theory develop ed from the attempt to understand informationpro cessing as

p ects of systems A collo quial denition of information pro cessing might b e the trans

formation of a given input to a desired output However in order to apply the notion of

information pro cessing to complex systems and to relate it to dynamical systems theory

the notion must b e enriched to include the production of information as well as its storage

transmission and logical manipulation In addition the engineeringbased notion of desired

output is not necessarily appropriate in this context the fo cus here is often on the intrin

sic informationpro cessing capabilities of a dynamical system not sub ject to a particular

computational goal

Beginning with Kolmogorovs and Sinais adaptation of Shannons communication theory

to mechanics in the late s there has b een a continuing eort to relate a nonlinear

systems informationpro cessing capability and its temp oral b ehavior One result is that a

deterministic chaotic system can b e viewed as a generator of information Another is

that the complexity of predicting a chaotic systems b ehavior grows exp onentially with time

The complexity metric here called the KolmogorovChaitin complexity uses a

universal Turing machine as the deterministic prediction machine The relationship b etween

the diculty of prediction and dynamical randomness is simply summarized by the statement

that the growth rate of the descriptive complexity is equal to the information pro duction

rate These results give a view of deterministic chaos that emphasizes the pro duction of

randomness and the resulting unpredictability They are probably the earliest connections

between dynamics and computation

The question of what structures underlie information pro duction in dynamical systems

has received attention only more recently The rst and crudest prop erty considered is the

amount of memory a system employs in pro ducing apparent randomness The idea is

that an ideal random pro cess uses no memory to pro duce its informationit simply ips a

coin as needed Similarly a simple p erio dic pro cess requires memory only in prop ortion to

the length of the pattern it rep eats Within the memorycapacity view of dynamics b oth

these typ es of pro cesses are simplemore precisely they are simple to describ e statistically

Between these extremes though lie the highly structured complex pro cesses that use b oth

randomness and pattern storage to pro duce their b ehavior Such pro cesses are more complex

to describ e statistically than are ideal random or simple p erio dic pro cesses The tradeo

between structure and randomness is common to much of science The notion of statistical

complexitywas intro duced to measure this tradeo

Computation theory is concerned with more than information and its pro duction and

storage These elements are taken as given and instead the fo cus is on how their combina

tions yield more or less computational p ower Understandably there is a central dichotomy

between machines with nite and innite memory On a ner scale distinctions can b e

drawn among the ways in which innite memory is organizedeg as a stack a queue or a

parallel array Given such considerations the question of the intrinsic computational struc

ture in a dynamical system b ecomes substantially more demanding than the initial emphasis

on gross measures of information storage and pro duction

Several connections in this vein have b een made recently In the realm of continuousstate

dynamical systems Crutcheld and Young lo oked at the relationship b etween the dynamics

and computational structure of discrete time series generated by the logistic map at dierent

parameter settings They found that at the onset of chaos there is an abrupt jump

in computational class of the time series as measured by the formal language class required

to describ e the time series In concert with Feigenbaums renormalization group analysis of

the onset of chaos this result demonstrated that a dynamical systems computational

capabilityin terms of the richness of b ehavior it pro ducesis qualitatively increased at a

phase transition

Rather than considering intrinsic computational structure a numb er of engineering

suggestions have b een made that there exist physically plausible dynamical systems imple

menting Turing machines These studies provided explicit constructions for several

typ es of dynamical systems At this p oint it is unclear whether the resulting computational

systems are genericie likely to b e constructible in other dynamical systemsor whether

they are robust and reliable in information pro cessing In any case it is clear that much

work has b een done to address a range of issues that relate continuousstate dynamics and

computation Many of the basic issues are now clear and there is a rm foundation for future

work

Dynamics and Computation in Cellular Automata

There has also b een a go o d deal of study of dynamics and computation in discrete spatial

systems called cellular automata CAs In manyways CAs are more natural candidates

for this study than continuousstate dynamical systems since they are completely discrete in

space in time and in lo cal state There is no need to develop a theory of computation with

real numb ers Unfortunately something is lost in going to a completely discrete system The

analysis of CA b ehavior in conventional dynamical systems terms is problematic for just this

reason Dening the analogs of sensitive dep endence on initial conditions the pro duc

tion of information chaos instability attractor smo oth variation of a parameter

bifurcation the onset of chaos and other basic elements of dynamical systems theory

requires a go o d deal of care Nonetheless Wolfram intro duced a dynamical classication

of CA b ehavior closely allied to that of dynamical systems theory He sp eculated that one

of his four classes supp orts universal computation It is only recentlyhowever that

CA b ehavior has b een directly related to the basic elements of qualitative dynamicsthe

attractorbasin p ortrait This has led to a reevaluation of CA b ehavior classication and

in particular to a redenition of the chaos and complexity apparent in the spatial patterns

that CAs generate

SubsequenttoWolframs work Langton studied the relationship b etween the average

dynamical b ehavior of cellular automata and a particular statistic of a CA rule table

He then hyp othesized that computationally capable CAs and in particular CAs capable

of universal computation will have critical values corresp onding to a phase transition

between ordered and chaotic b ehavior Packard exp erimentally tested this hyp othesis by

using a genetic algorithm GA to evolve CAs to p erform a particular complex computation

He interpreted the results as showing that the GA tends to select CAs close to critical

regionsie the edge of chaos

Wenow turn our discussion more sp ecically to issues related to dynamicalb ehavior

classes and computation in CAs We then present exp erimental results and a theoretical dis

cussion that suggest the interpretation given of the results byPackard is not correct Our

exp eriments however showsomeinteresting phenomena with resp ect to the GA evolution

of CAs whichwe summarize here Longer more detailed descriptions of our exp eriments

and results are given in

Cellular Automata and the Edge of Chaos

Cellular automata are one of the simplest frameworks in which issues related to complex

systems dynamics and computation can b e studied CAs have b een used extensively as

mo dels of physical pro cesses and as computational devices In its simplest

form a CA consists of a spatial lattice of cel ls each of which at time t can b e in one of k

states We denote the lattice size ie numb er of cells as N A CA has a single xed rule

used to up date each cell this rule maps from the states in the neighb orho o d of a celleg

the states of a cell and its nearest neighb orsto a single state which b ecomes the up dated

value for the cell in question The lattice starts out with some initial conguration of cell

states and at each time step the states of all cells in the lattice are synchronously up dated

We use the term state to refer to the value of a single cell eg or and the term

conguration to mean the pattern of states over the entire lattice

In this chapter we restrict our discussion to onedimensional CAs with k In a one

dimensional CA the neighb orho o d of a cell includes the cell itself and some number r of

neighb ors on either side of the cell All of the simulations describ ed here are of CAs with

spatially p erio dic b oundary conditions ie the onedimensional lattice is viewed as a circle

with the right neighb or of the rightmost cell b eing the leftmost cell and vice versa

The equations of motion for a CA are often expressed in the form of a rule tableThis

is a lo okup table listing each of the neighb orho o d patterns and the state to which the central

cell in that neighb orho o d is mapp ed For example the following is one p ossible rule table

for a onedimensional CA with k r Each p ossible neighborhood is given along

with the output bit s to which the central cell is up dated

s

In words this rule says that for each neighb orho o d of three adjacent cells the new state is

decided by a ma jorityvote among the three cells

The notion of computation in CAs can have several p ossible meanings but the

most common meaning is that the CA p erforms some useful computational task Here the

rule is interpreted as the program the initial conguration is interpreted as the input

and the CA runs for some sp ecied numberoftimestepsoruntil it reaches some goal

patternp ossibly a xedp oint pattern The nal pattern is interpreted as the output

An example of this is using CAs to p erform imagepro cessing tasks

Packard discussed a particular k r ruleinvented by Gacs Kurdyumov and

Levin GKL as part of their studies of reliable computation in CAs The GKL rule was

not invented for any particular classication purp ose but it do es have the prop erty that

under the rule most initial congurations with less than half s are eventually transformed

to a conguration of all s and most initial congurations with more than half s are

transformed to a conguration of all s The rule thus approximately computes whether

the density of s in the initial conguration whichwe denote as isabove the threshold

When initial congurations are close to the rule makes a signicant

c

numb er of classication errors

Packard was inspired by the GKL rule to use a GA to evolve a rule table to p erform this

task If then the CA should relax to a conguration of all s otherwise

c

it should relax to a conguration of all s This task can b e considered to b e a complex

computation for a k r CA since the minimal amount of memory it requires increases

with N in other words the required computation is spatially global and corresp onds to the



recognition of a nonregular language The global nature of the computation means that

information must b e transmitted over signicant spacetime distances on the order of N

and this requires the co op eration of many lo cal neighb orho o d op erations

In dynamical terms complex computation in a smallradius binarystate CA requires

signicantly long transients and spacetime correlation lengths Langton hyp othesized that

such eects are most likely to b e seen in a certain region of CA rule space as parameterized



See Hop croft and Ullman for an intro duction to formallanguage classes in computation theory

by For binarystate CAs is simply the fraction of s in the output bits of the

rule table For CAs with k is dened as the fraction of nonquiescent states in

the rule table where one state is arbitrarily chosen to b e quiescent and all states ob ey a

strong quiescence requirement Langton p erformed a number of Monte Carlo samples

of twodimensional CAs starting with and gradually increasing to
k ie the

most homogeneous to the most heterogeneous rule tables Langton used various statistics

such as singlesite entropytwosite mutual information and transient length to classify

CA average b ehavior at each value The notion of average b ehavior was intended

to capture the most likely b ehavior observed with a randomly chosen initial conguration

for CAs randomly selected in a xed subspace These studies revealed some correlation

between the various statistics and The correlation is quite go o d for very low and very

high values However for intermediate values in nitestate CAs there is a large degree

of variation in b ehavior

Langton claimed on the basis of these statistics that as is incremented from to

k the average b ehavior of CAs undergo es a phase transition from ordered xed

p oint or limit cycle after some short transient p erio d to chaotic apparently unpredictable

after some short transient p erio d As reaches a critical value the claim is that rules

c

tend to have longer and longer transient phases Additionally Langton claimed that CAs

close to tend to exhibit longlived complexnonp erio dic but nonrandompatterns

c

Langton prop osed that the regime roughly corresp onds to Wolframs Class CAs

c

and hyp othesized that CAs capable of p erforming complex computations will most likely b e

found in this regime

Analysis based on is one p ossible rst step in understanding the structure of CA

rule space and the relationship b etween dynamics and computation in CAs However the

claims summarized ab ove rest on a numb er of problematic assumptions One assumption

is that in the global view of CA space CA rule tables themselves are the appropriate lo ci

of dynamical b ehavior This is in stark contrast with the state space and the attractor

basin p ortrait approach of dynamical systems theory The latter approachacknowledges the

fact that b ehaviors in state space cannot b e adequately parameterized byany function of

the equations of motion suchas Another assumption is that the underlying statistics

b eing averaged eg singlesite entropy converge But many pro cesses are known for which

averages do not converge Perhaps most problematic is the assumption that the selected

statistics are uniquely asso ciated with mechanisms that supp ort useful computation

Packard empirically determined rough values of for onedimensional k r CAs

c

by lo oking at the dierencepattern spreading rate as a function of The spreading

rate is a measure of unpredictability in spatiotemp oral patterns and so is one p ossible

measure of chaotic b ehavior It is analogous to but not the same as the Lyapunov

exp onentforcontinuousstate dynamical systems In the case of CAs it indicates the average

propagation sp eed of information through spacetime though not the pro duction rate of

lo cal information At each a large numb er of CAs was sampled and for eachCA was

estimated The average over the selected CAs was taken as the average spreading rate at

the given The results are repro duced in Figure a As can b e seen at lowandhigh s

vanishes indicating xedp oint or shortp erio d b ehavior at intermediate it is maximal

indicating chaotic b ehavior and in the transition or regionscentered ab out 

c

and  it rises or falls gradually While not shown in Figurea for most values

s variance like that of the statistics used by Langton is high

The Original Exp eriment

Langtons empirical CA studies recounted ab ove addressed only the relationship b etween

and the dynamical b ehavior of CAs as revealed byseveral statistics Those studies did not

correlate or b ehavior with an indep endent measure of computation Packard addressed

this issue by using a genetic algorithm GA to evolve CA rules to p erform a particular

computation This exp erimentwas meant to test twohyp otheses rules able to p erform

complex computations are most likely to b e found near values and when rules are

c

evolved to p erform a complex computation evolution will tend to select rules near values

c

Packards exp eriment consisted of evolving binarystate onedimensional CAs with r

The complex computation is the task describ ed ab ove A genetic algorithm was

c

applied to a p opulation of rules represented as bit strings To calculate the tness of a string

the string was interpreted as the output bits of a rule table and the resulting CA was run

on a numb er of randomly chosen initial conditions The tness was a measure of the average

classication p erformance of the CA over the initial conditions

The results from this exp eriment are displayed in Figure b The histogram displays

the observed frequency of rules in the GA p opulation as a function of with rules merged

from a numb er of dierent runs with identical parameters but with dierentrandomnumber

seeds In the initial generation the rules were uniformly distributed over values The graph

b gives the nal generationin this case after the GA has run for generations The

rules cluster close to the two regions as can b e seen by comparison with the dierence

c

pattern spreading rate plot a Note that each individual run pro duced rules at one or the

other p eak in graph b so when the runs were merged together b oth p eaks app ear

Packard interpreted these results as evidence for the hyp othesis that when an ability for

complex computation is required evolution tends to select rules near the transition to chaos

He argues like Langton that this result intuitively makes sense b ecause rules near the

transition to chaos have the capability to selectively communicate information with complex

structures in spacetime thus enabling computation

Our Exp eriment

We p erformed an exp eriment similar to Packards The rules in the p opulation are repre

sented as bit strings each enco ding the output bits of a rule table for k r Thus

r 

the length of each string is

For a single run the GA we used generated a random initial p opulation of rules bit

strings with values uniformly distributed over Then it calculated the tness of each

rule in the p opulation by a metho d to b e describ ed b elow The p opulation was then ranked

by tness and the rules with lowest tness were discarded The rules with highest

tness were copied directly into the next generation To ll out the p opulation new rules

were generated from pairs of parents selected at random from the current generation Each

pair of parents underwent a singlep oint crossover whose lo cation was selected with uniform

probabilityover the string The resulting ospring were mutated at a numb er of sites chosen hosen e v olution of CAs h The rules with with the authors b er of randomly c t onGAev um ed from initial p opulations olv ed from initial p opulations uniformly of a large n olv ards exp erimen k ac to bins of length eac b er of runs These p opulations ev Graphs a and b are adapted from axis is divided in vided there b Results from P um t The histogram plots the frequencies of rules merged from the nal tmost bin The b est cross and mean circle tnesses are plotted for as pro the al for tnesses is also The histogram consists of bins of width The bin ab o terv erage dierencepattern spreading rate wing v ollo ertical scale w axis in F y classication task The histogram plots the frequencies of rules merged from the nal a The a c CAs as a function of are included in the righ r tains just those rules with h bin The eac con p ermission No v c Results fromgenerations our generation exp erimen ofdistributed in runs These p opulations ev generations generation uniformly of distributed a in n for the k Figure

Fraction of Rules 0.25 0.75 γ 0.5 1.0 0.0 Prob(rule) 0.0 .5050.75 0.5 0.25 λ 1.0 mean fitness best fitness (b) (a) (c)

from a Poisson distribution with a mean of

The tness of a rule is calculated as follows is run on randomly chosen initial

congurations on a lattice with N A new set of initial congurations is chosen each

generation and all rules in that generation are tested on it The initial congurations are

uniformly distributed over densities in with exactly half having and exactly half

having is run on each initial conguration for approximately iterations the

actual number is chosen probabilistically to avoid overtting iterations is the measured

maximum amount of time for the GKL CA to reachaninvariant pattern over a large number

of initial congurations on lattice size

s score on a given initial conguration is the fraction of correct bits in the nal

conguration For example if the initial conguration has then s score is the

fraction of s in the nal conguration Thus gets partial credit for getting some of

the bits correct A rule generating random strings would therefore get an average score of

approximately s tness is its average score over all initial congurations For more

details and for justications for these parameters see Mitchell et al

The results of our exp eriment are given in Figure c This histogram displays the

observed frequency of rules in the p opulation at generation as a function of merged

from dierent runs with identical parameters but dierentrandomnumb er seeds The

b est and mean tnesses of rules in each bin are also displayed

There are a numb er of striking dierences b etween Figures b and c

 In Figure b most of the rules in the nal generations cluster in the regions dened

c

by Figure a In particular in Figure b approximately of the mass of the

distribution is in bins and combined where bins are numb ered left to

right In Figure c these bins contain only of the mass of the distribution

there are no rules in bins or and there are only rules in bin out of

a total of rules represented in the histogram

 In Figure b there are rules in every bin In Figure c there are rules only in the

central six bins

 In b oth histograms there are two p eaks surrounding a central dip As in the original

exp eriment in our exp erimenteach individual run pro duced rules at one or the other

p eak so when the runs were merged together b oth p eaks app ear In Figure b how

ever the two p eaks are lo cated roughly at bins and and thus are centered around

and resp ectively In Figure c the p eaks are lo cated roughly at

bins and and thus are centered around and resp ectivelyThe

ratio of the two p eak spreads is thus approximately

 In Figure b the two highest bins are roughly ve times as high as the central bin

whereas in Figure c the two highest bins are roughly three times as high as the

central bin

Figure c also gives an imp ortant calibration the b est and mean tness of rules in

each bin The b est tnesses are all b etween and except the leftmost bin which

has a b est tness of Under this tness function the GKL rule has tness  on all

lattice sizes the GA never found a rule with tness ab ove on lattice size and the

measured tness of the b est evolved rules was muchworse on larger lattice sizes The

tnesses of the rules in Figure b were not given byPackard though none of those

rules achieved the tness of the GKL rule

Discussion of Exp erimental Results

Why Do the Rules Cluster Close to

What accounts for these dierences b etween Figures b and c In particular why did

the evolved rules in our exp eriment tend to cluster close to rather than the two

c

regions

There are two reasons discussed in detail b elow Go o d p erformance on the

c

task requires rules with close to and the GA op erators of crossover and mutation

intrinsically push any p opulation close to

Itcanbeshown that correct or nearly correct p erformance on the task requires

c

rules close to Intuitively this is b ecause the task is symmetric with resp ect to the

exchange of s and s Supp ose for example a rule that carries out the task

c

has This implies that there are more neighb orho o ds in the rule table that map

to output bit than to output bit This in turn means that there will b e some initial

congurations with on which the action of the rule will decrease the number of s

c

And this is the opp osite of the desired action If the rule acts to decrease the numb er of s

on an initial conguration with it risks pro ducing an intermediate conguration with

c

whichthenwould lead under the original assumption that the rule carries out the

c

task correctly to a xed p oint of all s misclassifying the initial conguration A similar

argument holds in the other direction if the rules value is greater than This informal

argumentshows that a rule with  will misclassify certain initial congurations

Generally the further away the rule is from the more such initial congurations

there will b e Such rules may nonetheless classify many initial congurations correctly or

partially correctlyHowever we exp ect any rule that p erforms reasonably well on this task

in the sense of b eing close to the GKL rules average tness across lattice sizesto have

a value close to This selection pressure is one force pushing the GA p opulation to

We note that not surprisingly the GKL rule has

A second force pushing rules to cluster close to isacombinatorial drift force

by which the random actions of crossover and mutation apart from any selection force tend

to push the p opulation towards The results of exp eriments measuring the relative

eects of this force and the selection force in our exp eriment are given elsewhere

Implications of the Requirementfor 

The theoretical argument that the task requires rules with  invalidates

c

Packards use of this task as an evolutionary goal for testing the hyp othesis that rules ca

pable of p erforming complex computations are most likely to b e found close to regions

c

According to Figure a for k r CAs the values o ccur at roughly and

c

But for the classication tasks the range of values required for go o d p erformance

is simply a function of the task and sp ecicallyof Our exp erimental results along

c

with the theoretical argument for  given ab ove lead us to conclude that it is not

correct to interpret Figure b as evidence for the hyp othesis that CAs able to p erform

complex computations will most likely b e found close to Thisisanimportantconclu

c

sion since Packards work is the only published exp erimental study directly linking

with computational ability in CAs

Since classication is only one particular class of tasks this conclusion do es not directly

invalidate sp eculations ab out a generic relationship b etween and computational abilityin

CA However there is to date no theoretical or exp erimental evidence for such a relationship

and an alternative framework for analyzing computation in CAs suggests that there is no

such relationship Moreover it is likely that like classication any particular

nontrivial computational task will have asp ects that require certain ranges of not

necessarily those close to or are not reected in at all

c

Aside from p erforming particular computational tasks it has b een known for some time

that some CAs eg the Game of Life CA are capable in principle of universal computation

this was proved for the Game of Life by explicit construction The Game of Life has

 Langton sketched a construction of some comp onents of a universal computer

c

similar to those used in the construction for the Game of Life in another particular two

dimensional CA in the complex regime However these particular constructions do not

establish any generic relationship b etween and the ability for complex or even universal

c

computation

In summarywe conclude that there is no evidence for a generic relationship b etween

and computational ability in CA and no evidence that an evolutionary pro cess with compu

tational capability as a tness goal will preferentially select CAs at a sp ecial region

c

We do not know for certain what accounted for the dierences b etween our exp erimental

results and those obtained byPackard We sp eculate that the dierences are due to additional

mechanisms in the GA used in the original exp eriment that were not rep orted byPackard

For example the original exp eriment included a numb er of additional sources of randomness

such as the regular injection of new random rules at various values and a much higher

mutation rate than that in our exp eriment These sources of randomness mayhave

slowed the GAs search for hightness rules and prevented it from converging on rules close

to The key observable for this the tness of the evolved CAs was not rep orted by

Packard Our exp erimental results and theoretical analysis indicate that the clustering

close to seen in Figure b is almost certainly an artifact of mechanisms in the particular

c

GA that was used rather than a result of any computational advantage conferred bythe

c

regions To test the robustness of our results wehave p erformed a wide range of additional

exp eriments Not only have the results held up but these exp eriments have p ointed to a

number of novel mechanisms that control the interaction of evolution and computation

What Causes the Dip at

Aside from the many dierences b etween Figure b and Figure c there is one rough

similarity the histogram shows two symmetrical p eaks surrounding a central dip We found

that in our exp eriment this feature is due to a kind of symmetry breaking on the part

of the GA this symmetry breaking actually imp edes the GAs ability to nd a rule with

p erformance at the level of the GKL rule In short the mechanism is the following On each

run the b est strategy found by the GA is one of two equally t strategies

Strategy If the initial conguration contains a suciently large blo ck of adjacent

or nearly adjacent s then increase the size of the blo ckuntil the entire lattice

consists of s Otherwise quickly relax to a conguration of all s

Strategy If the initial conguration contains a suciently large blo ck of adjacent

or nearly adjacent s then increase the size of the blo ckuntil the entire lattice

consists of s Otherwise quickly relax to a conguration of all s

These two strategies rely on lo cal inhomogeneities in the initial conguration as indicators

of Strategy assumes that if there is a suciently large blo ck of s initiallythenthe

is likely to b e greater than and is otherwise likely to b e less than Strategy

makes similar assumptions for suciently large blo cks of s Such strategies are vulnerable

to a numb er of classication errors For example a rule might create a suciently sized

blo ck of s that was not present in an initial conguration with and increase its

size to yield an incorrect nal conguration But as is explained in rules with

for Strategy and rules with for Strategy are less vulnerable to such errors

than are rules with For example a rule with maps more than half of the

neighb orho o ds to and thus tends to decrease the initial Due to this it is less likely to

create a suciently sized blo ck of s from a lowdensity initial conguration

The symmetry breaking involves deciding whether to increase blo cks of s or blo cks of

s The GKL rule is p erfectly symmetric with resp ect to the increase of blo cks of s and

s The GA on the other hand tends to discover one or the other strategy and the one

that is discovered rst tends to takeover the p opulation moving the p opulation s to one

or the other side of

The shap e of the histogram in Figure c thus results from the combination of a number

of forces the selection and combinatorial drift forces describ ed ab ove push the p opulation

toward and the errorresisting forces just describ ed push the p opulation away

from Details of the ep o chs the GA undergo es in developing these strategies are

describ ed in

It is imp ortant to understand how in general such symmetry breaking can imp ede an

evolutionary pro cess from nding optimal strategies This is a sub ject we are currently

investigating

Conclusion

In this chapter wehave reviewed some general ideas ab out the relationship b etween dynami

cal systems theory and the theory of computation In particular wehave discussed in detail

work by Langton and byPackard on the relation b etween dynamical b ehavior and compu

tation in cellular automata Langton investigated correlations b etween and CA b ehavior

as measured by several coarse statistics While there app ears to b e a relationship b etween

high and low and CA b ehavior as measured by these statistics the relationship is weak for

intermediate due to high variance in the statistics there Packards exp erimentwas meant

to directly test the hyp othesis that computational ability is correlated with regions of CA

c

rule space

Wehave presented theoretical arguments and results from an exp eriment similar to

Packards From these we conclude that Packards interpretation of his results was not

correct Webelieve that those original results were due to mechanisms in the particular GA

used in that exp eriment rather than to intrinsic computational prop erties of CAs

c

In addition as wehave noted sp ecic prop erties of the task invalidate it as an

c

evolutionary goal for testing the evolution to hyp othesis Wehave also noted that any

c

particular nontrivial computational task is likely to have prop erties that require certain

ranges of lamb da not related to or are not reected in at all

c

The results presented here do not disprovethehyp othesis that computational capability



can b e correlated with phase transitions in CA rule space Indeed this general phenomena

has already b een noted for other dynamical systems as noted in the intro duction More

generally the computational capacityofevolving systems mayvery well require dynamical

prop erties characteristic of phase transitions if such systems are to increase their complexity

Wehave shown only that the published exp erimental supp ort cited for hyp otheses relating

and computational capability in CAs was not repro duced One problem is that these

c

hyp otheses have not b een unambiguously formulated If the hyp otheses put forth by Langton

and Packard are interpreted to mean that any rule p erforming complex computation

as exempliedbythe task must b e close to thenwehaveshown it to b e

c c

false with our argument that correct p erformance on the task requires

c

If instead the hyp otheses are concerned with generic statistical prop erties of CA rule

spacethe average b ehavior of an average CA at a given then the notion of average

behavior must b e b etter dened Additionally more appropriate measures of dynamical

behavior and computational capabilitymust b e formulated and the notion of the edge

of chaos in CAs must also b e well dened Static parameters estimated directly from the

equations of motion as is from the CA rule table are only the simplest rst step at making

suchhyp otheses and terms welldened and are excellent examples of the problems one

encounters their correlation with dynamical b ehavior is weak and they havefartoomuch

variance when viewed over CA space

Classifying CA b ehavior and analyzing the typ es of computation that CA b ehavior sup

p orts requires a structural view of CAs that go es b eyond quantifying degrees of apparent

disorderapparent disorder is precisely what andvarious meaneld statistics are meant

to indicate The rst steps have b een taken in this direction by delineating various struc

tural elements in CAp erio dic and p ositiveentropy domains intervening walls particles

and particle interactions Employing this approach one can determine the intrinsic

computational capability in CA b ehavior For example this approachgives a metho d for

constructing nonlinear lters that remove p erio dic and chaotic domains from spacetime

data pro duced by a CA The resulting ltered congurations typically reveal how the CA

p erforms its information pro cessing in terms of particles that transmit information over



There are several direct inferences concerning computation in CAs and phase transitions that can b e

drawn from existing results For example individual CAs have b een known for some time to exhibit phase

transitions with the requisite divergence of correlation length required for innite memory capacity

long spacetime distances and particleparticle interactions that p erform logical op era

tions A summary of this typ e of analysis of the GKL rule in terms of particles is given

in

Let us close by reemphasizing that our studies do not preclude a future rigorous and

useful denition of the phrase edge of chaos in the context of cellular automata Nor

do they preclude the discovery that it is asso ciated with a CAs increased computational

capability Finally they do not preclude adaptive systems moving to such dynamical regimes

in order to take advantage of the intrinsic computational capability there In fact the present

work is motivated by our interest in this last p ossibility And the immediate result of that

interest is this attempt to clarify the underlying issues in the hop e of facilitating new progress

along these lines

Acknowledgments

This researchwas supp orted by the Santa Fe Institute under the Core Research Adaptive

Computation and External Faculty Programs and by the University of California Berkeley

under contract AFOSR Thanks to Doyne Farmer Jim Hanson Erica Jen Chris

Langton Wentian Li Cris Mo ore and Norman Packard for many helpful discussions and

suggestions concerning this pro ject Thanks also to Emily Dickinson and Terry Jones for

technical advice

References

E Berlekamp J H Conway and R Guy Winning ways for your mathematical plays Aca

demic Press New York NY

L Blum M Shub and S Smale On a theory of computation over the real numb ers Bul l

AMS

A A Brudno Entropy and the complexity of the tra jectories of a dynamical system Trans

Moscow Math Soc

G Chaitin On the length of programs for computing nite binary sequences J ACM

M Creutz Deterministic Ising dynamics Ann Phys

J P Crutcheld and J E Hanson Turbulent pattern bases for cellular automata PhysicaD

J PCrutcheld and N H Packard Symb olic dynamics of onedimensional maps Entropies

nite precision and noise Intl J Theo Phys

J P Crutcheld and N H Packard Symb olic dynamics of noisy chaos PhysicaD

J P Crutcheld and K Young Inferring statistical complexity Physical Review Letters

J P Crutcheld and K Young Computation at the onset of chaos In W H Zurek ed

itor Complexity Entropy and the Physics of Information pages AddisonWesley

Redwood City CA

D Farmer T Tooli and S Wolfram editors Cel lular Automata Proceedings of an Inter

disciplinary Workshop North Holland Amsterdam

M J Feigenbaum Universal b ehavior in nonlinear systems PhysicaD

P Gacs G L Kurdyumov and L A Levin Onedimensional uniform arrays that wash out

nite islands Probl Peredachi Inform

P Grassb erger Toward a quantitative theory of selfgenerated complexity Intl J Theo

Phys

H A Gutowitz editor Cel lular Automata MIT Press Cambridge MA

J E Hanson and J P Crutcheld The attractorbasin p ortrait of a cellular automaton

Journal of Statistical Physics

J H Holland Adaptation in Natural and Articial Systems MIT Press Cambridge MA

Second edition First edition

J E Hop croft and J D Ullman Introduction to Automata Theory Languages and Compu

tation AddisonWesley Reading

A N Kolmogorov Entropy p er unit time as a metric invariant of automorphisms Dokl Akad

Nauk SSSR Russian Math Rev vol no b

A N Kolmogorov Three approaches to the concept of the amount of information Prob Info

Trans

C G Langton Computation at the edge of chaos Phase transitions and emergent computa

tion PhysicaD

C G Langton Computation at the edge of chaos Phase transitions and emergent computa

tion PhD thesis The UniversityofMichigan Ann Arb or MI

M Mitchell J P Crutcheld and P T Hrab er Evolving cellular automata to p erform

computations Mechanisms and imp ediments PhysicaD in press

M Mitchell P T Hrab er and J P Crutcheld Revisiting the edge of chaos Evolving cellular

automata to p erform computations Complex Systems

C Mo ore Unpredictability and undecidabili ty in dynamical systems Phys Rev Lett

S Omohundro Mo delling cellular automata with partial dierential equations PhysicaD

N Packard Complexityofgrowing patterns in cellular automata In J Demongeot E Goles

and M Tchuente editors Dynamical Behavior of Automata Theory and Applications Aca

demic Press New York

N H Packard Personal communication

N H Packard Adaptation toward the edge of chaos In J A S Kelso A J Mandell and

M F Shlesinger editors Dynamic Patterns in Complex Systems pages Singap ore

World Scientic

K Preston and M Du Modern Cel lular Automata Plenum New York

A Rosenfeld Parallel image pro cessing using cellular arrays Computer

R Shaw Strange attractors chaotic b ehavior and information ow Z Naturforsh a

Ja G Sinai On the notion of entropy of a dynamical system Dokl Akad Nauk SSSR

T Tooli and N Margolus Cel lular Automata Machines A new environment for modeling

MIT Press Cambridge MA

S Wolfram Universality and complexity in cellular automata PhysicaD

S Wolfram editor Theory and applications of cel lular automataWorld Scientic Singap ore