Neutral Emergence and Coarse Graining Cellular Automata
Total Page:16
File Type:pdf, Size:1020Kb
NEUTRAL EMERGENCE AND COARSE GRAINING CELLULAR AUTOMATA Andrew Weeks Submitted for the degree of Doctor of Philosophy University of York Department of Computer Science March 2010 Neutral Emergence and Coarse Graining Cellular Automata ABSTRACT Emergent systems are often thought of as special, and are often linked to desirable properties like robustness, fault tolerance and adaptability. But, though not well understood, emergence is not a magical, unfathomable property. We introduce neutral emergence as a new way to explore emergent phe- nomena, showing that being good enough, enough of the time may actu- ally yield more robust solutions more quickly. We then use cellular automata as a substrate to investigate emergence, and find they are capable of exhibiting emergent phenomena through coarse graining. Coarse graining shows us that emergence is a relative concept – while some models may be more useful, there is no correct emergent model – and that emergence is lossy, mapping the high level model to a subset of the low level behaviour. We develop a method of quantifying the ‘goodness’ of a coarse graining (and the quality of the emergent model) and use this to find emergent models – and, later, the emergent models we want – automatically. i Neutral Emergence and Coarse Graining Cellular Automata ii Neutral Emergence and Coarse Graining Cellular Automata CONTENTS Abstract i Figures ix Acknowledgements xv Declaration xvii 1 Introduction 1 1.1 Neutral emergence 1 1.2 Evolutionary algorithms, landscapes and dynamics 1 1.3 Investigations with cellular automata 2 2 Cellular automata 5 2.1 Cellular automata grids 5 2.2 Updating a CA 5 2.3 CA dynamics 6 2.4 Conway’s Game of Life 7 2.5 Elementary cellular automata 8 2.6 Simple ECA behaviour 8 2.7 More complex ECA behaviour 10 2.8 Key points 11 3 Evolutionary algorithms 13 3.1 Genetic algorithms 14 3.2 A tension in evolution 18 3.3 Recombination as conservation 19 3.4 Recombination as innovation 20 3.5 Key points 21 4 Nonlinear dynamics 23 4.1 Pendulum 23 4.2 Bifurcations 25 4.3 Saddle node bifurcation 25 4.4 Transcritical bifurcation 26 4.5 Pitchfork bifurcation 27 4.6 Hopf bifurcation 28 4.7 Linear analysis 29 4.8 Phase portrait stability 31 4.9 Nullclines 31 4.10 Higher dimensional systems 32 4.11 Chaos in discrete systems 35 4.12 Bifurcation diagrams 36 4.13 Key points 37 5 Computation as a dynamical system 39 5.1 The λ parameter 39 5.2 A phase transition 40 5.3 Correlation 40 iii Neutral Emergence and Coarse Graining Cellular Automata 5.4 Growth dimension 41 5.5 Length of transients 42 5.6 Key points 43 6 NK landscapes and random Boolean networks 45 6.1 The NK model 45 6.2 Random Boolean networks 52 6.3 Key points 63 7 Entropy 65 7.1 Thermodynamics 65 7.2 Heat and Work 66 7.3 Entropy 67 7.4 The Second Law and Life 67 7.5 Gibbs Free Energy 68 7.6 Summary of thermodynamics 69 7.7 Information theory 70 7.8 Information theory 70 7.9 Shannon entropy, joint and conditional entropies 71 7.10 Information 72 7.11 Noiseless encoding 73 7.12 Key points 73 8 Emergence review 75 8.1 Why emergence? 75 8.2 Modelling flocking behaviour 75 8.3 Defining emergence 76 8.4 Downward causation 78 8.5 Supervenience and emergence 80 8.6 Levels and complexity in emergence 80 8.7 Emergence as a change of scope or resolution 81 8.8 Weak emergence 82 8.9 Strong emergence 83 8.10 Dynamics and emergence 83 8.11 Abstractions in flocking 85 8.12 Evolving flocking 85 8.13 Flocking through predation 86 9 Emergence 89 9.1 Defining emergence 89 9.2 A starting definition of emergence 89 9.3 Subjective emergence 90 9.4 Useful emergence 90 9.5 Independence and lossy emergence 91 9.6 Discontinuities and lossy emergence 92 9.7 Projection and sampling 93 9.8 Mappings in emergence 93 9.9 Neutral emergence 95 9.10 Neutral evolution 95 iv Neutral Emergence and Coarse Graining Cellular Automata 9.11 Quantitative emergence 95 9.12 Complexity and information 96 9.13 Kolmogorov complexity 97 9.14 Minimum model complexity 98 9.15 Information retention 99 9.16 Comment on Adami’s model 99 9.17 Quantifying emergence 100 9.18 Neutral emergence 101 9.19 Engineering emergence 102 9.20 Robustness in neutral emergence 103 9.21 The power of neutrality 104 9.22 Emergence is easy 105 9.23 Exploiting problem structure 105 9.24 ‘Good enough’ solutions 106 9.25 Key points 107 Using cellular automata to investigate neutral emergence 109 10 Coarse graining CAs 111 10.1 Predicting cellular automaton behaviour 111 10.2 How to find Life: identifying patterns and predicting behaviour 111 10.3 Predicting the future 114 10.4 Elucidation through elimination 114 10.5 Coarse graining and emergence 115 10.6 What is coarse graining? 115 10.7 Why coarse grain? 117 10.8 Coarse graining needs consistent mappings 117 10.9 Cell mappings are sufficient 118 10.10 How to coarse grain 119 10.11 Something of a waste 123 10.12 Different coarse grainings 124 10.13 Graining graphs 126 10.14 Moving to two dimensions 131 10.15 Explosive test cases 132 10.16 Partial coarse graining 133 10.17 A partial population 134 10.18 Finding a partial coarse graining 134 10.19 Partial graining graphs 138 10.20 Mappings 139 10.21 Taking the union of all mappings 148 10.22 Using known rule cases 149 10.23 Key points 150 11 Emergence and information 153 11.1 Types of information loss 153 11.2 Coarse graining and mutual information 154 11.3 How to calculate the MI 155 11.4 Calculating the MI 157 v Neutral Emergence and Coarse Graining Cellular Automata 11.5 Calculating the MI example 157 11.6 The significance of 1.5 160 11.7 Partial coarse graining and mutual information 160 11.8 Choosing MI test strings 162 11.9 Mutual information of different coarse grainings 168 11.10 Graining graphs and MI 173 11.11 Predicting good coarse grainings 173 11.12 Mappings and MI 176 11.13 Key points 179 12 Subjective emergence 181 12.1 Feature extraction in emergent systems 182 12.2 How to extract features from an elementary CA 182 12.3 Example of feature extraction 183 12.4 Forcing contiguous blocks 186 12.5 Towards directed coarse graining 187 12.6 Phase changes in cellular automata 187 12.7 Catching the transition 189 12.8 MI and phase transitions 192 12.9 Exploring restart position 193 12.10 A clearer transition graph 194 12.11 Coarse graining rule 130 to rules 128 and 213 195 12.12 Coarse graining rule 130 to rules 128 and 84 197 12.13 Relative graphs 198 12.14 The inevitable S-curve 199 12.15 Analysis of finding transition points 199 12.16 Overcoming the mutual information problem 200 12.17 A temporal dimension in mutual information 200 12.18 True temporal mutual information 204 12.19 Analysis of temporal mutual information 205 12.20 Analysis of different block shapes and sizes 206 12.21 Extra entropy: a better distinction 207 12.22 Liberal coarse graining 208 12.23 Applying extra entropy to other coarse grainings 209 12.24 Graphs of rule 130’s extra entropy 209 12.25 The best block size 213 12.26 Extra entropy of rule 130’s coarse grainings 214 12.27 Extra entropy of rule 140’s coarse grainings 217 12.28 Analysis of extra entropy 220 12.29 Directed coarse graining 222 12.30 Finding behaviour of interest 222 12.31 Creating an exception to a rule 223 12.32 Exceptions 224 12.33 Sparse exceptions 225 12.34 Exceptions at the coarse level 226 12.35 Coarse graining with exceptions 227 12.36 Another exception using rule 138 228 vi Neutral Emergence and Coarse Graining Cellular Automata 12.37 Adding an exception to a chaotic rule 230 12.38 Directed coarse graining through exceptions 235 12.39 Key points 235 13 Contributions 237 13.1 Emergence 237 13.2 Coarse graining and emergence in cellular automata 238 13.3 Emergence and information 239 14 Looking forward 241 14.1 Emergence and robustness 241 14.2 Exploiting problem structure 242 14.3 Developmental systems 242 A Coarse grainings 247 B MI of random strings 265 C Coarse graining distribution 267 D Phase changes initial condition 269 D.1 Investigating string length 270 D.2 Avoiding MI resonance 272 E Extra entropy of selected rules 275 E.1 Extra entropy of rule 130’s coarse grainings 275 E.2 Extra entropy of rule 140’s coarse grainings 280 E.3 Extra entropy for rule 43’s coarse grainings 284 E.4 Extra entropy of rule 192’s coarse grainings 288 References 291 vii Neutral Emergence and Coarse Graining Cellular Automata viii Neutral Emergence and Coarse Graining Cellular Automata FIGURES 2 Cellular automata Figure 2.1 A 2D cellular automaton grid. Each square is a cell. 5 Figure 2.2 The Moore neighbourhood for a regular 2D CA (left); the von Neumann neighbourhood… 6 Figure 2.3 The progression over time from all possible states. (Image from [48]) Garden of… 6 Figure 2.4 Patterns in Life 7 Figure 2.5 Elementary CA rule 58 8 Figure 2.6 The result of running rule 128 for twelve steps, starting with the initial condition… 9 Figure 2.7 Rule 128 9 Figure 2.8 A run of rule 138.