Evolutionary Informatics - 9In X 6In B2390-Fm FA6 Page Iv
Total Page:16
File Type:pdf, Size:1020Kb
February 2, 2017 12:44 Introduction to Evolutionary Informatics - 9in x 6in b2390-fm FA6 page iv Publishers’ page iv February 2, 2017 12:44 Introduction to Evolutionary Informatics - 9in x 6in b2390-fm FA6 page v CONTENTS Preface xiii About the Authors xxiii 1. Introduction 1 1.1. The Queen of Scientists & Engineers .............. 2 1.2. Science and Models .......................... 3 1.2.1. Computer models ....................... 4 1.2.2. The improbable and the impossible ........... 4 Notes ....................................... 5 2. Information: What Is It? 7 2.1. Defining Information ......................... 7 2.2. Measuring Information ........................ 10 2.2.1. KCS complexity ....................... 10 2.2.1.1. KCS information using prefix free programs ...................... 13 2.2.1.2. Random programming and the Kraft inequality ...................... 15 2.2.1.3. Knowability .................... 17 2.2.1.4. Application ..................... 18 2.2.2. Shannon information .................... 18 2.2.2.1. Twenty questions: Interval halving and bits .................. 21 2.2.2.2. Shannon information applied to interval halving ........................ 22 v February 2, 2017 12:44 Introduction to Evolutionary Informatics - 9in x 6in b2390-fm FA6 page vi vi Contents 2.3. Remarks .................................. 26 Notes ....................................... 26 3. Design Search in Evolution and the Requirement of Intelligence 29 3.1. Design as Search ............................ 29 3.1.1. WD-40TM and Formula 409TM .............. 30 3.1.2. Tesla, Edison and domain expertise ........... 30 3.2. Design by Computer ......................... 31 3.3. Designing a Good Pancake ..................... 32 3.3.1. A search for a good pancake #1 ............. 32 3.3.2. A search for a good pancake #2: Cooking times plus range setting ....................... 35 3.3.3. A search for a good pancake #3: More recipe variables ............................. 36 3.3.4. A search for a good pancake #4: Simulating pancakes on a computer with an artificial tongue using a single agent ..................... 38 3.3.5. A search for a good pancake #5: Simulating pancakes on a computer with an artificial tongue using an evolutionary search ............... 40 3.4. Sources of Knowledge ........................ 41 3.4.1. Designing antennas using evolutionary computing ............................ 43 3.5. The Curse of Dimensionality & the Need for Knowledge ............................. 46 3.5.1. Will Moore ever help? How about Grover? ...... 47 3.6. Implicit Targets ............................. 48 3.7. Skeptic Fallibility ........................... 50 3.7.1. Loss of function ........................ 52 3.7.2. Pareto optimization and optimal sub-optimality . 52 3.7.3. A man-in-the-loop sneaks in active information . 55 3.7.3.1. Evolving Tic-Tac-Toe to checkers to chess ....................... 55 3.7.3.2. Replacing the man-in-the-loop with a computer-in-the-loop .............. 55 February 2, 2017 12:44 Introduction to Evolutionary Informatics - 9in x 6in b2390-fm FA6 page vii Contents vii 3.8. A Smörgåsbord of Search Algorithms .............. 56 3.9. Conclusions ............................... 59 Notes ....................................... 59 4. Determinism in Randomness 67 4.1. Bernoulli’s Principle of Insufficient Reason .......... 69 4.1.1. “Nothing is that which rocks dream about” ..... 69 4.1.2. Bernoulli’s Principle (PrOIR) ............... 70 4.1.2.1. Examples ...................... 71 4.1.2.2. Criticisms of Bernoulli’s principle ..... 71 4.1.2.2.1. Model variations .......... 72 4.1.2.2.2. Vague definitions & ambiguity: Bertrand’s paradox ........ 78 4.1.2.2.3. Continuous versus discrete probability .............. 81 4.2. The Need for Noise .......................... 86 4.2.1. Fixed points in random events .............. 87 4.2.2. Importance sampling .................... 90 4.2.3. Limit cycles, strange attractors & tetherball ..... 92 4.3. Basener’s ceiling ............................ 93 4.3.1. Tierra ............................... 95 4.3.2. The edge of evolution .................... 98 4.4. Final Comments ............................ 99 Notes ....................................... 100 5. Conservation of Information in Computer Search 105 5.1. The Genesis ............................... 105 5.2. What is Conservation of Information? .............. 107 5.2.1. Deceptive counterexamples ................ 109 5.2.2. What does learning have to do with design? ..... 112 5.2.2.1. Sumo wrestlers can’t play basketball .... 112 5.2.3. A man-in-the-loop sneaks in active information . 117 5.2.3.1. Back room tuning ................ 119 5.3. The Astonishing Cost of Blind Search in Bits ......... 120 February 2, 2017 12:44 Introduction to Evolutionary Informatics - 9in x 6in b2390-fm FA6 page viii viii Contents 5.3.1. Analysis ............................. 121 5.3.2. The cost ............................. 123 5.4. Measuring Search Difficulty in Bits ............... 125 5.4.1. Endogenous information .................. 125 5.4.1.1. Two special cases ................. 127 5.4.1.2. Endogenous information of the Cracker Barrel puzzle .................... 128 5.4.2. Active information ...................... 130 5.4.2.1. Examples of sources of knowledge ..... 134 5.4.2.2. Active information per query ......... 135 5.4.2.2.1. A subtle distinction ........ 136 5.4.2.3. Examples of active information ....... 138 5.4.2.3.1. The Cracker Barrel puzzle . 138 5.4.2.3.2. The Monte Hall problem .... 140 5.4.2.3.3. A sibling problem ......... 141 5.4.2.3.4. Multiple queries .......... 144 5.4.3. Mining active information from oracles ........ 145 5.4.3.1. The Hamming oracle .............. 145 5.4.3.2. Weasel ware and variations of information mining ............... 151 5.5. Sources of Information in Evolutionary Search ........ 155 5.5.1. Population ........................... 155 5.5.2. Mutation rate .......................... 156 5.5.3. Fitness landscapes ...................... 156 5.5.3.1. Initialization .................... 160 5.6. Stairstep Information & Transitional Functional Viability .................................. 160 5.6.1. Baby steps ........................... 161 5.6.2. Developmental functionality and irreducible complexity ........................... 161 5.6.2.1. Example: Using an EAR_TATTER_ .... 164 5.6.2.2. Analysis ....................... 165 5.6.3. Irreducible complexity ................... 167 5.7. Coevolution ............................... 167 5.8. The Search for the Search ...................... 171 5.8.1. An example ........................... 171 February 2, 2017 12:44 Introduction to Evolutionary Informatics - 9in x 6in b2390-fm FA6 page ix Contents ix 5.8.2. The problem .......................... 173 5.8.2.1. The weak case ................... 174 5.8.2.2. The strict case ................... 174 5.8.3. Proofs .............................. 175 5.8.3.1. Preliminaries .................... 175 5.8.3.2. The weak case ................... 177 5.8.3.3. The strict case ................... 178 5.9. Conclusion ................................ 180 Notes ....................................... 181 6. Analysis of Some Biologically Motivated Evolutionary Models 187 6.1. EV: A Software Model of Evolution ............... 188 6.1.1. EV structure .......................... 188 6.1.2. EV vivisection ......................... 192 6.1.3. Information sources resident in EV ........... 194 6.1.4. The search ........................... 198 6.1.4.1. Search using the number cruncher ..... 198 6.1.4.2. Evolutionary search ............... 198 6.1.4.3. EV and stochastic hill climbing ....... 199 6.1.4.4. Mutation rate ................... 199 6.1.5. EV ware ............................. 200 6.1.6. The diagnosis ......................... 201 6.2. Avida: Stair Steps to Complexity Using NAND Logic . 205 6.2.1. Kitzmiller et al. versus Dover area school district ......................... 206 6.2.2. Boolean logic ......................... 207 6.2.3. NAND logic .......................... 209 6.2.3.1. Logic synthesis using NAND gates ..... 209 6.2.4. The Avida organism and its health ........... 213 6.2.5. Information analysis of Avida .............. 217 6.2.5.1. Performance .................... 219 6.2.5.1.1. The evolutionary approach . 219 6.2.5.1.2. The ratchet approach ....... 220 6.2.5.1.3. Comparison ............. 221 6.2.5.2. Minivida ....................... 221 6.2.5.2.1. The full program .......... 223 February 2, 2017 12:44 Introduction to Evolutionary Informatics - 9in x 6in b2390-fm FA6 page x x Contents 6.2.5.2.2. Remove the staircase ....... 224 6.2.5.2.3. Minimal instructions ....... 225 6.2.6. Avida is intelligently designed .............. 227 6.2.7. Beating a dead organism .................. 230 6.3. Metabiology ............................... 231 6.3.1. The essence of halting .................... 233 6.3.2. On with the search ...................... 234 6.3.3. The math: “intelligent design” in metabiology . 236 6.3.4. Resources ............................ 240 6.4. Conclusion: Sweeping a Dirt Floor ................ 241 6.4.1. Evolving a Steiner tree ................... 241 6.4.2. Time for evolution ...................... 242 6.4.3. Finis ............................... 243 Notes ....................................... 243 7. Measuring Meaning: Algorithmic Specified Complexity 251 7.1. The Meaning of Meaning ...................... 251 7.2. Conditional KCS Complexity ................... 253 7.3. Defining Algorithmic Specified Complexity (ASC) ..... 255 7.3.1. High ASC