On Mutation and Crossover in the Theory of Evolutionary Algorithms
Total Page:16
File Type:pdf, Size:1020Kb
ON MUTATION AND CROSSOVER IN THE THEORY OF EVOLUTIONARY ALGORITHMS by James Neal Richter A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science MONTANA STATE UNIVERSITY Bozeman, Montana April, 2010 c Copyright by James Neal Richter 2010 All Rights Reserved ii APPROVAL of a dissertation submitted by James Neal Richter This dissertation has been read by each member of the dissertation committee and has been found to be satisfactory regarding content, English usage, format, citations, bibliographic style, and consistency, and is ready for submission to the Division of Graduate Education. Dr. John T. Paxton Approved for the Department of Computer Science Dr. John T. Paxton Approved for the Division of Graduate Education Dr. Carl A. Fox iii STATEMENT OF PERMISSION TO USE In presenting this dissertation in partial fulfillment of the requirements for a doc- toral degree at Montana State University, I agree that the Library shall make it available to borrowers under rules of the Library. I further agree that copying of this dissertation is allowable only for scholarly purposes, consistent with \fair use" as pre- scribed in the U.S. Copyright Law. Requests for extensive copying or reproduction of this dissertation should be referred to ProQuest Information and Learning, 300 North Zeeb Road, Ann Arbor, Michigan 48106, to whom I have granted \the exclusive right to reproduce and distribute my dissertation in and from microform along with the non-exclusive right to reproduce and distribute my abstract in any format in whole or in part." James Neal Richter April, 2010 iv DEDICATION This work is dedicated to my father. I miss you every day Dad. James Franklin Richter (February 24, 1957 - January 14, 2006). I also dedicate this work to my mothers Maureen, Francile and Geneal, my lovely wife Stephanie and our four children, James Zachary, Kathleen, Brooklyn and Marcus. v ACKNOWLEDGEMENTS Dr. John Paxton has been my academic advisor since 1994 when I began under- grad studies in Computer Science. His encouragement and advice over the years have been invaluable. Dr. Alden Wright of the University of Montana acted as the external co-chair of the dissertation committee. His mentorship, expertise and collaboration was instru- mental in following this research path. Since I began PhD studies I've gotten married, lost a father, gained four children, had three different jobs, helped reboot and sell a software company then joined the buyer's software company to tackle bigger challenges. As a result this work was often left idle in drawer. I would like to thank John and Alden for their encouragement and patience through all that. Thomas Jansen, Michael Vose, Jon Rowe, Richard Watson, Lothar Schmitt, and the late Ingo Wegener are also acknowledged for their encouragement and advice. I would also like to express gratitude to the participants and organizers of the Theory of Evolutionary Algorithms conference series at Schloss Dagstuhl International Con- ference and Research Center for Computer Science, where the seeds of this research were germinated. The Departments of Computer Science and Mathematics have my gratitude for the education and support provided. Thanks also go out to my industry co-workers and supervisors at RightNow, Others Online and the Rubicon Project. Thanks also to Michael Guerini, who helped edit and beat this text into better shape. vi TABLE OF CONTENTS 1. INTRODUCTION ........................................................................................1 1.1 Document Organization........................................................................2 1.2 Framework and Contributions ...............................................................3 2. INTRODUCTION TO EVOLUTIONARY ALGORITHMS .............................4 2.1 Evolutionary Algorithms.......................................................................4 2.1.1 The Simple Genetic Algorithm.........................................................6 2.2 Pre-History of Evolutionary Algorithms.................................................8 2.2.1 Birth of Genetic Algorithms .......................................................... 10 2.2.2 Synthesis of Evolutionary Computation.......................................... 11 2.2.3 Population Size............................................................................. 12 2.2.4 The Selection Operators ................................................................ 13 2.2.5 The Mutation Operator................................................................. 15 2.2.6 The Crossover Operator ................................................................ 15 2.3 Common Difficulties ........................................................................... 17 2.4 Operators and Parameter Setting ........................................................ 19 2.5 Relation to Metaheuristic Optimization ............................................... 21 2.6 How do EAs really work?.................................................................... 21 3. HISTORICAL REVIEW OF EVOLUTION AND EVOLUTIONARY THEO- RIES..........................................................................................................23 3.1 Theory of Evolution ........................................................................... 23 3.2 Historical Anticipation and Reception of Darwinism ............................. 24 3.3 Early Interpretations .......................................................................... 26 3.4 Genetics and The Chromosome ........................................................... 28 3.5 Mutationism ...................................................................................... 29 3.6 Selection, Recombination and Linkage ................................................. 31 3.7 Population Genetics ........................................................................... 35 3.8 Modern Evolutionary Synthesis ........................................................... 41 3.9 Chromosomes and DNA...................................................................... 43 3.10 Neutral Theory of Evolution ............................................................... 45 3.11 Views on Recombination..................................................................... 48 3.12 Punctuated Equilibria ........................................................................ 57 3.13 Statistical versus Dynamical Theories .................................................. 57 3.14 Overconfidence in Evolutionary Theory ............................................... 58 3.15 Current Status of the Synthesis........................................................... 59 3.16 Relevance to Evolutionary Algorithms ................................................. 60 vii TABLE OF CONTENTS { CONTINUED 4. REVIEW OF EVOLUTIONARY ALGORITHM THEORY...........................61 4.1 Introduction to EA Theory ................................................................. 61 4.1.1 Theoretical Population Genetics..................................................... 61 4.1.2 Reductionist Approaches ............................................................... 71 4.1.3 Schema Theorem and Building Block Hypothesis ............................ 73 4.1.4 Markov Models ............................................................................. 76 4.1.5 Infinite Population and Dynamical Systems Models......................... 78 4.1.6 Statistical Mechanics..................................................................... 80 4.1.7 Time Complexity Analysis............................................................. 81 4.1.8 No Free Lunch Theorems............................................................... 83 4.2 Crossover versus Mutation .................................................................. 84 4.2.1 Hard Functions for Mutation based EAs......................................... 84 4.2.2 Hard Functions for Crossover based EAs ........................................ 88 4.3 Genetic and Population Diversity ........................................................ 90 4.4 Detailed Review of Relevant Theory .................................................... 90 4.5 Detailed Review of Time Complexity Analysis ..................................... 91 4.5.1 The (µ + λ) EA ............................................................................ 93 4.5.2 The (1 + 1) EA............................................................................. 94 4.5.3 Metropolis Algorithms and Simulated Annealing Hillclimbers .......... 96 4.5.4 Comparing 1 vs µ Population EAs ............................................... 100 4.5.5 Analyzing Crossover.................................................................... 102 4.6 Detailed Review of the Infinite Population Model............................... 106 4.6.1 Selection-Mutation Models .......................................................... 109 4.6.2 Course Grained Models ............................................................... 111 4.7 Summary......................................................................................... 112 5. IGNOBLE TRAILS FOR CONSTANT POPULATION SIZE...................... 113 5.1 Introduction..................................................................................... 113 5.2 Minimal Population Evolutionary Algorithms..................................... 115 5.2.1 The Steady-State (2+1) EA......................................................... 115 5.2.2 The Steady-State (2+1)