The Design and Analysis of a Computational Model of Cooperative Coevolution A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy at George Mason University by Mitchell A. Potter BA, Mathematics, Florida State University, 1978 MS, Computer Science, University of Texas at Austin, 1985 Director: Kenneth A. De Jong, Associate Professor Department of Computer Science Spring Semester 1997 George Mason University Fairfax, Virginia ii Copyright 1997 Mitchell A. Potter All Rights Reserved iii DEDICATION To Amy iv ACKNOWLEDGEMENTS Throughout my doctoral studies at George Mason University, I have interacted with many excellent faculty, staff, and students. I especially wish to acknowledge the support of my dissertation director, Ken De Jong. I have been extremely fortunate to have had access to his considerable insight into the field of evolutionary computation. I also thank my other committee members Ken Hintz, Eugene Norris, and Gheorghe Tecuci for their helpful comments and suggestions regarding this work. I am also grateful to doctoral students Jayshree Sarma, Alan Schultz, Bill Spears, and Haleh Vafaie for their companionship and their willingness to listen when I needed a sound- ing board for ideas; research librarians Maryalls Bedford, Amy Keyser, and Cathy Wiley at the Naval Research Laboratory for their help in tracking down references; Eric Bloedorn at the Machine Learning Laboratory at George Mason University for running AQ15 on the congressional voting records data set and providing me with the conjunctive descriptions and related performance data documented in chapter 6; Scott Fahlman at Carnegie Mellon University for use of his cascade-correlation simulator; and Anne Marie Casey for her help in editing this dissertation. Special thanks go to my wife, Amy, for her encouragement, patience, and understanding. This dissertation was written on a NeXT workstation and typeset with LATEX2e. The graphics were produced with Gnuplot, Mathematica, and Diagram!2. All the experi- ments were run on two large networks of Sun workstations at the Navy Center for Applied Research in Artificial Intelligence, and the Center for the New Engineer at George Mason University. This work was supported in part by the Office of Naval Research. I am extremely grateful to John Grefenstette at the Navy Center for Applied Research in Artificial Intelligence for making this financial support possible. v TABLE OF CONTENTS List of Figures viii List of Tables xi Abstract xii 1 Introduction 1 1.1 Motivation .................................... 1 1.2 CurrentApproaches ............................... 3 1.3 Objectives..................................... 3 1.4 Methodology ................................... 4 1.5 ProposedCoevolutionaryModel . ... 4 1.6 Contributions................................... 5 1.7 DissertationOutline .. .. .. .. .. .. .. .. 6 2 Background and Related Work 7 2.1 EvolutionaryComputation . 7 2.1.1 GeneticAlgorithms............................ 8 2.1.2 Evolution Strategies . 11 2.1.3 Evolutionary Algorithm Differences . 13 2.2 Issues in Evolving Coadapted Subcomponents . ....... 14 2.2.1 ProblemDecomposition . 14 2.2.2 Interdependencies Between Subcomponents . ..... 15 2.2.3 CreditAssignment ............................ 17 2.2.4 Population Diversity . 18 2.2.5 Parallelism ................................ 18 2.3 RelatedWork................................... 19 2.3.1 Single Population Approaches . 19 2.3.2 Multiple Population Approaches . 26 2.4 Limitations of Previous Approaches . ..... 28 3 Architecture 30 3.1 A Model of Cooperative Coevolution . 30 3.2 IssuesRevisited................................. 35 3.2.1 ProblemDecomposition . 35 3.2.2 Interdependencies Between Subcomponents . ..... 36 vi 3.2.3 CreditAssignment ............................ 36 3.2.4 Population Diversity . 37 3.2.5 Parallelism ................................ 37 3.3 Additional Advantages of the Model . 38 3.3.1 Speciation Through Genetic Isolation . 38 3.3.2 Generality................................. 39 3.3.3 Efficiency ................................. 39 3.4 ASimpleExample ................................ 39 4 Analysis of Sensitivity to Selected Problem Characteristics 43 4.1 Selection of Problem Characteristics . ....... 43 4.2 Methodology ................................... 44 4.3 Sensitivity to Random Epistatic Interactions . ......... 45 4.3.1 NK-LandscapeProblem . 46 4.3.2 ExperimentalResults .......................... 48 4.4 Sensitivity to Highly Ordered Epistatic Interactions . ............ 56 4.4.1 Coevolutionary Function Optimization . ..... 56 4.4.2 FunctionSeparability . 57 4.4.3 TestSuite................................. 58 4.4.4 ExperimentalResults .......................... 59 4.5 Sensitivity to Dimensionality . ..... 64 4.5.1 TestSuite................................. 64 4.5.2 ExperimentalResults .......................... 65 4.6 SensitivitytoNoise.............................. 68 4.6.1 TestSuite................................. 68 4.6.2 ExperimentalResults .......................... 68 4.7 Summary ..................................... 70 5 Basic Decomposition Capability of the Model 72 5.1 StringCoveringProblem. 73 5.2 EvolvingStringCovers. 73 5.3 Locating and Covering Multiple Environmental Niches . .......... 73 5.4 Finding an Appropriate Level of Generality . ....... 76 5.5 AdaptingtoaDynamicEnvironment. 80 5.6 Evolving an Appropriate Number of Species . ...... 83 5.7 Summary ..................................... 85 6 Case Studies in Emergent Problem Decomposition 87 6.1 Artificial Neural Network Case Study . 87 6.1.1 EvolvingCascadeNetworks . 88 6.1.2 The Cascade-Correlation Approach to Decomposition . ....... 90 6.1.3 Two-SpiralsProblem........................... 91 6.1.4 ExperimentalResults .......................... 92 6.2 ConceptLearningCaseStudy. 99 6.2.1 Evolving an Immune System for Concept Learning . 100 vii 6.2.2 The AQ Approach to Decomposition . 104 6.2.3 Congressional Voting Records Data Set . 105 6.2.4 ExperimentalResults . .. .. .. .. .. .. .. 105 6.3 Summary ..................................... 113 7 Conclusions 115 7.1 Summary ..................................... 115 7.2 FutureResearch ................................. 116 Bibliography 120 A Program Code for Cooperative Coevolution Model 132 B Parameter Optimization Problems 144 C Program Code for Coordinate Rotation Algorithm 153 viii LIST OF FIGURES 2.1 Canonical genetic algorithm . 10 2.2 Two-point crossover and mutation operators . ........ 11 2.3 Canonical (µ, λ)evolutionstrategy . .. .. .. .. .. .. 13 2.4 Match set, target set, and connection strengths before and after modification toamatchsetelement.............................. 17 2.5 An algorithm for modeling emergent fitness sharing in the immune system . 25 3.1 Canonical cooperative coevolution algorithm . ......... 31 3.2 Fitness evaluation of individuals from species S ................ 32 3.3 Model of species interaction . 33 3.4 Birthanddeathofspecies . 34 3.5 Average match score between target set and best collaborations . 41 3.6 Percent contribution of each species to best collaborations.......... 42 4.1 Standard genetic algorithm applied to 24-bit NK landscape with various lev- elsofepistasis................................... 49 4.2 Standard genetic algorithm and random search on 24-bit NK landscape with no epistasis (K = 0) and maximum epistasis (K = 23) . 50 4.3 Coevolution and standard genetic algorithm on two uncoupled 24-bit NK landscapeswithnoepistasis(K=0) . 51 4.4 Coevolution and standard genetic algorithm on two uncoupled 24-bit NK landscapeswithlowepistasis(K=3) . 52 4.5 Coevolution and standard genetic algorithm on two uncoupled 24-bit NK landscapes with moderate epistasis (K = 7) . 52 4.6 Coevolution and standard genetic algorithm on two uncoupled 24-bit NK landscapes with maximum epistasis (K = 23) . 53 4.7 Effect of optimizing coupled NK landscapes separately and merging the final solutions...................................... 54 4.8 Coevolution and standard genetic algorithm on two coupled 24-bit NK land- scapes(K=7andC=2)............................ 54 4.9 Coevolution and standard genetic algorithm on two coupled 24-bit NK land- scapes(K=7andC=4)............................ 55 4.10 Coevolution and standard genetic algorithm on two coupled 24-bit NK land- scapes(K=7andC=8)............................ 55 4.11 Coevolution and standard genetic algorithm on two coupled 24-bit NK land- scapes(K=7andC=16) ........................... 56 ix 4.12 Sensitivity of coevolution and standard genetic algorithm to coordinate ro- tationofAckleyfunction. 60 4.13 Sensitivity of coevolution and standard genetic algorithm to coordinate ro- tationofRastriginfunction . 60 4.14 Sensitivity of coevolution and standard genetic algorithm to coordinate ro- tationofSchwefelfunction. 61 4.15 Sensitivity of coevolution and standard genetic algorithm to coordinate ro- tation of extended Rosenbrock function . 63 4.16 Effect of a less greedy collaboration strategy on the optimization of the ro- tatedAckleyfunction .............................. 63 4.17 Sensitivity of coevolution and standard genetic algorithm to changes in di- mensionalityofspheremodel . 66 4.18 Sensitivity of coevolution and standard genetic algorithm to changes in di- mensionality of extended Rosenbrock function . ...... 67 4.19 Sensitivity of coevolution and standard genetic algorithm to changes in the standard deviation of noise in stochastic De Jong function . ......... 69 4.20 Sensitivity of coevolution and standard