Johan Høye ArtDev3D: An Artificial Development System Trondheim, March 2006 Norges teknisk-naturvitenskapelige universitet Faculty of Information Technology, Mathematics and Electrical Engineering Department of Computer and Information Science Master's thesis Programme of study: Master of Science in Informatics Supervisor: Pauline Haddow, IDI Co-Supervisor(s): Diego Federici, IDI Gunnar Tufte, IDI Foreword This master thesis is written by Johan Høye as a part of the Master of Science in Informatics degree at the Department of Computer and Information Science (DIS), Norwegian University of Science and Technology (NTNU). The following assignment description lays the basis for this thesis. “The assignment is to design and test a novel artificial development system implementing a biologically inspired development process used for the process of mapping the genotype to the phenotype in an evolutionary algorithm.” i Abstract Evolutionary algorithms (EAs) are a class of population-based stochastic search algorithms which have proven themselves to be powerful tools in optimization problems where the search space is complex, contains many local optima, and is so large that an exhaustive search is not possible. An application area where EAs have great potential is in the design of electronic circuits. However, for this type of task such a large representation is typically required for each of the proposed solutions that using an EA approach is not feasible because of the immense computational power this would require. This limitation of EAs is known as the scalability problem: EAs perform well when dealing with problems requiring a small solution representation, but when the required size for these representations increases the EAs quickly become too computationally expensive to be useful. Numerous approaches for dealing with the scalability problem have been proposed. One of the more promising approaches is inspired by the way nature copes with scaling: the process of an organism growing from a single fertilized cell and into a multi-cellular being, called development. By adapting some of the mechanisms of development to a computer program, the EA can evolve a relatively small genome which when developed i.e. decompressed, using this program will represent a solution. There are, however, some problems regarding this approach. One issue is that biological development is such a complex process that implementing it in all its detail is neither feasible nor desired, meaning a decision regarding which mechanisms to implement and which ones to leave out must be made. Another issue is the increased difficulty to evolve a good solution. This occurs because EAs depend on a gradual refinement of the solution to be effective, but with this approach a small change in the genome may lead to a large change in the corresponding solution. This is because in this approach there is no longer a direct correspondence between the genotype space and the solution space, so that what is adjacent in the genotype space may be far apart in the solution space. This means that even though gradual refinement is achieved in genotype space, the changes in the corresponding solution space may appear to be more or less random A novel artificial development system, designed and implemented from scratch, is presented in this thesis. A novel system was built because, although a number of other such system already have been implemented, they are all in the experimental stage, and this system is though to be a useful supplement to the existing ones, providing more material to base the understanding of what may be useful in an artificial development system on. An explorative approach was taken where the implemented system was put through a number of tests to investigate its capabilities. First the systems ability to develop a varied set of different shapes was investigated. Secondly, four parameters were tested for their effect on the system's ability to develop good solutions: the initial number of neighbours, the number of chemical types used (both part of a precondition), the number of cell types available to the system, and the degree of symmetry in the target shapes. The experiments performed showed that the system is able to develop a number of shapes. For the four investigated parameters, indications were found that each has a profound effect on the systems ability to develop a given target. iii Acknowledgements The following people deserves my gratitude for their help during my work on this thesis. Pauline Haddow for being my supervisor, for giving me valuable advice and for helping me to keep this thesis structured Diego Federici for fruitful discussions, and an overwhelming amount of creative suggestions Gunnar Tufte for valuable suggestions during the initial phase of this thesis Erik Wiborg and Bjarte Aune Bergstrøm for correcting my spelling errors and for giving me valuable feedback Sara and Jon Noé Høye for being my supportive and lovely family v Table of Contents Part I - Introduction and Background 1 1 Introduction...............................................................................................................................3 2 Motivation.................................................................................................................................5 2.1 Evolutionary Algorithms...................................................................................................5 2.1.1 Application Areas.......................................................................................................5 2.1.2 Problems....................................................................................................................5 2.2 The Scalability Problem....................................................................................................6 2.2.1 The Need For Scaling................................................................................................6 2.2.2 Approaches Towards Overcoming the Scalability Limitation...................................7 2.3 Development – The Way Nature Copes With Scaling.......................................................8 2.3.1 Scalability..................................................................................................................8 2.3.2 Flexibility...................................................................................................................8 2.3.3 Robustness.................................................................................................................9 2.4 Motivating Work...............................................................................................................9 3 Genetic Programming.............................................................................................................11 3.1 The Components..............................................................................................................11 3.2 The Mechanisms of Evolution........................................................................................12 3.2.1 Reproduction...........................................................................................................12 3.2.2 Crossover.................................................................................................................12 3.2.3 Mutation...................................................................................................................14 3.2.4 Fitness Function.......................................................................................................16 3.2.5 Selection Mechanism...............................................................................................16 3.3 The Algorithm.................................................................................................................16 4 Biological Development..........................................................................................................19 4.1 Overview.........................................................................................................................19 4.1.1 The DNA.................................................................................................................20 4.1.2 The Protein...............................................................................................................22 4.1.3 The Cell...................................................................................................................24 4.2 Protein Synthesis - From DNA to Protein.......................................................................28 4.2.1 Gene Expression......................................................................................................28 4.2.2 Control of Gene Expression....................................................................................32 4.3 Transcriptional Control....................................................................................................34 4.4 Development Mechanisms..............................................................................................36 4.4.1 Cell Growth – Division of Cells..............................................................................36 4.5 The Cell Cycle.................................................................................................................36 4.5.1 Differentiation..........................................................................................................38 4.5.2 Morphogenesis.........................................................................................................38