Building processes optimization : Toward an artificial ontogeny based approach Alexandre Devert Directed by Marc Schoenauer and Nicolas Bred`eche Submitted in total fulfilment of the requirements of the degree of Doctor of Philosophy June 21, 2009 Ecole Doctorale d’Informatique Universit´eParis-Sud 11 Contents 1 Introduction to Evolutionary Design 5 1.1 Anevolutionaryalgorithmsprimer . 7 1.1.1 Description .......................... 8 1.1.2 Lamarckian and Baldwinian evolutions . 9 1.1.3 Onthe designof anevolutionaryalgorithm . 10 1.2 Simulationandevaluation . 11 1.2.1 Designofanevaluationfunction . 13 1.2.2 Theusageofsimulation . 14 1.3 Representations for Evolutionary Design . .. 15 1.3.1 Blockstacks.......................... 15 1.3.2 Explicit and implicit representations . 18 1.3.3 Modularity .......................... 19 1.3.4 Treestructures .. .. .. .. .. .. .. .. .. .. 21 1.3.5 Grammarsandrewritingsystems . 23 1.3.6 Theevolvabilityissue . 26 1.3.7 Cellularrepresentations . 26 1.4 Preview ................................ 29 2 AguidedtourofEvolutionaryDesign 31 2.1 Explicitrepresentations . 31 2.1.1 Blockstacks.......................... 31 2.1.2 Legostructures.. .. .. .. .. .. .. .. .. .. 32 2.1.3 Antennae ........................... 33 2.1.4 Shapes............................. 34 2.1.5 Robots............................. 35 2.1.6 Tensegritystructures. 36 2.1.7 TopologicalOptimumDesign . 37 2.1.8 Opticalfibersdesign . 38 2.1.9 Bladeshapedesignforwindturbines. 39 2.2 Implicitrepresentations . 40 2.2.1 KarlSims’creatures . 40 2.2.2 Hornby’sL-systems . 41 2.2.3 EnvironmentawareL-systems. 44 2.3 Cellularrepresentations . 45 2.3.1 Virtualcreatures . 46 2.3.2 Skyscraperframes . 47 2.3.3 Trussstructures . .. .. .. .. .. .. .. .. .. 48 2.3.4 Environmentawareness . 50 2.4 Conclusion .............................. 51 1 CONTENTS 3 An Evolutionary algorithm for blueprints 55 3.1 TheBlindBuilderrepresentation . 55 3.1.1 Variationoperators. 57 3.2 Designofsimple3dobjects . 57 3.2.1 Experimental setup and hierarchical fitness . 58 3.2.2 ThePillarexperiment . 59 3.2.3 TheBridgeexperiment . 60 3.3 Anempiricalanalysis. 62 3.3.1 Experimentalsettings . 64 3.3.2 ExperimentalProtocol. 65 3.3.3 Results ............................ 66 3.4 Discussion............................... 71 3.5 Conclusion .............................. 72 4 Embryogenic flags 75 4.1 Introductiontoembryogenicflags. 76 4.2 An insight of the embryogenic flag problem . 78 4.2.1 DirectSolutions .. .. .. .. .. .. .. .. .. .. 79 4.2.2 Non-Developmental Solutions . 80 4.2.3 Agenericdevelopmentalsolution . 80 4.2.4 Reaction-diffusionsystems. 81 4.2.5 Towards non-trivial developmental solutions . .. 83 4.3 SurveyoftheEmbryogenicFlags . 83 4.3.1 UpdateFunctionModels. 84 4.3.2 CellCommunications . 84 4.3.3 NeighbourhoodsandInputs . 85 4.3.4 Synchronous and Asynchronous Updates . 86 4.3.5 InitialSetup ......................... 86 4.3.6 Birth,GrowthandDeath . 86 4.3.7 StoppingCriterion . 87 4.3.8 Self-RepairingProperties . 88 4.3.9 RelevancetoEvolutionaryDesign . 88 4.4 Methodology ............................. 90 4.4.1 TheTargetPatterns . 90 4.4.2 Stoppingcriterion . 92 4.4.3 Patterndistance .. .. .. .. .. .. .. .. .. .. 93 4.4.4 Fitnessfunction .. .. .. .. .. .. .. .. .. .. 94 4.4.5 Steadystatestability. 94 4.4.6 Parametricoptimization . 94 4.5 Neural networks as update functions: NEAT . 95 4.5.1 Themodel .......................... 95 4.5.2 ControllerOptimization . 96 4.5.3 ExperimentalSetup . 97 4.5.4 Resultsanddiscussion . 98 4.6 Reservoir computing for update functions . 104 4.6.1 EchoStateNetworks. 104 2 CONTENTS 4.6.2 Controlleroptimization . 105 4.6.3 Resultsanddiscussion . 106 4.7 Adiffusionbasedapproach . 106 4.7.1 Themodel .......................... 108 4.7.2 ControllerOptimization . 109 4.7.3 Resultsanddiscussion . 109 4.8 Developmentandstability . 115 4.8.1 Experimentalsetup. 115 4.8.2 Resultsanddiscussion . 115 4.9 Conclusion .............................. 120 5 A cellular development model for structural truss design 125 5.1 Structuraltrusses. 125 5.1.1 OptimizationofTrussStructures . 127 5.2 EiffelBlob ............................... 127 5.2.1 ControlTopologies . 128 5.2.2 Controlandinformationdiffusion. 129 5.2.3 Geometrycontrol. 130 5.2.4 Mechanicalfeedback . 131 5.2.5 Initializationandhalting . 131 5.3 Experimentations. 132 5.3.1 Optimization of a developement process . 134 5.3.2 Comparison with the Direct Representation . 135 5.3.3 Propertiesof the Developement Process . 138 5.4 Conclusion .............................. 139 6 Discussion and Conclusion 143 6.1 Contributions ............................. 143 6.1.1 Representations for Evolutionary Design . 143 6.1.2 ExplicitBuildingProcesses . 143 6.1.3 EmbryogenicFlags . 144 6.1.4 Optimizing Cellular Development Processes . 145 6.1.5 StoppingCriterion . 146 6.1.6 A CellularRepresentationforTrusses . 146 6.1.7 Summary ........................... 147 6.2 Perspectives for cellular representations . 147 6.2.1 CurrentIssues .. .. .. .. .. .. .. .. .. .. 148 6.2.2 FutureDirections. 149 A The Springy Comb 151 A.1 Description .............................. 151 A.2 Implementation. .. .. .. .. .. .. .. .. .. .. .. 153 B Blocks stacks physics 155 B.1 Themodel............................... 155 B.2 Force,equilibriumandbalance . 155 3 CONTENTS C NEAT 157 C.1 Therepresentation . 157 C.2 Thevariationoperators . 157 C.2.1 Mutations........................... 158 C.2.2 Crossover ........................... 159 C.3 Thegenerationaloperator . 159 C.4 Issues .................................159 4 A long time ago, in a galaxy far, far away . Star Wars Episode IV: A New Hope 1 Introduction to Evolutionary Design Most of the living beings, without predators or catastrophic events like epidemic, would see their populations growing at an exponential rate. But those beings rely on the same limited resources, which can not sustain exponentially growing populations for a long time. Thus, there is a constant struggle for the vital resources. In addition, living beings usually breed individuals very similar to themselves, but with some tiny variations. Some of those variations might be able to give a slight advantage for access to the vital resources. Then those variations are likely to spread quickly and become a definitive character of a specie. This is a very abridged introduction to the natural selection theory of Charles R. Darwin, ignoring various biological facts, like the genetic material inner functioning, the role of the environment geography, . Such a summed-up version of the natural evolution is quite general, and we can replace beings by machines without much transgression to logic. In 2008, it would be yet a bit too soon to talk about self-reproducing autonomous machines like a mastered fact. But we can imagine without any difficulties a huge room filled by engineers. Each engineer conceives a plane for a predefined purpose. Those planes are candidate solutions. Once the conception is done, some prototypes of the corresponding planes are built. The prototypes are tested by test pilots, their designs are reviewed by an expert commission, the productions costs are estimated, etc. The candidates solutions are evaluated during an evaluation stage. And finally, amongst the plane designs from the engineers room, a few plane designs might be adopted and produced: this is the selection stage. An exceptional, versatile and cheap plane design is likely to be produced in massive quantities, while an average, specialized and expensive design will be produced in little quantities. The planes design evolved with a striking resemblance with natural evolution. Such a caricature view, rooms filled of engineers drawing planes, actually existed during the Second World War. In Germany, there was a high pres- sure to compensate for the lack of resources and industrial strength by cut- ting edge technologies. Combined to the frantic desperation and the fury of a darned regime, this generated an explosion of planes designs. Drastic amounts of innovative, groundbreaking designs were proposed, selected an tested at that time. The invention, the evaluation and the selection stages were performed in 5 CHAPTER 1. INTRODUCTION TO EVOLUTIONARY DESIGN Figure 1.1: The Triebfl¨ugel concept. It was designed to takeoff vertically, on rough airfields, using low quality fuel. The three wings would rotate around the plane body, propelled by a compact ramjet at their tip. Although the Pabst ramjet was tested, this design never made its way to production. Horten Ho 229 Heinkel He 162 Volksj¨ager Figure 1.2: Two German aircraft of the Second World War, amongst the less loose concepts. At the end of the war, the left one was about to make its first flight, while the mass production of the one at the right side was beginning. Hundreds of even more daring designs were studied as well, but sadly, good pictures of them are quite hard to find. 6 1.1. AN EVOLUTIONARY ALGORITHMS PRIMER very short cycles. German engineers of World War Two devised inverted arrow wings, delta wings, jet engine or rockets propelled flying wings 1, composite planes, suborbital bombers 2, asymmetric planes 3, variable geometry planes 4, or even aberrations like the Triebfl¨ugel design (showed on figure 1.1). An exploding, never quite equaled variety of very original designs came out from the German aircraft manufacturers. Few of those Germans designs made their way to the mass production and the battlefield, but the more reasonable ones are everywhere now: jet engine powered delta wings and inverted arrow wings, ...Indeed, most