Digital Control Networks for Virtual Creatures
Total Page:16
File Type:pdf, Size:1020Kb
Digital control networks for virtual creatures Christopher James Bainbridge Doctor of Philosophy School of Informatics University of Edinburgh 2010 Abstract Robot control systems evolved with genetic algorithms traditionally take the form of floating-point neural network models. This thesis proposes that digital control sys- tems, such as quantised neural networks and logical networks, may also be used for the task of robot control. The inspiration for this is the observation that the dynamics of discrete networks may contain cyclic attractors which generate rhythmic behaviour, and that rhythmic behaviour underlies the central pattern generators which drive low- level motor activity in the biological world. To investigate this a series of experiments were carried out in a simulated physically realistic 3D world. The performance of evolved controllers was evaluated on two well known control tasks — pole balancing, and locomotion of evolved morphologies. The performance of evolved digital controllers was compared to evolved floating-point neu- ral networks. The results show that the digital implementations are competitive with floating-point designs on both of the benchmark problems. In addition, the first re- ported evolution from scratch of a biped walker is presented, demonstrating that when all parameters are left open to evolutionary optimisation complex behaviour can result from simple components. iii Acknowledgements “I know why you’re here... I know what you’ve been doing... why you hardly sleep, why you live alone, and why night after night, you sit by your computer.” I would like to thank my parents and grandmother for all of their support over the years, and for giving me the freedom to pursue my interests from an early age. My time at Edinburgh would not have been so enjoyable without my friends and flatmates; in particular I must thank J.D. for his friendship and kindred humour, Big C. for the nights of Teviot and snakebite, Ljiljana for her remarkable ability to share a room with me for three years and not go insane, Javier, Sasha and Jude for the similar skills of flat sharing, Nicki, Evie and Marian for being there, and everyone in the university ju-jitsu club for providing years of social interactions, the complexity of which, when graphed, might put a neural network to shame. From my life before Edinburgh: Frank, John, Ian and Brian for the crazy nights in Consett, including the impromptu ju-jitsu demonstration that almost got us arrested. And from a time long past: the inhabitants of the DnC BBS who gave me the inspiration and motivation to sit every night playing with code, networks, and Unix. And to my younger brother, Paul, who as he points out, has had a job for a long time now... Without various pieces of software this project would not have been possible. I would like to thank the authors of the following open source systems which I use every day: Linux, the Open Dynamics Engine (ODE) and PyODE, Screen, KDE and KPDF/Okular, R, Python, teTeX, Vim, ZODB/ZEO, Git, Firefox, and MPlayer. This thesis was greatly improved thanks to valuable feedback from Gillian Hayes and Susan Stepney. iv Declaration I declare that this thesis was composed by myself, that the work contained herein is my own except where explicitly stated otherwise in the text, and that this work has not been submitted for any other degree or professional qualification except as specified. (Christopher James Bainbridge) v Table of Contents 1 Introduction 1 1.1 Motivation . 1 1.2 Contributions to knowledge . 7 1.3 Reproducibility . 11 1.4 Summary . 12 1.5 Organisation of the thesis . 14 2 Biological sensing and control 17 2.1 Neural networks . 17 2.2 Biological models . 22 2.3 Connectivity and resilience . 22 2.4 Pattern generation for muscle control . 28 2.5 Sensory feedback . 31 2.6 Brain computer interfaces . 32 2.7 Summary . 35 3 Synthetic neural networks 39 3.1 Similarities to biological networks . 42 3.2 Connectivity . 43 3.3 State and signal coding . 44 3.3.1 Example codings . 46 3.4 Models of single neuron dynamics . 49 3.4.1 Input function . 49 3.4.2 Activation function . 50 3.4.3 Output function . 50 3.4.4 Spiking models . 52 3.4.5 Continuous models . 57 vii 3.4.6 Reduced models . 61 3.5 Computational power . 75 3.6 Robot control . 76 3.7 Training . 76 3.7.1 Backpropagation . 77 3.8 Hardware acceleration . 79 3.9 Summary . 82 4 Other networks 83 4.1 Analog circuits . 84 4.2 Digital circuits . 85 4.3 Asynchronous circuits . 86 4.4 Genetic regulatory networks . 89 4.5 Boolean networks . 94 4.5.1 Random boolean networks . 96 4.6 Generalised logical networks . 102 4.7 Cellular automata . 106 4.8 The edge of chaos . 110 4.9 Summary . 111 5 Genetic evolution 115 5.1 Natural selection . 116 5.2 Genetics in nature . 117 5.3 Genetic algorithms . 124 5.4 Modularity of the genome . 126 5.5 Fitness function . 127 5.6 Fitness surface . 128 5.7 Search behaviour . 131 5.8 Genotype encoding . 132 5.8.1 Example encodings . 134 5.9 Population models . 135 5.10 Initial population creation . 136 5.11 Parent selection . 136 5.12 Reproduction . 138 5.13 Summary . 140 viii 6 Evolution of specific genotypes 143 6.1 Problem specific operators . 144 6.2 Evolving flocking . 146 6.3 Ant colonies . 148 6.4 Evolving neural networks for robot control . 150 6.4.1 Timeline . 152 6.5 Evolving reduced continuous neural models . 157 6.6 Evolving reduced spiking neural models . 158 6.7 Evolving modular hierarchical neural networks . 158 6.8 Evolving genetic regulatory networks . 161 6.9 Evolving cellular automata . 161 6.9.1 Evolving cellular automata neural networks . 164 6.10 Evolving analog circuits . 168 6.10.1 Evolving FPAA robot control . 168 6.10.2 Evolving FPGA oscillators . 169 6.10.3 Evolving FPGA frequency discriminators . 169 6.10.4 Evolving on an analog FPTA . 170 6.10.5 Evolving fault tolerance . 171 6.11 Evolving digital circuits . 172 6.11.1 Optimising gate count . 173 6.11.2 Optimising power . 174 6.11.3 Evolving digital circuits for robot control . 176 6.12 Evolving pattern generators . 176 6.12.1 Timeline . 177 6.13 Evolving morphology . 180 6.13.1 Timeline . 181 6.14 Evolving robot morphology and control . 193 6.14.1 Timeline . 194 6.15 Evolving modular robots . 211 6.16 Summary . 212 7 Overview so far 217 8 Software design 219 8.1 Creature morphology . 219 8.2 Morphogenesis . 221 ix 8.3 Evolution of the morphology . 223 8.4 Example morphologies . 224 8.5 Neurogenesis . 225 8.6 Neural network topologies . 233 8.7 Neuron quantisation . 236 8.8 Evolution of the neural networks . 237 8.9 The software . 238 8.10 Simulated physics . 244 8.10.1 Collision detection . 245 8.11 Sensors and actuators . 246 8.12 Motor models . 246 8.12.1 Stimulus-response curves of PD motor controller . 247 8.13 Physics simulation problems . 250 8.14 Tasks . 256 8.15 Summary . 256 9 Software testing 257 9.1 Testing neuron models . 259 9.1.1 Explanation of graphs . 260 9.1.2 Sine model . 262 9.1.3 Sigmoid model . ..