View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by The IT University of Copenhagen's Repository CA-NEAT: Evolved Compositional Pattern Producing Networks for Cellular Automata Morphogenesis and Replication Stefano Nichele∗y, Mathias Berild Osey, Sebastian Risiz, Gunnar Tuftey ∗Oslo and Akershus University College of Applied Sciences, Oslo, Norway [email protected] yNorwegian University of Science and Technology, Trondheim, Norway fmathiabo@stud, [email protected] zIT University of Copenhagen, Copenhagen, Denmark [email protected] Abstract—Cellular Automata (CA) are a remarkable example found in nature can be intractable, e.g. a cellular au- of morphogenetic system, where cells grow and self-organise tomata system with hundreds of cellular states and large through local interactions. CA have been used as abstractions neighbourhoods or a deep neural network with millions of of biological development and artificial life. Such systems nodes and weights. An appropriate mapping which scales have been able to show properties that are often desirable but well and at the same time allows solutions to be evolved difficult to achieve in engineered systems, e.g. morphogenesis incrementally, starting with a solution encoded into a and replication of regular patterns without any form of small genome that is gradually complexified by adding centralised coordination. However, cellular systems are hard new degrees of freedom, is desired. However, exploring a to program (i.e. evolve) and control, especially when the high-dimensional space in search for a solution can take number of cell states and neighbourhood increase. prohibitively long time, regardless of the encoding [4]. In this paper, we propose a new principle of mor- In this work a cellular system is used as test bed of phogenesis based on Compositional Pattern Producing Net- morphogenetic engineering. A traditional CA table-based works (CPPNs), an abstraction of development that has encoding is replaced by a Compositional Pattern Produc- been able to produce complex structural motifs without ing Networks (CPPNs) mapping, a developmental encod- local interactions. CPPNs are used as Cellular Automata ing often used in systems without local interactions. In genotypes and evolved with a NeuroEvolution of Augmenting our work CPPN is used as developmental encoding based Topologies (NEAT) algorithm. This allows complexification on local interactions, i.e. a true morphogenetic cellular system. The cellular automata CPPNs are evolved through of genomes throughout evolution with phenotypes emerging a NeuroEvolution of Augmenting Topologies (NEAT) al- from self-organisation through development based on local gorithm, a method that evolves increasingly complex net- interactions. In this paper, the problems of 2D pattern works. The approach is termed CA-NEAT. All cells in morphogenesis and replication are investigated. Results show the systems are uniform, i.e. they share the same genome that CA-NEAT is an appropriate means of approaching network. Two benchmark problems are investigated: 2D cellular systems engineering, especially for future applica- morphogenesis and replication of structures of increasing tions where natural levels of complexity are targeted. We complexities. argue that CA-NEAT could provide a valuable mapping for The paper is organised as follows: Section 2 gives morphogenetic systems, beyond cellular automata systems, background information on CA, CPPNs and NEAT, to- where development through local interactions is desired. gether with a motivation for using them together in a Index Terms—Cellular Automata, Compositional Pattern morphogenetic engineering system. Section 3 describes Producing Network (CPPN), NeuroEvolution of Augmenting the investigated problems and the experimental setup. Sec- Topologies (NEAT), EvoDevo, Artificial Life. tion 4 presents the results of the experiments, which are further discussed in Section 5. Finally, Section 6 describes some possible directions for future work and Section 7 1. Introduction concludes the paper. Appendix A contains some graph- ical visualisations of the solutions found in the outlined Complex self-architecturing systems are difficult to experiments. program, i.e. by top-down engineering. Kowaliw and Banzhaf [1], [2] argue that the bottom-up methodol- 2. Background and Motivation ogy of artificial development is an appropriate means of approaching complex systems engineering. However, achieving some sort of self-architecturing properties, e.g. 2.1. Cellular Automata morphogenesis or self-replication, is not trivial. One way of ”programming” such developmental systems is through Cellular Automata (CA) were first studied in the artificial evolution, i.e. an evolutionary and developmental 1940s by von Neumann and Ulam [5]. CA were inspired approach (EvoDevo) [3]. Searching for a solution for an by biological organisms, and introduced as models that artificial EvoDevo system that targets levels of complexity could emulate some of these organisms’ interesting and remarkable properties, such as multi-cellular development (e.g. embryogenesis), reproduction (clonal or sexual) and robustness (e.g. self-repair). CA have been extensively studied, in particular within the field of artificial life [6]. Michael J. Flynn has spec- ulated that CA and cellular computing might be the path forward [7] for novel bio-inspired computational machines. Matthew Cook proved that a CA of a certain type, e.g. Rule 110, can be Turing complete [8]. A CA consists of a grid of very simple units called cells. A cell can be in one out of a finite set of states. Sip- per [9] described the three core principles of the cellular computing paradigm: • Simplicity: a cell is simple and can do very little by itself. (a) (b) • Vast Parallelism: the number of cells is very large, much larger than the number of processors Figure 1: An example composition of the sigmoid, si- in a conventional parallel computer. nusoid and hyperbolic tangent functions. The discrete coordinates of (b) are first normalised to [−1:0; 1:0] and • Locality: all interactions between cells take place on a purely local basis. No cell knows or controls then mapped to various output values through the CPPN the entire system. (a). The cells in a CA use the information available from their neighbours and a set of rules to transition from one 2.2. CPPNs state to the next, i.e. CA transition function. Depending on the starting state of the whole system and the transition Artificial Neural Networks (ANNs) [13, Chapter 1] function, it is possible to observe interesting emergent have been used in many different applications related to or self-organising behaviour over time and space, e.g. artificial life and intelligence, such as robotics or machine ordered, periodic, chaotic patterns. These interesting CA learning. An ANN is a directed graph structure, with often enter an attractor [10], [11]. If a sequence of states vertices (referred to as neurons) and edges (referred to repeats periodically it is referred to as cyclic attractor, as connections). This is inspired by neuroscience, with and if the CA stabilises into a permanent state it is called the brain consisting of neurons and synapse connections. a point attractor. ANNs are useful because they consists of many discrete parts that can be individually or collectively tuned by 2.1.1. Transition Functions. Langton [6] formally de- some adaptive process, and are easily expanded. The scribes finite CA as consisting of a finite set of cell universal approximation theorem [14] states that relatively Σ K = jΣj α states of size , a finite input alphabet , simple ANNs can approximate a wide variety of functions, ∆ N and a transition function . Each cell has a -sized and the field of deep learning [15] shows that a large neighbourhood. The number of possible neighbourhoods complex structure with enough tuning can perform very can be expressed by equation (1). complex tasks, such as image classification [16] or natural language processing [17]. A Compositional Pattern Pro- jαj = jΣN j = KN (1) ducing Network (CPPN), which was introduced by Stanley The transition function for a CA must thus encode in 2007 [18], is a special type of ANN that is employed a mapping of jαj different inputs to one of K states. as an artificial development encoding. Like an ANN, a The number of possible unique transition functions is thus CPPN consists of a set of nodes with activation functions, N K(K ). weights and biases, as well as weighted connections be- Traditionally ∆ has been encoded as a complete map- tween nodes. Likewise, external values are input to the ping ∆ : ΣN ! Σ, which can be implemented as a first layer, then undergo transformation by weights and lookup table. When working with non-trivial CA where activation functions before being output by the final layer. both K and N can be relatively large numbers, it becomes In contrast to ANNs, which are usually structured a problem to store the mapping ∆ in an efficient way, and with neurons of the same activation functions arranged the space of possible ∆ becomes too large to be explored in layers, the CPPN has few such restrictions on topology by exhaustive enumeration. and layer-wise heterogeneity and often employs a vari- Elementary CA transition functions have been studied ety of different activation functions. Different activation extensively, and it has been reported that most rules lead functions are included to capture specific patterns seen in to ”uninteresting” behaviour, either falling into an ”order” natural development, such as a Gaussian function to create which is either static or repeating periodically, or into symmetric patterns, or sine functions to create repeating chaos, where all useful information is quickly lost in noise. patterns. It has been speculated that it is in the critical border region Additionally, CPPNs are normally applied across a between these behaviours where interesting computations broader range of possible inputs than ANNs. For example, can occur [6]. In order to find these ”interesting” ∆, smart Figure 1 shows a CPPN and its output when mapped heuristic searches are often applied [12], e.g.