1 Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks Andrea Soltoggio, Kenneth O. Stanley, Sebastian Risi Abstract—Biological neural networks are systems of extraor- is to simulate natural evolution in-silico, as in the field of dinary computational capabilities shaped by evolution, develop- evolutionary computation (Holland, 1975; Eiben and Smith, ment, and lifelong learning. The interplay of these elements leads 2015). Sub-fields of evolutionary computation such as evo- to the emergence of biological intelligence. Inspired by such intricate natural phenomena, Evolved Plastic Artificial Neural lutionary robotics (Harvey et al., 1997; Nolfi and Floreano, Networks (EPANNs) employ simulated evolution in-silico to breed 2000), learning classifier systems (Lanzi et al., 2003; Butz, plastic neural networks with the aim to autonomously design and 2015), and neuroevolution (Yao, 1999) specifically research create learning systems. EPANN experiments evolve networks algorithms that, by exploiting artificial evolution of physical, that include both innate properties and the ability to change computational, and neural models, seek to discover principles and learn in response to experiences in different environments and problem domains. EPANNs’ aims include autonomously behind intelligent and learning systems. creating learning systems, bootstrapping learning from scratch, In the past, research in evolutionary computation, particu- recovering performance in unseen conditions, testing the com- larly in the area of neuroevolution, was predominantly focused putational advantages of particular neural components, and de- on the evolution of static systems or networks with fixed riving hypotheses on the emergence of biological learning. Thus, neural weights: evolution was seen as an alternative to learning EPANNs may include a large variety of different neuron types and dynamics, network architectures, plasticity rules, and other rules to search for optimal weights in an artificial neural factors. While EPANNs have seen considerable progress over the network (ANN). Also, in traditional and deep ANNs, learning last two decades, current scientific and technological advances in is often performed during an initial training phase, so that artificial neural networks are setting the conditions for radically weights are static when the network is deployed. Recently, new approaches and results. Exploiting the increased availability however, inspiration has originated more strongly from the of computational resources and of simulation environments, the often challenging task of hand-designing learning neural fact that intelligence in biological organisms considerably networks could be replaced by more autonomous and creative relies on powerful and general learning algorithms, designed processes. This paper brings together a variety of inspiring by evolution, that are executed during both development and ideas that define the field of EPANNs. The main methods and continuously throughout life. results are reviewed. Finally, new opportunities and possible As a consequence, the field of neuroevolution is now pro- developments are presented. gressively moving towards the design and evolution of lifelong Index Terms—Artificial Neural Networks, Lifelong Learning, learning plastic neural systems, capable of discovering learn- Plasticity, Evolutionary Computation. ing principles during evolution, and thereby able to acquire I. INTRODUCTION knowledge and skills through the interaction with the envi- ronment (Coleman and Blair, 2012). This paper reviews and Over the course of millions of years, evolution has led to the organizes the field that studies evolved plastic artificial neural emergence of innumerable biological systems, and intelligence networks, and introduces the acronym EPANN. EPANNs are itself, crowned by the evolution of the human brain. Evolution, evolved because parts of their design are determined by an development, and learning are the fundamental processes that arXiv:1703.10371v3 [cs.NE] 8 Aug 2018 evolutionary algorithm; they are plastic because parts of their underpin biological intelligence. Thus, it is no surprise that structures or functions, e.g. the connectivity among neurons, scientists have tried to engineer artificial systems to reproduce change at various time scales while experiencing sensory- such phenomena (Sanchez et al., 1996; Sipper et al., 1997; motor information streams. The final capabilities of such Dawkins, 2003). The fields of artificial intelligence (AI) and networks are autonomously determined by the combination artificial life (AL) (Langton, 1997) are inspired by nature and of evolved genetic instructions and learning that takes place biology in their attempt to create intelligence and forms of as the network interacts with an environment. life from human-designed computation: the main idea is to EPANNs’ ambitious motivations and aims, centered on the abstract the principles from the medium, i.e., biology, and autonomous discovery and design of learning systems, also utilize such principles to devise algorithms and devices that entail a number of research problems. One problem is how reproduce properties of their biological counterparts. to set up evolutionary experiments that can discover learning, One possible way to design complex and intelligent sys- and then to understand the subsequent interaction of dynamics tems, compatible with our natural and evolutionary history, across the evolutionary and learning dimensions. A second Department of Computer Science, Loughborough University, LE11 3TU, open question concerns the appropriate neural model abstrac- Loughborough, UK, [email protected] tions that may capture essential computational principles to Department of Computer Science, University of Central Florida, Orlando, FL, USA, [email protected] enable learning and, more generally, intelligence. One further IT University of Copenhagen, Copenhagen, Denmark, [email protected] problem is the size of very large search spaces, and the high 2 computational cost required to simulate even simple models determine the ultimate capabilities of complex biological of lifelong learning and evolution. Finally, experiments to neural systems (Deary et al., 2009; Hopkins et al., 2014): autonomously discover intelligent learning systems have a different animal species manifest different levels of skills wide range of performance metrics, as their objectives are and intelligence because of their different genetic blueprint sometimes loosely defined as the increase of behavioral com- (Schreiweis et al., 2014). The intricate structure of the brain plexity, intelligence, adaptability, evolvability (Miconi, 2008), emerges from one single zygote cell through a developmental and general learning capabilities (Tonelli and Mouret, 2011). process (Kolb and Gibb, 2011), which is also strongly affected Thus, EPANNs explore a larger search space, and address by input-output learning experiences throughout early life broader research questions, than machine learning algorithms (Hensch et al., 1998; Kolb and Gibb, 2011). Yet high levels specifically designed to improve performance on well-defined of plasticity are maintained throughout the entire lifespan and narrow problems. (Merzenich et al., 1984; Kiyota, 2017). These dimensions, The power of EPANNs, however, derives from two au- evolution, development and learning, also known as the phylo- tonomous search processes: evolution and learning, which genetic (evolution), ontogenetic (development) and epigenetic arguably place them among the most advanced AI and machine (learning) (POE) dimensions (Sipper et al., 1997), are essential learning systems in terms of open-endedness, autonomy, po- for the emergence of biological plastic brains. tential for discovery, creativity, and human-free design. These The POE dimensions lead to a number of research ques- systems rely the least on pre-programmed instructions because tions. Can artificial intelligence systems be entirely engineered they are designed to autonomously evolve while interacting by humans, or do they need to undergo a less human-controlled with a real or simulated world. Plastic networks, in particular process such as evolution? Do intelligent systems need to recurrent plastic networks, are known for their computational learn, or could they be born already knowing? Is there an power (Cabessa and Siegelmann, 2014): evolution can be a optimal balance between innate and acquired knowledge? valuable tool to explore the power of those computational Opinions and approaches are diverse. Additionally, artificial structures. systems do not need to implement the same constraints and In recent years, progress in a number of relevant areas has limitations as biological systems (Bullinaria, 2003). Thus, set the stage for renewed advancements of EPANNs: ANNs, inspiration is not simple imitation. in particular deep networks, are becoming increasingly more EPANNs assume that both evolution and learning, if not successful and popular; there has been a remarkable increase strictly necessary, are conducive to the emergence of a strongly in available computational power by means of parallel GPU bio-inspired artificial intelligence. While artificial evolution is computing and dedicated hardware; a better understanding of justified by the remarkable achievements of natural evolution, search, complexity, and evolutionary computation allows for the role of learning has gathered significance in recent years. less naive approaches; and finally,