Evolving Neural Networks Using Behavioural Genetic Principles Maitrei Kohli A dissertation submitted in partial fulfilment of the requirements for the degree of Doctor of Philosophy Department of Computer Science & Information Systems Birkbeck, University of London United Kingdom March 2017 0 Declaration This thesis is the result of my own work, except where explicitly acknowledged in the text. Maitrei Kohli …………………. 1 ABSTRACT Neuroevolution is a nature-inspired approach for creating artificial intelligence. Its main objective is to evolve artificial neural networks (ANNs) that are capable of exhibiting intelligent behaviours. It is a widely researched field with numerous successful methods and applications. However, despite its success, there are still open research questions and notable limitations. These include the challenge of scaling neuroevolution to evolve cognitive behaviours, evolving ANNs capable of adapting online and learning from previously acquired knowledge, as well as understanding and synthesising the evolutionary pressures that lead to high-level intelligence. This thesis presents a new perspective on the evolution of ANNs that exhibit intelligent behaviours. The novel neuroevolutionary approach presented in this thesis is based on the principles of behavioural genetics (BG). It evolves ANNs’ ‘general ability to learn’, combining evolution and ontogenetic adaptation within a single framework. The ‘general ability to learn’ was modelled by the interaction of artificial genes, encoding the intrinsic properties of the ANNs, and the environment, captured by a combination of filtered training datasets and stochastic initialisation weights of the ANNs. Genes shape and constrain learning whereas the environment provides the learning bias; together, they provide the ability of the ANN to acquire a particular task. Ontogenetic adaptation was implemented via a local gradient-based search method, while phylogenetic evolution was implemented via a Darwinian, fitness-based selection approach. The project was structured as follows. Chapter 2 presents the novel neuro-evolutionary approach for evolving populations of neural networks, inspired from BG principles. In chapter 3, the framework is applied to an exemplar problem domain drawn from psychology, that of English past-tense acquisition within the field of child language development. This domain is notable because it is a quasi-regular or dual-natured task. Chapter 3 also introduces the analytical technique of assessing the ‘heritability’ of performance in a population of ANNs. Populations are created comprising identical and non-identical ‘twins’, so specified by the similarity of their artificial genomes. Heritability provides a scalable summary statistic of the net effect of all internal parameters on learning. However, it can only be quantified when variable also exists in the quality of the environment. The findings from the experimental evaluations demonstrated the effectiveness of the model and provided a basis to extend it to capture population-level differences within developmental settings. In the second half of the project, the framework was extended to model transfer learning, with a special focus on heterogeneous tasks. This simulated the neuroevolutionary scenario wherein population members can be required to learn tasks different from those for which they were selected. Chapter 4 lays out the theoretical issues in this field. Large-scale simulations, involving over 200,000 networks, then identified and tested two key factors that modulated the performance of the transfer model – the type of selection operator (chapter 5) and the nature of source task (chapter 6). By transferring the ‘general ability to learn’, the transfer model enabled a population of ANNs to acquire successfully five different heterogeneous tasks. Analyses of heritability and environmentability were utilised to reveal which factors were most responsible for variation in performance, and its improvement across generations. Crucially, these large-scale experiments demonstrated that it is possible for a population of ANNs to acquire multiple heterogeneous tasks, and that the heritability metric can be utilised to identify when negative transfer effects may occur. As discussed in chapter 7, the BG-inspired method therefore presents concrete progress in optimising those neurocomputational properties of ANNs relevant to enhance learning across multiple problem domains. 2 Acknowledgements I am truly grateful to all the people who supported me through this incredible journey. My sincerest thanks to My parents, you are my biggest strength and inspiration. I am so grateful for your unconditional love, prayers, and unwavering belief in me. Most of all, I am extremely grateful for everything you do for me. I dedicate this work to you. My brother, Jivak, for your constant support, encouragement, funny banter and for always being there for me. Thanks for being the best brother ever. My grandma, for loving me so much and for endless blessings. My best friend, Areej, through good times and bad you have been there for me always. Thank you for everything. Mrs. La Young Jackson, for guiding and helping me through visa processes numerous times. Ms. Kristina Freris, for advising and encouraging me when things got tough and overwhelming. I am really thankful. Oriental bank of commerce, for giving me education loan and thereby making finances easier for me. Dept. of Computer Science, Birkbeck, for awarding me with 3-year BEI studentship. Systems Group, for helping me numerous times with Matlab and Condor. Members of Developmental Neuro-cognition Lab, for giving me several opportunities to present my work and for all your constructive feedback and suggestions. Thanks to Prof. Michael Thomas for making me a member of DNL. My co-supervisor, Prof. Michael Thomas, for your guidance, encouragement, valuable feedback, hard-work and patience with me. You have taught me to always strive for perfection and to keep the goals and standards high. Working with you has been a great privilege and I will always be grateful to you. Finally, my supervisor and mentor, Prof. George Magoulas. Words cannot describe how grateful I am to you for all the invaluable advise, support, guidance, encouragement, feedback, hard-work and patience with me. I have learned so much from you and it has been a great honour to work with you. Thank you for not only being my supervisor but also the perfect role- model. I will always be earnestly grateful to you. 3 Table of Contents 1. Introduction 10 1.1 Research Questions 12 1.2 Methodology 13 1.3 Thesis Structure and Contribution 16 2. Behavioural Genetics inspired framework for evolving populations of neural networks: combining learning & evolution 2.1 Overview 21 2.2 Evolution and Learning and interactions therein 21 2.3 Combining Evolution and Learning using ANNs 25 2.3.1 Frameworks for combining Evolution with Learning 27 2.3.1.1 Evolving ANN connection weights 28 2.3.1.2 Evolving ANN Architectures 30 2.3.1.3 Evolving ANN Learning Rules 33 2.3.2 What is next 35 2.4 Behavioural Genetics 38 2.4.1 Genotype, Phenotype and Environment 38 2.4.2 Methods employed in BG research – twin studies & GCTA 40 2.4.3 Environment 41 2.4.4 Genetic and Environmental Influences 42 2.4.5 Heritability 44 2.4.6 Evolution and Selection 46 2.4.7 Generalist Genes, Pleiotropy and Polygenicity 48 2.5 BG as a framework for neuro-evolution 49 2.5.1 Evolutionary and Learning Task(s) 51 2.5.2 Simulating variations in genetic influences 52 2.5.2.1 Encoding structural and learning parameters into genome 52 2.5.2.2 Calibrate the range of variation in genome 53 2.5.2.3 Genotype – Phenotype Mappings 54 2.5.3 Simulating variations in environmental influences 56 2.5.3.1 Simulating shared environmental influences 56 2.5.3.2 Simulating non-shared environmental influences 57 2.5.4 Generating population of ANNs 58 2.5.5 Training and performance assessment 61 2.5.5.1 Fitness Evaluation 62 2.5.5.2 Computing Heritability 62 2.5.6 Selection 63 2.5.6.1 Roulette wheel selection (Stochastic selection) 63 2.5.6.2 Truncation selection (Deterministic selection) 64 2.5.6.3 Selection and sexual reproduction 65 2.5.6.4 Breed next generation and repeat 66 2.6 Summary and contribution of the chapter 66 3. Neuro-evolutionary framework for capturing population variability across language development: Modelling children’s past tense formation 3.1 Overview 68 3.2 An introduction to language acquisition 68 3.3 Computational modelling of past tense acquisition 73 3.4 Learning English past tense through Evolution 75 3.4.1 English past tense dataset 77 3.5 Experiment Design 78 3.5.1 How was behaviour (performance) measured 80 3.6 Roulette wheel selection based experiment results 82 3.6.1 Results and Analysis 82 3.7 Truncation Selection based experiment results 98 3.7.1 Results and Analysis 98 3.8 Analysing the effects of selection 113 4 Table of Contents 3.9 Summary and contribution of the chapter 115 4. Behavioural Genetics inspired model for Transfer Learning 4.1 Overview 117 4.2 Introduction to Transfer Learning 118 4.3 Research issues in transfer learning 121 4.3.1 What to transfer? 121 4.3.2 How to transfer? 122 4.3.3 When to transfer? 125 4.3.4 How to assess task relatedness or how to model task similarity? 125 4.4 Heterogeneous Transfer: introduction and issues 126 4.5 What is next? 127 4.6 Extending the BG inspired model to transfer learning 129 4.6.1 How to choose tasks – related or heterogeneous 131 4.6.2 Simulating neurocomputational variation (What to transfer?) 132 4.6.3 How was shared environmental variation implemented? (What to 133 transfer?) 4.6.3.1 Initial weights of ANNs as representatives of non-shared 135 environment (What not to transfer) 4.6.4 Role of using twins population (Determining task relatedness and 135 avoiding negative transfer) 4.6.5 Implementation of transfer approach (How to transfer?) 136 4.6.6 Factors affecting transfer of ‘ability to learn’ 139 4.7 Summary and contribution of the chapter 141 5.