University of Plymouth PEARL https://pearl.plymouth.ac.uk 04 University of Plymouth Research Theses 01 Research Theses Main Collection 2019 Efficient Evolution of Neural Networks Pagliuca, Paolo http://hdl.handle.net/10026.1/15144 University of Plymouth All content in PEARL is protected by copyright law. Author manuscripts are made available in accordance with publisher policies. Please cite only the published version using the details provided on the item record or document. In the absence of an open licence (e.g. Creative Commons), permissions for further reuse of content should be sought from the publisher or author. EFFICIENT EVOLUTION OF NEURAL NETWORKS by PAOLO PAGLIUCA A thesis submitted to the University of Plymouth in partial fulfilment for the degree of DOCTOR OF PHILOSOPHY School of Computing, Electronics and Mathematics [In collaboration with ISTC-CNR (Institute of Cognitive Sciences and Technologies)] September 2018 Copyright Statement This copy of the thesis has been supplied on condition that anyone who consults it is understood to recognise that its copyright rests with its author and that no quotation from the thesis and no information derived from it may be published without the author’s prior consent. Acknowledgments First and foremost, I would like to thank my supervisor, Stefano Nolfi, who supported me during the last four years, even when I encountered some personal problems. His passion, knowledge, advice and trust have been fundamental for me to keep on track and succeed in completing this work. He is a special person and I am proud to work with him. I am also grateful to all my colleagues at the Institute of Cognitive Sciences and Technologies, in particular the LARAL group (in sparse order Onofrio Gigliotta, Diana Giorgini, Vito Trianni, Alessandra Vitanza, Filippo Cantucci, Jonata Tyska Carvalho, Nicola Milano, Luca Simione, Dario Albani, Tomassino Ferrauto, Gianluca Massera, Eliseo Ferrante, Giuseppe Morlino), for their contribution to my work, but especially for the funny moments spent in these years, in particular during our after lunch table- soccer matches. Their friendship is really important for me. I find it difficult to imagine a better work environment. I would thank also my second supervisor, Angelo Cangelosi, who always gave me support during these years, and the Plymouth University. A special thank goes to Silvia Felletti, a colleague who has become one of my best friends. She constantly encouraged me when research did not progress. We shared plenty of funny moments, including table-soccer matches, lunches and dinners, evenings spent by playing table games, etc. A particular mention is for Monica, who is a special friend. I am also grateful to other people known at the Institute for relatively short periods (Chiara, Cristina and Martina). I am infinitely grateful to my loving family (my mother Nadia, my father Domenico, my brother Antonello, my sister Nina, my aunt Andreina and my grandmother Anna), who supported me all days and gave me all I needed to have a good life. Although sometimes I do not appreciate some aspects of their characters, they are special people and I love them. I thank my best friends Franco, Andrea, Andrea, Ivano, Valerio and Luca for all the moments spent together during the last twenty years. Without them, I did not manage to arrive at this point of my life. We shared many important moments and we never stopped to support each other, especially when some tragic episodes happened. I would also thank some other important friends met during my University studies (in random order Simone, Valentina, Cosmo and Matteo). Moreover, I would thank my friend Silvia, a person that deserves all of my respect for what she achieved in her life. A thank you also goes to my former colleagues Emanuele, Lara, Luca, Marco, Flavio, Marco, Marco, Giuseppe, Emanuele, Valeriano, Pierpaolo, Daniele, Luciano, Marco, Antonio and Mario. They have been very important for my personal growth, although I was not able to show them my love. Finally, I would infinitely thank the people met during the last three years, who helped and still help me become a better person. This work is dedicated to the loving memory of my grandmother Rosa, who gave me her infinite love till the end, my grandfather Gino, a lovable person and a hard worker, Loreto and Margherita, two special people that made me feel like a son and that I loved so much and Loredana, who has seen me grow since I was a child. Abstract This thesis addresses the study of evolutionary methods for the synthesis of neural network controllers. Chapter 1 introduces the research area, reviews the state of the art, discusses promising research directions, and presents the two major scientific objectives of the thesis. The first objective, which is covered in Chapter 2, is to verify the efficacy of some of the most promising neuro-evolutionary methods proposed in the literature, including two new methods that I elaborated. This has been made by designing extended version of the double-pole balancing problem, which can be used to more properly benchmark alternative algorithms, by studying the effect of critical parameters, and by conducting several series of comparative experiments. The obtained results indicate that some methods perform better with respect to all the considered criteria, i.e. performance, robustness to environmental variations and capability to scale-up to more complex problems. The second objective, which is targeted in Chapter 3, consists in the design of a new hybrid algorithm that combines evolution and learning by demonstration. The combination of these two processes is appealing since it potentially allows the adaptive agent to exploit a richer training feedback constituted by both a scalar performance objective (reinforcement signal or fitness measure) and a detailed description of a suitable behaviour (demonstration). The proposed method has been successfully evaluated on two qualitatively different robotic problems. Chapter 4 summarizes the results obtained and describes the major contributions of the thesis. Author’s declaration At no time during the registration for the degree of Doctor of Philosophy has the author been registered for any other University award without prior agreement of the Doctoral College Quality Sub-Committee. Work submitted for this research degree at the University of Plymouth has not formed part of any other degree either at the University of Plymouth or at another establishment. Relevant scientific seminars and conferences were regularly attended at which this work was presented. Two articles have been accepted for publication in refereed journals. This study was carried out in collaboration with the Istituto di Scienze e Tecnologie della Cognizione (ISTC) - Consiglio Nazionale delle Ricerche (CNR), Rome. This thesis contains works being the result of collaborations with other researchers. The author contribution to the reported works over the total was about 80% for the work described in chapter 2 and 60% for the work described in chapter 3. Word count for the main body of this thesis: 34130 Publications Paolo Pagliuca and Stefano Nolfi (2015). “Integrating learning by experience and demonstration in autonomous robots”. Adaptive Behavior 23 (5), pp 300-314. DOI: https://doi.org/10.1177/1059712315608424 Paolo Pagliuca, Nicola Milano and Stefano Nolfi (2018). “Maximizing adaptive power in neuroevolution”. PloS one 13 (7). DOI: https://doi.org/10.1371/journal.pone.0198788 Nicola Milano, Paolo Pagliuca and Stefano Nolfi (2019). “Robustness, evolvability and phenotypic complexity: insights from evolving digital circuits”. Evolutionary Intelligence 12 (1), pp 83-95. DOI: https://doi.org/10.1007/s12065-018-00197-z Paolo Pagliuca and Stefano Nolfi (2019). “Robust optimization through neuroevolution”. PloS one 14 (3). DOI: https://doi.org/10.1371/journal.pone.0213193 Signed: Date: Contents Acknowledgments .......................................................................................................................... Author’s declaration ....................................................................................................................... Contents ......................................................................................................................................... List of Figures ................................................................................................................................ List of Tables ................................................................................................................................. List of Acronyms …………………………………………………………………………………. Chapter 1. Neuroevolution ............................................................................................................ 2 Introduction ................................................................................................................................... 2 1.1. Neuroevolution .............................................................................................................. 2 1.1.1. Basic algorithm ..................................................................................................... 2 1.1.2. Advantages and disadvantages of neuroevolution ................................................ 3 1.1.3. Applications domains ............................................................................................ 5 1.2. Research topics in neuroevolution ...............................................................................