A Study of Genetic Programming and Grammatical Evolution for Automatic Object-Oriented Programming by Kevin Chizoba Igwe Submitted in fulfilment of the academic requirements for the degree of Master of Science in the School of Mathematics, Statistics, and Computer Science, University of KwaZulu-Natal, Pietermaritzburg April 2016 As the candidate’s supervisor I have/have not approved this thesis/dissertation for submission Signed: _______________________ Name: Prof. Nelishia Pillay Date: _______________________ PREFACE The experimental work described in this dissertation was carried out in the School of Mathematics, Statistics, and Computer Science, University of KwaZulu-Natal, Pietermaritzburg, from February 2014 to April 2016, under the supervision of Professor Nelishia Pillay. These studies represent original work by the author and have not otherwise been submitted in any form for any degree or diploma to any tertiary institution. Where use has been made of the work of others it is duly acknowledged in the text. _______________________ _______________________ Supervisor: Prof. Nelishia Pillay Candidate: Kevin Chizoba Igwe i DECLARATION 1: PLAGIARISM I, Kevin Chizoba Igwe (student number: 212553209) declare that: 1. The research reported in this dissertation, except where otherwise indicated or acknowledged, is my original work. 2. This dissertation has not been submitted in full or in part for any degree or examination to any other university. 3. this dissertation does not contain other persons’ data, pictures, graphs or other information, unless specifically acknowledged as being sourced from other persons; 4. This dissertation does not contain other persons’ writing, unless specifically acknowledged as being sourced from other researchers. Where other written sources have been quoted, then: a. their words have been re-written but the general information attributed to them has been referenced b. where their exact words have been used, their writing has been placed inside quotation marks, and referenced 5. This dissertation does not contain text, graphics or tables copied and pasted from the internet, unless specifically acknowledged, and the source being detailed in the dissertation and in the References sections. Signed: _____________________ Candidate: Kevin Chizoba Igwe Date: _______________________ ii DECLARATION 2: PUBLICATIONS DETAILS OF CONTRIBUTION TO PUBLICATIONS that form part and/or include research presented in this thesis Publication 1: Igwe, K., Pillay, N.: A Comparative Study of Genetic Programming and Grammatical Evolution for Evolving Data Structures. In: Proceedings of the 2014 Pattern Recognition Association of South Africa, pp. 115 – 121, Cape Town, South Africa (2014). Publication 2: Igwe, K., Pillay, N.: A Study of Genetic Programming and Grammatical Evolution for Automatic Object Oriented Programming: A Focus on the List Data Structure. In: Proceedings of the 7th World Conference on Nature and Biologically Inspired Computing (NaBIC 2015), pp. 151 – 163, Pietermaritzburg, South Africa (2015). _______________________ _______________________ Supervisor: Prof. Nelishia Pillay Candidate: Kevin Chizoba Igwe iii ABSTRACT Manual programming is time consuming and challenging for a complex problem. For efficiency of the manual programming process, human programmers adopt the object- oriented approach to programming. Yet, manual programming is still a tedious task. Recently, interest in automatic software production has grown rapidly due to global software demands and technological advancements. This study forms part of a larger initiative on automatic programming to aid manual programming in order to meet these demands. In artificial intelligence, Genetic Programming (GP) is an evolutionary algorithm which searches a program space for a solution program. A program generated by GP is executed to yield a solution to the problem at hand. Grammatical Evolution (GE) is a variation of genetic programming. GE adopts a genotype-phenotype distinction and maps from a genotypic space to a phenotypic (program) space to produce a program. Whereas the previous work on object- oriented programming and GP has involved taking an analogy from object-oriented programming to improve the scalability of genetic programming, this dissertation aims at evaluating GP and a variation thereof, namely, GE, for automatic object-oriented programming. The first objective is to implement and test the abilities of GP to automatically generate code for object-oriented programming problems. The second objective is to implement and test the abilities of GE to automatically generate code for object-oriented programming problems. The third objective is to compare the performance of GP and GE for automatic object-oriented programming. Object-Oriented Genetic Programming (OOGP), a variation of OOGP, namely, Greedy OOGP (GOOGP), and GE approaches to automatic object-oriented programming were implemented. The approaches were tested to produce code for three object-oriented programming problems. Each of the object-oriented programming problems involves two classes, one with the driver program and the Abstract Data Type (ADT) class. The results show that both GP and GE can be used for automatic object-oriented programming. However, it was found that the ability of each of the approaches to automatically generate code for object-oriented programming problems decreases with an increase in the problem complexity. The performance of the approaches were compared and statistically tested to determine the effectiveness of each approach. The results show that GE performs better than GOOGP and OOGP. iv ACKNOWLEDGMENTS The author gratefully acknowledges the financial support of the National Research Foundation (NRF). The NRF shall not be held responsible or be given attributes for opinions expressed and conclusions arrived at. Those are of the author. My gratitude goes to almighty God, my rock and my strength, who has sustained me till this very moment. I would like to thank my supervisor, Professor Nelishia Pillay for her guidance on every step I took to produce this research work. A million thanks to my family and friends who have encouraged and supported me. Special thanks to my brother Mr D.O Igwe for providing a shelter for me during this research period. Also, to Ms Chiedza Mlingwa for proofreading this dissertation and to Ms Jane C. Nnam for her words of encouragement. v TABLE OF CONTENTS PREFACE ................................................................................................................................... i DECLARATION 1: PLAGIARISM ......................................................................................... ii DECLARATION 2: PUBLICATIONS ................................................................................... iii ABSTRACT .............................................................................................................................. iv ACKNOWLEDGMENTS ......................................................................................................... v TABLE OF CONTENTS .......................................................................................................... vi LIST OF TABLES ................................................................................................................. xiii LIST OF FIGURES ................................................................................................................. xv LIST OF EQUATIONS ....................................................................................................... xviii CHAPTER 1: INTRODUCTION ........................................................................................ 1 1.1 Purpose of the Study ................................................................................................... 1 1.2 Objectives .................................................................................................................... 2 1.3 Contributions ............................................................................................................... 2 1.4 Dissertation Layout ..................................................................................................... 3 CHAPTER 2: GENETIC PROGRAMMING ...................................................................... 6 2.1 Introduction ................................................................................................................. 6 2.2 Introduction to Genetic Programming Algorithm ....................................................... 6 2.3 Control Model ............................................................................................................. 8 2.3.1 Generational Control Model ................................................................................ 8 2.3.2 Steady state control Model................................................................................... 9 2.4 Program Representation .............................................................................................. 9 2.5 Function and Terminal Set ........................................................................................ 10 2.5.1 Function Set ....................................................................................................... 11 2.5.2 Terminal Set ....................................................................................................... 11 2.6 Sufficiency and Closure ...........................................................................................