University of South Florida Scholar Commons Graduate Theses and Dissertations Graduate School October 2019 Interactive Fitness Domains in Competitive Coevolutionary Algorithm ATM Golam Bari University of South Florida Follow this and additional works at: https://scholarcommons.usf.edu/etd Part of the Computer Sciences Commons Scholar Commons Citation Bari, ATM Golam, "Interactive Fitness Domains in Competitive Coevolutionary Algorithm" (2019). Graduate Theses and Dissertations. https://scholarcommons.usf.edu/etd/8623 This Dissertation is brought to you for free and open access by the Graduate School at Scholar Commons. It has been accepted for inclusion in Graduate Theses and Dissertations by an authorized administrator of Scholar Commons. For more information, please contact [email protected]. Interactive Fitness Domains in Competitive Coevolutionary Algorithm by ATM Golam Bari A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy Department of Computer Science and Engineering College of Engineering University of South Florida Major Professor: Alessio Gaspar, Ph.D. Ali Yalcin, Ph.D. Lu Lu, Ph.D. R. Paul Wiegand, Ph.D. Srinivas Katkoori, Ph.D. Date of Approval: September 20, 2019 Keywords: Interactive Evolutionary Algorithm, Optimization, Solution Concept, Pareto Dominance, Concept Inventory Copyright c 2019, ATM Golam Bari Dedication I would like to dedicate my dissertation to my parents Mst Rawshan-ara Begum & Late Md. Yakub Ali; to my father in law Late Md. Reazul Haque and my loving wife Mst Rokeya Reaz. Acknowledgments I would like to express my heartiest gratitude to my supervisor, Dr. Alessio Gaspar for his never ending support and encouragement during my research period at USF. I would like to express my deepest appreciation to my committee members, Dr. Ali Yalcin, Dr. Lu Lu, Dr. R. Paul Wiegand, Dr. Srinivas Katkoori for providing their valuable insights regarding my research. I am also thankful to Dr. R. Paul Wiegand, Dr. Anthony Bucci, Dr. Jennifer L. Albert, Dr. Amruth Kumar for their feedback during my research paper writings. I would also like to thank my committee Chairperson Dr. John N Kuhn. I am thankful to the members of our CEREAL lab; Stephen Kozakoff, Kok Cheng Tan, Dmytro Vitel, Himank Vats, Paul Burton for providing feedback and discussion on my research. I am very thankful to Gabriel Franco, Laura Owczarek, Mayra Morfin, Kim Blumer, Lashanda Lightbourne who have always helped me without compliant even if I requested some academic administrative assistance at the very last minute. I am also thankful to Catherine Burton to provide feedback on the format of my dissertation. I am extremely grateful to my wife Rokeya Reaz whose encouragement, inspiration and caring support me during this journey. I am indebted to so many. This dissertation is based in part upon work supported by the Association for Com- puting Machinery's SIGCSE Special Projects 2015 award, and the National Science Foun- dation under awards #1504634, #1502564, and #1503834. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. Thank you. Table of Contents List of Tables vi List of Figures viii Abstract x Chapter 1: Introduction 1 1.1 Background 1 1.1.1 Interactive Evolutionary Algorithms 3 1.1.2 Competitive Coevolutionary Algorithms 4 1.1.3 (Co)evolutionary Approaches in Educational Software 6 1.2 Candidate Solution Evaluation 7 1.2.1 Evaluation in Interactive Evolutionary Algorithms 8 1.2.2 Evaluation in Competitive Coevolutionary Algorithms 8 1.3 Challenges and Opportunities in Candidate Evaluation of Com- petitive Coevolutionary Algorithm 9 1.3.1 Number Games 10 1.3.2 Objective and Subjective Fitness 10 1.3.3 Coevolutionary Pathologies 11 1.3.3.1 Loss of Gradient 12 1.3.3.2 Overspecialization, Also Known as Focusing 12 1.3.3.3 Cycling, Also Known as Intransitivity 14 1.3.4 Opportunities in Competitive Coevolutionary Algorithm 15 1.3.4.1 Solution Concepts 15 1.3.4.2 Ideal Evaluation 17 1.3.4.3 Coevolutionary Dimension Extraction (CDE) 18 1.3.4.4 Archiving 19 1.3.5 Discussion on Challenges and Opportunities in CCoEA 19 1.4 Challenges and Opportunities in Candidate Evaluation for Inter- active Evolutionary Algorithm 21 1.4.1 User Fatigue 21 1.4.2 Techniques to Reduce User Fatigue 22 1.4.2.1 Selecting Better Solutions 22 1.4.2.2 Statistical Ranking of Solutions 23 1.4.2.3 Evaluation of Solutions by Human Evaluator 24 i 1.4.2.4 Limiting User Intervention 25 1.4.2.5 Surrogate Model 27 1.4.2.6 Various Schemes of Mitigating User Fatigue 28 1.5 Competitive (Co)Evolutionary Algorithms and Their Applications: Analysis from the IEEE Xplore Meta Data Repository 29 1.6 Discussions 35 Chapter 2: Elementary Coevolution 37 2.1 Coevolutionary Interaction Dynamics 37 2.2 Experiment#1 - Elementary Coevolution 39 2.2.1 Problems 39 2.2.2 Algorithms 39 2.2.3 Methods 41 2.2.4 Results 44 2.3 Experiment#2 - Noisy Elementary Coevolution 44 2.3.1 Problems 44 2.3.2 Algorithms 45 2.3.3 Methods 45 2.3.4 Results 45 2.4 Experiment #3 - Noisy Teacher-Learner Coevolution 46 2.4.1 Problems 46 2.4.2 Algorithms 47 2.4.3 Methods 48 2.4.4 Results 48 2.5 Guidelines for Evolving Practice Problems Based on Elementary Coevolution 50 2.6 Noisy Teacher-Learner Coevolution Revisited 51 2.7 Experiment#4 - Three Properties of Noisy Teacher-Learner Coevolution 53 2.7.1 Problems 53 2.7.2 Algorithms 55 2.7.3 Methods 55 2.7.4 Results 56 2.7.5 Implications Regarding Design Guidelines 56 2.7.6 Analysis of the Interaction in Experiment #4 57 2.7.6.1 Property 1 - Mutation Effect 58 2.7.6.2 Property 2 - rand(r) Effect 59 2.7.6.3 Property 3 - Combined Effect 61 2.7.7 Experimental Validation of the Three Properties 64 2.7.7.1 Experimental Parameters 64 2.7.7.2 Performance Metrics 64 2.7.7.3 (M; r = 2) Analysis 65 2.7.7.4 Implications Regarding Design Guidelines 67 ii 2.8 Experiment#5 - Genes' Bounds 68 2.8.1 Problems 68 2.8.2 Algorithms 68 2.8.3 Results 68 2.8.4 Implications Regarding Design Guidelines 69 2.9 Experiment #6 - Noise and Genes' Bounds 69 2.9.1 Problems and Algorithms 69 2.9.2 Results 69 2.9.3 Implications Regarding Design Guidelines 70 2.10 Guidelines for Evolving Practice Problem by Noisy Teacher-Learner Coevolution 70 2.11 Minimal Guidelines for Interactive Noisy Teacher-Learner CCoEA 71 2.11.1 Measure the Progress in Coevolution 72 2.11.2 Design Solution Concepts 72 2.11.3 Consider the Role of Solution Concepts 73 2.11.4 Determine Minimum Number of Evaluations 73 2.11.5 Develop the Outcome of an Interaction 73 2.11.6 Develop Techniques to Mitigate User Fatigue 74 2.11.7 Focus on Noisy Evaluations 74 2.11.8 Validate the Design Guidelines 74 2.11.9 Validate the Evolution 75 Chapter 3: EvoParsons: Evolving Parsons Puzzles Based on Elementary Coevolution 76 3.1 Evolutionary Parsons Puzzle 76 3.2 Experiment #7 - Evaluating Evolved Practice Problems by Simulation 79 3.2.1 Problems and Algorithms 80 3.2.1.1 Evaluating Evolved Practice Problems on CDE Metrics 80 3.2.2 Methods 81 3.2.3 Results 84 3.2.4 Implications Regarding Design Guidelines 86 3.3 Experiment #8 - Evaluating Evolved Puzzles in EvoParsons 88 3.3.1 Problems and Algorithms 88 3.3.1.1 Mapping Integer Vectors into Parsons Puzzles 89 3.3.1.2 EvoParsons Architecture - EA, Broker and Learners 91 3.3.1.3 User Fatigue, Sparse Interaction Matrix and Se- lection Policies 93 3.3.2 EvoParsons Interaction with Real Human Students 94 3.3.2.1 Quantitative Perspective 96 3.3.2.2 Qualitative Perspective 97 3.3.3 Design Guidelines Validation 101 3.4 Validations of EvoParsons 103 iii 3.4.1 Summary of Findings 103 3.4.2 Threats to Validity 104 3.4.3 Improvement of EvoParsons 105 Chapter 4: Accelerating Progress in Elementary Coevolution 107 4.1 Need for Progress Acceleration 107 4.2 Definitions and Formalism 108 4.3 Experiments # 9 Relaxing Selection Criteria in P-PHC-P 108 4.3.1 Problems and Algorithms 109 4.3.1.1 Base Selection in P-PHC-P 110 4.3.1.2 Competitive Shared Fitness Based Relaxed Selection 111 4.3.2 Methods 113 4.3.3 Results 115 4.3.4 Relaxation vs Convergence, Exploration and Monotonic Progress 115 4.3.4.1 Convergence and Quality of the Solutions 115 4.3.4.2 Relaxation vs Exploration 116 4.3.4.3 Relaxation vs Monotonic Progress 117 4.4 Experiment #10 Pareto Front Based Relaxed Selection 118 4.4.1 Problems and Algorithms 119 4.4.2 Methods 121 4.4.3 Results 121 4.5 Relaxation Based on Interaction Space of Pareto Coevolution 121 4.6 Experiment #11 - Stepping Stones of Interaction Space 123 4.6.1 Interactive Domains and Interaction 124 4.6.2 Interaction Space 124 4.6.2.1 Properties of Interaction Space 126 4.6.2.2 Traversal of Interaction Space and Selection Con- ditions 127 4.6.3 Methods 127 4.6.4 Results 128 4.6.5 Observations on Interaction Space Based Relax Conditions 128 4.7 Experiment # 12 - Proposed Relaxed Selection Conditions 129 4.7.1 Selection Methods 129 4.7.2 Problems and Algorithms 132 4.7.3 Results 132 4.7.4 Exploration vs Diversity in Relaxed Section Conditions 133 4.8 Exp#13: Progress vs Distribution of Candidate Solutions in Pareto Layers 134 4.8.1 Problems and Algorithms 135 4.8.2 Methods 135 4.8.3 Results 135 iv 4.9 Exp#14: Diversity and Convergence in Relaxed Pareto Coevolution 137 4.9.1 Problems and Algorithms 137 4.9.2 Methods 138 4.9.3 Results 138 Chapter 5: Dominance Relations: Synthetic to EvoParson's Interaction 141 5.1 Dominance Relations in Interaction Space 141 5.2 Experiment #15 The Effect of Flipping Interaction