Optimizing an Evolutionary Approach to Machine Generated Artificial
Total Page:16
File Type:pdf, Size:1020Kb
Optimizing an Evolutionary Approach to Machine Generated Artificial Intelligence for Games Master’s Thesis MTA 161030 Aalborg University Medialogy Medialogy Aalborg University http://www.aau.dk Title: Abstract: Optimizing an evolutionary approach to machine generated artificial intelligence This thesis presents an investiga- for games tion into how to effectively optimize the production of machine generated Theme: game AI, exploring the behavior tree Master’s Thesis model and evolutionary computation. The optimization methods focus on Project Period: providing a ‘proof of concept’ that Spring Semester 2016 a system can be designed and im- plemented, through a series of stud- Project Group: ies, being capable of producing game MTA161030 AIs with alternative behaviors within a playthrough of a game. The con- Participant(s): struction of these behaviors should be Andrei Vlad Constantin informed by the evaluation of pre- vious behaviors, as well as show a Richard Alan Cupit quantifiable improvement in perfor- mance. The studies evaluate the per- Konstantinos Monastiridis formance of a generated AI for the game XCOM 2, a Turn-Based Tactics Supervisor(s): video game. The AIs will be evalu- Martin Kraus ated by running combat simulations Copies: 1 against the standard AI implemented by its developers. Ultimately, the re- Number of Pages: 90 sults of the process led to an user ex- periment, in which the most successful Date of Completion: machine generated game AI won 50% May 24, 2016 of matches. The content of this report is freely available, but publication (with reference) may only be pursued due to agreement with the author. Contents List of Figures vii List of Tables ix 1 Introduction 1 2 Background 3 2.1 Game AI . .3 2.1.1 Perspectives . .4 2.1.2 History . .4 2.1.3 Modern Video Game AI . .4 2.2 Behavior Trees . .5 2.2.1 Overview . .5 2.2.2 Uses in game industry and research . .8 2.3 Evolutionary Algorithms . 10 2.3.1 Genetic Algorithms . 10 2.3.2 Genetic Programming . 17 2.3.3 Uses in game industry and research . 17 2.4 Evolving Behavior Trees . 18 2.5 Platform of Application . 19 2.5.1 Turn Based Tactic Games . 19 2.5.2 XCOM 2 . 20 3 Project Statement 23 4 Design and Implementation 25 4.1 Mod Implementation . 25 4.1.1 Game Systems . 25 4.1.2 Normalization . 26 4.1.3 Environmental Cover . 29 4.1.4 Default AI . 30 4.2 Genetic Algorithm Implementation . 30 v vi Contents 4.2.1 Chromosome Design . 31 4.2.2 Example Chromosome Implementation . 32 4.2.3 WatchMaker Framework . 33 4.2.4 Generational Evolution Engine . 33 5 Experiment And Results 37 5.1 Pilot Test . 37 5.1.1 Design . 37 5.1.2 Analysis of Data . 42 5.2 Study 1 . 46 5.2.1 Design . 46 5.2.2 Analysis of Data . 52 5.3 Study 2 . 57 5.3.1 Design . 57 5.3.2 Analysis of Data . 61 5.4 Study 3 . 64 5.4.1 Design . 65 5.4.2 Analysis Of Data . 67 5.4.3 Final Evaluation . 70 5.5 User Testing . 71 6 Discussion and Conclusion 77 6.1 Discussion . 77 6.1.1 Dynamic Elitism . 77 6.1.2 Fitness Function . 78 6.1.3 Chromosome Structure . 78 6.1.4 Unit Conditions and Decisions . 79 6.2 Conclusion . 79 7 Future Directions 81 7.1 Further Development . 81 7.2 Alternative Directions . 82 Bibliography 83 Appendices 85 A. Extra Content . 85 B. Unit Condition Implementation . 86 C. Unit Decision Implementation . 87 D. Questionnaire . 88 E. Classifying Evaluation Matches . 89 List of Figures 2.1 Sequence checking for Ammunition and if so, the agent Reloads and the Sequence returns Success. .7 2.2 The Selector will return Success when either of depicted Actions return Success. .8 2.3 Visual representation of a string encoded chromosome, holding the solution variables . 12 2.4 Visual representation of a string encoded chromosome, holding ran- dom variables . 12 2.5 Single-point crossover performed an a string. 15 2.6 Two-point crossover performed an a string. 16 2.7 Visualization of Uniform Crossover - The ‘H’ characters represent the positive result of a coin toss. 16 2.8 Visualization of Input (top) and Output (bottom) string from a ge- netic mutation operator.The character H represents a successful coin toss. 17 4.1 XCOM 2 unit movement UI. 28 4.2 An example decision tree structure, with example string representa- tions . 32 4.3 Complete example chromosome . 33 4.4 Call to instantiate and Evolution Engine of type string, using the Generational Evolution Engine interface . 34 4.5 Example fitness evaluator code . 34 5.1 Chromosome encoding for the first action point . 38 5.2 Example of crossover producing undesirable offspring . 40 5.3 Average fitness % and win % of candidates per generation of Pilot Test....................................... 42 5.4 Graph showing the number of Unit Conditions contained within candidate solutions who won a minimum of one game . 44 vii viii List of Figures 5.5 Graph showing the number of Unit Decisions contained within can- didate solutions who won a minimum of one game . 45 5.6 Chromosome decision structure for Study One . 47 5.7 Chromosome encoding for the first action point . 47 5.8 Average fitness % and win % of candidates per generation for Study One. Fitness average does not include modifier . 53 5.9 Graph showing the number of Unit Conditions contained within candidate solutions who won a minimum of one game at Study One 55 5.10 Graph showing the number of Unit Decisions contained within can- didate solutions who won a minimum of one game at Study One ........................................ 56 5.11 Average fitness % and win % of candidates per generation for Study 2. Fitness average does not include modifier . 61 5.12 Occurrence of Unit Conditions within candidates winning 3 or more consecutive matches at Study 2 . 62 5.13 Occurrences of Unit Decisions within candidates winning 3 or more consecutive matches at Study 2 . 63 5.14 Example chromosome structures showing the irrelevance of the or- der of the Unit Conditions . 65 5.15 Average fitness % and win % of candidates per generation for Study 3. Fitness average does not include modifier . 67 5.16 Occurrence of Unit Conditions within candidates winning 3 or more consecutive matches at Study 3 . 69 5.17 Occurrences of Unit Decisions within candidates winning 3 or more consecutive matches at Study 3 . 69 5.18 Playtime differences between XCOM2 and XCOM Enemy Unknown/Enemy Within . 73 5.19 Correlation between participants play times in descending order and combat outcome statistics . 75 5.20 Most evolved candidate AI: ’BCDgdcdaihfADEigidfdeg’ UC&D break- down..................................... 76 1 Classification error rates for a trained K-nearest neighbor classifier, evaluating both sets . 89 2 Confidence matrix of a K-nearest neighbor classifier evaluating the allOnlyResims set . 90 List of Tables 4.1 Example Unit Condition characters for the GA to choose from, and their identifiers. 31 4.2 Example Unit Decision characters for the GA to choose from, and their identifiers. 32 5.1 Set of Unit Conditions used in the pilot test. 39 5.2 Set of Unit Decisions used in pilot test. 39 5.3 Pilot Test’s Solution space size. 40 5.4 Candidates that won multiple games over the course of the evolution. 43 5.5 Set of Unit Conditions used in Study 1. 48 5.6 Set of Unit Decision used in Study 1. 49 5.7 Solution space size for study 1 . 49 5.8 Success of elite candidates produced by the first 3 generations of Study 1 . 50 5.9 Elite candidate performance during Study One. 53 5.10 Elite candidate performance during Study One. 54 5.11 Set of Unit Conditions used in the Study 2. 57 5.12 Set of Unit Decision used in Study 2. 58 5.13 Size of Solution space from Study 2. 58 5.14 Elite candidate performance during Study 2. 62 5.15 Combat performance information about candidates that failed to win consecutive matches. 64 5.16 Study One’s Solution space size. ..