University of California Santa Cruz Learning from Games
Total Page:16
File Type:pdf, Size:1020Kb
UNIVERSITY OF CALIFORNIA SANTA CRUZ LEARNING FROM GAMES FOR GENERATIVE PURPOSES A dissertation submitted in partial satisfaction of the requirements for the degree of DOCTOR OF PHILOSOPHY in COMPUTER SCIENCE by Adam J. Summerville June 2018 The Dissertation of Adam J. Summerville is approved: Professor Michael Mateas, Chair Professor Noah Wardrip-Fruin Professor Santiago Ontañón Dean Tyrus Miller Vice Provost and Dean of Graduate Studies Copyright © by Adam J. Summerville 2018 Table of Contents List of Figures vi List of Tables xiv Abstract xvi Dedication xvii Acknowledgments xviii 1 Introduction 1 1.1 Research Contributions ........................... 5 I Learning From Game 7 2 How Humans Understand Games 8 2.1 Human Annotation Tasks .......................... 17 2.2 Entity Persistence .............................. 17 2.3 Camera Motion ................................ 18 2.4 Room Detection ............................... 21 2.4.1 Teleportation ............................. 23 2.4.2 Traversal ............................... 25 2.5 Animation Cycles ............................... 32 2.6 Mode Dynamics ................................ 34 2.7 Conclusion .................................. 36 3 Mappy – A System for Map Extraction and Annotation via Observa- tion 37 3.1 NES Architecture and Emulation ...................... 41 3.2 Operationalizing Entity Persistence ..................... 43 3.3 Inferring Camera Motion .......................... 49 iii 3.4 Determining Room Transitions ....................... 59 3.4.1 Automatic Mapping ......................... 61 3.5 The Detection of Animation Cycles ..................... 61 3.6 The Extraction of Dynamics Modes .................... 67 3.6.1 Proximate Cause Learning ...................... 80 3.7 Mappy Annotation Synthesis ........................ 92 3.8 Conclusion .................................. 94 4 Case Study – A Comparison of Human and Machine Annotations for Super Mario Bros. 96 4.1 Machine Annotation ............................. 103 4.2 Case Study – Other Mappy Output ..................... 110 4.3 Discussion ................................... 115 II The Generation of Game Maps 122 5 Procedural Generation of Two Dimensional Rooms via Machine Learn- ing 123 5.1 Two Dimensional Room Generation .................... 127 5.2 Required Properties of A General Machine Learned Room Generator .. 130 5.3 Existing PCGML Systems and their Generality .............. 136 5.3.1 Markov Methods ........................... 136 5.3.2 Neural Methods ........................... 138 5.3.3 Matrix Factorization Methods .................... 139 5.4 Graphical Methods .............................. 140 5.5 PCGML Generality Comparison ...................... 141 6 Kamek – A Machine-Learned Two Dimensional Room Generator 143 6.1 Data Formulation ............................... 149 6.1.1 Sampling Methodology ........................ 152 6.1.2 Choice of Space-Filling Curve .................... 154 6.1.3 Evaluation of Linearization Traversals ............... 159 6.2 The Impact of Meta-Information on Quality ................ 160 6.3 Model Capacity ................................ 166 6.4 On the Importance of Following the Flow of Play ............. 171 6.5 A Comparison of Human and Machine Annotation ............ 172 6.6 Combining Machine-Learned and Search-Based Content Generation .. 173 7 Map Generation 180 7.1 Digression on Euclidean Vs Non-Euclidean Topology ........... 187 7.2 Data Requirements .............................. 188 iv 8 Case Study – Agahnim 191 8.1 Generation Process .............................. 197 8.2 Results ..................................... 198 8.3 Discussion ................................... 202 III The Evaluation of Generative Methods 205 9 Existing Methods for Generative Evaluation 206 10 Aside – A Discussion of Metrics for Platformers 215 10.1 Previous Computational Metrics ...................... 217 10.2 Novel Computational Metrics ........................ 219 10.3 Empirical Results ............................... 226 10.3.1 Metrics Selection with LASSO ................... 226 10.3.2 LASSO Prediction Results ...................... 228 10.3.3 Previous Neural Network Model .................. 229 10.3.4 Metrics Selected by LASSO ..................... 230 10.3.5 Discussion ............................... 234 10.3.6 Case Studies ............................. 242 10.4 Conclusion .................................. 244 11 Analyses of Generative Space 246 11.1 Kernel Density Estimation and Visual Inspection ............. 248 11.1.1 Comparison to the Original Corpus ................. 250 11.2 Expressive Volume .............................. 259 11.2.1 Case Study – Expressive Volume as Function of Training Corpus 260 11.3 Joint Distribution Testing of Generative Spaces .............. 264 12 Analysis of Individual Pieces of Content 272 12.1 Avoiding Cherry-Picking ........................... 273 12.2 Plagiarism As Selection Criterion ...................... 276 12.3 Metric Distance as Selection Criterion ................... 281 12.3.1 Case Study – Kamek ......................... 283 12.4 Discussion ................................... 288 13 Conclusions And Future Work 291 13.1 Future Work ................................. 293 Bibliography 295 v List of Figures 2.1 A scene from Super Mario Bros. where semantic representation knowl- edge would classify entities that are animate in color and the inanimate entities as gray-scale. The human is obviously animate, and the anthro- pomorphic mushroom with eyes and feet would reasonably be intuited to be mobile. Note that this intuition is incorrect as the mushroom above the ?-block has the same movement profile as the Goomba. ....... 20 2.2 Two rooms in Adventure that are traversed between. The player exits through the bottom of the first room and winds up in the top ofthe second. ..................................... 23 2.3 The interstitial shown in Super Mario Bros. before the player starts a “world” (a set of rooms). ........................... 24 2.4 Three frames from the interstitial sequence in The Legend of Zelda that occurs when the player enters a dungeon. ................. 24 2.5 Three frames from the traversal interstitial sequence in The Legend of Zelda that occurs when the player traverses between rooms connected in a Euclidean manner. ............................. 26 2.6 Three frames from the traversal interstitial sequence in Metroid. .... 26 2.7 The original frame is shown on the top, and the results of each input on the resultant frame are shown on the bottom. ............... 29 2.8 The three images that make up Mario’s run cycle in Super Mario Bros. 30 2.9 The eight images that make up Batman’s run cycle in Batman: The Return of the Joker .............................. 30 2.10 A phénakistiscope with a man pumping water from Mclean’s Optical Illusions [122]. Reprinted with permission[151]. .............. 30 2.11 The two halves of a thaumatrope from 1825 which would display a vase full of flowers when used. Reprinted with permission [63]. ........ 31 2.12 A reproduction of a Victorian zoetrope. Reprinted with permission [54]. 32 2.13 One thousand frames of Mario’s animations in Super Mario. Each frame of animation is assigned a unique identifier, and the progression of these is shown in the top image. The current frame, sampled periodically, is shown on the bottom. ............................ 34 vi 3.1 The architecture for Mappy. Mappy takes in a game and a set of input traces. These are passed to an instrumented emulator that provides entity locations which are used for determining overall camera behavior and entity tracks. The camera behavior is used to determine when a room begins or ends. The entity tracks are used to determine what animations occur at given points in time, and finally, along with said animation information, are used to learn an automata describing the properties and dynamics for each of the types of entities found. All of this information combined is used in the resultant map graph. ............... 39 3.2 The controller for the Nintendo Entertainment System. ......... 40 3.3 An exaggerated example of differences in two frames from Super Mario Bros.. The initial frame is slightly reduced in opacity for this illustration. This would lead to the apparent motion of the mushroom moving to the right and the lower Goomba moving to the left. .............. 44 3.4 An exaggerated example of differences in three frames from Mega Man. The initial frame is slightly reduced in opacity for this illustration. In the left image, Mega Man starts at one position (frame 1) and moves right to (frame 2). On the right Mega Man abruptly changes direction and moves left from (frame 2) to (frame 3) ....................... 45 3.5 Two frames of Super Mario Bros., the optical flow between them (as achieved via the technique described in ”Two-Frame Motion Estimation Based on Polynomial Expansion” by Gunner Farneback in 2003” [91]), and the desired optical flow. Mario, the mushroom, and the camera move slightly to the right, while the Goomba moves to the left. Black in the optical flow images denotes no motion, while a higher intensity denotes more motion. The desired flow field should have all background elements with a slight (6 pixels) amount of motion, the Goomba with higher (9 pixels), Mario with slight (1 pixel), and the mushroom with no apparent motion; however, sky presents a large field in which no flow can be discerned. ................................. 51 3.6 An example of template matching of two different template patterns