Combinational Machine Learning Creativity A
Total Page：16
File Type：pdf, Size：1020Kb
COMBINATIONAL MACHINE LEARNING CREATIVITY A Dissertation Presented to The Academic Faculty By Matthew Guzdial In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the College of Computing Georgia Institute of Technology August 2019 Copyright c Matthew Guzdial 2019 COMBINATIONAL MACHINE LEARNING CREATIVITY Approved by: Dr. Devi Parikh Dr. Mark Riedl, Advisor School of Interactive Computing School of Interactive Computing Georgia Institute of Technology Georgia Institute of Technology Dr. Briak Magerko Dr. Ashok Goel School of Literature, Media, and School of Interactive Computing Communication Georgia Institute of Technology Georgia Institute of Technology Dr. Charles Isbell Dr. Michael Mateas School of Interactive Computing Computational Media Georgia Institute of Technology University of California, Santa Cruz Date Approved: June 24, 2019 To my parents, my first supporters and first inspirations. ACKNOWLEDGEMENTS It is impossible for me to be fully satisfied with these acknowledgements. I have been blessed with incredible support from all possible corners: academic colleagues, industry partners, family, and friends. Let me begin with a blanket statement of thanks for this support, as I fear an exhaustive list would have to be a secondary document. First and most clearly this work would not have been possible without the support of my advisor. Mark has provided opportunities for my research work, support and feedback when I have needed it, and trust when I have not. In my fellow PhD students I have found community, friends, and academic colleagues that have pushed me to improve. I thank Boyang ‘Albert’ Li for mentoring me through my first PhD research project and my in- ternship at Disney Research. Alex Zook and Kristin Siu were both incredible guides and collaborators throughout the early years of my PhD, collaborating on both research projects and multiple whiteboards of Pokemon´ drawings. Brent Harrison and I started in the lab at the same time (though Brent was a post-doc), and I am immensely grateful to have spent three years researching with him. Lara Martin, Chris Purdy, and Upol Ehsan helped rein- vigorate the lab and myself, creating valuable research community and becoming valued friends. It’s been a joy and an inspiration watching Zhiyu Lin and more recently Raj Am- manabrolu become excellent researchers, and I anticipate even greater future success for the both of them. I have had the honor of mentoring a rather large number of undergraduates. I greatly appreciate this opportunity and the ways that their work has supported and influenced my own research trajectory. I would especially like to thank Nicholas Liao, who in camping out in the lab one day in an attempt to get involved in undergraduate research indirectly started this aspect of my academic career. Of course many academics outside of Mark’s research lab have impacted my research. In particular, I would like to thank Brian Magerko who took a chance on me as an under- iv graduate, and thus launched my interest in research. Thanks as well to Mikhail Jacob, who I first met while working with Brian, and who would go on to be an invaluable colleague. We worked together on the 2018 International Conference on Computational Creativity, and supported each other across research projects, thesis proposals, and thesis defenses. Outside of Georgia Tech, I would like to thank Gillian Smith, who in an alternate uni- verse was my advisor. Thankfully in this universe I am lucky enough to count Gillian as an excellent colleague, collaborator, and friend. Thanks to Mike Cook for inspiring my research into game generation in the first place and for being an all around great guy. Spe- cial thanks to Diana Ford and the entire Unity Research team for believing in my work and granting me a 2018 Unity Graduate Fellowship. I would also like to thank Joseph Os- born, Sam Snodgrass, and Adam Summerville. While we were at separate universities and they’ve all since graduated, going through parallel PhD programs, collaborating with them on papers, and working with them on the Knowledge Extraction from Games workshop has all been hugely helpful and rewarding. I could keep going forever. There are so many more people who have impacted my research directly and indirectly. However, my greatest supporters have been my family, who I cannot thank enough. I want to especially thank my parents, Mark Guzdial and Barbara Ericson. They were my very first academic inspirations and their support and advice has been invaluable. But my greatest thanks must be reserved for Jack for their support and patience. They have been my first editor and a springboard for ideas. They have kept me from overworking myself, and if that failed, they took care of me through burn out. I could not have done this without them. v TABLE OF CONTENTS Acknowledgments . iv List of Tables . xii List of Figures . xiv Chapter 1: Introduction . 1 1.1 The Case for Combinational Creativity . 2 1.2 The Case for Combinational Machine Learning Creativity . 4 1.3 The Case for Video Games as Domain . 4 1.4 Thesis Statement . 5 1.5 Outline . 6 Chapter 2: Situating the Work . 8 2.1 Computational Creativity . 8 2.1.1 Boden’s Theory of Creativity . 8 2.1.2 Discovery Systems . 9 2.1.3 Analogical Reasoning . 10 2.1.4 Combinational Creativity . 11 2.1.5 Conceptual Blending . 13 vi 2.1.6 Amalgamation . 14 2.1.7 Compositional Adaptation . 14 2.2 Procedural Content Generation . 15 2.2.1 Computational Creativity and Games . 15 2.2.2 Procedural Content Generation via Machine Learning . 16 2.2.3 Game Rules . 17 2.2.4 Automated Game Design . 17 2.3 Machine Learning . 18 2.3.1 Object and Scene Modeling . 19 2.3.2 Machine Vision . 19 2.3.3 Automated Game Playing . 19 2.3.4 Automated Understanding . 20 2.3.5 Automated Game Understanding . 21 2.3.6 Knowledge Reuse in Neural Networks . 21 Chapter 3: Automated Level Design . 23 3.1 Background . 24 3.2 System Overview . 25 3.3 Model Learning . 26 3.3.1 Defining and Categorizing Level Chunks . 26 3.3.2 Probabilistic Graphical Model . 28 3.4 Generation . 31 3.4.1 Generating Level Plans . 31 vii 3.4.2 Generating Level Chunks . 33 3.5 Evaluation . 34 3.5.1 Experimental Setup . 34 3.5.2 Methodology . 36 3.5.3 Study Results and Discussion . 37 3.5.4 Model Evaluation . 40 3.6 Conclusions . 41 Chapter 4: Game Engine Learning . 43 4.1 Background . 44 4.2 System Overview . 45 4.2.1 Parsing Frames . 45 4.2.2 Engine Learning . 47 4.2.3 Limitations . 52 4.3 Evaluation . 53 4.3.1 Frame Accuracy Evaluation . 53 4.3.2 Gameplay Learning Evaluation . 56 4.4 Conclusions . 58 Chapter 5: Conceptual Blending of Machine Learned Models . 59 5.1 Background . 60 5.2 Conceptual Blending of Level Design Models . 61 5.2.1 Deriving S-Structure Graphs . 62 5.2.2 Blending S-Structure Graphs . 63 viii 5.3 Blending Evaluation: Lost Levels . 66 5.3.1 Example Output . 70 5.4 Underwater Castle Case Study . 71 5.5 Conclusions . 71 Chapter 6: Conceptual Expansion . 73 6.1 Background . 74 6.1.1 Amalgamation . 75 6.1.2 Conceptual Blending . 76 6.1.3 Compositional Adaptation . 77 6.2 Output Space Comparisons . 77 6.3 Problem Statement . 80 6.4 Conceptual Expansion . 81 6.5 Case Study: House Boat . 84 6.5.1 Metrics . 85 6.5.2 Case Study Results . 86 6.6 Case Study: Combining Machine-learned Game Level Models . 88 6.6.1 Results . 90 6.7 Discussion . 93 6.8 Conclusions . 94 Chapter 7: The Video Game Invention Problem . 96 7.1 Background . 98 7.2 Game Graph Representation . 99 ix 7.2.1 Level Design Learning . 99 7.2.2 Ruleset Learning . 101 7.2.3 Game Graph . 102 7.3 Variation 1: Goal-Driven Conceptual Expansion over Game Graphs . 105 7.3.1 Evaluation . 108 7.3.2 Results . 110 7.3.3 Discussion . 111 7.4 Variation 2: Human-in-the-Loop, Heuristic-based Game Generation . 112 7.4.1 Full Games Case Study . 116 7.4.2 Discussion . ..