A STUDY OF TECHNIQUES FOR MEASURING ENJOYMENT IN VIDEO GAMES CONTAINING PROCEDURAL GENERATION By ELIZABETH A. MATTHEWS A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2019 c 2019 Elizabeth A. Matthews I dedicate this to everyone who believed in me, even when I didn't. To my friends and family: you laid the groundwork for me to start running. To my past self: your struggle got me to the end. To those in the future: know that you can do it too. ACKNOWLEDGMENTS First, I would like to thank my advisor, Dr. Juan E Gilbert, for believing in me since the beginning. I would also like to thank my committee, for being interested and invested in my progress and success. Thanks to my parents, Robin A Matthews and Geoffrey B Matthews, for their experience and guidance through a stressful time. Thanks to everyone in the Human Experience Research Lab at the University of Florida for their help, friendship, and support. Specifically, thank you to Rua Williams, Briana Posadas, and DeKita Moon for their thorough feedback on this dissertation. I would like to extend special thanks to Bodie Lee, for always being there for me. Finally, I would like to thank my friends outside of academia, for their listening and encouragement. 4 TABLE OF CONTENTS page ACKNOWLEDGMENTS..................................4 LIST OF TABLES......................................8 LIST OF FIGURES.....................................9 ABSTRACT......................................... 13 CHAPTER 1 INTRODUCTION................................... 15 1.1 Problem and Motivation............................. 15 1.2 Procedural Content Generation......................... 15 1.3 Game Enjoyment................................. 16 1.4 Overview of Goals, Research Questions, and Thesis Statement......... 17 1.5 Contributions................................... 18 1.6 Organization of the Dissertation......................... 19 2 RELATED WORK / LITERATURE REVIEW..................... 20 2.1 Defining Game Enjoyment............................ 20 2.1.1 GameFlow................................ 20 2.1.2 Motivational States............................ 24 2.1.3 Emotional States............................. 26 2.1.4 Needs Satisfaction............................ 30 2.1.5 Engagement............................... 32 2.1.6 Summary................................. 38 2.2 Measuring Game Enjoyment........................... 38 2.2.1 Subjective Measurements......................... 38 2.2.2 Objective Measurements......................... 40 2.2.3 List of Measurement Tools........................ 43 2.2.4 Summary................................. 48 2.3 Procedural Generation in Video Games..................... 48 2.3.1 Independent Procedural Content Generation............... 49 2.3.2 Experience Driven Procedural Content Generation............ 50 2.3.3 Summary................................. 51 3 IMPLICATIONS.................................... 52 3.1 Problem Statements and Proposed Solution................... 52 3.2 Research Questions and Hypotheses to be Tested................ 52 5 4 APPROACH...................................... 54 4.1 User Study Design Overview........................... 54 4.2 Participants................................... 56 4.3 Game Types................................... 56 4.4 Game Enjoyment Metric and Measurement................... 58 4.5 Data Collection and Analysis.......................... 59 4.5.1 Demographics.............................. 61 4.5.2 The Game Experience Questionnaire................... 62 4.5.3 The Fang et al. Questionnaire...................... 62 4.5.4 Physiological Data with the Empatica.................. 62 5 2D+3D INFINITE RUNNER ENGINE......................... 66 5.1 Game Style Choice................................ 66 5.2 Procedural Content Generation......................... 66 5.3 Static Content Generation............................ 66 5.4 Manual Designs................................. 67 6 ATLAS CHRONICLE................................. 69 6.1 Story Abstraction................................ 69 6.2 Physics Engine.................................. 70 6.3 Terrain Generation................................ 71 6.4 Recursive Process................................ 72 6.5 Terrain Mapping................................. 72 6.6 Testing Game Engine.............................. 73 6.6.1 Storyline and Progression........................ 73 6.6.2 2D Engine................................ 74 6.7 Static Content Generation............................ 74 6.8 Manual Designs................................. 74 7 DATA AND ANALYSIS................................ 83 7.1 Data....................................... 83 7.2 Analysis..................................... 83 7.2.1 Objective Measurements......................... 84 7.2.2 Subjective Measurements......................... 86 7.2.2.1 Analysis Procedure....................... 87 7.2.2.2 Decline of Enjoyment..................... 91 7.2.2.3 Gender and Gamer Label Differences............. 93 7.3 Results...................................... 94 8 SUMMARY AND FUTURE WORK.......................... 113 8.1 Summary and Research Questions Revisited................... 113 8.2 Contributions................................... 113 8.3 Limitations and Future Work.......................... 114 6 8.4 Conclusions.................................... 114 APPENDIX A LEVEL DESIGN MATERIALS............................. 116 A.1 Infinite Runner Designer............................. 116 A.2 RPG Designer.................................. 117 B USER STUDY MATERIALS............................. 119 B.1 Recruitment Flyer................................ 119 B.2 Screening Form................................. 120 B.3 Informed Consent Form............................. 121 B.4 Instructions.................................... 124 B.5 Game Experience Questionnaire......................... 126 B.5.1 In-Game GEQ.............................. 126 B.5.2 Scoring Guide............................... 126 B.6 Fang et al. Questionnaire............................ 127 C ADDITIONAL STUDY DATA............................. 128 C.1 Subjective Data................................. 128 C.1.1 Before and After by Generation Type.................. 128 C.1.2 Game and Generation.......................... 141 C.1.3 Gender.................................. 143 REFERENCES........................................ 145 BIOGRAPHICAL SKETCH................................. 157 7 LIST OF TABLES Table page 4-1 Table of participant controlled randomized order assignment. Letters represent the following: s = static, m = manual, and p = procedural content generation, Run = infinite runner game, RPG = role playing game. Order uses a Latin Square of size 3 for both game types................................... 65 7-1 A traditional numerical contingency table for GEQ03 ``I felt bored.'' The left column indicates the numerical response value. The other columns are for each segment during which the data is collected. The numbers in the segment columns represent the total for each response value in that segment................... 102 7-2 A traditional numerical contingency table with superscript indicating the pairwise test results for GEQ03 ``I felt bored.'' Numbers are the same as Table 7-1...... 102 A-1 Controls for the Infinite Runner Designer Program................... 117 A-2 Table for manual color mapping............................ 118 8 LIST OF FIGURES Figure page 1-1 Two variations of dungeon layouts in Rogue [1].................... 19 4-1 The typical features of an infinite runner with the PCG in green. The user controls a runner with constant velocity by pressing a button to jump. Procedural generation controls the jump distance, the next platform's height relative to the current platform, the next platform's length, and optional obstacles. Good PCG assures that the layouts are always possible............................... 63 4-2 The features of a possible RPG with PCG setup. A story is defined by important locations of interest and connections between them. PCG then varies the relative cardinal directions, distances, and landmass shapes around the locations. Good PCG would assure that the progression of locations in the story is not broken by impossible land features that would otherwise prevent the subject from following the intended storyline.................................. 64 4-3 Mockup of the setup for the study. Participant will be seated a comfortable distance from the computer monitor, between one to three feet................ 64 5-1 Example of the implemented 2D side-scrolling jumping game designed with 2D+3D infinite runner engine [2]. The user controls the square by jumping between platforms. 68 6-1 Example of story abstraction with three LOI and three restrictions. The restrictions between A/B and B/C are traversable restrictions and the restriction between A/C is not.......................................... 75 6-2 The three states in which a slide spring can exist. Top image: the connection between A and B is less than MAX and greater than MIN distances. Middle image: the distance between A and B is less than MIN, force is applied to push A and B away from each other. Bottom image: the distance between A and B is greater than MAX, force is applied to pull A and B towards each other............... 75 6-3 The process from the coordinates generated by the space manager to 2D map.... 76 6-4 Recursive process for continent