A DATA-DRIVEN APPROACH FOR PERSONALIZED DRAMA MANAGEMENT A Thesis Presented to The Academic Faculty by Hong Yu In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the School of Computer Science Georgia Institute of Technology August 2015 Copyright © 2015 by Hong Yu A DATA-DRIVEN APPROACH FOR PERSONALIZED DRAMA MANAGEMENT Approved by: Dr. Mark O. Riedl, Advisor Dr. David Roberts School of Interactive Computing Department of Computer Science Georgia Institute of Technology North Carolina State University Dr. Charles Isbell Dr. Andrea Thomaz School of Interactive Computing School of Interactive Computing Georgia Institute of Technology Georgia Institute of Technology Dr. Brian Magerko Date Approved: April 23, 2015 School of Literature, Media, and Communication Georgia Institute of Technology To my family iii ACKNOWLEDGEMENTS First and foremost, I would like to express my most sincere gratitude and appreciation to Mark Riedl, who has been my advisor and mentor throughout the development of my study and research at Georgia Tech. He has been supportive ever since the days I took his Advanced Game AI class and entered the Entertainment Intelligence lab. Thanks to him I had the opportunity to work on the interactive narrative project which turned into my thesis topic. Without his consistent guidance, encouragement and support, this dissertation would never have been successfully completed. I would also like to gratefully thank my dissertation committee, Charles Isbell, Brian Magerko, David Roberts and Andrea Thomaz for their time, effort and the opportunities to work with them. Their expertise, insightful comments and experience in multiple research fields have been really beneficial to my thesis research. A lot of friends and colleagues have been generous and helpful to make my study fun and memorable in the entertainment intelligence lab. Many thanks go to Boyang Li, Alexander Zook, Brian O'Neill, Kristin Siu, Matthew Guzdial, Brent Harrison, Stephen Lee-Urban, Yangfeng Ji, and Spencer Frazier. I would also like to wish my thanks to Jessica Celestine, Cynthia Bryant, and everyone in the School of Interactive Computing for their help and support during my study at Georgia Tech. Finally, I would like to thank my family for always being supportive and standing by me through good times and bad. I deeply thank for their love, support, and reassurances in all my pursuits. iv TABLE OF CONTENTS DEDICATION .................................. iii ACKNOWLEDGEMENTS .......................... iv LIST OF TABLES ............................... viii LIST OF FIGURES .............................. x SUMMARY .................................... xiii I INTRODUCTION ............................. 1 1.1 Research Questions and Intended Contributions...........5 1.2 Reader's Guide..............................8 II BACKGROUND AND RELATED WORK ............. 10 2.1 Narrative and Interactive Narrative.................. 10 2.2 Branching Story Graph......................... 12 2.3 Drama Manager............................. 16 2.3.1 Search-based and Declarative Optimization-based Drama Man- ager................................ 16 2.3.2 Planning-Based Drama Manager................ 20 2.3.3 Other Types of Drama Manager................ 21 2.3.4 Player Agency and Player Guidance in Interactive Narrative 23 2.4 Player Modeling............................. 24 2.4.1 Player Modeling in Computer Games............. 25 2.4.2 Player Modeling in Interactive Narrative........... 26 2.5 Collaborative Filtering and Its Application in Computer Games... 28 2.5.1 Collaborative Filtering..................... 28 2.5.2 Application in Computer Games................ 29 III DATA-DRIVEN PLAYER MODELING ALGORITHM ..... 31 3.1 Prefix Tree Representation....................... 32 3.2 Prefix Based Collaborative Filtering.................. 34 v 3.2.1 Collaborative Filtering Algorithms............... 36 3.2.2 Model Learning Algorithms................... 37 3.2.3 Story Recommendation Algorithms.............. 39 3.3 Evaluation of the Player Modeling Algorithm............. 41 3.3.1 Story Library.......................... 42 3.3.2 User Interface.......................... 44 3.3.3 Model Training on Human Players............... 45 3.3.4 Evaluation of Story Recommendation............. 47 3.3.5 Experiment on Human Players without Considering History. 49 3.3.6 Evaluation of Using Robin's Laws as Prior Knowledge.... 51 3.3.7 Experiment with Simulated Players.............. 52 3.4 Discussion and Conclusions....................... 57 IV PERSONALIZED DRAMA MANAGER ............... 61 4.1 Multi-option Branching Story Graph.................. 62 4.2 Player Option Preference Modeling.................. 64 4.3 Personalized Guidance Algorithm................... 66 4.4 Target Full-length Story Selection................... 67 4.4.1 Highest Rating Algorithm.................... 68 4.4.2 Highest Expected Rating Algorithm.............. 68 4.5 Personalized Drama Manager Algorithm................ 71 4.6 Evaluation of the Personalized DM................... 72 4.6.1 Stories and User Interface.................... 73 4.6.2 Training the Personalized Drama Manager.......... 75 4.6.3 Testing the Personalized Drama Manager........... 77 4.6.4 Player Agency Study...................... 81 4.7 Discussion and Conclusions....................... 84 V FURTHER IMPROVEMENT OF THE PERSONALIZED DRAMA MANAGER ................................. 87 5.1 Reduce the Requirement for the Amount of Training Data...... 87 vi 5.1.1 Impact of the Amount of Data Collection on Player Experience 87 5.1.2 Use Active Learning to Select Training Stories........ 90 5.2 Incorporate Author's Preference.................... 95 5.3 Conclusion................................ 97 VI CONCLUSIONS AND FUTURE WORK .............. 98 6.1 Summary................................. 98 6.2 Contributions and Potential Applications............... 99 6.3 Future Work............................... 101 6.3.1 Repeated Visits to the Same Branching Point......... 101 6.3.2 Building Player Models Using Other Features......... 102 6.4 Conclusion................................ 103 APPENDIX A | STORY LIBRARY USED IN THE HUMAN STUD- IES ....................................... 104 REFERENCES .................................. 130 vii LIST OF TABLES 1 The average RMSE for NMF and pPCA algorithms with different pa- rameters................................... 47 2 The average ratings for the random and personalized full-length stories. The accuracies are the percent of pairs in which the average rating of the personalized stories is higher than the average rating of the random stories.................................... 48 3 The experiment results of the comparison experiment without consid- ering history................................ 51 4 The RMSE for NMF algorithms with Robin's Laws prior on human player data................................. 52 5 The experiment results for the simulated players using several varia- tions of the DM algorithm......................... 55 6 The training results of the branch transition probability model using three probabilistic classification algorithms................ 76 7 The comparison of the personalized drama manager with three target selection algorithms............................ 79 8 The sensitivity analysis of the players' ratings in the bootstrapping phase as the numbers of discarded training stories change....... 80 9 The results for the player experience study. The significant compar- isons are mark with * (p value < 0.05).................. 84 10 The average response to the three boredom related questions for dif- ferent number of training stories..................... 89 11 The average response to the three boredom related questions for dif- ferent amount of feedback collected.................... 89 12 The comparison of averages of prefix and option ratings for different numbers of training stories........................ 90 13 The plot point and page number mappings for The Abominable Snowman.107 14 The plot point and page number mappings for The Lost Jewels of Nabooti.111 15 The plot point and page number mappings for Space And Beyond... 112 16 The plot point and page number mappings for Journey Under The Sea. 113 17 The options used in Figure 28....................... 115 18 The options used in Figure 28 (Continued)................ 116 viii 19 The options used in Figure 28 (Continued)................ 117 20 The options used in Figure 28 (Continued)................ 118 21 The options used in Figure 28 (Continued)................ 119 22 The options used in Figure 28 (Continued)................ 120 23 The options used in Figure 28 (Continued)................ 121 24 The options used in Figure 29....................... 122 25 The options used in Figure 29 (Continued)................ 123 26 The options used in Figure 29 (Continued)................ 124 27 The options used in Figure 29 (Continued)................ 125 28 The options used in Figure 29 (Continued)................ 126 29 The options used in Figure 29 (Continued)................ 127 30 The options used in Figure 29 (Continued)................ 128 31 The options used in Figure 29 (Continued)................ 129 ix LIST OF FIGURES 1 Illustration of the branching story graph for one of the CYOA books| The Abominable Snowman........................3 2 Aristotle's dramatic arc.......................... 11 3 A sample branching story graph..................... 12 4 The branching story tree representation for the branching story graph in Figure3................................