Interactivity in Game Audio: Instances in which Listening Improves a Player’s Gameplay A Dissertation Submitted to the Faculty of the University of Minnesota By Ryan Clarke Thompson In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy Advised by Peter Mercer-Taylor December 2017 Copyright Ryan Clarke Thompson, 2017. 2 Acknowledgements This dissertation, and my entire graduate career at the University of Minnesota, would not have been possible without an incredible amount of support from family, friends, and colleagues. First, above all, I must thank Addie Thompson for her unwavering support. In addition to helping with one portion of the third chapter directly, she continues to inspire and motivate me in all aspects of my life, and we shape each other far more than words on this page can fairly attest. Thanks to my parents Rod and Debbie for believing when my brother Tom (thanks for all the games growing up) and I said that all of this gaming would be useful someday. Both of us have done well in proving that true. Thanks also to John, Verlaine, and Amanda Hall for providing support and encouragement throughout graduate school, and for allowing me to ramble on and on about a subject that rarely reaches southwest Iowa. Everyone at the University of Minnesota has made my graduate school experience wonderful, and I have learned something important from every faculty member with whom I have interacted. Sumanth Gopinath continues to demonstrate a scholarly ideal that many chase; in addition, his own research on the intersections of music and technology made me believe that my own work was possible as ludomusicology was only beginning to get a foothold in the academy. It is my hope to someday read something he is yet to get his hands on. David Grayson is an incredible reminder that there is tremendous value in learning a broad range of investigative techniques. As I begin my own teaching, I i hope to be as effective a teacher (his source studies course in particular often enters my thoughts) and as gregarious a storyteller. Scott Lipscomb has long been an example of how someone pursuing scholarship in video games can thrive across multiple disciplines. He seems to have more than just two feet in the doors of far more than two fields of study. As someone who will be working outside of a music department, I will continue to appreciate him for providing a great example of how an interdisciplinary career can develop from the confines of a single office in a single building. My advisor, Peter Mercer-Taylor, has routinely placed far more trust and authority in me than I deserved. I was allowed to make big mistakes and occasionally fail spectacularly, only to be given a compassionate helping hand whenever I needed one. His outpouring of support of my family’s frankly ridiculous decision to raise two children in the middle of graduate school is the experience every graduate student deserves, and I am very lucky to have been his student. As someone writing the bulk of the dissertation away from my home campus, I have been especially supported by friends and colleagues on social media – often my only connection to fellow scholars and musicians. I could list almost every scholar I know here, but will limit myself to extending special thanks to Jim Buhler, Frank Lehman, Bryn Hughes, Julianne Grasso, Reba Wissner, Matt Brounley, Ashley Greathouse, and the anonymous Musicology Duck (whose secret is safe with me) for making the Internet a friendlier, funnier, and more scholarly place for me to inhabit. ii To the folks at OverClocked ReMix, thank you for providing a place where I could talk about video game music with other crazy awesome people. I couldn’t ask for better friends or belong to a better community of musicians. In particular, thanks to Andrew and Katie Luers, Andrew and Jillian Aversa, Stevo Bortz, Valerie Legendre, Jarel Jones, Shariq Ansari, Nabeel Ansari, Aaron Wu (who helped with the appendix), Jeremy Waters, Amy Hsieh, Larry Oji, and David Lloyd. Lastly, I’d be remiss to not extend a special thank you to Will Gibbons, Steven Reale, and Dana Plank. Coordinating and leading our joint presentation in San Francisco was the high point of my graduate school experience, and presenting at GDC made me believe that I could really become a scholar and leader across the multiple disciplines that combine and intersect to form ludomusicology. I look forward to every time we cross paths, and can’t wait to see what each of you do next. iii Dedication for Owen and Oliver, in the hopes of demonstrating the value of telling stories. iv Table of Contents List of Tables…………………………………………….vi List of Figures…………………………………………... vii Introduction………………………………………………1 Chapter 1: Identification, Identity, and Spatial Awareness in Valve’s Left 4 Dead………………………………………11 Chapter 2: Using Sonic Domains to Communicate to Spectators in Competitive Multiplayer Games………………..49 Chapter 3: Empirically Measuring the Impact of Game Audio on Player Performance……………………………..93 Chapter 4: Using Video Games to Understand Musical Performance………………………………………117 Conclusion……………………………………………….155 Bibliography……………………………………………...163 Appendix……………………………………………........167 v List of Tables Table 1: Relationships Between Spawning Motives in Left 4 Dead 2. Page 34. Table 2: Musical Effects of Gameplay on Super Mario Maker. Page 113. vi List of Figures Figure 1.1. Loading screens from Left 4 Dead. Page 18. Figure 1.2. Transcription of “At the Gravesite” from Night of the Living Dead. Page 21. Figure 1.3. Transcription of title music from Left 4 Dead. Page 22. Figure 1.4. Transcription of “Tower Block” from 28 Days Later. Page 23. Figure 1.5. Scale utilized in composition of Left 4 Dead as described by Mike Morasky. Page 24. Figure 1.6. Transposition of Figure 1.5 to match the score of 28 Days Later. Page 24. Figure 1.7. Transcription of Shade Man’s theme from Mega Man 7. Pages 26-27. Figure 1.8. Transcription of “Bloody Tears” from Castlevania II: Simon’s Quest. Page 27. Figure 1.9. Transcription of More Rooms from Resident Evil. Page 29. Figure 1.10. Transcription of scale used in composing Left 4 Dead. Page 30. Figure 1.11. Left 4 Dead scale rendered as two overlapping octatonic scale fragments. Page 31. Figure 1.12. Hunter theme from Left 4 Dead. The Hunter pounces on players, pinning them to the ground. Page 32. Figure 1.13. Jockey theme from Left 4 Dead 2. The Jockey jumps on players and directs their movements. Page 32. Figure 1.14. Smoker theme from Left 4 Dead. The Smoker ensnares players from a distance. Page 32. Figure 1.15. Spitter theme from Left 4 Dead 2. The Spitter lobs corrosive acid from long range. Page 32. Figure 1.16. Boomer theme from Left 4 Dead. The Boomer vomits on players, blinding them before exploding near players. Page 33. Figure 1.17. Charger theme from Left 4 Dead 2. The Charger lunges at players, running them into walls and off cliffs if he grabs them. Page 33. vii Figure 1.18. Instruction manual from Super Mario Bros. 2. Page 39. Figure 1.19. Screenshot of ending to Super Mario Bros. 2. Page 40. Figure 1.20. Internet meme describing Super Mario Bros. 3 as a stage play. Page 41. Figure 1.21. Tank theme from Left 4 Dead. Page 44. Figure 2.1. Second screen of Super Mario Bros. stage 1-1. Page 53. Figure 2.2a, b. Before and after hitting the first invisible block in Super Mario Bros. Page 54. Figure 2.3. Screen capture of stage select screen from Mega Man 3. Page 56. Figure 2.4. Transcription of “Horizon of Light and Shadow,” from Bravely Default. Page 60. Figure 2.5. Transcription of “Main Theme,” from Final Fantasy. Page 62. Figure 2.6. Transcription of “Eternal Wind,” from Final Fantasy III. Page 63. Figure 2.7. Transcription of “Main Theme,” from Final Fantasy II. Page 65. Figure 2.8. Transcription of Theme of Vale, from Golden Sun. Page 66. Figure 2.9. Transcription of “Kids Run Through the City Corner,” from Final Fantasy VI. Page 67. Figure 2.10. Transcription of Issac’s battle theme, from Golden Sun. Pages 69-70. Figure 2.11. Transcription of Jenna’s combat theme, from Golden Sun: The Lost Age. Page 72. Figure 2.12. “The Decisive Battle,” from Final Fantasy VI. Pages 73-75. Figure 2.13. Sound effect accompanying the “Cura” spell, from Final Fantasy VI. Page 75. Figure 3.1. Eight images from the same room of Mega Man 5, representing the endless sequence of appearing blocks in the first Proto Man stage. Pages 97-98. viii Figure 3.2. The arrow marks Mario’s x-axis position upon downbeat (here, at the point of departure for a required jump) of background music for the musically synchronized version of the Overworld stage. Page 104. Figure 3.3. The arrow marks Mario’s x-axis position upon the downbeat of the background music for the musically synchronized version of the Castle stage. Page 105. Figure 3.4. Sequence of multiple jumps in quick succession for both Overworld (above) and Castle (below) stages. Page 106. Figure 3.5. Both of the two “difficult” challenges require players to trust that the coins will guide them to safety over their own instinctual desires to leap to safety. Page 108. Figure 3.6. Final data from Table 2 expressed in a bar plot. Page 114. Figure 4.1. Sound Shapes screenshot. Page 123. Figure 4.2. Sound Shapes. Image from the “Cities” stage. Page 124.