Title page Submitted in fulfilment of the requirements for the Doctor of Philosophy Adelaide, January 2020 University of Adelaide Elder Conservatorium of Music Thesis Video Game Acoustics: Perception-Based Sound Design for Interactive Virtual Spaces Submitted by Benjamin Kym Probert 1146412 Table of Contents AbstractAbstract I DeclarationDeclaration II AcknowledgementsAcknowledgements III ExampleExample Videos Videos Link Link III List ofList Figures of Figures IV List ofList Tables of Tables VI ChapterChapter 1 Introduction 1 Introduction 1 1 1.1 Theoretical Background 5 1.1 Theoretical Background 5 1.2 Methodology 8 1.2 Methodology 7 1.3 Original Contribution 11 1.3 Original Contribution 10 1.4 Thesis Structure 13 1.4 Thesis Structure 12 Chapter 2 Literature Review 14 2.1Chapter Video 2 Game Literature Acoustics Review 15 13 2.2 2.1Sound Video Theory game Acoustics 26 14 2.3 2.2Perception Sound Theory of Acoustics 31 24 2.4 2.3Conclusion Perception of Acoustics 35 28 2.4 Conclusion 32 Chapter 3 Video Game Acoustics and Sound Design 37 3.1Chapter Reverberation 3 Video Game Acoustics and Sound Design 39 34 3.2 3.1Sound Reverberation Propagation 56 36 3.3 3.2Spatial Sound Signatures Propagation 87 53 3.4 3.3Camera Spatial Perspective Signatures 97 84 3.5 3.4Pre -renderedCamera Perspective Acoustics 105 94 3.6 3.5Dialog Pre and-rendered Voice Acoustics 110 101 3.7 3.6Sound Dialog Medium and Voice 120 106 3.8 3.7User Sound Space Medium 127 116 3.9 3.8Conclusion User Space 144 123 3.9 Conclusion 139 Chapter 4 Perceptual Functions of Video Game Acoustics 146 4.1Chapter Sound 4 Perception Perceptual and Functions Simulation of Video Game Acoustics147 141 4.2 4.1Reinforcement Sound Perception and Simulation 151 142 4.3 4.2Distance Reinforcement and Power 159 146 4.4 4.3Spatial Distance Presence and Power 170 154 4.5 4.4Materialisation Spatial Presence 182 165 4.6 4.5Conclusion Materialisation 191 177 4.6 Conclusion 186 Chapter 5 Conceptual Frameworks 193 5.1Chapter Sound 5 Power Conceptual Framework Frameworks 195 188 5.2 5.1Player Sound Circumstances Power Framework Framework 201 190 5.3 5.2Voice Player Acoustics Circumstances Framework Framework 209 196 5.4 5.3Coupled Voice Rooms Acoustics Framework Framework 215 204 5.5 Conclusion 227 5.4 Coupled Rooms Framework 210 5.5 Conclusion 222 Chapter 6 Conclusion 230 Chapter 6 Conclusion 223 Reference List 240 GlossaryReference List 250 233 Glossary 242 Abstract Video game acoustics are the various aspects of sound physics that can be represented in a video game, as well as the perception and interpretation of those sound physics by a player. At its core, the research here aims to identify the many functions and considerations of acoustics in interactive virtual spaces, while also building a theoretical foundation for video game acoustics by gathering relevant research from a wide variety of disciplines into a single video game context. The writing here also functions as an informative resource for video game sound designers and is primarily written for that audience. Through a review of the literature it is found that there is research available across many different disciplines that is relevant to video game acoustics, but none that bring it all together and fully explore acoustics in a video game context. Small discussions related to the topic occur sporadically throughout various fields, however there are few of any detailed focus and even fewer with video game sound designers as their intended audience. This scattering and dilution of relevant information validates the need for its distillation into a dedicated discussion. The writing here addresses this gap in the literature and in doing so uncovers aspects of video game acoustics that have not previously been given adequate attention. This thesis accomplishes its aims by combining an interdisciplinary background with an emphasis on simplification to suit the creative field of game sound design. A theoretical foundation is built from several different disciplines, including Acoustics, auditory perception, acoustic simulation, sound theory, spatial presence, film sound, and of course game sound. A twofold physics/perception approach is used to analyse video game acoustics. The human perception of sound has various strengths and weaknesses, which help to identify the aspects of sound physics that are important to provide a player as well as aspects that may be ignored for efficiency reasons. The thesis begins by revealing the many considerations and implications of incorporating acoustics into a video game, followed by an exploration of the perceptual functions of acoustics in virtual spaces. Several conceptual frameworks are then offered to address some of the problems discovered in the previous sections. By the end of the thesis it will be shown that the main purpose of video game acoustics is to provide a player with a natural experience of sound. People working in the video game industry may use the research presented here to cultivate an understanding of how humans can interact with video games through sound physics, and why it is important to improve the quality of this interaction. i Declaration I certify that this work contains no material which has been accepted for the award of any other degree or diploma in my name, in any university or other tertiary institution and, to the best of my knowledge and belief, contains no material previously published or written by another person, except where due reference has been made in the text. In addition, I certify that no part of this work will, in the future, be used in a submission in my name, for any other degree or diploma in any university or other tertiary institution without the prior approval of the University of Adelaide and where applicable, any partner institution responsible for the joint-award of this degree. I give permission for the digital version of my thesis to be made available on the web, via the University’s digital research repository, the Library Search and also through web search engines, unless permission has been granted by the University to restrict access for a period of time. I acknowledge the support I have received for my research through the provision of an Australian Government Research Training Program Scholarship. Benjamin Kym Probert II Acknowledgements Thanks to my supervisors Christian Haines and Stephen Whittington for their support and guidance. Also, thanks to Kimi Coaldrake and Charles Bodman Rae for logistical and bureaucratic assistance. Special thanks to Kym Probert for letting his adult son “temporarily” move back home. And thanks to Theo, Toby, Joel, Chris, and Bella. Example Videos Link tiny.cc/VGAC or www.youtube.com/channel/UCJakBVfThrKJgrDSL2hFdOQ/videos Headphones must be used. iii List of Figures 1. A single sound wave in an enclosed space increasing in complexity over time. Page 2 2. The relative timing of a sound wave passing a point while being reflected inside a space. 39 3. Direct sound. 40 4. Early reflections coming from various directions. 41 5. Occlusion of a sound source. 56 6. Direct sound being obstructed by an object while a reverb effect remains unchanged. 59 7. An obstruction that could be incorrectly presented as an occlusion. 60 8. A top-down view of a sound source and a listener in different rooms that are connected by an 67 opening (a doorway). 9. A sound source and a listener, each with their own reverb element in their respective rooms. 68 10. The sound source is heard by the listener directionally from the doorway, and the angle 69 between the two affects the frequency spectrum. 11. The level of the sound is affected by the total distance travelled. 69 12. The sound source's reverb is heard directionally from the doorway, with the level depending on 70 the listener's distance from the doorway. 13. The D:R ratio functioning as a cue to the distance between the sound source and doorway. 71 Overall level changes depending on the listener’s distance from doorway. 14. The sound coming through the doorway undergoes Sound Power Reduction (SPR) before being 73 reverberated in the listener’s room. 15. A sound source in the listener's room undergoes Sound Power Reduction (SPR) before being 75 reverberated in a neighbouring room. The subsequent reverberation is heard by the listener directionally from the doorway. 16. A sound source inside a warehouse being heard as occluded by a listener on the outside. The 79 warehouse’s reverberation is heard directionally from the warehouse. 17. A sound source outside a warehouse being heard as occluded by a listener on the inside. The 79 outside’s reverberation would sound enveloping, as the listener is technically still ‘inside’ it. 18. A listener leaves the warehouse on the same side as the sound source. The direct sound and 79 outdoor reverberation are heard normally. 19. A listener leaves the warehouse through an opening facing away from the sound source. The 79 outdoor reverberation is heard normally but the direct sound is obstructed. 20. A sound source with directivity. The different shades could represent differences in level or 83 frequency spectrum, or both. 21. A player in the centre of a room, with omnidirectional and enveloping late reverberation. 90 22. A player in a corner of a room, with slightly directional late reverberation. 90 23. A player in a hallway, with directional late reverberation. 91 24. A clean unaffected sound (top), and the same sound with pre-rendered reverb applied 106 (bottom). 25. Size constancy and sound power constancy compared. Interpreted size remains consistent with 165 real size in spite of retinal projection. Similarly, interpreted sound power remains consistent with real sound power in spite of loudness at the ear.