The Impact of Multichannel Game Audio on the Quality of Player Experience and In-Game Performance
Total Page:16
File Type:pdf, Size:1020Kb
The Impact of Multichannel Game Audio on the Quality of Player Experience and In-game Performance Joseph David Rees-Jones PhD UNIVERSITY OF YORK Electronic Engineering July 2018 2 Abstract Multichannel audio is a term used in reference to a collection of techniques designed to present sound to a listener from all directions. This can be done either over a collection of loudspeakers surrounding the listener, or over a pair of headphones by virtualising sound sources at specific positions. The most popular commercial example is surround-sound, a technique whereby sounds that make up an auditory scene are divided among a defined group of audio channels and played back over an array of loudspeakers. Interactive video games are well suited to this kind of audio presentation, due to the way in which in-game sounds react dynamically to player actions. Employing multichannel game audio gives the potential of immersive and enveloping soundscapes whilst also adding possible tactical advantages. However, it is unclear as to whether these factors actually impact a player’s overall experience. There is a general consensus in the wider gaming community that surround-sound audio is beneficial for gameplay but there is very little academic work to back this up. It is therefore important to investigate empirically how players react to multichannel game audio, and hence the main motivation for this thesis. The aim was to find if a surround-sound system can outperform other systems with fewer audio channels (like mono and stereo). This was done by performing listening tests that assessed the perceived spatial sound quality and preferences towards some commonly used multichannel systems for game audio playback over both loudspeakers and headphones. There was also a focus on how multichannel audio might influence the success of a player in a game, based on their in-game score and their navigation within a virtual world. Results suggest that surround-sound game audio is preferable over more regularly used two-channel stereo systems, because it is perceived to have higher spatial sound quality and there is an improvement in player performance. This illustrates the potential for multichannel game audio as a tool to positively influence player experiences, a core goal many game designers strive to achieve. 3 Contents Abstract 2 Table of Contents 3 List of Tables 9 List of Figures 10 Acknowledgements 13 Declaration of Authorship 14 1 Introduction 15 1.1 Statement of Hypothesis . 17 1.2 Description of Hypothesis . 17 1.3 Statement of Ethics . 18 1.4 Thesis Structure . 18 1.5 Contributions to the Field . 20 2 Concepts for Spatial Hearing 22 2.1 The Transmission of Sound . 22 2.2 The Human Hearing System . 24 2.3 Sound Intensity and Sound Pressure . 26 2.4 The Frequency Domain . 28 2.5 Room Acoustics . 29 2.6 Binaural hearing . 33 2.6.1 Interaural Time Difference (ITD) . 34 2.6.2 Interaural Level Difference (ILD) . 36 2.6.3 Spectral Cues . 37 4 2.6.4 Head-related Transfer Function (HRTF) . 38 2.7 Summary . 42 3 Multichannel Audio Playback 43 3.1 Mono . 44 3.2 Stereo . 44 3.3 Surround-sound . 47 3.3.1 Surround-sound codecs . 50 3.3.2 Stereo Mix-down . 52 3.4 Virtual Home Theatre (VHT) . 53 3.5 Ambisonics . 54 3.6 Spatial Attributes . 55 3.6.1 Distance . 56 3.6.2 Localisation . 56 3.6.3 Depth . 57 3.6.4 Width . 58 3.6.5 Envelopment . 58 3.7 Summary . 59 4 Multichannel Video Game Audio 61 4.1 A History of Multichannel Audio in Gaming . 61 4.2 Observations on Multichannel Game Audio . 67 4.2.1 Formats . 68 4.2.2 Perspective . 70 4.2.3 Use of Centre Channel . 73 4.2.4 Music Panning . 75 4.3 Video Game Content Case Studies . 76 4.3.1 Game Selection Criteria . 77 4.3.2 The Last of us: Remastered . 78 4.3.3 Alien: Isolation . 80 4.3.4 P.T. 83 4.3.5 Ratchet and Clank: Tools of Destruction . 85 4.4 Summary . 87 5 5 Determining the Player Experience 89 5.1 Quality of Experience . 90 5.1.1 QoE Definition . 90 5.1.2 Utilitarian QoE assessment . 91 5.1.3 Analytic QoE assessment . 93 5.1.4 QoE test framework . 94 5.2 Influencing Factors on QoE . 95 5.2.1 Audio as an influence on QoE . 97 5.2.2 Spatial audio as an influence on audio quality . 98 Loudspeaker Rendering . 100 Headphone Rendering . 101 5.2.3 Game Audio Quality . 102 5.3 Summary . 103 6 Perceived Spatial Quality and Player Preferences 104 6.1 Research Question . 105 6.2 Pilot Study . 105 6.2.1 Method . 106 6.2.2 Pilot Study Design . 106 Participants . 106 6.2.3 Pilot Study Procedure . 107 Materials . 107 Questionnaire . 109 6.2.4 Pilot Study Analysis of Results . 111 Perceived Spatial Quality . 112 Preference . 112 6.2.5 Pilot Study Discussion . 114 6.3 Main Experiment . 115 6.3.1 Method . 115 6.3.2 Experimental Design . 115 Participants . 117 6.4 Experimental Procedure . 117 6.5 Results Analysis . 118 6 6.5.1 Overall spatial sound quality . 119 6.5.2 Individual attribute scores comparison . 119 6.5.3 Preference . 122 6.6 Discussion . 123 6.7 Summary . 126 7 Headphone Based Audio Rendering and Player Preferences 128 7.1 Research Question . 129 7.2 Method . 129 7.3 Experimental Design . 130 7.3.1 Participants . 130 7.4 Experimental Procedure . 131 7.4.1 Materials . 131 VHT Rendering Process . 132 Stereo Mix-down Process . 134 7.4.2 Questionnaire . 135 7.5 Analysis of Results . 135 7.6 Discussion . 137 7.7 Summary . 140 8 The Impact of Multichannel Audio on Player Performance 141 8.1 Research Question . 143 8.2 Method . 143 8.3 Experimental Design . 144.