INTERACTIVE AND ADAPTW AUDIO FOR HOME VIDEO GAME CONSOLES Jonathan B. Fish M.A., Simon Fraser University, 2003 THESIS SUBMllTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS (COMMUNICATION) in the School of Communication O Jonathan B. Fish 2003 SIMON FRASER UNIVERSITY July 2003 All rights reserved. This work may not be reproduced in whole or in part, by photocopy or other means, without permission of the author. APPROVAL NAME: Jonathan Fish DEGREE: MA TITLE OF THESIS: Interactive and Adaptive Audio for Home Video Game Consoles EXAMINING COMMITTEE: CHAIR: Prof-Gary McCarron -/ Professor, School of Communication, SFU Prof. Stephen Kline Supevisor Professor, School of Communication, SFU Prof. Kenneth Newby, Examiner Assistant Professor, Interactive Arts, Computing Arts and Design Sciences, SFU Surrey Date: &b a6'/0I PARTIAL COPYRIGHT LICENSE I hereby grant to Simon Fraser University the right to lend my thesis, project or extended essay (the title of which is shown below) to users of the Simon Fraser University Library, and to make partial or single copies only for such users or in response to a request from the library of any other university, or other educational institution, on its own behalf or for one of its users. I further agree that permission for multiple copying of this work for scholarly purposes may be granted by me or the Dean of Graduate Studies. It is understood that copying or publication of this work for financial gain shall not be allowed without my written permission. Title of Thesis/Project/Extended Essay Interactive and Adaptive Audio for Home Video Game Consoles TJ, 27/03 (date) ABSTRACT The video game industry, more suitably referred to as interactive entertainment, serves to immerse players into a fantasy universe for an engaging, virtual, participatory experience. While visuals have been the focus of previous studies of the video game medium, recent advances in video game audio demonstrate that game sound is no longer merely an underlying component in crafting the overall, interactive experience. The thesis analyzes the interactive nature of video game audio, with its real-time audio tools that allow audio components to adapt and conform to random gameplay actions taken by the individual player. Specific games from the different genres of First-Person Shooter, Sports Simulator, and Fantasy Sports games are studied using soundscape study concepts to determine how interactive and adaptive audio fimctions spanning different interactive game audio constructs. The methodology draws its data from multiple sources, including personal observations, literature surveys, case studies and interviews with professional video game sound designers. The dissertation brings together the discrete areas of soundscape studies (including acoustic and psychoacoustic concepts), video-game audio technology, and theories of interactivity in order to lay the groundwork for the audio-textual analyses of the games examined. The thesis argues that interactive audio, within the immersive context of the virtual gaming environment, creates a feedback cycle which serves to empower the participant, and that the surrogate interactive soundscapes of video game environments become vehicles that make the interactive partnership between user and machine possible. The study shows that a communicational approach in soundscape studies can be readily applied to video-game soundscapes similar to its use in deconstructing sonic environments in the real world. The role of audio interactivity in immersing the player is analyzed by isolating the sonic elements within the virtual sonic environments inherent in specific games, analyzing their functional role within the gaming context, and documenting the experiences of professional sound designers obtained during interviews. Dedicated to my father, Dr. William W. Fish ACKNOWLEDGEMENTS I would like to thank all those whose invaluable support made this thesis possible. Without the help of those listed, this labour of love could not have come to completion. First and foremost, I wish to extend my sincerest gratitude to my mentor, Barry Truax, for his ongoing support, scrupulous supervision, expert advice, and insightful knowledge. I have been privileged to work with Barry, who has been a meticulous editor, but also whose patience and friendly support will never be forgotten. A special thanks to David Murphy, whose technical assistance (as well as moral support) served to make the presentation of this thesis possible. Thanks to Aaron Marks for his support in granting me permission to incorporate from his inspirational and invaluable text, The Complete Guide to Game Audio. I wish to express my gratitude to Kenneth Newby for hls feedback and insight, and for a thought-provoking discussion. Thanks to Stephen Kline for his guidance prior to the thesis in helping me explore the area of interactive entertainment media, and the issues and debates surrounding the vibrant medium of video games. I would also like to acknowledge the friendly fhculty at the School of Communication, as well as the rest of the helphl SFU staff for their services. I wish to express my sincerest appreciation to the sound artists at Electronic Arts Inc., for sharing their experiences and expertise, and for giving so selflessly of their time. I wish to also recognize Electronic Arts Canada headquarters in Burnaby for approving the NDA, making the interviews in this study possible. Finally, I would like to show appreciation to my fiiends and colleagues who have supported me along the way, especially my beautiful, loving girlfriend, Joanne, for helping me find my motivation when suffering from writer's block, or just being there as a shoulder to cry on. Thanks to my wonderful hily, for helping me bear the weight of this undertaking. Last but not least, thank you Dad, for being my inspiration and for always believing in me. I couldn't have done this without you, and I will always remember your strength and dedication as the driving forces motivating me to persevere. We did it, Dad! TABLE OF CONTENTS .. Approval ............................................................................................................................. 11 ... Abstract ..............................................................................................................................111 Dedication ........................................................................................................................... iv Acknowledgments ............................................................................................................... v Table of Contents ................................................................................................................ vi Chapter I . An Interactive Medium ................................................................................... 1 1.1 Video Games .................................................................................................... 5 1.1.1Video Game Audio: 'We've Heard It All Before ...Haven't We? ....... 5 1.1.2 Video Game Audio: An Aural History .............................................. 6 1.1.3 At Present: "Hear We Are" ................................................................ 12 1.2 Interactivity & Immersion ................................................................................. 13 Chapter I1 .Sound: An Ear-Opening Experience ............................................................. 23 2.1 A "Sound" Approach ......................................................................................... 23 2.2 Speech - Music - Soundscape ............................................................................27 2.2.1 Speech: More than Just Hearsay ........................................................ 27 2.2.2 Music: 'Organized Sound' .................................................................. 28 2.2.3 Soundscape: Keeping an Ear to the Ground ........................................ 30 2.3 Psychoacoustics & Auditory Perception: 'Clairaudience'. ................................... 34 Chapter I11 . Interactive Audio in Video Games: More than Meets the Ear.................... 40 3.1 Video Game Audio: Genres .............................................................................. 40 3.1.1 First Person Shooter Games ............................................................... 41 3.1.2 Graphic Adventure Games ................................................................. 42 3.1.3 Role-Playing Games .......................................................................... 43 3.1.4 Sports Games .................................................................................... 44 3.1.5 Music Simulation Games ................................................................... 45 3.2 Video Game Audio: Home Console Systems .....................................................46 3.2.1 Sony Playstation 2 ...................................................................... 47 3.2.2 Nintendo GameCube .........................................................................48 3.2.3 Microsoft Xbox ................................................................................. 49 3.3 Technological Advances in Video Game Audio: Loud & Clear ......................... 50 3.3.1 Linear Audio ..................................................................................... 50 3.3.2 Interactive & Adaptive Audio ............................................................56 Chapter IV . Principles of Design: