NASA/TM_2000-209606 3-D Sound for Virtual Reality and Multimedia Durand R. Begault Ames Research Center, Moffett Field, California National Aeronautics and Space Administration Ames Research Center Moffett Field, California 94035 August 2000 Acknowledgments Without the assistance and support of Justine, Pat, Denise, Chloe, Nigel, tre, Bill, Lars, Joel, Rick, Dr. B, Jenifer, Reuben, and many others, the completion of this book would not have been possible. About the Cover Cylindrical Surface Plot of the Head-Related Transfer Function: Magnitude Response as a Function of Frequency over Azimuth Angle on a Radial Axis, by William Martens of E-mu/Creative Technology Center. The cover photograph shows a visualization of the magnitude response (gain) of the head-related transfer function (HRTF) measured at the eardrum position of the anthropomorphic mannequin KEMAR. HRTFs were measured for 19 loudspeaker directions circling the side of the head facing the loudspeaker placed at ear level. The surface was constructed by interpolating the gain within each of 50 log-spaced frequency bands for the 19 HRTFs using a bicubic spline. The lowest band was centered on 55 Hz, the highest on 21,331 Hz. The distance of the surface from the origin and the color indicates the gain at a particular frequency and azimuth, which ranges from blue-black at the lowest gain (-43.9 dB) to a desaturated yellow at the peak gain (14.6 dB). Available from: NASA Center lbr AeroSpace Information National Technical Information Service 7121 Standard Drive 5285 Port Royal Road Hanovcr, MD 21076-1320 Springfield, VA 22161 301-621-0390 703-605-6OO0 Table of Contents Preface to the NASA Technical Memorandum version ................................................................... ix Preface .............................................................................................................................................. x Overview ...................................................................................................................................... xii CHAPTER ONE .............................................................................................................................. 1 Virtual Auditory Space: Context, Acoustics, and Psychoacoustics .................................................. 1 Context .......................................................................................................................................... I Source-Medium-Receiver Model: Natural versus Virtual Spatial Hearing ........................................ 4 Application Types ............................................................................................................................. 7 Virtual Audio: A Special Case ...................................................................................................... 1 I Components .............................................. •.................................................................................. 12 So What Is 3-D Sound Good For? ................................................................................................ 13 Surround versus 3-D Sound ......................................................................................................... 16 Characteristics of Sound Sources .................................................................................................... 17 Describing Waveforms ................................................................................................................. 17 Periodic and Aperiodic Waveforms .............................................................................................. 17 Digital Sound and Signal Processing ............................................................................................ 20 Harmonics ................................................................................................................................... 20 Fourier Analysis ........................................................................................................................... 21 Amplitude Envelope .................................................................................................................... 25 Perception ....................................................................................................................................... 25 Psychoacoustics and Applications ................................................................................................ 25 Perceptual Correlates of Frequency, Intensity, and Spectral Content ............................................. 27 Cognition ..................................................................................................................................... 29 CHAPTER TWO ........................................................................................................................... 31 Overview of Spatial Hearing Part I: Azimuth and Elevation Perception ..................................... 31 Interaural Time and Intensity Cues ............................................................................................... 31 Lateralization ........................................................................................................................... 3 i Physical Basis of Lateralization ................................................................................................ 32 ITD Envelope Cue .................................................................................................................... 33 Perception of Lateralization ..................................................................................................... 34 The Precedence Effect ................................................................................................................. 36 ITD, liD, and Barn Owls: A Neurological Processing Model ......................................................... 37 Head Movement and Source Movement Cues ................................................................................ 39 Head Movement ........................................................................................................................... 39 Moving Sound Sources ................................................................................................................ 40 Spectral Cues Provided by the Pinnae ............................................................................................ 40 Ambiguous ITD and liD Cues ................................................... .................................................. 40 The Head-Related Transfer Function ............................................................................................ 41 A Do-it- Yourself Experiment with Spectral Modification ........................................................... 44 45 HRTF Magnitude Characteristics ........................................................................................ _.... HRTF Phase Characteristics ...................................................................................................... 45 Localization with HRTF Cues ............................................ •.......................................................... 49 Spectral Cues Provided by the HRTF ........................................................................................ 49 Spectral Band Sensitivi_. .......................................................................................................... 52 Localization of Actual Sound Sources with HRTF Cues ............................................................ 53 Localization of Azimuth and Elevation of Virtual Sources ......................................................... 5 7 Nonindividualized HRTFs ......................................................................................................... 57 V Reversals .................................................................................................................................. 62 CHAPTER THREE ........................................................................................................................ 69 Overview of Spatial Hearing Part 11: Sound Source Distance and Environmental Context ......... 69 3-D Sound, Distance, and Reverberation ....................................................................................... 69 Distance Cues .................................................................................................................................. 69 Intensity, Loudness Cues ......................................................................... ..................................... 70 Influence of Expectation and Familiarity ..................................................................................... 76 Spectral and Binaural Cues to Distance ......................................................................................... 76 Spectral Changes and Sound Source Distance ......................................................................... 77 Frequency Dependent Attenuation at Large Distances .............................................................. 78 Binaural Cues and Intensity; Auditory. Parallax ....................................................................... 79 Inside-the-Head Localization: Headphone Distance Error ............................................................ 79 Reverberation ................................................................................................................................