Calhoun: The NPS Institutional Archive Theses and Dissertations Thesis Collection 1998-09-01 Facilitating rich acoustical environments in virtual worlds Hoag, Kenneth J. Monterey, California. Naval Postgraduate School http://hdl.handle.net/10945/8386 DUDLEY KNOX LIBRARY NAVAL POSTGRADUATE SCHOOL MONTFPFY CA 93943-5101 NAVAL POSTGRADUATE SCHOOL Monterey, California THESIS FACILITATING RICH ACOUSTICAL ENVIRONMENTS IN VIRTUAL WORLDS by Kenneth J. Hoag Sr. September 1998 Thesis Advisor: Rudolph Darken Second Reader: Russell Storms Approved for public release; distribution is unlimited DOCUMENTATION PAGE Form Approved REPORT OMBNo. 0704-0188 Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instruction, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302, and to the Office of Management and Budget, Paperwork Reduction Project (0704-0188) Washington DC 20503. 1. AGENCY USE ONLY (Leave blank) 2. REPORT DATE 3. REPORT TYPE AND DATES COVERED September 1998 Master's Thesis 4. TITLE AND SUBTITLE 5. FUNDING NUMBERS Facilitating Rich Acoustical Environments in Virtual Worlds 6. AUTHOR(S) Hoag Sr., Kenneth J. 8. PERFORMING ORGANIZATION REPORT 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) NUMBER Naval Postgraduate School Monterey, CA 93943-5000 9. SPONSORING /MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSORING / MONITORING AGENCY REPORT NUMBER 11. SUPPLEMENTARY NOTES The views expressed in this thesis are those of the author and do not reflect the official policy or position of the Department of Defense or the U.S. Government. 12a. DISTRIBUTION / AVAILABILITY STATEMENT 12b. DISTRIBUTION CODE Approved for public release; distribution is unlimited. 13. ABSTRACT (maximum 200 words) The visual aspect of virtual environments has advanced at a rapid pace. The audio aspect, however, has not kept pace. Current methods of building virtual models do not address the graphical and audio aspects in an integrated fashion. Furthermore, graphical programming tools have not addressed sound in a satisfactory manner. As proof of concept, a modeling tool was developed to allow a user to build both the visual and the auditory environment simultaneously. A rendering application was developed that would display and browse a graphical environment, an audio environment, or a complete graphical/audio environment. This thesis demonstrates that building both the auditory and the visual geometry simultaneously allows for rapid, easy development of both the visual and the auditory environment Enhancements and recommendations to current software technologies and modeling languages are introduced. New models to represent audio are introduced. 14. SUBJECT TERMS 15. NUMBER OF PAGES Virtual Audio, Virtual Environment, 3-D Audio, Spatialized Sound, Audio Environment 97 16. PRICE CODE 17. SECURITY 18. SECURITY CLASSIFICATION 19. SECURITY CLASSIFICATION 20. LIMITATION OF ABSTRACT CLASSIFICATION OF REPORT OF THIS PAGE OF ABSTRACT Unclassified Unclassified Unclassified UL NSN 7540-01-280-5500 Standard Form 298 (Rev. 2-89) Prescribed by ANSI Std. 239-18 Approved for public release; distribution is unlimited FACILITATING RICH ACOUSTICAL ENVIRONMENTS IN VIRTUAL WORLDS Kenneth J. Hoag Sr. Captain, United States Marine Corps B.S., University of Southwestern Louisiana, 1986 Submitted in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE IN COMPUTER SCIENCE from the NAVAL POSTGRADUATE SCHOOL September 1998 DUDLEY KNOX LIBRARY NAVAL POSTGRADUATE SCHOOL MONTEREY CA 93943-5101 ABSTRACT The visual aspect of virtual environments has advanced at a rapid pace. The audio aspect, however, has not kept pace. Current methods of building virtual models do not address the graphical and audio aspects in an integrated fashion. Furthermore, graphical programming tools have not addressed sound in a satisfactory manner. As proof of concept, a modeling tool was developed to allow a user to build both the visual and the auditory environment simultaneously. A rendering application was developed that would display and browse a graphical environment, an audio environment, or a complete graphical/audio environment. This thesis demonstrates that building both the auditory and the visual geometry simultaneously allows for rapid, easy development of both the visual and the auditory environment. Enhancements and recommendations to current software technologies and modeling languages are introduced. New models to represent audio are introduced. VI 1 TABLE OF CONTENTS I. INTRODUCTION 1 A. MOTIVATION 1 B. RESEARCH OBJECTIVES 2 C. SCOPE 3 D. ASSUMPTIONS 4 E. THESIS ORGANIZATION 4 H. BACKGROUND 7 A. FUNDAMENTALS OF SOUND 7 1. Pitch 8 2. Loudness 8 3. Timbre 9 B. PSYCHOACOUSTICS 11 1. How do we hear sound differences? 1 2. How does the ear hear the pitch? 1 3. How do we organize sounds? 12 C. PRIMARY LOCALIZATION CUES 13 D. SOUND DIFFUSION 15 E. VIRTUAL SOUND 16 F. AUDIO ENGINEERING 17 G. DSP HARDWARE 17 1. Audio Processor Features of the SGI Indigo 17 2. Creative Labs 18 3. Aureal family of sound cards 19 4. Acoustetron 20 H. WHAT IS IMPORTANT? 20 m. REAL TIME SOUND/PREVIOUS WORK 23 A. RELATED RESEARCH 23 1. Implementing 3D sound into specific virtual environments 23 2. Real-time sound generation 24 B. DSP APPLICATION PROGRAMMER'S INTERFACES 25 1. DirectSound 25 2. A3D 26 3. A2D 26 4. A3D2.0 26 5. Environmental Audio/EAX 27 6. SGI Audio Library 27 C. FAMILIES OF MODELING AND RENDERING TOOLS 28 1. Coryphaeus Software, Inc 28 2. Paradigm Simulation, Inc 28 3. Virtual Reality Modeling Language (VRML) 29 D. QUICK SYNOPSIS AND OPINION 30 TV. APPROACH 33 A. THE THREE STATE APPROACH 33 B. THE APP-CULL-PLAY APPROACH 35 C. AUDITORY MODELS 37 1. Aureflexion (auditory reflection) model 38 2. Audication (Auditory location) Model 38 D. CONCLUSION 39 V. IMPLEMENTATION 41 A. SELECTING HARDWARE 41 vu B. SELECTING SOFTWARE 43 1. Acoustical Server 43 2. Modeling Tool , 43 3. Rendering Tool 44 C. TYPEDEFS 44 D. ACOUSTELIB 47 E. GAUDTvTEW 49 F. GAUDIFLY 57 G. SUMMARY 60 VI. DISCUSSION 61 A. WHAT WOULD BE NECESSARY TO MAKE THIS A PRODUCTION MODEL? 61 1. AcousteLib 61 2. GAUDIVIEW 62 3. GAUDIFLY 62 B. WHAT IS REALLY NEEDED? 63 1. Modelers 63 2. Programming 64 VH. CONCLUSIONS AND RECOMMENDATIONS 65 A. MODELING 65 B. PROGRAMMING 65 C. NODES/DATATYPES 66 D. FUTURE WORK 69 1. BUILD A GEOMETRY/AUDIO INTEGRATED MODELER IN JAVA 69 2. IMPLEMENT THE AUDIO ENVIRONMENT IN VRML 69 3. MORE PRECISE IMPLEMENTATIONS OF SOUNDS 70 LIST OF REFERENCES 73 BIBLIOGRAPHY 77 INITIAL DISTRIBUTION LIST 81 vm LIST OF FIGURES Figure 1: A Sound Wave 7 Figure 2: Frequency 8 Figure 3: Amplitude 9 Figure 4: Timbre 10 Figure 5: Interaural Time Delay (TTD) 14 Figure 6: Interaural Intensity Difference 15 Figure 7: Sound Diffusion 16 Figure 8: The three states of a sound in the real world 33 Figure 9: The three states of a sound as represented in a virtual world 34 Figure 10: APP-CULL-DRAW process of graphical rendering 35 Figure 11: APP-CULL-PLAY process of audio rendering 36 Figure 12: Integration of graphical and audio processes 37 Figure 13: Relationships between Audio Models and the three states of sound 39 Figure 14: File Selection Box 50 Figure 15: GAUDIEVIEW showing a simple room textured with wood 52 Figure 16: Head node and Sound node 53 Figure 17: GAUDrVTEW in action 56 Figure 18: Screencapture of GAUDIFLY 60 Figure 19: Resolving the applications with Figure 12 63 IX LIST OF TABLES Table 1: Comparison of Sound Card Features 20 Table 2: Technology Requirements 21 Table 3: Comparison of APIs 27 Table 4: Acoustical Materials 45 Table 5: Room structure 45 Table 6: Head structure 46 Table 7: Sound structure 46 Table 8: Preliminary Audio Structure 47 Table 9: Functions implemented in Acoustelib 48 Table 10: File Dialog Code 49 Table 1 1: Code to create a room 51 Table 12: Modifying a texture 52 Table 13: Loading a sound in GAUDIVIEW 55 Table 14: Code to get the value in a specified field 56 Table 15: Declaring the audio structures in GAUDIFLY 57 Table 16: Applying Viewer's Coordinates to the sound device 58 Table 17: Command line parameters for AUDEFLY 58 Table 18: Modifications to cmdline.C 60 Table 19: Proposed Acoustical Data Type 67 Table 20: Proposed Texture Node 67 Table 21: Sample code for a ceiling made of celotex 68 Table 22: Sample code for a Room 69 xi Xll ACKNOWLEDGEMENTS First and foremost I would like to thank my wife, Rachel, for the support she has shown and the understanding she has displayed. For half of my life, you have been there for me. Thanks for tolerating all of my pet projects, of which this thesis is the closest to being completed. Thanks for asking me if I was putting in more work than the instructor really required, because often I was too involved to see that I was overshooting the requirements. Second, I would like to thank my children, Kayleigh, Jordan, and Jamey. You helped me on projects when I asked, rode your bikes with me so that I would have to run faster, and helped me to keep my thinking fresh. Dr. Darken, thanks for all of the help. I have seldom worked for anyone that would tell me what they wanted, then let me go about accomplishing it. The non- micromanagement was very productive. Thanks for letting me go out and experiment with a little bit of everything. Can you believe, I actually stayed with a topic for more than a few weeks? Thanks to Major Russell Storms for being my second reader and one of the only people around that could really get down into the weeds when discussing sound.