Audiosity= Audio+ Radiosity
Total Page:16
File Type:pdf, Size:1020Kb
AUDIOSITY = AUDIO + RADIOSITY A THESIS SUBMITTED TO THE FACULTY OF GRADUATE STUDIES AND RESEARCH IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE IN COMPUTER SCIENCE UNIVERSITY OF REGINA By Hao Li Regina, Saskatchewan September 2009 © Copyright 2009: Hao Li Library and Archives Bibliotheque et 1*1 Canada Archives Canada Published Heritage Direction du Branch Patrimoine de I'edition 395 Wellington Street 395, rue Wellington Ottawa ON K1A 0N4 Ottawa ON K1A 0N4 Canada Canada Your file Votre reference ISBN: 978-0-494-65704-1 Our file Notre reference ISBN: 978-0-494-65704-1 NOTICE: AVIS: The author has granted a non L'auteur a accorde une licence non exclusive exclusive license allowing Library and permettant a la Bibliotheque et Archives Archives Canada to reproduce, Canada de reproduire, publier, archiver, publish, archive, preserve, conserve, sauvegarder, conserver, transmettre au public communicate to the public by par telecommunication ou par I'lntemet, preter, telecommunication or on the Internet, distribuer et vendre des theses partout dans le loan, distribute and sell theses monde, a des fins commerciales ou autres, sur worldwide, for commercial or non support microforme, papier, electronique et/ou commercial purposes, in microform, autres formats. paper, electronic and/or any other formats. The author retains copyright L'auteur conserve la propriete du droit d'auteur ownership and moral rights in this et des droits moraux qui protege cette these. Ni thesis. Neither the thesis nor la these ni des extraits substantiels de celle-ci substantial extracts from it may be ne doivent etre imprimes ou autrement printed or otherwise reproduced reproduits sans son autorisation. without the author's permission. In compliance with the Canadian Conformement a la loi canadienne sur la Privacy Act some supporting forms protection de la vie privee, quelques may have been removed from this formulaires secondaires ont ete enleves de thesis. cette these. While these forms may be included Bien que ces formulaires aient inclus dans in the document page count, their la pagination, il n'y aura aucun contenu removal does not represent any loss manquant. of content from the thesis. ••I Canada UNIVERSITY OF REGINA FACULTY OF GRADUATE STUDIES AND RESEARCH SUPERVISORY AND EXAMINING COMMITTEE Hao Li, candidate for the degree of Master of Science in Computer Science, has presented a thesis titled, Audiosity = Audio + Radiosity, in an oral examination held on September 18, 2009. The following committee members have found the thesis acceptable in form and content, and that the candidate demonstrated satisfactory knowledge of the subject material. External Examiner: Dr. Shaun M. Fallat, Department of Mathematics and Statistics Supervisor: Dr. David Gerhard, Department of Computer Science Committee Member: Dr. Yiyu Yao, Department of Computer Science Committee Member: Dr. Xue Dong Yang, Department of Computer Science Chair of Defense: Professor Kenneth Runtz, Faculty of Engineering and Applied Science ABSTRACT It is challenging to render spatialized audio within a dynamic environment in real-time with limited computational resources. Further complicating this process is the fact that compared to direct sound, reflected sound is usually more expensive and time consuming. In this thesis, we introduce a rapid method that we have named audiosity, which generates reflected sound in an enclosed audio environment containing obsta cles. Audiosity adopts the radiosity rendering technique from computer graphics, and applies it to audio animation. At each frame of the animation, audiosity computes the sound energy that propagates between surfaces. The result is the final sound energy in the scene at the state of equilibrium. By treating the audio environment in its entirety, the audiosity scheme accelerates the reflected sound computation. More over, a filter grid is pre-calculated and stored so that the speed of the audio animation process can be increased further. In this thesis, our only concerns are defused reflections during sound propagation. In addtion, due to other restrictions of the graphics radiosity technique, we focus on iii rendering without time delay. Although it is a rough approximation, we will justify this reasonable simplification to accelerate the entire audiosity scheme, so it works within real-time applications which are required to respond to events as they happen simultaneously. We also propose a modification to the original audiosity system providing for the inclusion of delay components in order to render reverberation, but this adds computational complexity. A testing program is provided, along with experiments and evaluations which show that the audiosity scheme is efficient as well as matching our expectations. iv ACKNOWLEDGEMENTS I would like to acknowledge and extend my sincere gratitude to all those who have helped me during my graduate studies and research, especially in the preparation of this thesis. Foremost, I would like to thank my supervisor, Dr. David Gerhard, for the freedom he gave me, which allowed me to explore various research topics in the Computer Audio field. He has provided me with invaluable guidance and infinite wisdom in my research and graduate study. Further appreciation is extended to my thesis committee members, Dr. Xue Dong Yang and Dr. Yiyu Yao for their time and expertise in improving this thesis. In addition, thank the University of Regina, Faculty of Graduate Studies and Research, and the Department of Computer Science for funding and other helps. Lastly, I would like to give my thanks to my parents, Wancai Li and Miling Zhang, for both emotional and financial supports throughout my life. This thesis is also dedicated to Vivi for her unending encouragement and support. v POST DEFENSE ACKNOWLEDGEMENTS I would like to express thanks to my external examiner, Dr. Shaun Fallat, for his insightful comments and constructive suggestions. Final thanks are given to Prof. Ken Runtz who presided at my defense. VI CONTENTS ABSTRACT iii ACKNOWLEDGEMENTS v POST DEFENSE ACKNOWLEDGEMENTS vi 1 INTRODUCTION 1 1.1 TERMINOLOGY AND MOTIVATION 1 1.2 OUTLINE OF THE THESIS 5 2 BACKGROUND OF AUDIO ANIMATION AND RADIOSITY 7 2.1 SPATILIZATION IN AUDIO ANIMATION 7 2.2 FRESNEL ZONE AND CLEARANCE 10 2.3 TSINGOS'AUDIO ANIMATION RENDERING TECHNIQUE ... 14 2.4 LIMITATIONS OF CURRENT AUDIO RENDERING TECHNIQUES 17 2.5 RADIOSITY 18 vii 2.5.1 Form Factor 21 2.5.2 Radiosity Equation 22 2.6 LIMITATIONS OF RADIOSITY 26 3 FIRST-PASS ENERGY DISTRIBUTION 29 3.1 INTRODUCTION 29 3.2 DIFFERENCE BETWEEN GRAPHICS EMITTERS AND SOUND EMITTERS 30 3.3 FIRST-PASS ENERGY DISTRIBUTION 32 3.4 USAGE OF FIRST-PASS ENERGY DISTRIBUTION 38 4 AUDIOSITY 41 4.1 INTRODUCTION 41 4.2 DIRECT SOUND VS. REFLECTED SOUND 42 4.3 AUDIOSITY SCHEME 45 4.3.1 Solid angle 46 4.3.2 Render with inversed matrix 48 4.3.3 Audio form factor 52 4.3.4 Rendering and output 54 4.3.5 Sound attenuation function 55 4.3.6 Rendering algorithm 56 viii 4.4 FILTER GRID 60 4.5 AUDIOSITY AND TIME DELAY 64 5 EXPERIMENTS AND EVALUATIONS 69 5.1 GUI OF THE TESTING PROGRAM 69 5.2 REASONABLENESS 71 5.2.1 Experiment #1: test with empty environment 72 5.2.2 Experiment #2: test with sound blockers 73 5.3 PERFORMANCE 76 6 CONCLUSION AND FUTURE WORK 82 REFERENCES 86 IX LIST OF TABLES 5.1 Grid construction time 78 5.2 Grid retrieval time 79 x LIST OF FIGURES 2.1 Sampled signals 9 2.2 Presnel zone 12 2.3 Tsingos audio animation rendering technique [47] 16 2.4 The Cornel Box [8] 20 2.5 Form factor 22 2.6 Radiosity equation 24 3.1 Emitter arrangement 1 33 3.2 Emitter arrangement 2 34 3.3 Emitter arrangement 3 35 3.4 Emitter arrangement 4 36 4.1 Direct sound vs. reflected sound 43 4.2 A determinant reflected sound situation 45 4.3 Solid angle [34] 47 xi 4.4 Audiosity rendering architecture 57 4.5 Equalization filter 58 4.6 Equalizer in iTunes 60 4.7 A sound filter grid 62 4.8 Retrieve data from the filter grid 63 4.9 Reverberation 65 4.10 Audiosity with delay time 67 5.1 GUI of the testing program 70 5.2 Experiment 1 73 5.3 Experiment 1 result chart 74 5.4 Experiment 2 75 5.5 Experiment 2 result chart 76 5.6 Fresnel zones for different frequency bands 77 5.7 Grid construction time 78 5.8 Grid retrieval time 80 xii 1 INTRODUCTION 1.1 TERMINOLOGY AND MOTIVATION In real-time computer audio simulation within a dynamic sound environment, the final audio playback is a combination of all audio waveforms sent out by sound sources and affected by the environment. Certainly, during the propagation, sound will be obscured by obstacles and be reflected by surfaces, so that the physical layout of the environment affects the perceived sound. These effects, along with the sound attenuation during traveling, embed the information of the environment into the final sound, which is then received by the listener. One of the ultimate goals in computer audio is to make the result reasonable and match our expectation. In more technical terms, we hope to create a realistic illusion of sound sources within a spatial environment. Objects near the sound source and receiver, such as obstacles and flat surfaces, affect the perceived sound based on the physical arrange ment of those objects. Such spatial information needs to be encoded into the audio 1 CHAPTER 1. INTRODUCTION 2 to ensure the listener can receive this information through limited number of loud speakers (two for stereo, five or more for a surround sound system). In computer audio, this procedure of simulating the audio behaviour based on the physical layout of the environment, including sound sources, sound receivers, and nearby objects, is called audio spatialization.