A Beam Tracing Approach to Acoustic Modeling for Interactive Virtual Environments
Total Page:16
File Type:pdf, Size:1020Kb
A Beam Tracing Approach to Acoustic Modeling for Interactive Virtual Environments Thomas Funkhouser, Ingrid Carlbom, Gary Elko, Gopal Pingali, Mohan Sondhi, and Jim West Bell Laboratories Abstract we must pay more attention to producing realistic sound in order to create a complete immersive experience in which aural cues com- Virtual environment research has focused on interactive image gen- bine with visual cues to support more natural interaction within a eration and has largely ignored acoustic modeling for spatialization virtual environment. First, qualitative changes in sound reverbera- of sound. Yet, realistic auditory cues can complement and enhance tion, such as more absorption in a room with more lush carpets, can visual cues to aid navigation, comprehension, and sense of presence enhance and reinforce visual comprehension of the environment. in virtual environments. A primary challenge in acoustic model- Second, spatialized sound can be useful for providing audio cues ing is computation of reverberation paths from sound sources fast to aid navigation, communication, and sense of presence [14]. For enough for real-time auralization. We have developed a system that example, the sounds of objects requiring user attention can be spa- uses precomputed spatial subdivision and ªbeam treeº data struc- tialized according to their positions in order to aid object location tures to enable real-time acoustic modeling and auralization in inter- and binaural selectivity of desired signals (e.g., ªcocktail partyº ef- active virtual environments. The spatial subdivision is a partition of fect). The goal of this work is to augment a previous interactive 3D space into convex polyhedral regions (cells) represented as a cell image generation system to support real-time auralization of sound adjacency graph. A beam tracing algorithm recursively traces pyra- based on realistic acoustic modeling in large virtual environments. midal beams through the spatial subdivision to construct a beam tree We hope to use this system to support virtual environment appli- data structure representing the regions of space reachable by each cations such as distributed training, simulation, education, home potential sequence of transmission and specular re¯ection events at shopping, virtual meetings, and multiplayer games. cell boundaries. From these precomputed data structures, we can A primary challenge in acoustic modeling is computation of generate high-order specular re¯ection and transmission paths at reverberation paths from a sound source to a listener (receiver) [30]. interactive rates to spatialize ®xed sound sources in real-time as the As sound may travel from source to receiver via a multitude of user moves through a virtual environment. Unlike previous acoustic re¯ection, transmission, and diffraction paths, accurate simulation modeling work, our beam tracing method: 1) supports evaluation of is extremely compute intensive. For instance, consider the simple reverberation paths at interactive rates, 2) scales to compute high- example shown in Figure 1. In order to present an accurate model order re¯ections and large environments, and 3) extends naturally of a sound source (labeled `S') at a receiver location (labeled `R'), to compute paths of diffraction and diffuse re¯ection ef®ciently. We we must account for an in®nite number of possible reverberation are using this system to develop interactive applications in which a paths (some of which are shown). If we are able to model the user experiences a virtual environment immersively via simultane- reverberation paths from a sound source to a receiver, we can render ous auralization and visualization. a spatialized representation of the sound according to their delays, attenuations, and source and receiver directivities. Key Words: Beam tracing, acoustic modeling, auralization, spatialized sound, virtual environment systems, virtual reality. S R 1 Introduction Interactive virtual environment systems combine graphics, acous- tics, and haptics to simulate the experience of immersive exploration of a three-dimensional virtual world by rendering the environment Figure 1: Example reverberation paths. as perceived from the viewpoint of an observer moving under real- time control by the user. Most prior research in virtual environment Since sound and light are both wave phenomena, acoustic mod- systems has focused on visualization (i.e., methods for rendering eling is similar to global illumination in computer graphics. How- more realistic images or for increasing image refresh rates), while ever, there are several signi®cant differences. First, the wavelengths relatively little attention has been paid to auralization (i.e., rendering of audible sound fall between 0.02 and 17 meters (20kHz to 20Hz), spatialized sound based on acoustical modeling). Yet, it is clear that more than ®ve orders of magnitude longer than visible light. As a ¡ result, though re¯ection of sound waves off large walls tends to be Princeton University primarily specular, signi®cant diffraction does occur around edges of objects like walls and tables. Small objects (like coffee mugs) have signi®cant effect on the sound ®eld only at frequencies beyond 4 kHz, and can usually be excluded from models of acoustic en- vironments, especially in the presence of other signi®cant sources of re¯ection and diffraction. Second, sound travels through air 106 times slower than light, causing signi®cantly different arrival times for sound propagating along different paths, and the resulting acous- tic signal is perceived as a combination of direct and re¯ected sound (reverberation). The time distribution of the reverberation paths S ¢ d of the sound in a typical room is much longer than the integration period of the perception of sound by a human. Thus, it is important d R to accurately compute the exact time/frequency distribution of the c reverberation. In contrast, the speed of light and the perception of Sc light is such that the eye integrates out the transient response of a a S light source and only the energy steady-state response needs to be calculated. Third, since sound is a coherent wave phenomenon, the b calculation of the re¯ected and scattered sound waves must incor- Sa Sb porate the phase (complex amplitude) of the incident and re¯ected wave(s), while for incoherent light, only the power must be summed. Figure 2: Image source method. Although acoustic modeling has been well-studied in the con- text of non-interactive applications [34], such as concert hall design, their expected computational complexity has exponential growth. In there has been relatively little prior research in real-time acoustic general, © ¥ virtual sources must be generated for re¯ections modeling for virtual environment systems [15]. Currently available in environments with surface planes. Moreover, in all but the sim- auralization systems generally model only early specular re¯ections, plest environments (e.g., a box), complex validity/visibility checks while late reverberations and diffractions are modeled with statisti- must be performed for each of the © ¥ virtual sources since not cal approximations [1, 25, 40, 53]. Also, due to the computational all of the virtual sources represent physically realizable specular complexity of current systems, they generally consider only simple re¯ection paths [6]. For instance, a virtual source generated by geometric arrangements and low-order specular re¯ections. For in- re¯ection over the non-re¯ective side of a surface is ªinvalid.º Like- stance, the Acoustetron [17] computes only ®rst- and second-order wise, a virtual source whose re¯ection is blocked by another surface specular re¯ections for box-shaped virtual environments. Video in the environment or intersects a point on a surface's plane which games provide spatialized sound with ad hoc localization methods is outside the surface's boundary (e.g., £¥ in Figure 2) is ªinvisi- (e.g., pan effects) rather than with realistic geometrical acoustic ble.º During recursive generation of virtual sources, descendents modeling methods. The 1995 National Research Council Report on of invalid virtual sources can be ignored. However, descendents of Virtual Reality Scienti®c and Technological Challenges [15] states invisible virtual sources must still be considered, as higher-order that ªcurrent technology is still unable to provide interactive systems re¯ections may generate visible virtual sources (consider mirroring £ © ¥ with real-time rendering of acoustic environments with complex, re- over surface ). Due to the computational demands of vis- alistic room re¯ections.º ibility checks, image source methods are practical only for acoustic In this paper, we describe a beam tracing method that computes modeling of few re¯ections in simple environments [32]. high-order specular re¯ection and transmission paths from ®xed sources in large polygonal models fast enough to be used for au- ralization in interactive virtual environment systems. The key idea 2.2 Radiant Exchange Methods behind our method is to precompute and store spatial data struc- tures that encode all possible transmission and specular re¯ection Radiant exchange methods have been used extensively in computer paths from each audio source and then use these data structures to graphics to model diffuse re¯ection of radiosity between patches compute reverberation paths to an arbitrarily moving observer view- [21]. Brie¯y, radiosity methods consider every patch a potential point for real-time auralization during