Advanced Visualization and Interactive Applications in the Immersive Cabin
Total Page:16
File Type:pdf, Size:1020Kb
Advanced Visualization and Interactive Applications in the Immersive Cabin Kaloian Petkov and Arie Kaufman, Fellow, IEEE Department of Computer Science and CEWIT Stony Brook University Stony Brook, NY 11794-4400, USA [kpetkov,ari]@cs.stonybrook.edu Abstract— We present a visualization and interaction framework enclosed CAVEs exist [2], they present a number of for the Immersive Cabin (IC), which is a 5-wall CAVE-like engineering challenges that result in significant cost increases. immersive 3D environment. The back-end rendering is provided by a cluster of 5 workstations with 2 NVIDIA Quadro GPUs We have proposed and constructed the Immersive Cabin each. We utilize a pair of projectors for each wall and an external (IC) [3] as a robust and affordable design for a fully enclosed LCD shutter system that is synchronized with active stereo visualization environment. The IC consists of 4 projection glasses. Each workstation produces a synchronized pair of walls and a floor and the design is compatible with multiple stereoscopic images for a single wall in the IC. If only one GPU is projectors per surface and the new 120Hz stereo projectors. dedicated to rendering, the second one is utilized as a Our design utilizes an automatic door to provide access into the computational device using C/C++ and the NVIDIA CUDA IC, which compares favorably cost-wise with facilities that use extensions. In addition to the depth cues and surround immersion heavy duty machinery to move the back wall. Our facility has from the visualization system, we use wireless optical head and been used for a number of projects and in this paper we present hand tracking to further enhance the data exploration a summary of the advances in terms of the software framework, capabilities. Combined with a range of interaction and navigation the IC instruments and some of the applications. tools, our system can support a variety of interactive applications, including architectural and automotive pre-visualization, urban planning, medical imaging, and simulation and rendering of physical phenomena. II. EQUIPMENT AND CONSTRUCTION A. Immersive Cabin Keywords-Virtual Reality, CAVE, Immersive Cabin, GPU, Virtual Colonoscopy The IC is a compact, fully enclosed visualization environment with 5 projection surfaces and an automatic door I. INTRODUCTION as part of the back wall. Each projection node is equipped with The size of data has grown in recent years to the point the following: where traditional techniques for its visualization and analysis Dual Hitachi SXGA+ projectors are not sufficient. It is not uncommon to be presented with multi-gigabyte volumetric datasets from fluid dynamics Beacon SX+ shutter and synchronization simulations, high resolution CT scans, or data structures so complex that projections onto 2D display are ineffective. A IR emitter number of techniques have been developed for the The layout of the installation is illustrated in Fig. 1 and visualization of large-scale complex data. Facilities such as the additional details about the construction are available in the CAVE [1] and devices such as Head Mounted Displays paper by Qiu et al. [3]. The shutters in front of the projectors (HMDs) provide a much larger field of view into a virtual are connected to the Beacon system and operate at 120Hz. The environment and further improve the perception of spatial IR emitter is also connected to the Beacon system and allows relationships between data by utilizing stereoscopic rendering. for synchronization with the LCD shutter glasses. The 5 Despite their advantages, both solutions present a number of Beacon systems are daisy-chained so that all the shutters in the challenges that ultimately limit their appeal and traction outside IC are synchronized. Each pair of projectors is driven by a the visualization field. HMDs are usually bulky and heavily single workstation with two NVIDIA Quadro FX 4600 boards. wired, and although they present a visually unobstructed view Although perfect synchronization between the GPUs and the into the data, their usage is associated with eye strain, projectors is possible with the use of NVIDIA G-Sync boards, headaches and other medical issues. CAVEs on the other hand our setup can operate with software-based framelocking. The offer a more natural visualization environment that is amenable G-Sync boards are mandatory for installations without an to augmented reality applications. However, building fully external shutter system, e.g. when using native 120Hz enclosed facilities remains a challenge and many installations projectors for stereoscopic rendering. have a limited number of display surfaces. Although fully This work was partially supported by NSF grants CCF-0702699, CCF- 0448399, DMS-0528363, DMS-0626223, NIH grant CA082402, and the Center of Excellence in Wireless and Information Technology (CEWIT). Figure 2. Blue cloud represents the tracking coverage areas seen simultaneously by (left to right) 3, 6 and 8 cameras. The positions near the center of the IC that are seen by 6 cameras are optimal for head and gesture tracking since optical occlusions are minimized. C. Input Devices Our system supports standard input devices such as keyboards and mice, although they are not particularly suitable for the exploration of virtual environments. We primarily rely Figure 1. Diagram for an installation of the Immersive Cabin. on wireless gamepads as the main navigation tool and the framework currently supports the Logitech Rumblepad 2 and the Xbox 360 wireless controllers. In our experiments, the Our cluster contains a single head node which is 3DConnexion Space Navigator has provided superior responsible for managing all the machines and processing all performance for certain medical and architectural applications user interactions. In addition, we have a 32-node cluster with as it combines 6 degrees of freedom in a very intuitive and dual CPUs at each node and an NVIDIA Quadro FX 4500 compact device. However, in practice, the absence of native graphics board. These machines are not used for direct output wireless models greatly complicates the setup. to the projection surfaces in the IC and are instead dedicated to Our IR tracking system is deployed to provide a different off-line rendering and natural phenomena simulations. modality of user interactions. Head tracking is used to interact with the visual system directly by allowing the user unobstructed movement inside the volume of the IC. The B. Tracking System position and orientation of the user are both taken into account The IC utilizes an infrared tracking system from when rendering the virtual world, which allows, for example, NaturalPoint that contains 8 IR cameras with wide-angle peeking around a corner or simulating an object that is inside lenses, 2 synchronization hubs and the ARENA motion capture the IC. In addition, the tracking system allows the development software. The cameras are mounted along the top edge of the of a more intuitive user interface that relies on natural gestures IC walls and provide sufficient coverage for head and gesture instead of tracked devices. tracking throughout most of the interior space. The synchronization hubs are connected to the sync signal of the III. RENDERING FRAMEWORK Beacon shutter system so that the pulsed IR illumination from A. Framework Design the cameras does not interfere with the IR signal for the shutter glasses. We use between 3 and 5 markers to define rigid bodies Designing a flexible framework for distributed GPU for the tracking software and special care has to be taken so rendering is generally a challenging task. Compared to a that the marker arrangements are sufficiently different in the traditional 3D engine, our system needs to support a network of topological sense. For the LCD shutter glasses we use a 5 tape workstations with single or multiple rendering pipes, multiple marker pattern that reduces the occurrence of optical display surfaces per node and stereo rendering. Support for occlusions, while for other objects, 3 markers may be more traditional display layouts, such as multiple angled sufficient. In the case of hand gloves, protruding spherical monitors on a desk, is essential for debugging and application markers are used as occlusions prevent the efficient use of flat development. The Chromium library [4] provides support for tape markers. rendering on a cluster on nodes by directly manipulating the stream of rendering commands issued by a graphics API such Fig. 2 illustrates the coverage areas for tracking that we as OpenGL. A set of stream filters are used at each rendering achieve in the IC. The blue cloud represents the area that is node to create a parallel graphics system in which the seen by a specified number of cameras and we provide the visualization application is not necessarily aware of the results for coverage by 3, 6 and 8 cameras. A point needs to be underlying rendering platform. While this approach greatly seen by at least 3 cameras in order to obtain its position in 3D simplifies the process of creating Virtual Reality applications, space and a rigid body can contain between 3 and 6 points. network bandwidth becomes a major bottleneck with modern Although most of the volume in the IC is covered by at least 3 applications that may utilize hundreds of high resolution cameras, the illustration in Fig. 2 does not account for optical textures, advanced shaders and multiple rendering passes per occlusions which are a significant problem. Rather, we have frame [5]. CGLX [6] follows a similar approach by providing a optimized the camera placement for optimal tracking near the replacement for the OpenGL library that abstracts the center of the IC and towards the front screen. visualization surfaces. VRJuggler [7] and Equalizer [8] on the other hand operate at a higher level using a runtime library to manage multiple instances of the application on the cluster computers. With this approach, only the user input is transmitted over the network which results in significant bandwidth savings.