Lattice-based Immersive Visualization A Dissertation Presented by Kaloian Petkov to The Graduate School in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in Computer Science Stony Brook University December 2013 Copyright by Kaloian Petkov 2013 Stony Brook University The Graduate School Kaloian Petkov We, the dissertation committee for the above candidate for the Doctor of Philosophy degree, hereby recommend acceptance of this dissertation. Arie Kaufman – Dissertation Advisor Distinguished Professor and Chair, Computer Science Department Xianfeng Gu - Chairperson of Defense Associate Professor, Computer Science Department Klaus Mueller Professor, Computer Science Department Amitabh Varshney Professor, Computer Science Department, University of Maryland This dissertation is accepted by the Graduate School Charles Taber Dean of the Graduate School ii Abstract of the Dissertation Lattice-based Immersive Visualization by Kaloian Petkov Doctor of Philosophy in Computer Science Stony Brook University 2013 The sizes of data are increasing at a very rapid pace in many applications, including medical visualization, physical simulations and industrial scanning. Some of this growth is not only due to the development of high resolution medical and industrial scanners, but also due to the wide availability of high performance graphics processing units (GPUs) which allow for the interactive rendering of large datasets. However, the increase of problem sizes has generally outpaced the increase of onboard GPU memory. At the same time, the resolution of the traditional display systems has not improved significantly compared to the exponential growth of computing power. We have developed a comprehensive approach that tackles the efficiency of the data representation through lattice-based techniques, as well as the visualization capabilities for exploring that data. We have constructed the Immersive Cabin and the Reality Deck facilities, along with a set of visualization techniques, to address the challenge of growing data sizes. In terms of sampling lattices, we have developed a Computational Fluid Dynamics (CFD) simulation framework based on the lattice Boltzmann method and using optimal sampling lattices. Our focus is specifically on the Face-centered Cubic lattice (FCC), which can achieve a stable simulation with only 13 lattice velocities while at the same time improving the sampling efficiency compared to using the traditional Cartesian grid. We demonstrate the resulting fD3Q13 LBM model for use in highly interactive smoke dispersion simulations. The simulation code is coupled with our visualization framework, which includes a high-performance volume renderer and support for virtual reality systems. The volume rendering is further enhanced with a novel LOD scheme for large volume data that allows for mixing the optimal sampling lattices in adjacent levels of the hierarchy with a computationally cheap indexing function. iii We have developed a visualization framework for the Immersive Cabin which supports the traditional virtual reality components, such as distributed rendering, stereo, tracking, rigid-body physics and sound. The lattice-based visualization and simulation techniques, including the support for mixed-lattice hierarchies, are integrated in the framework and can be combined with the mesh rendering and the rigid-body physics simulation. Based on our experience in the Immersive Cabin, we have designed and constructed the Reality Deck, which is the world's first 1.5 gigapixel immersive display. The Reality Deck contains 416 high resolution LCD monitors in a 4-wall surround layout, and similarly to the Immersive Cabin, uses a unique automatic door that is also a display surface. The graphics are generated on an 18-node cluster with 24 displays connected to each node using the AMD Eyefinity technology. We have extended the Immersive Cabin visualization framework to support the Reality Deck and developed a new gigapixel image renderer targeted at scientific and immersive visualization. We have developed a set of visualization techniques for the exploration of large and complex data in both of our facilities. Conformal Visualization is a novel retargeting approach for partially-enclosed VR environments, such as the Immersive Cabin and the Reality Deck, to allow for the complete visualization of the data even when display surfaces are missing. Our technique uses conformal mapping to ensure that shape is preserved locally under the transformation and we demonstrate its use for the visualization of both mesh and lattice data. In our Frameless Visualization technique, the traditional framebuffer is replaced with reconstruction from a stream of rendered samples. This approach allows smooth user interaction even when rendering on a gigapixel display, such as the Reality Deck, or when using computationally-expensive visualization algorithms. Our system generates low-latency image samples using an optimal sampling lattice in the 2D+time space, as well as importance-driven higher quality samples. Finally, we have developed the Infinite Canvas visualization technique for horizontally-enclosed visual environments. As the user moves through the physical space of the facility, the graphics outside of the field of view are updated to create the illusion of an infinite continuous canvas. The Infinite Canvas has been used for the visual exploration of gigapixel datasets that are an order of magnitude larger than the surface area of the Reality Deck, including very large image collections. iv To my parents. vi Contents List of Figures xi List of Tables xxi Acknowledgements xxiii Publications xxv 1 Introduction 1 1.1 Motivation . 1 1.2 Contributions . 3 2 Background 5 2.1 Sampling Lattices . 5 2.2 Lattice Reconstruction . .. 8 2.3 Multi-resolution Techniques for Volumetric Data . .... 9 2.4 The Lattice Boltzmann Method . 10 2.4.1 Definition . 10 2.4.2 Boundary Conditions . 12 2.5 Immersive Visualization . 13 2.6 Frameless Rendering . ... 15 3 LBM Fluid Simulation and Visualization on Optimal Sampling Lattices 17 3.1 Introduction . .. 17 3.2 The fD3Q13 Lattice . .. 19 3.3 Validation . 24 3.3.1 Poiseuille Flow . 24 3.3.2 Flow Past a Sphere . 26 vii CONTENTS 3.4 LBM Implementation with CUDA . 29 3.4.1 Programming Model . 29 3.4.2 LBM with CUDA . 30 3.5 Smoke Simulation in Urban Environment . 33 3.5.1 Urban Modeling . 33 3.5.2 Modeling of Smoke in Urban Environment . 34 3.5.3 Rendering . 38 3.5.4 Results . 41 4 Hierarchical Volume Rendering on Mixed Lattices 43 4.1 Introduction . .. 43 4.2 Mixed-Lattice Hierarchy Construction . ..... 44 4.3 Mixed-lattice Rendering . .. 46 4.4 Implementation and Results . 49 4.4.1 Reconstruction Filters . 49 4.4.2 Hierarchical Rendering . .. 53 5 Immersive Environments 57 5.1 The Immersive Cabin . .. 58 5.1.1 Equipment and Construction . 59 5.1.2 Rendering Framework . 61 5.1.3 Remote Visualization . 63 5.1.4 Applications . 66 5.2 The Reality Deck . 70 5.2.1 Equipment and Construction . 71 5.2.2 Rendering Framework . 78 5.2.3 Applications . 80 5.3 Case Studies in Immersive Visualization . ... 85 5.3.1 Case Study: World-wide data visualization . 85 5.3.2 Case Study: Fusion of GIS and procedural modeling for driving simulation in New York........................................... 87 5.3.3 Case Study: Immersive Medical Visualization . .. 91 viii CONTENTS 6 Distributed Immersive Visualization 97 6.1 Conformal Visualization . .. 98 6.1.1 Theoretical Background . .. 99 6.1.2 Implementation Details . 105 6.1.3 Visualization Techniques . 110 6.1.4 Results . 114 6.1.5 Evaluation . 124 6.2 Frameless Visualization . .. 132 6.2.1 Algorithm Overview . 133 6.2.2 Adaptive Sampling . 134 6.2.3 Implementation Details . 137 6.2.4 Results . 138 6.3 The Reality Deck Infinite Canvas . .... 146 6.3.1 Basic Formulation . 147 6.3.2 Spiral Navigation . 150 6.3.3 Multiuser Visualization . 152 6.3.4 Results . 153 7 Conclusions 157 7.1 Summary of Contributions . 157 7.2 Future Research . .... 158 Bibliography 161 ix CONTENTS x List of Figures 2.1 The BCC lattice can be constructed by (d) adding one lattice site at the center of every cell of the CC lattice (a), or by (e) interleaving two CC lattices. The FCC lattice can be constructed by (b) placing sites at the face centers of the CC unit cell, or by (c) interleaving four CC lattices. 6 2.2 Unit cell and Voronoi cell for the (a) FCC and (b) BCC lattices. Unit cells (illustrated with the black edges) are the cuboctahedron and the rhombic dodecahedron, respectively. The Voronoi cells (colored in red) are the rhombic dodecahedron and the truncated octahedron, respectively. 6 2.3 Curved boundary treatment. The filled circles represent the discretized geometric object. The clear circles represent fluid cells. The blue circle at position xw is the intersection point between the surface of the geometric object and a lattice link. 14 3.1 Hexagonal 2D lattice, D2Q7. The 6 nearest neighboring sites (green dots) of a site (red dots) form a regular hexagon. The Voronoi cell (blue) is also a regular hexagon. 19 3.2 A 3D BCC lattice (a) can be constructed by adding one lattice site at the center of every cell of the CC lattice. A 3D FCC lattice (b) can be constructed by placing sites at the centers of the square surfaces of every cell of the CC lattice. A 3D HCP lattice (c) can be constructed by layering 2D HCP lattices. In (c), the blue sites form the 2D HCP lattice, which is the first layer of the 3D HCP lattice. The green sites represent the second layer. The entire 3D HCP lattice contains alternating blue and green layers. ..