LINKOPING¨ STUDIES IN SCIENCE AND TECHNOLOGY DISSERTATIONS, NO. 1043 Efficient Methods for Direct Volume Rendering of Large Data Sets Patric Ljung DEPARTMENT OF SCIENCE AND TECHNOLOGY LINKOPING¨ UNIVERSITY, SE-601 74 NORRKOPING,¨ SWEDEN NORRKOPING¨ 2006 Efficient Methods for Direct Volume Rendering of Large Data Sets c 2006 Patric Ljung [email protected] Division of Visual Information Technology and Applications, Norrk¨oping Visualization and Interaction Studio Department of Science and Technology Link¨oping University, SE-601 74 Norrk¨oping,Sweden ISBN 91-85523-05-4 ISSN 0345-7524 Online access: http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-7232 Printed in Sweden by LTAB, Link¨oping 2006. To my wife Jenny and our children Henry & Hanna iv Abstract Direct Volume Rendering (DVR) is a technique for creating images directly from a representation of a function defined over a three-dimensional domain. The technique has many application fields, such as scientific visualization and medical imaging. A striking property of the data sets produced within these fields is their ever increasing size and complexity. Despite the advancements of computing resources these data sets seem to grow at even faster rates causing severe bottlenecks in terms of data transfer bandwidths, memory capacity and processing requirements in the rendering pipeline. This thesis focuses on efficient methods for DVR of large data sets. At the core of the work lies a level-of-detail scheme that reduces the amount of data to process and handle, while optimizing the level-of-detail selection so that high visual quality is maintained. A set of techniques for domain knowledge encoding which significantly improves assessment and prediction of visual significance for blocks in a volume are introduced. A complete pipeline for DVR is presented that uses the data reduction achieved by the level-of-detail selection to minimize the data requirements in all stages. This leads to reduction of disk I/O as well as host and graphics memory. The data reduction is also exploited to improve the rendering performance in graphics hardware, employing adaptive sampling both within the volume and within the rendered image. The developed techniques have been applied in particular to medical visualiza- tion of large data sets on commodity desktop computers using consumer graphics pro- cessors. The specific application of virtual autopsies has received much interest, and several developed data classification schemes and rendering techniques have been mo- tivated by this application. The results are, however, general and applicable in many fields and significant performance and quality improvements over previous techniques are shown. Keywords: Computer Graphics, Scientific Visualization, Medical Imaging, Vol- ume Rendering, Raycasting, Transfer Functions, Level-of-detail, Fuzzy Classification, Virtual Autopsies. vi Acknowledgments My first and sincere thanks go to my supervisor and friend, Professor Anders Ynner- man. It has been a truly pleasant, rewarding and exciting journey with much good hard labor, and a few very late nights. Claes Lundstrom,¨ my constant collaborator. It has been, and still is, such a great and inspiring opportunity to work together. Matthew Cooper for continuous support and proof-reading submissions during the middle of the night. My ex-room mate, Jimmy Johansson, for stimulating research discussions and fetching me coffee on oc- casions of submission deadline distress. My other former and present colleagues at VITA, NVIS and CMIV, you have made this such an exciting and challenging aca- demic environment. This work has been supported by the Swedish Research Council, grant 621-2001- 2778 and 621-2003-6582, and the Swedish Foundation for Strategic Research, grant A3 02:116. My very grateful thanks go to: My mom, dad and brother for their love, support and trust. My parents in-law for their love, friendship and the shade in their relaxing garden at Loftahammar where I could work on my thesis. My loving and beautiful wife Jenny and our lively and adorable kids Henry and Hanna. You always remind me that there is so much in life to cherish. Patric Ljung at the summerhouse in Loftahammar, July 2006. Photo taken by Hanna, aged 5. viii Contents Foreword xiii 1 Introduction 1 1.1 Direct Volume Rendering . 1 1.2 Volume Data . 2 1.3 Transfer Functions . 4 1.4 Rendering Images of Volumes . 5 1.5 User Interaction . 6 1.6 Research Challenges . 7 1.7 Contributions . 8 2 Aspects of Direct Volume Rendering 9 2.1 Volumetric Data Sets . 9 2.1.1 Voxel Data and Sampling . 10 2.1.2 Linear Data Structures . 10 2.2 Preprocessing and Basic Analysis . 11 2.2.1 Volume Subdivision and Blocking . 11 2.2.2 Block Properties and Acceleration Structures . 12 2.2.3 Hierarchical Multiresolution Representations . 12 2.3 Level-of-Detail Management . 14 2.3.1 View-Dependent Approaches . 14 2.3.2 Data Error Based Approaches . 14 2.3.3 Transfer Function Based Approaches . 15 2.4 Encoding, Decoding and Storage . 15 2.4.1 Transform and Compression Based Techniques . 15 2.4.2 Out-of-Core Data Management Techniques . 17 2.5 Advanced Transfer Functions . 18 2.5.1 Data Classification and Segmentation . 18 2.6 Rendering . 19 2.6.1 Texture Slicing Techniques . 19 2.6.2 GPU-based Raycasting . 20 2.6.3 Multiresolution Volume Rendering . 21 3 Improving Direct Volume Rendering 23 3.1 Flat Multiresolution Blocking . 23 3.2 Knowledge Encoding in Transfer Functions . 25 3.2.1 Transfer Function-Based Block Significance . 26 3.2.2 Level-of-Detail Selection . 27 x CONTENTS 3.3 Exploiting Spatial Coherence . 29 3.3.1 Partial Range Histograms and Range Weights . 30 3.3.2 Enhanced Histograms . 31 3.4 Pipeline Processing . 34 3.5 Sampling of Multiresolution Volumes . 36 3.5.1 Nearest Block Sampling . 36 3.5.2 Interblock Interpolation Sampling . 37 3.5.3 Interblock Interpolation Results . 38 3.6 Raycasting on the GPU . 39 3.6.1 Adaptive Object-Space Sampling . 40 3.6.2 Multivariate Raycasting . 41 3.6.3 Adaptive Screen-Space Sampling . 42 3.7 Case Study I: Brute Force Methods . 45 3.8 Case Study II: Virtual Autopsies . 46 4 Conclusions 49 4.1 Summary of Conclusions . 51 4.2 Future Research . 51 Bibliography 53 Paper I: Interactive Visualization of Particle-In-Cell Simulations 59 Paper II: Transfer Function Based Adaptive Decompression for Volume Rendering of Large Medical Data Sets 67 Paper III: Extending and Simplifying Transfer Function Design in Medical Volume Rendering Using Local Histograms 77 Paper IV: Multiresolution Interblock Interpolation in Direct Volume Rendering 87 Paper V: The α-histogram: Using Spatial Coherence to Enhance Histograms and Transfer Function Design 97 Paper VI: Adaptive Sampling in Single Pass, GPU-based Raycasting of Multiresolution Volumes 107 Paper VII: Multi-Dimensional Transfer Function Design Using Sorted Histograms 119 Paper VIII: Local histograms for design of Transfer Functions in Direct Volume Rendering 131 Paper IX: Full Body Virtual Autopsies Using A State-of-the-art Volume Rendering Pipeline 145 List of Papers Papers I through IX are included in this dissertation. I Patric Ljung, Mark Dieckmann, Niclas Andersson and Anders Ynnerman. In- teractive Visualization of Particle-In-Cell Simulations. In Proceedings of IEEE Visualization 2000. Salt Lake City, USA. 2000. II Patric Ljung, Claes Lundstrom,¨ Anders Ynnerman and Ken Museth. Transfer Function Based Adaptive Decompresion for Volume Rendering of Large Medical Data Sets. In Proceedings of IEEE/ACM Symposium on Volume Visualization 2004. Austin, USA. 2004. III Claes Lundstrom,¨ Patric Ljung and Anders Ynnerman. Extending and Simpli- fying Transfer Function Design in Medical Volume Rendering Using Local His- tograms. In Proceedings EuroGraphics/IEEE Symposium on Visualization 2005. Leeds, UK. 2005 IV Patric Ljung, Claes Lundstrom¨ and Anders Ynnerman. Multiresolution Interblock Interpolation in Direct Volume Rendering. In Proceedings of Eurographics/IEEE Symposium on Visualization 2006. Lisbon, Portugal. 2006. V Claes Lundstrom,¨ Anders Ynnerman, Patric Ljung, Anders Persson and Hans Knutsson. The α-histogram: Using Spatial Coherence to Enhance Histograms and Transfer Function Design. In Proceedings Eurographics/IEEE Symposium on Visualization 2006. Lisbon, Portugal. 2006. VI Patric Ljung. Adaptive Sampling in Single Pass, GPU-based Raycasting of Mul- tiresolution Volumes. In Proceedings Eurographics/IEEE International Workshop on Volume Graphics 2006. Boston, USA. 2006. VII Claes Lundstrom,¨ Patric Ljung and Anders Ynnerman. Multi-Dimensional Trans- fer Function Design Using Sorted Histograms In Proceedings Eurographics/IEEE International Workshop on Volume Graphics 2006. Boston, USA. 2006. VIII Claes Lundstrom,¨ Patric Ljung and Anders Ynnerman. Local histograms for de- sign of Transfer Functions in Direct Volume Rendering. To appear in IEEE Trans- actions on Visualization and Computer Graphics. 2006. IX Patric Ljung, Calle Winskog, Anders Persson, Claes Lundstrom¨ and Anders Yn- nerman. Full Body Virtual Autopsies Using A State-of-the-art Volume Rendering Pipeline. To appear in IEEE Transactions on Visualization and Computer Graph- ics (Proceedings Visualization 2006). Baltimore, USA. 2006. xii CONTENTS X Patric Ljung. Masters Thesis: Interactive Visualization of Particle In Cell Simu- lations, LiTH-ITN-EX–00/001–SE. Department of Science and Technology, Lin- koping¨ University, Linkoping,¨ Sweden. 2000 XI M. E. Dieckmann, P. Ljung, A. Ynnerman, and K. G. McClements. Large-scale numerical simulations of ion beam instabilities in unmagnetized astrophysical plasmas. Physics of Plasmas, 7:5171-5181, 2000. XII L. O. C. Drury, K. G. McClements, S. C. Chapman, R. O. Dendy, M. E. Dieck- mann, P. Ljung, and A. Ynnerman. Computational studies of cosmic ray electron injection. Proceedings of ICRC 2001. 2001. XIII Interactive Visualization of Large Scale Time Varying Data sets. Patric Ljung, Mark Dieckmann, and Anders Ynnerman. ACM SIGGRAPH 2002 Sketches and Applications. San Antonio, USA. 2002. XIV M. E. Dieckmann, P. Ljung, A. Ynnerman, and K. G. McClements. Three-Dimen- sional Visualization of Electron Acceleration in a Magnetized Plasma.