LiU-ITN-TEK-A--10/045--SE Real-time DVR Illumination Methods for Ultrasound Data Erik Sundén 2010-06-07 Department of Science and Technology Institutionen för teknik och naturvetenskap Linköping University Linköpings Universitet SE-601 74 Norrköping, Sweden 601 74 Norrköping LiU-ITN-TEK-A--10/045--SE Real-time DVR Illumination Methods for Ultrasound Data Examensarbete utfört i vetenskaplig visualisering vid Tekniska Högskolan vid Linköpings universitet Erik Sundén Handledare Patric Ljung Examinator Karljohan Lundin Palmerius Norrköping 2010-06-07 Upphovsrätt Detta dokument hålls tillgängligt på Internet – eller dess framtida ersättare – under en längre tid från publiceringsdatum under förutsättning att inga extra- ordinära omständigheter uppstår. Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervisning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säkerheten och tillgängligheten finns det lösningar av teknisk och administrativ art. Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sammanhang som är kränkande för upphovsmannens litterära eller konstnärliga anseende eller egenart. För ytterligare information om Linköping University Electronic Press se förlagets hemsida http://www.ep.liu.se/ Copyright The publishers will keep this document online on the Internet - or its possible replacement - for a considerable time from the date of publication barring exceptional circumstances. The online availability of the document implies a permanent permission for anyone to read, to download, to print out single copies for your own use and to use it unchanged for any non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional on the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility. According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement. For additional information about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its WWW home page: http://www.ep.liu.se/ © Erik Sundén Abstract Ultrasound (US) volume data is noisy, so traditional methods for direct volume ren- dering (DVR) are less appropriate. Improved methods or new techniques are required. There are furthermore a high performance requirement and limited pre-processing to be considered in order for it to be used interactively, since the volume data might be time- varying. There exist numerous techniques for improving visual perception of volume rendering, and while some perform well and produce a visually enhanced result, many are designed and compared for use with medical data that has a high signal-to-noise ra- tio. This master thesis describe and compare recent methods for DVR illumination, in the form of ambient occlusion or direct/indirect lighting from an external light source. New designs and modifications are introduced for efficiently and effectively enhancing the visual quality of DVR withUS data. Furthermore, this thesis addresses the issue of how clipping is performed during rendering and for the different illumination tech- niques, which is commonly used in ultrasound visualization. This diploma work was conducted at Siemens Corporate Research in Princeton, NJ where the partially open source framework XIP is developed. The framework was extended further to include modern methods for DVR illumination that are described in detail within this thesis. Finally, presented results show that several methods can be used to visually enhance the visualization within highly interactive frame-rates. placeholder Acknowledgements I would like to thank my supervisor Patric Ljung for his support and expertise in this field during this work. Special thanks goes to my examiner Karljohan Lundin Palmerius as well as my colleagues Christoph Ruß, Oliver Kutter and Stefan Lindholm for taking part in important discussions and the development of this work. Foremost, I would like to thank my family as well as my beloved Kristin, how has supported me and endured too much time apart in order for me to finish my thesis. placeholder Contents 1 Introduction 1 1.1 Motivation........................................1 1.2 Aim............................................1 1.3 Outline..........................................2 1.4 Limitations........................................2 2 Background 3 2.1 Volumetric Ultrasound Data...............................3 2.1.1 3D US.......................................3 2.1.2 4D US.......................................4 2.1.3 Problem Areas..................................4 2.2 GPU Pipeline and Programming............................5 2.3 Direct Volume Rendering................................6 2.3.1 Optical Properties................................6 2.3.2 Approximation of the Volume Rendering Integral...............7 2.3.3 Ray Casting....................................8 2.3.4 Texture Slicing..................................8 2.3.5 Transfer Functions and Lookup Tables.....................9 2.3.6 Opacity Correction................................9 2.4 Illumination........................................ 10 2.4.1 Local Volume Illumination........................... 10 2.4.2 Global Volume Illumination........................... 11 3 Related Work 13 3.1 Ambient Occlusion.................................... 13 3.2 Lighting and Shadows.................................. 14 4 Approach 17 4.1 Develop Environment.................................. 17 4.2 Ambient Occlusion.................................... 17 4.3 Lighting and Shadows.................................. 18 4.4 Possible Optimizations Aspects............................. 18 4.4.1 Render to Texture................................ 18 4.4.2 Clipping...................................... 19 CONTENTS 5 Theory and Implementation 21 5.1 Algorithms for correct and visually improved interactive rendering......... 21 5.1.1 Stochastic Jittering................................ 21 5.1.2 Self Shading.................................... 21 5.2 Local Ambient Occlusion................................ 22 5.2.1 Sampling Distribution.............................. 23 5.2.2 Local Directional Occlusion........................... 24 5.2.3 Performance Optimizations........................... 24 5.3 Normal Hemisphere Sampling.............................. 25 5.4 Screen Space Ambient Occlusion............................ 26 5.5 Shadow Mapping..................................... 27 5.5.1 Projective Texturing............................... 28 5.5.2 Variance Shadow Mapping........................... 28 5.5.3 Depth Map Generation............................. 29 5.5.4 Approximation of Subsurface Scattering.................... 30 5.6 Volume Lighting with Half-Angle Slicing........................ 31 5.6.1 Scattering within Half-Angle Slicing...................... 32 5.7 Interactive Pre-calculated Volumetric Lighting.................... 33 5.7.1 Permutation................................... 34 5.7.2 Render to 3D Texture.............................. 34 5.7.3 Scattering within IVL.............................. 35 5.7.4 Local Gradient-free Shading........................... 36 5.8 Clipping.......................................... 36 5.8.1 Clipping for LAO................................. 37 5.8.2 Clipping for IVL................................. 37 5.9 Manipulated Light Source................................ 38 6 Results and Discussion 39 6.1 Ambient Occlusion.................................... 39 6.2 Direct and Indirect Lighting............................... 40 6.3 Benefits and Limitations................................. 41 7 Conclusions and Possibilities 43 7.1 Known Limitations.................................... 44 7.2 Further Work....................................... 44 Glossary 48 References 49 A Additional Screenshots 53 List of Figures 2.1 Volume Imaging Ultrasound Station (ACUSON SC2000 from Siemens Healthcare).4 2.2 The programmable graphics pipeline...........................5 2.3 Optical properties that occur when light interact with media, which affects the radiance along the ray. Illustration based on [6](p.5)..................7 2.4 Volume ray casting ofCT head with emission-absorption model only.........8 2.5 Illustration of volume ray casting(left) and texture slicing(right)...........9 2.6 Representation of a 1D LUT, which is used within most rendered screenshots in this thesis............................................9 2.7 Illustration of the volumetric illumination models known as single scattering(left), single scattering with attenuation(middle) and multiple scattering(right). Based on [6](p.104)......................................... 10 2.8 DVR of non-smoothed ultrasound dataset rendered with shadows(left), ambient occlusion(middle) and volumetric shadowing with scattering(right). Meth- ods used within these screenshots are shadow mapping(left), local ambient