LiU-ITN-TEK-A--11/045--SE Illustrative Visualization of Anatomical Structures Erik Jonsson 2011-08-19 Department of Science and Technology Institutionen för teknik och naturvetenskap Linköping University Linköpings universitet SE-601 74 Norrköping, Sweden 601 74 Norrköping LiU-ITN-TEK-A--11/045--SE Illustrative Visualization of Anatomical Structures Examensarbete utfört i medieteknik vid Tekniska högskolan vid Linköpings universitet Erik Jonsson Examinator Karljohan Lundin Palmerius Norrköping 2011-08-19 Upphovsrätt Detta dokument hålls tillgängligt på Internet – eller dess framtida ersättare – under en längre tid från publiceringsdatum under förutsättning att inga extra- ordinära omständigheter uppstår. Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervisning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säkerheten och tillgängligheten finns det lösningar av teknisk och administrativ art. Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sammanhang som är kränkande för upphovsmannens litterära eller konstnärliga anseende eller egenart. För ytterligare information om Linköping University Electronic Press se förlagets hemsida http://www.ep.liu.se/ Copyright The publishers will keep this document online on the Internet - or its possible replacement - for a considerable time from the date of publication barring exceptional circumstances. The online availability of the document implies a permanent permission for anyone to read, to download, to print out single copies for your own use and to use it unchanged for any non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional on the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility. According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement. For additional information about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its WWW home page: http://www.ep.liu.se/ © Erik Jonsson Abstract Illustrative visualization is a term for visualization techniques inspired by tradi- tional technical and medical illustration. These techniques are based on knowl- edge of the human perception and provide effective visual abstraction to make the visualizations more understandable. Within volume rendering these expressive visualizations can be achieved using non-photorealistic rendering that combines different levels of abstraction to convey the most important information to the viewer. In this thesis I will look at illustrative techniques and show how these can be used to visualize anatomical structures in a medical volume data. The result of the thesis is a prototype of an anatomy education application, that makes use of illustrative techniques to have a focus+context visualization with feature enhancement, tone shading and labels describing the anatomical struc- tures. This results in an expressive visualization and interactive exploration of the human anatomy. 1 Acknowledgements I would like to thank my supervisor Karl-Johan Lundin Palmerius and Lena Tibell at the Department of Science and Technology, Linköping University for their help and assistance throughout the thesis work. Thanks also to Daniel Forsberg at the Department of Biomedical Engineering, Linköping University for providing the human body data set together with the segmented data. 2 Contents 1 Introduction 7 1.1 Motivation . .7 1.2 Purpose & Goal . .7 1.3 Limitations . .8 1.4 Outline . .8 2 Background 9 2.1 Anatomy Education . .9 2.1.1 Dissections . .9 2.2 Volume Rendering . 10 2.2.1 Volume Rendering Integral . 10 2.2.2 Segmented Volume Data . 11 2.2.3 Ray Casting . 11 2.2.4 GPU-based Ray Casting . 12 2.2.5 Transfer Functions . 14 2.2.6 Local Illumination . 14 2.3 Illustrative Visualization . 15 2.3.1 Medical Illustrations . 15 2.3.2 Visual Abstraction . 16 2.3.3 Cut-away Views and Ghosted Views . 16 2.3.4 Visibility Control . 18 2.3.5 Textual Annotations . 18 2.4 Voreen . 19 3 Theory 21 3.1 The Importance-aware Composition Scheme . 21 3.2 The Tone Shading Model . 22 4 Implementation 24 4.1 Illustrative Ray Casting . 24 4.1.1 Segmentation Classification . 24 4.1.2 Tone Shading . 25 4.1.3 Importance-aware Composition . 26 4.2 Labeling of Segmented Data . 28 4.2.1 Segment Description File . 29 4.2.2 Layout Algorithm . 29 4.2.3 Rendering . 30 4.3 Anatomy Application . 31 4.3.1 Design and User Interface . 31 4.3.2 Focus+Context Widget . 31 4.3.3 Labeling Widget . 32 5 Conclusion 35 5.1 Results . 35 5.1.1 Result of the Importance-aware Composition . 35 5.1.2 Result of the Tone Shading . 37 5.1.3 Result of the Anatomy Application . 37 5.1.4 Performance . 38 5.2 Discussion . 40 5.2.1 The Illustrative Techniques . 40 5.2.2 The Anatomy Application . 40 5.3 Future work . 41 5.3.1 Additional Features . 41 4 List of Figures 2.1 The front and back face from the bounding box of the volume . 13 2.2 The ray casting technique via rasterization . 13 2.3 A transfer function represented by a 1D texture . 14 2.4 Cut-away and ghosted illustration of a sphere . 17 2.5 Medical illustrations by Leonardo da Vinci . 17 2.6 The standard workspace in VoreenVE . 19 3.1 Tone shading of a red object with blue/yellow tones . 23 4.1 1D TF textures stored in a 2D segmentation TF texture . 25 4.2 Tone shading parameters . 26 4.3 Importance Measurements Parameters . 27 4.4 Convex hull: A set of points enclosed by an elastic band . 30 4.5 The placement of labels . 30 4.6 The network of the anatomy application . 32 4.7 Layout of the Labeling widget . 33 5.1 The intensity measurement . 36 5.2 The gradient magnitude, silhouetteness and background measure- ment.................................. 36 5.3 Focus+context visualization . 36 5.4 Comparision of Blinn-Phong shading and Tone shading . 37 5.5 The Anatomy Application: Selection on Pericardium . 38 5.6 The Anatomy Application: The Digestive and Urinary System . 39 List of Tables 5.1 Performance measurements of front-to-back composition and importance- aware composition with different settings on importance measure- ments (IM) and early ray termination (ERT). 38 5.2 Performance measurement of tone shading and Blinn-Phong shad- ing using front-to-back composition. 38 Chapter 1 Introduction In this Master’s thesis an illustrative volume rendering system has been devel- oped at the division for Media and Information Technology, Department of Sci- ence and Technology at Linköping University. Illustrative techniques are used in the system to achieve an expressive visualization of anatomical structures. The thesis serves as a fulfillment of a Master of Science degree in Media Technology at Linköping University, Sweden. 1.1 Motivation The study of medicine and biology has always relied on visualizations to learn about anatomical relationships and structures. In these studies are dissections often used to support the anatomical learning with both visual and tactile expe- rience. However, the use of dissection is declining for schools that have anatomi- cal education [14]. High schools and universities are more often using other aids such as textbooks, plastic specimens and simulators to support their anatomy education. The computerized aids offer many new possibilities, where simulators and educational software lets the user explore anatomical structures in three dimen- sions. Often these applications are using surface rendering to render pre-modeled 3D models. However, through a technique called volume rendering the structures can be rendered directly from the medical data. Volume rendering has for a long time been considered as much slower than surface rendering, but with newer GPU’s it is possible to achieve interactive frame rates. With volume rendering it is possible to acquire renders that better corresponds with the real material. The density values in the medical data sets are directly mapped to RGBA values for the pixels in the rendered images. This allows for fuzzy surfaces with varying opacity, where surface and internal details can be rendered together, for example material such as soft tissue and blood vessels. 1.2 Purpose & Goal In this thesis an interactive volume visualization system for illustrative visual- ization and exploration of medical volume data is proposed. The purpose with the thesis is to develop a volume rendering application for anatomy education, which allows the user to interactively explore anatomical structures in a medical 7 8 Introduction data set. The goal of using illustrative techniques is to achieve an expressive vi- sualization, where complex data is conveyed in an intuitive and understandable way. Otherwise, the information can quickly overwhelm the user, which makes it harder for the user to convey the information. The goal with the thesis is to achieve illustrative visualization of anatomical structures and to show its use in an application for anatomy education. 1.3 Limitations The application in this thesis is based on research material and is developed as a proof-of-concept, where the potential of the methods are evaluated. This means that the user’s satisfaction is not evaluated and no user requirements are collected. Otherwise, the user’s need and opinions in such an application would have been questioned.