A Method for Viewing and Interacting with Medical Volumes in Virtual Reality Jordan King Williams Iowa State University
Total Page:16
File Type:pdf, Size:1020Kb
Iowa State University Capstones, Theses and Graduate Theses and Dissertations Dissertations 2019 A method for viewing and interacting with medical volumes in virtual reality Jordan King Williams Iowa State University Follow this and additional works at: https://lib.dr.iastate.edu/etd Part of the Computer Sciences Commons Recommended Citation Williams, Jordan King, "A method for viewing and interacting with medical volumes in virtual reality" (2019). Graduate Theses and Dissertations. 17125. https://lib.dr.iastate.edu/etd/17125 This Thesis is brought to you for free and open access by the Iowa State University Capstones, Theses and Dissertations at Iowa State University Digital Repository. It has been accepted for inclusion in Graduate Theses and Dissertations by an authorized administrator of Iowa State University Digital Repository. For more information, please contact [email protected]. A method for viewing and interacting with medical volumes in virtual reality by Jordan Williams A thesis submitted to the graduate faculty in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE Co-majors: Human Computer Interaction; Computer Engineering Program of Study Committee: Eliot Winer, Major Professor Adarsh Krishnamurthy James Oliver The student author, whose presentation of the scholarship herein was approved by the program of study committee, is solely responsible for the content of this thesis. The Graduate College will ensure this thesis is globally accessible and will not permit alterations after a degree is conferred. Iowa State University Ames, Iowa 2019 Copyright © Jordan Williams, 2019. All rights reserved. ii TABLE OF CONTENTS LIST OF FIGURES ....................................................................................................................... iv LIST OF JOURNAL FIGURES .................................................................................................... iv LIST OF TABLES ......................................................................................................................... vi ABSTRACT .................................................................................................................................. vii CHAPTER 1: INTRODUCTION ................................................................................................... 1 Uses of Medical Imaging Technology in the Field ..................................................................... 1 Volume Rendered Medical Images ............................................................................................. 2 VR HMDs ................................................................................................................................... 2 VR Controllers ............................................................................................................................ 3 Thesis Organization..................................................................................................................... 4 CHAPTER 2: BACKGROUND ..................................................................................................... 5 Graphic Rendering Techniques ................................................................................................... 5 Medical Imaging ......................................................................................................................... 8 2D medical imaging................................................................................................................. 8 Volume rendering methods ................................................................................................... 11 Virtual Reality ........................................................................................................................... 15 History of VR Viewing Methods........................................................................................... 16 Current HMDs ....................................................................................................................... 19 VR Controllers ....................................................................................................................... 21 Interacting with Medical Volumes in VR.............................................................................. 24 HMD Volume Rendering Concerns .......................................................................................... 27 CHAPTER 3: SOFTWARE AND HARDWARE COMPONENTS OF THE MEDICAL VOLUME VIEWER ..................................................................................................................... 31 OpenSceneGraph ....................................................................................................................... 31 VIPRE ....................................................................................................................................... 31 OpenVR ..................................................................................................................................... 32 Development Hardware............................................................................................................. 32 CHAPTER 4: INTERACTING WITH GPU RAYTRACED MEDICAL VOLUMES USING COMMODITY VIRTUAL REALITY HMDS ............................................................................ 33 Foreword ................................................................................................................................... 33 Abstract ..................................................................................................................................... 33 Introduction ............................................................................................................................... 34 iii Background ............................................................................................................................... 39 Volume Rendering of Medical Images .................................................................................. 39 Viewing Medical Images in VR ............................................................................................ 40 VR Hand Controls and Interaction ........................................................................................ 41 Methodology ............................................................................................................................. 42 Volume Rendering Components ........................................................................................... 42 Development Hardware ......................................................................................................... 44 Base Volume Rendering Functionality .................................................................................. 44 Adding OpenVR viewer to VIPRE ....................................................................................... 46 Raycasting modifications ...................................................................................................... 47 Finding eye position relative to volume ................................................................................ 49 GPU clipping planes .............................................................................................................. 49 Tissue Density Windowing ................................................................................................... 51 Designing a cross-platform control scheme .......................................................................... 52 Evaluation of feasibility and performance ............................................................................. 54 Conclusions and Future Work ............................................................................................... 59 References ................................................................................................................................. 60 CHAPTER 5: SUMMARY AND FUTURE WORK ................................................................... 63 REFERENCES ............................................................................................................................. 65 iv LIST OF FIGURES Figure 1: Raytraced volumetric chest cavity, courtesy of [4] ......................................................... 6 Figure 2: The programmable OpenGL Pipeline. Image from adobe.com/devnet/flashplayer/articles/vertex-fragment-shaders.html ........................................... 7 Figure 3: Clockwise from top: X-ray of chest, CT scan of chest, and MRI of head ...................... 9 Figure 4: Diagram of ray computation for volume raytracing, courtesy of [19] .......................... 12 Figure 5: Diagram of shear warp procedure, courtesy of Lacroute [20] ...................................... 15 Figure 6: Acer 3D display and Nvidia 3D shuttering glasses ....................................................... 16 Figure 7: Iowa State University’s C6 CAVE (left), Ivan Sutherland's Sword of Damocles HMD (right) ............................................................................................................................................ 18 Figure 8: From top-left proceeding clockwise, Oculus Rift, HTC Vive, Lenovo Explorer, HP headset, Del Visor, Samsung Odyssey, Acer headset (center) ..................................................... 19 Figure 9: Clockwise from top: HTC Vive Wand, Samsung HMD Odyssey controller, Oculus touch .............................................................................................................................................