The Optical Coherence Tomography Microsurgical Augmented Reality System (OCT-MARS): A Novel Device for Microsurgeries Samantha Horvath CMU-RI-TR-16-56 Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Robotics. The Robotics Institute Carnegie Mellon University Pittsburgh, Pennsylvania 15213 September 2016 Thesis Committee: Dr. George Stetten Dr. John Galeotti Dr. Roberta Klatzky Dr. Thomas Furness Sponsors: NIH R01EY021641 NSF GRFP DGE-1252522 DoD CDMRP Research to Prevent Blindness Copyright © 2016 by Samantha Horvath. All rights reserved 1 Abstract This thesis describes the development and testing of the Optical Coherence Tomography Microsurgical Augmented Reality System (OCT-MARS). This system allows surgeons to view real-time medical image data as an in situ overlay within the surgical field. There are a number of clinical applications for which real time, in situ visualization of otherwise transparent structures of the eye would be beneficial to surgeons. The primary motivating application for this project is the surgical treatment of glaucoma. We have built a projection system capable of producing flat and tilted images in the normal field of view of the microscope with sufficient brightness and resolution to be viewed under magnification. We have studied the perception of tilted surfaces under magnification and found that OCT images provide sufficient stereo information to be correctly perceived. Finally, we have tested stereo perception under magnification using surgically relevant tasks to evaluate the effectiveness of the system. 2 Acknowledgements Advisors Dr. George Stetten and Dr. John Galeotti Thesis committee members Dr. Roberta Klatzky and Dr. Thomas Furness Fellow students, esp. Tejas Mathai, Jihang Wang, Kori MacDonald and Daniel Freer The Drs. Horvath. They told me I shouldn’t do this. They were probably right. 3 4 Table of Contents List of Figures ............................................................................................................................ 7 List of Tables ............................................................................................................................. 9 1 Introduction ...................................................................................................................... 11 2 Background ...................................................................................................................... 13 2.1 OCT Details ............................................................................................................. 13 2.2 Clinical motivation ................................................................................................... 14 2.3 Augmented Reality Systems .................................................................................... 15 2.4 3D Perception Under Magnification ........................................................................ 16 3 Aim 1: The OCT-MARS ................................................................................................. 19 3.1 OCT Scanner ............................................................................................................ 19 3.1.1 Hardware Basics .............................................................................................. 19 3.1.2 Real time 3D volume scanning ........................................................................ 20 3.1.3 Hardware Modifications .................................................................................. 20 3.1.4 Software modification – LabVIEW ................................................................. 21 3.1.5 Software modification – C++/CUDA .............................................................. 21 3.2 Laser Refocusing System ......................................................................................... 21 3.2.1 Overview .......................................................................................................... 21 3.2.2 Microvision Projector ...................................................................................... 21 3.2.3 Single Lens Focusing ....................................................................................... 22 3.2.4 Multiple Lens Focusing ................................................................................... 23 3.3 OCT MARS Version 1 ............................................................................................. 25 3.4 OCT MARS Version 2 ............................................................................................. 25 3.4.1 Design .............................................................................................................. 25 3.4.2 Depth of Field Analysis ................................................................................... 26 3.4.3 Verification ...................................................................................................... 27 3.5 Calibration................................................................................................................ 29 3.5.1 Step 1 ............................................................................................................... 29 3.5.2 Step 2 ............................................................................................................... 29 3.6 Limitations of the experimental OCT system .......................................................... 30 3.6.1 Initial System Damage ..................................................................................... 30 3.6.2 Issues with PSI support .................................................................................... 30 3.6.3 Theoretical Concerns ....................................................................................... 30 4 Aim 2: Slant Perception in Microscopy ........................................................................... 33 4.1 Scale-invariant Texture Generation ......................................................................... 33 4.1.1 Implementation ................................................................................................ 34 4.1.2 Spectral analysis ............................................................................................... 37 4.1.3 Invariance under scale ...................................................................................... 38 5 4.2 Experiment 1 ............................................................................................................ 39 4.2.1 Experiment 1A ................................................................................................. 39 4.2.2 Experiment 1B ................................................................................................. 44 4.2.3 Discussion ........................................................................................................ 45 4.3 Experiment 2 ............................................................................................................ 46 4.3.1 Experiment 2A ................................................................................................. 46 4.3.2 Experiment 2B ................................................................................................. 50 4.4 Experiment 3 ............................................................................................................ 51 4.4.1 Overview .......................................................................................................... 51 4.4.2 Lighting Changes ............................................................................................. 52 4.4.3 Alignment Tool ................................................................................................ 52 4.4.4 LEAP Motion Tracker ..................................................................................... 54 4.4.5 Masked OCT Stimulus ..................................................................................... 55 4.4.6 Experiment 3A ................................................................................................. 56 4.4.7 Experiment 3B ................................................................................................. 58 5 Aim 3: Preclinical Validation .......................................................................................... 61 5.1 Overview .................................................................................................................. 61 5.2 Custom phantoms..................................................................................................... 61 5.3 Commercial/industry phantoms ............................................................................... 62 5.4 System Test Images ................................................................................................. 63 5.4.1 OCT MARS v1 ................................................................................................ 63 5.4.2 OCT MARS v2 ................................................................................................ 64 5.5 Limitations of this study .......................................................................................... 65 6 Conclusion ....................................................................................................................... 67 6.1 OCT-MARS Implementation ................................................................................... 67 6.1.1 Current Progress ..............................................................................................