University of Central Florida STARS Electronic Theses and Dissertations, 2004-2019 2018 Visionary Ophthalmics: Confluence of Computer Vision and Deep Learning for Ophthalmology Dustin Morley University of Central Florida Part of the Computer Sciences Commons Find similar works at: https://stars.library.ucf.edu/etd University of Central Florida Libraries http://library.ucf.edu This Doctoral Dissertation (Open Access) is brought to you for free and open access by STARS. It has been accepted for inclusion in Electronic Theses and Dissertations, 2004-2019 by an authorized administrator of STARS. For more information, please contact [email protected]. STARS Citation Morley, Dustin, "Visionary Ophthalmics: Confluence of Computer Vision and Deep Learning for Ophthalmology" (2018). Electronic Theses and Dissertations, 2004-2019. 5793. https://stars.library.ucf.edu/etd/5793 VISIONARY OPHTHALMICS: CONFLUENCE OF COMPUTER VISION AND DEEP LEARNING FOR OPHTHALMOLOGY by DUSTIN MORLEY M.S. University of Central Florida, 2016 B.S. University of Central Florida, 2012 A dissertation submitted in partial fulfilment of the requirements for the degree of Doctor of Philosophy in the Department of Computer Science in the College of Engineering and Computer Science at the University of Central Florida Orlando, Florida Spring Term 2018 Major Professor: Hassan Foroosh c 2018 Dustin Morley ii ABSTRACT Ophthalmology is a medical field ripe with opportunities for meaningful application of computer vision algorithms. The field utilizes data from multiple disparate imaging techniques, ranging from conventional cameras to tomography, comprising a diverse set of computer vision chal- lenges. Computer vision has a rich history of techniques that can adequately meet many of these challenges. However, the field has undergone something of a revolution in recent times as deep learning techniques have sprung into the forefront following advances in GPU hardware. This de- velopment raises important questions regarding how to best leverage insights from both modern deep learning approaches and more classical computer vision approaches for a given problem. In this dissertation, we tackle challenging computer vision problems in ophthalmology using methods all across this spectrum. Perhaps our most significant work is a highly successful iris registration algorithm for use in laser eye surgery. This algorithm relies on matching features extracted from the structure tensor and a Gabor wavelet – a classically driven approach that does not utilize modern machine learning. However, drawing on insight from the deep learning revolution, we demonstrate successful application of backpropagation to optimize the registration significantly faster than the alternative of relying on finite differences. Towards the other end of the spectrum, we also present a novel framework for improving RANSAC segmentation algorithms by utilizing a convolutional neural network (CNN) trained on a RANSAC-based loss function. Finally, we apply state-of-the- art deep learning methods to solve the problem of pathological fluid detection in optical coherence tomography images of the human retina, using a novel retina-specific data augmentation technique to greatly expand the data set. Altogether, our work demonstrates benefits of applying a holistic view of computer vision, which leverages deep learning and associated insights without neglecting techniques and insights from the previous era. iii This dissertation is dedicated to my loving and supportive wife, Rebecca. Her patience and encouragement were critical throughout my graduate studies. I am also grateful to the many family members, friends, and colleagues who have regularly and enthusiastically offered encouragement. iv ACKNOWLEDGMENTS This work would not have been possible without the financial support of LENSAR, Inc. In addition to the financial support, LENSAR provided an ideal environment for me to complete my studies while working full time, and I am extremely grateful to LENSAR for that. To this end, I would like to specifically thank Gary Gray and Alan Connaughton for the roles they played in establishing and maintaining this environment. I would also like to thank Glen Martin, Art Newton, and Valas Teuma for their overall support and all of the great technical conversations we have had, as well as Keith Peck for providing valuable feedback during the development of some of the novel statistical methods used in this dissertation. I am grateful to many UCF professors for their instruction and guidance, as I can honestly say that I thoroughly enjoyed every course I took throughout my time as a graduate student in computer science. I would especially like to thank Dr. Hassan Foroosh, the chairman of my dissertation committee, for his professional and academic guidance. Also from my committee, I would like to thank Dr. Ulas Bagci for connecting me to the field of OCT image analysis, and Dr. Boqing Gong for planting the seeds that led to my research and contributions at the boundary between machine learning and more traditional computer vision methods. v TABLE OF CONTENTS LIST OF FIGURES . xi LIST OF TABLES . xiv CHAPTER 1: INTRODUCTION . 1 Medical Computer Vision . .3 Computer Vision in Ophthalmology . .4 Designed Algorithms vs. Learned Algorithms . .5 Backpropagation - With or Without Machine Learning . .7 CHAPTER 2: LITERATURE REVIEW . 11 RANSAC . 11 Automatic Iris Registration . 12 Convolutional Neural Networks . 16 Backpropagation . 19 Retina Fluid . 20 CHAPTER 3: COMPUTING CYCLOTORSION IN REFRACTIVE CATARACT SURGERY vi 21 Relevant Prior Work . 23 Proposed Method . 26 Boundary Detection . 27 Filtering and Unwrapping the Iris . 33 Feature Extraction . 35 Measuring Correlation Strength . 37 Extracting and Applying the Angle of Cyclotorsion . 40 Data Collection and Validation . 42 Experiments . 48 Impact of Pupil Dilation . 48 Efficacy of Masking Out Eyelids . 49 Importance of Centration for Unwrapping . 49 Radial Shear Efficacy and Error Rates . 50 Discussion . 60 Conclusion . 62 CHAPTER 4: IRIS REGISTRATION WITH OPTIMIZED UNWRAPPING . 63 vii Introduction . 63 Method . 64 Review of Prior Method . 66 Optimizing the Unwrapping Center . 69 Experiments . 77 Registration Efficacy . 77 Benefits of Backpropagation . 81 Significance of Final Unwrapping Center . 83 Conclusion . 86 CHAPTER 5: IMPROVING RANSAC SEGMENTATION THROUGH CNN ENCAPSU- LATION . 88 Introduction . 88 Related Work . 91 Method . 93 Preprocessing . 93 Feature Extraction . 94 Clutter Removal . 95 viii RANSAC Fitting and Backpropagation . 96 Parameters . 98 Experiments . 99 Base Configuration Definition . 100 Base Configuration Results . 100 Hyperparameter Variation . 105 Alternate Configurations . 106 Reduced Training Set . 107 Discussion . 107 CHAPTER 6: SIMULTANEOUS DETECTION AND QUANTIFICATION OF RETINAL FLUID WITH DEEP LEARNING . 109 Introduction . 109 Related Work . 109 Method . 110 Pre-Processing. 110 Data Augmentation. 111 CNN Architecture. 112 ix Post-Processing. 116 Results . 117 Experiments on RETOUCH Data Set . 118 Experiments on Alternate Data Set . 121 Conclusion . 124 CHAPTER 7: CONCLUSION . 125 APPENDIX : COPYRIGHT INFORMATION . 129 LIST OF REFERENCES . 135 x LIST OF FIGURES Figure 2.1: Google Scholar search results for "Convolutional Neural Network" over time. 17 Figure 3.1: Boundary detection for a LLS image . 29 Figure 3.2: Boundary detection for a Cassini image . 30 Figure 3.3: Image filtering procedure for eyelid interference detection. 32 Figure 3.4: Example results of eyelid interference detection. 32 Figure 3.5: Locating the innermost suction ring in an LLS image. 33 Figure 3.6: Unwrapped, DOG filtered iris (LLS top, topographer bottom). 35 Figure 3.7: Correlation measures as a function of proposed cyclotorsion angle. 39 Figure 3.8: Confidence score function based on peak height ratio. 41 Figure 3.9: Example registration result with highlighted matching sections. 43 Figure 3.10: Correlation plots extended to ±180 degrees. 46 Figure 3.11: Cyclotorsion-corrected correlation coefficient as a function of pupil size difference. ..