BAYESIAN MODELS ON MANIFOLDS FOR IMAGE REGISTRATION AND STATISTICAL SHAPE ANALYSIS by Miaomiao Zhang A dissertation submitted to the faculty of The University of Utah in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computing School of Computing The University of Utah August 2016 Copyright c Miaomiao Zhang 2016 All Rights Reserved The University of Utah Graduate School STATEMENT OF DISSERTATION APPROVAL The dissertation of Miaomiao Zhang has been approved by the following supervisory committee members: Preston Thomas Fletcher , Chair 10/30/2015 Date Approved Guido Gerig , Member 11/10/2015 Date Approved Sarang Joshi , Member 10/30/2015 Date Approved Robert Michael Kirby , Member 10/30/2015 Date Approved Laurent Younes , Member 10/30/2015 Date Approved and by Ross T. Whitaker, Chair of the School of Computing and by David B. Kieda, Dean of The Graduate School. ABSTRACT An important area of medical imaging research is studying anatomical diffeomorphic shape changes and detecting their relationship to disease processes. For example, neurode- generative disorders change the shape of the brain, thus identifying differences between the healthy control subjects and patients affected by these diseases can help with understanding the disease processes. Previous research proposed a variety of mathematical approaches for statistical analysis of geometrical brain structure in three-dimensional (3D) medical imaging, including atlas building, brain variability quantification, regression, etc. The critical component in these statistical models is that the geometrical structure is represented by transformations rather than the actual image data. Despite the fact that such statistical models effectively provide a way for analyzing shape variation, none of them have a truly probabilistic interpretation. This dissertation contributes a novel Bayesian framework of statistical shape analysis for generic manifold data and its application to shape variability and brain magnetic resonance imaging (MRI). After we carefully define the distributions on manifolds, we then build Bayesian models for analyzing the intrinsic variability of manifold data, involving the mean point, principal modes, and parameter estimation. Because there is no closed-form solution for Bayesian inference of these models on manifolds, we develop a Markov Chain Monte Carlo method to sample the hidden variables from the distribution. The main advantages of these Bayesian approaches are that they provide parameter estimation and automatic dimensionality reduction for analyzing generic manifold-valued data, such as diffeomorphisms. Modeling the mean point of a group of images in a Bayesian manner allows for learning the regularity parameter from data directly rather than having to set it manually, which eliminates the effort of cross validation for parameter selection. In population studies, our Bayesian model of principal modes analysis (1) automatically extracts a low-dimensional, second-order statistics of manifold data variability and (2) gives a better geometric data fit than nonprobabilistic models. To make this Bayesian framework computationally more efficient for high-dimensional diffeomorphisms, this dissertation presents an algorithm, FLASH (finite-dimensional Lie algebras for shooting), that hugely speeds up the diffeomorphic image registration. In- stead of formulating diffeomorphisms in a continuous variational problem, Flash defines a completely new discrete reparameterization of diffeomorphisms in a low-dimensional ban- dlimited velocity space, which results in the Bayesian inference via sampling on the space of diffeomorphisms being more feasible in time. Our entire Bayesian framework in this dissertation is used for statistical analysis of shape data and brain MRIs. It has the potential to improve hypothesis testing, classification, and mixture models. iv To my family CONTENTS ABSTRACT :::::::::::::::::::::::::::::::::::::::::::::::::::::::: iii LIST OF FIGURES ::::::::::::::::::::::::::::::::::::::::::::::::: ix LIST OF TABLES ::::::::::::::::::::::::::::::::::::::::::::::::::: xii ACKNOWLEDGEMENTS ::::::::::::::::::::::::::::::::::::::::::: xiii CHAPTERS 1. INTRODUCTION ::::::::::::::::::::::::::::::::::::::::::::::: 1 1.1 Problems and Challenges........................................3 1.2 Dissertation Statement and Contributions...........................4 1.3 Outline......................................................5 2. BACKGROUND AND RELATED WORKS :::::::::::::::::::::::: 7 2.1 Riemannian Manifolds...........................................7 2.1.1 Riemannian Metrics........................................7 2.1.2 Geodesics................................................8 2.1.3 Lie Groups...............................................9 2.1.4 Left and Right Invariant Metrics.............................. 11 2.1.5 Jacobi Fields.............................................. 11 2.1.6 Adjoint Representation...................................... 13 2.2 Statistics on Manifolds.......................................... 14 2.2.1 Bayesian Principal Component Analysis......................... 14 2.2.2 Principal Geodesic Analysis.................................. 15 2.2.3 Geodesic Regression and Splines............................... 17 2.3 Diffeomorphic Image Registration.................................. 19 2.3.1 Diffeomorphisms........................................... 19 2.3.2 Metrics on Diffeomorphisms.................................. 20 2.3.3 LDDMM With Geodesic Shooting............................. 21 2.3.4 Diffeomorphic Atlas Building................................. 23 3. BAYESIAN ESTIMATION OF REGULARIZATION AND ATLAS BUILDING :::::::::::::::::::::::::::::::::::::::::::::: 25 3.1 Related Work................................................. 25 3.2 A Bayesian Model for Diffeomorphic Atlas Building................................................. 26 3.3 Estimation of Model Parameters................................... 27 3.3.1 Hamiltonian Monte Carlo (HMC) Sampling...................... 29 3.3.2 The Maximization Step...................................... 30 3.4 Results....................................................... 30 3.4.1 Parameter Estimation on Synthetic Data........................ 31 3.4.2 Atlas Building on 3D Brain Images............................ 32 3.4.3 Image Matching Accuracy.................................... 33 3.5 Mixture Model of Diffeomorphic Multiatlas Building................... 35 3.5.1 Our Mixture Model......................................... 35 3.6 Inference..................................................... 37 3.6.1 E-step................................................... 37 3.6.2 M-step................................................... 38 3.7 Results....................................................... 38 3.7.1 Synthetic Data............................................ 38 3.7.2 OASIS Brain Data......................................... 39 3.8 Conclusion.................................................... 40 4. PROBABILISTIC PRINCIPAL GEODESIC ANALYSIS :::::::::::: 42 4.1 Related Work................................................. 42 4.2 Probabilistic Principal Geodesic Analysis............................ 43 4.2.1 Inference................................................. 44 4.3 Experiments.................................................. 47 4.3.1 Simulated Sphere Data...................................... 47 4.3.2 Shape Analysis of the Corpus Callosum......................... 48 4.4 Automatic Data Dimensionality Reduction.......................... 51 4.5 Conclusion.................................................... 51 5. BAYESIAN PRINCIPAL GEODESIC ANALYSIS OF DIFFEOMORPHIC SHAPE VARIABILITY ::::::::::::::::::: 52 5.1 Overview..................................................... 52 5.2 Probability Model.............................................. 54 5.3 Inference..................................................... 56 5.4 Results....................................................... 57 5.4.1 Synthetic Data............................................ 58 5.4.2 OASIS Brain Dataset....................................... 59 5.5 Conclusion and Future Work...................................... 63 6. LOW DIMENSIONAL LIE ALGEBRAS FOR GEODESIC SHOOTING ::::::::::::::::::::::::::::::::::::::::: 64 6.1 Overview..................................................... 64 6.2 Low-Dimensional Lie Algebras.................................... 66 6.2.1 Space of Bandlimited Velocity Fields and Metrics................. 67 6.2.2 Low-Dimensional Lie Bracket................................. 68 6.2.3 EPDiff Equation in Low-Dimensional Lie Algebras................ 70 6.3 Estimation of Diffeomorphic Image Registration...................... 71 6.3.1 Reduced Adjoint Jacobi Fields in Bandlimited Velocity Space........ 72 6.4 Complexity Analysis............................................ 73 6.5 Results....................................................... 75 6.5.1 Synthetic Data............................................ 75 6.5.2 3D Brain Image Registration................................. 75 6.5.3 Atlas Building............................................. 79 6.6 Conclusion and Future Work...................................... 80 vii 7. DISCUSSION AND FUTURE WORK ::::::::::::::::::::::::::::: 82 7.1 Summary of Contributions....................................... 82 7.2 Future Work.................................................. 84 7.2.1 Open Theoretical Problems.................................. 85 7.2.2 Related Future Work