Metric learning for diffeomorphic image registration. Fran¸cois-Xavier Vialard Metric learning for diffeomorphic image registration. Fran¸cois-XavierVialard Universit´eParis-Est Marne-la-Vall´ee joint work with M. Niethammer and R. Kwitt. IHP, March 2019. Metric learning for diffeomorphic image Outline registration. Fran¸cois-Xavier Vialard 1 Introduction to diffeomorphisms group and Riemannian tools 2 Choice of the metric 3 Spatially dependent metrics 4 Metric learning 5 SVF metric learning Metric learning for diffeomorphic image Example of problems of interest registration. Fran¸cois-Xavier Vialard Introduction to diffeomorphisms group and Riemannian tools Choice of the metric Spatially dependent metrics Metric learning SVF metric learning Given two shapes, find a diffeomorphism of R3 that maps one shape onto the other Metric learning for diffeomorphic image Example of problems of interest registration. 3 Fran¸cois-Xavier Given two shapes, find a diffeomorphism of R that maps one Vialard shape onto the other Introduction to diffeomorphisms group and Riemannian tools Different data types and different way of representing them. Choice of the metric Spatially dependent metrics Metric learning SVF metric learning Figure{ Two slices of 3D brain images of the same subject at different ages Metric learning for diffeomorphic image Example of problems of interest registration. 3 Given two shapes, find a diffeomorphism of R that maps one Fran¸cois-Xavier shape onto the other Vialard Introduction to diffeomorphisms group and Riemannian tools Choice of the metric Spatially dependent metrics Metric learning SVF metric learning Deformation by a diffeomorphism Figure{ Diffeomorphic deformation of the image Metric learning for diffeomorphic image Variety of shapes registration. Fran¸cois-Xavier Vialard Introduction to diffeomorphisms group and Riemannian tools Choice of the metric Spatially dependent metrics Metric learning SVF metric learning Figure{ Different anatomical structures extracted from MRI data Metric learning for diffeomorphic image Variety of shapes registration. Fran¸cois-Xavier Vialard Introduction to diffeomorphisms group and Riemannian tools Choice of the metric Spatially dependent metrics Metric learning SVF metric learning Figure{ Different anatomical structures extracted from MRI data Metric learning for diffeomorphic image A Riemannian approach to diffeomorphic registration. Fran¸cois-Xavier registration Vialard Introduction to diffeomorphisms group and Riemannian tools Choice of the metric Spatially dependent Several diffeomorphic registration methods are available: metrics Metric learning • Free-form deformations B-spline-based diffeomorphisms by D. SVF metric learning Rueckert • Log-demons (X.Pennec et al.) • Large Deformations by Diffeomorphisms (M. Miller,A. Trouv´e,L. Younes) • ANTS Only the two last ones provide a Riemannian framework. Metric learning for diffeomorphic image A Riemannian approach to diffeomorphic registration. Fran¸cois-Xavier registration Vialard Introduction to diffeomorphisms group and Riemannian tools n Choice of the metric • vt 2 V a time dependent vector field on R . Spatially dependent • 't 2 Diff , the flow defined by metrics Metric learning SVF metric learning @t 't = vt ('t ) : (1) Action of the group of diffeomorphism G0 (flow at time 1): Π: G0 × C ! C ; : Π('; X ) = ':X 2 1 R 1 2 Right-invariant metric on G0: d('0;1; Id) = 2 0 jvt jV dt. −! Strong metric needed on V (Mumford and Michor: Vanishing Geodesic Distance on...) Right invariant distance on G0 Z 1 2 2 d(Id;') = inf jvt j dt ; 2 V v2L ([0;1];V ) 0 −! geodesics on G0. V is a Reproducing kernel Hilbert Space (RKHS): (pointwise evaluation continuous) =) Existence of a matrix function kV (kernel) defined on U × U such that: hv(x); ai = hkV (:; x)a; viV : Metric learning for diffeomorphic image Matching problems in a diffeomorphic framework registration. Fran¸cois-Xavier Vialard 1 n Introduction to U a domain in R diffeomorphisms group 1 and Riemannian tools 2 V a Hilbert space of C vector fields such that: Choice of the metric Spatially dependent kvk1;1 6 CjvjV : metrics Metric learning SVF metric learning Right invariant distance on G0 Z 1 2 2 d(Id;') = inf jvt j dt ; 2 V v2L ([0;1];V ) 0 −! geodesics on G0. Metric learning for diffeomorphic image Matching problems in a diffeomorphic framework registration. Fran¸cois-Xavier Vialard 1 n Introduction to U a domain in R diffeomorphisms group 1 and Riemannian tools 2 V a Hilbert space of C vector fields such that: Choice of the metric Spatially dependent kvk1;1 6 CjvjV : metrics Metric learning V is a Reproducing kernel Hilbert Space (RKHS): (pointwise SVF metric learning evaluation continuous) =) Existence of a matrix function kV (kernel) defined on U × U such that: hv(x); ai = hkV (:; x)a; viV : Metric learning for diffeomorphic image Matching problems in a diffeomorphic framework registration. Fran¸cois-Xavier Vialard 1 n Introduction to U a domain in R diffeomorphisms group 1 and Riemannian tools 2 V a Hilbert space of C vector fields such that: Choice of the metric Spatially dependent kvk1;1 6 CjvjV : metrics Metric learning V is a Reproducing kernel Hilbert Space (RKHS): (pointwise SVF metric learning evaluation continuous) =) Existence of a matrix function kV (kernel) defined on U × U such that: hv(x); ai = hkV (:; x)a; viV : Right invariant distance on G0 Z 1 2 2 d(Id;') = inf jvt j dt ; 2 V v2L ([0;1];V ) 0 −! geodesics on G0. Metric learning for diffeomorphic image Variational formulation registration. Fran¸cois-Xavier Vialard Introduction to Find the best deformation, minimize diffeomorphisms group and Riemannian tools J (') = inf d(':A; B)2 (2) Choice of the metric Spatially dependent '2GV | {z } metrics similarity measure Metric learning SVF metric learning Metric learning for diffeomorphic image Variational formulation registration. Fran¸cois-Xavier Find the best deformation, minimize Vialard Introduction to diffeomorphisms group 2 J (') = inf d(':A; B) (2) and Riemannian tools '2G V | {z } Choice of the metric similarity measure Spatially dependent metrics Tychonov regularization: Metric learning SVF metric learning 1 2 J (') = R(') + 2 d(':A; B) : (3) | {z } 2σ Regularization | {z } similarity measure Riemannian metric on GV : Z 1 1 2 R(') = jvt jV dt (4) 2 0 is a right-invariant metric on GV . Main issues for practical applications: • choice of the metric (prior), • choice of the similarity measure. Metric learning for diffeomorphic image Optimization problem registration. Fran¸cois-Xavier Minimizing Vialard Introduction to 1 diffeomorphisms group 1 Z 1 and Riemannian tools J (v) = jv j2 dt + d(' :A; B)2 : t V 2 0;1 Choice of the metric 2 0 2σ Spatially dependent In the case of landmarks: metrics Metric learning k SVF metric learning 1 Z 1 1 X J (') = jv j2 dt + k'(x ) − y k2 ; 2 t V 2σ2 i i 0 i=1 In the case of images: Z 2 2 d('0;1:I0; Itarget ) = jI0 ◦ '1;0 − Itarget j dx : U Metric learning for diffeomorphic image Optimization problem registration. Fran¸cois-Xavier Minimizing Vialard Introduction to 1 diffeomorphisms group 1 Z 1 and Riemannian tools J (v) = jv j2 dt + d(' :A; B)2 : t V 2 0;1 Choice of the metric 2 0 2σ Spatially dependent In the case of landmarks: metrics Metric learning k SVF metric learning 1 Z 1 1 X J (') = jv j2 dt + k'(x ) − y k2 ; 2 t V 2σ2 i i 0 i=1 In the case of images: Z 2 2 d('0;1:I0; Itarget ) = jI0 ◦ '1;0 − Itarget j dx : U Main issues for practical applications: • choice of the metric (prior), • choice of the similarity measure. Metric learning for diffeomorphic image Why does the Riemannian framework matter? registration. Fran¸cois-Xavier Vialard Generalizations of statistical tools in Euclidean space: Introduction to diffeomorphisms group • Distance often given by a Riemannian metric. and Riemannian tools • Straight lines ! geodesic defined by Choice of the metric Spatially dependent Z 1 metrics 2 Metric learning Variational definition: arg min kc_kc(t) dt = 0 ; c(t) 0 SVF metric learning Equivalent (local) definition: rc_ c_ =c ¨ + Γ(c)(c _; c_) = 0 : • Average ! Fr´echet/Karcher mean. Variational definition: arg minfx ! E[d 2(x; y)]dµ(y)g 2 Critical point definition: E[rx d (x; y)]dµ(y)] = 0 : • PCA ! Tangent PCA or PGA. • Geodesic regression, cubic regression...(variational or algebraic) Metric learning for diffeomorphic image Karcher mean on 3D images registration. Fran¸cois-Xavier Vialard Introduction to diffeomorphisms group Init. guesses and Riemannian tools Choice of the metric Spatially dependent metrics Metric learning 1 iteration SVF metric learning 2 iterations 3 iterations 1 2 3 4 Ai Ai Ai Ai m Figure{ Average image estimates Ai , m 2 f1; ··· ; 4g after i =0, 1, 2 and 3 iterations. Metric learning for diffeomorphic image Interpolation, Extrapolation registration. Fran¸cois-Xavier Vialard Introduction to diffeomorphisms group and Riemannian tools Choice of the metric Spatially dependent metrics Metric learning SVF metric learning Figure{ Geodesic regression (MICCAI 2011) Metric learning for diffeomorphic image Interpolation, Extrapolation registration. Fran¸cois-Xavier Vialard Introduction to diffeomorphisms group and Riemannian tools Choice of the metric Spatially dependent metrics Metric learning SVF metric learning Figure{ Extrapolation of happiness Metric learning for diffeomorphic image registration. Fran¸cois-Xavier Vialard Introduction to diffeomorphisms group and Riemannian tools Choice of the metric Spatially dependent metrics Metric learning What metric to choose? SVF metric learning The norm on V is simply Z Z 2 1=2 2 kvkV = hv(x); (Lv)(x)i dx = (L v) (x) dx : Ω Ω Scale parameter important! 2 − kx−yk n kσ(x; y) = e σ2 kernel/operator (Id −σ∆) (5) • σ small: good matching but non regular deformations and more local minima. • σ large: poor matching but regular deformations and more global minima. Metric learning for diffeomorphic image Choosing the right-invariant metric registration. Right-invariant metric: Eulerian fluid dynamic viewpoint on Fran¸cois-Xavier Vialard regularization.