Diffeomorphic Shape Analysis
Laurent Younes Johns Hopkins University January 2014
Partially supported by ONR, NIH, NSF
Geilo, January 2014 Laurent Younes 1 Introduction
• Shape analysis is the study of variation of shapes in relation with other variables. • Once a suitable “shape space” is defined, analyzing shapes involves: – Comparing them (using distances) – Finding properties of datasets in this space – Testing statistical hypotheses
Geilo, January 2014 Laurent Younes 2 Some entry points to the literature
• Landmarks / point sets: Kendall, Bookstein, Small, Dryden… • Active shapes: Taylor, Cootes,… • Registration: Demons, Diffeomorphic Demons (Thirion, Guimond, Ayache, Pennec, Vercauteren,…) • Harmonic analysis: SPHARM (Gerig, Steiner,…) • Medial axis and related, M-REPS (Pizer, Damon,…) • Manifolds of curves / surfaces – Theory (Mumford, Michor, Yezzi, Mennucci, Srivastava, Mio, Klassen…) • Statistics on manifolds of curves/surfaces (Pennec, Fletcher, Joshi, Marron…)
Geilo, January 2014 Laurent Younes 3 On “Diffeomorphometry”…
• Precursors (fluid registration): Christensen-Miller-Rabbit; Thirion… • Large Deformation Diffeomorphic Metric Mapping – LDDMM – Images: Beg, Miller, Trouvé, Y.; – Landmarks Miller, Joshi; – Measures (Glaunès, Trouvé, Y.), Currents (Glaunès, Vaillant) – Hamiltonian methods (Glaunès, Trouvé, Vialard, Y…) • Metamorphosis (Miller, Trouvé, Holm, Y…)
Geilo, January 2014 Laurent Younes 4 Shape Spaces
• Shape spaces are typically modeled as differential manifolds, but… • Almost all practical methods use a shape representation in a linear space of “local coordinates” • Linear statistical methods can then be employed to analyze the data. • For some of these methods (e.g., PCA), it is also important that linear combinations of the representations make sense, too.
Geilo, January 2014 Laurent Younes 5 Example: Kendall Shape Space
• Consider the family of collections of N distinct points in d ! d N • This forms an open subset of Q = (! ) • Identify collections that are can be deduced from each other by rotation, translation and scaling. • It is a set of subsets of Q (quotient space). • It can be structured as a differential manifold. • The Euclidean metric on Q transforms into a Riemannian metric in the quotient space.
Geilo, January 2014 Laurent Younes 6 Example: Spaces of Curves
2 • The set Q of smooth functions q : [ 0 , 1 ] → ! is a Fréchet space. • Quotient out translation rotations scaling and change of parameter to obtain a shape space. • Sobolev norms on Q trickle down to Riemannian metrics on the shape space. • See papers from Michor, Mumford, Shah…
Geilo, January 2014 Laurent Younes 7 Shapes Spaces and Deformable Templates • Ulf Grenander’s metric pattern theory involves transformation groups that act on shapes. • The transformations have a cost, represented by an effort functional. • Under additional assumptions, this induces a metric in the shape space. • The construction that follows will be based on these principles. • The group is the group of diffeomorphisms. • The shapes are anything that can be deformed.
Geilo, January 2014 Laurent Younes 8 Tangent space representations
• There is a natural way to build linear representations when the the shape space is modeled as a Riemannian manifold, i.e.: – a topological space that can be mapped locally to a vector space on which differentials can be computed (with consistency relations between local maps). – with a metric which can be used to compute lengths, and shortest paths between two points, which are called geodesics.
Geilo, January 2014 Laurent Younes 9 Exponential Charts (I)
• Geodesics can be characterized by a second order differential equation on the manifold. • This equation has (in general) a unique solution given its initial position and first derivative. • Fix the initial position (“template”). Denote it S . • The space of all derivatives of curves in S that start from T S S is the tangent space to S at S (notation S ).
Geilo, January 2014 Laurent Younes 10 Exponential Charts (II)
• The exponential map associates to each vector v T S ∈ S the solution at time t =1 of the geodesic equation starting at S in the direction v. • It is a map from T S to S , with notation S
v ∈ T S exp (v) ∈ S S S • It can (generally) be restricted to a neighborhood of 0 in T S S over which it is one to one, providing so-called normal local coordinates or an exponential chart at S .
Geilo, January 2014 Laurent Younes 11 SHAPES AND DIFFEOMORPHISMS
Geilo, January 2014 Laurent Younes 12 General Approach
• We will define shape spaces via the action of diffeomorphisms on them. • This action will induce a differential structure on the shape space. • It also allows to compare them: two shapes are similar if one can be obtained from the other via a small deformation.
Geilo, January 2014 Laurent Younes 13 Notation
• Diffeomorphisms of d will be denoted φ, ψ, etc. They are invertible differentiable transformations with a differentiable inverse.
• Vector fields over d will be denoted v, w, etc. They are differentiable mappings from d onto itself.
Geilo, January 2014 Laurent Younes 14 Ordinary Differential Equations
• Diffeomorphisms can be built as flows associated to ordinary differential equations (ODEs). • Let ( t , x ) ! v ( t , x ) be a time-dependent vector field. Assume that v is differentiable in x with bounded first derivative over a time interval [ 0 , T ] . • Then there exists a unique solution of the ODE y ! = v ( t , y ) y(0) x with initial condition = 0 defined over the whole interval [ 0,T]
Geilo, January 2014 Laurent Younes 15 Flows
• The flow associated with the ODE y! = v ( t , y ) is the mapping (t,x) ! ϕ(t,x) such that ϕ ( t , x ) is the solution at time t of the ODE initialized with y ( 0 ) = x . • In other terms:
⎧∂tϕ = v(t,ϕ) ⎨ ⎩ϕ(0,x) = x
Geilo, January 2014 Laurent Younes 16 Flows (II)
• Assuming that v ( t , ⋅ ) is continuously differentiable with bounded derivative, uniformly in time, the associated flow is such that x ! ϕ(t,x) is a diffeomorphism at all times. • If v has more (space) derivatives, they are inherited by ϕ . • We will build diffeomorphisms as flows associated to vector fields that belong in a specified reproducing kernel Hilbert space.
Geilo, January 2014 Laurent Younes 17 Hilbert Spaces
• A Hilbert space H is an infinite-dimensional vector space with an inner-product, such that the associated norm that makes it a complete metric space. • The inner product between two elements in H is denoted h,h' H • The set of bounded linear functionals µ : H → ! is the topological dual of H, denoted H * . • We will use the notation µ h : = µ ( h ) for µ ∈H *,h ∈H ( )
Geilo, January 2014 Laurent Younes 18 Riesz Theorem
• The Riesz representation theorem states that H and H * are isometric. * • µ ∈ H if and only if there exists h ∈ H such that, for all h ' ∈ H : µ h′ = h,h' ( ) H • The correspondence h will be denoted K , so that µ ! H K : H * H and its inverse A : H ! H * H → H • We therefore have
µ h = K µ,h and h,h' = A h h′ ( ) H H H ( H )
Geilo, January 2014 Laurent Younes 19 RKHS
d • A Hilbert space H of functions defined over ! , with k values in ! is an RKHS, if H ⊂ L2 (!d ,!k ) ∩ C p (!d ,!k ) 0 ( p ≥ 0 ) with continuous inclusions in both spaces. (Here C p ( d , k ) is the space of p times differentiable 0 ! ! vector fields that converge to 0 (will all derivatives up to order p) at infinity, equipped with the supremum norm. • This means that for some constant C, and for all v ∈H max v , v ≤ C v ( 2 p,∞ ) H
Geilo, January 2014 Laurent Younes 20 RKHS
d k • If H is an RKHS, and x ∈ ! , a ∈ ! the linear form aδ : v ! aT v(x) x belongs to H * . • General notation: if θ is a (scalar) measure and a is a vector field defined on its support, a θ is the vector measure
(a | w) a(x)T w(x)d (x) θ = ∫ θ
Geilo, January 2014 Laurent Younes 21 RKHS
• The reproducing kernel of V is a matrix-valued function K V d d defined on ! × ! by
K H (x, y)a = K H (aδ y )(x) d with a,x ∈! K (x, y) • H is a k by k matrix. • We will denote K i the ith column of K and by K i j its i,j H H H entry. One has, in particular
K i (⋅,x), K j (⋅, y) = K ij (x, y) H H H H
Geilo, January 2014 Laurent Younes 22 Usual examples (scalar kernels)
I • k is the d by d identity matrix. • Gaussian kernel: K (x, y) ∝ exp(− x − y 2 /2σ 2 )I H ‖ ‖ k I • Cauchy Kernel: K (x, y) ∝ k H 2 2 1+ x − y σ • Power of Laplacian:
K (x, y) ∝ P ( x − y /σ )exp(− x − y /σ )I H m ‖ ‖ ‖ ‖ k with P = 1 , P = 1 + t and P P tP 0 1 ∂t m − m = − m−1
Geilo, January 2014 Laurent Younes 23 The Interpolation Problem
• Let H be a Hilbert space and µ , , µ ∈ H * . 1 … m • Solve the problem
⎡ h → min ⎢ H ⎢subject to µ h = λ ,k = 1,…,m ⎣ ( k ) k
m • Solution: h = a K µ where a , … , a are identified ∑ k=1 k H k 1 m using the constraints.
Geilo, January 2014 Laurent Younes 24 Application
• H is a scalar RKHS (k = 1). x , ,x ∈ d • 1 … m ! • Solve ⎡ h → min ⎢ H subject to h(x ) = λ ,k = 1,…,m ⎣⎢ k k
m • Solution: h(⋅) = K (⋅,x )a ∑ k=1 H k k
m with: K (x ,x )a = λ , k = 1,…,m ∑ l=1 H k l l k
Geilo, January 2014 Laurent Younes 25 Attainable Diffeomorphisms
• Fix V, an RKHS of vector fields with at least one derivative ( p ≥ 1 ). • We consider ODEs of the form y! = v ( t , y ) with v(t,⋅) ∈V • The flow evolution ( t , ) v ( t , ( t , ) ) can be interpreted ∂tϕ ⋅ = ϕ ⋅ as a control system in which the deformation, φ, is the state driven by the time-dependent vector field v (t,⋅) • The space of attainable diffeomorphisms is
G = ψ : ∃T,∃v(t,⋅), v(t,⋅) bounded,ψ = ϕ v (T,⋅) V { V } v where ϕ is the flow associated to v.
Geilo, January 2014 Laurent Younes 26 Attainable Diffeomorphisms
G d • V is a subgroup of the group of diffeomorphisms of ! • There is no loss of generality in taking T = 1 in the definition. • One defines a distance on G by V 1 v dV (ψ ,ψ ′) = min ‖v(t,)‖V dt :ϕ (1,ψ ) =ψ ′ {∫ 0 } and
1 2 2 v dV (ψ ,ψ ′) = min ‖v(t,)‖V dt :ϕ (1,ψ ) =ψ ′ {∫ 0 }
Geilo, January 2014 Laurent Younes 27 Attainable Diffeomorphisms
• Computing d ( ψ , ψ ′ ) = d ( i d ,ψ ( ψ ′ ) − 1 ) is equivalent to V V ! solving the optimal control problem
1 2 ‖v(t)‖V dt → min ∫0
subject to ϕ(0) = id, ∂ ϕ = v(t,ϕ), ϕ(1) =ψ (ψ ′)−1 t !
Geilo, January 2014 Laurent Younes 28 Riemannian Interpretation
d • V is the distance associated to the Riemannian metric −1 δϕ = δϕ !ϕ ϕ V −1 since, using ϕ ! " ϕ = v ,
⎧ 1 ⎫ d (ψ ,ψ ′) = min ‖ϕ!(t)‖ dt :ϕ(1) =ψ ′,ϕ(0) =ψ V ⎨∫ 0 ϕ ⎬ ⎩ ⎭ 1/2 ⎧ 1 ⎫ = min ‖ϕ!(t)‖2 dt :ϕ(1) =ψ ′,ϕ(0) =ψ ⎨∫ 0 ϕ ⎬ ⎩ ⎭
Geilo, January 2014 Laurent Younes 29 Differentiable Manifold
• Let M be a Hausdorff topological space. An n- dimensional local chart on M is a pair ( U , Φ ) where U is n open in M and Φ : U → V ⊂ ! is a homeomorphism. (U , ),(U , ) p • Two charts 1 Φ 1 2 Φ 2 are C -compatible if either U ∩ U = ∅ or Φ ! Φ − 1 is C p on Φ (U ∩U ) ⊂ n 1 2 2 1 1 1 2 ! • M is an n-dimensional C p manifold if it can be covered with local charts that are all pairwise C p compatible (which form an atlas).
Geilo, January 2014 Laurent Younes 30 Something like this
Geilo, January 2014 Laurent Younes 31 Tangent Vectors on Manifolds Tangent Vectors on a differentiable manifold are velocities of trajectories in the manifold. They represent infinitesimal displacements.
The collection of all tangent vectors at a given point is an n- dimensional vector space, called the tangent space at this point.
Geilo, January 2014 Laurent Younes 32 Riemannian Manifold
• A Riemannian manifold is a differentiable manifold with an inner product on each of its tangent spaces.
p ∈ M, v tangent to M at p → v p
Geilo, January 2014 Laurent Younes 33 Riemannian distance
• Adding the norms of infinitesimal displacements, one can compute the lengths of trajectories on a manifold. • The Riemannian distance is the length of the shortest path between two points: 1 d( p, p') = inf x!(t) dt : x(0) = p,x(1) = p' {∫ 0 x(t) } • Curves that achieve the shortest length are called (minimizing) geodesics. They extend the notion of straight lines to manifolds.
Geilo, January 2014 Laurent Younes 34 Back to Diffeomorphisms
• Manifold: group G of diffeomorphisms. It is an open space of a Banach or Fréchet space and has infinite dimension. • Infinitesimal displacements, δ ϕ , are vector fields. • The Riemannian metric is
δϕ = δϕ !ϕ −1 ϕ V
Geilo, January 2014 Laurent Younes 35 Exponential Charts
• On a Riemannian manifold, one can define radial coordinates or exponential charts. • Example: (Latitude, Longitude) on Earth. – Meridian lines stemming from the North Pole are geodesics. – A point on Earth is measured by specifying which meridian it belongs to (Longitude) and where it is on this meridian (latitude).
Geilo, January 2014 Laurent Younes 36 Exponential Charts
• Geodesics t ! γ ( t ) must satisfy a second-order differential equation. • They are uniquely defined by their initial conditions (γ (0),γ!(0)) • Definition: exp (v) = γ (1) p where γ is the geodesic with initial conditions γ (0) = p,γ!(0) = v is the exponential map on the manifold.
Geilo, January 2014 Laurent Younes 37 Exponential Charts
• Fixing p, the function v ! e xp ( v ) is a local chart p mapping a neighborhood of 0 in the tangent space to p to a neighborhood of p on the manifold. • It is called the exponential chart. • (On the sphere, longitude provides the direction of v and latitude provides the norm.) • The inverse map: p ' v such that p ' = e xp ( v ) is defined → p in a neighborhood of p and provides exponential coordinates (Riemannian logarithm).
Geilo, January 2014 Laurent Younes 38 Application to Diffeomorphisms
• Take M = G, a group of diffeomorphisms, p = i d , the identity map. • The Riemannian logarithm of ψ is obtained by solving the optimal control problem 1 2 ‖v(t)‖V dt → min ∫0 subject to ϕ(0) = id, ∂ ϕ = v ϕ, ϕ(1) =ψ . t !
• The logarithm is then given by v ( 0 ) .
Geilo, January 2014 Laurent Younes 39 EPDiff
• The geodesic equation on diffeomorphisms is called EPDiff. • It expresses the conservation of “momentum” along optimal trajectories.
⎪⎧∂tϕ = v !ϕ ⎨ ∂ A v + ad*A v = 0 ⎩⎪ t V v V where ad : w → Dv w − Dww v (id,v ) • Solving this equation with initial conditions 0 provides a local chart of G around the identity.
Geilo, January 2014 Laurent Younes 40 Shape Spaces
• Assume that the shape space M is an open subset of a Banach space Q . • Assume that diffeomorphisms act on shapes, with notation: (ϕ,q) ϕ ⋅q • Thus: id ⋅q = q ϕ ⋅(ψ ⋅q) = (ϕ !ψ )⋅q
Geilo, January 2014 Laurent Younes 41 Infinitesimal Action
• The infinitesimal action of vector fields: (v,q) v ⋅q is defined by
v ⋅q := ∂ (ϕ ⋅q) , ϕ = id, ( ) v ε ε =0 0 ∂εϕ ε=0 =
• Let ξ : v v ⋅ q . q • We assume that, for all q ∈ M , ξ : V → Q is well defined q and bounded.
Geilo, January 2014 Laurent Younes 42 Riemannian Metric on Shapes
• Define a metric on M via “Riemannian submersion”, yielding
δq = inf v :ξ v = δq q { V q } • The associated distance is then given by d (q ,q ) = inf d (id,ϕ) :ϕ ⋅q = q M 0 1 { G 0 1}
1 • Or: 2 ‖v(t)‖V dt → min ∫0 q(0) = q , ∂ q = ξ v, q(1) = q . subject to 0 t q 1
Geilo, January 2014 Laurent Younes 43 Large Deformation Diffeomorphic Metric Mapping • The generic “LDDMM” problem is
1 2 1 v(t,.) +U(q(1)) → min 2 ∫0‖ ‖V
subject to q(0) = q(0) and q!(t) = ξ v(t) q(t) • This is an infinite-dimensional optimal control problem, with v as control, q as state and U an end-point cost assumed to be differentiable. • Typically, U measures the discrepancy between q(1) and the “target” shape q . 1
Geilo, January 2014 Laurent Younes 44 Geodesics equations
• Optimal paths must satisfy * ⎧∂t q = ξqKV ξq p ⎪ ⎪ * ⎨∂t p + (∂q ξqv) p = 0 ⎪ * v = K ξ p ⎩⎪ V q with q(0) = q , p(0) p . 0 = 0 for some p M * . 0 ∈ • The mapping p q ( 1 ) provides a coordinate system 0 → equivalent to the exponential chart.
Geilo, January 2014 Laurent Younes 45 Example: Parametrized sets
• q: continuous embedding of S (Riemannian manifold) in d with ϕ ⋅q = ϕ q ξ v = v q q • If p is a measure on S, ξ * a = q * a and q
* (ξ K ξ p)(x) = K (q(x),q( y))dp( y) q V q ∫S V
Geilo, January 2014 Laurent Younes 46 Special case: Point Sets / Landmarks
q (q , ,q ) d • = 1 … N is a finite set of distinct points in . q ( (q ), , (q )) ϕ ⋅ = ϕ 1 … ϕ N ξ v = (v(q ),…,v(q )) q 1 N N * • Q = Q * = ( d ) N ξ p = p δ q ∑ k qk k=1
• Then N ( K * p) K (q ,q ) p ξq V ξq k = ∑ V k l l l=1
Geilo, January 2014 Laurent Younes 47 Point Set Matching LDDMM Problem
• Plug into the generic problem:
1 2 1 v(t,.) +U(q(1)) → min 2 ∫0‖ ‖V
subject to q(0) = q(0) and q!(t) = ξ v(t) q(t) the constraint that v ( t , ⋅) = K ξ * p ( t ) for some V q(t) N p(t) = p (t) { k }k=1
N • One then has 2 v(t) = pT K (q ,q ) p V ∑ k V k l l k,l=1
Geilo, January 2014 Laurent Younes 48 Reduction
• The problem becomes finite dimensional: N 1 1 p (t)T K (q (t),q (t)) p (t)dt +U(q(1)) → min 2 ∫0 ∑ k V k l l k,l=1 N subject to q(0) q(0) and q (t) K (q (t),q (t)) p (t) = !k = ∑ V k l l l=1 • This is a classical optimal control problem with state q and control p. • One can use the adjoint method to solve it numerically.
Geilo, January 2014 Laurent Younes 49 Adjoint Method
• Consider the general optimal control problem 1 g(q(t),u(t))dt +U(q(1)) → min ∫0 (0) subject to q(0) = q and q!(t) = f (q(t),u(t)) u • Consider q as a function of u, say q uniquely defined by
u (0) u u q (0) = q and q! (t) = f (q (t),u(t))
• Let 1 F(u) = g(qu (t),u(t))dt +U(qu (1)) → min ∫0
Geilo, January 2014 Laurent Younes 50 Adjoint Method
• The adjoint methods computes ∇F(u) • Introduce the Hamiltonian H ( p,q) pT f (q,u) g(q,u) u = − • Step 1: Given u, solve for the state equation q! = f (q,u) or q! = ∂ H p u
• Step 2: Set p(1) = −∇U(q(1)) • Step 3: Solve p! = − ∂ H (backward in time) q u • Step 4: Let F(u)(t) H( p(t),q(t)) ∇ = − ∂u
Geilo, January 2014 Laurent Younes 51 Data Terms
• Several types of data terms q ! U ( q ) have been developed depending on the interpretation made for q. • If q ( q , , q ) are labeled landmarks, use = 1 … N N 2 U(q) q q(1) = ∑ k − k k=1 • If q { q , , q } are unlabeled, introduce the measure = 1 … N
N µ = δ q ∑ qk k=1
Geilo, January 2014 Laurent Younes 52 Data Terms
2 • Let U ( q ) = ‖ µ − µ (1 ) ‖ with q q 2 µ = K (x, y)dµ(x)dµ( y) ∫∫ H • This is also applicable to weighted sums of point masses. • Example: discretize the line measure dl along a curve by N µq = (dlk )δ q ∑ k k=1 and use this representation for curve comparison. • Same idea for triangulated surfaces.
Geilo, January 2014 Laurent Younes 53 Data Terms
• For oriented curves and surfaces: use vector measures instead of scalar ones (involving normal vector). • Equivalent to current-based surface comparison (Vaillant- Glaunès). • Other variants have been developed (collections of curves, surfaces to sections,…)
Geilo, January 2014 Laurent Younes 54 Application: tracking tagged MRI
Geilo, January 2014 Laurent Younes 55 Matching curves from tagged MRI data
Geilo, January 2014 Laurent Younes 56 Matching Triangulated Surfaces
Geilo, January 2014 Laurent Younes 57