Deformation analysis for shape and image processing Laurent Younes Contents General notation 5 Chapter 1. Elements of Hilbert space theory 7 1. Definition and notation 7 2. Examples 8 3. Orthogonal spaces and projection 9 4. Orthonormal sequences 10 5. The Riesz representation theorem 11 6. Embeddings and the duality paradox 12 7. Weak convergence in a Hilbert space 15 8. Spline interpolation 16 9. Building V and its kernel 22 Chapter 2. Ordinary differential equations and groups of diffeomorphisms 35 1. Introduction 35 2. A class of ordinary differential equations 37 3. Groups of diffeomorphisms 44 Chapter 3. Matching deformable objects 49 1. General principles 49 2. Matching terms and greedy descent procedures 49 3. Regularized matching 70 Chapter 4. Groups, manifolds and invariant metrics 95 1. Introduction 95 2. Differential manifolds 95 3. Submanifolds 98 4. Lie Groups 99 5. Group action 102 6. Riemannian manifolds 103 7. Gradient descent on Riemannian manifolds 110 Bibliography 113 3 General notation The differential of a function ϕ :Ω → Rd, Ω being an open subset of Rd, is denoted Dϕ : x 7→ Dϕ(x). It takes values in the space of linear operators from Rd to Rd so that Dϕ(x) can be identified to d × k matrix (containing the partial derivatives of the coordinates of ϕ). d Idk is the identity matrix in R . ∞ CK (Ω) is the set of infinitely differentiable functions on Ω, an open subset of Rd. 5 CHAPTER 1 Elements of Hilbert space theory 1. Definition and notation A set H is a (real) Hilbert space if i) H is a vector space over R (i.e., addition and scalar multiplication are defined on H). 0 0 0 ii) H has an inner product denoted (h, h ) 7→ hh , h iH , for h, h ∈ H. This inner product is a bilinear form, which is positive definite and the associ- p ated norm is kh]kH = hh , hiH . iii) H is a complete space for the topology associated to the norm. Here are some explanations of iii). Converging sequences for the norm topology are sequences hn for which there exists h ∈ H such that kh − hnkH → 0. Property iii) means that if a sequence (hn, n ≥ 0) in h is a Cauchy sequence, i.e., it collapses in the sense that, for every positive ε there exists n0 > 0 such that khn − hn0 kH ≤ ε for n ≥ n0, then it necessarily has a limit: there exists h ∈ H such that khn − hk → 0 when n tends to infinity. If condition ii) is weakened to the fact that k.kH is a norm (not necessarily induced by an inner product), one says that H is a Banach space. If H satisfies i) and ii), it is called a pre-Hilbert space. On pre-Hilbert spaces, the Schwartz inequality holds: Proposition 1 (Schwartz inequality). If H is pre-Hilbert, and h, h0 ∈ H, then 0 0 hh , h iH ≤ khkH kh kH The first consequence of this property is Proposition 2. The inner product on H is continuous for the norm topology. Proof. The inner product is a function H × H → R. Letting ϕ(h, h0) = 0 hh , h iH , we have, by Schwartz inequality, and introducing a sequence (hn) which converges to h 0 0 0 |ϕ(h, h ) − ϕ(hn, h )| ≤ kh − hnkH kh kH → 0 which proves the continuity with respect to the first coordinate, and also with respect to the second coordinate by symmetry. ut Working with a complete normed vector space is essential when dealing with infinite sums of elements: if (hn) is a sequence in H, and if khn+1 + ··· + hn+kk can be made arbitrarily small for large n and any k > 0, then completeness implies P∞ that the series n=0 hn has a limit in h. In particular, absolutely converging series in H converge: X X (1) khnkH < ∞ ⇒ hn converges. n≥0 n≥0 7 8 1. ELEMENTS OF HILBERT SPACE THEORY We shall add a fourth condition to our definition of a Hilbert space iv) H is separable for the norm topology: there exists a countable subset S in H such that, for any h ∈ H and any ε > 0, there exists h0 ∈ S such that kh − h0k ≤ ε. In the following, Hilbert spaces will always be a separable Hilbert spaces (this is not always the case in the literature). A Hilbert space isometry between two Hilbert spaces H and H0 is a an invertible 0 0 0 0 linear map F : H → H such that, for all h, h ∈ H, hF h , F h iH0 = hh , h iH . A sub-Hilbert space of H is a subspace H0 of H (i.e., a non-empty subset closed 0 by linear combination) which is closed for the norm topology: if (hn) is a sequence 0 0 0 in H and khn − hkH → 0 for some h ∈ H, then h ∈ H . Topological closedness implies that H0 is itself a Hilbert space, since Cauchy sequences in H0 also are Cauchy sequences in H, hence converge in H, hence in H0 since H0 is closed. The next proposition shows that every finite dimensional subspace of a Hilbert space is closed, and hence a Hilbert subspace. Proposition 3. If K is a finite dimensional subspace of H, then K is closed in H Proof. let e1, . , ep be a basis of K. Let (hn) be a sequence in K which Pp converges to some h ∈ H: then hn may be written hn = k=1 aknek and for all l = 1, . , p: p X hhn , eliH = hek , eliH akn k=1 p p Let an be the vector in R with coordinates (akn, k = 1, . , p) and un ∈ R with coordinates (hhn , ekiH , k = 1, . , p). Let also S be the matrix with coefficients skl = hek , eliH , so that the previous system may be written: un = San. The matrix S is invertible: if b belongs to the null space of S, a quick computation shows that p 2 X T biei = b Sb = 0 i=1 H which is only possible when b = 0, since (e1, . , en) is a basis. We therefore have −1 an = S un. Since un converges to u with coordinates (hh , ekiH , k = 1, . , p) (by −1 continuity of the inner product), we obtain the fact that an converges to a = S u. Pp Pp But this implies that k=1 aknek → k=1 akek and since the limit is unique, we Pp have h = k=1 akek ∈ K. ut 2. Examples n 0 2.1. Finite dimensional Euclidean spaces. H = R with hh , h i n = Pn 0 R i=1 hihi is the standard example of a finite dimensional Hilbert space. 2.2. `2 space of real sequences. Let H be the set of real sequences h = P∞ 2 (h1, h2,...) such that i=1 hi < ∞. Then H is a Hilbert space. 3. ORTHOGONAL SPACES AND PROJECTION 9 2.3. L2 space of functions. Let k and d be two integers. Let Ω be an open subset of Rd. We define L2(Ω, Rd) as the set of all square integrable functions h :Ω → Rd, with inner-product Z 0 0 hh , h iL2 = h(x)h (x)dx Ω Integrals are taken with respect to the Lebesgue measure on Ω, and two functions which coincide everywhere except on a set of null Lebesgue measure are consid- ered as equal. The fact that L2(Ω, Rd) is a Hilbert space is a standard result in integration theory. 3. Orthogonal spaces and projection Let O be a subset of H. The orthogonal space to O is defined by ⊥ O = {h : h ∈ H, ∀o ∈ O, hh , oiH = 0} Theorem 1. O⊥ is a sub-Hilbert space of H. Proof. Stability by linear combination is obvious, and closedness is a conse- quence of the continuity of the inner product. ut When K is a sub-Hilbert space of H (a closed subspace) and h ∈ H, one can define the variational problem: 0 0 (PK (h)) : find k ∈ K such that kk − hkH = min {kh − k kH : k ∈ K} The following theorem is fundamental Theorem 2. If K is a closed subspace of H and h ∈ H, (PK (h)) has a unique solution, characterized by the property k ∈ K and h − k ∈ K⊥ Definition 1. The solution of problem (PK (h)) in the previous theorem is called the orthogonal projection of h on K and will be denoted πK (h). Proposition 4. πK : H → K is a linear, continuous function and πK⊥ = id − πK . 0 0 Proof. Let d = inf {kh − k kH : k ∈ K}. The proof relies on the following construction: if k, k0 ∈ K, a direct computation shows that 0 2 k + k 0 2 2 0 2 h − + kk − k kH /4 = kh − kkH + kh − k kH /2 2 H 0 2 0 k+k and the fact that (k + k )/2 ∈ K implies h − 2 ≥ d so that H 0 2 2 0 2 2 kk − k kH ≤ kh − kkH + kh − k kH /2 − d . Now, from the definition of the infimum, one can find a sequence kn in K such that 2 2 −n kkn − hkH ≤ d + 2 for each n. The previous inequality implies that 2 −n −m kkn − kmkH ≤ 2 + 2 /2 which implies that kn is a Cauchy sequence, and therefore converges to a limit k 0 which belongs to K and kk − hkH = d. If the minimum is attained for another k in K, we have, by the same inequality 0 2 2 2 2 kk − k kH ≤ d + d /2 − d = 0 .