HESSIAN OF THE RIEMANNIAN SQUARED DISTANCE FUNCTION ON CONNECTED LOCALLY SYMMETRIC SPACES WITH APPLICATIONS Ricardo Ferreira ∗Jo˜ao Xavier ∗ ∗ [email protected] [email protected] Instituto de Sistemas e Rob´otica (ISR), Instituto Superior T´ecnico Av. Rovisco Pais, 1049-001 Lisbon, Portugal Abstract: Many optimization problems formulated on Riemannian manifolds involve their intrinsic Riemannian squared distance function. A notorious and important example is the centroid computation of a given finite constellation of points. In such setups, the implementation of the fast intrinsic Newton optimization scheme requires the availability of the Hessian of this intrinsic function. Here, a method for obtaining the Hessian of this function is presented for connected locally-symmetric spaces on which certain operations, e.g. exponential, logarithm and curvature maps, are easily carried out. Particularly, naturally reductive homogeneous spaces provide the needed information, hence applications will be shown from this set. We illustrate the application of this theoretical result in two engineering scenarios: (i) centroid computation and (ii) maximum a posteriori (MAP) estimation. Our results confirm the quadratic convergence rate of the intrinsic Newton method derived herein. Keywords: Riemannian manifolds, Hessian of intrinsic distance, Intrinsic Newton method, Centroid computation, MAP estimation 1. INTRODUCTION AND MOTIVATION of the function to be optimized. In this paper, our interest lies on those optimization problems Due to its quadratic convergence rate near the so- involving the intrinsic squared Riemannian dis- lution, Newton’s method has for a long time been tance. As a special case, this subsumes the class the method of choice for optimization problems, of optimization problems known as centroid com- especially when high precision is required. This is putation. true in many fields, from engineering to numerical Many applications of centroid computation exist analysis where it is used extensively to obtain in the literature. For example (Moakher, 2002) many digits of precision. Intrinsic quasi-Newton mentions centroid computation in SO(3) for the and Newton algorithms for optimization problems study of plate tectonics and sequence-dependent on smooth manifolds have been discussed (among continuum modeling of DNA. In these, experimen- others) in (Gabay, 1982) and in (Edelman et tal observations are obtained with a significant al., 1998) and (Manton, 2002). Applications can amount of noise that needs to be smoothed. Pos- be found in robotics (Belta and Kumar, 2002), sig- itive definite symmetric matrices are used as co- nal processing (Manton, 2005), image processing variance matrices in (Pennec et al., 2004) for sta- (Helmke et al., 2004), etc. tistical characterization of deformations and en- Implementation of the intrinsic Newton scheme coding of principle diffusion directions in Diffusion relies on the availability of the intrinsic Hessian Tensor Imaging (DTI), expanding the range of category, here SE(n) is seen as SO(n) applications to medicine. Computation of centers Rn). Further, when dealing with constant× of mass also find applications for analyzing shapes sectional curvature manifolds, we show how in medical imaging, see (Fletcher et al., 2004). to decrease the computational complexity of It is also a mandatory step when considering the the algorithm. extension of the K-means algorithm to manifolds. Section 4 illustrates the application of the • theory to the problem of intrinsic cen- Most of these approaches rely on gradient meth- troid computation by a Newton method. ods for computing the intrinsic Riemannian cen- A pseudo-code implementation of the algo- troid with a few exceptions: H¨uper and Manton rithm is given. Results for the special Eu- in (H¨uper and Manton, 2005) developed a New- clidean group, the Grassmann manifold and ton method for the special orthogonal group and real projective plane are presented. Another (Absil et al., 2004) introduced a Newton method application, concerning MAP position esti- applicable to Grassmann manifolds which oper- mation in the context of robot navigation is ates on an approximation of the intrinsic cost shown as well. function, yielding the intrinsic centroid. Finally, some conclusions are drawn and di- 1.1 Contribution We present and discuss a • rections for future work are delineated in method for computing the intrinsic Hessian of the section 5. Riemannian squared distance function for locally symmetric Riemannian manifolds. This builds on 2. HESSIAN OF THE RIEMANNIAN our previous work in (Ferreira et al., 2006). In this SQUARED DISTANCE FUNCTION paper we simplify the description of the intrinsic Hessian by providing a concise matrix expression 2.1 Review of Newton’s Method in Rie- in each tangent space. This entails an appre- mannian Manifolds Let qk M henceforth des- ciable simplification with respect to (Ferreira et ignate the kth iterate in an optimization∈ method al., 2006), e.g. the need for parallel translation formulated on a Riemannian manifold M. New- of tangent vectors is dropped, thus relaxing the ton’s method on a manifold is essentially the same amount of required differential geometric func- as in Rn (see (Edelman et al., 1998), (Manton, tions initially imposed. 2002) and (H¨uper and Trumpf, 2004) for some generalizations). It generates a search direction Adding to the already described cases of the n d T M as the solution of the linear system embedded sphere S , special orthogonal group k ∈ qk SO (n) and symmetric positive definite matrices H dk = grad f(qk) , (1) Sym+(n), the method will also be shown to work · − for the special Euclidean group SE(n) and the where H is the bilinear Hessian tensor of the smooth cost function f : M R evalu- Grassmann manifold G(n,p). Notice that the real → n ated at q and grad f(q ) T M is its gra- projective plane P is a particular case of the k k ∈ qk Pn G dient. Some care is needed though, since the Grassmann manifold ( ∼= (n +1, 1)). As an example of other applications, a simple example Hessian and the gradient are not as simple Rn of MAP estimation is described and results are to find as in , but are in fact given as shown. the solutions of (df)qXq = grad f(q),Xq and Hess f(q)(X , Y ) = gradh f, Y , wherei q q q ∇Xq q ∈ 1.2 Paper Organization The paper is struc- M and Xq, Yq TqM are any tangent vectors. ∈ tured as follows: Here (df)q denotes the differential of the function Section 2 provides a short review of Newton’s f at the point q and denotes the Levi-Civita • connection of the manifold.∇ Here, , denotes the method on Riemannian manifolds and also h· ·i presents the main result of this article - the inner product on TqM. matrix version of the Hessian of the Rieman- Once a Newton direction has been obtained, it nian squared distance function in a given should be checked if it’s a descent direction (its tangent space. This enhances our previous inner product with the gradient vector should result in (Ferreira et al., 2006) and describes be negative). If so, the update equation qk+1 = it in an implementation-friendly format. expqk (αkdk), can be used to obtain a better esti- To construct the Hessian, several differential- mate. Here α is a step size, given for example by • k geometric operations on the manifold are Armijo’s rule, and exp denotes the Riemannian needed, e.g. computation of the curvature exponential map. If the inner product is negative, tensor. Section 3 mentions a method for ob- a safe negative gradient direction should be used. taining the needed computational tools for a class of commonly used manifolds: natu- Although the gradient is usually easy to compute, rally reductive homogeneous spaces (all the determination of the Hessian is more involved. manifolds mentioned earlier fall under this The next section describes a method to calculate it on particular Riemannian manifolds (connected and locally-symmetric where the curvature tensor ˆ i.e. [X]i = Xq, Fiq , let Rk be the matrix with and Riemannian logarithm maps are known). entries [R ] = F , (F ) and consider the k ij iq R j q 2.2 Hessian in Matrix Form This section D E T eigenvalue decomposition Rk = EΛE . Here λi starts with a slightly upgraded version of the main will be used to describe the i’th diagonal element result presented in (Ferreira et al., 2006), stating: of Λ. Then the Hessian matrix (a representation for the bilinear Hessian tensor on the finite di- Theorem 1. Consider M to be a connected locally- mensional tangent space with respect to the fixed symmetric n-dimensional Riemannian manifold basis) is given by: with curvature endomorphism R. Let Bǫ(p) be a T Hk = EΣE (2) geodesic ball centered at p M and r : B (p) ∈ p ǫ → R the function returning the intrinsic (geodesic) where Σ is diagonal with elements σi given by σi = ˆ T ˆ distance to p. Let γ : [0, r] Bǫ(p) denote ctgλi (r). Hence Hess(kp)q(Xq, Yq)= X HkY . the unit speed geodesic connecting→ p to a point q B (p), where r = r (q), and letγ ˙ γ˙ (r) ∈ ǫ p q ≡ This result follows from theorem 1 by decompos- be its velocity vector at q. Define the function ing each vector with respect to the considered R 1 2 kp : Bǫ(p) , kp(x) = 2 rp(x) and consider basis and noting that, due to the symmetries of any X , Y →T M. Then q q ∈ q the curvature endomorphism, the operator has a k null eigenvalue associated with the eigenvectorγ ˙ q. Hess(kp)q(Xq, Yq)= Xq , Yq + D E n 3. IMPLEMENTATION ctg (r) X ⊥, E Y , E . λi q iq q iq 3.1 Naturally Reductive Homogeneous D E Xi=1 Spaces The theory of naturally reductive ho- where Ei TqM is an orthonormal basis mogeneous spaces, henceforth denoted by NRHS, q ⊂ which diagonalizes the linear operator : TqM provides a practical way of obtaining the needed R → TqM, (Xq ) = R(Xq, γ˙q)γ ˙ q with eigenvalues λi, curvature endomorphism R, Riemannian expo- R this means (Ei )= λiEi .