Optimization Algorithms on Matrix Manifolds
Total Page:16
File Type:pdf, Size:1020Kb
00˙AMS September 23, 2007 © Copyright, Princeton University Press. No part of this book may be distributed, posted, or reproduced in any form by digital or mechanical means without prior written permission of the publisher. Index 0x, 55 of a topology, 192 C1, 196 bijection, 193 C∞, 19 blind source separation, 13 ∇2, 109 bracket (Lie), 97 F, 33, 37 BSS, 13 GL, 23 Grass(p, n), 32 Cauchy decrease, 142 JF , 71 Cauchy point, 142 On, 27 Cayley transform, 59 PU,V , 122 chain rule, 195 Px, 47 characteristic polynomial, 6 ⊥ chart Px , 47 Rn×p, 189 around a point, 20 n×p R∗ /GLp, 31 of a manifold, 20 n×p of a set, 18 R∗ , 23 Christoffel symbols, 94 Ssym, 26 − Sn 1, 27 closed set, 192 cocktail party problem, 13 S , 42 skew column space, 6 St(p, n), 26 commutator, 189 X, 37 compact, 27, 193 X(M), 94 complete, 56, 102 ∂ , 35 i conjugate directions, 180 p-plane, 31 connected, 21 S , 58 sym+ connection S (n), 58 upp+ affine, 94 ≃, 30 canonical, 94 skew, 48, 81 Levi-Civita, 97 span, 30 Riemannian, 97 sym, 48, 81 symmetric, 97 tr, 7 continuous, 194 vec, 23 continuously differentiable, 196 convergence, 63 acceleration, 102 cubic, 70 accumulation point, 64, 192 linear, 69 adjoint, 191 order of, 70 algebraic multiplicity, 6 quadratic, 70 arithmetic operation, 59 superlinear, 70 Armijo point, 62 convergent sequence, 192 asymptotically stable point, 67 convex set, 198 atlas, 19 coordinate domain, 20 compatible, 20 coordinate neighborhood, 20 complete, 19 coordinate representation, 24 maximal, 19 coordinate slice, 25 atlas topology, 20 coordinates, 18 cotangent bundle, 108 basis, 6 cotangent space, 108 For general queries, contact [email protected] 00˙AMS September 23, 2007 © Copyright, Princeton University Press. No part of this book may be distributed, posted, or reproduced in any form by digital or mechanical means without prior written permission of the publisher. 222 INDEX covariant derivative, 94 image, 193 covector, 108 inverse, 193 covector field, 108 on, 193 covering, 193 onto, 193 critical point, 54 projection, 29 curve, 33 range, 193 restriction, 26 deflating subspace, 7 smooth, 24, 97 derivation, 37 at a point, 37 Gauss-Newton, 186 derivative, 38 generalized eigenvalue problem, 7 directional, 32, 92 geodesic, 102 descent mapping, 67 minimizing, 103 determinant Givens rotation, 58 derivative, 196 gradient, 46, 74, 196 diffeomorphism, 24 gradient-related, 62 differentiable, 24 Gram-Schmidt, 58 Lipschitz continuously, 148 graph, 28 differentiable structure, 19 Grassmann manifold, 6, 32 differential, 24, 38 qf, 173 Hausdorff, 20, 192 dimension Heine-Borel, 193 of subspace, 6 Hessian, 113 directional derivative, 195 Hessian operator, 197 distance horizontal distribution, 43 locally equivalent, 163 horizontal lift, 43, 50, 83 Riemannian, 46 horizontal space, 43, 48 distribution, 101, 120 ICA, 13 Eckart-Young-Mirsky theorem, 11 image, 191, 193 eigenpair, 6 immersed submanifold, 25 leftmost, 7 immersion, 38 eigenspace, 6 canonical, 24 extreme, 7 independent component analysis, 13 eigenvalue, 6 injection, 193 leftmost, 7 injectivity radius, 148 eigenvector, 5 inner iteration, 140 embedding space, 25 inner product, 45 epipolar constraint, 15 interior eigenvalues, 75 equivalence class, 27 invariant, 29 equivalence relation, 27 invariant subspace, 6, 7, 82, 85 Euclidean gradient, 46 leftmost, 7 Euclidean group, 14 rightmost, 7 Euclidean space, 45, 190 simple, 133 exponential, 112 spectral, 6, 128, 133 exponential map, 102 inverse, 193 exponential retraction, 103 Jacobi correction equation, 126 fiber, 194 Jacobi’s formula, 196 Finsler manifold, 53 Jacobian, 111 fixed point, 67 Jacobian matrix, 71 flag manifold, 29 JDCG, 167 foot, 34 Fr´echet differentiable, 195 Kantorovich’s theorem, 132 Frobenius norm, 11, 23 kernel, 191 function, 193 Koszul formula, 97 differentiable, 24 domain, 193 least squares, 11, 185 For general queries, contact [email protected] 00˙AMS September 23, 2007 © Copyright, Princeton University Press. No part of this book may be distributed, posted, or reproduced in any form by digital or mechanical means without prior written permission of the publisher. INDEX 223 Leibnizian, 37 submultiplicative, 190 length of a curve, 46 normal coordinates, 103 level set, 194 normal neighborhood, 102 Levenberg-Marquardt, 187 normal space, 47, 99 Lie bracket, 96 normalized essential manifold, 15 limit, 63 normed vector space, 190 limit point, 64, 192 notation limit set, 64 Ω, 194 linear convergence factor, 69 O, 194 Lipschitz constant, 198 o, 194 Lipschitz-continuous, 198 local rigidity, 55 oblique manifold, 12, 29 locally equivalent distances, 163 Olsen formula, 131 locally optimal conjugate gradient, 89 one-form field, 108 LOCG, 78 one-to-one correspondence, 193 Lojasiewicz’s inequality, 67 open set, 191 operator, 190 manifold, 19 bilinear, 190 dimension, 19 bilinear positive-definite, 190 linear, 22 bilinear symmetric, 190 nonlinear, 22 eigenvalue, 191 quotient, 28 eigenvector, 191 Riemannian, 69 invertible, 191 topology, 21 singular, 191 manifold structure, 20 order of convergence, 68 map, see function, 193 orthogonal complement, 191 mapping, see function, 193 orthogonal group, 27 matrix orthogonal projection, 191 commutator, 82 orthonormal, 6 identity, 189 orthonormal basis, 190 inverse, 189 invertible, 23, 189 paracompact, 21, 52 nonsingular, 189 parallel translation, 104 orthogonal, 189 parallel vector field, 104 orthonormal, 189 parameterization, 20 singular, 189 partition of unity, 20 skew-symmetric, 189 pencil, 7 square, 189 polar decomposition, 58 symmetric, 189 polarization identity, 106 matrix quotient manifold, 29 positive-definite, 113 matrix manifold, 17, 29 preimage, 193 matrix representation, 31 Procrustes problem, 12 matrix submanifold, 25 product manifold, 23 matrix-free, 10 product topology, 192 metric, 46 projection module, 53 canonical, 28 Moore-Penrose inverse, 186, 191 natural, 28 of function, 29 neighborhood, 192 pseudo-inverse, 131, 186, 191 Newton equation, 111 pullback, 55, 140 Newton vector, 111 norm, 190 qf, 58 consistent, 190 QR decomposition, 58 Frobenius, 191 thin, 196 induced, 190 quotient, 28 mutually consistent, 190 quotient manifold, 28, 83 operator, 190 Riemannian, 49, 83 spectral, 191 quotient topology, 193 For general queries, contact [email protected] 00˙AMS September 23, 2007 © Copyright, Princeton University Press. No part of this book may be distributed, posted, or reproduced in any form by digital or mechanical means without prior written permission of the publisher. 224 INDEX range, 191, 193 subspace rank, 24, 26 linear, 6 Rayleigh quotient, 8 topological, 193 generalized, 7, 84 subspace topology, 193 Rayleigh quotient iteration, 130 surjection, 193 real projective space, 30 symmetric operator, 191 regular value, 25 symmetric part, 81 residual, 180 restriction, 6, 26 T1, 192 retraction, 76 T2, 192 second-order, 107 tangent bundle, 36 Riemannian connection, 112 tangent map, 38 Riemannian distance, 46 tangent space, 34 Riemannian Hessian, 105 as vector space, 34 Riemannian manifold, 45 tangent vector, 34 Riemannian metric, 45 coordinates, 35 horizontally invariant, 100 realization, 34 Riemannian quotient manifold, 49 to a curve, 33 Riemannian submersion, 49 Taylor expansion, 198 Riemannian trust region, 141 tCG, 143 Ritz value, 129 thin SVD, 104 Ritz vector, 129 topological space, 192 root, 91 topology, 191 RTR, 141 basis, 192 finer, 192 Hausdorff, 192 saddle point, 66 of a manifold, 21 saddle-point problem, 130, 133 product, 192 search direction, 54 quotient, 193 second covariant derivative, 109 subspace, 193 second-countable, 20, 192 vector space, 193 sequence total space, 28 convergent, 63 trace, 7, 189 similarity transformation, 6 transpose, 189 singular values, 11 truncated CG, 143 skew-symmetric, 42 trust-region subproblem, 140 skew-symmetric part, 81 smooth, 19, 24, 197 unstable, 67 span, 6, 31 spectrum, 6 vector field, 36 sphere, 27 coordinate, 37 stable point, 67 on a curve, 102 star-shaped neighborhood, 102 vector space, 189 stationary point, 54 normed, 190 step size, 54 vector transport, 169 Stiefel manifold associated retraction, 170 noncompact, 23 velocity, 101 orthogonal, 26, 80 vertical space, 43 structure space, 29, 42 subimmersion, 52 Whitney sum, 169 submanifold, 25 embedded, 25, 47 zero of a function, 91 open, 21 regular, 25, 52 Riemannian, 47 submersion, 24, 38 canonical, 24 Riemannian, 49, 100 For general queries, contact [email protected].