sign in sign up
<<
Home , Divergence theorem, Eugenio Beltrami, Exterior derivative, Laplace operator

Appendix A Differential Forms and Operators on

In this appendix, we introduce differential forms on manifolds so that the integration on manifolds can be treated in a more general framework. The operators on manifolds can be also discussed using differential forms. The material in this appendix can be used as a supplement of Chapter 2.

A.1 Differential Forms on Manifolds

Let M be a k- and u =[u1, ··· ,uk] be a on M.A differential form of degree 1 in terms of the coordinates u has the expression

k i α = aidu =(du)a, i=1

1 k  1 where du =[du , ··· , du ]anda =[a1, ··· ,ak] ∈ C . The differential form is independent of the chosen coordinates. For instance, assume that v = [v1, ··· ,vk] is another coordinate system and α in terms of the coordinates v has the expression n i α = bidv =(dv)b, i=1 i n  1 ∂v ∂v 1 ··· ∈ where b =[b , ,bk] C .Write = j .By ∂u ∂u i,j=1

T ∂v dv =du , ∂u we have T ∂v dvb =du b, ∂u 340 Appendix A Differential Forms and Operators on Manifolds and by T ∂v a = b, ∂u we have (du)a =(dv)b. From differential forms of degree 1, we inductively define exterior differential forms of higher degree using wedge product, which obeys usual associate law, distributive law, as well as the following laws:

dui ∧ dui =0, dui ∧ duj = −duj ∧ dui, a ∧ dui =dui ∧ a = adui, where a ∈ R is a . It is clear that on a k-dimensional manifold, all forms of degree greater than k vanish, and a form of degree k in terms of the coordinates u has the expression 1 k 1 ω = a12···kdu ∧···∧du ,a12···k ∈ C . (A.1) To move differential forms from one manifold to another, we introduce the inverse image (also called pullback) of a f.LetM1 and M2 be two manifolds, with the coordinate systems u and v respectively. Let μ ∈ r C (r  1) be a mapping from M2 into M1, μ(y)=x, y ∈ M2.Letg be the parameterization of M2: y = g(v). We have

u =(μ ◦ g)(v).

r Hence, u is a C function of v.Letf be a function on M1. Then there is a function h on M2 such that

h(y)=f(μ(y)) y ∈ M2.

We denote the function h by μ∗f and call it the inverse image of f (by μ). Clearly, the inverse image has the following properties: 1. μ∗(f + g)=μ∗f + μ∗g; ∗ ∗ ∗ 1 2. μ (fg)=(, μ f)(μ g), f ∈ C ; i 3. if df = i aidu ,then ∗ ∗ ∗ i d(μ f)= (μ ai)d(μ u ). i We now define the inverse image of a differential form as follows. Definition A..1. Let ω be a differential form of degree j:

ω = fdun1 ∧···∧dunj . Then the inverse image of the differential form ω (by μ) is defined by

μ∗ω =(μ∗f)d (μ∗un1 ) ∧···∧d(μ∗unj ) . A.2 over Manifold 341

The properties of inverse image for functions therefore also hold for differen- tial forms. Besides, by Definition A.1, the following properties hold.

supp(μ∗ω) ⊂ μ(supp(ω)) and ∗ ∗ ∗ ∗ ∗ μ dω =d(μ ω),μ(ω1 ∧ ω2)=μ ω1 ∧ μ ω2.

A.2 Integral over Manifold

Let M be a compact k-manifold having a single coordinate system (M,u) and ω be a k-form on M given in (A.1). Let D = u(M) ⊂ Rk. Then the integral of ω over M is defined by 1 k ω = a12···kdu ∧···∧du . M D We often simply denote ω by ω if no confusion arises. Notice that if M the support of a12···k is a subset of D,thenwehave 1 k 1 k a12···kdu ∧···∧du = a12···kdu ∧···∧du . D Rk To extend the definition of the integral over a manifold, which does not have a global coordinate system, we introduce the partition of unity and the orientation of a manifold.

Definition A..2. Assume that {Ui} is an open covering of a manifold M. A countable set of functions {φj} is called a partition of unity of M with respect to the open covering {Ui} if it satisfies the following conditions: ∞ 1. φj ∈ C has, compact support contained in one of the Ui;  2. φj 0and j φj =1; 3. each p ∈ M is covered by only a finite number of supp(φj ). It is known that each manifold has a partition of unity. We shall use partition of unity to study orientation of manifolds. Orien- tation is an important topological property of a manifold. Some manifolds are oriented, others are not. For example, in R3 the M¨obius strip is a non- oriented 2-dimensional manifold, while the sphere S2 is an oriented one. To define the orientation of a manifold, we introduce the orientation of a vector .

Definition A..3. Let E = {e1, ··· , en} and B = {b1, ··· , bn} be two bases of the space Rn.LetA be a , which represents the linear transformation from B to E such that

ei = Abi,i=1, 2, ··· ,n. 342 Appendix A Differential Forms and Operators on Manifolds

We say that E and B have the same orientation if det(A) > 0, and that they have the opposite orientations if det(A) < 0. Therefore, for each n ∈ Z+, Rn has exactly two orientations, in which the orientation determined by the canonical o.n. is called the right-hand orientation, and the other is called the left-hand orientation. The closed k-dimensional half-space is defined by

k  k H = {[x1, ··· ,xk] ∈ R : xk  0}. (A.2)

Definition A..4. A connected k-dimensional manifold M (k  1) is said to be oriented if, for each point in M, there is a neighborhood W ⊂ M and a k k diffeomorphism h from W to R or H such that for each x ∈ W ,dhx maps the given orientation of the space TxM to the right-hand orientation of Rk. By the definition, if M is a connected oriented manifold, there is a differential { } ∈ ∩ structure (Wi,hi) such that at each p hi(Wi) hj(Wj ), the linear trans- ◦ −1 Rk → Rk ◦ −1 formation d(hi hj )p : , satisfies det d(hi hj )p > 0foreach p. We give two examples below to demonstrate the concept of orientation. Example A..1. Let U =[−π, π] × (−1, 1). M¨obius strip M2 is given by the parametric equation g : U → R3: u u u T g(u, v)= 2+v sin cos u, 2+v sin cos u, v cos . (A.3) 2 2 2 However, g is not a diffeomorphism from U to g(U) ⊂ R3 since it is not one-to-one (on the line u = π). To get an on M2, we define u u u T g˜(u, v)= −2+v cos cos u, −2+v cos cos u, v sin , 2 2 2 (A.4) and set U o =(−π, π) × (−1, 1). Then {(g(U o),g−1), (˜g(U o), g˜−1)} is a differ- entiable structure on M2 with g(U o) ∪ g˜(U o)=M2. Note that the following lines T L1 = {[2, 0,t] : −1

The opposite signs show that the M¨obius strip M2 is not oriented. Example A..2. The spherical S2 defined in Example 2.1 is oriented. 2 Recall that {(W1,h1), (W2,h2)} is an atlas on S . The transformation from A.2 Integral over Manifold 343 h2 =(˜u, v˜)toh1 =(u, v) is given in (2.25). The Jacobian of the transforma- tion is negative everywhere, for we have 1 det(df(˜ ˜))=− < 0. u,v (˜u2 +˜v2)2

2 Hence, S is oriented. Since det(df(˜u,v˜)) < 0, in order to obtain the same orientation over both W1 and W2, we can replace h2 by −h2. The definition of integrable functions on a manifold M needs the concept of zero-measure set on manifold. To avoid being involved in the measure theory, we define the integration over a manifold for continuous functions only. Definition A..5. Let M be a connected and oriented k-dimensional man- ifold, and Φ = {φj}j∈Γ be a partition of unity of M so that the support of each φj ∈ Φ is contained in the domain of a coordinate system. Let ω be a continuous differential form of degree k on M. The integral of ω over M is defined as ω = φj ω, j∈Γ provided the converges. 1 If M is compact, then a partition of unity of M is a finite set. Hence, ω over a compacted manifold always exists. It is clear that the integral is independent of the chosen partition. As an application, we introduce the formula of a manifold (M,g), where g is a parameterization of M. We define the volume differentiation of (M,g)bythek-form

dV = |Gg| du, where Gg is the Riemannian metric generated by g. Then the volume of M is V (M)= 1. M

The integral on an oriented manifold has the following properties. Theorem A..1. Assume that M is k-dimensional oriented manifold. Let −M denote the manifold having the opposite orientation with respect to M. Then we have the following. 1. For any continuous differential form ω of degree k on M, ω = − ω. −M M

2. For a, b ∈ R,

aω1 + bω2 = a ω1 + b ω2. M M M 344 Appendix A Differential Forms and Operators on Manifolds

3. Assume that at each coordinate system (W, u), the k-form ω can be rep- resented as a du1 ∧···∧duk,wherea is continuous and a  0. Then ω  0, M where the equality holds if and only if a =0. 4. If μ is a diffeomorphism from a k-dimensional oriented manifold M2 to a k-dimensional oriented manifold M1, keeping the orientation, and ω is a continuous k-form on M2,then μ∗ω = ω. M1 M2 To establish Stokes’ Theorem, we introduce the concept of the manifold with boundary. Let the boundary of the closed half-space Hk (see (A.2)) be defined by the hyperplane

∂Hk = Rk−1 × 0 ⊂ Rk.

In general, a manifold with boundary is defined as follows. Definition A..6. A k-dimensional manifold M ⊂ Rn is called the one with boundary if for each x ∈ M, there is a neighborhood W diffeomorphic to an open subset of Hk. The boundary of M, denoted by ∂M, is the set of the points mapped from the points in ∂Hk. It is easy to see that ∂M is a (k−1)-manifold. Note that each point x ∈ ∂M is also a point in M. Hence, the tangent space at x,say,TxM,isak-dimensional space with respect to M. However, since ∂M is (k − 1)-dimensional, its tan- gent space at x,say,Tx(∂M), is (k − 1)-dimensional space with respect to ∂M,andTx(∂M) ⊂ TxM.ThenTx(∂M) divides TxM into two half-spaces k k k k H+ and H−: all outward vectors are in H+ while all inward vectors in H−.It can be proved that if M is oriented, then ∂M is oriented. The orientation of M induces an orientation of its boundary ∂M in the following way: We first select a positive oriented basis {v1, ··· , vk} of TxM so that for x ∈ ∂M the vector v1 is an outward vector and the set {v2, ··· , vk} is a basis of Tx(∂M). Then the positive orientation of ∂M is determined by the basis {v2, ··· , vk}. This orientation on the boundary is called induced orientation. We now present the famous Stokes’ Theorem. Theorem A..2.(Stokes). Let M be a connected, oriented, and compact k-manifold with boundary and ω be a (k − 1)-form in C1.Then dω = ω, (A.5) M ∂M where the orientation over ∂M is induced from that over M. A.3 Laplace-Beltrami on Manifold 345 A.3 Laplace-Beltrami Operator on Manifold

The , named after Pierre-Simon Laplace, operating on func- tions defined on Euclidean spaces, can be generalized to operate on func- tions defined on Riemannian manifolds. This more general operator goes by Laplace-Beltrami operator, after Laplace and Eugenio Beltrami. Like the Laplace operator, the Laplace-Beltrami operator is defined as the of the , and is a linear operator mapping functions to functions. The operator can be extended to operate on as the divergence of the covariant . Alternatively, the operator can be generalized to oper- ate on differential forms using the divergence and . The resulting operator is called the Laplace-de Rham operator. Because in this book only Laplace-Beltrami operator is used, we introduce it in this section, skipping the discussion about the Laplace-de Rham operator. Let (M,g) be a connected, oriented, and compact Riemannian k-manifold with the metric Gg,and{(W, u)} be a differentiable structure on M.Letthe k-form having the expression du1 ∧···∧duk in terms of the coordinates u be denoted by du. A function f defined on M is called square integrable if the k-form |f|2dv is integrable. The space of all square integrable functions on M, denoted by H(M), is a equipped with the inner product f,h = fh M The Laplace-Beltrami operator is a generalization of the Laplace operator on in (2.5). Recall that the Laplace operator on C2(Rn)is defined as Δf = ∇·∇f = div grad f, where “div” denotes the divergent operator and “grad” denotes the gradient operator. We first generalize them on C2(M). Let |G| denote the determinant of the metric matrix [gij ]: |G| = det([gij ]). Let the coordinate mapping on M ∂ 1 ··· k be denoted by h and the coordinates under h be [u , ,u ]. Write ∂i = i .  ∂u The divergence of a vector field X =[X1, ··· , Xk] on M is defined by

k 1 div X = ∂ |G|X . | | i i G i=1 The following can be derived form the Stokes’ Theorem. Theorem A..3. Let M be a connected, oriented, and compact k-manifold def with boundary, and G = Gg is a metric on M.Let{(W, u)} be a coordinate T 1 system on M.LetX =[X1, ··· , Xk] be a C vector field on M. Define ˆ the (k − 1)-form diu by

ˆ diu =du1 ∧···∧dui−1 ∧ dui+1 ∧ duk. 346 Appendix A Differential Forms and Operators on Manifolds

Then the following equation holds. k ˆi div X = |G|Xid u. M ∂M i=1 The gradient of a scalar function f on M is a vector field defined by grad f =[(gradf)1, ··· , (grad f)k], with k i ij (grad f) = g ∂jf, j=1 ij ij where [g ] is the inverse matrix of Gg =[gij ]: [g ][gij ]=In. Definition A..7. The Laplace-Beltrami operator on C2(M) is defined by

k k 1 Δf = div grad f = ∂ |G|gij ∂ f . (A.6) | | i j i=1 j=1 G Assume that f ∈ C1 is a compactly supported function on M and X ∈ C1 is an k-vector field on M. Applying the Divergence Theorem on the vector field fX and considering f =0on∂M,wehave k ˆi div fX = f |G|Xid u =0. M ∂M i=1 Hence, we obtain ∇f,X = − f div X. M M Particularly, choosing X =gradh for a function h ∈ C2 and applying (2.40), we get − fΔh = ∇f,∇h , (A.7) M M which yields ∇f2 = − fΔf. (A.8) M M From (A.7), we also obtain another useful formula fΔh = hΔf. (A.9) M M ij When the Riemannian metric Gg is the identity so that gij = g = δij . Then the formulas for divergent, gradient, and Laplace-Beltrami operators are reduced to the traditional expressions: ∂ ∂ grad f = , ··· , , ∂u1 ∂uk A.3 Laplace-Beltrami Operator on Manifold 347

k div X = ∂iXi, i=1 and k ∂2f Δf = . ∂(ui)2 i=1 Index

A 300 low-rank VSV approximation, 307 Algorithm d LTSA algorithm, 226 -rank approximation SVD al- MVU algorithm, 186 gorithm, 301 PCA algorithm, 100, 101 ARPACK, 102 PRAT of Type 1, 330 CMDS algorithm, 126 PRAT of Type 2, 330 data neighborhood, 58 PRAT of Type 3, 330 Dijkstra’s algorithm, 153, 158, projection RAT (P-RAT), 320 163 random projection algorithm, Dijktra’s algorithm, 19 135 Dmaps algorithm, 271 randomized algorithms, 311 Dmaps algorithm of Graph-Laplacian randomized anisotropic type, 272 transformation (RAT), 319 Dmaps of Laplace-Beltrami type, randomized greedy algorithm, 274 315 eigenface, 109 randomized low-rank matrix in- EM-PCA algorithm, 102, 104 terpolation, 312 fast anisotropic transformation randomized SVD algorithm, (FAT), 316 311, 314 fast DR algorithms, 299 robust graph connection FAT, 320 algorithm, 85 FAT Isomaps, 322 RP face projection, 145 Floyd’s algorithm, 158 SDP, 186 graph connection-checking algo- SDP software package–CSDP, rithm, 84 187 Greedy anisotropic transforma- SDP software package–DSDP, tion (GAT), 318 187 greedy low-rank matrix approx- SDP software package–PENSDP, imation, 310 187 HLLE algorithm, 256 SDP software package–SBmeth, Implicitly Restarted Arnodi Method, 187 102 SDP software package–SDPA, interpolative RAT (IRAT), 319 187 Isomap algorithm, 156, 158, 186 SDP software package–SDPLR, L-ULV(A) rank-revealing, 305 187 L-ULV(L) rank-revealing, 304 SDP software package–SDPT3, landmark MVU, 194, 198 187, 188 Leigs algorithm, 240, 241 SDP software package–SeDumi, LLE algorithm, 207, 208 low-rank approximation algorithm, 187 350 Index

self-tuning Dmaps algorithm, D 276 semidefinite programming, 181 semidefinity programming, 186 Data, 51 ε-neighborhood, 52 SPCA algorithm, 104 B square distance, 56, 57 -coordinates, 56 k-nearest neighbors, 19 UTL rank-revealing, 304 k UTV Expansion Pack, 306 -neighborhood, 52 UTV Tools, 306 barycentric coordinates, 204 VSV rank-revealing, 306 compressive sensing, 22 Application configuration, 79, 82, 115, 117, 2-dimensional visualization, 192 125 classification, 17 configurative points, 117 data feature extraction, 2 constraints on DR output, 82 data visualization, 2 data compression (DC), 22 eigenfaces, 108 data geometry, 51 face recognition, 2, 3, 109, 145 data graphs, 60 fingerprint identification, 2 data matrix, 30 hyperspectral image analysis, 2, data models in DR, 79 112 dataset, 3 keyword search, 6 density of data, 292 LLE in image ordering , 214 dissimilarities, 79, 115, 116 pattern recognition, 17 dissimilarity, 51, 79, 119, 152 RP in document sorting, 146 exact configuration, 123 sensor localization, 190, 192 extrinsic , 52, 79 Supervised LLE, 215 feasible set, 186 text document classification, 2 geometric structure, 18 visual perception, 172 graph, 152, 156 visualization, 18 graph neighborhood, 83 high-dimensional data, 1, 51 HSI data, 152 B intrinsic configuration dimension, 123 Barycentric coordinates intrinsic dimension, 79 Hermite mean value interpola- landmarks, 194 tion, 205 lossless compression, 22 Lagrange interpolating property, lossy compression, 22 204 manifold coordinate representa- , 204 tion, 222 positivity, 204 manifold neighborhood, 83 multiscale diffusion features, 292 multiscale diffusion geometry, C 290 multiscale geometric structures, Curse of the dimensionality, 8 267 concentration of distances, 11 nearest neighbors, 12 concentration of norms , 11 neighborhood, 51, 52, 90, 152 diagonals of cube, 11 neighborhood system, 18, 60 empty space phenomenon, 9 noise, 80 tail probability, 10 noise estimation, 81 volume of a thin spherical shell, object vectors, 79 9 observed data, 18 volume of cubes and spheres, 9 olivetti faces, 3 point, 3 Index 351

point cloud, 3 successor set, 65 similarities, 79, 115, 116 symmetric digraph, 65 similarity, 18, 51, 79, 152 symmetric pair of nodes, 64 source coding, 22 tail of arc, 64 tangent coordinate representation, weakly connected digraph, 65 222 weight matrix, 65 tangent coordinates, 251 Dimension text documents, 6 extrinsic , 12, 17 training set, 108 intrinsic configuration vector, 3 dimension, 82, 83 visible data, 18 intrinsic dimensions, 12 Data graph intrinsic dimensions estimation, connection consistence, 83 12 consistent data graph, 83 Lebesgue covering dimension, 14 graph consistent conditions, 83 linearly intrinsic dimension, 13 local dimension consistence, 83 manifold dimension, 14 Differential form, 339 topological dimension, 14 inverse image, 340 Dimensions, 12 wedge product, 340 Distance degree, 339 l∞distance, 53 Digraph angular distance, 53 adjacency matrix, 64 derivative distance, 55 arc, 64 diffusion distance, 267 arrow, 64 directionally weighted distance, boundary in-volume of cluster, 55 66 distance in homogeneous boundary out-volume of cluster, Sobolev spaces, 55 66 edge distance, 55 direct successor, 64 Euclidean distance, 119, 152 directed edge, 64 Fourier distance, 54 directed path, 65 Fourier phase distance, 54 head of arc, 64 Fourier phase-difference in-degree, 64 distance, 55 in-neighborhood, 65 distance, 19, 46, 152, in-volume of cluster, 66 153, 175 in-volume of node, 66 graph distance, 152, 153, 175 inverted arc, 64 Mahalanobis distance, 56 inverted path, 65 Manhattan distance, 118 isomorphism, 64 path distance, 153 label graph, 64 S-distance, 175 linear digraph, 65 template distance, 53 oriented graph, 64 wavelet distance, 54 out-degree, 64 Dmaps, 267 out-neighborhood, 65 graph-Laplacina type, 269 out-volume of cluster, 66 Laplace-Beltrami type, 269 out-volume of node, 66 self-tuning Dmaps, 276 predecessor, 64 DR,1,3,12 predecessor set, 65 input data of the first type, 79 regular digraph, 65 centralization constraint on out- regular digraph of in-degree k, put, 83 65 classical multidimensional scal- regular digraph of out-degree k, ing (CMDS), 123 65 classical multidimensional scal- strongly connected digraph, 65 ing (LMDS), 115 subgraph, 65 352 Index

CMDS, 126, 191 eigenfaces, 109 diffusion maps, 267 face space, 108 generalized Leigs, 240 illumination variations, 111 geometric approach, 17, 18 large-scale features, 111 geometric embedding, 18 small-scale features, 111 hard dimensionality reduction, 17 Function Hessian locally linear embedding coordinate function, 32 (HLLE), 249 feature function, 267 input data of the second type, Gamma function, 33 81 homogeneous , 32 Isomaps, 151, 181 linearly independent, 32 , 18 local tangent coordinate Landmark MVU, 181 functions, 256 Laplacian eigenmaps, 235, 249 Taylor’s formula, 33 linear data model, 79 Linear method, 18 Hessian functional, 250 LMVU, 194 Hessian functional in manifold local tangent space alignment (LTSA), coordinates, 252 221 Hessian functional in tangent locally linear embedding (LLE), coordinates, 252 203, 206 linear functional, 31, 32 maximum variance unfolding (MVU), local tangent functional, 256 181 manifold Hessian functional, 250 MDS, 181 tangent Hessian functional, 252 multidimensional scaling (MDS), 115 multivariate analysis, 18 G MVU, 181 nonlinear data model, 80 Graph, 51 nonlinear method, 18 k-connected graph, 62 orthogonality constraint on out- k-edge-connected graph, 62 put, 83 k-regular graph, 61, 67 output data, 82 k-vertex-connected graph, 62 PCA, 126, 299 adjacency matrix, 62, 73 principal component analysis (PCA), adjacent nodes, 61 95 boundary volume of cluster, 63 random projection (RP), 131 cluster, 63 scaled output, 83 complete graph, 61, 70, 115 semidefinite embedding (SDE), connected component, 62, 70 181 contracting operator, 75 SLLE, 215 contraction, 75 soft dimensionality reduction, 18 degree of vertex, 61 spectral method, 18 diameter, 72 t-SLLE, 215 digraph, 60, 64 directed graph, 60, 64 Dirichlet sum, 69 E disconnected graph, 63 edge set, 156 Embedding edgeless graph, 60 linear embedding, 30 edges, 60 orthogonal embedding, 13, 30 end of edge, 60 end vertex of edge, 60 F endpoint of edge, 60 equivalent, 62 Face recognition Index 353

extended neighborhood of node, facial databases, 3 61 fingerpringt, 1 graph distance, 72 fingureprints, 1 graph refinement, 185 hand-writing letters and digits, harmonic , 69, 73 1 homomorphic graphs, 62 handwriting letters and digits, 5 homomorphism, 62 hyperspectral images, 1 isolated point, 67 hyperspectral images (HSI), 6 isolated vertex, 70, 73 images, 1 isomorphism, 62 speech signals, 1 label graph, 62 text documents, 1 Laplacian, 67 videos, 1 , 73 Hyperspectral image Laplacian on weighted graph, 74 band-image, 113 locally complete graph, 185 hyperspectral signature, 112 loop, 61 raw data, 52 neighborhood of node, 61 spectral curve, 112 node degree, 63 virtual band-image, 113 node volume, 63 nodes, 60 non-normalized Laplacian matrix, I 67 path, 62 Image, 52 points, 60 eigenface, 4 Rayleigh quotient, 69 gray-level, 52 regular graph, 61 intensity, 52 self edge, 61 Intrinsic dimension estimating simple digraph, 64 method simple graph, 60 capacity dimension, 16 spectral analysis, 67 correlation dimension, 15 spectral graph theory, 18, 51, 67 fractal dimension, 15 spectrum, 68 geometric approach, 14 subgraph, 61 multiscale estimation, 16 trivial graph, 60 nearest neighbor distance, 15 undirected graph, 60 projection technique, 14 unweighted graph, 63 Isomaps vertex, 60 ε-Isomap, 156 vertical set, 156 k-Isomap, 156 volume, 72 constant-shift technique, 156 volume of vertex, 74 weight, 62 weight matrix, 18, 62, 206 K weighted graph, 18, 62 weighted Laplacian matrix, 74 Kernel constant-shifted Isomap kernel, 154 H diffusion kernel, 291 Dmap kernel, 269 Heat diffusion Dmaps kernel of graph- heat capacity, 292 Laplacian type, 270 mass density, 292 Dmaps kernel of Laplace- thermal conductivity, 292 Beltrami type, 270 thermal diffusivity, 292 Gaussian kernel, 35, 237 High-dimensional data Gaussian-type diffusion kernel, facial images, 3 267 354 Index

graph Laplacian, 239 embedded, 13 HLLE kernel, 251, 252, 255 global theory, 29 Isomap kernel, 154, 158 hyperplane, 13, 31, 38, 79 Leigs kernel, 239, 269 linear, 13 LLE kernel, 208 linear manifold, 29 LTSA kernel, 227 local theory, 29 non-normalized diffusion kernel, M¨obius strip, 342 288 manifold geometry, 29 Normalization of Dmaps kernels, manifold with boundary, 344 269 minimum branch separation, PCA kernel, 100 177 sparse, 20 minimum radius of , 177 neighborhood, 14 L open covering, 14 orientation, 342 Linear DR method parameterization, 12, 36 classical multidimensional scal- parametrization, 29 ing (CMDS), 18 , 43 principal component analysis (PCA), S-curve, 22 13, 18 sampling data set, 51 random projection, 18 simple manifold, 38 LLE spherical surface, 342 invariance constraint, 206 submanifold, 38 sparseness constraint, 206 Swiss roll, 22 LTSA tangent place, 38 global alignment, 225 tangent space, 20, 29, 40, 222 local coordinate representation, tangent vector field, 43 224 volume, 343 coordinate system, 36 M Half-space, 342 induced orientation, 344 Machine learning, 55, 107 integral, 341 examples, 107 oriented manifold, 344 feature extractor, 107 Mapping feature function, 107 coordinate mapping, 36 feature vector, 55 diffeomorphism, 36 manifold learning, 29 differentiable homeomorphism, semi-supervised learning, 191 36 supervised learning, 107, 191 diffusion maps, 290 unsupervised learning, 107, 191 feature mapping, 290 Manifold, 18 homeomorphism, 36 k-manifold, 36 JL-embeddings, 133 of curve, 46 Lipschitz embedding, 131 atlas, 38 Lipschitz mappings, 131 compact manifold, 175 random projection, 133 connected manifold, 175 smooth, 36 convex manifold, 175 Matrix, 30 coordinate neighborhood, 36 array of distance matrices, 82 curvature, 16 best low-rank approximation, developable surface, 22 300 differentiable structure, 38, 42 best rank-revealing approxima- differential manifold, 36 tion, 300 discrete form, 51 centered data matrix, 121 Index 355 centered Gram matrix, 185 tangent , 252 centering Gram matrix, 119, , 30 122, 124 MDS centering symmetric matrix, 122 classical MDS (CMDS), 118, 123 centralizing matrix, 121 classical multidimensional scal- Cholesky decomposition, 120 ing, 123 coordinate-change matrix, 42 metric MDS, 117 covariance matrix, 98 metric multidimensional scaling data matrix, 119, 120 (MMDS), 123 diagonal matrix, 30 nonmetric MDS, 118 Euclidean distance matrix, 115, replicated MDS, 118 119, 120 unweighted MDS, 118 Euclidean square-distance matrix, weighted MDS, 118 119, 124 Metric general inverse, 31 canonic metric, 45 Gram matrix, 115, 119, 120 Euclidean metric, 45, 118, 119, greedy low-rank approximation, 152 309 geodesic metric, 152, 156 Hadamard product, 127, 254 graph metric, 156 Hadamard square, 309 Minkowski metric, 118 Hessian matrix, 33 Riemannian metric, 43 low-rank approximation, 299 manifold Hessian matrix, 250 matrix decomposition, 299 N matrix with orthonormal columns, 30 Nonlinear DR kernel, 90 matrix with orthonormal rows, distance approach, 88 30 weight approach, 89 Nystr¨om approximation, 307 Nonlinear DR method orthogonal matrix, 30 autoencoders, 21 positive definite matrix, 30 diffusion maps (Dmaps), 20 positive semi-definite (psd) ma- generative topographic map, 21 trix, 120 Hessian locally linear embedding positive semi-definite matrix, 30 (HLLE), 20 QR decomposition, 31 Isomaps, 19 random matrix, 131 Kohonen map, 21 random matrix of Type-1, 134 Laplacian eigenmaps (Leigs), 20 random matrix of Type-2, 134 local multidimensional scaling, random matrix of Type-3, 134 19 rank, 30 local tangent space alignment rank-revealing factorization, 299 (LTSA), 20 rank-revealing LU factorization, locally linear embedding (LLE), 302 19 rank-revealing QR factorization, machine learning, 21 302 manifold sculpting, 19 rank-revealing ULV factorization, maximum variance unfolding 303 (MVU), 19 rank-revealing URV factorization, neural network, 21 303 NeuroScale, 21 rank-revealing UTV factorization, self-organizing feature map 302 (SOFM), 21 Schur decomposition, 302 self-organizing map (SOM), 21 shifted Gram matrix, 121 semidefinite embedding (SDE), similarity matrix, 81 19 SVD decomposition, 299 356 Index

O first principal component, 99 first principal direction, 99 principal components, 99 Operator principal directions, 99 , 39 random component, 99 diffusion operator, 268 dilation, 31 discrete Laplacian, 238 S discrete Laplace-Beltrami, 237 discrete local Hessian, 255 Space divergent, 49, 346 affine space, 186 exponential formula, 236 dual, 31 feature extractor, 292 feature space, 290 gradient, 33, 39, 49, 346 Hilbert space, 32 Hessian, 20, 249, 250, 252 Housdorff space, 36 Laplace-Beltrami, 20, 49, 70, 235, Lebesgue space, 33 236, 268 metric space, 16 Laplacian, 34, 67 Neumann heat diffusion, 20, 35, 236, 268 T Nystr¨om approximation, 307 positive and self-adjoint, 236 Theorem projection, 30 Courant-Fischer Theorem, 69 , 31 Divergence Theorem, 48, 345 self-joint operator, 68 Inverse function theorem, 39 semi-group, 290 Johnson-Lindenstrauss Lemma, shift, 31 133 Young and Householder Theo- P rem, 123 Stokes’ Theorem, 344 PCA principal components, 96, 99, 108 U principal directions, 96, 99, 108 PDE Undirected graph Bobin boundary condition, 34 connected graph, 62, 65 Cauchy initial problem, 35 connected vertices, 62 diffusion kernel, 35 disconnected graph, 62 Dirichlet boundary condition, 34 disconnected vertices, 62 exponential formula, 35 isomorphic graphs, 62 Green’s function, 35 linear graph, 61 heat diffusion equation, 35 Laplacian eigenvalue problem, 34 Neumann boundary condition, 34 W

R Wavelet, 54 discrete wavelet transform, 54 DWT, 54 Random vector, 99, 100 multi-level DWT, 54 direction of maximum variance, 99