Manifold parametrizations by eigenfunctions of the Laplacian and heat kernels
Peter W. Jones*, Mauro Maggioni†‡, and Raanan Schul§
*Department of Mathematics, Yale University, 10 Hillhouse Avenue, New Haven, CT 06510; †Department of Mathematics, Duke University, Box 90320, Durham, NC 27708; and §Department of Mathematics, University of California, Box 951555, Los Angeles, CA 90095-1555
Communicated by Ronald R. Coifman, Yale University, New Haven, CT, October 29, 2007 (received for review September 3, 2007)
We use heat kernels or eigenfunctions of the Laplacian to construct Bcr (z). We call this method ‘‘heat triangulation’’ in analogy with local coordinates on large classes of Euclidean domains and Ri- triangulation as practiced in surveying, cartography, navigation, emannian manifolds (not necessarily smooth, e.g., with C ␣ metric). and modern GPS. Indeed, our method is a simple translation of These coordinates are bi-Lipschitz on large neighborhoods of the these classical triangulation methods. domain or manifold, with constants controlling the distortion and The embeddings we propose can be computed efficiently and the size of the neighborhoods that depend only on natural geo- therefore, together with the strong guarantees we prove, are metric properties of the domain or manifold. The proof of these expected to be useful in a variety of applications, from di- results relies on novel estimates, from above and below, for the mensionality reduction to data set compression and navigation. heat kernel and its gradient, as well as for the eigenfunctions of the Given these results, it is plausible to guess that analogous Laplacian and their gradient, that hold in the non-smooth category, results should hold for a local piece of a data set if that piece has and are stable with respect to perturbations within this category. in some sense a ‘‘local dimension’’ approximately d. There are Finally, these coordinate systems are intrinsic and efficiently com- certain difficulties with this philosophy. The first is that graph putable, and are of value in applications. eigenfunctions are global objects and any definition of ‘‘local dimension’’ must change from point to point in the data set. A spectral geometry ͉ nonlinear dimensionality reduction second difficulty is that our manifold results depend on classical estimates for eigenfunctions. This smoothness is often lacking in n many recent applications, one attempts to find local param- graph eigenfunctions. Ietrizations of data sets. A recurrent idea is to approximate a For data sets, heat triangulation is a much more stable object high dimensional data set, or portions of it, by a manifold of low than eigenfunction coordinates because dimension. A variety of algorithms for this task have been Y proposed (1–8). Unfortunately, such techniques seldomly come heat kernels are local objects; Y with guarantees on their capabilities of indeed finding local if a manifold M is approximated by discrete sets X, the parametrization (but see, for example, refs. 8 and 9) or on corresponding graph heat kernels converge rather nicely to the manifold heat kernel (4, 5); quantitative statements on the quality of such parametrizations. MATHEMATICS Y Examples of such disparate applications include document anal- one has good statistical control on smoothness of the heat ysis, face recognition, clustering, machine learning (10–13), kernel, simply because one can easily examine it and because ʦ 2 ٌ ʦ 2 nonlinear image denoising and segmentation (11), processing of one can use the Hilbert space {f L : f L }; articulated images (8), and mapping of protein energy land- Y our results that use eigenfunctions rely in a crucial manner on scapes (14). It has been observed in many cases that the Weyl’s lemma, whereas heat kernel estimates do not. eigenfunctions of a suitable graph Laplacian on a data set The philosophy used in this paper is as follows. provide robust local coordinate systems and are efficient in Step 1. Find suitable points yj,1Յ j Յ d and a time t so that dimensional reduction (1, 4, 5). The purpose of this paper is to the mapping given by heat kernels provide a partial explanation for this phenomenon by proving an ͑ 3 ͑ ͒ ͑ ͒͒ analogous statement for manifolds as well as introducing other x Kt x, y1 ,...,Kt x, yd coordinate systems via heat kernels, with even stronger guaran- tees. Here, we should point out the 1994 paper of Be´rard et al. is a good local coordinate system on B(z, cr). (This is heat (15) where a weighted infinite sequence of eigenfunctions is triangulation.) shown to provide a global coordinate system. (Points in the Step 2. Use Weyl’s lemma to find suitable eigenfunctions i so ᐉ ϭ ٌ Ϸ ٌ ʦ j manifold are mapped to 2.) To our knowledge, this was the first that (with Kj(x) Kt(x, yj)) one has ij (x) cj Kj (x), x B(z, result of this type in Riemannian geometry. If a given data set cr) for an appropriate constant c. has a piece that is statistically well approximated by a low dimensional manifold, it is then plausible that the graph eigen- Results functions are well approximated by the Laplace eigenfunctions Euclidean Domains. We first present the case of Euclidean do- of the manifold. One of our results is that, with the normalization mains. Although our results in this setting follow from the more that the volume of a d-dimensional manifold M equals one, any general results for manifolds discussed in the next section, the suitably embedded ball Br (z)inM has the property that one can case of Euclidean domains is of independent interest, and the find (exactly) d eigenfunctions that are a ‘‘robust’’coordinate exposition of the theorem is simpler. system on Bcr (z) (for a constant c depending on elementary We consider the heat equation in ⍀, a finite volume domain properties of M). In addition, these eigenfunctions, which de- in ޒd, with either Dirichlet or Neumann boundary conditions: pend on z and r, ‘‘blow up’’ the ball Bcr (z) to diameter at least one. In other words, one can find d eigenfunctions that act as a ‘‘microscope’’ on Bcr (z) and ‘‘magnify’’ it up to size ϳ1. Another Author contributions: P.W.J., M.M., and R.S. designed research, performed research, of our results is as follows. We introduce simple ‘‘heat coordi- analyzed data, and wrote the paper. nate’’ systems on manifolds. Roughly speaking (and in the The authors declare no conflict of interest. language of the previous paragraph), these are d choices of ‡To whom correspondence should be addressed. E-mail: [email protected]. manifold heat kernels that form a robust coordinate system on © 2008 by The National Academy of Sciences of the USA
www.pnas.org͞cgi͞doi͞10.1073͞pnas.0710175104 PNAS ͉ February 12, 2008 ͉ vol. 105 ͉ no. 6 ͉ 1803–1808 Downloaded by guest on September 26, 2021 Ѩ Ѩ We let rM (z0) ϭ sup{r Ͼ 0:Br (u(z0)) ʕ u(U)}. Observe that, ͩ⌬ Ϫ ͪu͑x, t͒ ϭ 0 ͩ ⌬ Ϫ ͪ u͑x, t͒ ϭ 0 when g is at least C 2, r can be taken to be the inradius, with local ͭ Ѩt or ͭ Ѩt M ͉ ϭ Ѩ ͉ ϭ coordinate chart given by the exponential map at z. We denote u Ѩ⍀ 0 vu Ѩ⍀ 0 by ʈgʈ␣ٙ1 the maximum over all i, j of the ␣ ٙ 1-Ho¨lder norm of Ѩ⍀ gij in the chart (U, u). The natural volume measure d on the Here, v is the outer normal on . Independently of the ͌ ⌬ ⍀ manifold is given, in any local chart, by det g; conditions 0.6 boundary conditions, we will denote by the Laplacian on . ⌬ For the purpose of this paper, in both the Dirichlet and guarantee that detg is uniformly bounded below from 0. Let M Neumann case, we restrict our study to domains where the be the Laplace Beltrami operator on M In a local chart, we have spectrum is discrete and the corresponding heat kernel can be 1 written as ⌬ f͑x͒ ϭϪ Ѩ ͑ ͱdet ggij͑u͑x͒͒Ѩ f ͒͑u͑x͒͒, [0.7] M ͱdet g j i i, jϭ1 ϩϱ ⍀ Ϫ ij ͑ ͒ ϭ ͑ ͒ ͑ ͒ jt where (g ) is the inverse of gij. Conditions 0.6 are the usual Kt z, w j z j w e , [0.1] jϭ0 uniform ellipticity conditions for the operator 0.7. With Dirichlet 2 or Neumann boundary conditions, ⌬M is self-adjoint on L (M, Յ where the {j} form an orthonormal basis of eigenfunctions of ). We will assume that the spectrum is discrete, denote by 0 Յ ⅐⅐⅐Յ Յ ⌬, with eigenvalues 0 Յ 0 Յ ⅐⅐⅐Յ j Յ ....Wealso require that 0 j its eigenvalues and by { j} the corresponding the following Weyl’s estimate holds, i.e., there is a constant orthonormal basis of eigenfunctions, and write Eqs. 0.1 and 0.2 ⍀ CWeyl,⍀ such that for any T Ͼ 0 with replaced by M.
d ͕ Յ ͖ Յ ⁄2͉⍀͉ Theorem 2. Let (M, g), z ʦ M and (U, u) be as above. Also, assume # j : j T CWeyl,⍀ T . [0.2] ͉M͉ ϭ 1. There are constants c1,...,c6 Ͼ 0, depending on d, cmin, This condition is always satisfied in the Dirichlet case, where in cmax, ʈgʈ␣ٙ1, ␣ ٙ 1, and CWeyl,M, such that the following hold. Let) ϭ d/2 Յ ␥ ϭ fact CWeyl,⍀ can be chosen independent of ⍀.) It should however Rz rM (z). Then there exist i1,...,id and constants c6Rz 1 be noted that these conditions are not always true and the ␥1 (z),...,␥d ϭ ␥d (z) Յ 1 such that Neumann case is especially problematic (16, 17). (a) the map Theorem 1. Embedding via Eigenfunctions, for Euclidean Domains. Let ⌽ ͑ ͒ 3 ޒd : Bc R z [0.8] ⍀ be a domain in ޒd satisfying all the conditions above, rescaled 1 z ͉⍀͉ ϭ Ͼ so that 1. There are constants c1,...,c6 0 that depend only 3 ͑␥ ͑ ͒ ␥ ͑ ͒͒ [0.9] x 1 i1 x ,..., d id x on d and CWeyl,⍀, such that the following hold. For any z ʦ ⍀, let Յ Ѩ⍀ d/2 Յ Rz dist (z, ). Then there exist i1,...,id and constants c6 Rz such that for any x1, x2 ʦ B (z, c1 Rz) ␥1 ϭ ␥1 (z),...,␥d ϭ ␥d(z) Յ 1 such that c c 2 ͑ ͒ Յ ʈ⌽͑ ͒ Ϫ ⌽͑ ͒ʈ Յ 3 ͑ ͒ (a) the map dM x1, x2 x1 x2 dM x1, x2 . [0.10] Rz Rz ⌽ : B ͑z͒ 3 ޒd [0.3] c1Rz (b) the associated eigenvalues satisfy
3 ͑␥ ͑ ͒ ␥ ͑ ͒͒ [0.4] Ϫ2 Ϫ2 x 1 i1 x ,..., d id x Յ Յ c4Rz i1,..., id c5Rz .
satisfies, for any x1, x2 ʦ B (z, c1 Rz), Remark. In both Theorem 1 and Theorem 2, the constants ␥j are given by c c 2 ʈ Ϫ ʈ Յ ʈ⌽͑ ͒ Ϫ ⌽͑ ͒ʈ Յ 3 ʈ Ϫ ʈ x1 x2 x1 x2 x1 x2 ; [0.5] Ϫ1 Rz Rz ⁄2
(b) the associated eigenvalues satisfy ␥ ϭ ͑zЈ͒2dzЈ j ij ͑ ͒ BcR z , Ϫ2 Յ Յ Ϫ2 z c4Rz i1,..., id c5Rz .
,where c is some fixed constant, depending on d, cmin, cmax, ʈgʈ␣ٙ1 . Remark 1. The dependence on the constant CWeyl,⍀ is only ␣ ٙ 1, and C needed in the Neumann case. Weyl Remark 2. The constant CWeyl,M is only needed in the Neumann case. Manifolds with ␣ Metric. C The results above can be extended to Remark 3. Most of the proof is done on one local chart certain classes of manifolds. To formulate a result corresponding containing z which we choose (one which contains a large to Theorem 1, we must first carefully define the manifold enough ball around z). An inspection of the proof shows that we analogue of dist (z, Ѩ⍀). Let M be a smooth, d-dimensional use only the norm ʈgʈ␣ٙ1 of the g restricted to this chart. In compact manifold, possibly with boundary. Suppose we are given particular, the theorem holds also for R Յ r (z). ␣ ␣ Ͼ ʦ z M a metric tensor g on M is C for some 0. For any z0 M, Remark 4. When rescaling Theorem 2, it is important to note let (U, u) be a coordinate chart such that z0 ʦ U and Ϫ1 that if f isaHo¨lder function with ʈfʈC ␣ٙ1 ϭ A and fr (z) ϭ f(r ʈ ʈ ϭ ␣ٙ1 Ͻ i) gil (u(z )) ϭ ␦il; z), then fr C ␣ٙ1 Ar . Since we will have r 1, fr satisfies a) ʈ ʈ ϭ ␣ٙ1 Յ ϭ ʈ ʈ 0 .d, better Ho¨lder estimate then f, i.e., fr C ␣ٙ1 Ar A f C ␣ٙ1ޒ ii) for any x ʦ U, and any , ʦ) Remark 5. We do not know, in both Theorem 1 and Theorem d d 2, whether it is possible to choose eigenfunctions such that ␥1 ϭ ϭ ␥ ͑ ͒ʈʈ2 Յ ij͑ ͑ ͒͒ ij͑ ͑ ͒͒ ... d. c g ޒd g u x i j g u x ivj min Another result is true. One may replace the d chosen eigen- i, jϭ1 i, jϭ1 functions above by d chosen heat kernels, i.e., {Kt(z, yi)}iϭ1,...,d. Յ ͑ ͒ʈʈ ʈ ʈ cmax g ޒd v ޒd. [0.6] In fact, such heat kernels arise naturally in the main steps of the
1804 ͉ www.pnas.org͞cgi͞doi͞10.1073͞pnas.0710175104 Jones et al. Downloaded by guest on September 26, 2021 proofs of Theorem 1 and Theorem 2. This leads to an embedding where one really uses the Ho¨lder condition is an estimate on map with even stronger guarantees: how much the gradient of a (global) eigenfunction changes in a ball. Theorem 3. Heat Triangulation Theorem. Let (M, g), z ʦ M and (U, (c) We will use Brownian motion arguments (on the manifold). u) be as above, where we now allow ͉M͉ ϭϩϱ. Let Rz Յ min{1, r (z)}. Let p ,...,p be d linearly independent directions. There To have existence and uniqueness, one needs smoothness M 1 d assumptions on the metric (say, C 2). Therefore, we will first ␣ , are constants c ,...,c Ͼ 0, depending on d, c , c , ʈgʈ␣ٙ 1 6 min max 1 2 ٙ 1, and the smallest and largest eigenvalues of the Gramian matrix prove the theorem in the manifold case in the C metric category, and then use perturbation estimates to obtain the (pi, pj)i,jϭ1,...,d, such that the following holds. Let yi be so that yi Ϫ Յ Յ ϭ result for gij ʦ C ␣. To this end, we will often have dependence z is in the direction pi, with c4Rz dM ( yi, z) c5Rz for each i ␣ ϭ 2 ⌽ 3 ޒd on the C norm of the coordinates of the gij even though we 1,...,d and let tz c6Rz. The map : Bc R (z) , defined by 1 z will be (for a specific lemma or proposition) assuming the g 3 ͑ d ͑ ͒͒ d ͑ ͒ 2 x Rz Ktz x, y1 ,...,Rz Ktz x, yd ) [0.11] has C entries. (d) We will use estimates from ref. 18. The theorems in ref. 18 satisfies, for any x , x ʦ B (z), 1 2 c1Rz are stated only for the case of d Ն 3. Our theorems are true ϭ ϭ c2 c3 also for the case d 2 (and trivially, d 1). This can beseen d ͑x , x ͒ Յ ʈ⌽͑x ͒ Ϫ ⌽͑x ͒ʈ Յ d ͑x , x ͒. ˜ ϭ ϫ ޔ R M 1 2 1 2 R M 1 2 indirectly by considering M : M and noting that the z z eigenfunctions of M˜ and the heat kernel of M˜ both factor. This holds for the manifold and Euclidean case alike and The idea of the proof of Theorem 1 and 2 is as follows. We start depends only on estimates for the heat kernel and its gradient. by fixing a direction p1 at z. We would like to find an eigenfunc- Remark 6. One may replace the (global) heat kernel above with Ϫ tion such that ͉Ѩ տ R 1 on B (z). To achieve this, we a local heat kernel, i.e., the heat kernel for the ball B (z, R ) with i1 p1 i1 z c1Rz z start by showing that the heat kernel has large gradient in an the metric induced by the manifold and Dirichlet boundary ϳ Ϫ1 conditions. In fact, this is a key idea in the proof of all of the annulus of inner and outer radius Rz around y1 (y1 chosen above theorems. such that z is in this annulus, in direction p1). We then show that Remark 7. All theorems hold for more general boundary the heat kernel and its gradient can be approximated on this annulus by as the partial sum of 0.1 over eigenfunctions which condi-tions. This is especially true for the Heat Triangulation Ϫ2 Ϫd/2 satisfy both ϳ R and R ʈʈ 2 տ 1. By the Theorem, which does not even depend on the existence of a z z L (Bc1Rz (z)) spectral expansion for the heat kernel. pigeon-hole principle, at least one such eigenfunction, let it be Example 1. It is a simple matter to verify this theorem for the i1, has a large partial derivative in the direction p1. We then d ϭ ϭ ٌ Ќٌ ޒ case where the manifold in . For example, if d 2, Rz 1, and consider i1 and pick p2 i1 and by induction we select i1, ϭ ϭ Ϫ ϭ Ϫ z 0, y1 ( 1, 0) and y2 (0, 1). Then if Kt (x, y)isthe ..., id, making sure that at each stage we can find ik, not ͉Ѩ ͉ ϳ Ϫ1 Euclidean heat kernel, previously chosen, satisfying p i Rz on Bc R (z). We finally ⌽ ϭ k k 1 z show that the : ( i1,..., id) satisfies the desired properties.
3 ͑ ͑ ͒ ͑ ͒͒ MATHEMATICS x K1 x, y1 , K1 x, y2 Step 1. Estimates on the heat kernel and its gradient. Let K be the Dirichlet or Neumann heat kernel on ⍀ or M, corresponding is a (nice) biLipschitz map on B1/2 ((0, 0)). (The result for to one of the Laplacian operators considered above associated arbitrary radii then follows from a scaling argument.) This is with g. We have the spectral expansion because on can simply evaluate the heat kernel Kt (x, y) ϭ Ϫ(͉xϪy͉2) [1/(4 t)]e /4t.InB1/2 ((0, 0)), ϩϱ Ϫ ͑ ͒ ϭ jt ͑ ͒ ͑ ͒ 1 1 1 1 Kt x, y e j x j y . Ϫ ⁄4 Ϫ ⁄4 K ͑x, y ͒ ϳ e ͑1, 0͒ and ٌK ͑x, y ͒ ϳ e ͑0, 1͒. jϭ0ٌ 1 1 2 1 2 2
Շ When working on a manifold, we can assume in what follows Notation. In what follows, we will write f (x) c1,...,cn g(x) if there that we fix a local chart containing BRz (z). exists a constant C depending only on c1,...,cn, and not on f, g,orx, such that f(x) Յ Cg (x) for all x (in a specified domain). Assumption A.1. Let the constants ␦0, ␦1 Ͼ 0 depend on d, cmin, cmax, We will write f(x) ϳc ,...,c g(x) if both f(x) Շc ,...,c g(x) and g(x) n 1 n ʈgʈ␣ٙ , ␣ ٙ 1. We consider z, w ʦ ⍀ satisfying (␦ /2) R Ͻ t1/2 Ͻ 1 Շ f(x). We will write a ϳC2 b for a, b vectors, if a ϳC2 b for 1 1 z c1,...,cn C1 i C1 i ␦ R and ͉z Ϫ w͉ Ͻ ␦ R . all i. 1 z 0 z ʦ ␣ ␦ The Proofs Proposition 4. Under Assumption A.1, let g C , 0 sufficiently small, and ␦ is sufficiently small depending on ␦ . Then there are The proofs in the Euclidean and manifold case are similar. In this 1 0 constants C1, C2, CЈ1, CЈ2, C9 Ͼ 0, that depend on d, ␦0, ␦1, cmin, cmax, section, we present the main steps of the proof. A full pre- ʈ ʈ ␣ ٙ Ј Ј sentation is given in ref. 2. Some remarks about the manifold g ␣ٙ1, 1} and C1, C2, C9 dependent also on CWeyl, such that case: the following hold: (a) As mentioned in Remark 3, we will often restrict to working (i) the heat kernel satisfies on a single (fixed) chart in local coordinates. When we Ϫd K ͑z, w͒ ϳ C2 t ⁄2 ; [0.12] discuss moving in a direction p, we mean in the local t C1 coordinates. (ii) if (1/2)␦ R Ͻ ͉z Ϫ w͉, p is a unit vector in the direction of z Ϫ ʈ ʈ 0 z b) Let us say a few words about how the dependence on g ␣ٙ1 w, and q is arbitrary unit vector, then) comes into play. Generally speaking, in all places except one ٙ ␣ (which we will mention momentarily), the 1-Ho¨lder Ј Ϫd Rz Ј Ϫd Rz ϳ C2 ⁄2 ͉Ѩ ͑ ͉͒ϳC2 ⁄2 ͒ ͑ ٌ͉ Kt z, w Ј t and pKt z, w Ј t condition is used to get local bi-Lipschitz bounds on the C1 t C1 t perturbation of the metric (resp. the ellipticity constants) from the Euclidean metric (resp. the Laplacian). The place [0.13]
Jones et al. PNAS ͉ February 12, 2008 ͉ vol. 105 ͉ no. 6 ͉ 1805 Downloaded by guest on September 26, 2021 We start by restricting our attention to eigenfunctions do not z Ϫ w Ϫd R Ϫd R ͯѨ ͑ ͒ Ϫ Јͳ ʹ ⁄2 zͯՅ ⁄2 z have too high frequency. Let ⌳ (A) ϭ { : Յ AtϪ1} and ⌳ qKt z, w C2 q, t C9t , [0.14] L j j H ʈz Ϫ wʈ t t Ϫ1 c (AЈ) ϭ {j : j Ͼ AЈt } ϭ⌳L (AЈ) . A first connection between the heat kernel and eigenfunc- where C 3 0 as ␦ 3 0 (with ␦ fixed); 9 1 0 tions is given by the following truncation Lemma. ij 2 (iii) if in addition g ʦ C , (1/2)␦0Rz Ͻ ͉z Ϫ w͉, and q is as above, then for s Յ t, Lemma 1. Assume g ʦ C 2. Under Assumption A.1, for A Ͼ 1 large enough and AЈϽ1 small enough, depending on ␦0, ␦1, C1, C2, CЈ1, Ϫd Ϫd R ⁄2 ⁄2 z CЈ2 (as in Proposition 4), there exist constants C3, C4 (depending on Շ ٌ͉ ͑ ͉͒ Շ Ј ͒ ͑ Ks z, w C2 t , Ks z, w C2 t t A, AЈ as well as {d, cmin, cmax, ʈgʈ␣ٙ1, ␣ ٙ 1}) such that
Ϫd R (i) The heat kernel is approximated by the truncated expansion ⁄2 z ͉Ѩ ͑ ͉͒ Շ Ј [0.15] and qKs z, w C2 t ; t C4 Ϫ t K ͑z, w͒ ϳ ͑z͒ ͑w͒e j . t C3 j j jʦ⌳ ͑A͒ (iv) C1, C2 both tend to a single function of {d, cmin, cmax, ␦0, L CWeyl}, as ␦1 tends to 0 with ␦0 fixed; (ii) If (1/2)␦0 Rz Ͻ ͉z Ϫ w͉ and p is a unit vector parallel to z Ϫ 2 (v) if g ʦ C , then also CЈ1, CЈ2, C9 can be chosen independently of w, then ͉ ͉ Յ ϩϱ CWeyl. Furthermore, the above estimates also hold for M . Ϫ K ͑z, ⅐͒ ϳ C4 ͑zٌ͒ ͑⅐͒e jt ٌ w t C3 j w j At this point we can side track and choose heat kernels {Kt jʦ⌳L͑A͒പ⌳H͑AЈ͒ 2 (⅐, y )} ϭ , with t ϳ R , that provide a local coordinate chart i i 1,...,d z Ϫ Ѩ K ͑z, ⅐͒ ϳ C4 ͑z͒Ѩ ͑⅐͒e jt. with the properties claimed in Theorem 3. p t C3 j p j 2 ʦ⌳ ͑ ͒പ⌳ ͑ Ј͒ Proof of Theorem 3. We start with the case g ʦ C . Let us j L A H A consider the Jacobian ˜J (x), for x ʦ Bc R (z), of the map 1 z (iii) C3, C4 both tend to 1 as A 3 ϱ and AЈ 3 0. ⌽˜ ϭ Ϫd ϩd/2͑ 2͒⌽ : Rz t t/Rz . This lemma implies that in the heat kernel expansion, we do not need to consider eigenfunctions corresponding to eigenval- ͉˜ Ϫ Ј͗ Ϫ ʈ Ϫ ʈ͘ Ϫ1͉ Յ Ϫ1 By 0.14 we have Jij (x) C2 pi,(x yj)/ x yj Rz C9Rz , with ues larger than AtϪ1. However, in our search for eigenfunctions Ј C2 independent of CWeyl. As dictated by Proposition 4, by with the desired properties, we need to restrict our attention ␦ ␦ choosing 0, 1 appropriately (and, correspondingly, c1 and c6), further, by discarding eigenfunctions that have too small a ⑀ we can make the constant C9 smaller than any chosen , for all gradient around z. As a proxy for gradient, we use local energy. entries, and for all x at distance no greater than c1 Rz from z, Recall Avez ( f) ϭ ( ͉f͉2)1/2, and let ϭ ϭ 2 ⌽ R BR(z) where we use t tz c6 Rz for ˜ . Therefore, for c1 small enough z compared to c4 we can write Rz˜J (x) ϭ Gd ϩ E(x), where Gd is ⌳ ͑ ␦ ͒ ϭ͕ ʦ ͑⌬͒ 1 ͑ ͒ Ն ͖ E z, Rz, 0, c0 : j : Ave ␦ j c0 . 0Rz the Gramian matrix ͗pi, pj͘ (indepedent of x), and ͉Eij (x)͉ Ͻ ⑀, for 2 ʦ Ϫ1 Ϫ ⑀ ʈʈ Յ ʈ ˜ ʈ Յ x Bc1Rz (z). This implies that Rz ( min Cd ) J (x) Ϫ1 ϩ ⑀ ʈʈ The truncation Lemma 1 can be strengthened into Rz ( max Cd ) , with Cd depending linearly on d, where max and min are the largest and, respectively, smallest eigen- ʦ 2 ⑀ Lemma 2. Assume g C . Under Assumption A.1, for C3, C4 close values of Gd. At this point, we choose small enough, so that the enough to 1 (as in Lemma 1), and c small enough (depending on above bounds imply that the Jacobian is essentially constant in 0 , d, c , c , ʈgʈ␣ٙ , ␣ ٙ 1, and C ), there exist constants C B (z), and by integrating along a path from x to x in B (z), min max 1 Weyl,M 5 c1Rz 2 2 c1Rz C (depending only on C , C , c , and C ) such that the heat ⌽ ⌽˜ 6 3 4 0 Weyl,M we obtain the Theorem ( and differ only by scalar multipli- kernel satisfies cation). We note that ⑀ ϳ 1/d suffices. To get the result when g ␣ Ϫ is only C we use perturbation techniques for the heat kernel (2). K ͑z, w͒ϳC6 ͑z͒ ͑w͒e jt t C5 j j We proceed towards the proof of Theorem 1 and 2. The jʦ⌳L͑A͒പ⌳E͑z,Rz,␦0,c0͒ following steps aim at replacing appropriately chosen heat ␦ Ͻ ͉ Ϫ ͉ ⌳ ϭ⌳ പ ⌳ Ј പ ⌳ kernels by a set of eigenfunctions, by extracting the ‘‘leading and if (1/2) 0Rz z w , then, if : L (A) H (A ) E ␦ terms’’in their spectral expansion. (z, Rz, 0, c0), z ϭ Step 2. Heat kernel and eigenfunctions. Let AveR ( f) ( B (z) Ϫ R Ѩ ͑ ͒ϳC6 ͑ ͒Ѩ ͑ ͒ jt 2 1/2 pKt z, w j z p j w e . ͉f͉ ) . We record the following (2): C5 jʦ⌳ ij ʦ ␣ Ͻ Proposition 5. Assume g C . There exists b1 1, that depends C , C tend to 1 as C , C tend to 1 and c tends to 0. ʈ ʈ ␣ ٙ 5 6 3 4 0 .on d, cmin, cmax, g ␣ٙ1, 1 such that the following holds. For an Step 3. Choosing appropriate eigenfunctions ⌬ Յ eigenfunction j of M, corresponding to the eigenvalue j, and R The set of eigenfunctions with eigenvalues in ⌳ (as in Lemma R , the following estimates hold. For w ʦ B (z) and x, y ʦ z b1R 2) is well suited for our purposes, in view of:
Bb1R (z), Lemma 3. ʦ 2 ␦ ͉ ͑ ͉͒ Շ ͑ 2͒ z ͑ ͒ Assume g C . Under Assumption A.1, for 0 small j w P1 jR AveR j enough, there exists a constant C7 depending on {C1, C2, CЈ1, CЈ2, C5, ʈ ʈ ␣ ٙ ␦ ␦ ʈٌ ͑ ͒ʈ Շ Ϫ1 ͑ 2͒ z ͑ ͒ 1} and C8 depending on { 0, cmin, cmax, g ␣ٙ1, 1} such that j w R P2 jR AveR j the following holds. For any direction p there exist j ʦ ⌳ :ϭ⌳L (A) പ⌳ Ј പ⌳ ␦ ʈٌ ͑x͒ Ϫ ٌ ͑y͒ʈ H (A ) E (z, Rz, 0, c0) such that ͒ j j Շ Ϫ1Ϫ␣ٙ1 ͑ 2͒ z ͑ ٙ1 R P3 jR AveR j␣ ʈx Ϫ yʈ C Ϫ1 z ͉Ѩ ͑ ͉͒ ϳ 8 1 p j z C Rz Ave ␦ j, 7 uRz ij 2 with constants depending only on d, cmax, cmin, ʈg ʈ␣ٙ1, and P1 (x) ϭ (1/2)ϩ (3/2)ϩ (5/2)ϩ (1 ϩ x) , P2 (x) ϭ (1 ϩ x) , P3 (x) ϭ (1 ϩ x) , with and moreover, if ʈz Ϫ zЈʈ Յ b1Rz, where b1 is a constant that depends the smallest integer larger than or equal to (dϪ2)/4. on C7, C8, d, cmin, cmax, ʈgʈ␣ٙ1, ␣ ٙ 1, then 
1806 ͉ www.pnas.org͞cgi͞doi͞10.1073͞pnas.0710175104 Jones et al. Downloaded by guest on September 26, 2021 ʦ ޒkϩ1 ¥kϩ1 ϭ ͗¥kϩ1 pkϩ1. Assume a and nϭ1 anpn 0, then nϭ1 anpn, ͘ϭ0 for all l ϭ 1,...,k ϩ 1, i.e., a solves the linear systemٌ 1.2 1.2 jl Akϩ1 a ϭ 0. But Akϩ1 is lower triangular with all diagonal entries non-zero, hence a ϭ 0. , For l Յ k, we have ٌ͗j , pkϩ1͘ϭ0 and, by Lemma 3, ٌ͉͗j 1 1 ͉͘ տ Ϫ1 ⌽ ϭ l ⌽ϭ⌽ l pl Rz . Now let k ( j1,..., jk) and d. We start by showing that ʈٌ⌽͉z (w Ϫ z)ʈ տd 1/Rz ʈw Ϫ zʈ. Indeed, suppose that ʈٌ⌽k͉z (w Ϫ z)ʈ Յ c/Rz ʈw Ϫ zʈ, for all k ϭ 1,...,d. For c small 0.8 0.8 enough, this will lead to a contradiction. Let w Ϫ z ϭ ¥l alpl.We have (using say Lemma 3) 0.6 0.6 1 ͉ ͉ ʈٌ⌽ ͉ ͑ Ϫ ͒ʈ ϭ Ѩ ⌽ ͉ տ ͉ ͉ Ϫ k z w z al p k z ͩ ak c al ͪ . Ͱ l Ͱ Rz 0.4 0.4 lՅk lϽk
͉ ͉ Յ ¥k l ʈ Ϫ ʈ ͉ ͉ Յ ʈ Ϫ By induction, ak lϭ1 c w z . For c small enough, ai w 0.2 0.2 zʈ/d. This is a contradiction since ʈ¥i aipiʈ ϭ ʈw Ϫ zʈ and ʈpiʈ ϭ 1. We also have, by Proposition 5,
ʈz Ϫ wʈ ␣ٙ1 1 ͪ ͩ ʈٌ⌽͉ Ϫ ٌ⌽͉ ʈ Շ 1.2 1 0.8 0.6 0.4 0.2 1.2 1 0.8 0.6 0.4 0.2 w z . [0.16] Rz Rz Fig. 1. A non-simply connected domain in ޒ2. The dark circle is the neigh- borhood to be mapped, and grayscale intensity represents two (left and right) Finally, by ensuring ʈz Ϫ wiʈ/Rz is smaller than a universal eigenfunctions for the embedding. Each of them has about half an oscillation constant for i ϭ 1, 2, we get from Eq. 0.16 in the neighborhood, and these two half-oscillations are in roughly orthog- onal directions. 1 ʈ⌽͑w ͒ Ϫ ⌽͑w ͒ʈ ϭ ͯ͵ ٌ⌽͉ ϩ͑ Ϫ ͒ ͑w Ϫ w ͒dtͯ 1 2 tw1 1 t w2 1 2 0 C Ϫ1 z ͉Ѩ ͑ Ј͉͒ ϳ 8 1 p j z C Rz Ave ␦ j. 7 0Rz 2 1 ͒͒ ϭ ͯ͵ ٌ͑⌽͉ ϩ ٌ͑⌽͉ ϩ͑ Ϫ ͒ Ϫ ٌ⌽͉ This lemma yields an eigenfunction that serves our purpose in w1 tw1 1 t w2 w1 a given direction. From this point onwards we let, with abuse of 0 z Ϫ1 notation, j(zЈ) ϭ͑Ave1 j) ⅐ j(zЈ). To complete the proof 1 ␦ R 1 2 0 z ⅐ ͑w Ϫ w ͒dtͯ տ ͵ ʈw Ϫ w ʈdt 1 2 1 2 of the theorems, we need to cover d linearly independent Rz
0 MATHEMATICS directions. Pick an arbitrary direction p1.ByLemma 3 we can Ϫ1 Ϫ1 find j1 ʦ ⌳, (in particular j1 ϳ t ) such that ͉Ѩp j (z)͉ Ն c0R . l l z 1 տ ʈ Ϫ ʈٌ Let p2 be a direction orthogonal to j1 (z). We apply again c0 w1 w2 , Ͻ Ϫ1 ͉Ѩ ͉ Ն Ϫ1 Rz Lemma 3, and find j At so that p j (z) c0Rz . Note 2 2 2 that necessarily j2 j1 and p2 is linearly independent of p1. In fact, which proves the lower bound 0.5. To prove the upper bound of Ѩ ϭ by choice of p2, p2 j1 0. We proceed in this fashion. By 0.5, we observe that from Proposition 5 we have the upper bound z induction, once we have chosen j1,..., jk (k Ͻ d), and the ͉Ѩ ͉Շ 2 Ϫ pl il (z) AveRz ( il), and since il is L -normalized, the right p p ͉Ѩ z ͉ Ն c R 1 l ϭ ϳ Ϫd/2 ␥ corresponding 1,..., k, such that pl jl ( ) 0 z , for hand side can be as big as Rz . Therefore, we can choose l ϩ ͗ ٌ ϭ ͘ 1,..., k, we pick pk 1 orthogonal to { jn}n 1,...,k and apply as in the statement of the theorem so as to satisfy the upper Ϫ1 ϩ ͉Ѩ ͉ Ն Lemma 3, that yields jk 1 such that pkϩ1 jkϩ1 (z) c0Rz . bound 0.5. This completes the proof for the Euclidean case. ϩ ϭ Ѩ ϭ ϩ ʦ 2 We claim that the matrix Ak 1 : ( pn jm)m,n 1,...,k 1 is lower In the manifold case, we consider first the case g C , and thus triangular and {p1,..., pkϩ1} is linearly independent. Lower- let ␣ ٙ 1 ϭ 1 in what follows. Let Rz be as in the theorem. We triangularity of the matrix follows by induction and the choice of take c1 Յ (1/2)␦0 chosen so that
Fig. 2. Example of localization. When the entrance in the room is small enough, all of the eigenfunctions that Theorem 1 selects for mapping a neighborhood Ϫd/2 ϳ of a point z in the small room need to be rescaled by a factor Rz to map a neighborhood of z to a ball of size 1.
Jones et al. PNAS ͉ February 12, 2008 ͉ vol. 105 ͉ no. 6 ͉ 1807 Downloaded by guest on September 26, 2021 il͑ ͒ Ϫ ␦il͉ ϭ ͉ il͑ ͒ Ϫ il͑ ͉͒ Ͻ ʈ ʈ ʈ Ϫ ʈ␣ٙ1 Ͻ ⑀ 2 ͉ ,g x g x g z g ␣ٙ1 x z 0 norm arbitrarily well by a C (M) metric. By the above lemma and our main theorem for the case of C 2 metric, we obtain the ʦ ␣ for all x B2c1Rz (z). For this g, the above is carried on in local theorem for the C case. coordinates. It is then left to prove that the Euclidean distance in the range of the coordinate map is equivalent to the geodesic Examples distance on the manifold. For all x, y ʦ Bc R (z) 1 z Example 2. Mapping with eigenfunctions, non simply-connected 1 x Ϫ y domain. We consider the planar, non-simply connected domain ٙ1 ⍀ ʦ ⍀␣ Յ ͵ Ͱ Ͱ ͑ ϩ ʈ ʈ␣ٙ ͒ in Fig. 1. We fix a point z , as in the figure, and display ͒ ͑ dM x, y ʈ Ϫ ʈ 1 g 1t dt x y ޒd 0 two eigenfunctions whose existence is implied by Theorem 1. Example 3. Localized eigenfunctions. In this example, we show that Շ ͑ ϩ ʈ ʈ ͒ʈ Ϫ ʈ ٙ1 1 g ␣ٙ1 x y . the factors ␥1,...,␥d in Theorem 1 and 2 may in fact be required␣ to be as small as Rd/2. We consider the “two-drums” domain ␥ 3 z For the converse, let : [0, 1] M be the geodesic from x to y. in Fig. 2, consisting of a unit-size square drum, connected by a ␥ is contained in B (x) on the manifold, whose image in the 2dM(x,y) small aperture to a small square drum, with size /N, where is local chart is contained in B ϩʈ ʈ␣ٙ (x). We have 2(1 g 1)dM(x,y) the golden ratio. The with of the connecting aperture is ␦/N, for small ␦. For this domain, for small enough ␦, and for z in the d տ ͑ Ϫ ʈ ʈ␣ٙ ͒ʈ Ϫ ʈ smaller square, it can be shown that all possible eigenfunctionsޒտ ͑ Ϫ ʈ ʈ␣ٙ ͒͵ ʈ␥͑ ͒ʈ ͒ ͑ dM x, y 1 g 1 ˙ t 1 g 1 x y . ␥ that may get chosen in the theorem are localized in the smaller square This is essentially a consequence of the fact that the Finally, when g ʦ C ␣ we will need: proper frequencies of the two drums, for ␦ ϭ 0, are all irrational with respect to each other, and therefore eigenfunctions are il il Ͼ ʈ Ϫ ilʈ ϱ 3 ʈ ʈ ␣ ␦ Lemma 4. Let J 0 be given. If g˜n g L (BR(z)) n 0 with g˜n C perfectly localized on each drum. For small enough, a pertur- uniformly bounded, then for j Ͻ J bation argument shows that the eigenfunctions will be essentially localized on each drum. But the eigenfunctions localized on the ʈ Ϫ ʈ ϱ 3 j ˜j,n L ͑ ͑ ͒͒ n0, 2 ϱ BR z small drum, being normalized to L norm, will have L norm as Ϫd/2 ␥ large as Rz , and therefore the lower bound for the i’s is sharp. ʈٌ͑ Ϫ ͒ʈ ϱ͑ ͑ ͒͒ 3 j ˜j,n L BR z n 0, ͉ Ϫ ˜ ͉ 3 ACKNOWLEDGMENTS. We thank K. Burdzy, R. R. Coifman, P. Gressman, H. j j,n n0. Smith, and A. D. Szlam for useful discussions during the preparation of the manuscript, as well as IPAM for hosting these discussions and more. P.W.J., ϭ Ϫ2 To conclude the proof of the theorem, let J c5Rz , depending M.M., and R.S. are grateful for partial support from, respectively, NSF DMS ␣ .on d, (1/2)cmin,2cmax, ʈgʈ␣ٙ1, ␣ ٙ 1. We may approximate g in C 0501300, NSF DMS 0650413 and ONR N0001407-1-0625, NSF DMS 0502747
1. Coifman RR, Lafon S, Lee AB, Maggioni M, Nadler B, Warner F, Zucker SW (2005) 10. Niyogi P, Matveeva I, Belkin M (2003) Regression and Regularization on Large Graphs Geometric diffusions as a tool for harmonic analysis and structure definition of data: (Univ of Chicago, Chicago), Tech Rep. Diffusion maps. Proc Natl Acad Sci USA 102:7426–7431. 11. Szlam AD, Maggioni M, Coifman RR (2006) A General Framework for Adaptive 2. Jones PW, Maggioni M, Schul R (2007) Universal local manifold parametrizations Regularization Based on Diffusion Processes on Graphs (Dept of Computer Science, via heat kernels and eigenfunctions of the Laplacian, http://arxiv.org/abs/ Yale Univ, New Haven, CT), Tech Rep 1365. 0709.1975. 12. Mahadevan S, Maggioni M (2007) Proto-value functions: A spectral framework for 3. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embed- solving Markov decision processes. J Mach Learn Res 8:2169–2231. ding. Science 290:2323–2326. 13. Szlam AD, Maggioni M, Coifman RR, Bremer JC, Jr (2005) Diffusion-driven multiscale 4. Belkin M, Niyogi P (2003) Using manifold structure for partially labelled classification. analysis on manifolds and graphs: Top-down and bottom-up constructions. Proc SPIE Adv NIPS 15. 5914:59141D. 5. Coifman RR, Lafon S (2006) Diffusion maps. Appl Comput Harm Anal 21:5–30. 14. Das P, Moll M, Stamati H, Kavraki LE, Clementi C (2006) Low-dimensional, free-energy 6. Tenenbaum JB, de Silva V, Langford JC (2000) A global geometric framework for landscapes of protein-folding reactions by nonlinear dimensionality reduction. Proc nonlinear dimensionality reduction. Science 290:2319–2323. Natl Acad Sci USA 103:9885–9890. 7. Zhang Z, Zha H (2002) Principal Manifolds and Nonlinear Dimension Reduction via 15. Be´rard P, Besson G, Gallot S (1994) Embedding Riemannian manifolds by their heat Local Tangent Space Alignment (Dept of Computer Science, Pennsylvania State Univ, kernel. Geom Funct Anal 4:374–398. University Park, PA), Tech Rep CSE-02-019. 16. Netrusov Y, Safarov Y (2005) Weyl asymptotic formula for the Laplacian on domains 8. Donoho DL, Grimes C (2003) Hessian eigenmaps: New locally linear embedding tech- with rough boundaries. Commun Math Phys 253:481–509. niques for high-dimensional data. Proc Natl Acad Sci USA 100:5591– 5596. 17. Hempel R, Seco L, Simon B (1991) The essential spectrum of Neumann laplacians on 9. Donoho DL, Grimes C (2002) When Does Isomap Recover Natural Parameterization of some bounded singular domains. J Funct Anal 102:448–483. Families of Articulated Images? (Dept of Statistics, Stanford Univ, Stanford, CA), Tech 18. Gru¨ter M, Widman K-O (1982) The Green function for uniformly elliptic equations. Man Rep 2002-27. Math 37:303–342.
1808 ͉ www.pnas.org͞cgi͞doi͞10.1073͞pnas.0710175104 Jones et al. Downloaded by guest on September 26, 2021