Tohoku Math. Journ. 30 (1978), 561-573.

INDEX OF SOME GAUSS-CRITICAL SUBMANIFOLDS

YOSIO MUTO

(Received May 13, 1977, revised February 7, 1978)

Introduction. As is well-known, the Gauss map is an excellent device in classical where curves and surfaces in a Euclidean three-space are studied. The same is true when the Gauss map is applied to an m-dimensional submanif old in a Euclidean n-space. In this case the image lies in the Grassmann manifold G(m, n -- m) which is not a sphere nor a projective space if m satisfies 1 < m < n --1. Let M be an m-dimensional compact orientable C°° submanif old in a Euclidean n-space En such that the Gauss map F: M -~ G(m, n -- m) is regular. We consider only the case 1 < m < n -1, for, if m = n --1, then F maps every closed hypersurf ace onto the (n -1)-sphere, whereas, if m =1, a simpler method may be available. Nevertheless, one of the motives of the present study lies in the fact: Let C be a closed curve with positive in a Euclidean three- space. Then the Gauss image of C in the standard sphere has the least length when and only when C lies in a plane. Assuming the standard Riemannian metric on G(n, n - m), we get a volume form on F(M). From the pull back of this volume form to M we get an integral Vol*(F(M)). As the Gauss image 1'(M) is immersed in the Grassmann manifold, Vol*(I'(M)) is not always the volume of F(M). When M moves smoothly in En, Vol* (F(M)) moves in R. Thus we can consider a submanifold Mo which is a critical point of Vol*(1,(M)). Mo is called a Gauss-critical submanif old and is denoted by GCS. As it is always the case with critical points, there arises the problem of finding the index. The purpose of the present paper is to prove that a submanif old M which lies in a linear subspace Em' as a closed hypersurf ace with positive is a GCS whose index is zero. A theorem related to this result has been obtained by Chern and Lashof [1], namely, THEOREM OF CHERN AND LASHOF. Let is M - E7 be an immersion of an m-dimensional compact manifold M into a Euclidean n-space En. Then the total absolute curvature z(M, i, En) is equal to 2 if and only 562 Y. MUTO if the immersion i is an imbedding and iM is a convex hypersurface in a linear subspace Em+1 of E~. See also N. H. Kuiper [2] for related topics. In the integral Vol*(T(M)) and also in the total absolute curvature the second fundamental form plays an essential role, but the relation between them is difficult to find. An obvious difference is that Vol*(T(M)) is defined only when the Gauss map T is regular in the present study. In § 1 we recall the Gauss map together with some symbols used in the present paper. Vol*(T(M)) is defined. In §2 a general formula of the second variation of Vol*(T(M)) is given. In § 3 we consider an infinitesimal deformation of a submanif old of En starting from a closed hypersurf ace Mo of Em+1. In § 4 the formula of the second variation is obtained. In § 5 an integral inequality on a closed hypersurf ace of Em+1 with positive second fundamental form is obtained and with the use of this inequality the Main Theorem (Theorem 5.6) is proved. 1. Gauss map and Gauss-critical submanif olds. Let (M, g) be an m-dimensional closed C~ submanif old of En, where g is the Riemannian metric induced by immersion. Let x'z(h, i, j, ... = 1, •.., n) be the rectangular coordinates in En and yFc(,c,~, p, ... =1, ., m) the local coordinates in any coordinate neighborhood of M such that the immersion M -~ En is given locally by C°° functions xla = x' (y1 ... ym) M is assumed to be covered by a set of such coordinate neighborhoods. We define (1.1)

Then gut are the covariant components of the Riemannian metric g. The contravariant components are g~a which satisfy gg1a = (a, ,9, 7, ... = 1, ..., m). We use them for lowering and raising the indices in Greek letters. Let be the Christoffel symbols derived from gut and V2 the symbol of Van der Waerden-Bortolotti's covariant differentiation. Then

(1.2) where a2xh = ax'/aye, is the second fundamental tensor. We define (1.3) GAUSS-CRITICAL SUBMANIFOLDS 563

Then G, is non-negative and we can define an m-form w on M whose local expression is

(1.4) w = [det (G ~)]"2dy1 A ... A dym . From w we define

(1.5)

Now the Grassmann manifold G(rn, n - m) is the space of m-planes 11 of EL passing the origin and the Gauss map F carries a point p of M into an element 11 of G(rn, n - m) which is an rn-plane parallel to Mp. On the other hand the Grassmann manifold bears the standard Riemannian metric g of K. Leichtweiss [3], [5] and the ratio of any line element in (M, g) to its Gauss image is given by [gu2dy~dy2]"2: [G 2dy"dy']1~2 Thus, if F is regular, namely det (Guy) > 0, vo is the volume form of the Gauss image [4]. But, as F is an immersion, 7 differs in general from Vol(F(M)) and is denoted by Vol*(F(M)). Let Mo be one of such submanif olds and {M(t), t e R} a set of m- dimensional closed submanif olds such that M(0) = Mo. Moreover, we assume that, if I = [- E, s] for some > 0, there exists a C°° map Q: Mo x I --} E~ where cp(M,, t) = M(t) for t e I. For sufficiently small ~, F is regular for all M(t), -~ < t ~, and we put 7(t) = Vol*(F(M(t))). If t = 0 is a critical point of 7(t) for every such set {M(t), t e I}, we say that Mo is a Gauss-critical submanif old and denote it by GCS. The equation of GCS was obtained in [4]. A submanif old of En which lies in a subspace Em+l of En as a closed hypersurf ace with positive second fundamental form is a GCS. The purpose of the present paper is to prove that the index is zero for such a submanif old. 2. Second variation of Vol*(F(M)). If v is the volume form of (M, g), we get from (1.4) and (1.5)

(2.1)

for V of*(F(M(t))). As, in the second member, GF~,g12, and v depend on t, in order to get the derivatives d7/dt and d27/dt2 we prefer for the present the following expression,

(2.2) 564 Y. MUTO where dy = [det(g2)]1'2v~-does not depend on t. Notice that we take coordinate neighborhoods such that p e Mo and cp(p, t) e M(t) have always the same local coordinates and the immersion M(t) -+ En is expressed locally by (2.3) ~h = xh(y1f ..., ym; t) From (2.2) we get

(2.4)

(2.5)

Let us define (G-1)'`2 by

( -1)fia f Then we have

(2.6)

(2.7)

If we use vector fieldsh, ~1'tdefined by

we get from (1.1) and (1.2) GAUSS-CRITICAL SUBMANIFOLDS 565 where Bt = g2 B. We also obtain from (1.3)

(2.8)

(2.9)

3. A submanif old Mo of En which lies in Em+1 as a closed hyper- surface with positive second fundamental form. Now let us consider the case where M(0) = Mo is imbedded in En as a closed hypersurf ace with positive second fundamental form in a subspace Em+1 of En. As M(t), t ~ 0, need not be confined to that Em', the vector fields s&Jand 'l h are arbitrary for t = 0. Let C (X = m + 2, ..., n) be orthonormal normal vectors of Em' which are constant vectors. Let us take a rec- tangular coordinate system of En such that the Em' under consideration is given by xm+2= ... = xn = 0 and the components satisfy = 0. We also obtain

(3.1) for Ma where NIt is the unit normal vector field lying in Em+1 and h,,, the second fundamental form of Mo as a hypersurf ace of Em+1. 566 Y. MUTO

We also get

(3.2) ©uNh = - h~aBa, (3.3) = a~hv~ and the equation of Gauss, (3.4) = h~~hvh-- hL~h With the use of these equations we want to get formulas for d7/dt and d27/dt2 at t = 0. In the following calculation all quantities are to be evaluated at t = 0. Then we get

(3.5)

(3.6)

We also get from (1.3) and (3.1)

(3.7) G~a = , (G-')~`2 _ where k~~ are defined by kah~a = ~. In order to apply Green's theorem in the subsequent calculation we rewrite (2.4) and (2.5) in the following form where v is the volume form of M0,

(3.8)

(3.9)

From (3.7) we get [det (G~,)/det (g)]112 = det (h12) . Hence, in view of (2.6), (2.8), and (3.1), we can calculate the integrand A of (3.8) as follows, GAUSS-CRITICAL SUBMANIFOLDS 567

But the first term in the last member vanishes on integration by Green's theorem. We use the symbol if two members are equal except such a divergence term, hence

On the other hand we have ©~[det (hu~)k~aNh] = det (h~~)[(k~JV~hyIckP6 - k~Jka~v~hy~)Nh+ k~'V~N"] .

Hence we get A 0 because of (3.2) and (3.3). This proves d7/dt = 0 at t = 0 and the following theorem [4]. THEOREM 3.1. A C~ submanif old Mo of En which lies in a linear subspace Em+1 as a closed hypersurface with positive second fundamental form is a Gauss-critical submanif old. REMARK. Our assumption is that the Gauss map of M is regular. Then it is an inevitable consequence that we consider only the case of positive second fundamental form. As a consequence there exists the matrix (k~2).

4. The second derivative (d27/dt2)o when Mo is a hypersurf ace of an Em' with positive second fundamental form. We have proved in 3 that (d7/dt)o vanishes for the submanif old Mo under consideration, and this implies that the terms of B involving ih do not contribute to the integral (d271dt2)0. In order to get an expression for (d27/dt2)o which is convenient for us in finding the index, we need some lengthy calculation as is predictable from (2.7), (2.8), (2.9). We first obtain after some straightforward calculation 568 Y. MUT 0

Substituting these formulas into (2.7) we obtain

(4.1)

If we put

(4.2) GAUSS-CRITICAL SUBMANIFOLDS 569 where C are as defined in 3, we get

hence B is reduced to

(4.3) wherE

(4.4)

(4.5)

(4.6)

Now in order to prove

(4.7) we first prove

(4.8)

The next calculation resembles that carried out in 3, but is a little more complicated: we get 570 Y. MUTO

and into the second member we substitute

which is the result of the Ricci identity and the Gauss equation. Again we get

hence

As we have

in view of (3.2), (3.3), we get (4.8). Now we can reduce Bl + B2 to a formula where only the first derivatives of s h are contained: we get from (4.5) and (4.8) GAUSS-CRITICAL SUBMANIFOLDS 571

Substituting this result and (4.6) into Bl + B2 + B3 we get

and this is equivalent to (4.7). From (4.3) and (4.7) we get the following lemma. LEMMA 4.1. Let Mo be a C°° submanifold of En lying in a linear subspace Em' as a closed hypersurface with positive second fundamental form. It we consider an infinitesimal deformation of such a submanifold Mo in En, we get for the second derivative of 7 defined in § 1

where v is the volume form of (M0, g), h~2 is the second fundamental form of Mo as a by persurf ace of Em', k' ' is given by kuah2a = ~,

C being orthonormal normal vectors of Em+l which are constant vectors and h is the vector field of deformation. 5. Index of a Gauss-critical submanif old Mo. We first prove the following theorem. THEOREM 5.1. If (M, g) is a C°° closed hypersurface of Em' with the positive second fundamental form h,2, then the following inequality holds for any C°° function f :

(5.1)

where v is the volume form of (M, g) and kf`' satisfies kI`ah,a = ~j. To prove Theorem 5.1 we need some lemmas. LEMMA 5.2. k~ak°Pa QaaP is non-negative for any tensor a~1. PROOF. As h,12 is positive, k'`2 is also positive and, if we fix any point, we can put k~a = k~~Pa where k~ > 0 for ,9 = 1, • • • , m. Hence we get k~ak°papaaap= LJ~,a k~ka(a~gQ)2> 0. LEMMA 5.3. 572 Y. MUTO

PROOF. We have in view of Lemma 5.2

namely, mk~~k°PV~V Qf V«V pf - (kaV sVof )2 0. Then this lemma is immediately obtained. LEMMA 5.4.

PROOF. Applying Green's theorem we get

As we have Vp(det (hft~)k~ak°p)VsV af - V~(det (h,2)k~~k°p)VpV Q f = Vp(det (h,')(ksakcp - kp~k°'))V~V Qf = 0 , GAUSS-CRITICAL SUBMANIFOLDS 573 we get Lemma 5.4.

As we assume ma > 1, hence m - 1 > 0, we get from Lemma 5.3 and Lemma 5.4

(5,2)

LEMMA 5.5.

PROOF. If S `' = g' ~g''S,3,, we have (St9 - S,, )(SPA Ss) > 0 1 hence

S~~Sa - > 0 . Lemma 5.5 is proved if we put S~Q = k91V,3Vaf . From Lemma 5.5 and inequality (5.2) we get (5.1), hence Theorem 5.1 is proved. The following Main Theorem is a direct consequence of Theorem 5.1 together with Lemma 4.1. THEOREM 5.6. If an m-dimensional C~ submanif old M of En lies in a linear subspace Em+' as a closed hypersurface with the positive second fundamental form, then the index of M as a critical point of the integral Vol*(I'(M)) is zero.

REFERENCES

[1] S. S. CHERN AND R. K. LASHOF, On the of immersed manifolds, Amer. J. Math., 79 (1957), 306-318; II, Michigan Math. J., 5 (1958), 5-12. 2] N. H. KUIPER, Immersions with minimal absolute curvature, [Colloque de Geometrie Differentielle Globale, Bruxelles, 1958, 75-87. [3] K. LEICHTWEISS,Zur Riemannschen Geometric in Grassmannschen Mannigf altigkeiten, Math. Z., 76 (1961), 334-366. [4] Y. MUTO, The Gauss map of a submanifold in a Euclidean space, J. Math. Soc. Japan, 30 (1978), 85-100. [5] Y. C. WONG, Differential geometry of Grassmann manifolds, Proc. Nat. Acad. of Sciences, 57 (1967), 589-594. 2262-150, TOMIOKA-CHO,KANAZAWA-KU, YOKOHAMA236, JAPAN