Finiteness of Orbit Structure for Real Flag Manifolds

JOSEPH A. WOLF*

FINITENESS OF ORBIT STRUCTURE FOR REAL FLAG MANIFOLDS

ABSTRACT. Let G be a reductive real Lie group, a an involutive automorphism of (7, and L = G u the fixed point set of a. It is shown that G has only finitely many L-conjugacy classes of parabolic subgroups, so if P is a parabolic subgroup of G then there are only finitely many L-orbits on the real flag manifold G/P. This is done by showing that G has only finitely many L-conjugacy classes of a-stable Cartan subgroups. These results extend known facts for the case where G is a complex group and L is a real form of G.

Key words and phrases: flag manifold, reductive Lie group, semisimple Lie group, parabolic subgroup, Caftan subgroup, Cartan subalgebra. AMS Subject Classification (1970) Primary: 22-50, 22-70, 53-66 Secondary: 14-45, 57-47

1. INTRODUCTION

One knows [4] that there are only finitely many conjugacy classes of Cartan subalgebras in a reductive real Lie algebra. If G is a complex reductive Lie group and L is a real form, it follows [8] that there are only finitely many L-eonjugacy classes of parabolic subgroups of G. In particular, if X= G/P is a complex flag manifold of G, then [8] there are only finitely many L-orbits on X. Here we extend these results to the case where G is a reductive real Lie group and L is the fixed point set of an involutive automorphism, and we indicate the scope of applicability of the extension.

2. CONJUGACY OF CARTAN SUBALGEBRAS

If G is a Lie group then Go denotes its identity component, g denotes its Lie algebra, and Int (g) denotes the inner automorphism group {Ad (g):g e Go) of g.

THEOREM 1. Let g be a reductive real Lie algebra, a an involutive automorphism of g, and 1= g~ its fixed point set. Let L o denote the analytic subgroup oflnt (g ) for I. Then there are only finitely many Lo-conjugacy classes of a-stable Cartan subalgebras of g. Proof. It suffices to consider the case where g is semisimple and has no proper a-stable ideal. We now assume that, and we fix the notation g = I +m where rrt= {x~g:tr(x)= -x}.

* Research partially supported by NSF Grant GP-16651.

Case 1: Int(g) is compact. This case follows directly from Kostant's proof [4] that g* = I +ira has only finitely many conjugacy classes of Cartan subalgebras. Kostant shows that every Cartan subalgebra of g* is conjugate to a a-stable one and that g* has only finitely many Lo-conjugacy classes of a-stable Cartan subalgebras D*. Then ~)* ~ D = ([9* n I) + i(D* n ira) gives our finiteness assertion.

Case 2: Int(g) is complex and a is conjugate-linear. Then I is a real form of g, and our assertion is Kostant's result [4] on finiteness of the number of conjugacy classes of Cartan subalgebras of I.

Case 3: Int (g) is complex and a is complex-linear. This is the nontrivial case. Fix a Cartan involution 8 of g that commutes with a. Its fixed point set go is a a-stable compact real form of g, so go=iO+m 0 where 1°=Ing° and m °=mc~g°. Let Lg denote the analytic subgroup of Lo for [o. Case 1 tells us that gO has only finitely many Lo°-conjugacy classes of a-stable Caftan subalgebras. Since gO is a real form of g, this says that g has only finitely many Lo°-conjugacy classes of 0-stable a-stable Cartan subalgebras. L~-conjugacy implies Lo- conjugacy. Now we need only show that every Lo-conjugacy class of a-stable Caftan subalgebras of g contains a 0-stable algebra. Let ~ be an Lo-conjugacy class of a-stable Cartan subalgebras of g. We first consider the situation if D~¢' then D:m i.e. [)c~I=0. Fix t)~ and let [)~ be its real form spanned by the roots. If ~b is an b-root then a*q~= -~b because Dcrrt. Thus g has a a-stable compact real form g' with i D# = g'. Let 0' denote the Cartan involution of g that is complex conjugation over g'. Evidently 0' (t))= t). Thus, to exhibit a 8-stable element of it suffices to prove that 0 and 0' are L0-conjugate, and for this we need only check that g'= gO' is Lo-conjugate to gO. As g' is a-stable, I °' = I c~ g' is a compact real form of I, hence is Lo-conjugate to I °. Thus we may assume I°'= I °. That assumption made, g'= = I ° +m °' just as gO= i0 +m o. Since I ° is a real form of I, I°c 0 °r and 00' is complex linear, we now have I ~ goo'. (=8 '-1.8 is a complex linear automorphism of g and trivial on [, so (Ira commutes with every element of Loire. If ~1,~= + 1 then 0 = + 0' on m, so m °' = rrt e or m °" =im °. If ra °' = m ° then g' = go as desired. If m °" = ira ° then g'= I ° +ira °, which is noncompact, contradicting the fact that 8' is a Cartan involution. Now we assume ~1,,~ + 1. g is simple because it has no proper ORBIT STRUCTURE FOR REAL FLAG MANIFOLDS 379 a-stable ideal, so now (go, p) is an irreducible hermitian symmetric pair, and re=m+ +m_ direct sum of I-invariant I-irreducible subspaces that are interchanged by 0 and also by 8'. Thus we have a complex number g with

~1~+=~ and ~l~.=~. Let x± era± and compute

x+ +x_ = 02(x+ +x_) = (o'¢)'(x+ + x_) = O'~(O'~x_ + O'~x+) = +

Thus ~ = 1 and ~ = Ad (z) where z is central in L~. Now Om°" = O' Ad (z) m °' = • = 0'11l °" = 1110', SO

m e' = (m"" n m °) + (m °' c~ ira").

The second summand vanishes because g' is compact. We conclude g'= ga. In summary: if JY is an Lo-conjugacy class of a-stable Caftan subalgebras of g, and if be.~ implies I~=m, we have shown that ~Y' contains a P-stable algebra. Now let .Yg be any Lo-conjugacy class of a-stable Cartan subalgebras of g. Let ~)~ ~¢g and split [~ = (b n I)+ (I~ nm). Passing to the derived algebra of the g-centralizer of ~ n I, we reduce to the case just considered, obtaining a P-stable Lo-conjugate of ~). This completes the argument for Case 3.

Case 4: the general case. Let n=dimg, r=rank g and N=(n) - 1. Use Pliicker coordinates to view the Grassmannian of r-dimensional subspaces of gc as a subvariety of the complex projective space PN(C). Let Loc denote the analytic subgroup of Int(g c) with Lie algebra I c. Now every LC-eon - jugacy class of Cartan subalgebras of gc is an LC-orbit on pN (C) under the rational representation A' (Adln t (gc)lLc). If O is such an orbit, then [6, Lemma 1.1 ] ~ c~ P N(R) is a finite union of Lo-orbits. Case 4 now follows from Cases 2 and 3. Q.E.D.

3. CONJUGACY OF PARABOLIC SUBGROUPS

If G is a connected complex semisimple Lie group with Lie algebra g, and if P is a complex Lie subgroup with Lie algebra p, then one has equivalent conditions (i) G/P is compact, (ii) G/P is a compact simply connected kaehler manifold, 380 JOSEPH A. WOLF

(iii) G/P is a complex projective variety, (iv) G/P is a closed G-orbit in a projective representation. Under those circumstances, P is a parabolic subgroup of G, p is a parabolic subalgebra of g, and one can see that P= {#~G:Ad(g) p =p}. Let G be a complex Lie group, g its Lie algebra, ~ the solvable radical of fl, and n:g ~ fl/~ the projection. The parabolic subalgebras of g are the n -1 (q) where q is a parabolic subalgebra of g[~. The parabolic subgroups of G are the normalizers P= {g~G:Ad(g) p =p} where p is a parabolic subalgebra of g; then p is the Lie algebra of P. Let G be a real Lie group and fl its Lie algebra. Then parabolic subalgebra of g means a subalgebra p = g c~ q where q is a parabolic subalgebra of gc stable under complex conjugation over ft. The parabolic subgroups of G are the normalizers P = {g ~ G: Ad (g) p = p } where p is a parabolic subalgebra of fl, and then P has Lie algebra p.

THEOREM 2. Let G be a real Lie group, tr an involutive automorphism of G, and L= G° its fixed point set. Then there are only finitely many Lo-conjugacy classes of parabolic subgroups of G. Proof. We will show that g has only finitely many Lo-eonjugacy classes of parabolic subalgebras. For this we may assume G connected and simply connected. Now G has ¢-stable Levi decomposition GI" S where S is the solvable radical and we may replace G by its semisimple quotient G/S ~- Gt. Thus we may assume g semisimple. Let p be a parabolic subalgebra of g. Then p c~ a (p) is a-stable and contains a Cartan subalgebra of g, so it contains a a-stable Cartan subalgebra D of g. There are only finitely many parabolic subalgebras of g containing any given Caftan subalgebra, and Theorem 1 says that there are only finitely many possibilities for D up to Lo-conjugacy. Thus there are only finitely many possibilities for p up to L0-conjugacy. Q.E.D.

4. APPLICATION TO REAL FLAG MANIFOLDS

By real flag manifold we mean a homogeneous space X= G/P where G is a real Lie group and P is a parabolic subgroup. Since P is its own normalizer in G, we may view Xas the set of all G-conjugates of P under x~ {g e G: g (x) = = x}. If S is a subgroup of G, then the orbit S(x) corresponds to the S-conjugacy class of {g~G:g(x)=x}. Now Theorem 2 gives us

THEOREM 3. Let G be a real Lie group, P a parabolic subgroup, and X=G/P the corresponding real flag manifold. Let ~ be an involutive automorphism of G and L= G¢ its fixed point set. Then there are only finitely many ORBIT STRUCTURE FOR REAL FLAG MANIFOLDS 381

Lo-orbits on X. In particular, L and Lo each has both open and closed orbits on X, and each L.orbit is a finite union of Lo-orbits. A trivial consequence: X has only finitely many topological components. Here note that X is connected if and only if every G-conjugate of P is Go- conjugate to P. In the complex case (G is a complex Lie group, P is a complex parabolic subgroup and L is a real form of G) there is just one dosed Lo-orbit on every topological component of X= G/P [8, Theorem 3.3] Here is an example in the real case where there are two closed orbits. Let G= SL (2; R) and

a_ 1 :a, beR with aGO so that X= G/P is the circle R u {~ } bounding the upper half plane, the action being

: xt-~ --. cx+d

Let tr be conjugation by ('0 then L=G ~ is the Cartan subgroup

{(0 a0t) : 0#a~R l with Lo given by a>0. As (0 a-10) : x~-~a2x,

L and Lo have the same orbits on X, which are two dosed orbits {0} and {~}, and two open orbits {xeR:x>O} and {xeR:x

5. APPLICATION TO SYMMETRIC R-SPACES

A symmetric R-space is a compact Riemannian symmetric space X=K/V which admits a connected effective Lie transformation group G such that (i) K= G and (ii) if Y is a G-stable Riemannian factor of ,Y then G acts on Y as a noncompact group. Then [5] G is semisimple, K is a maximal corn- 382 JOSEPH A. WOLF pact subgroup of G, and, up to finite covering, X is a real flag manifold G/P. It often happens that G contains an interesting noneompact form L of K.

EXAMPLE: Grassmann Manifolds. Let F denote the field of real, complex or quaternion numbers. If n>0 then U(n; F) denotes its unitary group, which is the orthogonal group O(n), the complex unitary group U(n) or the unitary symplectic group Sp(n). The grassmannian of k-planes in F k+~ is the compact Riemannian symmetric space

X = K/V = U(k + l; F)/U(k; F) x U(l; F).

Under the action of the general linear group G=GL(k+I; F) it is a real flag G]P where

P= { g~G:g hasmatrix /*l*'fl\~[,jj.

Let L be the F-unitary group of any of the nondegenerate F-Hermitian forms h(x, y)= (xlYl +'" +Xp~p)- (xp+ lyp+ ~ +-.. +xp+q~p+~) where p +q=k+l. Then Witt's Theorem (over F) gives us the L-orbit structure of X.

EXAMPLE: Hermitian Symmetric Spaces. Here X=K/V is an Hermitian symmetric space of compact type and G = K c. The dual bounded symmetric domain D = K*/V where K* is a certain noncompact real form of G, and the Borel embedding realizes D as a certain open K*-orbit on X. One knows the K*-orbit structure of X ([7-9]). Let L be an arbitrary real form of G.

EXAMPLE: ~ilov Boundaries of Tube Domains. Let X= K/V be an hermitian symmetric space of compact type and D = K*/V the dual bounded symmetric domain. Suppose that D is holomorphically equivalent to a tube domain. Then its Bergman-~ilov boundary is a symmetric R-space S= VIE= K*]W; see [3]. Let L be an isotropy subgroup of K* on an open K*-orbit in X. If one expresses the open orbit as K* (c~x0) where x0 is the base point and ez is a partial Cayley transform then L = K* n Ad (c~) V c, a possibly-noncompact form of V. These examples all fit into the following pattern, and the result allows us to apply Theorem 3 to symmetric R-spaces. Here note that any reductive subalgebra of g is invariant under a Cartan involution.

THEOREM 4. Let fl be a semisimple Lie algebra, 0 a Cartan involution, and g =~ + s the eigenspace decomposition under O, so that ~ = go is the maximal compactly embedded subalgebra. Suppose that g has no proper O-stable ideal, ORBIT STRUCTURE FOR REAL FLAG MANIFOLDS 383 i.e. that (g, [) is an irreducible Riemannian symmetric pair of noncompact type. Let I be a O-stable subalgebra of g such that I c'~[c. (1) There is a complex linear automorphism ~ of g c that preserves the compact form ~ + is and sends I c to ~c. (2) g is stable under the involutive automorphism a=~-10~, and I=g ~. Remark. I suspect that Theorem 4 is true for g reductive and (g, ~) assumed irreducible. But this does not affect the application to symmetric R-spaces. Proof. We first check that (1) implies (2). Since ~(Ic)=[ c, the fixed point set (gc),=~-l.(gc)o=~-l(~c)=[c which is stable under 0. Thus 0 and a commute. Let m c denote the (- 1)-eigenspace of a, so gc = ic +m c under a. Then gc = ([c c~ [) + ([c c~ is) + (lrt c n [) + (rrt c c~ i~) + + (I c nil) + (ic c~ ~) + (m c niI) + (m c c~ is).

Each of these eight summands is o-stable because a preserves ~c, sc and ~+i~. That gives a-stability of g=(ICc~)+(ICc~s)+(mCc~)+(mCc~s), and now go = g c~ (gc)~ = g c~ [c = I. We prove (1). Let g*=t+is and I*=(Ic~l)+i(Ins). If g is not absolutely simple, then g* ~ ~@l with ~ embedded diagonally and 0 interchanging the two summands by 0 (x, y) = (y, x). Let vj: g* ~ Ij denote projection to the jth summand. I* is in neither summand because 0I*= I*, so vj:I* ~_~1. That gives p~Aut (~) with I* = {(x,/~x):xe~). Notice that/~ 2 = 1 because I*--0I*= {(BY, Y):Y~). Now ~= 1 x/~ is an automorphism of g* that interchanges I* and the diagonally embedded t. If g is absolutely simple and rank ~=rank ~, then g* is simple and its subalgebra ~ is a maximal subalgebra of maximal rank. The Borel-de Sieben- thal classification [1 ] shows that any two subalgebras of g* isomorphic to must be Aut(g*)-conjugate. Now suppose g absolutely simple with rank t < rank ft. If g* is of classical type then the inclusion ~ ~ g* is one of

so (n) ~ stt (n), sp (n) ~ stt (2n), so (p) • so (q) ~ so (p + q).

Looking at degrees of representations we see that [ q g* and l* ~g* are equivalent representations. If ~/is the equivalence we take a=r/[ 0," If g* is of exceptional type then [ ~ g* is one of

sp(4)~e6 or f4~e6

and Dynkin's results [2] show that any two subalgebras of g* isomorphic to must be Aut(g*)-conjugate. Q.E.D. 384 JOSEPH A. WOLF

BIBLIOGRAPHY

1. Borel, A. and de Siebenthal, J,: 'Les sous-groupes ferm6s de rang maximum des groupes de Lie dos', Comm. Math. Helv. 23 (1949), 200-221. 2. Dynkin, E. B.: 'Semisimple Subalgebras of Semisimple Lie Algebras', Math. Sb. 30 (72), 349--462; also in Am. Math. Soc. Transl. Set. 2, 6, 111-244. 3. KoHmyi, A. and Wolf, J. A.: 'Realization of Hermitian Symmetric Spaces as Gener- alized Half-Planes', Ann. Math. 81 (1965), 265-288. 4. Kostant, B.: 'On the Conjugacy of Real Cartan Subalgebras', Proc. Nat. Acad. Sci. U.S.A. 41 (1955), 967-970. 5. Nagano, T.: 'Transformation Groups on Compact Symmetric Spaces', Trans. Am. Math. Soc. 118 (1965), 428-453. 6. Richardson, R. W.: 'Principal Orbit Types for Real-Analytic Transformation Groups', Am. J. Math. 95 (1973), 193-203. 7. Takeuchi, M.: 'On Orbits in a Compact Hermitian Symmetric Space', Am. J. Math. 90 (1968), 657-680. 8. Wolf, J. A.: 'The Action of a Real Semisimple Group on a Complex Flag Manifold, I: Orbit Structure and Holomorphic Arc Components', Bull. Am. Math. Soc. 75 (1969), 1121-1237. 9. Wolf, J. A.: 'Fine Structure of Hermitian Symmetric Spaces', in Symmetric Spaces: Short Courses Presented at Washington University (ed. by Boothby and Weiss), Mareell Dekker, New York, 1972.

Author's address: Joseph A. Wolf, University of California, Berkeley, Calif. 94720, U.S.A.

(Received May 3, 1974)