J. Math. Soc. Japan Vol. 18, No. 1, 1966

On some double coset decompositions of complex semi-simple Lie groups

By Kazuhiko AOMOTO

(Received March 6, 1965)

It is well known that a homogeneous space X of a complex semi-simple Lie group GC by a Borel subgroup B is a compact complex Kahler manifold. A compact form Gu of GC also operates transitively on X and so X can be considered as a homogeneous space of Gu. This manifold was studied pro- foundly by Borel [1], Borel-Hirzebruch [2], Bott [3], and by Kostant [15].. This manifold is intimately related to the theory of finite dimensional repre- sentations of Gu (or of GC). On the other hand, Gelf and and Graev [11] paid attention to that since it is also very important for the theory of unitary representations of a non compact real form G of GC which is a real semi simple Lie group. They studied the orbits of G in X when G is of AI-type (or of AIII-type) and, in fact, they showed that one can construct some new irreducible unitary representations of G, so-called "discrete series" in their sense, which are realized on some suitable spaces of functions on these orbits [12]. In this paper we shall show that, in general, the space of the double cosets B\GC'/G is finite, i. e. the number of the G-orbits in X is finite. In several examples, we shall calculate the number of G-orbits. In particular,, open orbits in X seem very interesting. These are complex manifolds on which G operates holomorphically, and can be considered as homogeneous spaces of G by its Cartan subgroups. It seems to be interesting to determine whether they are Stein manifolds or not, and further to study the structure of their cohomologies, because these properties have a close connection with representations of G. The methodd which I have made use of is essentially the same as that of Suldin in [12]. In § 1, we first prove that the subgroup of all elements are fixed by an. anti-linear involutive automorphism of a connected, simply connected complex. semi-simple group, which derives easily from the well known Cartan's theoremm on symmetric spaces. The decomposition theorem of Bruhat-Chevalley deduces. the theorem (Theorem 1) Let GC be a connected complex semi-simple groin, o be an anti-linear involutive automorphism of GC, b be the set of all elements g of G which satisfy 2 K. AOMOTo

Q(g)=g+1. For an arbitrary Borel subgroup of Ge, there exists a finite subset ?P' of such that any element g of P can be written g= b • g1 • a(b)1, where b is in B, g1 is an element of ¶. This is a generalization of Suldin's result. Let o be the set of all elements of the form g . Q(g)1 where g is an arbitrary element in Gc. Ob- viously, Po is contained in 0. But it is not obvious if ~o coincides with & In 2 we show that, in general, ~o does iot coincides with P and this problem has a close relation with the classification of real semi-simple Lie groups. For the real form G which Q determines, the double coset B\GC/G is finite. This is the main result of § 3. G operates on the flag manifold B\G We show that open G-orbits in this manifold are all isomorphic with A\G where A is a fundamental Cartan subgroup of G. It is known that A\G has a complex structure, and moreover it has a canonical holomorphicimbedding in this way. This is a generalization of Harish-Chandra's theory (see [9]). The double coset space B\GC/G has a close connection with the conjugate classes of Cartan subgroups of A. This approach was obtained by H. Matsumoto (unpublished). In § 4, we show several examples when GC is of simple classical type and shall count the number of double cosets Professor N. Iwahori has kindly given me many useful advices. In par- ticular Lemma 3' which is a generalization of Chevalley's theorem was com- municated to me by him. Owing to this lemma, the proof of Theorem 1 has become much simpler and I have been able to formulate it in more general situation than I have expected. I must be grateful to Mr. T. Yokonuma and Mr. M. Takeuchi who have helped me in many points.

§1. Let g be a non-compact real semi-simple Lie algebra, c be its complexi- fication, 6 be an involutive semi-linear automorphism of gCdefined by 6(X+V'-1 Y) = X-~/-1 Y , X, Y g . There exists a compact real form gu of gCwhich is invariant with respect to o. The restriction of Q to gu is an involutive automorphism of g,, denoted by o a =6• :Let B be an involutive semi-linear automorphism of gCdefined by B(X+/ii Y)=X--/--1 Y, X, Y~gu. Then 8 transforms g into itself. 6 = 6 I g is an involutive automorphism of g. and gu are decomposed in the following way : Double coset decompositions 3

~u=+4V/-1, where = q n 1u, p = g n V-1. It is a Cartan decomposition. Let GC be a connected, complex semi-simple Lie group with the Lie algebra gC. We assume i can be extended to an automorphism of GC. We define a subgroup G of GC : G= {g~ GC ; d(g)=g}. Obviously the Lie algebra of G is g. G may not be connected. If GC is simply connected, o can be extended uniquely to an automorphism of GC. NOTATIONS. Let g be the Lie algebra of a Lie group G, and 15 be a subalgebra of g, then there exists a unique analytic subgroup H of G corre- sponding to 1. It is denoted by Hc- in G or simply by Hclj. We shall frequently make use of this notation. LEMMA1. (Cartan) Let Gu be a simply connected compact semi-simple group, cu be its Lie algebra. We consider an involution i on c corresponding to a non-compact real semi-simple algebra. Then gu is decomposed as follows : where ={X9;a(X)=X}, p={XEgu;a(X)=-X} U is extended uniquely to an automorphism of Gu. Let K~ in G. Then K= {x Gu; a(x) = x}, that is, the subgroup of 6-fixed elements is connected. For the proof see Helgason [13], p. 272 Theorem 7.2. LEMMA2. Let GC be a simply connected complex semi-simple Lie group, Gu be its maximal compact subgroup, which is simply connected, as well known. Let GC~g C, Gu ~u, K~ as in Lemma 1. Then the subgroup G of Q-fixed elements of GC is connected. PROOF. It is well known that GC= Gu • exp (/-1c) (unique expression). The map exp : /1c-* exp ~/-i g is a diffeomorphism. Obviously Gun G K. Let g= u • exp X (u Gu, X '/-1c) be an element of G. Then Q(g) = Q(u) • Q(exp X) = u • exp X. On the other hand we have v(Gu) = Gu, 3(exp s-i gu)= exp /-1 gu. Therefore 6(u) = u, 6(exp X) = exp X. By Lemma 1, we have u K, a(X) = X and X n ~/-1 au=p. We have proved thus G C K • exp p. But K • exp p is connected and is contained in G. Therefore we obtain finally G = K • exp p. Let GC be a connected complex semi-simple group with GC-gC, and G2,. a maximal compact subgroup with Gu - gu. Then Gu is connected. Let T be a maximal torus of Gu with Tit. Then t is a Cartan subalgebra of g . Its complexification tC is a Cartan subalgebra of g . Let tCc AC and B be a Borel subgroup of GC containing AC. Let fl(T) be the normalizes of T in Gu. W = 12(T)/T is a finite group (Weyl group) and operates on t as linear trans- 4 K. AoMoTo formations via the adjoint representation. For each w E W, we choose once for all a representative element w in J2(T ), and denote it by w = w(w). Then one has the Bruhat decomposition: LEMMA 3. (1) GC = U B • w(w) • B (disjoint sum). w =W For the proof, see [4]. We need a more precise decomposition which is essentially the same as the Chevalley's [5]. Let V--1 t=I)R. We denote its dual (over R) by fjR, which is identified as the real part of the dual space (tC)* of tC (over C). We consid er two orderings 01 and 02 on I. Let d be roots of gC with respect to 15R, and P1i P2 be positive roots with respect to 01, 02, respectively. Then

(2) 9C= CEa+ CE_a+tC aEP1 aEP1 and (3) cC = CEa+ CE-a+t° aEP2 aEP2 where [H, EQ] = a(H)Ea, (HE tC, a E 4). Put nc = CEa , n2 = CE. aEP1 aEP2

For A E (tC)* and w E W we define w(A) E (tC)* by w(A) (H) = A(w-1(H)), H E tC . Then for any a E d, one has Adw • (CEa) = CEw(a) with w = w(w). We put P2(w) = {a E P2, w(a) E P1}

(4) P211(w)= {a E P2, w(a) E -P1} n2(w) = CE, n2 (w) = CEa, cEP2(w) cEP2'(w) then n2 = n2(w)+n2 (w) (direct sum), and n2(w), n2 (w) are subalgebras of n2 . Put b1 = tC+ni , b2 = tC +n2 . Let B1 c 1,1, B2 c b2, Ni c nC, N2 - n2 , Ne(w) r ne(w), N2 (w) i n~'(w). Then N2 =N2(w) • N2'(w) which is a unique factorization, i. e. N2(w)nN2'(w)= {1}. LEMMA 3'. GC = U B1 w(w)B2 (disjoint union) WEW (5) = U B1 • w(w)N2(w) (unique expression) WEW i. e. the map B1 XN2'(w)--~ B1w(w)N2 (w) defined by (b, n)-~ bw(w)n is bijective. Double coset decompositions 5

For the proof see [5]. Let g =-{-, gu =~~/=1 and o, o, a, o be defined as before. Let fj be a Cartan subalgebra which is invariant by 8. Then _ tj~+bp (direct sum), th _ t) _ )n P, bu = +/-1 ~ is a Cartan subalgebra of gu, 1R= +Llp and ~C= C is a Cartan subalgebra of gC. Let GC be a connected complex semi-simple Lie group in which d is defined and G be the subgroup of GC consisting of all Q-fixed elements. We remark that G need not be connected. We fix an order on fj~ and we denote the root system of gC with respect to ~jCby d, and positive roots by P. Thus gC= E CEa+ 2 CE-a+ LlC. aEP aEP Let ttC= CE,, b = ttC+ff , B (a Borel subgroup) ~ b, NC ttC, AC jjC, t)u ~ T aEP C Gu. We put U(C) = of, f = fjC~-ni. For any 2 (IC)* we define Q(2) by Q(2) (H) _ 2(Q(H)) for H Put P1= Q(P), then of = CE. By Lemma 3' aEPl we have : PROPOSITION 1. (6) GC= U B. w(w)Q(B) (disjoint union) WEW (7) = U B . w(w)Nf'(w) (unique expression), WEW where Nl (w) c nf'(w), nf'(w) _ CE, P1(w) _ {a U(P); wa aEPi' (w) PROPOSITION 2. If g GC satisfies &(g) = g-1, g can be described as follows : (8) g = z . a'w , Q(z)1=z.-wa a(z)-1 where z NC, w 12(T) (normalizer of T in Ga), a, a' E AC, w = w(w), w E W. PROOF. Let g= nawn1, n NC, a AC, w = w(w) I2(T), n1 E Nl (w) in Proposition 1. The equality d(g) = g-1 is written as follows : nwan1= Q(nl)-la(a)-1Q(w)-16(n)-1, Q(n)1 a(NC) = Nl. Put Q(n)1 = n2n2 , n2 Ni(w), n~' E N; (w). Because of the disjointness, we

have &(w)-1 = wa0 where a0 T, and nwan1 -' = a(n1)-1Q(a)-'wa0n'n" 2 2- -~(n1)-1w1non"., 2 = 4(n1)-'wln'wl 1wln2, where w1= 6(a)-'wa0. U(n1)-'wln2w11is an element of NC and n2 is an element of NC. Therefore, by Proposition 1, we have n = Q(n1)-1w1n2w11 , wa = w1, n1= n2 and n = Q(n2)-la(ne)-1= 6(nl)-lwln2wi 1 Hence from Q(n2)1 = Q(n1)1 follows 6(n2)-1= wln'w11, n2 is an element of Nf(w) and therefore n2= n32, for a suitable n3 Nf(w). Substituting n2 by n32,

N 6 K. AOMOTO

we obtain Q(n3)-2 = win32wi 1= (win3wj1)2, where Q(n3) NC, w1n3wj 1 NC. Since the correspondence between nC and NC is bijective, we have a(n3)-1= win3w11.

Thus n = Q(n1)-lw1n'2w1-1, 3 g = Q(n1y 1w1n'2w1-swan1 3 - =Q(n1)-1w1n'w1-1 3 w13n'w1-1 • wan 1 = 6(n1)-1&(nlwinani, where a(n3n1)-1 E NC. We put z = Q(n3n1)-1, then we have g = z • wa&(z)-1, which we had to prove. wa in (8) must satisfy (9) Q(wa) = (wa)-1. A L1Rin GC is an analytic isomorphism. AC = T • A is the direct product where Q(T) = T, and Q(A) = A. Let a = t • , t T, E A. Then (8) can be written as: 6(w)o(t)o() = t-1-1w-1, o(w)' _ wao, aq lw_16(t)6(e) = t-1-1w-1 and ao lw_1Q(t)w • w-1Q()w = Therefore

(10) ao lw-1G(t)w = t-1 , w-ra()w Let e =1 and1 E A. Then w-1Q(e1)2w=-2, wr1Q(1)w = 1. Consequently

(11) wa = wte = wt i = wte1e1= we1w_lwte1 = U(e1)-lwte1 Q and Ad w operate on lju as linear transformations and Ad w ~u = w. For any H E i o o w(H) = 6(Ad w(H)) is an involution, because (o o w)2(H) = Q[Ad w {o [Ad w(H)1 }] = Ad Q(w) Ad w(H) = Ad a(H) = H. Let t%z {X Etju, 6ow(X)=X} (12) b ;-{X Abu, 6ow(X)=-X}. Then bu = ~u +hu is a direct sum. Let lju c Tw, lj, Tw. Then T = Tw • Tw, where Tw n Tw may be non-trivial. We also define two subgroups `Tw and `Tw as follows :

I `Tw = {t E T ; Q(t) = wtw-1 } (13) `Tw = {t E T ; Q(t) = wt-lw-1} .

Obviously `TW T,, `Tw Tw with the index [`Tw : Tw ] < oo, [`Tw : T, ] < oo. Let t = exp (H.+H_), H~ tju, H_ l;. Then

(14) wt = w exp (H~+H_) = w exp H- exp H~ =w(exp H_ expH~=w exp-1 H-w`lw exp 4-FL exp H~ 2 2 2

-1 = Q exp 2 H_ w exp H.~ exp 21 H_ .

Because of Q(wt) = (wt)-1 we have Q(w exp H~) = (w exp H~.)-1, Q(w)a(exp H~ ) = exp (-H.). w-', ao 1w-1Q(exp H.) = exp (-H~) • w-1 and a-1 exp (H~) = exp (--H~). Thus

z Double coset decompositions T

(15) (exp H~)2= ao , H. E i . The set of elements H~ which satisfy (15) is finite which may be empty,y t = exp H. is contained in Tw and satisfies t2 = a0. If t and t1 (elements of Tw) satisfy P = t1= a0, then (16) (t tl 1)2=1, t ti 1 E Tw and t ti 1E 'T;. Conversely, suppose t tl 1 E `Tw, then t2 = ti. If t, t1 E T, satisfy wt = awU(a)-1 for a suitable element a E AC, then (17) t - t1 mod T;. More generally, suppose that (18) wt = bw1t1Q(b)-1, for b E B, w = co(w),wt = w(w1), t E T,, t1 E T1 which satisfy t2 = a0, ti = a and ao 1= Q(w)w, al 1= o(w1)w1. By (6), w =w1 and so w = w1. Therefore Tw = T,, ao = a1. Let b = na, n E NC, a E AC, then (18) is rewritten as follows wt = nawt1Q(a)-1d(n)-1. Let o(n)-1= nin2', n1 NI(w), n2' E NI (w) as in Proposition 1. Then wt = nawt1 Q(a)-1nln~= nw'n;w'-1. w'nf where w' = awt1Q(a)-1. Here nw'niw'-1 is containedd in NC and n~ E Nf'(w). By Proposition 1 nw'niw'-1=1, wt= w', nf' =1 and wt = awt16(a)-1 (a E AC). By (17) t ; t1 mod Tw. We define P, ~o, ', ?o, [ P ], [P1 in the following way : 'P_ {gE GC; o(g)=g-1} ~~_ {gE GC; g=g1U(g1)-1 for a certain element g1 E GC} (19) ' _ {wa ; a~ = a0, ao 1= Q(w)w, a E Tw, w = co(w),w E W } (The set of such elements a is denoted by Tw)

Two elements of ? (or of ?P'0)wa and weal are said to be equivalent if and only if w = w1 and a - a1 mod (Tw n Ti). The set of these equivalence classes of ? (or /') is denoted by [l'] (or [J). We shall also denote by [ P ] (or [?1To])the set of representatives in /T (or ?o) of all the equivalence classes [?P'] (or [?P]). We have arrived at: THEOREM1. (The Borel subgroup) B in Proposition 1 operates on P, by' taking (20) b : g--> b . g • Q(b)-1for any g E 0 (b E B). This operation is not generally transitive, and 0 is decomposed into finite B-orbits. This orbit set is identified with [ P ]. More precisely, in any B-orbit $ K. AoMOTo

there exists an element wa of ?P'; and two elements of contained in the same orbit are in the same equivalence class of [?]. PROPOSITION4. Let o o w and 6 o wl are conjugate with respect to W, that is, Q W1= wo (6 o w)wo1 for a suitable wo. Then we can take as w(w1)the element w1= ~(wo)wwo1, where wo= w(wo). Moreover if wa ? (or wathen weal E ?P' (or a respectively) with a1= woawo1, and conversely. Consequently, the property whether wa is an element of ?o or not depends only on the conjugate rclass of a o w with respect to W. PROOF. For any t T 6(wltwr1) = Q(6(wa)wwo1twow-16(wo)-1) = wo6(wwo1twow-1)wo 1= a • wl(t). Therefore we can take w1= w(w1) and 6(wo)wawo1= o(wo)wwo1woawo 1= wla1 where w1= woawo1. The decomposition T = TwTw is isomorphic to the de- composition T = TiTwl.

2. The relation between "P and P0. Obviously P We consider the problem whether P coincides with 'P or not? Let g be a real semi-simple Lie algebra, g = +p be a Cartan decomposition and B be an involution automorphism determining this decomposition: 0(X+ Y) = X-Y for X , Y E p. Let G be a connected real semi-simple Lie group with finite centre corresponding to 9. Let Kc l in G. K is a maximal com- pact subgroup of G. Let P = exp p, then G = K • P is a Mostow decomposition. Let lj be a 0-invariant Cartan subalgebra of g. Then b _ +b (t = ~~_ n p) is a direct sum. We define ~. = bR= 4/-1 u and t)C (the complexification of i ). We also define the dual spaces lj~ (over R) and (LIC)*(over C) of l)R and if respectively. I can be identified with (f)*. We :introduce the order of t R as follows: We take the orders of (t))* and ~(~/-lll i)* and define the order of (bC)*, compatible with those of (j)* and (4/-J1)* as follows, (21) llR32>O=21lj,>O or AIlj~=O and 2~~/-llj >O. ;Let d be a root-system of c with respect to f, and ,4~ be the set of positive root: di (or do) be the set of positive roots a such that a lj, > 0 (or a 1ll~ = 0 and a "l-1l,>0). Then d~=di tJdo. Let n _ CEa, n.=n+ng, iIC aEdi ~ aEdO} (CEa+CE_a)+l)i, where I is the complexification of t and m = mCn g. m is reductive and 0-invariant. Therefore m is decomposed into m=m1+m~, m=l-m, m~=mnp. Double cosec decompositions 9

Let n~ mN., c Ap, mr M, exp m,=M~ and MK =Mn K. M~ is contained in P. Then, LEMMA 4. (Harish-Chandra) i) M= MK • Mp and the map MK x mP--~ M, defined by

(22) (k, X )-k k exp X is an analytic diffeomorphism. ii) The map Kxm~xA~xN~--G defined by

(23) (k,X,a,n)->kexpX•a•n is an analytic diffeomorphism. For the proof see [10]. We apply this lemma to GC itself in the following situation. GC : a connected complex semi-simple Lie group gC, 6: (anti-linear) involutive automorphism of c with respect to a non- compact real form g, such that this automorphism can be lifted to GC, gu : 6-invariant compact real form of gC and gu r Cu, 6: (anti-linear) involutive automorphism of 9C with respect to cu, o'=61gu, B=OIg. Let lj, Llu, 4, 4~, 41, 4o be as in the beginning of § 2. Let nC = CEa, n+ = CEa, no = CEa, mC= ] (CEa+CE-a)T b~. uEd+ aEdl uEda aEdO

N Then mC=ml +m2 where xii1=mC n gu, m2z mCn ~/_1 gu, mC M, MN1=M N nGu, M2=Mnexp(~/-1gu), M-M1•M2, R~A and ljCr AC. We have GC= Cu • ANC (Iwasawa decomposition) = GuANoN+ = Cu . No AKAPN+C = C N 1M2A~N+N = Gu1V12ApN+N . By Lemma 4 the map Gux M2x A~X N --~GC defined by (24) (u, m, a, n) -> u m a n

N and exp : m2--~ M2, exp : n+ * N+ defined by X ---~exp X, Y-~ exp Y respectively are analytic diffeomorphisms. Now, suppose v Cu n ~o, v = g • &(g)-1, g E Cc. If we write g= uman, then v = uman Q(uman)-1= uman a{n)-1Q(a)-1Q(m)-1Q(u)-1 and (*) u-'v • a(u) = man Q(n)1U(a)1a(m)1 Q(m2)= m2, Q(n) = n and therefore a(M2) =1212, a(N) = N+, [mC, 4] C 4, and

N 10 K. AoNIoTo

&(a) = a for a A~. The right hand side of (*) is equal to manQ(n)-1a-16(m)-1= mo(m)-1aQ(m)n6(n)-1Q(ln)-1a-1

N because any element of A~ commutes with any one of M2. Now mU(m)~1 M1M2i aQ(m)nQ(n)-1Q(m)-1a-1 NC. Therefore, by (24) we have u-1v • Q(u) = m&(m)1, nU(n)-1=1 and m • &(m)-1 exp ('/-1 gu), obviously, a(G) = G,. Put exp (/i11 gu)= P. Then P • P n Gu = {1}. For, suppose p, p1 E P such that pp1 E Gu, then e(pp1) = pp1, e(p)e(p1) = pp1, p-1p 1= PP, therefore p2 = P? and p-1= p1 i, e. pp1=1. Thus we have proved : u-1vQ(u)=1, m&(m)1 =1, no(n)-1 =1 and (25) v = u • Q(u)-1 We define subsets of Gu : 0 (26) = {u E Gu ; a(u) = u1} po = {u E Gu ; u = v • Q(v)-1 for a suitable v E Gu}. It is well known that Do coincides with the totally geodesic submanifold exp and is a compact symmetric space [14]. We have proved: PROPOSITION5. 0 n ~o = e0, that is, if any element u of Cu satisfies u = g 6(g)-' for a suitable g of GC, then there exists v of Cu such that u = v • a(v)1 and therefore u can be written exp Y for a suitable Y E COROLLARY.In order that tP _ P may hold, it is necessary and sufficient that &=E. Proof follows from Theorem 1 and Proposition 5. LEMMA5. Let G be a compact, connected Lie group. i) Let S be a toral subgroup of G. Suppose a is an element of G which commutes with each member of S. Then there exists a torus T such that T a and TS. ii) Any maximal torus of G is conjugate with each other. For the proof see [13], p 212 and p 247. Let g be any compact semi-simple Lie algebra, G' be the automorphism group of g, G be the connected component of G' containing the identity. As is well known, G' is a compact group, G is a normal subgroup of G' and [G': G]

(30) u = v • ezo(v)-' and e2 = z-'a(z)-' . Further, if the equality

holds for two elements ez, 1z1 Tu Z satisfying e2 = z-'a(z)-', i = zj lo'(z1)-1 and for some element v Gu, then there exist n N (normalizer of Tu) and zo E Z such that (32) e1z1= n • ezzoar(n)-1 and zo Zn 0 . PROOF. Q • Ad u is an involutive automorphism of gu. By Lemma 6 there exist v1 Gu and 1 Tu such that o • Ad u = Ad v1• U • Ad Ad v1' and Ad u = Ad 6(v1)evl1. Therefore u = a(v1)ezvi' for some element z of Z. We get (30) by putting v=Q(v1). (32) follows easily from Lemma 7. Let c be the projection of N onto W =N/Tu. W operates on ~/-1 hu = h as a group of linear transformations. U also operates on L1R•Let W6 be the subgroup of W consisting of elements which commute with 6, and NQ= c-'(W0). We take a representative element w in c(w) for each w W and denote it by w = w(w). Suppose ez Tu Z be contained in Oo, then by (32) ez = n • z00(n)1 (n N, zo Zn Oo). Therefore c(n) = c(a(n)). We put c(n) = w, and n = wa, w = ca(w), a Tu. w lies in W0. We have z = w0(a)zoa-'6(w)-' = w6(a)a-'zo6(w)' = wa(w)-1z0U(w)U(a)a-'6(w)-'. a(a)a-' Tu and w makes i and rju invariant respectively, because w W0. Therefore a(w)a(a)a-1a(w)-' TI';. We define the set (33) I' = {w • a(w)-' ; w = w(w), w W0}C Tu . Then ez I' • (Zn (9). T;. But F • (Zn Oo) • Tu C Oo holds because any ele- ment r • z • exp H1 of F • (Z n (9). TI';, where r = w • 0(w)-', z Z'-' (9, H1 I);, can be written

r • z • exp H1=exp 2 1 H1 • w0(w)-'z exp 21 Hl -1 = exp H1 • w • z • q(w)-10 exp 1 2H1 2 and therefore rz exp H1 belongs to Oo. We have proved : PROPOSITION 7.

(34) (Tu . Z) r 9 = r • (Z n O)Tu n (Tu Z) COROLLARY. In case Cu is the adjoint group of gu, ai is defined on Cu and Z= {1}, and therefore (35) T, n oo = F • (Tu -' Tu) • PROPOSITION 8. In order that any element u C 0 is contained in Oo, it is Double coset decompositions 13 necessary that 0. • Ad u is conjugate to 6 in the adjoint group and this condi- tion is sufficient when Gu coincides with the adjoint group itself. We can now answer if 0 coincides with Oo or not, by calculating (34) or (35). REMARK. Let G7~ be the adjoint group, gu be the Lie algebra of Gu and B(X, Y) (X, Y E Gu) be the Killing form of gu, then (36) B(crX, Y) is a symmetric bilinear form on c, where Q is any involutive automorphism. Suppose, further, gu be simple. If two involutions o, z define real simple algebras (non-compact) of the same-type, then the signature B(aX, Y) and B(vX, Y) are equal. But the converse is not always true, though it was con- jectured by E. Cartan (See [13]). The exceptional case can take place when g is of type Dn. Therefore we can not determine whether u 0 belongs to Oo or not, by calculating the signature, when g is of type Dn. But in other cases, it is possible to determine whether u E Do or not by calculating the sign. We shall use this method later in an example.

§ 3. The finiteness of double cosets B\GC/G. It follows easily ?1To=' n Oo where Oo = exp For any element u E Oo there exists an element u 9 such that u = u2. There may exist many such elements, but we shall fix one element of them and denote it by /u. We define (37) ~/[11= {4/; wa [ 'o]} Suppose g GC (in the situation of Theorem 1), then g • a(g)1 cbo. By Theorem 1 there exist b B, wa [P01 such that

(38) g.6(g)2^b • wa Let ~lwa = r, then wa = 12 = r ~(r)-1 and (38) becomes g • 6(g)-1= b • r • Q(br)-1 where (br)-1 is a-fixed. Therefore (br)-lg E G and (39) gEbrGCB•r•G. Conversely, for rl, 12 E ~/[ F j such that rl Br2G we have ri = b • r2 • Q(b)-1 for some b B. Consequently, r; = r2 by Theorem 1 and r1= 12. We now have: THEOREM 2. (In the situation of Theorem 1.) GC= B • G holds. The double coset B\GC/G is finite and can be identified with /[?P0] canonically. It is well known that any Borel subgroup of GC is conjugate with each other, and therefore the Borel subgroup B in Theorem 2 may be replaced by an arbitrary Borel subgroup of G c. Thus 14 K. AoMOTO

THEOREM 2'. Let GC be any connected complex semi-simple Lie group, B be any Borel subgroup of GC, and G be any real form of GC (connected or not), then the double coset B\GC/G is finite. In Theorem 2 the right cosec B\GC is a compact complex algebraic manifold on which real semi-simple Lie group G operates. G does not operate transitively, in general. But we have found the number of orbits is finite by Theorem 2. Next we shall calculate the dimension of each orbit. Let r = s/wa, wa [ P ]. Br is a point of B\GC. The G-orbit of Br is B7G and its isotropy subgroup is the group S :

(40) S== {x G ; rxr-1 E B}. Let rxr-1= b, b B, x = r-1br. x G is equivalent to Q(7b1) = r-'br, that is r 6(b)r-~ = r-1br, r2ai b)r-2 = b and (41) wa&(b)(wa)-1= b. We have C b, ACC B. We put o1 = Ad (wa) Q. Q1 operates on bC as a conjugate isomorphism. Let f/1 be the set of 61-fixed elements of )C• Then lj1 is a real form of bC, and therefore Ad r-1() is a Cartan subalgebra of ~. As 1 contains a regular element of g, and AC is a Cartan subgroup of GC, G n Ad r-1(AC) = Al is a Catran subgroup of G in the sense of Harish-Chandra. Therefore S always contains a Cartan subgroup of G. We define the subsets of positive roots: P+1= {a E P, a1(a) > 0}, (42) P!1= {a E P, a1(a)<0}. The Lie algebra of S is (43)) = Ad r-1(n1+I1) where

(44) n1= R(Ea+iJ1Ea)+ ~' V--1 R(Ea-C'1Ea) aEP+1 aEP+1 is a subalgebra of nC. In particular, if P.1= ~b then S = Al is an abelian group. If t) is a fundamental Cartan subalgebra of g, - LIt+LIp, R - /-11+bP and if we take the order of (tjR)* (the dual space of L1R)such that for 2 E lj

(45) 2>0=2~ ~/-II >0 or 2 ~/-1 Ij =0 and 2 Ij, > 0 . Double coset decompositions 15

For any a d, we have a 4/_f )f ~ 0, as is easily proved. Seeing that 6(P) _ -P and w(P) ~ P, for any w * 1 W,

(46) 61(P) _ - P

N is equivalent to w =1. Let nC = CE_a, and no ~ NC in GC, _ CEa+I C a=P aEP N and b r B in GC. When (46) holds, wa = a (a T), a2 =1, &1(NC)= NC and = h. has the minimal dimension which is equal to the dimension of B\G C, and S\G is an open orbit in B\GC. In other cases, the dimension of S\G is smaller than that of B\GC and therefore is not an open orbit. We shall see later some examples of open orbits. The number of these orbits is equal to [(T~'r E) : Tar T j. Let n = dim g, l = rank of g, m = number of P, m1= dim a1 (l+2m = n). Then dim = l+m1 and (47) dim S \G = n-l-m1= 2m-m1. LEMMA 8. Let g be a compact Lie algebra, G be the group of inner auto- morphisms on g. Suppose i be the involution on g such that g = +~/-1 ~, _ {6X=X}, /-1= {6X=--X}. Let Kr in G. Then any two maximal abelian subalgebras in ~/-1 are conjugate with each other with respect to K. PROOF. g is a direct sum of its centre c and its derived algebra g' = [g, g' : g = g'+c, g' and c are both invariant with respect to o. Therefore g' = c=c~+c_, where g =g'n , g_=g'nc~=cn , c_=cand = g'_+c_. Any maximal abelian subalgebra ~j_ in contains c_, and therefore is written in the following form -_b--+c ~'-=~-ng' Iis a maximal abelian subalgebra in g'_. Thus we have reduced this problem into that of Cartan's well-known theorem about conjugacy of I[141. q. e, d. Let I) is a Cartan subalgebra of g with the largest non-compact part. We define an order of i as follows :

(48) Am, 2>O= AJl,>0 or 2H/-1~~>0, AIf)p=o. We define d (root-system), P (positive roots), P~ _ {aEP, a ~~, > 0}, Pa ={aEP, a I /-1 t > o, a ! jp = 0}, P = P~ U Po and g = J (CEa+CE-a)+i . b = CEa aEP aEP +IjC is a Lie algebra of a Borel subgroup B of GC. Put nC = CEa, n+ aEP = CEa and no = CEa. aEP+ aEP0 In case of w=1, (44) becomes

(49) n1=nCng, =ni+)a, 16 K, AoMoTo

where is a maximal abelian subalgebra of ~. In this case, has a maximal dimension and such is a maximal solvable subalgebra of g [14]. Moreover, it seems to me that this is the only case where has a maxi- mal dimension, but I can not prove it in general. Let T = where T Ijr, T-c V -1 i . We claim: PROPOSITION 9. (50) Tin&o=T,.nT_ PROOF. Let a T n Do then a2 =1. Let gu be the centralizer of a in g and cur Gu in Gu. a lies in the centre of Gu. As gu is invariant with respect to Q, so is Gu. Gu contains T. Since a E 9, there exists k E K such that a E k . T- • k-1. But any elements of k T- • k-1 commute with a, so kT-k_1 C Gu, and Ad k(//-1 Ij~) is a maximal abelian subalgebra in cu n 4%/-1 ~. By Lemma 8 there exists k1 Gun K such that kT-k_1 = k1T-kj 1, because 's,/-1 Ijp C gun 4/--1. Hence a kT-k-1 = k1T-kl 1 and k1 commutes with a, a E T~. q. e. d. COROLLARY. T~~ [7'o] consists of only one element. CONJECTURE. The number of G-orbits in B\GC which have the smallest dimension is one, and this orbit is isomorphic to the coset space G by the maxi- mal solvable subgroup S defined by (49). Next we shall find the relation between orbit spaces and conjugate classes of Cartan subalgebras of g. The same result was obtained by H. Matsumoto independently. We recall that o = {u6(u)1 n(T) ; u Gu} where J2(T) is a normalizer of T in Gu. Let T c(t) = ~u• If uor(u)-1 E J2(T), Ad u-1(Iju) = I, is also Q-invariant. The converse is also true. u6(u)1 J2(T) if and only if Ad u-1(Llu) is c-invariant. LEMMA 9. Let Iju and Iu be two 6-invariant Cartan subalgebras. Iju = Ijr ~u n r,-1 b~ = bu n 4/-1 ~, = Ijun 4/-1. Then Ij = Ij1+ v, Ij' = l +i are both Cartan subalgebras of g. Moreover, in order that Ij and Ij' are conjugate with each other under the group of inner automorphisms Ad (G) of g, it is necessary and sufficient that Ij and Ij' are conjugate only under Ad K The last condition is equivalent to that Iju and h, are conjugate under Ad K. PROOF. It follows immediately from Theorem 2 and Proposition 5 in [19] p. 383. Let i be the standard Cartan subalgebra of g in each conjugate class (i =1, 2, ... , r) such that Ij2= (Ij2n I)+(l n ). We define W = n I+Iji n Let Iju Tu in Gu, ~2(Tu)= n1 be the normalizer of Tu in G. Suppose wa E a be written in the form wa = u6(u)1, u E G. Ad u1(Ii) is conjugate to some Double coset decompositions 17 w under Ad K: Ad u1(b) = Ad k(in) k E K. We fix an element Yj for each j such that (51) Ad i 'O ) Y3 Gu . Then Ys1uk 72' and (52) wa = ua(u)-1= Yjna(n)-1c(Y j)-1 (53) Y31~Yj=j . If wa and w1a1 be in the same class of [/T], and wa is represented in the form (52), then w1a1 can be written

(54) w1a1= Y;(na')6(na')-1o(r~)-1 for suitable a' E T j by Theorem 1, and conversely. The set (56) '~ = {na (n)-1; n E 2j } 1 <_j < r is canonically isomorphic to the coset space of J2j: J2j/(ilj n K). In virtue of (54), [ P ] is canonically isomorphic to the set

(57) C)U j1j/(~j n K). Tu . j=1 We have arrived at: THEOREM3. Let 1j1,b2, , ljr be standard Cartan subalgebras of g and lju,Q, , t be their compact form. Let Tu be a maximal torus corresponding to t and ~j be the normalizer of T, in Gu for each j. Then [P] N U n5/(n' n K) . T, = U W/Wj 1nj~r 1nj<_r where W = ~j/T j and Wj = (~lj n K) • T j/Tu is a subgroup of W.

§ 4. Examples. NOTATION. (We shall make use of Sugiura's notations in [19] from now on.) D(m, n, C) = the set of complex matrices with m lines and n columns. 9(m, n, R) = the set of real matrices with m lines and n columns. gr(n, C) = l!J1(n,n, C). gl(n, R) = J (n, n, R). u(n) = the set of skew hermitian matrices. du(n) = the set of skew hermitian matrices whose traces are zero. ?r(n, C) (or ~(n, R)) = the set of complex (or real) square matrices whose traces are zero. He(n) = the set of hermitian matrices. Sy(n, C) (or Sy(n, R)) = the set of complex (or real) symmetric matrices, o(n, C) (or o(n)) = the set of complex (or real) skew symmetric matrices. tX = transposed matrix of X. det X = determinant of X. X = complex conjugate of X. T r(X) = trace of X. 1. GC= SL(n, C), G = SL(n, R), Gu = SU(n), K= SO(n), then Z = {scalar 18 K. AoMOTo matrices with determinants 1}, = {skew symmetric, real}, ~/--1 _ {symmetric, skew hermitian, trace = 0}. We put n = 2v or 2v+1 according as n is even or odd. Al 22 We denote the diagonal matrix by Diag [2122 •• 21. . Let l' be a fundamental Cartan subalgebra, l b - T+ :

The lowest term 1 appears only in case of n = 2v+1. If e2 =1, = Diag [±(1,1), ±(1,1), , ±(1,1),1]. But these are con- tained in Oo. X= Diag [J-1 cot "-1 co /-1 co2 11W2' f+11 +2co +co +1 = O cpi real. If we put cp2= 0 or ir, 1 <_ i cpy+1= 0 (mod 2irZ integers) suitably, we obtain all e, e2 =1. On the other zo (z) =1 for any z Z. And if z 0, then 2 =1 . Z C 9 and therefore ez Oo. By Proposition 6, 0 = Oo and P = tbo. This fact is well-known and can be proved by a direct method. If GC is replaced by the ad joint group of SL(n, C), 0 * Oo in general. This occurs for example in case n = 2v and Double coset decompositions 19

a(g.Z)=g•Z=g.Z, o(g•Z).(g•Z)=g2•Z=Z, i(g•Z)=g 1 •Z. But g.Z can not be written in the form g1 . gl iZ for any element of SL(n, C). For, if weput gl 1 = b~(11g12, whereg11 vX v matrix,then /~ --g12 g11 = A gl' 12), g21 g22 g22 g21 g21 g22 22 v=1 . -g12 ` Ag11, g11_ g22 = Ag21 g21 ~g22 g12 = 12 2g12, g22 = I A I2g22 for some A E C. But these equations are impossible except for g12= g22= 0 and therefore except for gj 1=0, which is impossible. This fact is deduced also from Proposition 8 and the classification of real simple Lie algebras of An type. Let B = {upper triangular matrices} SL(n, C),

n (60) = {H= Diag [t1t2 ... tn], ti =0, t1 real}, t = ~/-1 .

For any H E , 6(H) = -H therefore i is commutable with each member of W (Weyl group with respect to l) and (61) (c • w)2 =1= w2 =1 for w E W, where W is the symmetric group of degree n.

(62) w[V 1 t1 V-1 t2 ... ~/-1^tn] = [4/-1 t~1 V-1 t~2 ... ~1-1 ten] where1(2" • n is a permutationof(1, 2, 3, , n). Forany two elements 122 n w, w' of W, • w is conjugate with i • w' under W, w is conjugate with w' under W. The conjugate classes of W are represented by cyclic permutations. The classes of elements of order 2 are : 1, (12), (12)(34), ... , (12)(34) ... (2u-l, 2u). We denote by [w] the number of conjugate elements of w. Case w=1, w(w) =1, a(w)w=1, 6 • w(H) _ -IH, H E l , bu = {0}, bi=b' u Tw = {1} (Notations are the same as in (12), (13)) T w= {1}. The total =1. Casew = (12), [w] = () Z 20 K. AoMoTo

6 • w Diag [/-1t1 -/-1 t2 ••• ~/-1 t] = Diag [-/it2, -~-1 -~/-1 t3, ... , -s,/_1 tj u = Diag [/T t1, -/1t1, 0, ... , 0] 1j; = Diag [~/-1 t1, ,/-1 t1, ,/-1 t3, ... , V-1 tj Tu = Diag [A1, Ai 1, 1, ... ,1] Tu =Diag [A1, A1,A3, ... ,An] aETw, a2=1=a=[±(1,1),1, ••• ,1]. Thesetwo elements are congruent with each other mod T;. Thetotal= ().n Case w = (12)(34) ••• (2r-1, 2r) 1 _< r _< v if n = 2v+l, 1 _< r < v-1 if n = 2. 2

Cw] = n(n-1)(n-2) 2rxr!... (n-2r+1)

Two a, g 1), ±(1 ... ±(1 1].

2r

All these elements are congruent with each other mod (Tw(1 Tw). The total n(n-1)(n-2) ••• (n--2r+1) 2rxr! . Case w=(12)(34) ... (2v-1, 2v), n=2v

Tw={Tw3aa2=1}= {a = Diag [±(1 ,1), ±(1 1)] }

. Double cosec decompositions 21 t213=0}to = where{Diag [/-1t1 /-1t1 /-1 t3 -/-1t3 ... ,'-1 t2v-1 /1 t2v-1] cT w= {Diag [A1Al2323 ... A2_1 '2)-1]A1~3 ...A2,-1 =1} , Therefore any element of Tw is congruent modulo Tw to either Diag [11 1] =1 or Diag [-1-1,1 ••• 1] which is not congruent to each other. The total = 2 X n(n-1)(n--2) 2 X (n-2v+1) Thus, the number of ['o] is equal to v! =1+2)+2!n 1 n(n-1)(n 2 2)(n-3) + ... + 1 n(n-1) ... (n-2r+1) 2 r! 2 n(n-1) ••• (n-2v+1) n -2+1 v v!X2y +...+ 2n(n_-1) ••• (n-2v+1) v!X2~ n=2v. For example n = 2 o =3; n = 3 o =4; n = 4 o =13. It is when and only when w['./1t14-1 t2 -1 tn] = w[~l -1 to ./ t1] that defined by (40) has the smallest dimension, and has the smallest dimension in only one case if n = 2v+1 and in two cases if n = 2v. The G-orbits in B\GC corre- sponding to these are open in B\GC and isomorphic to homogeneous spaces S\G, where S is a fundamental Cartan subgroup of G (see (47)). These manifolds are very important in the theory of unitary representa- tions as Gelfand and Graev have shown in [11]. They have shown, when n =4, that these manifolds have explicit parametric representations. When n = 2v, it is easily generalized as follows : Let Pn-1 be the projective space of (n-1) complex dimension, z= (zii' : z22': : zn2') be v homogeneous coordinates 1 < i c v, and z = (zc1)zc2) ... z(v') Pn-1 X Pn-1 X ... x P1 _ Q,

z11) z91) ... zc1) z12) z 2 ... zn2) J (z) = the determinant zly' z2v' •• • zn~' 2(1) 2(1) ... 2(1)

z1 (v) z 2(v) ... zn, (v) We define Q~, Q- Q~ ={zcQ; d(z)>0} Q-={zu Q; 4(z)<0}. Then G = SL(n, R) operates transitively on Q~' or Q- by defining

vnYA 22 K. AoMoTo

z~(z' g)1= zjgj1 for gE G g=(gij) j=1 These two manifolds Q~, Q- are isomorphic to S\G where S is a fundamental Cartan subgroup. When n = 2v+1, B\GC has the only one open dense G-orbit. Finally we give a table showing the dimension of G-orbits in B\GC when n=4. dim B\GC =12

2. GC= SL(n, C), G,~= SU(n), n = 2v

Q.: gC~ X11 1'12 __, X21 X22 -"2X212 X11~21 Xi~ ., v x v degree

lj = {Ding [t1 ... tv f l ... tLi], Re t; = 0} j=1 is a fundamental Cartan subalgebra of g

lju = {Diag [~/-1 t1 ••• ~i-1 t„ ~/-1 t~~1 ~/-1 tj t; =0, t; real} =1 ~' ~~un = f, Tu ~ un /-1p=/-1~~ fir- {Diag [~/-1t1 ~/-1t -V-1t1 -~/-1tv] t; real} Tu ~a, a2=1=a=Diag[±1 ••• ±1 ±1 .•• ±1] the i-th and (i+v)-th elements have the same sign. These elements do not always belong to Oo. For example, we take a = Diag [-1,1.1, 1, --1, ••• ,1]. The elements in '-1 which commute with

y a have a reduced form :

nvU Double coset decompositions 23:3

where -X11- X22, X21= Xi2 = skew symmetric of degree v-1. The set of ele~ ments of gu which commute with a is a subalgebra gu of gu, g' is o 4nvariant..

Let s/-1Ij be any maximal abelian subalgebra contained in ~/T' and let -/-1Iy cT;". If a E e0, then a E Tu', because a lies in the centre of C' which is an analytic group corresponding to g' and because ~/-1is also maximall abelian in Any element of such ~/-1j, for example, has a form

H= Diag [~/-1 v, ~/-1t 2 •. • /-1t, ~/-1r, /-1t2 ...

when(±t)+=o.j =2 j z But a can not be written in the form exp H, and so a does not lie in 9.

. We have found that Q Oo and so 'P 'Pa. We must make in experimentt whether wa E P is contained in ?o or not. H E Iju, H= Diag [V-1t1/-1 t2 ... ~/-1 tn] (H)= Diag [-~/-1 ty+1... --it, -~/-1 t1 ... -~/-1 t~] w(H) = Diag [J-1t1 J_1 t2 •• • ~/-1 tZn] 6 • w(H) = Diag [-~/-1 ti~~1... -'/-1t~n -~/--1 t;1 ... -~/-1 t]. w E W and the conjugacy of 6 • w is equivalent to that of -a• • w under W. Case 1) -o • w =1, w = v+11 2 •• 2vv +i1 •• • v2v 24 K. AOMOTO

.,u_ J' lu ={O}, Tw = {1}. Therefore Tw = c. Case 2) =(1, y+1) ... (r, y+r), r

~(w) • w = Diag [1... 1-1•. -11... 1 -1...-1]

r v-r r v-r

u = {Diag [i/-1t1... ~/-1tr 0 ... p _4J_1t1... _ J_ltr o O]}. But Tw (6(w) • w)~1, therefore 7j=O. Case 3) -a' • w = (1 v+l)(2 v+2) •.. (v 2v), w =1, c (w) • w =1 It= {Diag [s/-1t1/-1t _,s/_+ t1 ... -,/-1 t j tti real} Double cosec decompositions 25

fju = {Drag• [~/_i t1 ••• ~/-1 ty ~/-1 t1 /-it] )J ti =0, ti real} . ti=1

Tw = Diag [±1 ± 1 ± 1 ±1] i-th and i-j-u-th are of the same sign. In this case ~/-it)~ is also maximal abelian in ~/-1p, therefore by Proposition 7, Tw n 9 = Tw ~1Tu = Tw ~1T;. Any element in Tw n Oo is congruent to each other modulo T;. The number of [ P ] is equal to v!2 An open orbit in B\GC is precisely the one defined by w =1, and so there is only one. 3. GC= SL(n, C), Gu = SU(n), Q(X) _ -JpgtXJpq, X E SL(n, C) (or &(n, C)), where

p p q --> 4.-+ F---- _ _ _ X11 X12 X13 X ` X31X21 X32X22 X33X23 E 9C Q(X) - ( ttX22tX21 123 tX13tX12tX 11 tX33tX32tX31 ) X 11 X 12 X13

g = {X = X21 tXll X23 ; X12, X21 E u(p), X33 E u(q)} tX 23 tx13 x33

X11 X12 X13 = {X= X12 X11 X13 X11, X12E u(p), X13 E 9J1(p, q), X33 E u(q)} tx 13 _t 13 x33 ={X= -X12X11 _ X11X12 -x13);X13 X11EHe(p), X12 Eu(p), X13E 9)1(p, q)} tX 13 tX13 0

( 26 K. AoMoTo

= {H= Diag [tl+~/-1 u1, ..., t+/-1u,pp-t1+/-1u1, ..., -tp+~-lup,

p q /-1up+1, ... , ~/-1up+q], 2 uj+ u1= 0, tj, u; real} j=1 j=1 ju= {H=Diag [/-1(tl+ul), ... ~/_1(tp+up), 1(-tl+ul), ... , -1(-tp+up), ~/-1 up+1, ... ,-1 up+q]

where t1, uj = real, 2 uj+ up+j = 0} j=1 j=1 Z&={1}, n=odd, ={±1}, n=even, Z~Oo=Tu neo= {1} if q>0, _ {±1} if q=0. We put tl+ul = Z1, ... , tp+up = zp, -t1+u1 = zp+1, ... , -tp+up = r2p, up+1 = Z2p+l, ... , up+q = 2n. Then 6 Diag [~/-1 z1, ... , ~/ 1 zn] = Diag ['V-1 zp-1, ... , V/_ 1 z2p,

~/_1z 1>... >^1 Zp,-1 Z2p+1,... p p Q= p+1 1 p+22 ... 2pp p+1 1 ... p2p 2p+1 ... n E W 12 ••• n f or any w w, w= , 6 • w Diag [~/-1v1i , -1 r] =Diag [zip+1~ ~1 22 ... lrt Vi2p,zi1, zip, z~2p+1, , ran] representative in conjugate class of order 2 : 6• w =id, (1, P+1), (1, p+l)(2, p+2), ... , (l, p+1)(2, p+2) ... (p , 2p), (1, p+1) ... (p, 2p)(2p+1, 2p+2), ... , (1, p+1) ... (p, 2p)(2p+1, 2p+2) ... (2p+2v-1, 2p+2v),where v = [ 2 .

Case (1) a• w-id, W= p -~ 1 p+22...... p2 p p+l 1 ... p2p 2p+1 .., n

~11 K12 X13 X'22 X2123

gu X = X21 X22 X23 -p 6(X) _" K12 K11 n13 31 32 X33 X32 X31 X33 Double coset decompositions 27

Llu -- Llu Tw = T , Tw = {Dlag [el, e2, ... en~

whose determinant =1, ez = ±1}

Jpq • wa = Diag [E1, ... } ep, ep+1... ' e2p' e2p+1 ... ' en]

where e2= ±1. If wa • Ad wa is conjugate to 6 under Ad Gu. There- fore the real semi-simple algebra which o • Ad wa defines is isomorphic to g. Let gu = gu -1-gu where gu = {X ~ gu, 6 • Ad wa X = X }, gu = {X ~ c, 1 • Ad wa X = -X}. Suppose the number of e~ which is equal to -1 is m, dim gu = m2 +(n-m)2-1. On the other, dim = p+(n-p)2-1. If dim gu = dim , m.= p or n-p. As the determinant of wa is equal to 1.

(_ 1)pe~e2... en =1, (-1)'--m = 1, p+m - 0 mod (2Z) n=odd= m=p, n=even= m=p or m=n-p.

Jpq • wa is conjugate to Jpq under Gu when and only when wa Do. This is equivalenttom = p. Thereforewao m= p. Thetotal number = (). n Case (ii) a • w = (1, pal) (r, per) 1 < r _

- 12 ... r r+1..• p p+1... p4-r p+r+1... 2p 2p+1... n w- ... r p+r+1... 2p p+1..i p+r r+1... p 2p+1 ... n

p 28 K. AOMOTO

u = C~-l ti ... '/--1 tr V-1 tr+i ... ~/--1 tp s-1 ti ... /_l tr \/--l tp+r+i ... ~/-1 t2p ~/--1 t2p+i ... v' l tn] V_ 1 tr O ... 0 _ v_ Lti ... __ 's/__ l tr 0 ... 00... 0]. Any element a' of Tw is congruent to (*) a = Diag[1~ sr+i... sp1 ...1 sp+r+i... ~2pE2p+i ... ~n~ r r mod Tw a2 =1. None of these two elements are congruent mod T;. Double coset decompositions 29

(-1)r+i ... Ep Ep+r+i ... SC2p~2p+i ... ~~= 1. Let m = the number of Ei which is equal to -1. Then, dim gu = (r+m)2 +(n-r-m)2-1, (-1)n+p-r =1, m+ p-r . 0 mod 2Z where t+; = gu is a Cartan decomposition with respect to Jpq • Ad wa. If dim gu = dim , rim = p or n-p, that is, m = p-r or m = n-p-r. The signature of the symmetric matrix Jpq • wa must be equal to that of Jpq, which occurs only when rim = p. Thus wa E ?1Toof the form (*) r+m = p (m = p-r). The number of such elements = (n_2r)p -r Case (iii) • w=(1 , pal) (p, 2p)(2p-1, 2p+2) ••• (2p+2r-1, 2p+2r) r>_ 1

W 12 ••• 2p 2p+22p+1 2p+12p+2 ••• 2p+2r2p+2rH-12p+2r 2p+2r+1 30 K. AOMOTO

1t2p+1 Ju = {Diag [1/-1 t1 ••. ~/-l tp-l ti ... ~/-l tp ~/-1 t2p~-1~l-

"' ~l-1 t2p+2r-1 / -1 t2p+2r-1 ~ 1 t2p+2r+1'" ~/-1 tn]}

For any wa Double coset decompositions 31

The signature of J • wa is not equal to that of J. Therefore wa E ?PO. The total number of [?'o] is equal to

n+ n n-2 ++ 1 n n-2 (n-2r+2 (n-2r p 2)p-1) r! 2 2 2 p-r

+ ... + _1- n n-2 ... (n-2p+2\_ ------n p! 2 2 2 r=o 2p-rr ! (n-2p+r) !.

We take, as a Borel subgroup B, the subgroup of elements of the form :

The part of oblique lines is not zero, the remaining part is zero.

_ CEa MR= {Diag [t1+u1, ... , tp+up, -t1 u1, ... , -tp+up, u2p~1,... , un] cEP

p q 2 u+ u2p+1= 0, tj, uj real} , j=1 j=1

P= l(ti+ui)-(tj+uj) 1 c i

ii Diag [t1+u1i ... , tp+up, -t1+u1, ... , -tp+up, u2p+1, ... , un~ = Diag [t1-u1, ... , tp-up, -t1-u1, ... , -t--u, u2p 1, ... , un~

N IjCc AC, nC= NC, nC= NC, B = AC • NC, TtC= CE_a. If in (44), n1 =0, the w • 6 aEP transforms P into -P. Let w • a = zo on h = ~/-1 f~, v0(P) _ -P, Then w • _ -zo on Iju, where o' is the restriction of Q to gu. Because w • 6 is con- tained in W, -zo E W. On the other -v0(P) = P. Therefore -zo =1 and w • Q =1, w = 6 on i5,, 0.. w -=1. The elements of [?'o] corresponding to this w,have been calculated inCase (i), and the number isequal to (). n There- forethe number ofopen orbits is equal to (i). 32 K. AoMoTo

4. GC= Sp(v, C), G _ S2,(y,R), Gu = SUp(v),c={(u1 -tX12; XilE ~Y(v, X 21 X 11 C); X12,X21 ESy(v, C)}, U : X g_ {(u1X X ~ X21 _ X1112 , X11 R); X12,

X21ES/i, R)}, = X12X11 X11X12 ; X11 O(v), X12 Sy(v, R)}, = {(iiX12X X11i2); X11,X12 ESy(v, R)}, ~) = ~1u= {Diag [t1, ... , tv,-t1, .•.,-tv], t~E R}, hu = W (Weylgroup) = Cw • (Z2)v (semi-direct, while Cw is the symmetric group of order v, (Z2)v is the direct sum of v cyclic groups of order 2). P • A E W operates on t as follows : P . A (t1, t2, ... , tY) = (e 1t11, s~2t12,... , ~iytiY) where P • (t1, t2, ••• > t)=(t21, v t22, , t), 2 v A • (t1, t2, ••• , t)=(1 v s t1, 2t2, ••• , Y t Y) e=±1. P-1 AP (t1, t2, ... , tv)= (t1, fi22t2,... , Es~tv),putting ...~v -1 P-1 = =1, 2,... , v, ... ~ZY ... ~lv t s H= Diag [s/-1 t2, , js/-1 tv, -/f t1, •., /-1 tv] H (t1, t2, ... , tv) (coordinate of H). Suppose (P. A)2=1, then PAPA=P2P-IAPA=1, therefore P2=1, P-1APA=1. P is written as the sum of transpositions, for example, P _' (I1, 2)(j3, 14), , (12r-1,12r).

(P-1AP. A)(t1, t2, ... , t)=(s2.1s1t1, v 1 s 22~s2t2, ... , s2.~sY v t)=(t1,v t2, ... , t v )-~ Consequently, =1, s12s2=1, ... , s~~sv=1. Since P 2 =1, i1 .- il, i2 = i2, • • • , 2Y = 2 y, and so szsss =1 1 < s < v. Thus Ej1 = E92, x13-E14, ~~2r-1_ ~J2r and for other indices j, s;-arbitrary. The characterization of conjugate classes of order 2 in W is the following (see [18]) : Let the compliment of the set {I1, 12, 13,j4, , jar-1, 12r} in {1, 2, , v} be {k1, k2, , -2r}, and the number of sks 1 < s -<_ v-2r which are equal to 1 be p. If PA and P'A' in W are conjugate to each other, then (r, p) coincides with (r', p'). Conversely if (r, p) = (r', p'), then PA and P'A' are conjugate in W. We choose representative in each conjugate class, P=(12)(34) (2r-12r) A=(1...1 1 ••• 1)

2r v-2r+1 A=(1...1 1 ••• -1)

2r A=(1...1 -i..' -1) 0_<2r<_v.

2r

Now we will ask if Do coincides with 0. Double coset decompositions 33

Cartan subalgebra of g and lies in . The analytic subgroup Tu corresponding to this i' is

Since Z (centre of Gu) is contained in Tu, TuZ n 0 = Tun 0. Let E Tun 0, them =1, has the following form : = Diag [± 1, ± 1, •• • , ± 1, +_1, • • , ± 1] where i-th and (i+v)-th elements are equal. On the other hand, 34 K. AOMOTO

We can write in the form exp X, by taking BZ= it or 0. Therefore by Proposition 6, 0 = 6. Now, to return to the point, we take t, =

o(H)=-H for any HEto.

For w E W, (aT• w)2 =1 is equivalent to w2 =1. i • w is conjugate to a • wl if and only if w is conjugate to w1. In case a • w(tl, ... , tv) = (t1, t2, ... , tv), w(t1, ... , t)=(-t1,v--t2, ... , -tv), Double coset decompositions 35

u =tju, bu ={0}, Tw=T, Tw={1},Tu ={Diag[±1 ±1 ••• ±1 ±1±i•••±1]

v v i-th and i+v the elements are equal}. None of elements of Tw is not congruent with each other. Therefore [Y''0]= [?'] D Tw. The total = 2. . This is just equal to the number of open orbits in B\GC. In most general case : 2r+ p <_v , w(H) = (t2,tlt tott3, ... , t2r,tir-1, t2r+1,, ,, t2r-hA,--t2r+A+1, ... , -tY)

2r p w(H) `_ (- t2, tot .-' t3, ... , -- t2r, _ t2r-U _ t2r4.1,... , - tir+A, tir4.p+U... tY) Illu ` {(tl, tU t3, t3, ... , t2r-1, t27-1, t2r+1, ... , t2r+A, ... 0)} ,

2r p v-2r- p lJZ - {(t1, t1, t3, _ t3, ... , t2r-1, -` tir-1, 0 ... 0, tir+A+1,... , t1)} ,

2r p v-2r- p 36 K. AOMOTO

2r p Tw = {Diag ~2, ~2, ... ,fir, fir, &2r+1,... , ~2r+P,1 ... 1 E1, E2,E2, ,fir, fir, ~2r+1, , ~2r+P,1 ... 1] } e j = ± 1 .

Any element of Tw is congruent to one of those elements

Diag [1,1 ... 1, ~2r+17... , ~2r+P,1... 1,1... 1, ~2r+1,... , E2r+P,1...1] , 2r v-2r-p 2r which are not congruent among themselves. Therefore the number of T wn [?o] is equal to 2. . The total

= 1 v v-2 v-2r-~2 x v-2r X 2p= v! 2p r! ^ 2 2 2 p r! 2rp! (v_2r- p) !' Finally, we have obtained our conclusion that the number of [?P is equal to

f v-2r v 12P [+1 v! 3v-2r P E r ! 2rP!(v-2r-- P) ! = r ! 2r(v2r) -- ! where H(z) is the Hermite polynomial of degree v. 5. GC= {g SL(n, C) ; tgfg =J} n = 2v+1= 2p+q, Gu= GCn SU(n), G = {g E GC; tgJpgg=Jpq}, where Double coset decompositions 37

qC= {X r(n, C), tXJ~JX = o}, 6 (involutive automorphism of g) : X --~JpgXJpq(X E gu), X11 ~1 X12 ~u= qCn ~1(n)= { -t~l 0 -'tl ; X11 it(), X12E O(v, C), ~1E CV} X12 ~1 - cX11

q=+p, Y11 Y12 1 "13 Y12 -tY12 Y _ Y11,Y13 22 '12 tY12 Y24 Y12E JJ (p, )-p, C), ={X= -tLl1 tL12 0 tL/I _t/2 t 1 E lgyp,L/2 E Cv-p, Y13 Y12 t -2Y11 712 Y22E _ Y24E O(v-p, C)}. `tY 12 Y24 L)2 tY12 tY22

4J-1 Y11 Y12 / 1 t1 /_'1 713 -Y12 __t712 0 0 t712 0 -/--1~= {X= /-1t~1 0 0 -/-1tL/1 0

-/-i Y13 -Y 12 v'-1, v -1' 711 712 t Y 12 0 0 -t Y12 0 Y11E Sy(p, R), Y12E JJ1(p,v-p, C), Y13E o(p) , t)1E Rp} ~/ Tf (=maximal abelien subalgebra of ~/-1~) 38 K. A,oMoTo

= {Diag [V-1t1, ,'s,/-1t, 0 ... 0, 0, - V--- t1, ... , -V'-1t, p0 0J}.

p q-1 p q-1 2^ 2

u =1 +'V-1 tj~ _ {Diag [V-1 t1, ... , V -_1 tv, 0, -'/-1 t1, ... , -'/-1 t~] tjER}. For H= Diag ['/-1 t1, ,'V-1 tY, 0, -V-1 t1, , -V-1 t j E lju whose coordinate is (t1, t2, , t~) E R~, o(H) = (-t1, , - tp, tp+1, , tv). The Weyl group is the same as that of C, type. For any w E W, 6 • w E W, and so 6 • w and w can be written in the following way : a • w = P' • A', w = P • A, P, P' (= symmetric group of degree v) ; A, A' E Z2 (v products of cyclic groups of order 2). The conjugate classes of a 'iv of order 2 are : a • w = P'. A', P' = (12)(34) ... (2r-1, 2r) A' =(1 ••• 1, -1 ••• -l,1 ••• 1) E Z2 ,

2r s v -2r -s that is, aw : (t1, t2, ... , t2r-1, t2r, t2r~U ... , t2r ~s, ... , ty) (t2, t1, t4, t3, , tv) ,

Tw = (A1, 21, ... , 2r, 'ar, 1 ... 1, "2r+S+1,... , ~v) , - S v-2r-s Tw = (21, Aj-1,... , Ar, 21, , A2r+1,... , 22r+s,1... 1).

S v-2r-s

Each element of Tw is congruent mod Tw to one and only one of the elements :

(**) t = (1 ... 1,1 ... 1, n2r+SF1, ... , Aj) A~=1, 2r+s < j < v .

2r s v-2r-s We consider three cases, separately, case a) p >_2r+s, b) 2r+s>p > 2r, c) 2r>p.

X11 0 X12 Case a) w = 0 (-1)p-S 0 , where X11= Diag [0...0, i...1,

2r s X12 0 X11 p-2r-s v-p Double coset decompositions 39

x11 0 x12

to1=a(w)•w=1, [?Ju]Bwt= 0 (_ 1)p-s 0 t, t has the form (**). X21 0 x22 Then Y11 0 Y12

cot Jpq= 0 ~_ 1)p-s 0 where

Y21 0 Y22 4a K. AoMOTo

r Y12= Diag [0 0, 1 ... 1, 0 ... 0].

2r s v-2r-s

Let c be the number of 2~ 2r+s+1 < j<_ v which are equal to -1. The iit •Jpq and Jpq are conjugate with respect to Gu if and only if p = 2r+s+2c (p-s = even) or p = 2r+s+2c+1 (p-s = odd). This can be proved by an ele- mentary method or by the Remark in § 2. Therefore the number of such v-2r--s elements is equal to([P2r5 _ - 2 x11 0 X12

Case b) w = 0 (-1)s_p 0 , where X11=Diag [0 ••• 0,

2r x12 0 X11 p-2r 0...o,1...1], 2r+s-p v-2r-s Double coset decompositions 41

Y11 0 Y12 tp 1= a(w)w = 1, (u t Jpq = 0 ~_ 1)S_p 0 where

Y12 0 Yll 42 K. AoMoTo

Y12=Diag [0 ••• 0, 1 ••• 1, 0 ••• 0].

2r s v-2r-s In this case, any c >_ 0 can not satisfy 2r+s+2c = p (s-p= even) or 2r+s-F2c +1 = p (s-p = odd). Therefore, no such wt lies in ?PO. Case c) For wt E [?], Y11 0 Y12 wt•Jpq= 0 s 0 , s=1 or --1 Y12 0 Y22 according as p = even or odd respectively.

Y12=Diag [0 ••• 0, 1 1, 0 01.

2r s v-2r-s

In this case, also, wt •Jpq is not conjugate to Jpq• Therefore, wt •Jpq Thus, the number of [?o] is equal to : v-2r-s v ! v-2r p 2r--s X r ! 2r(v-2r) x s r,sz0

If 6•w=1 r = s =0, and therefore the number of open orbits is equal to v tr Double coset decompositions 43

G is not connected. Let Go be its connected component of the identity. Then [G : Go] = 2. G = G U 1G0 for a suitable r G. But a fundamental Cartan subgroup S of Go is compact and is also a maximal torus of Gu. Open orbits are all isomorphic to S \G = S \G0 U r-1Sr\Go (disjoint sum), so that the v numberof openorbits of Go in B\GC is Cp jX2. 2~ In particular, if p =1, there is only one G-open orbit which is separated into two Go-orbits. This manifold seems to be closely related to the discrete series of infinitesimal irreducible unitary representations of Go constructed by T. Hirai [20].

University of Tokyo

References

[ 1 ] A. Borel, Kahlerien cosec spaces of semi-simple Lie groups, Proc. Nat. Acad. Sci. U.S.A., 40 (1954), 1147-1151. [ 2 ] A. Borel and F. Hirzebruch, Characteristic classes and homogeneous spaces, I., Amer. J. Math., 80 (1958), 458-538. [ 3 ] R. Bott, Homogeneous vector bundles, Ann. Math., 66 (1957), 203-248. [4] F. Bruhat, Sur les representations induites des groupes de Lie, Bull. Soc. Math. France, 84 (1956), 97-205. [5 ] C. Chevalley, Sur certains groupes simples, Tohoku Math. J., 7 (1955), 14-66. [6] F. Gantmacher, Canonical representations of automorphisms of a complex semi- simple Lie group, Rec. Math. (Mat. Sb.), 5 (1939), 101-146. [7] F. Gantmacher, On the classifications of real simple Lie groups, Rec. Math. (Mat. Sb.), 5 (1939), 217-250. [81 F. Gantmacher, The theory of matrices I, II, Chelsea Publishing, New York, 1959. [9] Harish-Chandra, Representations of semi-simple Lie groups, IV, V, VI, Amer. J. Math. 77 (1955), 743-777, 78 (1956), 1-41, 564-628. [10] Harish-Chandra, Fourier transforms on a semi-simple Lie algebra I, II, Amer. J. Math., 79 (1957), 193-257, 653-686. [11] M. Gelfand and M. I. Graev, Unitary representations of real unimodular groups, Izv. Acad. Nauk SSSR. Ser. Mat., 17 (1953), 189-248, (Russian). [12] M. I. Graev, Unitary representations of real simple Lie groups, Trudy Moskov, Mat. Ob., 7 (1958), 333-389, (Russian). [13] S. Helgason, Differential geometry and symmetric spaces, Academic Press, 1962. [14] N. Iwahori, Lie algebras and Chevalley groups, Lecture note given at Uni- versity of Tokyo, 1964, mimeographed. [15] B. Kostant, Lie algebra cohomology and the generalized Borel-Weil theorem, Ann, of Math., 74 (1961), 329-387. [16] H. Matsumoto, Quelques remarques sur les groupes algebriques reels, Proc. Japan Acad., 40 (1964), 4-7. [17] S. Murakami, On the automorphisms of real semi-simple Lie algebras, J. Math. 44 K. AOMOTO

Soc. Japan, 4 (1952), 103-133; 5 (1953), 105-112. [18] W. Specht, Gruppentheorie, Springer, 1956. [19] M. Sugiura, Conjugate classes of Cartan subalgebras in real semi-simple Lie algebras, J. Math. Soc. Japan, 11 (1959), 375-434. [20] T. Hirai, On infinitesimal operators of irreducible representations of the Lorentz group of n-th order, Proc. Japrn Acad., 38 (1962), 83-87.