A CARTAN DECOMPOSITION FOR NON-SYMMETRIC REDUCTIVE SPHERICAL PAIRS OF RANK-ONE TYPE AND ITS APPLICATION TO VISIBLE ACTIONS ATSUMU SASAKI Abstract. A Cartan decomposition for symmetric pairs plays an important role to study not only orbit geometry of the symmet- ric spaces but also harmonic analysis on them. For non-symmetric reductive pairs, there are examples of generalizations of Cartan de- compositions for some spherical complex homogeneous spaces such as complex line bundles over the complexified Hermitian symmet- ric spaces and triple spaces. This paper provides new examples of a Cartan decomposition for non-symmetric reductive pairs, namely, reductive non-symmetric spherical pairs of rank-one type. We also show that the action of some compact group on a non-symmetric reductive spherical homogeneous space of rank-one type is strongly visible. 1. Introduction Let G be a connected real semisimple Lie group and H a connected closed subgroup of G which is reductive in G. We take a Cartan in- volution θ of G satisfying θ(H) = H and set K := Gθ. Then, K is a maximal compact subgroup of G. Let g0, k0 and h0 be the Lie algebras of G, K and H, respectively. We write g0 = k0 + p0 for the correspond- ing Cartan decomposition of the Lie algebra g0 and g0 = h0 + q0 for the direct sum decomposition with respect to the Killing form of g0. Then, g0 is decomposed into the direct sum as follows: g = k h + k q + p h + p q . 0 0 ∩ 0 0 ∩ 0 0 ∩ 0 0 ∩ 0 arXiv:1807.11664v3 [math.RT] 15 Oct 2018 In this setting, if there exists an abelian subspace a0 in p0 q0 such that ∩ (1.1) G = K(exp a0)H, then we say that (1.1) is a Cartan decomposition for the pair (G,H), or for the homogeneous space G/H. 2010 Mathematics Subject Classification. Primary: 22E46, Secondary: 32M05, 22E60, 14M17. Key words and phrases. Spherical pair; Cartan decomposition; Cayley algebra; exceptional Lie group. This was partially supported by Grand-in-Aid for Scientific Research (C) (No. 15K04797), Japan Society for the Promotion of Science. 1 2 A. SASAKI If H coincides with K, then (1.1) is a usual Cartan decomposition. In the case where (G,H) is a symmetric pair (note that H is not nec- essary compact), the decomposition (1.1) holds (cf. Flensted-Jensen [4] and Rossmann [13]). Recently, we have investigated a decompo- sition GC = GuBHC with an abelian B in GC for some complex ho- mogeneous spaces GC/HC which are not symmetric but spherical such as SO(4n +2, C)/SL(2n +1, C), E6(C)/Spin(10, C) ([15, 18]), triple space (G′ G′ G′)/G′ where G′ = SL(2, C) ([3]), and with a subset × × B such as SL(2n +1, C)/Sp(n, C) ([16]) and SO(8, C)/G2(C) ([17]). We expect that non-symmetric reductive real spherical pairs (G,H), which are in connection with finite-multiplicity property of represen- tations (see [8] and references therein), have a Cartan decomposition (1.1) for some abelian a0. From this viewpoint, the author is studying a Cartan decomposi- tion (1.1) for reductive spherical pairs. This class contains complex symmetric pairs. Then, our interest is non-symmetric spherical ones. Here is a quick review on reductive spherical pairs. Let GC be a connected complex semisimple Lie group and HC a complex closed subgroup of GC. The pair (GC,HC) is called spherical, or the complex homogeneous space GC/HC is spherical, if a Borel subgroup of GC has an open orbit in GC/HC. The classification of reductive spherical pairs (GC,HC), namely, it is spherical and HC is a reductive Lie group, has been given by Kr¨amer [9] when GC is simple and by Brion [2], Mykytyuk [12] when GC is semisimple. In this paper, we consider non-symmetric reductive spherical pairs (GC,HC) which are of rank-one type, namely, (GC,HC) satisfies one of Table 1.1. Type GC HC R-1 SO(7, C) G2(C) ′ R-1 Spin(7, C) G2(C) R-2 G2(C) SL(3, C) Table 1.1: Non-symmetric reductive spherical pairs of rank-one type Now, we remark on Type R-1 and Type R-1′. The double cover- ing group homomorphism Spin(7, C) SO(7, C) induces the double → ′ covering map from Spin(7, C)/G2(C) (Type R-1 ) to SO(7, C)/G2(C) (Type R-1). Since the definition of spherical homogeneous space is a local condition, it is sufficient for the classification to consider spher- ical homogeneous spaces up to coverings. In the study of (1.1), our argument for Type R-1′ is parallel to that for Type R-2, and our main result for Type R-1′ provides that for Type R-1. For this reason, we will also treat Type R-1′ in this paper. DECOMPOSITION FOR RANK-ONE REDUCTIVE SPHERICAL PAIRS 3 The terminology ‘rank-one type’ comes from the geometric sense, namely, the corresponding homogeneous space is of rank one. More- over, it can be explained from the representation theory. By Vinberg– Kimelfeld [20], a reductive pair (GC,HC) is spherical if the space C[GC/HC] of regular functions on the complex homogeneous space GC/HC is multiplicity-free as a representation of GC, and vice versa. The set of highest weights of GC occurring in the multiplicity-free irreducible decomposition of C[GC/HC], called the support of GC/HC, is a semi- group. If (GC,HC) is one of Table 1.1, then the rank of the support equals one (see [9]). Now, let us explain our main result. Let (GC,HC) be a reductive spherical pair of rank-one type and θ a Cartan involution of GC satis- fying θ(HC)= HC. Let g, h be the Lie algebras of GC,HC, respectively. We use the same letter θ to denote the differential automorphism on θ θ g. We set gu = g and Gu := GC. Then, gu is a compact real form of g and the Lie algebra of Gu. The corresponding Cartan decompo- sition of the Lie algebra g is given by g = gu + √ 1gu. Let q be the orthogonal complement of h in g with respect to the− Killing form of g. Under the setting, we give new examples of a Cartan decomposition (1.1). Namely, we prove: Theorem 1.1. Let (GC,HC) be a non-symmetric reductive spherical pair of rank-one type. Then, one can take a one-dimensional abelian subspace a in √ 1g q such that 0 − u ∩ GC = Gu(exp a0)HC. The purpose of this paper is to provide an explicit description of a0 satisfying Theorem 1.1 under the realization of GC and HC as matrix groups. Thanks to Theorem 1.1, we find strongly visible actions on non- symmetric complex reductive homogeneous spaces. We state: Theorem 1.2 (see Theorem 6.1). Let (GC,HC) be a non-symmetric reductive spherical pair of rank-one type. Then, the Gu-action on the complex homogeneous space GC/HC is strongly visible (see Section 6 for the definition). This paper is organized as follows. In Section 2, we explain a matrix realization of Lie groups which appear in Table 1.1. In Sections 3–5, we prove Theorem 1.1 for each reductive spherical pair of rank-one type by giving an explicit description of an abelian group A. In particular, we deal with Type R-2 in Section 3, Type R-1′ in Section 4 and Type R-1 in Section 5. In Section 6, we show Theorem 1.2, in particular, we explain that Theorem 1.1 is an essential part of our proof of Theorem 1.2. The author would like to express his thank to an anonymous referee for careful comments and suggestions. 4 A. SASAKI 2. Preliminaries 2.1. Realization of exceptional Lie group of Type G2. In this subsection, we explain realizations of G2(C), and its maximal com- pact subgroup G2 as subgroups of the special orthogonal group on the complexified Cayley algebra. Let C be a Cayley algebra over R. This algebra C has the stan- dard basis e0,...,e7 with e0 as the unit element of C such that the following relations{ hold,} which we will fix in this paper: e2 = e , e2 = e (i =1, 2,..., 7), e e = e e = e , 0 0 i − 0 1 2 − 2 1 3 (2.1) e1e4 = e4e1 = e5, e1e6 = e6e1 = e7, e2e5 = e5e2 = e7, e e = −e e = e , e e = −e e = e , e e = −e e = e . 2 6 − 6 2 4 3 5 − 5 3 6 3 4 − 4 3 7 Let ( ,) be an inner product on C satisfying (e , e )= δ (0 i, j 7). · · i j ij ≤ ≤ We denote by Re(C)= Re0 the real part of C by Im(C)= Re1+ +Re7 the imaginary part of C. Then, Im(C) is the orthogonal complement··· of Re(C) in C with respect to ( , ). We identify the special orthogonal· · group SO(C, ( , )) with SO(8) = g SL(8, R): tgg = I where tg denotes the transposed· · matrix of { ∈ 8} g and I8 the unit matrix. Similarly, we have SO(Im(C), ( , )) = g SO(C, ( , )) : ge = e SO(7). We define G as the automorphism· · { ∈ · · 0 0} ≃ 2 group AutR(C) of C, namely, G = g SO(8) : (gx)(gy)= g(xy) (x, y C) . 2 { ∈ ∈ } Then, G2 is a connected and simply connected compact simple Lie group of exceptional type with Lie algebra g2.