Borel Orbits and Invariants of Classical Symmetric Subgroups on Multiplicity-Free Grassmannians (I)

Borel Orbits and Invariants of Classical Symmetric Subgroups on Multiplicity-free Grassmannians (I)

Huajun Huang

Department of Mathematics and Statistics, Auburn University, AL 36849, USA Email: [email protected]

Abstract Let K be a symmetric subgroup of a complex classical group G. Let BK denote a Borel subgroup of K , and PG a parabolic subgroup of G. We completely classify the BK orbits and invariants on Grassmannians of

G = GLn(C), for (1) K = GLn1 (C)×GLn2 (C) with n1 +n2 = n, (2) K = SOn(C), and (3) K = Spn(C)(n even). The work forms the framework of classifying the other BK -orbits on K multiplicity-free flag varieties G/PG . An extension of Witt’s theorem is discovered as a byproduct, which is useful in constructing PK elements for K = SOn and K = Spn (n even). AMS Classification 14M15, 14L24, 15A72 Keywords Borel orbit, invariant, flag variety, Bruhat decomposition, Witt’s theorem.

1 Introduction.

1.1 Backgrounds.

The family of multiplicity-free representations is a basic research topic in the representation theory and invariant theory. It includes many important examples with nice geometric structures and combinatorial properties [1, 7]. Fix an algebraic closed ﬁeld of characteristic not 2. Let K be a connected reductive linear algebraic group. Denote by BK a Borel subgroup, and PK a parabolic subgroup, of K . A regular K -action on a variety X is a group homomorphism ρ : K → Aut(X), which is also a regular map of varieties. It induces a regular K -representation on the regular function ring R(X) of X : ρ∗(g)(f(x)) = f(g−1(x)), g ∈ K, f ∈ R(X), x ∈ X. The K -action ρ is multiplicity-free if ρ∗ is a multiplicity-free representation, that is, ρ∗ can be decomposed into a direct sum of distinct (up to isomorphisms) irreducible representations. From highest weight theory, a K -action on variety X can be determined by the induced BK -action on X . Moreover, there is a neat result relating multiplicity- free K -actions on X to the BK -orbits on X :

1 Theorem 1.1 (Servedio [15], Vinberg [16]) Let a connected reductive linear algebraic group K act regularly on an irreducible aﬃne variety X . The following statements are equivalent: (1) The K -action is multiplicity-free.

(2) There are ﬁnitely many BK -orbits on X .

(3) There exists a Zariski dense and open BK -orbit.

Suppose K is the symmetric subgroup of a connected reductive algebraic group G, and X = G/PG is a flag variety (which is projective) of G. If K acts multiplicity-freely on X , then it acts multiplicity-freely on the affine line bundle of X . By Theorem 1.1, K acts multiplicity-freely on G/PG if and only if there are finitely many BK -orbits on G/PG , if and only if there is a Zariski dense open BK -orbit on G/PG . It motives us to investigate the geometric properties of these BK orbits. Another motivation comes from the fact that the structure of BK \G/PG is closely linked to both the Bruhat decomposition BG\G/BG and the Iwasawa decomposition K\G/BG . To some extend BK \G/PG reflects the mixture of these two decompositions. Let G be a complex classical group. The triples (G, K, X) where

(1) X = G/PG is a ﬂag variety of G. (2) K is a symmetric subgroup of G. (3) K acts multiplicity-freely on X . have been classiﬁed by Horvath, Howe, and Wallach [6] (cf. Table 1). This paper begins a series whose goal is to classify the BK -orbits and invariants on X = G/PG for the triples (G, K, X) above. The present paper treats three basic cases (cf. (1.2)) which will be used to understand more complicated situations to be done in later papers. Some related double cosets have been studied in [2, 4, 13, 14].

1.2 The list of multiplicity-free ﬂag varieties.

Since the base ﬁeld is C, we use GLn to denote GLn(C), and use GL(V ) to n represent the natural representation of GLn(C) on V := C . Likewise for the other complex classical groups. Let h, i denote a non-degenerate symmetric (resp. symplectic) bilinear form on V preserved by SO(V ) (resp. Sp(V ) when dim V is even). A ﬂag of V is a sequence of nested subspaces in V :

V := {V0 ( V1 ( ··· ( V`}. (1.1)

2 We always assume V0 = {0} for convenience. Call J := {dim Vi | i = 1, ··· , `} the dimension set of V . There are several types of flags corresponding to different classical groups and their parabolic subgroups: (1) The group G := GL(V ) acts transitively on generic flags V of a fixed dimension set J . The collection of these flags forms a flag variety of GL(V ), denoted by F(J, V ) or F(J). A stabilizer of a generic flag in GL(V ) is a parabolic subgroup of GL(V ) and vice versa. So fixing V` = V in V , there is a bijection between the generic flags V and the parabolic subgroups PGL(V ) of GL(V ), by corresponding a flag to its stabilizer in GL(V ). We may identify GL(V )/PGL(V ) with the flag variety of V .

When V is a complete flag, that is, dim Vi = i for i = 0, ··· , ` where ` = dim V in (1.1), the stabilizer of V in GL(V ) is a Borel subgroup BGL(V ) of GL(V ). When V = {{0} ( E ( V } contains exactly one nontrivial proper subspace, the flag variety of V is called a Grassmannian of GL(V ), denoted by Gr(dim E,V ) or Gr(dim E, n). We may identify E with V = {{0} ( E ( V }. (2) When G := SO(V ) is an isometry group preserving a symmetric h, i, we ⊥ call V in (1.1) a self-dual flag if Vi = V`−i for i = 1, ··· , `, and call V an isotropic flag if V` is isotropic, namely hv, wi = 0 for v, w ∈ V` . There is a bijection between isotropic flags and self-dual flags of V , by viewing an isotropic flag as “the first half part” of a self-dual flag. The group SO(V ) acts transitively on the set of isotropic flags of a fixed dimension set J , that is, the isotropic flag variety Fo(J, V ) or Fo(J, n). The stabilizer of an isotropic flag V in SO(V ) is a parabolic subgroup PSO(V ) of SO(V ). So we identify SO(V )/PSO(V ) with the isotropic flag variety of V . A complete self-dual flag is a complete (generic) flag which is self-dual. A complete isotropic flag is “the first half part” of a complete self-dual flag. The stabilizer of a complete isotropic flag or a complete self-dual flag in SO(V ) is a Borel subgroup BSO(V ) of SO(V ). When an isotropic flag V = {{0} ( E} contains exactly one nontrivial proper subspace, the isotropic flag variety of V is called an isotropic Grassmannian of SO(V ), denoted by Gro(dim E,V ) or Gro(dim E, n). We may identify E with V = {{0} ( E}. (3) When G := Sp(V ) is an isometry group preserving a symplectic h, i, similarly we can define isotropic flag, self-dual flag, complete isotropic flag and complete self-dual flag. The isotropic flag variety of dimension set J is denoted by Fs(J, V ) or Fs(J, n). We may identify Sp(V )/PSp(V ) with an isotropic flag variety. The stabilizer of a complete isotropic flag or a complete self-dual flag in Sp(V ) is a Borel subgroup BSp(V ) of Sp(V ). The isotropic Grassmannian of an isotropic flag {{0} ( E} for Sp(V )

3 is denoted by Grs(dim E,V ) or Grs(dim E, n). We may identify E with V = {{0} ( E}.

Table 1 lists all multiplicity-free K -actions of irreducible classical symmetric pairs (G, K) on ﬂag varieties X = G/PG , classiﬁed by Horvath, Howe and Wallach (the exceptions G = K × K for simple classical K were partially studied by Littelmann [11]). Denote the block sizes of F({d1, ··· , d` = n}) by the sequence (d1, d2 − d1, ··· , d` − d`−1). They are the diagonal block sizes in matrix representation of PGLn where GLn/PGLn = F({d1, ··· , d`}).

Table 1: K -multiplicity-free Flag Varieties X for (G, K, X)

Symmetric Pairs X = G/PG ∈ {F(J), Fo(J, n), Fs(J, n)}, (G, K) J = {d1, ··· , d`}, 0 < d1 < ··· < d` ≤ n.

n1 = 1 All ﬂag varieties F(J).

(GLn, GLn1 × GLn2 ) n1 = 2 Gr(d1, n); F({d1, d2, n}). n1 ≤ n2, n1 + n2 = n. n1 ≥ 3 Gr(d1, n); F({d1, d2, n}) with at least one block size equal to 1. (GLn,SOn) Gr(d1, n). Gr(d1, n). (GLn, Spn)(n even) F({d1, d2, n}) with at least one block size equal to 1. F({d1, d2, d3, n}) with at least three block sizes equal to 1. n1 = 1 All isotropic ﬂag varieties Fo(J, n). n (SOn,SOn1 × SOn2 ) n1 = 2 Gro(d1, n); Fo({d1, 2 }, n) when n is even. n n1 ≤ n2, n1 + n2 = n. n1 ≥ 3 Gro(1, n); Gro(b 2 c, n). n1 = 2 Grs(d1, n); Fs({d1, d2}, n).

(Spn, Spn1 × Spn2 ) n1 = 4 Grs(d1, n); Fs({1, 2}, n). n n1 ≤ n2, n1 + n2 = n, n1 ≥ 6 Grs(d1, n) with d1 ≤ 3; Grs( 2 , n); n1 and n2 are even. Fs({1, 2}, n). n (Spn, GLn/2)(n even) Grs(1, n); Grs( 2 , n). n n (SOn, GLn/2)(n even) Gro(d1, n) with d1 ≤ 3; Gro( 2 −1, n); Gro( 2 , n); n n n Fo({1, 2}, n); Fo({1, 2 }, n); Fo({ 2 − 1, 2 }, n).

The (Grassmannians / isotropic Grassmmanians) constitute a major part in Table 1. In particular, all Grassmannians Gr(d, n) are K -multiplicity-free for

4 G = GLn and the following K :  GL × GL with n + n = n,  n1 n2 1 2 K = SOn, (1.2)  Spn with even n.

It turns out that the orbit structures and invariants of BK \X for the other (G, K, X) triples in Table 1 are closely related to these three families. So it is of fundamental importance to study these three families ﬁrst. Acknowledgement: The paper origins from part of the author’s PhD dissertation. The author is grateful to his advisor Roger Howe for insightful directions. The author is indebted to referees for many valuable remarks, to Tin-Yau Tam for his helpful suggestions.

2 Preliminaries.

2.1 Basic settings.

For convenience, given p ∈ N, we set 0 Np := {1, ··· , p} and Np := {0, ··· , p}. (2.1)

n Let G be a group acting on V := C with a given basis Γ. We use [g]Γ to denote the matrix representation of g ∈ G, and use [v]Γ to denote the coordinate vector of v ∈ V with respect to Γ. If M ∈ Cn×d has full column rank, and a d-space E ⊆ V is the column space of M with respect to basis Γ, then we denote [E]Γ = Col(M).

Obviously Col(M) = Col(Mg) for every g ∈ GLd . Thus Gr(d, n) can be n×d identiﬁed with a subvariety of C /GLd . Note that the Pl¨ucker coordinates of E ∈ Gr(d, n) can be derived from [E]Γ = Col(M) by collecting the determinants of d × d submatrices of M . When V = U ⊕ W , the projections of v ∈ V onto U and W components are

PU v ∈ ({v} + W ) ∩ U, PW v ∈ ({v} + U) ∩ W. (2.2)

Clearly v = PU v + PW v. Similarly for PU E and PW E provided E ⊆ V . Given a nondegenerate, symmetric or symplectic bilinear form h, i on V , there is an induced orthogonal relation on V : for u, w ∈ V , u ⊥ w ⇐⇒ hu, wi = 0. (2.3)

5 Deﬁne the bilinear form matrix of two vector sequence Γ1 = (u1, ··· , up) and Γ2 = (w1, ··· , wq) by hu , w i B(Γ1;Γ2) := i j p×q . (2.4)

In particular, the Gram matrix B(Γ1;Γ1) is abbreviated as B(Γ1). Then T B(Γ2;Γ1) = ±B(Γ1;Γ2) , where the sign depends on whether h, i is symmetric (+) or symplectic (−).

For subspaces F1 = span Γ1 and F2 = span Γ2 , we set

rank hF1,F2i := rank B(Γ1;Γ2), rank Fi := rank B(Γi)(i = 1, 2). (2.5)

The settings are independent of Γ1 and Γ2 since ⊥ ⊥ rank hF1,F2i = dim F1 − dim(F1 ∩ F2 ) = dim F2 − dim(F2 ∩ F1 ).

A basis Γ = {e1, ··· , en} of V is called a self-dual basis if for 1 ≤ i ≤ j ≤ n, ( 1 if i + j = n + 1; hei, eji = δi+j,n+1 = (2.6) 0 otherwise. Set δ sgn (j − i)δ Jn := i+j,n+1 n×n ,Kn := i+j,n+1 n×n , (2.7) where Kn is deﬁned only for even n. Then Γ is a self-dual basis if and only if ( J when h, i is symmetric; B(Γ) = n Kn when h, i is symplectic. The matrix representations of the complex isometry groups with respect to a self-dual basis Γ are: T [SO(V )]Γ = g ∈ GLn | g Jng = Jn, det g = 1 , (2.8) T [Sp(V )]Γ = g ∈ GLn | g Kng = Kn . (2.9)

A basis {e1, ··· , en} and a flag V = {{0} = V0 ( V1 ( ··· ( Vk} of V are compatible, if every Vi is spanned by a subset of {e1, ··· , en}. They are strictly compatible if Vi = span (e1, ··· , edim Vi ) for i ∈ Nk . It is obvious that: (1) For every generic flag V of V , there exists a basis strictly compatible with V ; (2) For every self-dual flag V of V , there exists a self-dual basis strictly compatible with V . So when V is a flag of G = GL(V ),SO(V ), or Sp(V ), the matrix representation of the stabilizer of V in G (i.e. a parabolic subgroup) consists of block upper triangular matrices with the same block sizes as V . In particular, the matrix representation of a Borel subgroup of G consists of upper triangular matrices under a suitable basis.

6 2.2 The Bruhat decompositions.

Let BG be a Borel subgroup of a reductive linear algebraic group G. The Bruhat decomposition of G is a G = BG ω BG

ω∈WG where WG is the Weyl group of G corresponding to the roots in BG .

In vector space V , the Bruhat decomposition BG\G/BG ' WG for a classical group G ∈ {GL(V ),SO(V ), Sp(V )} says that: Given two complete flags (resp. self-dual flags) of G, there is a basis (resp. self-dual basis) of V compatible with both flags. The Bruhat decomposition may be interpreted in many different ways. For example, it may be viewed as the BG -orbit description on the complete flag variety G/BG , which corresponds to the multiplicity-free G-action on G/BG by Theorem 1.1. It may also be viewed as the BG × BG -orbit description on G ' (G × G)/G. Note that G by diagonal embedding forms a symmetric subgroup of G × G. Hence G × G acts multiplicity-freely on (G × G)/G. Theorem 1.1 then implies the Bruhat decomposition, which is a (complexified) Iwasawa decomposition [3]:

BG\G/BG ' BG×G\(G × G)/G.

In particular, the Bruhat decomposition for GLn is equivalent to the LU decomposition. It is the Borel orbit description of the GLn -GLm duality for n = m.

We describe the matrix invariants of the Bruhat decomposition BG\G/PG for G ∈ {GLn,SOn, Spn}. All Borel subgroups of G are inner conjugate to each 0 −1 0 −1 other. If g1BGg1 = BG and g2PGg2 = PG for ﬁxed g1, g2 ∈ G, then there is 0 0 a bijection between BG\G/PG and BG\G/PG given by: −1 0 −1 0 BG g PG ←→ g1 (BG g PG)g2 = BG (g1 gg2) PG.

So we may simply choose BG and PG in the way that BG consists of all upper triangular matrices in G and PG consists of all block upper triangular matrices in G according to a fixed partition γ := (t1, ··· , tk) of n. Given g ∈ Cm×n , let g[ij] be the submatrix formed by the last i rows and the first j columns of g, the “southwest” i × j corner of g. Similarly, let g[ij] be the submatrix formed by the first i rows and the first j columns of g, the “northwest” i × j corner of g. Denote [ij] [ij] r (g) := rank g , r[ij](g) := rank g[ij]. (2.10)

The Weyl group WG of G may be realized as a group consisting of signed permutation matrices.

7 Lemma 2.1 With the above settings, the double coset BG g PG for g ∈ G is uniquely determined by the invariant matrix r[isj ](g) where s := t + n×k j 1 ··· + tj . Let ω be a signed permutation matrix in BG g PG . Then under the row partition 1n = (1, ··· , 1) and the column partition γ, the (i, j)-block of ω has a nonzero entry if and only if (r[isj ] − r[(i−1)sj ] − r[isj−1] + r[(i−1)sj−1])(g) = 1. (2.11) The left side of the above expression equals to 0 if the (i, j)-block of ω has no nonzero entry.

0 [ij] Proof Suppose g = bgp where b ∈ BG and p ∈ PG . Let b be the submatrix formed by the last i rows and the last j columns of b. Then ∗ ∗ ∗ ∗ ∗ ∗ p ∗ ∗ ∗ = [sj sj ] = . 0[isj ] [ii] [isj ] [ii] [isj ] g ∗ 0 b g ∗ 0 ∗ ( b)(g )(p[sj sj ]) ∗

0[isj ] [is ] [is ] Thus rank g = rank g j . This shows the invariance of r j on BG g PG .

If ω ∈ BG g PG is a signed permutation matrix (the existence of such ω is guar- anteed by the Bruhat decomposition), then obviously r[isj ](ω) is the number of nonzero entries in ω[isj ] . So the left side of (2.11) represents the number of nonzero entry (i.e. ±1 entry) in the (i, j)-block of ω according to the row partition 1n = (1, ··· , 1) and the column partition γ. Conversely, if two signed permutation matrices ω1, ω2 ∈ G have the same amount (0 or 1) of nonzero −1 entry in each partitioned block, then ω1 ω2 ∈ G is a block diagonal matrix ac- −1 cording to the row and column partition γ. So ω1 ω2 ∈ PG , which implies that ω = ω (ω−1ω ) ∈ B ω P .Therefore, r[isj ](g) uniquely determines the 2 1 1 2 G 1 G n×k double coset BG g PG .

Some discussions on rank invariants, determinantal ideals and Schubert varieties could be found in [12].

2.3 Types.

Let G ∈ {GL(V ),SO(V ), Sp(V )} and BG the stabilizer of a complete ﬂag or a complete self-dual ﬂag of V :

V = {{0} = V0 ( V1 ( ··· ( Vn = V }.

The BG -orbits on a Grassmannian G/PG (where PG is a maximal parabolic) were classiﬁed by matrix rank invariants in Lemma 2.1. It is convenient to interpret these BG -invariants geometrically when we classify the BG -orbits.

Suppose V is ﬁxed. Let V−1 = ∅. Deﬁne the type of a vector v ∈ V as

TV (v) = i if v ∈ Vi − Vi−1. (2.12)

8 0 Notice that BG stabilizes every Vi . The type TV (v) ∈ Nn is invariant on BG(v) ⊆ Gr(1,V ). Likewise, deﬁne the type of a subspace E ⊆ V as

TV (E) = {i ∈ Nn | E ∩ Vi 6= E ∩ Vi−1} = {TV (v) | v ∈ E − {0}}. (2.13)

Clearly TV (E) ⊆ Nn is invariant on BG(E) ⊆ Gr(dim E,V ). Suppose Γ is a basis (or a self-dual basis) strictly compatible with V , so that [BG]Γ consists of upper triangular matrices in [G]Γ . Suppose [E]Γ = Col (M) for M ∈ Cn×d . The following identities are easy: [id] r (M) = |TV (E) ∩ (Nn − Nn−i)| , n [(n+1−i)d] [(n−i)d] o TV (E) = i ∈ Nn | (r − r )(M) = 1 .

For a parabolic subgroup PG and a ﬂag

E = {{0} = E0 ( E1 ( ··· ( Ek} ∈ G/PG,

Lemma 2.1 is equivalent to that BG(E) ∈ BG\G/PG is uniquely determined by k the invariants (TV (Ei))i=1 .

When V = U ⊕ W and G1 × G2 ⊆ GL(U) × GL(W ), there are similar type invariants and matrix rank invariants for the BG1×G2 actions on ﬂag varieties of V . We leave the details to Section 3.

3 The B -actions on Gr(d, n) with n +n = n. GLn1 ×GLn2 1 2

n Suppose V = C = U ⊕ W where dim U = n1 and dim W = n2 . We identify

G = GLn as GL(V ), and K = GLn1 ×GLn2 as GL(U)×GL(W ), respectively. First we give a geometric description of B orbits and invariants on GLn1 ×GLn2 Gr(d, V ). Then we quote a nice remark from an anonymous referee to understand the problem from another viewpoint. A Borel subgroup B of K stabilizes a complete ﬂag in U : GLn1 ×GLn2

U = {{0} = U0 ( U1 ( ··· ( Un1 = U}, (3.1) and stabilizes a complete ﬂag in W :

W = {{0} = W0 ( W1 ( ··· ( Wn2 = W }. (3.2)

For a subspace E ⊆ V , we deﬁne the following types: − + TU (E) := TU (E ∩ U),TU (E) := TU (PU E), (3.3) − + TW (E) := TW (E ∩ W ),TW (E) := TW (PW E). (3.4)

9 All of them are B invariants. GLn1 ×GLn2 + − + − Denote Dom δ = TU (E) − TU (E) and Im δ = TW (E) − TW (E). Deﬁne the U -W map of E as: + + δU : Dom δ → Im δ, δU (i) := min{T (v) | T (v) = i}. v∈E W U Deﬁne the W -U map of E as: + + δW : Im δ → Dom δ, δW (j) := min{T (v) | T (v) = j}. v∈E U W Both of them are clearly B invariants. GLn1 ×GLn2

−1 Theorem 3.1 We have δU = δW . Moreover,

|Dom δW | = |Dom δU | = dim E − dim(E ∩ U) − dim(E ∩ W ). (3.5)

Proof The formula (3.5) is equivalent to the following obvious identities:

dim PU E − dim(E ∩ U) = dim PW E − dim(E ∩ W ) = dim E − dim(E ∩ U) − dim(E ∩ W ).

Now we show that δW ◦ δU acts identically on Dom δU .

For every i ∈ Dom δU , by the deﬁnition of δU , there is v ∈ E such that + + TU (v) = i, TW (v) = δU (i). + By the deﬁnition of δW , we must have δW ◦δU (i) ≤ TU (v) = i. If δW ◦δU (i) = i for all i ∈ Dom δU , then we are done. Otherwise, let i ∈ Dom δ satisfy that 0 δW ◦ δU (i) < i. Then there exists v ∈ E where + 0 + 0 TW (v ) = δU (i),TU (v ) = δW ◦ δU (i) < i. + 0 + Since TW (v ) = TW (v), there is λ ∈ C such that + 0 + + 0 TW (v − λv ) < TW (v) = δU (i),TU (v − λv ) = i.

This contradicts the deﬁnition of δU . So δW ◦ δU = id|Dom δU . −1 Likewise, δU ◦ δW acts identically on Dom δW . Thus δW = δU .

−1 From now on, we can write δ = δU and δ = δW . Deﬁne + + TU,W (v) = (TU (v),TW (v)) for v ∈ V. (3.6) Also deﬁne [ [ [ TU,W (E) := {(i, 0)} {(0, j)} {(k, δ(k))}. (3.7) − − k∈Dom δ i∈TU (E) j∈TW (E) Evidently T (E) is B invariant. The following corollary is easy. U,W GLn1 ×GLn2

10 Corollary 3.2 There is a basis {vpq | (p, q) ∈ TU,W (E)} of E such that TU,W (vpq) = (p, q) for (p, q) ∈ TU,W (E). Li Fix a basis ΓU = {e1, ··· , en} of U where Ui = t=1 Cet in U , and a basis Lj ΓW = {en1+1, ··· , en} of W where Wj = t=1 Cen1+t in W . Then Γ = ΓU ∪ ΓW is a basis of V , and the matrix representation

[B ] = B n1 × B n2 (3.8) GLn1 ×GLn2 Γ GL(C ) GL(C ) where BGL(Cm) consists of m × m nonsingular upper triangular matrices.

Theorem 3.3 The B orbit of E ∈ Gr(d, n) may be represented by GLn1 ×GLn2 the canonical form M M M Cei Cen1+j C ek + en1+δ(k) . (3.9) − − k∈Dom δ i∈TU (E) j∈TW (E) The set T (E) consists of invariants uniquely determining B (E). U,W GLn1 ×GLn2

Proof It suﬃces to prove the canonical form. Let {vpq | (p, q) ∈ TU,W (E)} be the basis of E in Corollary 3.2. Deﬁne g ∈ GL(V ) in the way that (1) g(et) = et + + for t ∈ Nn where t∈ / TU (E) and t − n1 ∈/ TW (E); (2) for (p, q) ∈ TU,W (E), we let ( g(PU vpq) = ep if p 6= 0;

g(PW vpq) = en1+q if q 6= 0. Clearly g ∈ B sends E to the canonical form (3.9). GLn1 ×GLn2 Remark 3.4 Every E ⊆ V induces a diagram: E (3.10) p OO ppp OOO ppp OOO ppp OO w pp OOO E wp ' ' E E∩W E∩U u MM qq JJ ' uu MMM qq JJ ' uu MM qq JJ uu MM qq JJJ uu MM& & xqx qq JJ zu E % PU E E∩U⊕E∩W PW E II M t II ' rr MM ' tt II rr MM tt II rr MMM tt II rrr MM tt $ $ xrr ϕ M& yty PU E PW E E∩U / E∩W

PU E δ PW E TU ( E∩U ) / TW ( E∩W ) The isomorphism ϕ is independent of U and W . The U -W map δ is induced by U and W to make the above diagram commute. δ is related to a Weyl group element in the Bruhat decomposition of GL|Dom δ| corresponding to E .

11 Remark 3.5 A referee remarks that the fibration a Gr(d, V ) −→ F({a1, a2},U) × F({b1, b2},W ) a1≤a2≤n1, b1≤b2≤n2 a2−a1=b2−b1 defined by E 7−→ ({E ∩ U ⊂ PU E}, {E ∩ W ⊂ PW E}) is B -equivariant. The fiber over a point ({A ⊂ A }, {B ⊂ B }) is GLn1 ×GLn2 1 2 1 2 isomorphic to GL(A1/A1) and the stabilizer acts on GL(A2/A1) through the product of two Borel subgroups of GL(A2/A1) acting on the left and on the right. Hence both in the base and in the fiber the orbits are Bruhat cells.

Remark 3.6 The number of B -orbits on Gr(d, n) is GLn1 ×GLn2 min{n1,n2,d,n−d} X n1 n2 n − 2t BGL ×GL \Gr(d, n) = t! . n1 n2 t t d − t t=0

It is obtained by enumerating the possible TU,W (E) for all E ∈ Gr(d, n).

Let us explore the matrix version. Suppose E ∈ Gr(d, n) and M [E] = Col ,M ∈ Cn1×d,N ∈ Cn2×d, Γ N M so that E is the column space of with respect to Γ. By (3.8), every N b ∈ B satisfies that [b] = b × b ∈ B n1 × B n2 and GLn1 ×GLn2 Γ 1 2 GL(C ) GL(C ) b1 0 M b1M [b(E)]Γ = Col = Col . 0 b2 N b2N Denote M M [id] r[(i,j),d] := rank for (i, j) ∈ N0 × N0 . (3.11) N N [jd] n1 n2 Recall that M [id] is the submatrix formed by the last i rows and the first M d columns of M . From matrix multiplication, r[(i,j),d] for a fixed N M E ∈ Gr(d, n) is independent of the choice of , and it is BGL ×GL N n1 n2 invariant.

Corollary 3.7 The B orbit of E ∈ Gr(d, n) is uniquely determined GLn1 ×GLn2 by the invariant lattice r[(i,j),d] . The lattice is equivalent to (i,j)∈N0 ×N0 n1 n2 TU,W (E) in the way that [(i,j),d] M 0 0 0 0 r = TU,W (E) ∩ (N × N − N × N ) . N n1 n2 n1−i n2−j

12 Conversely, let r[(n+1,j),d] = r[(i,m+1),d] = d and s[(i,j),d] := r[(i,j),d] − r[(i,j−1),d] − r[(i−1,j),d] + r[(i−1,j−1),d], 0 0 then (i, j) ∈ TU,W (E) for (i, j) ∈ Nn × Nm if and only if M s[(n+1−i,m+1−j),d] = −1 (otherwise it is 0). N

It is easy to verify the identities for the canonical form (3.9). The proof is omitted.

Remark 3.8 The Zariski dense open B orbit on Gr(d, n) makes GLn1 ×GLn2 M every r[(i,j),d] maximal. It can be illustrated by a Col as follow: N

d ≤ n1 ≤ n2 n1 < d ≤ n2 n1 ≤ n2 < d n1 = 4, n2 = 5, d = 3, n1 = 3, n2 = 5, d = 4, n1 = 3, n2 = 4, d = 6,   0 0 0   0 0 1 0    0 0 1  0 0 1 0 0 0    0 1 0 0   0 1 0     0 1 0 0 0 0     1 0 0 0     1 0 0     1 0 0 0 0 0  M    0 0 0 0     0 0 0     0 0 1 0 0 0  N    1 0 0 0     0 0 0     0 0 0 1 0 0     0 1 0 0     1 0 0     0 0 0 0 1 0     0 0 1 0   0 1 0  0 0 0 0 0 1 0 0 0 1 0 0 1

If we deﬁne the semi direct sum of U to W as

U W = {0 ( U1 ( ··· ( Un1 ( U + W1 ( ··· ( U + Wn2 }. (3.12) Then B stabilizes two complete ﬂags in V : GLn1 ×GLn2 0 V = U W, V = W U.

Corollary 3.9 Each of the following is a collection of B -invariants GLn1 ×GLn2 uniquely determining the B orbit of E : GLn1 ×GLn2  A1. T (E)  U,W  ± ± A2. TU (E) ∪ TW (E) ∪ {δ}  A3. T (E) ∪ {δ}  V A4. TV0 (E) ∪ {δ} h i  E+Ui+Wj A5. dim  Ui⊕Wj (i,j)∈N0 ×N0  n1 n2  h i A6. dim[E ∩ (Ui ⊕ Wj)]  (i,j)∈N0 ×N0 n1 n2

13 The proofs are easy and omitted. In particular, the following identities hold:

E + Ui + Wj 0 0 0 0 dim = TU,W (E) ∩ (Nn1 × Nn2 − Ni × Nj ) (3.13) Ui ⊕ Wj M = r[(n1−i,n2−j),d] . (3.14) N 0 0 dim[E ∩ (Ui ⊕ Wj)] = TU,W (E) ∩ (Ni × Nj ) (3.15) M = d − r[(n1−i,n2−j),d] . (3.16) N

Remark 3.10 The matrix invariants r[(i,j),d] can be generalized to broader situations. Let B consist of m × m nonsingular upper triangular matri- GLmi i i ces. Consider the Q B action on a ﬂag variety of L Cmi with the i∈I GLmi i∈I dimension set (d , ··· , d = P m ). There is a Q B -invariant lattice 1 k i∈I i i∈I GLmi

[(ti)i∈I ,dj ] Y 0 r for (ti)i∈I ∈ Nmi and j ∈ Nk. i∈I Likewise, we can construct an invariant lattice for the double coset spaces Y Y X X P \Cm×n/ P for m := m and n := n . GLmi GLnj i j i∈I j∈J i∈I j∈J

4 The BSOn -actions on Gr(d, n).

2 n 4.1 The orbits and invariants on BGLn \S (C ).

Before discussing BSOn \Gr(d, n), we review the classical results of the orbits 2 n and invariants on BGLn \S (C ) solved by W. Hodge and D. Pedoe [5]. 2 n Consider the GLn -actions on S (C ), the space of n × n symmetric matrices: −1 T −1 2 n g(M) := (g ) Mg , g ∈ GLn,M ∈ S (C ).

Let BGLn consist of nonsingular upper triangular matrices. Denote by Qn ⊆ 2 n S (C ) the set of all symmetric subpermutations. In other words, s ∈ Qn if and only if s is symmetric, and each row or column of s has at most one nonzero entry 1. As usual, let r[ij](M) be the rank of the submatrix formed by the ﬁrst i rows and the ﬁrst j columns of a matrix M .

Theorem 4.1 (Hodge-Pedoe [5]) Suppose M,M 0 ∈ S2(Cn). Then

B (1) M GL∼ n M 0 ⇐⇒ r (M) = r (M 0) . [ij] n×n [ij] n×n

14 s (2) There is a unique symmetric subpermutation sM = ij n×n in BGLn (M). Precisely,

sij = (r[ij] − r[i(j−1)] − r[(i−1)j] + r[(i−1)(j−1)])(M).

The second statement was proved in [5], while the ﬁrst was implied there. The 2 n second statement BGLn \S (C ) 'Qn is an extension of the Gram-Schmidt process. We may view sM as the canonical form of BGLn (M).

2 3 Example 4.2 The coset space BGL3 \S (C ) may be represented by the symmetric subpermutations as follow: h 1 0 0 i h 0 1 0 i h 0 0 1 i h 1 0 0 i (1) Rank 3: 0 1 0 , 1 0 0 , 0 1 0 , 0 0 1 . 0 0 1 0 0 1 1 0 0 0 1 0 h 1 0 0 i h 0 1 0 i h 1 0 0 i h 0 0 1 i h 0 0 0 i h 0 0 0 i (2) Rank 2: 0 1 0 , 1 0 0 , 0 0 0 , 0 0 0 , 0 1 0 , 0 0 1 . 0 0 0 0 0 0 0 0 1 1 0 0 0 0 1 0 1 0 h 1 0 0 i h 0 0 0 i h 0 0 0 i (3) Rank 1: 0 0 0 , 0 1 0 , 0 0 0 . 0 0 0 0 0 0 0 0 1 h 0 0 0 i (4) Rank 0: 0 0 0 . 0 0 0 2 3 Totally, there are 14 BGL3 -orbits on S (C ).

4.2 The orbits and invariants on BSOn \Gr(d, n).

Now consider BSOn \Gr(d, n). Let h, i be a nondegenerate symmetric bilinear n form on V = C . Let SOn be the corresponding isometry group. A Borel subgroup BSOn is the stabilizer of a complete self-dual ﬂag

V = {{0} = V0 ( V1 ( ··· ( Vn = V }. Li There is a self-dual basis Γ := {e1, ··· , en} of V such that Vi = t=1 Cet for i ∈ Nn . So [SOn]Γ is given in (2.8) and T [BSOn ]Γ = {g ∈ BGLn | g Jng = Jn, det g = 1} consists of upper triangular matrices in [SOn]Γ . ⊥ Given E ⊆ V , we denote Erad := E ∩ E , E[i] := E ∩ Vi , and

R[ij](E) := rank hE[i],E[j]i where the rank was deﬁned in (2.5). Both (dim E ) and R (E) are [i] n [ij] n×n

BSOn -invariants, since BSOn stabilizes V and preserves h, i.

Remark 4.3 R[ij] and r[ij] are related. By the deﬁnition of the bilinear form matrix in (2.4) and Theorem 4.1, the suﬃcient and necessary condition for 2 n 0 0 0 0 M ∈ S (C ) to be written as M = B(Γ ) for a sequence Γ = (v1, ··· , vn) ⊆ E 0 0 where E[i] = span {v1, ··· , vi} (i ∈ Nn ) is r (B(Γ0)) = R (E) . (4.1) [ij] n×n [ij] n×n

15 Denote TV (E/Erad) := TV (E) − TV (Erad). Deﬁne the structure map of E as:

µE : TV (E/Erad) → TV (E/Erad), where µE(i) := j if and only if

(R[ij] − R[i(j−1)] − R[(i−1)j] + R[(i−1)(j−1)])(E) 6= 0. Deﬁne the structure matrix of E as X SE := EiµE (i), (4.2) i∈TV (E/Erad) n×n where Eij ∈ C is the matrix with 1 in the (i, j)-entry and 0 elsewhere.

Lemma 4.4 The structure map µE is well deﬁned and symmetric: 2 µE = id|TV (E/Erad).

00 00 00 Proof By Remark 4.3, there is Γ = (v1, ··· , vn) ⊆ E such that E[i] = span {v00, ··· , v00} and R (E) = r (B(Γ00)) . By Theorem 4.1, 1 i [ij] n×n [ij] n×n 00 00 s Γ may be chosen in the way that B(Γ ) = ij n×n ∈ Qn . Then

sij = (R[ij] − R[i(j−1)] − R[(i−1)j] + R[(i−1)(j−1)])(E). Thus s 00 SE = ij n×n = B(Γ ) ∈ Qn.

So each i ∈ TV (E/Erad) corresponds to exactly one j ∈ TV (E/Erad) such that sij 6= 0. We conclude that µE is well deﬁned and symmetric.

Together with Theorem 4.7 (to be presented), we can describe all possible SE .

Remark 4.5 The structure matrix SE ∈ Qn is determined by: r (S ) = R (E) . (4.3) [ij] E n×n [ij] n×n The collection of structure matrices for all subspaces of V consists of all sym- s metric subpermutations S = ij n×n ∈ Qn , such that sij = 0 for all i+j ≤ n, and each row or column of S + JnSJn has at most one nonzero entry.

Lemma 4.6 For i ∈ Nn we have ⊥ i∈ / TV (E) ⇐⇒ n + 1 − i ∈ TV (E ). ⊥ Equivalently, TV (E) ∪ n + 1 − TV (E ) = Nn .

16 Proof ⊥ ⊥ i∈ / TV (E) ⇐⇒ E ∩ Vi = E ∩ Vi−1 ⇐⇒ (E ∩ Vi) = (E ∩ Vi−1) ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⇐⇒ E + Vi = E + Vi−1 ⇐⇒ E + Vn−i = E + Vn+1−i ⊥ ⇐⇒ n + 1 − i ∈ TV (E ).

Now let us construct a basis ΓE := {ΓE(1), ··· , ΓE(n)} of V with respect to

(Γ,BSOn (E)) as below:  e + ei, i ∈ Dom µE, µE(i) > i > n + 1 − µE(i)  n+1−µE (i)  √1 (e + e ), i ∈ Dom µ , µ (i) = i 6= n + 1 − i  2 n+1−i i E E  ( ΓE(i) := √ n + 1 − i ∈ Dom µ (4.4) 2e , E  i  µE(n + 1 − i) = n + 1 − i 6= i  ei, otherwise It is easy to check that the entries of B(ΓE) are 0 and 1. Moreover, the (i, j)- E E entry of B(Γ ) is 1 if and only if i + j = n + 1 or j = µE(i). We use Γ to construct a canonical form of BSOn (E) as follow.

Theorem 4.7 The BSOn -orbit of E ∈ Gr(d, n) may be represented by the canonical form: M CΓE(i).

i∈TV (E)

The followings are collections of BSOn -invariants. Each of them uniquely determines BSOn (E) in BSOn \Gr(d, n):  h i A1. (dim E[i])n and R[ij](E)  n×n A2. TV (E) and µE  A3. TV (E) and SE

A lemma is needed to prove the theorem.

Lemma 4.8 For E ∈ Gr(d, n), there is a basis Γ = {v1, ··· , vn} of V where  a. T (v ) = i for v ∈ Γ,  V i i  L b. E = Cvi, i∈TV (E) (4.5)  c. hvi, vji = 1 ⇐⇒ i + j = n + 1 or j = µE(i),  d. hvi, vji = 0 for the other situations.

Below proof also works for Lemma 5.8 with slight adjustments indicated here.

17 00 00 00 Proof There is a vector sequence Γ := (v1, ··· , vn) ⊆ E such that 00 00 00 E[i] = span {v1, ··· , vi }, B(Γ ) = SE. 00 Denote vi := vi for i ∈ TV (E), and ΓTV (E) := {vi | i ∈ TV (E)}. Then ΓTV (E) satisﬁes (4.5) after we change Γ to ΓTV (E) in there.

Suppose now TV (E) ⊆ Ψ ⊆ Nn and ΓΨ = {vi | i ∈ Ψ} satisfy (4.5). Given t ∈ Nn − Ψ, we will ﬁnd a vector vt ∈ V such that Ψ1 := Ψ ∪ {t} and

ΓΨ1 := ΓΨ ∪ {vt} satisfy (4.5).

⊥ ⊥ (1) If t ∈ TV ((span ΓΨ) ), then we select v ∈ (span ΓΨ) with TV (v) = t. Clearly v ∈/ span ΓΨ by t ∈ Nn − Ψ. n+1 (a) When v is isotropic, let vt := v. This includes all t < 2 situations

since V is self-dual. Clearly a. and b. of (4.5) hold for ΓΨ1 . As to c. ⊥ and d., by t ∈ TV ((span ΓΨ) ) and Lemma 4.6 we have n+1−t∈ / Ψ. Now hvt, vji = 0 and t + j 6= n + 1 for every j ∈ Ψ. Thus c. and d.

of (4.5) hold for ΓΨ1 . n+1 (b) When t = 2 (this will not happen when h, i is symplectic), Lemma n+1 ⊥ p 4.6 implies that 2 ∈/ TV (v ). Denote vt := v/ hv, vi. Then

(4.5) holds for ΓΨ1 . n+1 (c) Otherwise, t > 2 , and v is non-isotropic (this will not happen when h, i is symplectic). By t ∈ Nn − Ψ and Lemma 4.6, there is ⊥ w ∈ (span ΓΨ) such that TV (w) = n + 1 − t. Also hv, wi 6= 0 by Lemma 4.6. Denote hv, vi v := v − w ∈ (span Γ )⊥. t 2hv, wi Ψ

Then vt is isotropic and TV (vt) = t. Similarly (4.5) holds for ΓΨ1 . ⊥ (2) If t∈ / TV ((span ΓΨ) ), then n + 1 − t ∈ Ψ by Lemma 4.6. By assumption n+1 vn+1−t ∈ ΓΨ and t 6= 2 (since t∈ / Ψ and n + 1 − t ∈ Ψ). Denote

Ψ−1 := Ψ − {n + 1 − t}. ⊥ We have t ∈ TV ((span ΓΨ−1 ) ) by Lemma 4.6. By above discussion, ⊥ there is an isotropic vector vt ∈ (span ΓΨ−1 ) such that TV (vt) = t. Then hvt, vn+1−ti= 6 0. By adjusting a scalar multiple of vt , we can make hvt, vn+1−ti = 1 (when h, i is symplectic, we let hvt, vn+1−ti =

sgn(n + 1 − 2t) instead). Likewise, (4.5) holds for ΓΨ1 .

In all situations, we extend Ψ to Ψ1 = Ψ ∪ {t} and extend ΓΨ to ΓΨ1 , while keeping (4.5) true. The lemma then follows from induction.

Proof of Theorem 4.7 The basis ΓE = {ΓE(1), ··· , ΓE(n)} of V defined in (4.4) satisfies a, c, d, of (4.5). Suppose Γ is defined in Lemma 4.8. Let

18 E E g ∈ GL(V ) send Γ to Γ . Then B(Γ ) = B(Γ) = SE and g preserves V . We have g ∈ {±In}· BSOn . At least one of ±g is in BSOn . Therefore, M ±g(E) = CΓE(i)

i∈TV (E) can be chosen as the canonical form of BSOn (E).

Clearly A1∼A3 in Theorem 4.7 are all BSOn -invariants. The canonical form L CΓE(i) is uniquely ﬁxed by T (E) and µ . Moreover, (dim E ) i∈TV (E) V E [i] n and T (E) determine each other, and R (E) , µ , S , determine each V [ij] n×n E E other by (4.2) and (4.3). So the combinations A1∼A3 are equivalent. Each combination of them uniquely determines BSOn (E).

Remark 4.9 A referee remarks that the BSOn orbits on Gr(d, V ) may be viewed from ﬁbration as well. The ﬁbration

a 0 Gr(d, V ) −→ Gro(d ,V ) 0 0 n d ≤d, d ≤ 2 ⊥ defined by E 7→ E ∩ E is BSOn -equivariant, and the fiber over an isotropic subspace I of dimension d0 is isomorphic to the symmetric space of nondegenerate subspaces of I⊥/I of dimension d−d0 . The stabilizer of I acts on this fiber ⊥ through a Borel subgroup of SO(I /I). Hence the BSOn -orbit decomposition 0 of Gr(d, V ) is reduced to the usual Bruhat decomposition in Gro(d ,V ) and to Bruhat decomposition in a symmetric space, which is not complicated in this case and for which one may consult the paper [13]. This argument applies in the symplectic case also.

Example 4.10 Let (2 + 3, 1 + 4) represent the sequence (e2 + e3, e1 + e4). n Likewise for the others. The BSOn orbits of subspaces in C for n ≤ 4 can be represented by bases of the canonical forms as below (the trivial situations

BSOn \Gr(0, n) and BSOn \Gr(n, n) are omitted):

n Gr(d, n) Bases of canonical forms for BSOn \Gr(d, n) Orbits 2 Gr(1, 2) (1); (2); (1 + 2). 3 3 Gr(1, 3) (1); (2); (3); (1 + 3). 4 Gr(2, 3) (1, 2); (2, 3); (1, 3); (2, 1 + 3). 4 Gr(1, 4) (1); (2); (3); (4); (1 + 4); (2 + 3). 6 (1, 2); (1, 3); (2, 4); (3, 4); (1, 4); (2, 3); 4 Gr(2, 4) (1, 2 + 3); (2 + 3, 4); (2, 1 + 4); (3, 1 + 4); 13 (2 + 3, 1 + 4); (1 + 2, 4); (1 + 3, 4). Gr(3, 4) (1, 2, 3); (1, 2, 4); (1, 3, 4); (2, 3, 4); 6 (2, 3, 1 + 4); (1, 2 + 3, 4).

19 The following result roughly says that every BSOn orbit on Gr(d, n) may be 0 viewed as the “sums” of suitable BSOm orbits on Gr(d , m) for m = 1, 2, 4 and d0 ≤ 2. Such orbits are included in the above table except for trivial ones.

Theorem 4.11 (Combined Hyperbolic Plane Decomposition for On ) Let V , V , and h, i be the same as above. Given E ⊆ V , there is a self-dual basis Γe = {v1, ··· , vn} of V strictly compatible with V , such that we have the combined hyperbolic plane decomposition of V compatible with E :

Hi = Hn+1−i := Cvi + Cvn+1−i for i ∈ Nn, L L V = Hi (Hj ⊕ HµE (j)), S4 i∈ Ωt j∈Ω5 Lt=1 L E = (E ∩ Hi) E ∩ (Hj ⊕ HµE (j)) . S4 j∈Ω5 i∈ t=1 Ωt Moreover,  E ∩ Hi = {0}, i ∈ Ω1  E ∩ Hi = Cvi, i ∈ Ω2  E ∩ Hi = C(vn+1−i + vi), i ∈ Ω3  E ∩ Hi = Cvn+1−i ⊕ Cvi, i ∈ Ω4   E ∩ (Hj ⊕ HµE (j)) = C(vn+1−µE (j) + vj) + CvµE (j), j ∈ Ω5 where  Ω1 = N n − TV (E) − (n + 1 − TV (E))  d 2 e  Ω = T (E )  2 V rad Ω3 = {i ∈ TV (E/Erad) | µE(i) = i}  Ω4 = {i ∈ TV (E/Erad) | µE(i) > i = n + 1 − µE(i)}  Ω5 = {j ∈ TV (E/Erad) | µE(j) > j > n + 1 − µE(j)}

We can verify the above decomposition for the canonical forms in Theorem 4.7 and the basis Γ = {e1, ··· , en}. The decompositions are BSOn -equivariant and thus the above result follows.

Remark 4.12 The number of BSOn -orbits on Gr(d, n) is (1) When n is even, n X ( )! 3a 2c |B \Gr(d, n)| = 2 . SOn a! b! (2c)!! ( n − d + b)! (a,b,c)∈Ω(d,n) 2 (2) When n is odd, n−1 a c n+1 2 X ( 2 )! 3 2 ( 2 − 3 a − b − 2c) |BSO \Gr(d, n)| = . n a! b! (2c)!! ( n+1 − d + b)! (a,b,c)∈Ω(d,n) 2

20 Here    a + 2b + 2c = d,    Ω(d, n) = (a, b, c) ∈ Z3 | a, b, c ≥ 0, .   n+1  a + b + 2c ≤ 2 . The formulas are obtained by enumerating the range of a set of invariants determining BSOn (E) in Theorem 4.7. Example 4.10 conﬁrms these formulas.

Let us discuss the matrix invariants. Suppose [E]Γ = Col (M) for a matrix M ∈ Cn×d with full column rank. Then

0 BSOn 0 E ∼ E ⇐⇒ [E ]Γ ∈ {Col (gMb) | g ∈ [BSOn ]Γ, b ∈ GLd} . [id] [id] Evidently r (M) = d − dim E[n−i] . So (r (M))i∈Nn , (dim E[i])i∈Nn , and TV (E) are equivalent to each other. Suppose TV (E) = {t1 < t2 < ··· < td}. Denote ω := e¯t1 , e¯t2 , ··· , e¯td where n e¯i ∈ C takes 1 in the i-th entry and 0 elsewhere. View ω as the ﬁrst d columns of a permutation matrix in GLn . Then by Lemma 2.1 M has the decomposition

M = gωb for g ∈ BGLn , b ∈ GLd.

The column space of the ﬁrst k columns of gω is exactly [E[tk]]Γ . Let tpi be the maximal integer in TV (E) no greater than i. Then given (i, j) ∈ Nn × Nn , T T R (E) = R (E) = r (w g Jngw). [ij] [tpi tpj ] [pipj ] It relates R (E) to certain matrix invariants. [ij] n×n

Corollary 4.13 Let BSOn act on E ∈ Gr(d, V ). The lattices of ranks (r[id](M)) and r (ωT gT J gω) i∈Nn [pipj ] n d×d are BSOn -invariants uniquely determining BSOn (E).

The Zariski dense open BSOn orbit on Gr(d, n) let the above ranks maximal.

Remark 4.14 The dense open BSOn orbit on Gr(d, n) may be represented by Col (M) according to two situations illustrated below: d ≤ n/2 d > n/2 n = 8, d = 3, n = 7, d = 5,  0 0 1  0 1 0  0 0 0 0 1  1 0 0 0 0 0 1 0  0 0 0   1 0 0 0 0  M  0 0 0  0 1 0 0 0  1 0 0   0 0 1 0 0  0 1 0 0 0 0 1 0 0 0 1 0 0 0 0 1

21 The proof of Lemma 4.8 implies the following result, a special case of the extension of Witt’s Theorem (Theorem 6.1).

Theorem 4.15 Let φ : E → F be an isometry of E,F ∈ Gr(d, V ) such that

φ(E[i]) = F[i] . Then φ can be extended to an element φe ∈ BSOn .

Proof From Lemma 4.8, there is a basis Γ = {v1, ··· , vn} of V satisfying (4.5) 0 0 with respect to E and Γ. Let vi = φ(vi) for i ∈ TV (E). Then {vi | i ∈ TV (E)} 0 is a basis of F . By the proof of Lemma 4.8, the set {vi | i ∈ TV (E) = TV (F )} 0 0 0 can be extended to a basis Γ = {v1, ··· , vn} of V satisfying (4.5) with respect 0 0 to F and Γ . Let φe ∈ GL(V ) such that φ(vi) = vi then we are done.

In Section 6, we will describe a set of invariants (similar to those in Theorem

4.7) completely determining the PSOn orbits on Gr(d, n).

5 The BSpn -actions on Gr(d, n) (n even).

The discussions in this section are highly parallel to those in Section 4. We ﬁrst 2 n recall the orbits and invariants on BGLn \ ∧ (C )[5]. Then we apply them to investigate BSpn \Gr(d, n).

2 n 5.1 The orbits and invariants on BGLn \ ∧ (C ).

2 n Let GLn act on ∧ (C ), the space of n × n skew-symmetric matrices, by −1 T −1 2 n g(M) := (g ) Mg g ∈ GLn,M ∈ ∧ (C ).

Let BGLn consist of nonsingular upper triangular matrices. Denote by Kn ⊆ 2 n ∧ (C ) the set of all standard skew-symmetric subpermutations, that is, s ∈ Kn if and only if s is skew-symmetric, each row or column of s has at most one nonzero entry, and the nonzero (i, j)-entry of s is sgn (j − i). Again let r[ij] be the rank of the submatrix formed by the ﬁrst i rows and the ﬁrst j columns of a matrix.

Theorem 5.1 (Hodge-Pedoe [5]) For M,M 0 ∈ ∧2(Cn), we have:

B (1) M GL∼ n M 0 ⇐⇒ r (M) = r (M 0) . [ij] n×n [ij] n×n s (2) There is a unique standard skew-symmetric subpermutation sM = ij n×n

in BGLn (M). Precisely,

sij = sgn (j − i) · (r[ij] − r[i(j−1)] − r[(i−1)j] + r[(i−1)(j−1)])(M).

22 2 n The second statement BGLn \ ∧ (C ) 'Kn is analogous to Gram-Schmidt process. We may view sM as the canonical form of BGLn (M).

2 4 Example 5.2 The coset space BGL4 \ ∧ (C ) may be represented by the standard skew-symmetric subpermutations as follow: 0 1 0 0 0 0 1 0 0 0 0 1 −1 0 0 0 0 0 0 1 0 0 1 0 (1) Rank 4: 0 0 0 1 , −1 0 0 0 , 0 −1 0 0 . 0 0 −1 0 0 −1 0 0 −1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 −1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 (2) Rank 2: 0 0 0 0 , −1 0 0 0 , 0 0 0 0 , 0 −1 0 0 , 0 0 0 0 , 0 0 0 1 . 0 0 0 0 0 0 0 0 −1 0 0 0 0 0 0 0 0 −1 0 0 0 0 −1 0 0 0 0 0 0 0 0 0 (3) Rank 0: 0 0 0 0 . 0 0 0 0 2 4 Totally, there are 10 BGL4 -orbits on ∧ (C ).

5.2 The orbits and invariants on BSpn \Gr(d, n) (n even).

Let h, i be a nondegenerate symplectic bilinear form on V = Cn preserved by

Spn . A Borel subgroup BSpn is the stabilizer of a complete self-dual ﬂag

V = {{0} = V0 ( V1 ( ··· ( Vn = V }.

Let Γ = {e1, ··· , en} be a self-dual basis strictly compatible with V . Then [Spn]Γ is given by (2.9) and T [BSpn ]Γ = {g ∈ BGLn | g Kng = Kn} sgn (j − i) · δ where Kn = i+j,n+1 n×n . ⊥ Given E ⊆ V , we denote Erad = E ∩ E , E[i] = E ∩ Vi , and

R[ij](E) = rank hE[i],E[j]i. Both (dim E ) and R (E) are B -invariants. [i] n [ij] n×n Spn

Remark 5.3 A matrix M ∈ ∧2(Cn) can be written as M = B(Γ0) for a 0 0 0 0 0 sequence Γ = (v1, ··· , vn) ⊆ E with E[i] = span {v1, ··· , vi} for i ∈ Nn , if and only if r (B(Γ0)) = R (E) . (5.1) [ij] n×n [ij] n×n

Deﬁne the structure map of E as

µE : TV (E/Erad) → TV (E/Erad), where µE(i) := j if and only if

(R[ij] − R[i(j−1)] − R[(i−1)j] + R[(i−1)(j−1)])(E) 6= 0.

23 Deﬁne the structure matrix of E by: X SE := sgn (µE(i) − i) EiµE (i) (5.2) i∈TV (E/Erad)

n×n where Eij ∈ C is the matrix with 1 in the (i, j)-entry and 0 elsewhere.

Lemma 5.4 The structure map µE is well deﬁned and symmetric:

2 µE = id|Dom µE .

The proof is analogous to that of Lemma 4.4 and thus omitted.

Remark 5.5 The structure matrix SE ∈ Kn is determined by: r (S ) = R (E) . (5.3) [ij] E n×n [ij] n×n By Theorem 5.7 below, the range of structure matrices for subspaces of V is s the set of all standard skew-symmetric subpermutations S = ij n×n ∈ Kn , such that sij = 0 for all i + j ≤ n, and each row or column of S + KnSKn has at most one nonzero entry (i.e. if µE(i) 6= n + 1 − i then n + 1 − i∈ / Im µE ).

Lemma 5.6 For i ∈ Nn we have

⊥ i∈ / TV (E) ⇐⇒ n + 1 − i ∈ TV (E ).

⊥ Equivalently, TV (E) ∪ (n + 1 − TV (E )) = Nn .

See the proof of Lemma 4.6.

E E E We construct a basis Γ := {Γ (1), ··· , Γ (n)} of V with respect to (Γ,BSpn (E)), such that ( e + e , i ∈ Dom µ , µ (i) > i > n + 1 − µ (i) ΓE(i) := n+1−µE (i) i E E E (5.4) ei, otherwise

Then the (i, j)-entry of B(ΓE) equals to  1, if i > j, and i + j = n + 1 or j = µ (i),  E −1, if i < j, and i + j = n + 1 or j = µE(i), 0, otherwise.

The following result says that some vectors of ΓE form a basis of the canonical form of BSpn (E).

24 Theorem 5.7 The BSpn -orbit of E ∈ Gr(d, n) may be represented by the canonical form: M CΓE(i).

i∈TV (E)

The followings are collections of BSpn -invariants. Each of them uniquely determines BSpn (E) in BSpn \Gr(d, n):  h i A1. (dim E[i])n and R[ij](E) .  n×n A2. TV (E) and µE.  A3. TV (E) and SE.

Lemma 5.8 For E ∈ Gr(d, n), there is a basis Γ = {v1, ··· , vn} of V where  a. T (v ) = i for v ∈ Γ,  V i i  L b. E = Cvi, i∈TV (E) (5.5)  c. hvi, vji = sgn (j − i) ⇐⇒ i + j = n + 1 or j = µE(i),  d. hvi, vji = 0 for the other situations.

All above proofs are parallel to those in Section 4.2 and so omitted.

Example 5.9 Let (1 + 2, 4) represent the sequence (e1 + e2, e4). Likewise for n the others. The BSpn orbits of subspaces in C for n ≤ 4 can be represented by bases of the canonical forms as below: (the trivial situations BSpn \Gr(0, n) and BSpn \Gr(n, n) are omitted)

n Gr(d, n) Bases of canonical forms for BSpn \Gr(d, n) Orbits 2 Gr(1, 2) (1); (2). 2 Gr(1, 4) (1); (2); (3); (4). 4 4 Gr(2, 4) (1, 2); (1, 3); (2, 4); (3, 4); (1, 4); (2, 3); 8 (1 + 2, 4); (1 + 3, 4). Gr(3, 4) (1, 2, 3); (1, 2, 4); (1, 3, 4); (2, 3, 4). 4

The list is simpler than the one in Example 4.10, since n should be even and all vectors are isotropic here.

Every BSpn orbit of Gr(d, n) may be viewed as the “sums” of suitable BSpm orbits on Gr(d0, m) for m = 2, 4 and d0 ≤ 2. Such orbits are included in the above table except for trivial ones. This is the following theorem.

Theorem 5.10 (Combined Hyperbolic Plane Decomposition for Spn ) Let V , V , and h, i be the same as above. Given E ⊆ V , there is a self-dual basis

25 Γe = {v1, ··· , vn} of V strictly compatible with V , such that we have the combined hyperbolic plane decomposition compatible with E :

Hi = Hn+1−i := Cvi + Cvn+1−i for i ∈ Nn, L L V = Hi (Hj ⊕ HµE (j)), i∈Ω1∪Ω2∪Ω4 j∈Ω5 L L E = (E ∩ Hi) E ∩ (Hj ⊕ HµE (j)) . i∈Ω1∪Ω2∪Ω4 j∈Ω5 Moreover,  E ∩ H = {0}, i ∈ Ω  i 1  E ∩ Hi = Cvi, i ∈ Ω2 E ∩ H = Cv + Cv , i ∈ Ω  i n+1−i i 4  E ∩ (Hj ⊕ HµE (j)) = C(vn+1−µE (j) + vj) + CvµE (j), j ∈ Ω5 where  Ω1 = Nd n e − TV (E) − (n + 1 − TV (E))  2  Ω2 = TV (Erad) Ω = {i ∈ T (E/E ) | µ (i) > i = n + 1 − µ (i)}  4 V rad E E  Ω5 = {j ∈ TV (E/Erad) | µE(j) > j > n + 1 − µE(j)}

Since the above decomposition is BSpn -equivariant, we only need to verify the decomposition for the canonical forms in Theorem 5.7 and the basis Γ = {e1, ··· , en}, which is an easy task.

Remark 5.11 The number of BSpn -orbits on Gr(d, n) is n X ( )! 2a+c |B \Gr(d, n)| = 2 . Spn a! b! (2c)!! ( n − d + b)! (a,b,c)∈Ω(d,n) 2 Here Ω(d, n) is the same as in Remark 4.12. Namely,    a + 2b + 2c = d,    Ω(d, n) = (a, b, c) ∈ Z3 | a, b, c ≥ 0, .   n+1  a + b + 2c ≤ 2 . The formula is obtained by exploring the range of a set of invariants determining

BSpn (E) in Theorem 5.7. Example 5.9 conﬁrms the above formula.

We now determine some matrix invariants. Suppose [E]Γ = Col (M) for a n×d matrix M ∈ C with full column rank. Suppose TV (E) = {t1 < t2 < ··· < td} and ω = e¯t1 , e¯t2 , ··· , e¯td as in Section 4.2. Then M has the decomposition

M = gωb for g ∈ BGLn , b ∈ GLd.

26 Let tpi be the maximal integer in TV (E) no greater than i. Then given (i, j) ∈ Nn × Nn , [(n−i)d] dim E[i] = d − r (M), T T R (E) = R (E) = r (w g Kngw). [ij] [tpi tpj ] [pipj ]

Corollary 5.12 Let BSpn act on E ∈ Gr(d, V ). The lattices of ranks (r[id](M)) and r (ωT gT K gω) i∈Nn [pipj ] n d×d are BSpn -invariants uniquely determining BSpn (E).

Example 5.13 The Zariski dense open BSpn orbit on Gr(d, n) may be represented by Col (M) according to two situations illustrated below: d ≤ n/2 d > n/2 n = 12, d = 6, n = 12, d = 8,  0 0 0 0 1 0   0 0 0 0 0 0 1 0  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0  0 0 0 0 0 0   0 0 0 0 0 0 0 0   1 0 0 0 0 0   1 0 0 0 0 0 0 0   0 0 0 0 0 0   0 1 0 0 0 0 0 0  M  1 0 0 0 0 0   0 0 1 0 0 0 0 0   0 1 0 0 0 0   0 0 0 1 0 0 0 0   0 0 1 0 0 0   0 0 0 0 1 0 0 0   0 0 0 1 0 0   0 0 0 0 0 1 0 0  0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1

The following result can be derived from Lemma 5.8. It is parallel to Theorem 4.15, and is a special case of the extension of Witt’s Theorem (Theorem 6.1).

Theorem 5.14 Let φ : E → F be an isometry of E,F ∈ Gr(d, V ) such that

φ(E[i]) = F[i] . Then φ can be extended to an element φe ∈ BSpn .

In Section 6, we will describe a set of invariants (similar to those in Theorem

5.7) completely determining the PSpn orbits on Gr(d, n).

6 An extension of Witt’s theorem.

Witt’s theorem [10, pp589] claims that a local isometry between two subspaces can be extended to an isometry of the whole space. Surprisingly, if the local isometry preserves a self-dual flag, then it can still be extended to an isometry of the whole space preserving the self-dual flag. This is motivated by Theorem 4.15 and Theorem 5.14 in the preceding discussion. Let h, i be a nondegenerate symmetric or symplectic bilinear form of V = Cn . Let V = {{0} = V0 ( V1 ( ··· ( Vk = V } be a self-dual flag. A linear map

27 g : E → F between two subspaces of V is preserving V if g(V|E) = V|F . In other words, 0 g(E ∩ Vi) = g(E) ∩ Vi for i ∈ Nk. We have the following statement:

Theorem 6.1 (Extension of Witt’s theorem) Let V be a self-dual ﬂag of V . Then every isometry of subspaces φ : E → F preserving V can be extended to an isometry φe : V → V preserving V .

The theorem is a powerful tool, which allows us to construct desired parabolic subgroup elements in isometry groups. Precisely, let G ∈ {On,SOn, Spn} and V PG = G for the self-dual ﬂag V . If we have an isometry g of two subspaces preserving V , then the theorem guarantees that g can be extended to an element in {±In}· PG . On the other hand, the statement is not necessary true if V is not self-dual. Below is a counterexample. It remains an interesting question to ask what other situations also have the extension of Witt’s theorem. In [8] we provide two other extensions of Witt’s Theorem. Both are derived from Theorem 6.1.

4 Example 6.2 Let Γ := {v1, ··· , v4} be a basis of C . Let i M V = {{0} = V0 ( V1 ( ··· ( V4 = V },Vi = Cvt, t=1 be the complete flag which Γ is strictly compatible with. Define a symmetric bilinear form h, i by the bilinear form matrix 0 0 1 0 0 0 0 1 B(Γ) = 1 0 0 0 . 0 1 0 0 The stabilizer of the non-self-dual flag V in the isometry group of h, i is (" # ) a 0 0 ac [(O )V ] = 0 b −bc 0 a, b ∈ C×, c ∈ C . 4 Γ 0 0 a−1 0 0 0 0 b−1

The isometry φ(v2) := v1 + v2 preserves V . However, by the expression of V 4 [(O4) ]Γ above, it cannot be extended to an isometry of C preserving V .

Theorem 6.1 is included in the following slight generalization:

Theorem 6.3 Let dim V = dim U < ∞ where V and U are complex vector spaces both endowed with nondegenerate symmetric (or symplectic) bilinear forms. Let

V = {{0} = V0 ( V1 ( ··· ( Vk = V }, U = {{0} = U0 ( U1 ( ··· ( Uk = U},

28 be self-dual ﬂags of V and U respectively, with dim Vi = dim Ui for i ∈ Nk . If there is an isometry φ : E → F , E ⊂ V and F ⊂ U , such that

φ(E ∩ Vi) = F ∩ Ui for i ∈ Nk, then φ can be extended to an isometry φe : V → U sending V to U . The proof of Theorem 6.3 depends on a technical result.

Lemma 6.4 Let V and U be the same as in Theorem 6.3, and φ : E → F be an isometry sending E ⊂ V to F ⊂ U . Let

{0 ⊂ V1 ⊂ V2 ⊂ V }, {0 ⊂ U1 ⊂ U2 ⊂ U}, be self-dual ﬂags of V and U respectively, with dim Vi = dim Ui for i = 1, 2. If φ(E ∩ Vi) = F ∩ Ui for i = 1, 2, then φ : E → F can be extended to an isometry φ1 : E + V1 → F + U1 with φ1(V1) = U1 .

Proof In this proof only, write Ei = E ∩ Vi and Fi = φ(E ∩ Vi) for i = 1, 2. Let h, i represent the bilinear forms in both V and U . There exists Ee such that E2 ⊕ Ee = E . Let Fe = φ(Ee) then F2 ⊕ Fe = F .

We extend φ to φ1 by two steps induced by: ⊥ E1 / E1 + (V1 ∩ E ) / V1

0 φ|V1 φ |V1 φ1|V1 ⊥ F1 / F1 + (U1 ∩ F ) / U1

Note that all vectors in V1 and U1 are isotropic, which are convenient in constructing isometries. 0 ⊥ ⊥ (1) From φ : E → F to φ : E + (V1 ∩ E ) → F + (U1 ∩ F ): ⊥ ⊥ ⊥ dim(V1 ∩ E ) = dim V − dim(V1 ∩ E )

= dim V − dim(V2 + E) = dim V1 − dim Ee ⊥ = dim(U1 ∩ F ). ⊥ ⊥ Likewise, dim[E1 + (V1 ∩ E )] = dim[F1 + (U1 ∩ F )]. We extend ⊥ ⊥ ⊥ φ|E1∩E : E1 ∩ E → F1 ∩ F to a linear bijection 0 ⊥ ⊥ ⊥ φ |V1∩E : V1 ∩ E → U1 ∩ F . 0 0 ⊥ Let φ be the linear map combining φ and φ |V1∩E : 0 ⊥ ⊥ φ : E + (V1 ∩ E ) → F + (U1 ∩ F ) 0 0 ⊥ Then φ is an isometry since hφ (v), φ(e)i = hv, ei = 0 for v ∈ V1 ∩ E 0 ⊥ ⊥ and e ∈ E . It is an extension of φ, and φ (V1 ∩ E ) = U1 ∩ F .

29 0 ⊥ ⊥ (2) From φ : E + (V1 ∩ E ) → F + (U1 ∩ F ) to φ1 : E + V1 → F + U1 : There exist E0 and V 0 such that E0 ⊂ V 0 and

⊥ 0 ⊥ 0 E1 = (E1 ∩ E ) ⊕ E ,V1 = (V1 ∩ E ) ⊕ V .

0 0 It follows that dim V = rank hV1,Ei = rank hV , Eei = dim Ee. Let F 0 = φ0(E0). There exists U 0 such that F 0 ⊂ U 0 and

⊥ 0 ⊥ 0 F1 = (F1 ∩ F ) ⊕ F ,U1 = (U1 ∩ F ) ⊕ U .

0 0 Likewise dim U = rank hU1,F i = rank hU , Fei = dim Fe = dim Ee. ϕ1 We make the isomorphism V 0 ' (Ee)∗ as follow: 0 0 0 0 ϕ1(v )(e) = hv , ei, v ∈ V , e ∈ E.e

0 ϕ2 ∗ 0 0 Similarly for the isomorphism U ' (Fe) . Let φ1|V 0 : V → U be induced by the following commutative diagram:

ϕ1 V 0 / (Ee)∗

0−1 ∗ φ1|V 0 (φ )

ϕ2 U 0 / (Fe)∗

0 0 In other words, hφ1(v), φ (e)i = hv, ei for v ∈ V and e ∈ Ee. Clearly 0 0 φ1|E0 coincides with φ |E0 . Combining φ and φ1|V 0 together we get an 0 isometry φ1 : E + V1 → F + U1 extended from φ , with φ1(V1) = U1 .

Proof of Theorem 6.3 (and Theorem 6.1) We have an isometry φ : E → F and the self-dual ﬂags of V and U respectively:

V = {{0} = V0 ( V1 ( ··· ( Vk = V }, U = {{0} = U0 ( U1 ( ··· ( Uk = U},

According to Lemma 6.4, the isometry φ can be extended to an isometry φ1 : E + V1 → F + U1 with φ1(V1) = U1 ; the isometry φ1 can be extended to an isometry φ2 : E + V2 → F + U2 with φ2(V2) = U2 ; . . . . Repeating the process, eventually φ can be extended to an isometry φbk/2c : E + Vbk/2c → F + Ubk/2c with φbk/2c(Vi) = Ui for i ∈ Nbk/2c . By Witt’s Theorem φbk/2c can be extended to an isometry φe : V → U . It follows that φe(Vi) = Ui for i ∈ Nk .

The extension of Witt’s Theorem may be applied to determine the invariants on PSOn \Gr(d, n) and PSpn \Gr(d, n)(n even).

30 Theorem 6.5 Let K = SO(V ) or Sp(V ), where V = Cn is endowed with a nondegenerate symmetric or symplectic bilinear form h, i preserved by K . Let parabolic subgroup PK be the stabilizer of a self-dual ﬂag

V = {{0} = V0 ( V1 ( ··· ( Vk = V }.

The PK orbit of E ∈ Gr(d, V ) is uniquely determined by the PK invariants (dim E )k and R (E) . [i] i=1 [ij] k×k

Proof On one hand, (dim E )k and R (E) are clearly P invari- [i] i=1 [ij] k×k K ants. On the other hand, suppose there exist E,F ⊂ V such that (dim E )k = (dim F )k , R (E) = R (F ) . [i] i=1 [i] i=1 [ij] k×k [ij] k×k

Let dim E[i] = dim F[i] = di . Similar to Hodge-Pedoe’s theorem, there are bases {e0 , ··· , e0 } of E and {f 0 , ··· , f 0 } of F respectively, such that 1 dk 1 dk

di di M 0 M 0 0 0 0 0 E[i] = Cet,F[i] = Cft. hei, eji = hfi , fji . dk×dk dk×dk t=1 t=1 0 0 The linear map deﬁned by φ(ei) = fi is an isometry preserving V . By Theorem 6.1, φ can be extended to an isometry φe of V preserving V . The map φe may V be not in PK when K = SO(V ), since O(V ) may be disconnected. However, it is easy to ﬁnd g ∈ PK such that g(E) = φe(E) = F .

Basing on the above invariants, we can determine the PK orbits on Gr(d, V ) for K = SO(V ) and Sp(V ). More applications for the extension of Witt’s Theorem can be found out in [8] and [9], the consecutive papers of this series classifying the other BK orbits and invariants on K -multiplicity-free ﬂag varieties G/PG .

References

[1] C. Benson, G. Ratcliﬀ, On multiplicity-free actions, in: Representations of real and p-adic groups, Singapore University Press, Singapore, 2004, 221-304. [2] M. Brion, On orbit closures of spherical subgroups in ﬂag varieties, Comment. Math. Helv. 76 (2001), 263-299. [3] R. Goodman, N. Wallach, Representation and Invariants of the Classical Groups, Cambridge University Press, 1998. [4] A. Helminck, G. Schwarz, Orbits and invariants associated with a pair of spherical varieties: some examples, Acta Appl. Math. 73: 103-113, 2002. [5] W. Hodge, D. Pedoe, Methods of Algebraic Geometry, Volume 1, Cambridge University Press, 1994.

31 [6] J. Horvath, R. Howe, N. Wallach, Mutiplicity-free actions of symmetric subgroups on ﬂag varieties (I: classical groups), preprint. [7] R. Howe, Prospectives on Invariant Theory: Schur duality, multiplicity-free actions and beyond, in: the Schur Lectures (1992), Israel Mathematical Conference Proceedings 8 (1995), 1-182. [8] H. Huang, Borel orbits and invariants of classical symmetric subgroups on multiplicity-free Grassmannians (II), in preparation. [9] H. Huang, Borel orbits and invariants of two classical symmetric pairs on ﬂag manifolds, in preparation. [10] S. Lang, Algebra, Addison-Wesley, 1993. [11] P. Littelmann, On spherical double cones, Journal of Algebra, 166 (1994), 142- 157. [12] E. Miller, B. Sturmfels, Combinatorial Commutative Algebra, Graduate Texts in Math Vol. 227, Springer-Verlag, 2004. [13] R. Richardson, T. Springer, The Bruhat order on symmetric varieties, Geom. Dedicata 35: 389-436, 1990. [14] R. Richardson, T. Springer, Complements to “The Bruhat order on symmetric varieties”, Geom. Dedicata 49: 231-238, 1994. [15] F. Servedio, Prehomogeneous vector spaces and varieties, Transaction of Amer- ican Mathematical Society 76 (1973), 421-444. [16] E. Vinberg, Complexity of actions of reductive Lie groups, Functional Analysis and Application 20 (1986), 1-11.