LECTURE 5 1. Isotropic Grassmannians in This Section, We

Home , Grassmannian, Orthogonal complement, Projectivization

LECTURE 5

1. Isotropic Grassmannians In this section, we discuss the geometry of isotropic Grassmannians. Let V be an n-dimensional complex vector space. Let Q be a non-degenerate quadratic form. The form Q may either be symmetric or skew-symmetric. Since the rank of a skew- symmetric form is always even, in the latter case, we must assume that n is even. There are some diﬀerences in the discussion depending on whether Q is symmetric or skew-symmetric. Furthermore, when Q is symmetric, there are slight variations in detail depending on whether n is even or odd. We will now discuss these cases separately. 1.1. Preliminaries on quadrics. Let Q be a smooth quadric hypersurface in n−1 n P . Set m = b 2 c. The largest dimensional linear spaces contained in Q have projective dimension m−1. If n is odd, then the maximal dimensional linear spaces m(m+1) on Q form an irreducible family of dimension 2 . If n is even, then the maximal dimensional linear spaces contained in Q form two isomorphic families of dimension m(m−1) 2 . Two linear spaces belong to the same irreducible component if and only if their dimension of intersection is equal to m − 1 modulo 2 (see [GH] p. 735). More generally, we will be interested in linear spaces on quadric hypersurfaces with singularities. A quadric hypersurface in Pn−1 of corank r (or, equivalently, with a singular locus of dimension r − 1) is the cone over a smooth quadric hypersurface in Pn−1−r with vertex an (r − 1)-dimensional projective linear space. If Q is a quadric hypersurface of corank r in Pn−1, then the largest dimensional n−r−2 linear space on Q has dimension b 2 c+r. The space of linear spaces of maximal dimension on Q is irreducible if n − r is odd and has two irreducible components if n−r−3 n−r−2 n−r is even. Setting l = 2 in the former case and l = 2 in the latter case, the dimension of each irreducible component of the space of maximal dimensional (l+1)(l+2) l(l+1) linear spaces is 2 and 2 , respectively. In the latter case, two linear spaces belong to the same irreducible component if and only if their dimension of intersection is equal to l + r modulo 2. These claims follow from the previous paragraph since Q is a cone over a smooth quadric hypersurface in Pn−1−r.

Notation 1.1. Denote the Fano variety of s-dimensional projective linear spaces n−1 r contained in a quadric hypersurface Q ∈ P of corank r by Fs,n(Q).

Let Q ⊂ Pn−1 be a quadric hypersurface of corank r. Let s be a positive integer n−r−2 less than or equal to b 2 c + r. Consider the incidence correspondence of pairs of a point p of Q and an s-dimensional linear space containing p: r I = { (x, [Λ]) | x ∈ Λ ⊂ Q } ⊂ Q × Fs,n(Q). The automorphism group of Q acts transitively on the smooth points of Q. The s-planes that contain a smooth point p lie in the tangent linear space H at p. Q∩H is a quadric hypersurface of corank r +1. The intersection with a hyperplane 1 complementary to p is a quadric hypersurface of corank r and intersects all the s-planes containing p in an (s − 1)-dimensional linear space. We conclude that the space of s-dimensional linear spaces containing p has the same dimension as the space of (s−1)-dimensional linear spaces lying on a quadric hypersurface in Pn−3 of corank r. Therefore, by induction, we can calculate the general fiber dimension of the projection of I to Q and determine the dimension of I. The second projection r maps I onto Fs,n(Q) with fiber dimension s. We thus obtain a recursion relation r for the dimension of Fs,n(Q). A priori we need to check that the s-dimensional linear spaces that intersect the n−r−2 vertex in dimension greater than s − 1 − b 2 c do not form another irreducible r component (potentially of different dimension) of Fs,n(Q). It is easy to see that linear spaces that intersect the vertex in larger than the expected dimension are limits of linear spaces that intersect the vertex in the expected dimension. Observe that every linear space on a quadric is contained in a maximal dimensional linear space. Take a linear space Λ that intersects the vertex in the linear space Ω. Assume that the dimension of Ω is larger than expected. Take a linear space ∆ in n−r−2 Λ complementary to Ω. Take a linear space Γ of dimension b 2 c which contains ∆, but does not intersect the vertex of Q. Since the Grassmannian of s-planes in the span of Γ and Ω is irreducible, the claim follows. n−r−2 In case s < 2 , the space of s-dimensional linear spaces on Q is irreducible. n−r−2 r+1 If s ≥ 2 the recursion stops when we obtain a quadric of rank r in P or Pr with multiplicity 2. The former case occurs if n − r is even and the latter case occurs if n − r is odd. This allows us to calculate the dimensions of the spaces of n−r−2 s-dimensional linear spaces on Q recursively. It also proves that when s ≥ 2 , the spaces of s-dimensional linear spaces on Q is irreducible if n − r is odd and has two components if n − r is even. We have thus proved the following: n−1 n−r−2 Lemma 1.2. Let Q be a quadric hypersurface in P of corank r. If s < 2 , r then Fs,n(Q) is irreducible of dimension 2n − 3s − 4 (s + 1) . 2 n−r−2 r If s ≥ 2 and n − r is even, then Fs,n(Q) has two irreducible components each of dimension n − 2s + r − 2 (n − r − 2) (n − r) (s + 1) + . 2 8 n−r−2 r If s ≥ 2 and n − r is odd, then Fs,n(Q) is irreducible of dimension n − 2s + r − 3 (n − r − 1) (n − r + 1) (s + 1) + . 2 8 1.2. Preliminaries on orthogonal Grassmannians. Let W be an n-dimensional vector space endowed with a non-degenerate, symmetric, bilinear form Q. Set n m = b 2 c. Let 0 < k ≤ m denote a positive integer. Let OG(k, n) denote the k- dimensional subspaces of W isotropic with respect to the form Q, unless n = 2k. In the latter case, the parameter space of k-dimensional isotropic subspaces of W has two isomorphic irreducible components. OG(k, n) denotes one of these irreducible components. The orthogonal Grassmannian OG(k, n) is isomorphic to one irreducible compo- 0 nent of the Fano variety Fk−1,n(Q) of (k − 1)-dimensional projective linear spaces 2 on a smooth quadric hypersurface. The non-degenerate quadratic form Q defines the smooth quadric hypersurface in Pn−1. A linear space is isotropic with respect to Q if and only if its projectivization is contained in the quadric hypersurface defined by Q. In particular, by the discussion in §1.1, the dimension of OG(k, n) is k(2n − 3k − 1) 2

The cohomology of OG(k, n) is generated by the classes of Schubert varieties. There are minor differences in the cohomology of OG(k, n) depending on the parity of n due to the fact that when n is even, the half-dimensional isotropic subspaces form two connected components. For even n, the notation has to distinguish between these two connected components. For simplicity, we will first discuss the case of odd n, then describe the necessary modifications for even n. We begin by describing the Schubert varieties in OG(k, 2m + 1). Let λ denote a sequence m ≥ λ1 > λ2 > ··· > λs > 0 of strictly decreasing integers, where s ≤ k. Given λ, there is an associated sequence ˜ ˜ m − 1 ≥ λs+1 > ··· > λm ≥ 0 of strictly decreasing integers defined by requiring that there does not exist any ˜ parts λi for which λj +λi = m. In other words, the associated partition is obtained by removing the integers m − λ1, . . . , m − λs from the sequence m − 1, m − 2,..., 0. For example, if m = 6, then the partition associated to (6, 4) is (5, 4, 3, 1). The Schubert varieties in OG(k, 2m + 1) are parameterized by pairs (λ, µ), where λ is a strictly decreasing partition of length s and µ

m − 1 ≥ µs+1 > µs+2 > ··· > µk ≥ 0 is a subpartition of λ˜ (i.e., the parts of µ are a subset of the parts of λ˜) of length k − s. We will call such pairs of partitions allowed pairs. Observe that for maximal isotropic Grassmannians OG(m, 2m + 1), the partition µ = λ˜ is uniquely determined by the partition λ. Consequently, in the literature it is standard to omit the sequence µ and parametrize Schubert varieties by strict partitions λ. We will find it useful to record the dimensions of all the flag elements where a jump in dimension occurs, so we add µ to the notation. For non-maximal Grassmannians there are several notations in use. The advantage of our notation is that it minimizes the amount of calculation needed to determine the dimensions of the flag elements where a jump in dimension occurs. Since µ is a subpartition of λ˜ we can assume ˜ ˜ that it occurs as λis+1 , ··· , λik . Given a pair (λ, µ), the discrepancy dis(λ, µ) of the pair is defined by k X dis(λ, µ) = (m − k)s + (m − k + j − ij). j=s+1

Fix an isotropic flag F• ⊥ ⊥ 0 ⊂ F1 ⊂ F2 ⊂ · · · ⊂ Fm ⊂ Fm−1 ⊂ · · · ⊂ F1 ⊂ W. ⊥ Here Fi denotes the orthogonal complement of Fi with respect to the bilinear form. In terms of the geometry of the quadric hypersurface Q ⊂ Pn−1 we can ⊥ describe Fj as follows. A one-dimensional isotropic subspace corresponds to a 3 point p ∈ Q ⊂ Pn−1. The annihilator of that subspace corresponds to the tangent Pn 2 space to Q at the point p. We can take Q to be given by the equation i=1 Xi = 0. We can assume the isotropic subspace is generated by v = (1, i, 0,..., 0). The annihilator of v is given by vectors (v1, v2, . . . , vn) such that v1 + iv2 = 0. On the other hand, the tangent space to the quadric hypersurface at p corresponding to v is given by X1 + iX2 = 0. So the annihilator of a vector consists precisely of those vectors lying in the tangent hyperplane to the quadric at the point corresponding ⊥ to the vector. To find Fj we take the intersection of all the tangent hyperplanes at n−1−j the points of Fj. The intersection is the projective linear space P everywhere tangent to Q along the projectivization of Fj. µ The Schubert variety Ωλ(F•) is defined as the closure of the locus {[Λ] ∈ OG(k, 2m+1)| dim(Λ∩F ) = i for 1 ≤ i ≤ s, dim(Λ∩F ⊥ ) = j for s < j ≤ k}. m+1−λi µj Ps The codimension of a Schubert variety is given by i=1 λi + dis(λ, µ). We will µ µ denote the cohomology class of Ωλ by σλ . The description of the Schubert varieties in OG(k, 2m) requires minor modifications to account for the fact that the space of m-dimensional isotropic subspaces have two irreducible components. Let λ denote a sequence

m − 1 ≥ λ1 > λ2 > ··· > λs ≥ 0 of strictly decreasing integers where s ≤ k. When k = m and m is even (respectively, odd), we will assume that s is even (respectively, odd). Given λ, we can deﬁne an associated sequence λ˜ of strictly decreasing integers ˜ ˜ m − 1 ≥ λs+1 > ··· > λm ≥ 0 ˜ satisfying the condition that there does not exists λi such that λi + λj = m − 1. In other words, to obtain λ˜ remove from the sequence m − 1,..., 0 the integers m − 1 − λ1, . . . , m − 1 − λs. The Schubert varieties in OG(k, 2m) are parameterized by pairs (λ, µ), where λ is a strictly decreasing partition of length s and µ

m − 1 ≥ µs+1 > µs+2 > ··· > µk ≥ 0 is a subpartition of λ˜ of length k − s. We will call such pairs of partitions allowed pairs. As above, for maximal isotropic Grassmannians OG(m, 2m), the partition µ = λ˜ is uniquely determined by the partition λ, so it is often omitted from the notation. The pair (λ, µ) is a subpartition of a pair (λ0, λ˜0) of total length m defined as follows. If m and s have the same parity, then λ = λ0. If m and s have different parities, λ0 has length s + 1 and differs from λ in that it includes the smallest number between 0 and m − 1 not already occurring in λ and not adding to m − 1 with any of the parts in µ. The discrepancy dis(λ, µ) of the pair (λ, µ) is defined as follows: Since (λ, µ) is a subpartition of (λ0, λ˜0), we can assume that the parts occur as λ0 , . . . , λ0 , λ˜0 , ··· , λ˜0 . The discrepancy is defined as i1 is is+1 ik k X dis(λ, µ) = (m − k + j − ij). j=1

We will make the convention that Fm denotes an m-dimensional isotropic subspace in one of the irreducible components. By abuse of notation, we will denote ⊥ by Fm−1 an m-dimensional isotropic subspace in the other irreducible component. 4 ⊥ Note that strictly speaking the intersection of the quadric hypersurface with Fm−1 consists of the union of two m-dimensional isotropic subspaces one in each irreducible component. Our slight abuse of notation will make notation more compact. We will use this convention without further mention in the rest of the paper. The µ Schubert variety Ωλ(F•) is defined as the closure of the locus {[Λ] ∈ OG(k, 2m)| dim(Λ∩F ) = i for 1 ≤ i ≤ s, dim(Λ∩F ⊥ ) = j for s < j ≤ k}. m−λi µj P 0 The codimension of a Schubert variety is given by λi + dis(λ, µ). We will denote µ µ the cohomology class of Ωλ by σλ . µ The cohomology classes σλ , as (λ, µ) varies over all allowed pairs, form an ad- ditive basis of the cohomology ring of OG(k, n). Given an allowed pair (λ, µ) for OG(k, 2m + 1), there is a dual allowed pair (λc, µc) defined by c c c c λ1 = m − µk, . . . , λk−s = m − µs+1, µk−s+1 = m − λs, . . . , µk = m − λ1. Similarly, if (λ, µ) is an allowed pair for OG(k, 2m), define the dual pair (λc, µc) by setting c c c c λ1 = m−1−µk, . . . , λk−s = m−1−µs+1, µk−s+1 = m−1−λs, . . . , µk = m−1−λ1.

c c c µ µ If (λ, µ) and (λ , µ ) are dual allowed pairs, then σλ · σλc is equal to the Poincar´e dual of the point class. In this section, we recall basic facts concerning the geometry of isotropic Grass- mannians. Let n = 2m be a positive, even integer. Let V be an n-dimensional vector space over C. Let Q be a non-degenerate, skew-symmetric form on V . By Darboux’s Theorem, we can choose a basis for V such that in this basis Q is expressed as Pm T i=1 xi ∧ yi. A subspace W of V is called isotropic if w Qv = 0 for any two vectors v, w ∈ W . The dimension of an isotropic subspace of V is at most m. Given a vector space W , the orthogonal complement W ⊥ of W is deﬁned as the set of v ∈ V such that vT Qw = 0 for every w ∈ W . If the dimension of W is k, then the dimension of W ⊥ is n − k and the restriction of Q to W ⊥ has rank n − 2k (or, equivalently, corank k). The Grassmannian SG(k, n) parameterizing k-dimensional isotropic subspaces of V is a homogeneous variety for the symplectic group Sp(n). The Grassmannian SG(m, n) parameterizing maximal isotropic subspaces has dimension m(m + 1) dim(SG(m, n)) = . 2 This can be seen inductively. The dimension of SG(1, 2) =∼ P1 is one since every vector is isotropic with respect to Q. Consider the incidence correspondence

I = {(w, W ) | w ∈ P(W ) and [W ] ∈ SG(m, n)} parameterizing a pair of a maximal isotropic subspace W and a point w of P(W ). The first projection of the incidence correspondence I maps to P(V ) with fibers isomorphic to SG(m − 1, n − 2). The second projection maps the incidence correspondence to SG(m, n) with fibers isomorphic to P(W ). By the Theorem on the Dimension of Fibers [S, I.6.7] and induction, we conclude that the dimension of m(m+1) SG(m, n) is 2 . 5 The dimension of the isotropic Grassmannian SG(k, n) is m(m + 1) (m − k)(3k − m − 1) 3k2 − k dim SG(k, n) = + = nk − . 2 2 2 To see this, consider the incidence correspondence

I = {(W1,W2) | W1 ∈ SG(k, n),W2 ∈ SG(m, n),W1 ⊂ W2} parameterizing two-step flags consisting of a k-dimensional isotropic space contained in a maximal isotropic space. Since every k-dimensional isotropic space can be completed to a maximal isotropic space, the first projection is onto SG(k, n). The fibers of the first projection are isomorphic to the isotropic Grassmannian SG(m − k, n − 2k). The second projection is onto SG(m, n) with fibers isomorphic to G(k, m). The Theorem on the Dimension of Fibers [S, I.6.7] and the previous paragraph imply the claim. More generally, we will need to study spaces parameterizing k-dimensional linear r spaces isotropic with respect to a degenerate skew form Qn of corank r on an n- dimensional vector space. Naturally, n − r needs to be even. Since the restriction r of Qn to a linear space complementary to its kernel is non-degenerate, we conclude n−r n−r that the largest dimensional isotropic subspace has dimension r+ 2 . Set h = 2 . r Then the space of (r + h)-dimensional isotropic linear spaces with respect to Qn h(h+1) is isomorphic to SG(h, 2h) and has dimension 2 . Considering the incidence correspondence r I = {(W1,W2) | W1 ⊂ W2 isotropic with respect to Qn,

dim(W1) = k, and dim(W2) = h + r}, r we see that the space of k-dimensional isotropic subspaces of Qn has dimension h(h+1) h(h+1) (h−k)(h−k+1) 2 + k(h + r − k) if k ≥ h and 2 + k(h + r − k) − 2 if k < h. The cohomology of SG(k, n) is generated by the classes of Schubert varieties. Let 0 ≤ s ≤ k be a non-negative integer. Let λ• : 0 < λ1 < λ2 < ··· < λs ≤ m be a sequence of increasing positive integers. Let µ• : m > µs+1 > µs+2 > ··· > µk ≥ 0 be a sequence of decreasing non-negative integers such that λi 6= µj + 1 for any 1 ≤ i ≤ s and s < j ≤ k. Then the Schubert varieties in SG(k, n) may be indexed by pairs of admissible sequences (λ•; µ•). Fix an isotropic ﬂag ⊥ ⊥ F• = F1 ⊂ F2 ⊂ · · · Fm ⊂ Fm−1 ⊂ · · · F1 ⊂ V.

The Schubert variety Σλ•;µ• (F•) is deﬁned as the Zariski closure of the set of linear spaces {W ∈ SG(k, n) | dim(W ∩ F ) = i for 1 ≤ i ≤ s, dim(W ∩ F ⊥ ) = j for s < j ≤ k}. λi µj

In the literature, it is customary to denote Schubert classes in the cohomology of SG(m, n) by strictly decreasing partitions m ≥ a1 > a2 > ··· > as > 0 of length s ≤ m. In our notation, the sequence a• translates to the sequence λ• by setting ai = m + 1 − λi. Note that when n = 2m, the sequence λ• determines the sequence µ• by the requirement that λi 6= µj +1 for any 1 ≤ i ≤ s and s < j ≤ m. Therefore, it is common to omit the sequence µ• from the notation. We will not follow this convention. In Schubert calculus, many authors prefer to record Schubert classes so that the codimension will be easily accessible. Our notation has the advantage 6 that it is preserved under natural maps between Grassmannians arising from linear embeddings between ambient vector spaces. We will index Schubert classes in the cohomology of the Grassmannian G(k, n) by increasing sequences of non-negative integers a• : 0 < a1 < a2 < ··· < ak ≤ n.

The Schubert variety Σa• (F•) with respect to a ﬂag F• parameterizes k-dimensional subspaces W of V that satisfy dim(W ∩ Fai ) ≥ i for 1 ≤ i ≤ k.

2. The restriction problem The orthogonal or the symplectic Grassmannian naturally includes in the ordinary Grassmannian. The restriction problem asks for computing the induced map on cohomology in terms of the Schubert classes. This is the geometric analogue of the restriction problem in representation theory. Our solution to this problem will be via specialization. In the case of orthogonal Grassmannians, the quadratic form Q defines a smooth degree two hypersurface Q in PW . We will interpret OG(k, n) as the Fano variety of (k − 1)-dimensional projective linear subspaces on Q. We will also need to study singular quadric hypersurfaces. Over the complex numbers, the projective equivalence class of a quadric hypersurface is determined by its dimension and corank. Let Qri denote a quadratic form of corank r obtained by restricting Q to di i a vector space of dimension di. Let Lnj denote an isotropic linear space of (vector space) dimension nj. A restriction variety in OG(k, n) is defined in terms of a sequence r L ⊂ · · · ⊂ L ⊂ Q k−s ⊂ · · · ⊂ Qr1 n1 ns dk−s d1 of isotropic linear spaces and quadrics. (In Definitions 4.2 and 4.9, we will specify the conditions that these linear spaces and quadrics need to satisfy. For the purposes of the introduction we ignore these subtleties.) The restriction variety parameterizes the isotropic linear spaces that intersect L in a subspace of dimension j and Qri nj di in a subspace dimension k − i + 1 for every 1 ≤ j ≤ s and 1 ≤ i ≤ k − s. Schubert varieties are examples of restriction varieties with the property that the quadrics in the sequence are as singular as possible (i.e., di + ri = n). The strategy to calculate the class of a restriction variety is to specialize the quadrics in the sequence one at a time to become more singular until they are maximally singular. When we specialize the quadrics, the restriction variety breaks into a union of simpler restriction varieties. The process is governed by the following basic facts about quadrics.

• The corank bound. Let Qr2 ⊂ Qr1 be two linear sections of Q such that the d2 d1 singular locus of Qr1 is contained in the singular locus of Qr2 . Then r −r ≤ d −d . d1 d2 2 1 1 2 In particular, the corank of a sub-quadric in Q is bounded by its codimension. • The linear space bound. The largest dimensional isotropic linear space with r d+r respect to a quadratic form Qd has dimension b 2 c. A linear space of dimension r j intersects the singular locus of Qd in a subspace of dimension at least max(0, j − d−r b 2 c). d−2 • Irreducibility. A sub-quadric Qd of Q is reducible and equal to the union of two linear spaces of (vector space) dimension d − 1 meeting along a linear space of k−1 dimension d − 2. If n = 2k, then the linear spaces constituting Qk+1 belong to two distinct connected components. 7 r • The variation of tangent spaces. Let a quadric Qd be singular along a codimension j linear subspace M of a linear space L. Then the image of the Gauss r map of Qd restricted to the smooth points of L has dimension at most j − 1. In r other words, the tangent spaces to Qd along the smooth points of L vary at most in a (j − 1)-dimensional family. The corank bound determines the order of the specialization. We increase the corank of the smallest dimensional quadric Qri that satisfies d + r < d + r di i i i−1 i−1 by one, i.e., we replace Qri in the sequence with Qri+1. The algorithm is obtained di di by describing the flat limit of this specialization. Suppose that a general linear space parametrized by the restriction variety intersects the singular locus of Qri di in a subspace of dimension xi. The linear spaces parametrized by the flat limit intersect the singular locus of Qri+1 in a subspace of dimension x or x + 1. The di i i limit has more than one component when both cases are possible. ‘The linear space bound’ and ‘the variance of tangent spaces’ dictate which of the possibilities occur. In addition, if ri = di − 3, then by the ‘irreducibility’ property, the new quadric Qri+1 is reducible forcing the limit to possibly have more components. Surprisingly, di each of these components occur with multiplicity one in the limit. The algorithm is obtained by inductively applying this specialization to each irreducible component. We refer the reader to §5 for the precise statement of the algorithm and detailed examples. The case of SG(k, n) is similar. The computation depends on four very sim- r ple geometric principles. We now explain these principles. Let Qd denote a d- r dimensional vector space such that the restriction of Q has corank r. Let Ker(Qd) r denote the kernel of the restriction of Q to Qd. Let Lj denote an isotropic sub- ⊥ space of dimension j with respect to Q. Let Lj denote the set of w ∈ V such that T w Qv = 0 for all v ∈ Lj. Evenness of rank. The rank of a non-degenerate skew-symmetric form is even. r r Hence, d − r is even for Qd. Furthermore, if d = r, then Qd is isotropic. The corank bound. Let Qr1 ⊂ Qr2 and let r0 = dim(Ker(Qr2 ) ∩ Qr1 ). Then d1 d2 2 d2 d1 0 r r1 − r2 ≤ d2 − d1. In particular, d + r ≤ n for Qd. r The linear space bound. The dimension of an isotropic subspace of Qd is d+r bounded above by b 2 c. Furthermore, an m-dimensional linear space L satis- r d−r fies dim(L ∩ Ker(Qd)) ≥ m − b 2 c. The kernel bound. Let L be an (s + 1)-dimensional isotropic space such that r r dim(L ∩ Ker(Qd)) = s. If an isotropic linear subspace M of Qd intersects L − r ⊥ Ker(Qd), then M is contained in L . These four principles dictate the order of the specialization and determine the limits that occur. Given a flag, we will specialize the smallest dimensional non- r isotropic subspace Qd, whose corank can be increased subject to the corank bound, r ˜r+2 keeping all other flag elements unchanged. We will replace Qd with Qd . The branching rule simply says that under this specialization, the limit L0 of a linear space L satisfying rank conditions with respect to the original flag satisfies the same 0 ˜r+2 rank conditions with the unchanged flag elements and either dim(L ∩Ker(Qd )) = r 0 ˜r+2 r dim(L ∩ Ker(Qd)) or dim(L ∩ Ker(Qd )) = dim(L ∩ Ker(Qd)) + 1. Furthermore, both of these cases occur with multiplicity one unless the latter leads to a smaller 8 dimensional variety or the former violates the linear space bound. See Sections 3 and 8 for an explicit statement of the rule and for examples.

3. Combinatorics In this section, we present the rule for computing restriction coeﬃcients of OG(k, n) and SG(k, n) combinatorially.

3.1. The orthogonal case. Consider a sequence of n integers written from left to right. We say that a bracket or brace is in position i if i of the integers are to the left of the bracket or brace. Definition 3.1. Let 0 ≤ s ≤ k < n be integers. A orthogonal sequence of brackets and braces of type (k, n) is a sequence of n natural numbers, s right brackets ] and k − s right braces } such that: • Every bracket or brace occupies a positive position and each position is occupied by at most one bracket or brace. • Every number i in the sequence satisfies 0 ≤ i ≤ k−s. The positive integers in the sequence are non-decreasing from left to right and are to the left of every zero in the sequence. • Every bracket is to the left of every brace. • If 2k = n, a bracket in the k-th position may either be a bracket ] or a bracket decorated with a prime ]0. For example, 1]1]122]33]0000}00}00}000 is an orthogonal sequence of brackets and braces of type (7, 18) with s = 4. To be concrete, the first rule forbids 0]]0, 0}}0 (two brackets or two braces in the same position), 00]}00 (a bracket and a brace in the same position), ]100 (a bracket that is not in a position). The second rule forbids numbers that look like 1132 (3 is not allowed to be to the left of 2) or 11200300 (3 should be to the left of any zero). The third rule forbids 000}00]0 (a brace cannot be to the left of a bracket). Notation 3.2. We order the brackets in the sequence from left to right and the braces in the sequence from right to left. In our example, 1]11]2122]333]40000}300}200}1000 the small numbers above the brackets and braces indicate their order. Let ρ(i, j) denote the number of integers to the right of the i-th brace and to the left of the j-th brace. Let ρ(i, 0) denote the number of integers to the right of the i-th brace. In our example, ρ(3, 2) = 2, ρ(2, 1) = 2, ρ(1, 0) = 3. Let p(]i) and p(}i) denote the number of integers to the left of the i-th bracket and i-th brace, respectively. These record the positions of the brackets and braces. In our running example, p(]1) = 1, p(]2) = 2, p(]3) = 5, p(]4) = 7 and p(}3) = 11, p(}2) = 13, p(}1) = 15. Let l(i) denote the number of integers in the sequence that are equal to i. Let l(≤ i) denote the number of positive integers in the sequence that are less than or equal to i. In our running example, l(1) = 3, l(2) = 2, l(3) = 2, l(≤ 2) = 5, l(≤ 3) = 7. When we are discussing more than one sequence, we will write ρD, pD and lD for the invariants of the sequence D. We are now ready to define quadric diagrams, which are the main combinatorial objects of this paper. The first three conditions in the definition do not play a role in the algorithm. They are included for precision and the reader may ignore them in a first reading. The last three conditions are crucial and the reader should remember them. 9 Definition 3.3. A quadric diagram for OG(k, n) is an orthogonal sequence of brackets and braces of type (k, n) with s brackets such that the following conditions hold. (D1) l(i) ≤ ρ(i, i − 1) for 1 ≤ i ≤ k − s. (D2)2 p(]s) ≤ p(}k−s) + l(≤ k − s). (D3) Suppose that the integer 0 < i < k − s occurs in the sequence. If i + 1 does not occur in the sequence, either i = 1 and every position after a 1 is occupied by a bracket, or l(j) = ρ(j, j − 1) for every j > i + 1 and ρ(i + 1, i) = 1. (D4) There are at least three zeros to the left of }k−s. j (D5) Let xi be the number of brackets such that p(] ) ≤ l(≤ i). Then p(}i) − l(≤ i) x ≥ k − i + 1 − . i 2 (D6) The two integers immediately to the left of a bracket are equal. If there is only one integer to the left of a bracket and s < k, then the integer is 1. Remark 3.4. Quadric diagrams index restriction varieties, which will be introduced in the next section and are the main geometric objects of study in this paper. Example 3.5. Let us give a few examples to clarify the meaning of these conditions. The first condition says that the number of times i appears in the sequence is less than or equal to the number of integers between the i-th and (i − 1)-st braces. In particular, the following are forbidden 2220000}00}0 (l(2) = 3, but ρ(2, 1) = 2), 11000}0 (l(1) = 2, but ρ(1, 0) = 1). Let the right most bracket be at position p(]s) and the left most brace be at position p(}k−s). The second condition says that twice p(]s) is less than or equal to the sum of p(}k−s) and the number of positive integers in the sequence. For example, 00]00}0, 100]00}0 are allowed, but 000]00}0 is not (2p(]1) = 6 > p(}1) = 5). The third condition is a consequence of the order in the algorithm. The reader does not have to pay attention to it except in a few places in the proof of the algorithm, where it simplifies the dimension counts. The rule says that if a positive integer occurs in the sequence, then all the larger integers (less than or equal to k − s) also occur in the sequence except in two very special cases. For example, 1]1]330000}00}0}00 (all the 1s are followed by brackets) and 1]1330000}00}0}00 (2 is missing, but l(3) = ρ(3, 2) = 2 and l(2) = ρ(2, 1) − 1 = 0) are allowed, but 1]130000}00}0}00 is not (2 is missing, but l(3) = 1 6= ρ(3, 2) = 2). These conditions are preserved during the algorithm. The reader may ignore them in a first reading. The last three conditions are the important conditions that the reader has to remember. The fourth condition is self-evident. It allows 11]00]00}00 or 33000}00}00}0, but does not allow 1100}00. The sixth condition is also self-evident. It allows for 1]22]33]0000}00}0}0 or 22]22]2000}00000}0, but disallows 2]22]000}000}0 (there is only one integer to the left of ]1, but it is not 1) or 1234]0000}0}0}0}0 (the two numbers preceding the bracket are not equal). The fifth condition is the one that is hardest to visually verify without resorting to some counting. In words, it says that the number of integers that are to the right of the right-most i and to the left of the i-th brace has to be at least twice the total number of brackets and braces that are at positions greater than l(≤ i) and less than or equal to p(}i). For example, it disallows 10]00}0 (There are three zeros to the right of the 1 that are to the left of 10 }. There is one bracket and one brace in positions greater than 1 and less than or equal to 4. However, 3 6≥ 4). We are now ready to state the algorithm. We begin by defining a new set of sequences of brackets and braces associated to D. The new sequences Da and Db defined below may fail to be quadric diagrams, but we address such instances below. Definition 3.6. If there exists an index i in D such that l(i) < ρ(i, i − 1), let κ = max(i | l(i) < ρ(i, i − 1)). Let Da be the sequence of brackets and braces obtained by changing the (l(≤ κ) + 1)-st integer in the sequence D to κ. s i b If pDa (] ) > lDa (≤ κ), let η = min(i | pDa (] ) > lDa (≤ κ)). Let D be the sequence of brackets and braces obtained from Da by moving the bracket ]η to the position lDa (≤ κ). To clarify, let us give some examples. Let D = 233]0000}00}0}0. Then κ = 1. We change the integer in the position l(≤ 1) + 1 (in this case the left most 2) to 1 to obtain Da = 133]0000}00}0}0. We slide the first bracket in Da to the right of the 1 we added to the immediate right of it to obtain Db = 1]330000}00}0}0. Note that in this case both Da and Db are quadric diagrams. Next let D = 00]0]0000}0. Here κ = 1, so we turn the left most 0 into 1 to obtain Da = 10]0]0000}0. We slide the first bracket to the right of the 1 to its immediate right to obtain Db = 1]00]0000}0. Here note that Db is a quadric diagram, but Da fails condition (D6). We have to turn Da into a quadric diagram. Here is the algorithm that turns Da into a quadric diagram. Algorithm 3.7. • If Da fails condition (D5), discard it. Da does not lead to any quadric diagrams. • If Da satisfies condition (D5) but not condition (D6), change the (l(≤ κ) + 1)-st integer in the sequence to κ and move }κ one position to the left. Repeat until you reach a sequence of brackets and braces that satisfies condition (D6). Label the resulting sequence Dc. If Dc is a quadric diagram, we refer to it as a quadric diagram derived from Da. Otherwise, proceed to the next step. • If Da or Dc satisfy conditions (D5) and (D6), but fail condition (D4), replace Da or Dc with two identical diagrams Da1 and Da2 obtained by replacing }k−s (in Da or Dc) with ]s+1 in position p(}k−s) − 1 and turning the digits equal to k − s to 0. If 2p(]s+1) = n, then we use (]0)s+1 instead of ]s+1 in Da2 . We refer to Da1 and Da2 as quadric diagrams derived from Da. Furthermore, if 2k = n and 2p(]s+1) = n, then discard the diagram with ]s+1 (respectively, (]0)s+1) if s + 1 6= k mod (2) (respectively, if s + 1 = k mod (2)). In our example, we first turn Da = 10]0]0000}0 to 11]0]000}00. This diagram still fails condition (6), so we repeat to obtain 11]1]00}000. Now condition (D6) is satisfied, but condition (D4) fails. Since n = 8 = 2 · 4, we obtain the two diagrams 00]0]0]0000 and 00]0]0]00000. These are the two diagrams derived from Da. Let D = 000}000}000}, then κ = 3. We turn the left most 0 into 3 to obtain Da = 300}000}000}. In this case, there are no brackets to the left of the 3, so there is no Db. The sequence Da fails condition (D4). Since n is odd, we replace Da with two identical quadric diagrams Da1 = 00]0000}000} and Da2 = 00]0000}000}. Let D = 00]0000}00}0. Then Da = 20]0000}00}0 and Db = 2]00000}00}0. Neither of these diagrams satisfy condition (D6). We already know that we should replace Da with 22]000}000}0. Here is how to modify Db. 11 Algorithm 3.8. • If Db does not satisfy condition (D6), let ]j be the bracket for which it fails. Let i be the integer immediately to the left of ]j. Replace i with i − 1 and move }i−1 one position to the left. As long as the resulting sequence does not satisfy condition (D6), repeat this process either until the resulting sequence is a quadric diagram (in which case this is the quadric diagram derived from Db) or two braces occupy the same position. In the latter case, no quadric diagrams are derived from Db. In our example, we replace Db = 2]00000}00}0 with 1]00000}0}00, which is a quadric diagram. If our example had been D = 00]0000}0}0, then Db = 2]00000}0}0. Replacing 2 with 1 and moving }1 to the left would produce 1]00000}}00. Hence, in this case no quadric diagrams are derived from Db. We need one final definition. Given a sequence of brackets and braces such that s xκ+1 p(] ) > l(κ), if l(≤ i) < p(] ) for all i, set yxκ+1 = k − s + 1. Otherwise, let xκ+1 xκ+1 yxκ+1 = max(i | l(≤ i) ≤ p(] )). If there is a 0 to the left of ] , then yxκ+1 is xκ+1 k − s + 1. Otherwise, yxκ+1 is the largest integer that occurs to the right of ] , which is the first bracket occurring in a position greater than l(≤ κ). The condition xκ+1 p(] )−l(≤ κ)−1 = yxκ+1−κ will play an important role. In words, this condition says that the number of integers larger than κ to the left of ]xκ+1 is one more than the cardinality of the set of integers greater than κ (or zero) occurring to the left of ]xκ+1. In view of condition (D3), a sequence satisfying this equality looks like ··· κ + 1 κ + 2 ··· κ + l − 1 κ + l κ + l] ··· or ··· κ + 1 κ + 2 ··· κ + l − 1 00] ··· , where we have drawn the part of the sequence starting with the left most κ + 1 and ending with ]xκ+1. We are now ready to state the algorithm. Algorithm 3.9. Let D be a quadric diagram. If l(i) = ρ(i, i − 1) for every 1 ≤ i ≤ k − s, then return D and stop. Otherwise, let Da and Db be as above.

xκ+1 s (1) If p(] ) − l(≤ κ) − 1 > yxκ+1 − κ or p(] ) ≤ l(κ) in D, then return the quadric diagrams that are derived from Da. (2) If Da violates condition (D5), then return the quadric diagrams that are derived from Db. (3) Otherwise, return the quadric diagrams that are derived from both Da and Db. Remark 3.10. In the proof of Theorem 5.12, we will check in detail that Algorithm 3.9 always returns at least one quadric diagram. Briefly, Da does not lead to a quadric diagram only if it violates condition (D5). In that case, by the definition of κ, there has to be equality in condition (D5) for all indices κ ≤ i ≤ k − s in the diagram D. Then, condition (D4) implies that there has to be a bracket to the right a xκ+1 of κ in D ; and condition (D6) implies that p(] ) − l(≤ κ) − 1 = yxκ+1 − κ in D. Finally, while running Algorithm 3.36, if two braces occupy the same position, then condition (D5) is violated for the index κ − 1 in the diagram D. These considerations imply that there is a quadric diagram derived from Db by Algorithm 3.9 (see paragraph 6 of the proof of Theorem 5.12 for more details). 3.2. The symplectic case. Notation 3.11. Let 0 ≤ s ≤ k be an integer. A sequence of n natural numbers of type s for SG(k, n) is a sequence of n natural numbers such that every number is less than or equal to k − s. We write the sequence from left to right with a small gap to the right of each number in the sequence. We refer to the gap after the 12 i-th number in the sequence as the i-th position. For example, 1 1 2 0 0 0 0 0 and 3 0 0 2 0 1 0 0 are two sequences of 8 natural numbers of types 1 and 0, respectively, for SG(3, 8). Definition 3.12. Let 0 ≤ s ≤ k be an integer. A sequence of brackets and braces of type s for SG(k, n) consists of a sequence of n natural numbers of type s, s brackets ] ordered from left to right and k − s braces } ordered from right to left such that: (1) Every bracket or brace occupies a position and each position is occupied by at most one bracket or brace. (2) Every bracket is to the left of every brace. (3) Every positive integer greater than or equal to i is to the left of the i-th brace. (4) The total number of integers equal to zero or greater than i to the left of the i-th brace is even. Example 3.13. 11]200}0}00 and 300}20}10}0 are typical examples of sequences of brackets and braces for SG(3, 8) that have the two examples from Notation 3.11 as their sequences of natural numbers. When writing a sequence of brackets and braces, we often omit the gaps not occupied by a bracket or a brace. Example 3.14. Let us give several non-examples to clarify Definition 3.12. The first condition disallows diagrams such as ]0000} (the first bracket is not in a position), 0]]000, 000}}0, 00]}00 (two brackets, two braces, or a bracket and a brace occupy the same position, respectively). The second condition disallows diagrams such as 00}0]000 (a brace cannot be to the left of a bracket). The third condition disallows diagrams such as 100}30}20}0 (3 is to the right of the third brace and 2 is to the right of the second brace). The fourth condition disallows diagrams such as 1]2000}0}00 (the number of zeros to the left of the second brace, and the number of zeros and twos to the left of the first brace are odd). Notation 3.15. By convention, the brackets are indexed from left to right and the braces are indexed from right to left. We write ]i and }i to denote the i-th bracket and i-th brace, respectively. Their positions are denoted by p(]i) and p(}i). The position of a bracket or a brace is equal to the number of integers to its left. For notational convenience, we declare that, in a sequence of brackets and braces of type s for SG(k, n), the brace }k−s+1 denotes ]s and an integer in the sequence equal to k − s + 1 should be read as 0. Let l(i) denote the number of integers in the sequence that are equal to i. Let ri be the total number of positive integers less than or equal to i that are to the left of }i. For 0 < j < i, let ρ(i, j) = p(}j) − p(}i) and let ρ(i, 0) = n − p(}i). Equivalently, ρ(i, 0) (respectively, ρ(i, j)) denotes the number of integers to the right of the i-th brace (respectively, to the right of the i-th brace and to the left of the j-th brace). Example 3.16. For the sequence of brackets and braces 300}20}10}0 for SG(3, 8), 3 2 1 the positions are p(} ) = 3, p(} ) = 5, p(} ) = 7. We have ri = l(i) = 1, for 1 ≤ i ≤ 3, ρ(i, i − 1) = 2, for 2 ≤ i ≤ 3, and ρ(1, 0) = 1. Example 3.17. For the sequence of brackets and braces 1]22]00}00}0 for SG(4, 8), 1 2 2 1 the positions are p(] ) = 1, p(] ) = 3, p(} ) = 5, p(} ) = 7. We have r1 = l(1) = 1, l(2) = 2, and r2 = 3. Moreover, ρ(2, 1) = 2 and ρ(1, 0) = 1. Definition 3.18. Two sequences of brackets and braces are equivalent if the lengths of their sequence of numbers are equal, the brackets and braces occur at the same 13 positions, and the collection of digits that occur between any consecutive brackets and/or braces are the same up to reordering. Example 3.19. The sequences 1221]00200}000}00, 1122]20000}000}00 and the sequences 003}02}01}0, 300}20}10}0 are equivalent pairs of sequences. We can depict an equivalence class of sequences by the representative where the digits are listed so that between any two consecutive brackets and/or braces the positive integers precede the zeros and are listed in non-decreasing order. We will always use this canonical representative and often blur the distinction between the equivalence class and this representative. Definition 3.20. A sequence of brackets and braces is in order if the sequence of numbers consists of a sequence of non-decreasing positive integers followed by zeros except possibly for one i immediately to the right of }i+1 for 1 ≤ i < k − s. Otherwise, we say that the sequence is not in order. A sequence is in perfect order if the sequence of numbers consists of non-decreasing positive integers followed by zeros. Example 3.21. The sequences 300}20}10}000, 11]22]00}00}00, 1]33]0000}200}0}0 are in order. Furthermore, 11]22]00}00}00 is in perfect order. The sequences 11]00]100}000, 1]20000}1}0}00, 122]100}00}00 are not in order. Definition 3.22. A sequence of brackets and braces is saturated if l(i) = ρ(i, i − 1) for 1 ≤ i ≤ k − s. Example 3.23. The sequences 11]22]00}00}00 and 1]22]100}00}00 are saturated, whereas, 22]00}00}00 and 1]0000}00}000 are not. The next definition is a technical definition that plays a role in the proof and is a consequence of the order in which the game is played. The reader can define a symplectic diagram as a sequence of brackets and braces that occurs in the game and refer to the conditions only when necessary.

Deﬁnition 3.24. A symplectic diagram for SG(k, n) is a sequence of brackets and braces of type s for SG(k, n) for some 0 ≤ s ≤ k such that: (S1) l(i) ≤ ρ(i, i − 1) for 1 ≤ i ≤ k − s. s s (S2) Let τi be the sum of p(] ) and the number of positive integers between ] and }i. Then i 2τi ≤ p(} ) + ri. (S3) Either the sequence is in order or there exists at most one integer 1 ≤ η ≤ k − s such that the sequence of integers is non-decreasing followed by a sequence of zeros except for at most one occurrence of η between ]s and }η+1 and at most one occurrence of i < η after }i+1. j j−1 (S4) Let ξj denote the number of positive integers between } and } . If an integer i occurs to the left of all the zeros, then either i = 1 and there is a bracket in the position following it, or there exists at most one index j0 such

that ρ(j, j−1) = l(j) for j0 6= j > min(i, η) and ρ(j0, j0 −1) ≤ l(j0)+2−ξj0 . Moreover, any integer η violating order occurs to the right of }j0 . Remark 3.25. Conditions (S1) and (S2) are necessary to guarantee that symplectic diagrams represent geometrically meaningful objects. Conditions (S3) and (S4) are consequences of the order the game is played and describe the most complicated 14 possible diagrams that can occur. The reader can ignore these conditions. They are necessary to carry out the dimension counts and to prove that the algorithm is defined at each step. They are not needed in order to run the algorithm. Example 3.26. Let us give some examples to clarify Definition 3.24. Condition (S1) allows for diagrams such as 11]22]2]00}000}00 but disallows 22]3300}2}00}000 (there are two 3’s and three 2’s in the sequence but ρ(2, 3) = 1 and ρ(1, 2) = 2). Condition (S2) disallows diagrams such as 000]10}0 (r1 = 1, τ1 = 4, but 2 · 4 > 5 + 1). Condition (S3) allows for 2344]300}00}00}10}0 (a non-decreasing sequence of positive integers 2344 followed by a sequence consisting of one 3, one 1 and zeros), but disallows 22]110000}2200}0000}00 (there are two 1s and two 2s following the non-decreasing sequence 22) or 22]133]00}00}00}0 (there are two 3s following the non-decreasing sequence 22). Condition (S4) allows for diagrams such as 11]3300}00}1}000, 1]1]33]00}00}00}00, however, it disallows diagrams such as 144]00}00}00}00}0 (1 occurs in the initial non-decreasing part of the sequence, but 2 and 3 do not occur. 1 is not followed by a bracket and l(3) = 0 6= ρ(3, 2) = 2, l(2) = 0 6= ρ(2, 1) = 2). The next definition is crucial for the game and the reader should remember these conditions. Definition 3.27. A symplectic diagram is called admissible if it satisfies the following additional conditions. (A1) The two integers to the left of a bracket are equal. If there is only one integer to the left of a bracket and s < k, then the integer is one. h h (A2) Let xi be the number of brackets ] such that every integer to the left of ] is positive and less than or equal to i. Then p(}i) − r x ≥ k − i + 1 − i . i 2 Example 3.28. Condition (A1) disallows diagrams such as 11]23]00}00}00}00 (the digits preceding the second bracket are not equal), 2]200}00}00 (there is a bracket in position 1, but the first digit is not 1). Condition (A2) is hard to visualize without resorting to counting. Let p be the position of the rightmost bracket such that every digit to the left of p is positive and less than or equal to i. In words, condition (A2) says that the total number of zeros and integers greater than i in the sequence is at least twice the number of brackets and braces in positions p + 1 i through p(} ). The following diagrams violate condition (A2): 22}00}00 (x2 = 0, 2 2 p(} ) = r2 = 2, but 0 < 1), 200}2}00} (the number of braces up to p(} ) = 4 is 2; the number of zeros is 2, but 2 < 2 · 2), 11]33]00}00}1}000 (the total number of brackets and braces between positions 3 and 9 = p(}1) is 4. The number of zeros and integers greater than 1 is 6, but 2 · 4 > 6). Remark 3.29. The admissible symplectic diagrams are the main combinatorial objects in this paper. They represent symplectic restriction varieties, which are the main geometric objects of the paper and will be defined in the next section. The symplectic diagram records a non-necessarily isotropic flag. The corresponding symplectic restriction variety parameterizes isotropic spaces that satisfy certain rank conditions with respect to this flag. The definition of an admissible symplectic diagram reflects the basic facts about isotropic subspaces discussed in the introduction, as we will see in the next section. 15 Definition 3.30. The symplectic diagram D(σλ;µ) associated to the Schubert class σλ;µ in SG(k, n) is the saturated symplectic diagram in perfect order, where the brackets occur at positions λ1, . . . , λs and the braces occur at positions n−µs+1, ··· , n − µk.

Example 3.31. The symplectic diagram associated to σ2,4;4,2 in SG(4, 10) is 11]22]00}00}00.

Lemma 3.32. The diagram D(σλ;µ) is an admissible symplectic diagram.

Proof. Let n = 2m. Since 0 < λ1 < ··· < λs ≤ m < n − µs+1 < ··· < n − µk, the brackets and braces occur in different positions and the brackets are to the left of the braces. Since the sequence is saturated and in perfect order, the number of integers in the sequence equal to i is µk−i+1 − µk−i+2 ≤ µs+1 < m (with the k−s convention that µk+1 = 0), for 1 ≤ i ≤ k − s and occur to the left of } . Finally, the number of integers equal to zero or greater than or equal to i to the left of }i is n − 2µk−i+1 = 2(m − µk−i+1). Therefore, D(σλ;µ) satisfies all 4 conditions in Definition 3.12. By definition, D(σλ;µ) is saturated, so l(i) = ρ(i, i − 1) and conditions (S1) and (S4) hold. Since the diagram is in perfect order, (S3) holds and

τi = max(λs, µs+1) ≤ m. i On the other hand, p(} ) + ri = n − µk−i+1 + µk−i+1 = n = 2m ≥ 2τi. Therefore, D(σλ;µ) satisfies all the conditions in Definition 3.24. Finally, since λj 6= µi + 1 for any i, j, the two integers preceding a bracket must be equal. Furthermore, if λ1 = 1, µ1 ≥ 1. Hence, condition (A1) holds. For i 1 ≤ i ≤ k − s, k − i + 1 − (p(} ) − ri)/2 = k − i + 1 + µk−i+1 − m. From the sequence 0, 1, . . . , m − 1, remove the integers λ1 − 1, λ2 − 1, . . . , λs − 1 to obtain a sequence αm < αm−1 < ··· < αs+1. By assumption µk−i+1 = αj for some j ≥ k − i + 1. Hence, k − i + 1 + µk−i+1 − m ≤ αj − (m − j) = xi. To see the last equality, observe that xi is the number of integers λh that are less than or equal to µk−i+1 = αj. This number is equal to the number of integers (αj − (m − j)) between 0 and αj that do not occur in the sequence αm, . . . , αj. Hence, condition (A2) holds. We conclude that D(σλ;µ) is an admissible symplectic diagram. The game is defined on admissible symplectic diagrams. We will see in the next section that saturated admissible diagrams in perfect order represent Schubert varieties in SG(k, n). The goal of the algorithm is to transform every admissible symplectic diagram to a collection of saturated admissible diagrams in perfect order. Given an admissible symplectic diagram D, we will associate to it one or two sequences Da and/or Db of brackets and braces. Initially, neither Da nor Db has to be admissible. We will shortly describe an algorithm that modifies Da and Db so that they become admissible. The game records a degeneration of the flag elements represented by D. Definition 3.33. Let D be an admissible symplectic diagram of type s for SG(k, n). For the purposes of this definition, read any mention of k−s+1 as 0 and any mention of }k−s+1 as ]s. (1) If D is not in order, let η be the integer in condition (S3) violating the order. (i) If every integer η < i ≤ k − s occurs to the left of η, let ν be the leftmost integer equal to η + 1 in the sequence of D. Let Da be the canonical representative of the diagram obtained by interchanging η and ν. 16 (ii) If an integer η < i ≤ k − s does not occur to the left of η, let ν be the leftmost integer equal to i + 1. Let Da be the canonical representative of the diagram obtained by swapping η with the leftmost 0 to the right of }i+1 not equal to ν and changing ν to i. (2) If D is in order but is not a saturated admissible diagram in perfect order, let κ be the largest index for which l(i) < ρ(i, i − 1). (i) If l(κ) < ρ(κ, κ − 1) − 1, let ν be the leftmost digit equal to κ + 1. Let Da be the canonical representative of the diagram obtained by changing ν and the leftmost 0 to the right of }k+1 not equal to ν to κ. (ii) If l(κ) = ρ(κ, κ − 1) − 1, let η be the integer equal to κ − 1 immediately to the right of }κ. (a) If κ occurs to the left of η, let ν be the leftmost integer equal to κ in the sequence of D. Let Da be the canonical representative of the diagram obtained by changing ν to κ − 1 and η to zero. (b) If κ does not occur to the left of η, let ν be the leftmost integer equal to κ + 1. Let Da be the canonical representative of the diagram obtained by swapping η with the leftmost 0 to the right of }κ+1 not equal to ν and changing ν to κ. Let p be the position in D immediately to the right of ν. If there exists a bracket at a position p0 > p in Da, let q > p be the minimal position occupied by a bracket in Da. Let Db be the diagram obtained from Da by moving the bracket at position q to position p. Otherwise, Db is not defined. Example 3.34. Let D = 2300}10}0}0, then η = 1 violates the order and ν = 2 and 3 occur to the left of it. Hence, we are in case (1)(i) and Da = 1300}20}0}0 is obtained by swapping 1 and 2. Similarly, let D = 200]200}00}, then the second 2 violates the order and Da = 220]000}00}, Db = 22]0000}00}. Let D = 124400}00}1}0}00, the 1 in the ninth place violates the order and 3 does not occur to its left, so we are in case (1)(ii) and Da = 123400}10}0}0}00. Let D = 22]00}00}00, then D is in order and κ = 1. Since l(1) = 0 < ρ(1, 0) − 1, we are in case (2)(i) and Da = 12]00}10}00 and Db = 1]200}10}00. Let D = 3300}200}0}, then D is in order and κ = 3. Since l(3) = 2 = ρ(3, 2)−1, we are in case (2)(ii)(a) and Da = 2300}000}0}. Finally, let D = 330000}00}1}0, then D is in order and κ = 2. Since l(2) = 0 = ρ(2, 1) − 1 and 2 does not occur in the sequence, we are in case (2)(ii)(b) and Da = 230000}10}0}0. We will soon check that both Da and Db are symplectic diagrams; however, they do not have to be admissible. We now describe algorithms for turning them into admissible diagrams. Algorithm 3.35. If Da is not an admissible symplectic diagram, perform the following steps to turn it into an admissible diagram. Step 1. If Da does not satisfy condition (A2), let i be the maximal index for which condition (A2) fails. Define a new diagram Dc as follows. Let the two rightmost a i integers equal to i in D be in the places π1 < π2. Delete } and move the i in place π2 to place π1 + 1. Slide the integers in places π1 < π < π2 and brackets and braces in positions π1 < p < π2 one to the right. Add a bracket at position π1 + 1. Subtract one from the integers i < h ≤ k − s; and if i = k − s, change the integers 17 equal to k − s to 0. Let Dc be the resulting diagram and replace Da with Dc. If Da satisfies condition (A2), proceed to the next step. Step 2. If Da fails condition (A1), let ]j be the smallest index bracket for which it fails and let i be the integer preceding ]j. Change this i to i − 1 (k − s if i = 0) and move }i−1 (}k−s if i = 0) one position to the left. Repeat this procedure until the sequence of brackets and braces satisfies condition (A1). Let the resulting sequence be Dc. In both steps, we refer to Dc as a quadric diagram derived from Da. Algorithm 3.36. If Db does not satisfy condition (A1), run Step 2 of Algorithm 3.35 on Db. Explicitly, let ]j be the minimal index bracket for which (A1) fails. Let i be the integer immediately to the left of ]j. Replace i with i − 1 and move }i−1 one position to the left. As long as the resulting sequence does not satisfy condition (A1), repeat this process either until the resulting sequence is an admissible symplectic diagram (in which case, this is the symplectic diagram derived from Db) or two braces occupy the same position. In the latter case, no admissible symplectic diagrams are derived from Db. Example 3.37 (Examples of Algorithm 3.35). Let D = 22]33]00}00}00}00. Then a the diagram D = 12]33]00}00}10}00 fails condition (A2) since x1 = 0 < 1 = 5 − (10 − 2)/2. Hence, according to Step 1 of Algorithm 3.35, we replace Da with 11]1]22]00}00}000 (delete }1, move the 1 in position 9 to position 2 and slide everything in positions 2-8 one position to the right, add a bracket in position 2, and subtract 1 from the integers greater than 1). The latter is an admissible diagram. a Let D = 00}00}00. Then D = 22}00}00 fails condition (A2) since x2 = 0 < 1 − (2 − 2)/2. Hence, Step 1 of Algorithm 3.35 replaces Da with 00]00}00 (delete }2 and add a bracket in position 2), which is admissible. Similarly, if D = 11]33]00}00}00}00, then the diagram Da = 11]23]00}20}00}00 fails condition (A2) since x2 = 1 < 2. Hence, according to Step 1 of Algorithm 3.35, we replace Da with 11]22]2]00}000}00, which is admissible. If D = 22]2]200}0000}00, then the diagram Da = 12]2]200}1000}00 is not admissible since it fails condition (A1) for ]1. Step 2 of Algorithm 3.35 replaces Da first with 11]2]200}100}000 (change the 2 preceding ]1 to 1 and move }1 one position to the right). Note that this diagram fails condition (A1) for ]2. Hence, Step 2 replaces it with 11]1]200}10}0000 (change the 2 preceding ]2 to 1 and move }1 one position to the left). This diagram is admissible, hence it is the diagram derived from Da. Example 3.38 (Examples of Algorithm 3.36). Let D = 11]33]00}00}00}00, then Db = 11]2]300}20}00}00 fails condition (A1). Algorithm 3.36 replaces it with 11]1]300}20}0}000, which is admissible. Let D = 00]0000}00}00}, then Db = 3]30000}00}00} does not satisfy condition (A1) since the digit to the left of ]1 has to be 1. Algorithm 3.36 replaces Db first with 2]30000}0}000}, which still fails condition (A1). Hence, Algorithm 3.36 replaces this diagram with 1]30000}0}00}0, which is admissible. If D = 00]0000}2}0}, then Da = 30]2000}0}0} and Db = 3]20000}0}0}. They both fail condition (A1). When we run Algorithm 3.36 on Db, we turn the 3 into 2 and slide }2 one position to the left. In that case, we obtain 1]30000}}00}. Since two braces occupy the same position, no diagrams are derived from Db in this case. When we run Algorithm 3.35 on Da, we obtain the admissible diagram 33]200}00}0}. 18 Let D be an admissible symplectic diagram and let ν be as in Definition 3.33. Let π(ν) denote the place of ν in the sequence of integers. If p(]s) > π(ν), then ]xν−1+1 is the first bracket to the right of ν. If the integer to the immediate left of xν−1+1 ] is positive, let yxν−1+1 be this integer. Otherwise, let yxν−1+1 = k − s + 1. xν−1+1 The condition p(] ) − π(ν) − 1 = yxν−1+1 − ν plays an important role. In words, this condition says that the number of values larger than ν or equal to zero that the integers to the left of ]xν−1+1 attain is one more than the cardinality of the set of integers consisting of zero and integers larger than ν occurring to the left of ]xν−1+1. In view of conditions (S3), (S4) and (A1), a sequence satisfying this equality looks like ··· ν ν + 1 ··· ν + l − 1 ν + l ν + l] ··· or ··· ν ν + 1 ··· ν + l 00] ··· , where we have drawn the part of the sequence starting with the left most ν and ending with ]xν−1+1. We are now ready to state the algorithm. Algorithm 3.39. Let D be an admissible, symplectic diagram of type s for SG(k, n). If D is saturated and in perfect order, return D and stop. Otherwise, let Da and Db be defined as in Definition 3.33.

s xν−1+1 (1) If p(] ) ≤ π(ν) or p(] ) − π(ν) − 1 > yxν−1+1 − ν in D, then return the admissible symplectic diagrams that are derived from Da. (2) Otherwise, return the admissible symplectic diagrams that are derived from both Da and Db. We run the algorithm on two symplectic diagrams.

19 Example 3.40. 00}00}00 → 00]00}00 → 00]0]000 ↓ 1]100}00 In this example, ﬁrst Da = 22}00}00 is not admissible since the diagram fails condition (A2). Therefore, we replace it by 00]00}00. Next, Da = 10]10}00 and Db = 1]100}00. Da is not admissible since it does not satisfy condition (A2). Hence, we replace it by the admissible diagram 00]0]000. Db is admissible. Note that the last two diagrams are saturated and in perfect order, so the algorithm terminates. ∗ We will soon see that this calculation shows i σ2,4 = σ2,3; + σ1;2 in SG(2, 6). Finally, we give a larger example in SG(3, 10) that illustrates the inductive structure of the game. Example 3.41. 300}20}10}000 → 200}00}10}000 → 200]00}10}000 → 1]0000}00}000 ↓ ↓ 100]00}00}000 1]2200}00}000 . & ↓ 100]0]000}000 11]200}0}0000 1]1200}10}000 . & ↓ ↓ 000]0]0]00000 11]00]100}000 11]11]00}0000 1]1100}00}000 . & ↓ 11]1]0000}000 11]11]00}0000 1]1100]00}000

∗ We will see that this calculation shows i σ3,5,7 = σ3,4,5; + σ2,3;3 + 2σ2,4;4 + σ1,5;3 in H∗(SG(3, 10), Z). Deﬁnition 3.42. A degeneration path is a sequence of admissible symplectic diagrams

D1 → D2 → · · · → Dr such that Di+1 is one of the outcomes of running Algorithm 3.39 on Di for 1 ≤ i < r. The main theorem of this paper is the following. Theorem 3.43. Let D be an admissible symplectic diagram for SG(k, n). Let V (D) be the symplectic restriction variety associated to D. Then, in terms of the Schubert basis of SG(k, n), the cohomology class [V (D)] can be expressed as X [V (D)] = cλ;µσλ;µ, where cλ;µ is the number of degeneration paths starting with D and ending with the symplectic diagram D(σλ;µ). Theorem ?? stated in the introduction is a corollary of Theorem 3.43. 20 Definition 3.44. Let σa• be a Schubert class in G(k, n). If aj < 2j − 1 for some ∗ 1 ≤ j ≤ k, then i σa• = 0 and we do not associate a symplectic diagram to σa• . Suppose that aj ≥ 2j − 1 for 1 ≤ j ≤ k. Let u be the number of i such that ai = 2i − 1. For j such that aj 6= 2j − 1, let uj be the number of integers i < j such that ai = 2i − 1. Let vj be the number of integers i > j such that ai = 2i − 1. ∗ Then the diagram D(a•) associated to i σa• is a diagram consisting of u brackets at positions 1, 2 ··· , u and a brace for each aj > 2j − 1 at position aj − uj + vj. The sequence of integers consists of u integers equal to 1 followed by zeros except for one integer equal to k − j − vj + 1 immediately following the first bracket or k−j−vj +1 brace to the right of } (or in the first position if j + vj = 1) for each odd aj > 2j − 1.

Example 3.45. The diagram D(σ3,5,7) in SG(3, 8) is 300}20}10}0. The diagram D(σ1,3,6,7,10) in SG(5, 10) is 1]1]1]00}00}000. The diagram D(σ1,3,7,8,9,12) in SG(6, 14) is 1]1]1]300}0}00}00000

Remark 3.46. The reader will notice that D(σa• ) is the diagram obtained by running Algorithm 3.35 on the diagram that has a brace at positions aj and whose sequence consists of zeros except for one k−j +1 immediately to the right of }k−j+2 when aj is odd.

Lemma 3.47. If aj ≥ 2j −1 for 1 ≤ j ≤ k, then D(a•) is an admissible symplectic diagram.

Proof. The brackets occur at positions 1, . . . , u. Let aj and aj+l be two consecutive integers in the sequence a• satisfying ai > 2i − 1. Then the positions of the corresponding braces are aj −uj +vj and aj+l −uj+l +vj+l. Since uj+l = uj +l −1 and vj+l = vj − l + 1, the positions of the two braces differ by the quantity β = aj+l − aj − 2l + 2. If l = 1, β > 0. If l > 1, then aj < aj+1 = 2j + 1. Since aj+l ≥ 2j + 2l, β is also positive. The first brace corresponds to the smallest index j0 such that aj0 > 2j0 − 1 and occurs at position aj0 − (j0 − 1) + (u − j0 + 1) = u + aj0 − 2j0 + 2 ≥ u + 2. The number of positive integers less than or equal k−j−vj +1 to k − j − vj + 1 to the left of } is u (respectively, u + 1) if aj is even (respectively, odd). Hence, the number aj − uj + vj − u(−1) = aj − 2uj(−1) (where −1 occurs if aj is odd) of integers equal to zero or greater k − j − vj + 1 to the left of }k−j−vj +1 is even. Therefore, conditions (1)-(4) of Definition 3.12 hold. By construction, l(i) ≤ 1 for i > 1 and l(1) = u(+1) depending on whether the largest aj > 2j − 1 is even (or odd). In either case, one easily sees that l(1) ≤ ρ(1, 0). The number of positive integers to the left of }k−j−vj +1 is equal to u plus the number oj of odd al < aj such that al > 2l −1. Since 2(uj +oj) ≤ 2j ≤ aj, we have that 2(u + oj) ≤ aj − uj + vj + u = aj + 2vj and condition (S2) holds. The sequence is in order and the only integers other than k − u occurring in the initial part of the sequence are ones, which are followed by brackets. We conclude that all the conditions in Definition 3.24 hold. Since any bracket is preceded by 1, condition (A1) holds. Finally, for }k−j−vj +1, aj −uj +vj −u(−1) aj (−1) the quantity j + vj − 2 = j + u − 2 ≤ u (where −1 occurs if aj is odd) since aj > 2j − 1. We conclude that D(a•) is an admissible symplectic diagram.

Corollary 3.48. Let σa• be a Schubert class in G(k, n). If aj < 2j − 1 for some ∗ 1 ≤ j ≤ k, then set i σa• = 0. Otherwise, let D(σa• ) be the diagram associated to 21 σa• . Express ∗ X i σa• = cλ;µσλ;µ in terms of the Schubert basis of SG(k, n). Then cλ;µ is the number of degeneration paths starting with D(σa• ) and ending with the symplectic diagram D(σλ;µ). Proof. In Lemma 7.20, we will prove that the intersection of SG(k, n) with a general Schubert variety in G(k, n) with class σa• is a restriction variety of the form

V (D(σa• )). The corollary is immediate from this lemma and Theorem 3.43. We conclude this section by proving that Algorithm 3.39 is well-defined and terminates. The proof of Theorem 3.43 is geometric and will be taken up in the next two sections. Proposition 3.49. Algorithm 3.39 replaces an admissible symplectic diagram with one or two admissible symplectic diagrams. Proof. If D is a saturated symplectic diagram in perfect order, then the algorithm returns D and there is nothing further to check. We will first check that Da and Db are (not necessarily admissible) symplectic diagrams. The diagram Db is obtained from Da by moving a bracket to the left. Conditions (2), (3), (4) of Definition 3.12 and conditions (S1), (S2), (S3) and (S4) of Definition 3.24 are preserved under moving a bracket to the left. Since ν 6= 1 is the leftmost integer in D equal to a given integer, by condition (A1) for D, there cannot be a bracket at position p in D or Da. Hence, condition (1) is satisfied for Db. We conclude that if Da is a symplectic diagram, then Db is also a symplectic diagram. We will now check that Da is a symplectic diagram in each case. In case (1)(i), by condition (S3) for D, let η be the unique integer that violates the order. Since η is violating the order, η is to the left of }η+1. Da is obtained by swapping η and ν, the leftmost integer equal to η + 1. This operation does not change the positions of the brackets and braces and keeps l(i) fixed for every i. After the swap, every integer i is still to the left of }i for every i since η was to the η+1 left of } . Furthermore, the operation also preserves or decreases τi for every i. We thus conclude that conditions (1) through (4) of Definition 3.12 and condition (S1), (S2) and (S4) of Definition 3.24 hold for the diagram Da. After the swap, η is part of the non-decreasing initial sequence in Da. Hence, the diagram Da is either in order or η + 1 is the only integer violating the order. Condition (S3) holds for Da. We conclude that Da is a symplectic diagram. In case (1)(ii), let η be the unique integer that violates the order. Assume that η < i ≤ k − s does not occur to the left of η. Then i does not occur anywhere in the sequence and, in condition (S4) for D, i = j0. We claim that the i-th and (i − 1)-st braces in D must look like ···}iη}i−1 ··· . By conditions (S3) and (S4) for D, η is to the right of }i and to the left of }η+1. If η is between }i+h and }i+h−1 for h 6= −1, then since ρ(i + h, i + h − 1) = l(i + h) by condition (S4), the parity in condition (4) is violated for }i+h−1. We conclude that η is between }i and }i−1. Furthermore, 1 ≤ ρ(i, i − 1) ≤ l(i) + 2 − ξi = 1 by condition (S4). The formation of Da does not affect conditions (1) through (3) in Definition 3.12. Condition (4) holds for Da since the formation of Da changes the number of integers that are equal to zero or greater than j to the right of }j only when j = i and for }i it changes the number by two. Since the formation of Da only increases l(i) by one a and decreases or preserves l(j) for j 6= i, D satisfies (S1). Similarly, τi increases 22 by one and all other τj remain fixed or decrease. On the other hand, ri increases by two, hence Da satisfies condition (S2). There is one exception. If i = k − s and s every integer to the left of ] is positive, τk−s increases by two. Then, τk−s = rk−s, k−s a a hence 2τk−s ≤ p(} ) + rk−s and D satisfies (S2). The diagram D is either in order or η is still the only integer violating the order, hence Da satisfies (S3). Finally, the formation of Da changes l(i) = 1 and decreases l(i + 1) by one. Hence, l(i) = ρ(i, i − 1) for Da. By condition (S4) for D, we have that ρ(j, j − 1) = l(j) in Da for any j for which the equality held for D except for j = i + 1. Furthermore, a a ξi+1 = 1 in D , so ρ(i + 1, i) = l(i + 1) + 1 = l(i + 1) + 2 − ξi+1 in D . Hence (S4) holds for Da. We conclude that Da is a symplectic diagram. From now on assume that D is in order. Then there cannot be i ≥ κ such that i is immediately to the right of }i+1. Suppose there exists such an i. The number χ(i) and χ(i + 1) of zeros and integers greater than i, respectively i + 1, to the left of }i, respectively }i+1, has to be even. However, χ(i) = χ(i+1)+l(i+1)+ρ(i+1, i)−1. Since by assumption ρ(i + 1, i) = l(i + 1), we conclude that either χ(i) or χ(i + 1) cannot be even leading to a contradiction. In case (2)(i), changing ν to κ and the first zero to the right of }κ+1 does not change the positions of brackets and braces, it decreases l(κ+1) by one and increases l(κ) by two. Furthermore, the sequence Da is still in order, unless κ = k − s and there are zeros to the left of ]s. In the latter case, the κ to the right of ]s is the unique integer violating order. Since by assumption l(κ) < ρ(κ, κ − 1) − 1 in D, l(κ) ≤ ρ(κ, κ − 1) in Da. The parity of the integers equal to zero or greater than i also remains constant for all 1 ≤ i ≤ k − s. We conclude that conditions (1) through (4) in Definition 3.12 and conditions (S1) and (S3) in Definition 3.24 hold a for D . The quantity τi remains constant for i > κ and increases by one for i ≤ κ s unless κ = k − s, l(k − s) ≥ p(] ) and τk−s increases by two. In the latter case, τk−s k−s is less than or equal to both rk−s and p(} ) and (S2) holds. In the former case, rκ increases by two, hence (S2) holds for the index κ. Since ρ(κ, κ − 1) < l(κ) − 1, (S2) also holds for indices i < κ. If there exists an index i < κ in D such that i is not a 1 followed by a bracket, then in condition (S4) for D, we have that a j0 = κ. Furthermore, ρ(κ, κ − 1) = l(κ) + 2. Hence, the formation of D preserves the equalities in condition (S4) except for j = κ or κ + 1. In Da, we have that ρ(κ, κ − 1) = l(κ) and ρ(κ + 1, κ) = l(κ + 1) + 1 = l(κ + 1) + 2 − ξκ+1. We conclude that condition (S4) holds for Da. Therefore, Da is a symplectic diagram. Finally, the argument showing that Da is a symplectic diagram in case (2)(ii)(a) is identical to the argument in case (1)(i) and the argument in case (2)(ii)(b) is identical to the case (1)(ii), so we leave them for the reader. We conclude that both Da and Db are symplectic diagrams. However, they need not be admissible. We now check that Algorithms 3.35 and 3.36 preserve the fact that the resulting sequences are symplectic diagrams and output admissible symplectic diagrams. Da may fail to be admissible either because it fails condition (A1) or (A2) in a Definition 3.27. The formation of D from D does not change the quantities xh, h p(} ). In cases (1)(i) and (2)(ii)(a) the quantity rh either remains the same or a decreases. Hence, in these cases D satisfies condition (A2). In case (1)(ii), rh remains the same or decreases except for ri, which increases by two. Hence, the inequality in condition (A2) can only be violated for the index i by one. If it is i p(} )−ri violated, we conclude that in D, we have xi = k−i+1− 2 . Recall that in this 23 i i−1 case D looks like ···} η} ··· . Since i does not appear in D, xi = xi−1. Writing the inequality in (A2) for D and the index i − 1 and noting that ri−1 = ri + 1 i i−1 i p(} )−ri and p(} ) = p(} ) + 1, we see that xi = xi−1 ≥ k − i + 2 − 2 = xi + 1. Since D satisfies (A2), this is a contradiction. We conclude that Da satisfies (A2) also in the case (1)(ii). By similar reasoning, in cases (2)(i) and (2)(ii)(b), Da can violate the inequality in (A2) only for the index κ by one. After Step 1 of Algorithm 3.35, all the inequalities in condition (A2) remain unchanged or improve and }κ is eliminated. We conclude that after Step 1, the resulting diagram satisfies (A2). When the inequality in (A2) is violated for Da, it is violated for the index κ by at b most 1. When we form D in cases (2)(i) and (2)(ii)(b), xκ also increases by one. Hence, Db, when it exists, always satisfies (A2). Observe that the operation in Step 1 of Algorithm 3.35 preserves the fact that Da is a symplectic diagram. By construction, conditions (1)-(4) and (S1) and (S2) hold. The diagram resulting after Step 1 is in order, hence (S3) holds. The operation renames l(i) as l(i − 1) for i > κ + 1 and ρ(i + 1, i) as ρ(i, i − 1) for i > κ + 1. The operation does not change the quantities l(i) and ρ(i, i−1) when i < κ and replaces l(κ) and l(κ + 1) with their sum under the name l(κ). The quantities ρ(κ, κ − 1) and ρ(κ + 1, κ) are replaced by ρ(κ, κ − 1) + ρ(κ + 1, κ) − 1 and renamed ρ(κ, κ − 1). Hence, the equalities in condition (S4) are preserved. Since (A1) also holds for the resulting diagram Dc, we conclude that if Da fails condition (A2), then Step 1 of Algorithm 3.35 produces an admissible symplectic diagram. Observe that changing a digit to the left of a bracket and moving a brace one unit i to the left, increases xi and ri by one and decreases p(} ) by one. Hence, it preserves the inequality in condition (A2). It also preserves the conditions (1) through (4) and (S1) through (S4), with the possible exception of (1) in case p(}i+1) = p(}i)−1. Condition (A1) is violated for Da when there is a bracket in position p(ν) + 1 and it is violated only for that bracket. After l applications of Step 2 of Algorithm 3.35, Condition (A1) is still violated if there exists brackets at positions p(ν) + 1, p(ν) + 2, ··· , p(ν) + l. Since there are a finite number of brackets, this process stops and the resulting diagram satisfies condition (A1). In this case, the only brace that moves is }ν−1. Since l(ν) ≤ ρ(ν, ν − 1) in D, the intermediate sequences and the resulting sequence all satisfy condition (1). If Db does not satisfy condition (A1), then the only bracket that can violate it is the one in position p(ν). In this case, Algorithm 3.36 successively decreases the integer to the right of the bracket in p(ν) by one until it either becomes equal to the integer to its right or to one in case there isn’t an integer to its right. Hence, this algorithm terminates in finitely many steps. A diagram might violate condition (1) in the process, but in that case the diagram is discarded. Hence, after finitely many steps either the diagram is discarded or results in an admissible symplectic diagram. We conclude that Algorithm 3.39, replaces D with one or two admissible symplectic diagrams. Proposition 3.50. After finitely many applications of Algorithm 3.39, every admissible symplectic diagram is transformed to a collection of admissible symplectic diagrams in perfect order. Proof. If the diagram D is not in order, after one application of the algorithm either the diagram is in order or the integer violating the order increases or the position of the integer violating the order in the sequence decreases. Since these steps cannot go on indefinitely, after finitely many steps, the diagram is in order. Furthermore, 24 during the process either the number of braces decreases or the number of positive integers less than or equal to i, for 1 ≤ i ≤ k − s in the initial part of the sequence remains constant or increases. If the diagram is in order, then at each application of the algorithm either the number of braces decreases or the number of positive integers less than or equal to i, for 1 ≤ i ≤ k − s, in the initial part of the sequence increases. Since these cannot go on indefinitely, we conclude that repeated applications of the algorithm transform an admissible symplectic diagram into a collection of admissible symplectic diagrams in perfect order. Hence, the algorithm terminates in finitely many steps. 4. Restriction varieties in the orthogonal Grassmannians In this section, we introduce restriction varieties in orthogonal Grassmannians and discuss their basic properties. Restriction varieties are subvarieties of OG(k, n) that parameterize isotropic k-planes that intersect elements of a given flag in spec- ified dimensions. We do not require the flag to be isotropic; however, we need to impose some basic numerical restrictions in order to obtain geometrically meaningful subvarieties.

Notation 4.1. Let W be a vector space of dimension n. Let Q be a non-degenerate, symmetric bilinear form on W . We denote an isotropic linear space of (vector space) dimension n by L . In case 2n = n, L and L0 denote isotropic linear spaces j nj j nj nj in diﬀerent connected components. Let Qri denote a sub-quadric of corank r cut di i out by a d -dimensional linear section of Q. We denote the singular locus of Qri i di by Qri,sing. For convenience, we let r = 0 and d = n. di 0 0

Deﬁnition 4.2. A sequence of linear spaces and quadrics (L•,Q•) associated to OG(k, n) is a totally ordered set r L L ··· L Q k−s ··· Qr1 n1 ( n2 ( ( ns ( dk−s ( ( d1 of isotropic linear spaces L (or possibly L0 in case 2n = n) and sub-quadrics nj ns s Qri of Q such that di (1)2 ns ≤ dk−s + rk−s.

(2) 2(k − i + 1) ≤ ri + di for every 1 ≤ i ≤ k − s.

(3) ri+1 + di+1 ≤ ri + di ≤ n for every 1 ≤ i < k − s. (4) Qri−1,sing ⊆ Qri,sing for every 1 < i ≤ k − s. di−1 di (5) dim(L ∩ Qri,sing) = min(n , r ). nj di j i (6) Let x1 denote the number of isotropic subspaces in the sequence contained in the singular locus of Qr1 . For every 1 ≤ i ≤ k − s, either r = r = x , d1 i 1 1 or rl − ri ≥ l − i − 1 for every l > i. Furthermore, if rl = rl−1 > x1 for some l, then di − di+1 = ri+1 − ri for all i ≥ l and dl−1 − dl = 1. Remark 4.3. Conditions (1), (2) and (3) express basic facts about quadrics. Con- ditions (1) and (2) express the “Linear space bound” that the dimension of an isotropic linear space with respect to a quadratic form of corank r in d variables is at most half of d+r. Since L ⊂ Qrk−s , Condition (1) needs to be satisfied. Below, ns dk−s in defining restriction varieties, we will require the isotropic k-planes to intersect Qri in a subspace of dimension k −i+1. Hence, Condition (2) needs to be satisfied. di 25 Condition (3) expresses the “Corank bound” that a hyperplane section of a quadric of corank r can have corank at most r + 1. Conditions (4) and (5) express that the singular loci of the quadrics Qri are in the most special position. The singular di locus of the quadric Qri contains the singular locus of all the larger dimensional di quadrics in the sequence. Furthermore, isotropic linear spaces in the sequence of dimension greater (resp., less) than ri contain (resp., are contained in) the singular locus of Qri . Finally, Condition (6) is a technical condition: If a quadric Qri di di is more singular than the linear spaces in the sequence force it to be, then each quadric contained in Qri is more singular than the one larger quadric containing di it except in a very special case detailed in Condition (6). These conditions will automatically hold for all the varieties in our algorithm, hence the reader does not need to remember these conditions to implement the algorithm.

We will use sequences of brackets and braces introduced in the previous section for representing the geometric sequences.

Deﬁnition 4.4. Let (L•,Q•) be a sequence for OG(k, n). The sequence of brackets and braces associated to (L•,Q•) is a sequence of non-negative integers of length n, s right brackets and k − s right braces such that

(1) The sequence consists of ri − ri−1 integers equal to i for 1 ≤ i ≤ k − s placed in increasing order followed by a sequence of n − rk−s zeros.

(2) The right square brackets are placed after the nj-th integer in the sequence for 1 ≤ j ≤ s and the right braces are placed after the di-th integer in the sequence for 1 ≤ i ≤ k − s. In case 2n = n, we distinguish between L and L0 by writing ] and ]0, respec- s ns ns tively, for the right bracket after the ns-th digit.

Example 4.5. The sequence of brackets and braces 1]22]000}00}0 represents the 3 1 sequence L1 ⊂ L3 ⊂ Q6 ⊂ Q8. To determine the (vector space) dimension di of the span of the quadric Qri , we count the number of digits to the left of the i-th di brace. For example, there are 8 digits to the left of the right most brace, so d1 = 8. There are six digits to the right of the second brace, so d2 = 6. To determine ri, we count the number of positive digits less than or equal to i. In this example, there are 3 positive digits less than or equal to 2, so r2 = 3. There is a unique one, so r1 = 1. Finally, to determine nj, we count the number of digits to the left of the j-th square bracket. In this example, n1 = 1, n2 = 3. The reader will notice that the Zariski closure of the subvariety of OG(4, 9) parameterizing isotropic subspaces Λ that satisfy 3 1 dim(Λ ∩ L1) = 1, dim(Λ ∩ L3) = 2, dim(Λ ∩ Q6) = 3, dim(Λ ∩ Q8) = 4 3,1 is the Schubert variety Ω4,2. Note that the sequence µ in our notation for Schubert varieties denotes the codimensions (equivalently, coranks) of the quadrics deﬁning the variety, so it is very easy to read from the diagram.

The sequence of brackets and braces associated to (L•,Q•) is a sequence of brackets and braces in the sense of the previous section. Since n1 < ··· < ns < dk−s < ··· < d1, the brackets and braces occupy diﬀerent positions. Since the quadrics contain the linear spaces, the brackets are to the left of all the braces. The positive integers are increasing and less than or equal to the number of braces 26 and they are all to the left of the zeros by construction. The position of a bracket j i p(] ) is equal the dimension nj of the linear space Lnj . The position of a brace p(} ) is equal to the dimension of the span d of the quadric Qri . The dimension r of the i di i singular locus of Qri is the number of positive integers l(≤ i) less than or equal to di i. Finally, l(i) is ri − ri−1 and ρ(i, i − 1) = di−1 − di. Hence, these sequences satisfy conditions (D1) (which is equivalent to Condition (3)), (D2) (which is equivalent to condition (2)) and (D3) (which is equivalent to Condition (6)).

Deﬁnition 4.6. Given a sequence (L•,Q•), let xi denote the number of isotropic linear spaces L of the sequence contained in Qri,sing. Similarly, let y be the nj di j integer such that ryj −1 < nj ≤ ryj . If ri < nj for every 1 ≤ i ≤ k − s, set yj = k − s + 1.

Remark 4.7. We will require the (k − i + 1)-dimensional subspace contained in Qri to intersect Qri,sing in a subspace of dimension x . The index y is the smallest di di i j index i such that L is contained in the singular locus of Qri . By conditions (4) nj di and (5), every quadric of index at least yj will be everywhere singular along Lnj .

We need some further assumptions on the sequence (L•,Q•) before it reﬂects the properties of the corresponding variety.

1 1 Example 4.8. Consider the sequence L3 ⊂ Q5 ⊂ Q6 depicted by 100]00}0}0. By ‘the linear space bound’, any isotropic 3-plane in OG(3, 7) which is contained 1 1 in Q6 necessarily must contain the singular point of Q6. (Geometrically, any plane in a five dimensional quadric cone contains the vertex.) Hence the sequence L1 ⊂ 1 1 Q5 ⊂ Q6 1]0000}0}0 1 better reflects the geometric properties of isotropic 3-planes contained in Q6. Sim- 2 0 ilarly, consider the sequence Q4 ⊂ Q6 depicted by 2200}00}0. 2 The quadric Q4 is reducible. (Geometrically, a quadric surface which is singular 0 along a line is the union of two planes.) Hence, two sequences of the form L3 ⊂ Q6 000]000}0 better reflect the geometry of the corresponding variety. These examples motivate the following definition.

Deﬁnition 4.9. A sequence (L•,Q•) associated to OG(k, n) is admissible if the linear spaces and quadrics satisfy the following additional conditions:

(7) rk−s ≤ dk−s − 3. (8) For every 1 ≤ i ≤ k − s, d − r x ≥ k − i + 1 − i i . i 2

(9) For any 1 ≤ j ≤ s, there does not exist 1 ≤ i ≤ k − s such that nj − ri = 1.

27 Remark 4.10. If Condition (7) is violated, then Qrk−s would either be reducible dk−s or non-reduced. Condition (8) expresses the fact that in a quadric Qri , a linear di space of dimension k − i + 1 has to intersect the singular locus in dimension at di−ri least k − i + 1 − 2 (see Remark 4.7). Condition (9) expresses the fact that if n − r = 1 for some pair, then the tangent spaces to Qri would be constant along j i di L . Hence the (k − i + 1)-dimensional subspace contained in Qri would actually nj di be contained in Qri+1 with singular locus L . The reader should remember these di−1 nj three conditions in order to implement the algorithm.

Lemma 4.11. The sequence of brackets and braces associated to an admissible sequence is a quadric diagram. Conversely, every quadric diagram corresponds to an admissible sequence (L•,Q•).

Proof. We already saw that the sequence associated to (L•,Q•) is a sequence of brackets and braces that satisfies the conditions (D1), (D2) and (D3). Conditions (7), (8) and (9) translate to the conditions (D4), (D5) and (D6). If rk−s ≤ dk−s −3, then there are at least three zeros to the left of }k−s since the total number of k−s positive integers in the sequence (rk−s) is three less than the position of } . i Using the facts that di = p(} ) and ri = l(≤ i), Conditions (8) and (D5) are direct translations of each other. Finally, if the two digits preceding a bracket ]j are a < b, then nj − ra = 1 contradicting Condition (9). If a bracket is at the first position, then n1 = 1. If r1 = 0, then n1 − r1 = 1 contradicting Condition (9). Hence, the digit preceding ]1 must be 1. We conclude that conditions (D6) and (9) are equivalent. Finally, observe that Condition (8) implies Condition (2). We have included Condition (2) to simplify certain statements in the proof of the algorithm. We conclude that the data defining quadric diagrams and admissible sequences are equivalent.

Deﬁnition 4.12. Let (L•,Q•) be an admissible sequence for OG(k, n). A restriction variety V (L•,Q•) is the subvariety of OG(k, n) deﬁned as the Zariski closure of the following quasi-projective variety

V (L ,Q )0 := { [W ] ∈ OG(k, n) | dim(W ∩L ) = j, dim(W ∩Qri ) = k−i+1, dim(W ∩Qri,sing) = x }. • • nj di di i

Example 4.13. Schubert varieties in OG(k, n) are restriction varieties deﬁned with respect to sequences satisfying di + ri = n for all 1 ≤ i ≤ k − s (see Lemma 4.18). The intersection of a general Schubert variety in G(k, n) with OG(k, n) (when non- empty) is a restriction variety associated to a sequence where s = 0 and ri = 0 for 1 ≤ i ≤ k (see Proposition 6.2 for the precise statement). Hence, restriction varieties are a class of varieties that interpolate between the restriction of Schubert varieties in G(k, n) and Schubert varieties in OG(k, n).

Remark 4.14. A restriction variety does not have to be irreducible. For example, 000}0}0 in OG(2, 5) consists of two irreducible components. (Geometrically, the corresponding restriction variety parametrizes lines on a smooth quadric surface in P3.) When 28 the inequality in Condition (8) is an equality for an index i, then the (di + ri)/2- dimensional linear spaces in Qri form two irreducible components. The (k − i + 1)- di dimensional subspaces contained in Qri may be distinguished by their parity of the di dimension of their intersection with linear spaces in each of these components.

Deﬁnition 4.15. Let (L•,Q•) be an admissible sequence. An index 1 ≤ i ≤ k − s such that d − r x = k − i + 1 − i i i 2 is called a special index. For each special index, a marking m• of (L•,Q•) designates one of the irreducible components of di+ri -dimensional linear spaces of Qri as even 2 di and the other one as odd, such that

• If di1 + ri1 = di2 + ri2 , for two special indices i1 < i2, and the component containing a linear space V is designated even for i2, then the component containing V is designated even for i1 as well; and

• If 2ns = di + ri for a special index i, then the component to which Lns belongs is assigned the parity of s; and • If n = 2k, m• assigns the component containing Lk the parity that charac- terizes the component OG(k, 2k).

A marked restriction variety V (L•,Q•, m•) is the Zariski closure of the subvariety 0 of V (L•,Q•) parameterizing k-dimensional isotropic subspaces W , where, for each special index i, W intersects subspaces of dimension di+ri of Qri designated even 2 di (respectively, odd) by m• in a subspace of even (respectively, odd) dimension.

Proposition 4.16. The marked restriction variety V (L•,Q•, m•) associated to a marked admissible sequence is an irreducible variety of dimension s k−s X X dim(V (L•,Q•, m•)) = (nj − j) + (di + xi − 2s − 2i) (1) j=1 i=1 Proof. We prove this proposition by induction on k. Suppose k = 1. If s = 1, then clearly the variety is isomorphic to projective space of dimension n1 − 1 and the proposition holds. If s = 0, then the variety is isomorphic to a quadric hypersurface in Pd1−1 singular along a linear space of codimension at least three (by Condition (7) in Deﬁnition 4.9). Since such a quadric is irreducible of dimension d1 − 2, the base case of the induction follows. Now suppose that the proposition holds up to k − 1. If k − s = 0, then the proposition is immediate. In that case, the isotropic subspaces are contained in the Grassmannian G(k, nk) and the restriction variety is an ordinary Schubert variety

(Σnk−n1−k+1,...,nk−nk−1−1) in G(k, nk). The irreducibility and the dimension follow 0 from these considerations. We may assume that k − s > 0. Let (L•,Q•) be the sequence for OG(k−1, n) obtained from (L ,Q ) by omitting Qr1 from the sequence • • d1 0 (and subtracting one from the indices of the quadrics). Observe that (L•,Q•) is also an admissible sequence: Conditions (1)-(9) remain valid when we omit the largest 0 0 quadric. Let m• be the restriction of the marking m• to this new sequence, where m designates the same components of linear spaces as even if ri+di < r1+d1 and swaps 0 the designation for linear spaces with ri + di = r1 + d1. Let V (L•,Q•, m•) denote 0 the intersection of V (L•,Q•, m•) with V (L•,Q•) , the Zariski open set used to 29 0 0 0 0 define V (L•,Q•). We then have a morphism f : V (L•,Q•, m•) → V (L•,Q•, m•) 0 by taking the intersection of the linear spaces of dimension k in V (L•,Q•, m•) with Qr2 . By induction, we can assume that the the image is an irreducible variety of d2 dimension predicted by the proposition. We now study the fibers of this morphism. Fix a point [W ] in the image. By assumption, the dimension of intersection of W with the singular locus of Qr1 is x . Then any k-dimensional linear space d1 1 0 containing W has to be contained in the quadric Q cut out on Q1 by the linear space everywhere tangent to W . This is a quadric of corank r1 + k − 1 − x1 in a linear space of dimension d1 − (k − 1 − x1). We have to choose a k-plane containing W . We can choose a linear section Q00 of Q0 complementary to W . Choosing a k-plane is equivalent is to choosing a point on Q00. Hence, the dimension of the fiber is d1 − k + 1 + x1 − 2 − k + 1. Furthermore, by Condition (8) d − r x ≥ k − 1 1 . 1 2 If the inequality is strict, it follows that

r1 + k − 1 − x1 < (d1 − k + 1 + x1) − 2, hence Q00 and consequently the ﬁber is irreducible. If equality holds, then Q00 is a union of two linear spaces. The marking m• selects one of these components by specifying the parity of the dimension of intersection with the k-dimensional linear space. Hence, the ﬁber is irreducible. This concludes the proof.

Remark 4.17. Since Equation 1 does not depend on the marking m•, every irreducible component of the restriction variety V (L•,Q•) has dimension s k−s X X (nj − j) + (di + xi − 2s − 2i). j=1 i=1

Observe that V (L•,Q•) has an irreducible component for every marking m•. The markings m• parameterize the irreducible components of V (L•,Q•). Correspond- ingly, given a sequence D of brackets and braces, we deﬁne dim(D) by the expression s k−s X j X i (p(] ) − j) + (p(} ) + xi − 2s − 2i). j=1 i=1

Lemma 4.18. Schubert varieties in OG(k, n) are the restriction varieties where the admissible sequence deﬁning the restriction variety satisﬁes ri + di = n for every 1 ≤ i ≤ k − s. When n = 2k, we also require that the k-dimensional linear spaces to intersect the k-dimensional linear space Lk in the sequence in a subspace of the correct parity.

Proof. Set α = b(n − 1)/2c. Let the sequence λ be defined by setting λj = α + 1 − nj. Let the sequence µ be given by setting µk−i+1 = ri. We claim that the µ restriction variety V (L•,Q•) is the Schubert variety Ωλ. Since the sequence satisfies Conditions (4) and (5), it suffices to show that there does not exist nj and ri such that nj − ri = 1 for any i and j. This is guaranteed by Condition (9) defining admissible sequences. When 2k = n, we require that the length of λ have the same parity as k (alternatively, we could interpret a restriction variety with the wrong 30 parity as a Schubert variety for the other connected component of OG(k, 2k)). Note that under the assumptions of the lemma, the restriction variety is automatically irreducible. Suppose equality holds in Condition (8) for some i0. Then n is even. Condition (9) and the assumption on the sequence imply that equality holds for every i ≥ i0. In particular, equality must hold for the index k − s. Combining Conditions (9) and equality in Condition (8), we have d − r d + r n n ≥ r + 1 + s − s + 1 − k−s k−s = k−s k−s = . s k−s 2 2 2

Using Condition (1), we deduce that ns = n/2. Hence, the marking is uniquely determined by the sequence.

5. The Algorithm for computing the classes of restriction varieties in Grassmannians 5.1. The strategy and examples. The strategy to calculate the class of a restriction variety V (L•,Q•) is to specialize it into a union of Schubert varieties by successively making the quadrics in the sequence more singular. By the corank bound (Condition (3)), if r + d = r + d , then Qri is as singular as it can be i i i−1 i−1 di given that it is contained in Qri−1 , so its corank cannot be increased. If r + d = n di−1 i i for all 1 ≤ i ≤ k − s, then V (L•,Q•) is a Schubert variety and there is nothing further to do. Otherwise, there is a smallest dimensional quadric whose corank can be increased. We increase the corank of this quadric (ﬁxing all the other linear spaces and quadrics) by one by specializing the quadric in a pencil. As a result of this specialization, the restriction variety breaks into a union of restriction varieties each with multiplicity one. In the rest of this section, we will describe the components and show that they occur with multiplicity one. We ﬁrst discuss several fundamental examples that illustrate the possibilities.

0 Example 5.1. We ﬁrst compute the class of V (Q4) depicted by 0000}0 0 in OG(1, 5). Projectively, V (Q4) parametrizes points on a smooth quadric hyper- 4 0 surface Q in P that are contained in a smooth hyperplane section Q4. We specialize the hyperplane section until it becomes tangent to Q. This specialization replaces 0 1 Q4 with Q4 (singular at the point of tangency). In the process, the restriction variety is transformed to 1000}0. This is the quadric diagram Da described in §3. Observe that if the linear spaces had to intersect the singular locus, then they would just be the singular point of 1 1 Q4. The singular point has smaller dimension than the quadric Q4. That’s why in these cases the quadric diagrams derived from Db do not occur. The cohomology class of a smooth hyperplane section is the same as that of a singular hyperplane 0 1 1 section, hence V (Q4) and V (Q4) have the same cohomology class. Since V (Q4) is a Schubert variety with class σ1 in OG(1, 5), this concludes the calculation. During this process, a quadric may become reducible. As a slight variation, we 0 compute the class of V (Q3) depicted by 000}0 31 0 in OG(1, 4). Projectively, V (Q3) parametrizes points contained in a smooth conic in a smooth quadric surface Q in P3. We specialize the plane of the conic until 0 1 1 it becomes tangent to Q, replacing Q3 with Q3. Note that Q3 violates Condition (7) (its corank is two less than its ambient dimension). Geometrically, a singular conic is a union of two lines which belong to two diﬀerent rulings on the quadric surface Q. The sequence of brackets and braces 100}0 fails condition (D4). We replace it by the two quadric diagrams 00]00 and 00]000 according to §3. Hence, the restriction variety corresponding to the diagram 000}0 is replaced by the two restriction varieties corresdponding to 00]00 and 00]000. Geometrically, the class of a conic is the sum of the classes of two lines on the quadric one in each ruling. This concludes the calculation since the latter two 2 varieties are Schubert varieties with classes σ0 and σ , respectively. Hence, the 0 2 class of V (Q3) in OG(1, 4) is σ0 + σ . This example shows that in the algorithm, we have to replace a quadric by two linear spaces if the specialization forces the quadric to become reducible (or, equivalently, violate Condition (7)).

0 Example 5.2. Next we calculate the class of the restriction variety V (L2 ⊂ Q4) in OG(2, 5) depicted by 00]00}0. Geometrically, this example corresponds to calculating the class of the inclusion of OG(2, 4) in OG(2, 5). More concretely, we calculate the class of the space of 0 lines contained in the smooth quadric surface Q4 and intersect the line L2 (in a 0 point). Note that Q4 is a smooth hyperplane section of the ambient quadric Q 4 in P . We specialize it until it becomes tangent to Q at a point on L2. This 0 1 replaces Q4 with Q4, the quadric cone singular at the point of tangency. This is depicted by Da = 10]00}0, which violates condition (D5). By the linear space bound (Condition (8)), lines contained in a quadric cone in P3 all pass through 0 the vertex of the cone. Hence, the degeneration replaces V (L2 ⊂ Q4) with the 1 restriction variety V (L1 ⊂ Q4) 1]000}0. This is the quadric diagram Db deﬁned in §3. This restriction variety is the Schubert 1 b variety with class σ2. Note that in this case, this is the diagram derived from D and Da does not lead to any diagrams since it violates condition (D5). Geometrically, this corresponds to the fact that the lines are required to pass through the singular point.

0 Example 5.3. Finally, consider the variety V (L2 ⊂ Q6) in OG(2, 7). 00]0000}0 → 1]00000}0 ↓ 11]000}00 Geometrically, this variety parametrizes lines on a smooth quadric Q in P6 that 0 intersect a line L2 and are contained in a smooth hyperplane section Q6 of Q. As 0 before, we specialize the linear space deﬁning Q6 to be tangent to Q along a point 0 1 of the line L2, replacing Q6 with Q6. In the limit, there are two possibilities. In 1 the ﬁrst case, the lines may all pass through the singular point of Q6. This case 32 1 b (V (L1 ⊂ Q6)) is depicted by the quadric diagram D = 1]00000}0. In the second, case the lines intersect L2 in a point other than the vertex. This is denoted by the sequence of brackets and braces Da = 10]0000}0. Note that this sequence fails 1 condition (D6). By “the variation of tangent spaces”, the tangent spaces to Q6 are 1 constant along the line L2. Therefore, the lines in Q6 that intersect L2 in a point 2 other than the singular point have to be contained in the quadric Q5 given by the intersection of Q with the linear space everywhere tangent to Q along L2. This 2 possibility (V (L2 ⊂ Q5)) is depicted by 11]000}00, which is the quadric diagram derived from Da as in §3. Both of these varieties are Schubert varieties and occur 0 1 2 with multiplicity one in the limit. Hence, the class of V (L2 ⊂ Q6) is σ3 + σ2. Example 5.3 shows the basic branching. When we increase the corank of the quadric, the linear spaces intersect the new singular locus either in a larger dimensional vector space (unless this possibility leads to a smaller dimensional variety as in Example 5.1) or in the same dimensional vector space (unless this possibility is excluded by the linear space bound (Condition (8)) as in Example 5.2). Additional branching occurs when one of the quadrics become reducible (as in Example 5.1). The general rule is obtained by repeating these three fundamental examples. In fact, these examples capture all the geometric complexity of restriction varieties in orthogonal Grassmannians. Next we give a complicated example that illustrates the inductive structure of the Algorithm.

0 0 0 Example 5.4. Consider the restriction variety V = V (Q4 ⊂ Q6 ⊂ Q8) in OG(3, 9). Concretely, V is the intersection of OG(3, 9) with a general Schubert variety Σ3,2,1(F•) in G(3, 9). We calculate the class of V in terms of Schubert classes in OG(3, 9) as follows.

0000}00}00}0 → 3000}00}00}0 ×→2 000]000}00}0 →200]000}00}0 ×→2 000]0]0000}0 → 100]0]0000}0 ↓ 22]0000}00}0 → 1]20000}00}0 → 1]22000}00}0 ↓ 11]0000}0}00 → 11]2000}0}00 We explain the salient features of this example. In the ﬁrst two steps, we increase 0 the corank of the smallest dimensional quadric Q4 by one. After the second step, 2 we obtain Q4, which is a reducible quadric equal to the union of two linear spaces of dimension three. (in terms of the combinatorics of quadric diagrams 3300}00}00}0 violates condition (D4), so has to be replaced by two copies of 000]000}00}0.) Correspondingly, the restriction varieties breaks into two irreducible components 0 0 both isomorphic to the restriction variety V (L3 ⊂ Q6 ⊂ Q8). The symbol ×2 above the right arrow indicates that there are two components of the limit with the same class (though they are distinct varieties, each occurring with multiplicity one). 0 In the next two steps, we increase the corank of the quadric Q6 by one. After the 2 second step, either the linear spaces intersect the singular locus of Q6 and we get the restriction variety indicated by the down arrow or the linear spaces do not intersect 2 2 the singular locus of Q6. In the latter case, the tangent spaces to Q6 are constant 3 along L3. Hence, these linear spaces must intersect the quadric Q5 everywhere tangent along the three dimensional linear space in a two-dimensional subspace. 33 3 Note that the latter quadric Q5 is reducible, the union of two linear spaces. Hence, in this case there are two components which are indicated after the right arrow. (In terms of the combinatorics of quadric diagrams we have Da = 220]000}00}0 and Db = 22]0000}00}0. Db is a quadric diagram, but Da fails condition (D6), so we replace it with Dc = 222]00}000}0, which fails condition (D4). We have to replace Dc by two copies of 000]0]0000}0). The rest of the calculation is similar to the previous examples. We conclude that the class of the variety is equal to 1 3,1 3,2 4σ2,1 + 2σ4 + 2σ3 .

5.2. The algorithm. We now give the algorithm for computing the class of the variety V (L•,Q•) in terms of Schubert classes in OG(k, n). First, we begin with a slogan that can help guide the reader through the combinatorics. The Rule in Slogan Form: Increase the dimension of the singular locus of the smallest dimensional quadric allowed by the corank bound (Condition (3)) by one. The linear spaces intersect the new singular locus either in a subspace of the same dimension as before or in one larger dimension, unless one of these possibilities leads to a smaller dimensional variety or is precluded by the linear space bound (Condition (8)). This section and §3 make this slogan precise.

Definition 5.5. Let (L•,Q•) be an admissible sequence. We say that the quadric Qri is saturated if r = n − d . V (L ,Q ) is saturated if every quadric Qri , 1 ≤ i ≤ di i i • • di k − s, in its definition is saturated. If the admissible sequence contains a quadric which is not saturated, define the active index κ to be the largest index i for which ri − ri−1 < di−1 − di (where, by convention, we set r0 = 0 and d0 = n).

Remark 5.6. By Lemma 4.18, a saturated restriction variety is a Schubert variety. r If a quadric Qri in the deﬁnition of a restriction variety is not saturated, then Q j di dj is not saturated for any j ≥ i. In particular, the smallest dimensional quadric r Q k−s is not saturated. The quadric Qrκ is the smallest dimensional quadric in dk−s dκ the sequence (L•,Q•) which is not maximally singular given the larger quadrics containing it.

We will compute the class of V (L•,Q•) by successively increasing rκ by one, where κ is the active index. This corresponds to a specialization of the ﬂag deﬁning the restriction variety. In the process, V (L•,Q•) will specialize into a union of restriction varieties. Applying the degeneration to each of the resulting varieties, we will be able to decompose any restriction variety into a union of Schubert varieties.

Degeneration 5.7. Let Sing(Q) denote the singular locus of a quadric Q. To avoid multiple indices set L = L . Let p ∈ L ∩ Sing(Qrκ+1 ). Suppose that nxκ+1 dκ+1 p 6∈ Sing(Qrκ ). Recall that L is the smallest dimensional isotropic linear space dκ in (L ,Q ) that is not entirely contained in Sing(Qrκ ). It is understood that if • • dκ κ = k − s, the condition that p ∈ Sing(Qrκ+1 ) is vacuous. Similarly, if x = s, then dκ+1 κ p ∈ Qrκ ∩ Sing(Qrκ+1 ), but p 6∈ Sing(Qrκ ). dκ dκ+1 dκ Let S = Span(Qrκ ) and let U = Sing(Qrκ ). Let T = T Qrκ−1 be the tangent dκ dκ p dκ−1 space to Qrκ−1 at p. By Condition (5), Qrκ−1 is smooth at p so the tangent dκ−1 dκ−1 34 hyperplane exists. Moreover, since p is not a singular point of Qrκ , T cannot dκ contain Qrκ . We conclude that T ∩ S := M is a codimension one linear space. dκ On the other hand, since Qrκ+1 is singular at p, M automatically contains Qrκ+1 . dκ+1 dκ+1 Let N = Span(M,U). Note that N has dimension dκ. Consider the pencil of linear spaces determined by N and S. Since N and S have M in common, they span a linear space of dimension dκ + 1. In this linear space and in appropriate coordinates, this pencil can be expressed as tx + (1 − t)y, where y = 0 defines r N and x = 0 defines S. This pencil of linear spaces cut out a pencil Q κ(t) (t) of dκ sub-quadrics on Q. When t = 1, this is the original quadric Qrκ . When t = 0, dκ it is a quadric of corank r + 1. Note that all of these quadrics contain Qrκ+1 κ dκ+1 and are contained in Qrκ−1 . Consequently, there exists a one-parameter family of dκ−1 sequences (L (t),Q (t)), where only the quadric Qrκ(t)(t) varies in the pencil just • • dκ constructed. At a general t, the sequence is projectively equivalent to the original sequence. At the special point t = 0, the sequence (L•(0),Q•(0)) is equivalent to a sequence where rκ has been replaced by rκ + 1. Correspondingly, there is a one-parameter family of restriction varieties V (t) defined with respect to the flags (L•(t),Q•(t)). As long as t 6= 0, these varieties are isomorphic. Hence, they form a flat family. By the properness of the Hilbert scheme, there exists a flat limit V (0). Our algorithm is obtained by describing V (0).

Notation 5.8. For the rest of the paper, we will always use Degeneration 5.7. Given an admissible sequence (L•,Q•), (L•(t),Q•(t)) will denote the position of a a the flag at time t under this degeneration. To simplify notation, we will use (L•,Q•) to denote the special position of the flag at t = 0. The dimension of the linear spaces a a 0 0 and the dimension and corank of the quadrics in (L•,Q•) will be denoted by nj, di 0 0 and ri, respectively. Note that except for rκ, these invariants equal to those of 0 (L•,Q•) and rκ = rκ + 1. a a a Observe that the sequence of brackets and braces associated to (L•,Q•) is D defined in §3. The degeneration increases rκ by one. This is represented by changing the integer in the (rκ +1)-st place in the quadric diagram corresponding to (L•,Q•) to κ. a a The reader should note that the sequence (L•,Q•) does not have to be ad- a a missible. The algorithm will consist of decomposing (L•,Q•) into a collection of j j admissible sequences (L•,Q•). The flat limit will be supported along the union of the restriction varieties corresponding to these sequences. We replace V (L•,Q•) by j j a collection of restriction varieties V (L•,Q•) each occurring with multiplicity one. Hence, the cohomology class of V (L•,Q•) is the sum of the cohomology classes j j j j of V (L•,Q•). The varieties V (L•,Q•) will be “closer” to Schubert varieties. By j j j “closer” we mean that the admissible sequence (L•,Q•) will have either s = s + 1 j (one more linear space and one fewer quadric); or ri ≥ ri with strict inequality for at least one i (one of the quadrics will have a higher dimensional singularity). If we keep applying the algorithm to each of the varieties that are output, the varieties will eventually become saturated. Hence, we will express the class of V (L•,Q•) as a sum of Schubert cycles.

A reminder about our notation: Recall that κ denotes the active index of (L•,Q•). xi denotes the number of isotropic subspaces of the sequence contained in the 35 singular locus of Qri . In particular, if x < s, then L denotes the small- di i nxi+1 est dimensional isotropic space in the sequence strictly containing Qri,sing (in the di quadric diagram notation, L is depicted by the left most bracket such that nxi+1 one of the digits to its left is zero or greater than i). yj denotes the index of the largest dimensional quadric containing Lnj in its singular locus or yj = k − s + 1 if there are none (in terms of quadric diagrams, yj is the positive digit to the immediate left of the j-th bracket or yj = k − s + 1 if this digit is zero.) The condition n − r − 1 = y − κ means that the codimension of Qrκ,sing in L is xκ+1 κ xκ+1 dκ nxκ+1 one more than the number of quadrics in the sequence that contain Qrκ but do not dκ contain L in their singular locus. nxκ+1 Algorithm 5.9. We now give the algorithm that describes the maximal dimensional components of the ﬂat limit of Degeneration 5.7.

Step 1. If V (L•,Q•) is saturated (i.e., a Schubert variety), output V (L•,Q•) and stop. The algorithm terminates. Otherwise, • Let (La,Qa) be the sequence obtained by replacing Qrκ in (L ,Q ) with • • dκ • • 0 rκ+1 rκ Q = Q 0 . dκ dκ b b • If xκ < s, then let (L ,Q ) be the sequence obtained by replacing Ln 0 • • xκ+1 0 a a rκ in (L ,Q ) with Lr0 (the singular locus of Q 0 ). • • κ dκ and proceed to Step 2. Step 2. Depending on the case, replace V (L•,Q•) by the following union of restriction varieties and stop.

• If xκ = s or if nxκ+1 − rκ − 1 > yxκ+1 − κ in the sequence (L•,Q•), replace V (L•,Q•) with the restriction varieties obtained by running Algorithm 5.10 on a a (L•,Q•). 0 0 a a 0 dκ−rκ • If (L•,Q•) violates Condition (8) (i.e., xκ < k − κ + 1 − 2 ), then replace V (L•,Q•) with the restriction varieties obtained by running Algorithm 5.10 on b b (L•,Q•).

• If xκ < s, nxκ+1 − rκ − 1 = yxκ+1 − κ in the sequence (L•,Q•) and Condition 0 0 a a 0 dκ−rκ (8) is satisﬁed for (L•,Q•) (i.e., xκ ≥ k − κ + 1 − 2 ), then replace V (L•,Q•) with the restriction varieties obtained by running Algorithm 5.10 on both sequences a a b b (L•,Q•) and (L•,Q•).

α α Algorithm 5.10 (Normalizing the sequence). Given a sequence (L• ,Q• ) equal a a b b to (L•,Q•) or (L•,Q•) deﬁned in Algorithm 5.9, run the following loop on the sequence. We will call the process of replacing the sequence (L•,Q•) by the sequences produced by this algorithm normalizing the sequence. α α α α i. If the sequence (L• ,Q• ) is admissible, output the sequence (L• ,Q• ) and stop. Otherwise, proceed to [ii]. α α ii. If rk−s + 2 ≥ dk−s (i.e., Condition (7) is violated) in (L• ,Q• ), replace α α i i i i (L• ,Q• ) by two sequences (L•,Q•) for i = 1, 2, where (L•,Q•) is the sequence obtained from (Lα,Qα) by replacing Qrk−s with L unless • • dk−s dk−s−1 2(dk−s − 1) = n. If 2(dk−s − 1) = n, then in one of the sequences replace Qrk−s with L and in the other with L0 . If in addition 2k = n, dk−s dk−s−1 dk−s−1 discard the sequence that parameterizes linear spaces that has the wrong 36 parity for the dimension of intersection with Lk. For each of the sequences i i α α (L•,Q•), return to Step [i] and run the loop again setting (L• ,Q• ) = i i (L•,Q•). If rk−s + 2 < dk−s (i.e., Condition (7) holds), proceed to [iii]. α α iii. If Condition (9) is violated for (L• ,Q• ), while Condition (9) is violated, let µ be the largest index for which it is violated. Form a new sequence (Lβ,Qβ) by replacing Qrµ in (Lα,Qα) with Qrµ+1 . Discard the sequence • • dµ • • dµ−1 β β α α (L• ,Q• ) if dµ+1 = dµ − 1 in (L• ,Q• ). If there are no remaining sequences, β β the algorithm terminates and does not our put any sequences. If (L• ,Q• ) satisﬁes Condition (9), proceed to Step [i] and run the loop again setting α α β β (L• ,Q• ) = (L• ,Q• ).

a a We already observed that the sequence (L•,Q•) is represented by the sequence a b b of brackets and braces D defined in §3. Next observe that (L•,Q•) is represented b b b a a by D defined in §3. (L•,Q•) is obtained from (L•,Q•) by replacing the smallest r0 dimensional linear space containing the singular locus of Q κ with the singular locus dκ r0 of Q κ . This corresponds to shifting the left most bracket whose position is greater dκ than lDa (≤ κ) to the position lDa (≤ κ). The problem, as observed in §3, is that Da and Db need not be quadric diagrams. a a b b Equivalently, (L•,Q•) and (L•,Q•) may fail to be admissible. Algorithm 5.10 a a replaces them by admissible sequences. The sequence (L•,Q•) may fail to satisfy Conditions (7), (8), or (9). If it fails to satisfy Condition (8), this sequence does not lead to a variety supported on the flat limit. If it fails to satisfy Condition (7), then Algorithm 5.10 in Step (ii) replaces the sequence by two sequences. The r0 geometric meaning of this step is that the quadric Q κ is reducible consisting of a dκ union of two linear spaces. When n is even and the linear spaces have dimension n/2, they belong to two different connected components. These are distinguished in the algorithm. a a When (L•,Q•) fails to satisfy Condition (9) such as in the sequence represented by 10]0]0]00000}0, the loop in Step iii of Algorithm 5.10 increases the dimension of the singular locus of the quadric failing Condition (9) by one and decreases its dimension by one until Condition (9) is satisfied. In this case, the loop would produce the sequences represented by 11]0]0]0000}00, 11]1]0]000}000 and 11]1]1]00}0000, which satisfies Condition (9). Note however that Condition (7) may now fail to be satisfied, hence needs to be checked again. In Algorithm 5.10, it would have made more sense to swap Steps ii and iii. We write it this way for consistency with the case of flag varieties. b b The sequence (L•,Q•) may also fail to satisfy Condition (9). For example, the sequence represented by 3]0000}00}0} fails Condition (9). The loop in Step iii of Algorithm 5.10 increases the dimension of the singular loci and decreases the dimension of the quadrics containing the quadric failing Condition (9) successively. In this case, the loop would produce the sequences represented by 2]0000}0}00} and 1]0000}0}0}0, successively.

The geometric meaning of Step iii in Algorithm 5.10 is as follows. When ri = n − 1, by “the variation of tangent spaces”, the tangent space to Qri is constant j di along Lnj . Hence, if a linear space intersects Lnj , then it must be contained in this fixed tangent space. Therefore, the subspaces that are contained in Qri are di 37 already contained in the codimension one quadric cut out on Qri by the linear di space everywhere tangent to Qri along L . The dimension of this quadric is one di nj smaller and its singular locus contains Lnj . Step iii of the Algorithm 5.10 replaces Qri with this quadric. di The geometric meaning of Algorithm 5.9 is apparent. Step 1 checks whether a given restriction variety is a Schubert variety. If so, the algorithm stops. Other- wise, we increase the corank of Qrκ by one using Degeneration 5.7. There are two dκ possibilities. Either the linear spaces intersect the new singular locus of Qrκ+1 in a dκ a a vector space of dimension xκ (this possibility corresponds to the sequence (L•,Q•) and is depicted by Da) or they intersect the singular locus in a subspace of dimen- b b sion xκ + 1 (this possibility is depicted by the sequence (L•,Q•) and is depicted b b b by D ). Under the first condition in Step 2, the variety corresponding to (L•,Q•) has smaller dimension than the original variety V (L•,Q•). Therefore, the sequence b b (L•,Q•) does not lead to a component of the flat limit of the Degeneration 5.7. We a a replace the original sequence by sequences obtained from (L•,Q•). In the second a a case, (L•,Q•) violates Condition (8), hence the dimension of intersection of the r0 linear spaces with the singular locus Q κ has to increase. Therefore, the only possi- dκ b b bilities are derived from the sequence (L•,Q•). In the final case, sequences derived a a b b from both sequences (L•,Q•) and (L•,Q•) give components of the flat limit of the Degeneration 5.7. This is the geometric branching. From our description of the two algorithms, it is clear that Algorithm 5.9 and Algorithm 3.9 are the same algorithm, one phrased in terms of admissible sequences and the other in terms of the quadric diagrams representing them. In the rest of the section, we will work with the geometric algorithm. We will check shortly that Algorithm 5.9 replaces a restriction variety with restriction varieties. Hence, we can apply the algorithm to each of the resulting varieties until the end result is a collection of Schubert varieties. Before proceed- ing, we urge the reader to work through the examples in the beginning of this section.

Deﬁnition 5.11. A degeneration path for V1 is a sequence of restriction varieties V1 → V2 → · · · → Vm starting with V1 and ending with a Schubert variety Vm such that Vi+1 is one of the varieties assigned to Vi by Algorithm 5.9.

Theorem 5.12. The class of a restriction variety V is equal X [V ] = [Vi] where Vi are the restriction varieties produced by Algorithm 5.9. In particular, the µ coeﬃcient cλ in X µ µ [V ] = cλσλ is the number of degeneration paths starting with V and ending in a variety with µ cohomology class σλ . Furthermore, the algorithm respects the marking of restriction varieties.

Proof. We prove the theorem in three steps. We first check that Algorithm 5.9 transforms restriction varieties into a collection of restriction varieties of the same dimension. Then we interpret replacing Qrκ by Qrκ+1 in Step 1 of Algorithm 5.9 dκ dκ 38 as applying Degeneration 5.7. Using a dimension count, we show that the flat limit is supported along the varieties produced by the algorithm. Finally, we check that the flat limit is reduced at the generic point of each of these varieties. Theorem 5.12 follows. We begin by analyzing each case in the algorithm separately.

• If the sequence (L•,Q•) is saturated, then Lemma 4.18 implies that V (L•,Q•) is a Schubert variety. In this case, there is nothing further to do. Accordingly, Algo- rithm 5.9 terminates. From now on we may assume that (L•,Q•) is not saturated.

a a b b The new sequences (L•,Q•), (L•,Q•) formed in Step 1 may fail to be admissible. However, Conditions (1)-(6) are satisfied for them and for any of the sequences a a output by Algorithm 5.9. We begin by verifying this for (L•,Q•). Conditions (4) and (5) hold by construction. Since Conditions (1) and (2) hold for (L•,Q•) and replacing rκ by rκ + 1 can only increase the left-hand-side of the inequalities, a a Conditions (1) and (2) also hold for (L•,Q•). The active index κ is chosen so that Qrκ satisfies the strict inequality d + r < d + r ≤ n in Condition dκ κ κ κ−1 κ−1 (3). Increasing rκ by one can at worst turn these inequalities into equalities and improves the corresponding inequality for the index κ. Therefore, Condition (3) a a a a holds for (L•,Q•). The sequence (L•,Q•) satisfies Condition (6) by the choice of κ. The ranks of the quadrics ri remain unchanged for indices i 6= κ. The choice of κ implies that j − i ≤ di − dj = rj − ri for every j > i ≥ κ in (L•,Q•). Hence, 0 0 replacing rκ with rκ + 1 ensures the inequality ri − rκ ≥ i − κ − 1 for i > κ. 0 0 The inequalities for rκ − ri improve by one for κ > i. Finally, the second half of Condition (6) is also immediate from the choice of κ. We conclude that Conditions a a (1)-(6) hold for (L•,Q•). b b a a Next we note that the sequence (L•,Q•) is obtained from (L•,Q•) by replacing the linear space Ln 0 with the smaller dimensional linear space Lr0 . Replacing a xκ+1 κ linear space by a smaller dimensional one clearly preserves Conditions (1)-(4) and 0 (6). Since all the quadrics with corank r ≤ r are singular along L 0 , Condition i κ rκ b b (5) also holds. Hence, the sequence (L•,Q•) satisfies Conditions (1)-(6). Finally, we analyze how Algorithm 5.10 affects Conditions (1)-(6). We make the a a observation that if Condition (9) fails for (L•,Q•), then it fails only for the index b b κ. If Condition (9) fails for (L•,Q•), then it can only fail for indices i < κ. • In Step ii of Algorithm 5.10, the quadric Qrk−s is replaced by a linear dk−s

space Ldk−s−1 of dimension dk−s − 1. Conditions (2)-(6) are unaffected by this change. By assumption, we have dk−s ≤ rk−s + 2. Hence 2ns+1 = 2(dk−s − 1) ≤ dk−s + rk−s ≤ dk−s−1 + rk−s−1. This verifies Condition (1). • In Step iii of Algorithm 5.10, a quadric Qri is replaced by a quadric of di corank ri + 1 and ambient dimension di − 1. Note that all the inequalities in (1)-(3) are invariant under this transformation. Conditions (4) and (5) hold by construction. Condition (9) may fail to be satisfied for the index a a b b κ in (L•,Q•) or for some indices i < κ in (L•,Q•). In the former case the loop increases the rank rκ to that of at most rκ+1 and it is clear that the b b resulting sequence satisfies Condition (6). If (L•,Q•) violates Condition (9), it either violates it for all 1 ≤ i ≤ κ or only for κ − 1. In the first case, in (L•,Q•) we must have r1 = rκ = xκ in (L•,Q•) and the loop produces a sequence that satisfies the same equalities. Else rκ−1 = rκ in (L•,Q•) and by Condition (6) for (L•,Q•), dκ−1 − 1 = dκ and the loop in Step iii, 39 does not produce any sequences. Hence, the sequences produced in Step iii satisfy Conditions (1)-(6).

We conclude that all the sequences occurring in the algorithm satisfy Conditions (1)-(6). If Conditions (7)-(9) hold in a sequence for all the indices i ≥ α, then Algorithm 5.10 does not modify the quadrics with these indices. Hence, every intermediate sequence formed during Algorithm 5.10 also satisﬁes Conditions (7)-(9) for indices i ≥ α. a a Condition (8) may fail to hold for (L•,Q•). However, since (L•,Q•) is admissible, a a Condition (8) can only fail in (L•,Q•) for the index κ and the right hand side of the inequality can exceed the left hand side by at most 1/2. Moreover, in (L•,Q•), we must have the equality d − r x = k − κ + 1 − κ κ . κ 2 The choice of κ implies that equality holds in Condition (8) for all the indices i > κ in the sequence (L•,Q•). Since ri + di = rκ + dκ for all i > κ, we can rewrite the inequality in Condition (8) for the index i as

xi ≥ xκ + ri − rκ + κ − i.

By Condition (9), rκ+1 − rκ − 1 ≥ xκ+1 − xκ. Hence, we see that equality holds for the index κ + 1. By induction, it follows that equality holds for all the indices

κ ≤ i ≤ k − s. Furthermore, nxκ+1 − rκ − 1 = yxκ+1 − κ in (L•,Q•). Finally, note that if xκ = s, then equality for the index k − s implies that dk−s = rk−s + 2 contradicting Condition (7) for (L•,Q•). We conclude that if Condition (8) fails a a for (L•,Q•), then xκ < s and nxκ+1 − rκ − 1 = yxκ+1 − κ. Therefore, the cases in Algorithm 5.9 are exhaustive and mutually exclusive. We may assume from now a a on that the sequence (L•,Q•) satisfies Condition (8). We also conclude that the b b sequence (L•,Q•) satisfies both Conditions (7) and (8). Since xκ < s, Condition a a (7) has to hold for (L•,Q•). Replacing a linear space with a smaller linear space does not affect Condition (7). Replacing L 0 with L 0 increases the left hand xκ+1 rκ side of the inequality in Condition (8) by one without affecting the right hand side.

b b Therefore, (L•,Q•) is either admissible or fails Condition (9). As we observed while verifying Step iii of Algorithm 5.10 preserves Conditions (1)-(6), no new sequences are formed unless Condition (9) fails for all the indices 1 ≤ i ≤ κ − 1. In this case, any sequence formed in Step iii of Algorithm 5.10 clearly satisﬁes Condition (9), hence is admissible. Hence, every sequence formed in Step 4 of Algorithm 5.9 is admissible. a a Condition (7) may fail to hold for (L•,Q•) or while running Step iii of Algorithm a a 5.10 on (L•,Q•). This may happen in only one of two ways. The sequence (L•,Q•) either has dk−s = rk−s + 3 and κ = k − s; or dk−s = rk−s + 3 + 2α, κ = k − s and (L•,Q•) has α linear spaces of dimensions rk−s + 2, rk−s + 3, . . . , rk−s + α + 1 = ns. By the observation three paragraphs above, κ = k − s. Hence, either Condition (7) a a is directly violated for (L•,Q•) or Condition (7) is violated after applying Step iii of Algorithm 5.10 for the index κ α times. The equality dk−s = rk−s + 3 + 2α 40 follows by combining Condition (8) d − r s − α > s + 1 − k−s k−s 2 for (L•,Q•) with the inequality dk−s −rk−s −2α ≤ 3 that expresses that Condition (7) is violated after α-steps. In either of the two cases, Step ii of Algorithm 5.10 outputs admissible sequences. a a Finally, if Condition (9) fails for the sequence (L•,Q•), then, as observed above, it fails only for the index κ. Applying Step iii of Algorithm 5.10 either produces a sequence which is admissible or which violates Condition (7). In the latter case, running Step ii of Algorithm 5.10 outputs an admissible sequence. We conclude that all the sequences output by Algorithm 5.9 are admissible. We now analyze the dimensions of the corresponding varieties. • The expression in Equation (1) for the dimension of a restriction variety remains unchanged when we replace Qrκ with Qrκ+1. dκ dκ 0 • In Cases 2 and of the Algorithm, we have the equality n 0 −r = y 0 − xκ+1 κ xκ+1 b b κ. Hence, when we replace Ln 0 with Lr0 to form (L ,Q ), xi increases xκ+1 κ • •

by one for the indices κ ≤ i < yxκ+1. The dimension of the linear space with index x0 +1 decreases by n0 −r0 . All other terms in the expression κ xκ+1 κ in Equation (1) remain unchanged. • Step iii of Algorithm 5.10 increases xµ by one and decreases dµ by one, hence preserves the expression in Equation (1). • Finally, replacing Qdk−s−2 with L in Step ii of Algorithm 5.10 in- dk−s dk−s−1 creases the first sum in Equation (1) by dk−s − s − 2. It changes the second sum by −xs − dk−s + 2s + 2. Since we must have xs = s, we conclude that this step also preserves the expression in Equation (1). Combining these observations, we conclude that every sequence produced by Algorithm 5.9 is admissible and gives rise to a restriction variety of the same dimension as V (L•,Q•). The algorithm can be recursively applied to each of the resulting restriction varieties. It is clear that the algorithm must terminate in a collection of Schubert varieties. At each stage of the algorithm, either the corank of a quadric in the sequence increases by at least one or the number of quadrics in the sequence decreases. Since there are finitely many quadrics in the sequence and the corank of the quadrics are bounded above, eventually the sequence must become saturated. Then the resulting varieties are Schubert varieties. We now analyze Degeneration 5.7 to conclude that the support of the flat limit is the union of restriction varieties replacing V (L•,Q•) in Algorithm 5.9. In order to restrict the possible irreducible components of the support of the flat limit, we write down conditions that the linear spaces in the limit have to satisfy. We then observe that these conditions already cut out varieties of dimension equal to the dimension of V (L•,Q•). The following observation puts strong restrictions on the support of the flat limit. Observation 5.13. The linear spaces parameterized by the restriction varieties

V (L•(t),Q•(t)) intersect the linear spaces Lnj (t) in a subspace of dimension at least j and the quadrics Qri (t) in a linear space of dimension at least k − i + 1. di Similarly, they intersect Qri,sing(t) in a linear space of dimension at least x . Since di i intersecting a proper variety in at least a given dimension is a closed condition, 41 the linear spaces parameterized by the ﬂat limit V (0) have to intersect the linear spaces L (0) in a subspace of dimension at least j and the quadrics Qri (0) in a nj di subspace of dimension at least k − i + 1. Furthermore, they intersect Qri,sing(0) in di a subspace of dimension at least xi.

A quick inspection of the algorithm will reveal that in each of the limits either the linear spaces intersect the vertex of Qrκ+1(0) in a subspace of dimension x + 1 dκ κ and otherwise remain as unconstrained as possible given Observation 8.3; or the linear spaces continue to intersect Qrκ+1(0) in a subspace of dimension x and only dκ κ satisfy the constraints imposed by Observation 8.3. A priori in the limit the linear spaces could become more special. However, we claim that these loci have strictly smaller dimension and do not form an irreducible component of the support of the flat limit. We now verify this claim. Let Y be an irreducible component of the support of the flat limit of Degeneration 5.7. Then we can build a sequence of consisting of k linear spaces and quadrics such that the closure of the locus of linear spaces intersecting the i-th element in the sequence (counting in increasing dimension) in dimension i contains Y . We a a complete the linear spaces and quadrics in the sequence (L•,Q•) to a set of linear spaces and quadrics whose dimensions increase by one at each stage making sure that Conditions (4) and (5) of Definition 4.2 are satisfied. We then select those linear spaces and quadrics that have a jump in the dimension of intersection with a general linear space parameterized by Y . We thus obtain a set of k linear spaces and quadrics. By construction the closure of the locus X of linear spaces that intersect the i-th one in dimension i contains Y . Observation 8.3 implies that the i-th linear space or quadric in the sequence thus obtained has dimension less than or equal to the i-th linear space or quadric (counting in increasing dimension) in a a the sequence (L•,Q•). By Proposition 7.21, Equation (1) gives an upper bound on the dimension of X (note that we used the fact that the sequence is admissible in the proof only to deduce the equality). We now estimate the dimension of X. We obtain the sequence defining X by a a replacing linear spaces and quadrics in (L•Q•) by smaller dimensional ones in increasing order. We will do this in greater generality in preparation for the discussion of orthogonal flag varieties. 0 • If we replace a linear space of dimension ni+j in the (i+j)-th position with ∗ a a a linear space of dimension ni in the i-th position not contained in (L•,Q•), 0 then according to Equation (1) the dimension changes as follows. Let yi+j ∗ and yi be the smallest index quadrics containing the corresponding linear 0 ∗ spaces in their singular locus. The dimension decreases by ni+j − ni + j + 0 ∗ yi+j − yi . Since Conditions (6) and (9) hold for (L•,Q•), we have that 0 ∗ 0 ∗ ni+j − ni + yi+j − yi ≥ 0. Consequently the decrease in the dimension is 0 ∗ 0 ∗ at least j with equality when ni+j − ni + yi+j − yi = 0. 0 • If we replace the i-th largest quadric in a vector space of dimension di by ∗ the (i + j)-th largest quadric in a vector space of dimension di+j , then 0 ∗ 0 ∗ according to Equation (1) the dimension decreases by di − di+j + xi − xi+j. This decrease is at least j with strict inequality unless Condition (9) fails 0 for ri.

42 0 ri • Finally, if we replace the quadric Q 0 with the linear space Ln∗ , then the di j 0 ﬁrst sum in Equation (1) changes by nj − s − 1. The second sum changes 0 ∗ 0 by −di + (k − s − yj − xi) + (2s + 2). Hence, the total change is 0 ∗ 0 ∗ −di + nj + k + 1 − xi − yj , where y∗ denotes the index of the largest dimensional quadric containing nj L∗ in its singular locus. We rewrite this expression as follows: nj d0 − r0 d0 + r0 (k − i + 1 − b i i c − x0 ) + (n∗ − d i i e + k − s − y∗) + (−k + s + i). 2 i j 2 j The sum in the ﬁrst parentheses is strictly negative unless Condition (8) is violated or there is equality in Condition (8); otherwise it is zero. The sum in the second parentheses is strictly negative unless j = s + 1 and 0 0 di + ri = dk−s + rk−s; otherwise it is zero. Finally, the third sum is strictly negative unless i = k − s; otherwise it is zero.

Since our degeneration is ﬂat, Y has to have the same dimension as V (L•,Q•). Since X contains Y , our dimension calculation puts strong restrictions on X.

First, suppose xκ = s in (L•,Q•). Then by Conditions (6) and (9) for (L•,Q•), 0 0 0 0 0 a a nl − ri + yl − i > 0 for every l with nl > ri in (L•,Q•). Furthermore, Condition (9) a a 0 0 holds for (L•,Q•). If dk−s − rk−s > 2, then replacing any linear space or quadric with a smaller dimensional one strictly decreases the dimension. Note also that a a a a (L•,Q•) is admissible. In this case, we conclude that X has to be V (L•,Q•). Since a a V (L•,Q•) and V (L•,Q•) have the same dimension, we conclude that Y has to be 0 a a 0 0 rk−s a component of V (L•,Q•). If dk−s − rk−s = 2, then Q 0 is necessarily reducible dk−s 0 0 consisting of two linear spaces of dimension dk−s −1. If 2(dk−s −1) = n, then these linear spaces belong to two diﬀerent connected components. We can therefore 0 rk−s replace Q 0 with either of these linear spaces to obtain two sequences. Note that dk−s replacing any other linear space or quadric with a smaller dimensional one strictly decreases the dimension. Hence X has to be the variety corresponding to one of these sequences. Since X has the same dimension as V (L•,Q•), we conclude that Y has to be an irreducible component of X. Observe that Algorithm 5.9 selects the sequences corresponding to X.

Next, suppose that xκ < s and nxκ+1 − rκ − 1 > yxκ+1 − κ in (L•,Q•). Then a a the sequence (L•,Q•) is admissible. Furthermore, by our dimension calculation, a a replacing any linear space or quadric in (L•,Q•) leads to a strictly smaller dimen- a a sional locus. We conclude that X = V (L•,Q•) and Y has to be an irreducible a a component of V (L•,Q•). a a Next, suppose that xκ < s and Condition (8) is violated for (L•,Q•) for κ. Note that in that case, there must be an equality in Condition (8) in (L•,Q•) for the index κ. Hence, by Conditions (6) and (8), rκ−1 < rκ in (L•,Q•). By the “linear 0 rκ space bound”, every linear space of dimension k − κ + 1 contained in Q 0 must dκ intersect the singular locus of this quadric in dimension at least xκ + 1. Hence, we a a b b can replace the sequence (L•,Q•) with the sequence (L•,Q•). By our dimension calculation, replacing any linear space or quadric by a smaller dimensional one results in a variety of strictly smaller dimension. Hence, we conclude that Y has b b b b to be an irreducible component of V (L•,Q•). As we observed above, (L•,Q•) may 43 fail Condition (9) for i < κ. In that case, by the “variation of tangent spaces”, any linear space of dimension k − i + 1 intersecting L 0 in dimension x is necessarily rκ i contained in the quadric everywhere tangent to Q along L 0 . Hence, we can replace rκ b b the sequence (L•,Q•) as in Step iii of Algorithm 5.10 to obtain an equivalent deﬁnition of the same variety (Note that since rκ−1 < rκ in (L•,Q•), the deﬁnition of κ ensures that dµ − 1 < dµ−1 while running Step iii of Algorithm 5.10).

Finally, suppose that xκ < s and nxκ+1 − rκ − 1 = yxκ+1 − κ in (L•,Q•) a a and (L•,Q•) satisﬁes Condition (8). Let i ≤ κ be the smallest index such that 0 a a n 0 −r = y 0 −i in (L ,Q ). Note that by Conditions (6) and (9) for (L ,Q ), xi+1 i xi+1 • • • • there may be such indices precisely when rκ−1 = rκ, ri ≥ xκ and ri = rκ − κ + i + 1 in (L ,Q ). By our dimension counts, replacing L 0 with L 0 for an index • • xκ+1 rj i ≤ j ≤ κ can result in a sequence that has the same dimension as V (L•,Q•). Replacing any other linear space or quadric with a smaller dimensional one, gives a smaller dimensional variety. The rest of the analysis of this case is more subtle. We need to argue that unless j = κ, these loci do not occur in the limit. For a general linear space W ∈ V (L (t),Q (t)), let W = Qrj (t) ∩ W for i ≤ j < κ. t • • t,j dj t The tangent space to Qrj along W intersects L in a subspace of dimension dj t,j nxκ+1 rj + 1. By semi-continuity, this must be true for every linear space contained in V (L•(t),Q•(t)) and also in the limit V (L•(0),Q•(0)). However, the tangent space to Qrj at a general linear space parameterized by the variety associated to the dj a a sequence obtained from (L ,Q ) by replacing L 0 with L 0 intersects L in • • xκ+1 rj nxκ+1 0 dimension rj = rj. We conclude that the support of Y cannot equal such a locus. a a b b Hence X is the locus associated to one of the sequences (L•,Q•) or (L•,Q•). These sequences may fail to satisfy Condition (9). In that case, Step iii of Algorithm 5.10 b b replaces them by equivalent varieties unless for (L•,Q•) we have dκ−1 − 1 = dκ. In the latter case, by “the variation of tangent spaces”, the (k − κ + 2)-dimensional 0 rκ subspaces of the linear spaces W parameterized by X have to be contained in Q 0 . dκ 0 rκ−1 In other words, we have to replace Q 0 by a smaller quadric. By our dimension dκ−1 counts, such a locus has strictly smaller dimension, hence cannot support Y .

In order to conclude the proof, we need to verify that the limits all occur and are reduced at the generic point of each of these loci. This is a straightforward local calculation. Let U be the Zariski open set of our family of restriction varieties parameterizing linear spaces W (t) such that dim(W (t) ∩ Qrκ(t)(t)) = k − κ + 1. dκ Let Z be the family of restriction varieties obtained by applying Degeneration 5.7 to the admissible sequence obtained from (L•,Q•) by omitting the quadrics Qr1 ,...,Qrκ−1. Then there exists a natural morphism f : U → Z sending W (t) d1 dκ−1 to W (t) ∩ Qrκ(t)(t), which is smooth at the generic point of each of the irreducible dκ components of the ﬁber of Z at t = 0. We may, therefore, assume that κ = 1. Furthermore, without loss of generality, we may assume that n = dκ + rκ + 1 and xκ = 0. We will check that the multiplicity is one by exhibiting cycles that intersect V (L•,Q•) in one point and exactly one of the potential limits in one point. This will allow us to conclude that each of the limits occur with multiplicity one. There is a a a b b Schubert cycle in the class of the variety V (L•,Q•) (respectively, V (L•,Q•)) that b b occurs with coeﬃcient one and does not occur in the class of V (L•,Q•) (respectively, a a V (L•,Q•)). We use the dual of these Schubert cycles for our computation. Note 44 that by our assumptions on κ and n, di + ri = dκ + rκ = n − 1 for every i ≥ κ. Hence, n − dk−s + 1 = rk−s + 2 and 2(rk−s + 2) ≤ rk−s + dk−s + 1 = n.

First, suppose xκ = s(= 0) and dκ = rκ + 3 in (L•,Q•). In this case, this is the standard family of a quadric breaking into a union of two linear spaces. Both occur in the limit with multiplicity one. In this case there is nothing to check. Next suppose xκ = s(= 0) and dκ > rκ + 3 in (L•,Q•). Let βi = n − di + 1. Let S be the Schubert variety deﬁned with respect to a general isotropic ﬂag

Lβ1 ⊂ · · · ⊂ Lβk−s . 0 In case 2βk−s = n, we will always deﬁne a second Schubert variety S by replacing L with L0 . Note that under our assumptions V (Q ) is irreducible. Both βk−s βk−s • α 0 V (Q•) and V (Q• ) intersect S (and S when appropriate) in a reduced point. The r0 spans Span(Qri ) and Span(Q i ) intersect the linear space L in a one dimensional di di βi subspace for every 1 ≤ i ≤ k. Any k-dimensional linear space contained in V (Q•)∩ α S or V (Q• ) ∩ S must contain these one-dimensional subspaces. Hence, the k- dimensional linear space is uniquely determined as the Span((Qri ∩ L ), 1 ≤ i ≤ k) di i r0 or Span((Q i ∩ L ), 1 ≤ i ≤ k), respectively. By Kleiman’s Transversality Theorem di i [Kl1], we conclude that the intersection of the two varieties consists of a single reduced point. When 2βk−s = n, two general linear spaces in the class L = Lβk−s intersect in a unique point if n = 0 modulo 4 and are otherwise disjoint. A general linear space in the class L and a general linear space in the class L0 = L0 βk−s intersect in a unique point if n = 2 modulo 4 and are otherwise disjoint. When 0 0 V (Q•) has two components, repeating the argument with S , we conclude that both components occur with multiplicity one. a a Next, suppose that xκ(= 0) < s and (L•,Q•) fails to satisfy Condition (8). Let

αxκ+1 = α1 = n − rκ. Let αj = n − nj−1 for j > xκ+1. Let βi = n − di + 1. Let S be the Schubert variety deﬁned with respect to the linear spaces and quadrics

L ⊂ · · · ⊂ L ⊂ Qn−αs ⊂ · · · ⊂ Qn−α1 . β1 βk−s αs α1 Proposition 4.18 implies that S is a Schubert variety. We claim that S intersects b b both V (L•,Q•) and V (L•,Q•) in a unique, reduced point. The linear spaces Lβi b ri ri intersect the quadrics Qd and Q b in unique points. Any k-dimensional linear i di b b space in the intersection of S and V (L•,Q•) or S and V (L•,Q•) must contain the (k − s)-dimensional linear space Λ spanned by these points. In S ∩ V (L•,Q•), the n−αj quadrics everywhere tangent to Λ determine unique points in Qαj ∩Lnj for j > 0. b b In S ∩ V (L•,Q•), the quadrics everywhere tangent to Λ determine unique points in n−αj Q ∩ L for j > 1 and furthermore the k-plane has to contain the point L b ∩ αj nj rκ Qn−α1 (which is contained in the singular locus of all the quadrics). Hence, in both α1 cases, the k-dimensional linear space in the intersection is uniquely determined. We b b proved above that ns = n/2 in this case. Hence, both V (L•,Q•) and V (L•,Q•) b b are irreducible. Therefore, V (L•,Q•) occurs in the limit with multiplicity one.

Next, suppose xκ(= 0) < s and nxκ+1 − rκ − 1 > yxκ+1 − κ. In this case, let i0 denote the smallest index for which equality holds in Condition (8) in (L•,Q•). If there is no such index, set i0 = 0 and ri0 = 0. For nj ≤ ri0 , let αj = n − nj + 1.

For nj > ri0 , set αj = n − nj−1. Next, for each index i < i0, let li be the largest positive integer such that ri + li + 1 = nxi+li . If there does not exist such li, set 45 li = 0. Let βi = n − di + li + 1 for i < i0 and let βi = n − di + 1 for i ≥ i0. Let S be the Schubert variety deﬁned by the sequence L ⊂ · · · ⊂ L ⊂ Qn−αs ⊂ · · · ⊂ Qn−α1 . β1 βk−s αs α1 When 2β = n, deﬁne S0 by replacing L with L0 . Note that Proposition k−s βk−s βk−s 4.18 implies S is a Schubert variety. As in the previous cases, it is straightforward a a to see that S intersects both V (L•,Q•) and V (L•,Q•) in a unique, reduced point. 0 a a When appropriate, the same holds for S . We conclude that V (L•,Q•) occurs in the limit with multiplicity one.

Finally, suppose xκ(= 0) < s, nxκ+1 − rκ − 1 = yxκ+1 − κ and Condition (8) is a a a a b b satisfied for (L•,Q•). Then duals for V (L•,Q•) and V (L•,Q•) are obtained as in the previous cases. Let S be the Schubert variety defined exactly as in the previous paragraph. Let T be the Schubert variety defined by replacing αxκ+1 = n−nxκ+1+1 in the definition of S with αxκ+1 = n − rκ. Then it is straightforward to see that both S and T intersect V (L•,Q•) in a unique reduced point. S (respectively, T ) a a b b intersects V (L•,Q•) (respectively, V (L•,Q•)) in a unique, reduced point and has b b a a empty intersection with V (L•,Q•) (respectively, V (L•,Q•)). It follows that both limits occur with multiplicity one. Finally, by replacing S with S0 and T with T 0 when appropriate, it is easy to see that in case these varieties are reducible, both components occur with multiplicity one and that the algorithm preserves marking. This concludes the proof of the theorem.

Remark 5.14. From the analysis in the proof of Theorem 5.12, it follows that at each stage of the degeneration a restriction variety breaks into at most three irreducible components.

6. Applications of Algorithm 5.9 In this section we discuss a couple of immediate applications of Algorithm 5.9. The Introduction discusses other applications. 6.1. The moduli space of vector bundles on hyperelliptic curves. There is a beautiful, classical construction that associates to a general pencil of quadric hypersurfaces in P2g+1 a hyperelliptic curve C of genus g. In fact, every smooth hyperelliptic curve of genus g arises this way [GH, §6], [?]. We recall the construction for the reader’s convenience. 2g+1 Let Q1 and Q2 be general quadric hypersurfaces in P . Let tQ1 + uQ2 be the pencil generated by Q1 and Q2. Consider the incidence correspondence I parameterizing pairs (Q, C), where Q is a quadric hypersurface contained in the pencil and C is a connected component of the space of g-dimensional projective linear spaces on Q. The incidence correspondence I is irreducible and maps to P1 by the first projection π1. When Q is a smooth quadric, the space of g-dimensional projective linear spaces on Q has two connected components. Hence, I is a double cover of P1. When Q has corank one, then the space of g-dimensional projective spaces has only one component. Hence, π1 is ramified at the 2g + 2 points in the pencil that are quadrics of corank one. By the Riemann-Hurwitz formula, we conclude that I is a hyperelliptic curve of genus g. To see that there are 2g + 2 corank one quadrics in a general pencil, observe that the pencil can be identified with a (2g + 2) × (2g + 2) symmetric matrix whose 46 entries are linear homogeneous polynomials in t and u. The quadrics of corank one correspond to matrices with zero determinant. Since the determinant is a homogeneous polynomial of degree 2g + 2 in t and u, it will have 2g + 2 roots in P1. If the pencil is general, these roots will be distinct and the corresponding symmetric matrix will have corank exactly one. Furthermore, it is clear from this description that one can construct a pencil with any 2g +2 distinct roots. Hence, every smooth hyperelliptic curve of genus g arises via this construction.

Let C be a smooth hyperelliptic curve of genus g ≥ 2. Let MV2,o(Cg) denote the moduli space of rank two vector bundles with a fixed determinant of odd-degree on C. Realize C as a double cover of a pencil of quadric hypersurfaces in P2g+1. By a celebrated theorem of Desale and Ramanan [?], MV2,o(Cg) is isomorphic to the space of (g − 2)-dimensional projective linear spaces contained in this pencil of 2g+1 quadric hypersurfaces in P . Equivalently, if Q1 and Q2 are two smooth quadric hypersurfaces that generate the pencil, MV2,o(Cg) is isomorphic to the space of (g − 2)-dimensional projective linear spaces contained in both Q1 and Q2. We can view the space X parameterizing (g − 2)-dimensional projective linear spaces contained in Q1 as the orthogonal Grassmannian OG(g − 1, 2g + 2), which naturally includes in G(g − 1, 2g + 2). We can also view the space of (g − 2)- dimensional projective linear spaces contained in Q2 as a subvariety Y of G(g − 1, 2g+2). Of course, Y is isomorphic to X; however, its embedding in G(g−1, 2g+2) differs from that of X by translation with an element of PGL(2g+2). By Kleiman’s Transversality Theorem, X and Y intersect transversally. Therefore, the class of the intersection Y ∩ OG(g − 1, 2g + 2) in H∗(OG(g − 1, 2g + 2), Z) is the pull-back of the class of Y in H∗(G(g − 1, 2g + 2), Z) under the map induced by inclusion. g−1 The class of Y in G(g − 1, 2g + 2) is well-known to be 2 σg−1,g−2,...,2,1. There are several ways of calculating this class. First, it is the top Chern class of the vector bundle Sym2(S∗) on G(g − 1, 2g + 2), where S∗ denotes the dual of the tautological bundle of G(g−1, 2g+2). Calculating the top Chern class of Sym2(S∗) is a standard exercise in using the splitting principle. Alternatively, one can use degenerations for a more pleasant calculation. Very briefly, break the quadric into a union of two linear spaces using a general pencil Q + tL1L2. The flat limit of the space of (g − 2)-dimensional projective linear spaces contained in Q is the space of (g − 2)-dimensional projective linear spaces contained in L1 or L2 that intersect Q ∩ L1 ∩ L2 in (g − 3)-dimensional projective linear spaces (see [?] or [?]). Now inductively break Q ∩ L1 ∩ L2 into a union of linear spaces using a general pencil. Continuing this process for (g − 1) steps, we obtain 2g−1 flags of the form 2g 2g−2 2g−4 4 P ⊃ P ⊃ P ⊃ · · · ⊃ P , where P2g−2i is one of the two linear spaces obtained by degenerating the (2g − 2i)-dimensional quadric. Inductively, the flat limit of Y is the space of (g − 2)- dimensional projective linear spaces that intersect P2g−2i in a projective space of g−1 dimension g − 2 − i. We conclude that the class of Y is is 2 σg−1,g−2,...,2,1 in the cohomology of G(g − 1, 2g + 2). g−1 In conclusion, the class of MV2,o(Cg) is 2 times the class of the restriction variety associated to the Schubert class σg−1,g−2,...,1 in G(g − 1, 2g + 2). More g−1 explicitly, the class of MV2,o(Cg) in OG(g − 1, 2g + 2) is equal to 2 times the class of the restriction variety associated to the admissible sequence 0 0 0 0 Q5 ⊂ Q7 ⊂ · · · ⊂ Q2g−1 ⊂ Q2g+1. 47 Using Algorithm 5.9 the class can be easily computed. Here we give the class for the first few genera. 1 (1)[ MV2,o(C2)] = 2σ 1 3,1 (2)[ MV2,o(C3)] = 4σ0 + 4σ 3,1 1 4,1 (3)[ MV2,o(C4)] = 16σ2 + 16σ1,0 + 16σ1 3,1 1 5,1 4,1 5,4,1 3,1 (4)[ MV2,o(C5)] = 64σ3,1 + 64σ2,1,0 + 64σ2,1 + 32σ3,0 + 32σ3 + 32σ5,0 + 5,3,1 3,2 5,3,2 32σ5 + 32σ4,0 + 32σ4 More generally, one obtains a recursion in the genus for the class. Suppose that the class of MV2,o(Cg−1) in OG(g − 3, 2g) is given by X µ [MV2,o(Cg−1)] = cλ,µ[Ωλ], µ λ µ where Ωλ is defined with respect to a sequence (L• ,Q• ). Let t be the largest index ˜λ ˜µ of a linear space in the sequence such that nt = t. Define a new sequence (L• , Q• ) by setting L˜λ = Lλ for all 1 ≤ j ≤ s and Q˜µ,ri+1 = Qµ,ri for 1 ≤ i ≤ g − 3 − s. nj nj di+1 di Set Q˜µ,r1 = Qt . Then we have that d1 2g+1−t X ˜λ ˜µ [MV2,o(Cg)] = 2 cλ,µ[V (L• , Q• )].

Remark 6.1. When g = 2, MV2,o(C2) is a complete intersection of two quadric hypersurfaces in P5 [GH, §6]. Ciprian Manolescu (in private correspondence) posed the question whether MV2,o(Cg) can be a complete intersection for g > 2. In fact, one can ask for a much weaker property. Can MV2,o(Cg) be a complete intersection of ample divisors in OG(g − 1, 2g + 2)? The codimension of MV2,o(Cg) g(g−1) g−2,g−3,...,2,1 in OG(g−1, 2g+2) is 2 . The codimension of the Schubert variety σg g2+g is g + 1. Hence, the sum of the codimensions of these two varieties is 2 + 1. If g > 2, this is less than the dimension of OG(g−1, 2g+2). Hence, if MV2,o(Cg) were g−2,g−3,...,2,1 a complete intersection of ample divisors, σg ·[MV2,o(Cg)] 6= 0. However, the cup product of these classes is clearly zero since the one-dimensional vector space 0 deﬁning the Schubert variety can be chosen to not be contained in Q2g+1 deﬁning the restriction variety. Hence, we conclude that for g > 2, MV2,o(Cg) cannot be a complete intersection of ample divisors even in OG(g − 1, 2g + 2), let alone in (2g+2)−1 G(g − 1, 2g + 2) or P g−1 . 6.2. A geometric algorithm for computing the product of arbitrary Schu- bert cycles. The pull-back of a Schubert class under the inclusion j : OG(k, n) → G(k, n) can be expressed as a sum of classes of restriction varieties. Consider a

Schubert cycle Σλ1,...,λk deﬁned with respect to a general partial ﬂag

Fn−k+1−λ1 ⊂ Fn−k+2−λ2 ⊂ · · · ⊂ Fn−λk . The intersection of this flag with the quadric hypersurface Q leads to the sequence of quadrics Q0 ⊂ Q0 ⊂ · · · ⊂ Q0 . n−k+1−λ1 n−k+2−λ2 n−λk Note that since none of the quadrics are singular, the Conditions (3)-(6) of Defi- nition 4.2 are automatically satisfied. Similarly, since there are no linear spaces in the sequence, Condition (1) is automatically satisfied. However, Condition (2) may be violated. In that case, the corresponding variety is empty and the pull-back is 48 zero. From now on we assume that the sequence satisfies all the conditions in Def- inition 4.2. If the sequence is admissible, then the pull-back of the Schubert cycle is the class of the corresponding restriction variety. However, the sequence may fail to be admissible and thus the pull-back maybe the sum of classes of restriction varieties. We now describe how to express the pull-back as a sum of these. Since Condition (2) in Definition 4.2 is satisfied, n−k +i−λi ≥ 2i for all i. Suppose that equality holds for i ≤ α and the inequality is strict for i = α + 1. Then the quadric

Qn−k+1−λ1 consists of two points p1, p2. The linear spaces have to contain one of the p and be contained in the tangent space to Q along p . Then Q1 con- i i n−k+2−λ2 sists of two lines intersecting at pi. The linear spaces containing pi have to contain one of these lines. Continuing we deduce the following proposition.

Proposition 6.2. Let σλ1,...,λk be a Schubert cycle in G(k, n). Let j : OG(k, n) → G(k, n) be the natural inclusion. Then ∗ (1) j σλ1,...,λk = 0 unless n − k − i ≥ λi for every 1 ≤ i ≤ k. (2) Suppose that n−k −i = λi for i = 1, . . . , α and n−k −i > λi for i = α+1. Further suppose that if 2k = n, then α 6= k. Let (L•,Q•) be the admissible sequence L ⊂ L ⊂ · · · ⊂ L ⊂ L ⊂ Qα ⊂ · · · ⊂ Qα ⊂ Qα . 1 2 α−1 α n−k+α+1−λα+1 n−1−λk−1 n−λk ∗ α Then j σλ1,...,λk = 2 [V (L•,Q•)], where [V (L•,Q•)] denotes the cohomology class of the restriction variety V (L•,Q•). If 2α = 2k = n, then the class is 2α−1 times the Poincar´edual of a point.

7. Symplectic restriction varieties In this section, we interpret admissible symplectic diagrams geometrically. We introduce symplectic restriction varieties and discuss their basic geometric properties. Recall that Q denotes a non-degenerate skew-symmetric form on a vector space

V of dimension n. Let Lnj denote an isotropic subspace of Q of dimension nj. Let Qri denote a linear space of dimension d such that the restriction of Q to it has di i corank r . Let K denote the kernel of the restriction of Q to Qri . i i di

Definition 7.1. A sequence (L•,Q•) is a partial flag of linear spaces Ln1 ( ··· ( r L Q k−s ··· Qr1 such that ns ( dk−s ( ( d1 • dim(Ki ∩ Kh) ≥ ri − 1 for h > i. • dim(L ∩ K ) ≥ min(n , dim(K ∩ Qrk−s ) − 1) for every 1 ≤ j ≤ s and nj i j i dk−s 1 ≤ i ≤ k − s. The main geometric objects of this paper will be sequences satisfying further properties. Definition 7.2. A sequence is in order if

• Ki ∩ Kh = Ki ∩ Ki+1, for all h > i and 1 ≤ i ≤ k − s, and • dim(L ∩K ) = min(n , dim(K ∩Qrk−s )), for 1 ≤ j ≤ s and 1 ≤ i < k −s. nj i j i dk−s A sequence (L•,Q•) is in perfect order if

• Ki ⊆ Ki+1, for 1 ≤ i < k − s, and

• dim(Lnj ∩ Ki) = min(nj, ri) for all i and j. 49 Definition 7.3. A sequence (L•,Q•) is called saturated if di + ri = n, for 1 ≤ i ≤ k − s. The next definition is the analogue of Definition 3.24 and is a consequence of the order of specialization.

Deﬁnition 7.4. A sequence (L•,Q•) is called a symplectic sequence if it satisﬁes the following properties.

(GS1) The sequence (L•,Q•) is either in order or there exists at most one integer 1 ≤ η ≤ k − s such that

Ki ⊆ Kh for h > i > η and Ki ∩ Kh = Ki ∩ Ki+1 for i < η and h > i.

Furthermore, if Kη ⊆ Kk−s, then dim(L ∩ K ) = min(n , dim(K ∩ Qrk−s )) for i < η and nj i j i dk−s dim(L ∩ K ) = min(n , dim(K ∩ Qrk−s ) − 1) for i ≥ η. nj i j i dk−s If Kη 6⊆ Kk−s, then dim(L ∩ K ) = min(n , dim(K ∩ Qrk−s )) for all i. nj i j i dk−s (GS2) If α = dim(K ∩ Qrk−s ) > 0, then either i = 1 and n = α or there exists i dk−s α at most one j0 such that, for j0 6= j > min(i, η), rj − rj−1 = dj−1 − dj. Furthermore,

rj0 dj0−1 − dj0 ≤ rj0 − rj0−1 + 2 − dim(Kj0−1) + dim(Kj0−1 ∩ Q ) dj0

rj0 and Kη 6⊂ Q . dj0

Remark 7.5. Given a sequence (L•,Q•), the basic principles concerning skew- symmetric forms imply inequalities among the invariants of a sequence. The evenness of rank implies that di − ri is even for every 1 ≤ i ≤ k − s. The corank bound implies that r − dim(Qri ∩ K ) ≤ d − d . The linear space bound implies that i di i−1 i−1 i

2(ns + ri − dim(Ki ∩ Lns )) ≤ ri + di for every 1 ≤ i ≤ k − s. These inequalities are implicit in the sequence (L•,Q•).

Remark 7.6. For a symplectic sequence (L•,Q•), the invariants nj, ri, di to- gether with the dimensions dim(L ,K ) and dim(Qrh ∩K ) determine the sequence nj i dh i (L•,Q•) up to the action of the symplectic group. This will become obvious when we construct these sequences by choosing bases.

Deﬁnition 7.7. A symplectic sequence (L•,Q•) is admissible if it satisﬁes the following additional conditions:

(GA1) nj 6= dim(Lnj ∩ Ki) + 1 for any 1 ≤ j ≤ s and 1 ≤ i ≤ k − s.

(GA2) Let xi denote the number of isotropic subspaces Lnj that are contained in Ki. Then d − r x ≥ k − i + 1 − i i . i 2 The translation between sequences and symplectic diagrams. Symplectic sequences can be represented by symplectic diagrams introduced in §3. An isotropic linear space L is represented by a bracket ] in position n . A linear space Qri nj j di is represented by a brace } in position di such that there are exactly ri positive integers less than or equal to i to the left of the i-th brace. Finally, dim(Lnj ∩ Ki) 50 and dim(Qrh ∩K ), h > i, are recorded by the number of positive integers less than dh i or equal to i to the left of ]j and }h, respectively. 3 2 Example 7.8. 11]200}0}00 records a sequence L2 ⊂ Q5 ⊂ Q6, where L2 ⊂ 2 Ker(Q6). In the diagram, there is one bracket that occurs in position 2. There are two braces that occur in positions 5 and 6. We thus conclude that the sequence contains one isotropic subspace of dimension 2 (L2) and two non-isotropic subspaces of dimensions 5 (Q5) and 6 (Q6). There are two integers equal to 1 and one integer equal to 2 in the sequence. Hence, the corank of the restriction of Q to the six (re- 2 3 spectively, ﬁve) dimensional subspace Q6 (Q5) is two (three). Finally, since every 2 integer to the left of the bracket is equal to one, we conclude that L2 ⊂ Ker(Q6).

More explicitly, given a symplectic sequence (L•,Q•), the corresponding symplectic diagram D(L•,Q•) is determined as follows: The sequence of integers begins with dim(Ln1 ∩ K1) integers equal to 1, followed by dim(Ln1 ∩ Ki) − dim(Ln1 ∩ Ki−1) integers equal to i, for 2 ≤ i ≤ k − s, in increasing order, followed by n1 − dim(Ln1 ∩ Kk−s) integers equal to 0. The sequence then continues with dim(Lnj ∩ K1) − dim(Lnj−1 ∩ K1) integers equal to 1, followed by dim(Lnj ∩ Ki) − max(dim(Lnj−1 ∩ Ki), dim(Lnj ∩ Ki−1)) integers equal to i in increasing order, followed by nj −max(nj−1, dim(Lnj ∩Kk−s)) zeros for j = 2, . . . , s in increasing order. The sequence then continues with dim(Qrk−s ∩ K ) − dim(L ∩ K ) integers equal dk−s 1 ns 1 to 1, followed by dim(Qrk−s ∩ K ) − max(dim(Qrk−s ∩ K ), dim(L ∩ K )) inte- dk−s i dk−s i−1 ns i gers equal to i in increasing order, followed by zeros until position dk−s. Between positions d and d (i > k −s), the sequence has dim(Qri−1 ∩K )−dim(Qri ∩K ) i i−1 di−1 1 di 1 integers equal to 1, followed by dim(Qri−1 ∩K )−max(dim(Qri ∩K ), dim(Qri−1 ∩ di−1 h di h di−1 Kh−1)) integers equal to h in increasing order, for h ≤ i − 1, followed by zeros until position di−1. Finally, the sequence ends with n − d1 zeros. The brackets occur at positions nj and the braces occur at positions di.

Proposition 7.9. The diagram D(L•,Q•) is a symplectic diagram of type s for SG(k, n). Furthermore, if (L•,Q•) is admissible, then D(L•,Q•) is admissible.

Proof. By construction each bracket or brace occupies a position. Since n1 < n2 < ··· < ns < dk−s < ··· < d1, a position is occupied by at most one bracket or brace. Since nj < di for every 1 ≤ j ≤ s and 1 ≤ i ≤ k − s, every bracket occurs to the left of every brace. By construction, it is clear that dim(L ∩ K ) and dim(Qrh ∩ K ), nj i dh i for h ≥ i, are recorded by the number of positive integers less than or equal to i to the left of ]j and }h, respectively. Hence, every integer equal to i occurs to the left of }i. Finally, the total number of integers equal to zero or greater than i to the left of }i is equal to the rank of the restriction of Q to Qri . Since this rank di is necessarily even, the total number of integers equal to zero or greater than i to the left of }i is even. This shows that we have a sequence of brackets and braces of type s. The sequence of brackets and braces is a symplectic diagram. The corank bound implies that r − dim(Qri ∩ K ) ≤ d − d . The left hand side of the inequality i di i−1 i−1 i is represented by the number of integers equal to i in the sequence. The right hand side of the inequality is equal to the number of integers between }i and }i−1. We thus get the inequality l(i) ≤ ρ(i, i − 1) required by Condition (S1) in Definition 3.24. By the linear space bound, the largest dimensional linear space contained in Qri has dimension bounded by (d + r )/2. The invariant r is equal to both the di i i i 51 number of positive integers less than or equal to i contained to the left of }i and dim(Ki). The span of Lns and the kernels Kh for h ≥ i is an isotropic subspace of Qri . The dimension of this subspace is denoted by τ and is equal to the sum of di i s s i i p(] ) and the number of positive integers between ] and } . Hence, 2τi ≤ p(} ) + ri and condition (S2) of Definition 3.24 holds. If the sequence is in (perfect) order, then the corresponding sequence of brackets and braces is in (perfect) order. Assume the sequence is not in order. The definition of a sequence implies that, for i < k − s, there can be at most one i which is not to the left of }k−s. Suppose the sequence satisfies Condition (GS1). Then, there exists an integer η such that for i > η those integers that are not to the left of }k−s are to the immediate left of }i+1. Furthermore, condition (GS1) implies that the positive numbers up to η are in non-decreasing order and η is the only integer violating the order. Thus condition (S3) is satisfied. Finally, condition (GS2) directly translates to condition (S4). We conclude that the sequence of brackets and braces is a symplectic diagram.

If the sequence (L•,Q•) is admissible, then the corresponding symplectic diagram is also admissible. Let i be the minimal index such that Lnj ⊂ Ki. If there isn’t such an index, let i = k − s + 1. If i > 1, then condition (GA1) implies that j dim(Lnj ∩ Ki−1) ≤ nj − 2. Hence, the two integers preceding ] are equal to i (or 0 if i = k − s + 1). If i = 1, then all the integers preceding ]j are equal to 1. Furthermore, if nj = 1, condition (GA1) implies that Lnj ⊂ Ki for all 1 ≤ i ≤ k − s. We conclude that condition (A1) holds. The invariant xi is equal to both the number of isotropic subspaces Lnj contained in Ki and the number of brackets such that every integer to the left of it is positive and less than or equal i to i. Since di = p(} ), conditions (A2) and (GA2) are exactly the same. This concludes the proof of the proposition. Remark 7.10. Proposition 7.9 also explains the definition of a symplectic diagram in geometric terms. Condition (4) of Definition 3.12 is implied by the evenness of rank and simply states that di − ri has to be even. As discussed in the proof of Proposition 7.9, condition (S1) is a translation of the corank bound and condition (S2) is implied by the linear space bound. Conversely, we can associate an admissible sequence to every admissible symplectic diagram. By Darboux’s Theorem, we can take the skew-symmetric form Pm to be defined by i=1 xi ∧ yi. Let the dual basis for xi, yi be ei, fi such that j j xi(ej) = δi , yi(fj) = δi and xi(fj) = yi(ej) = 0. Given an admissible symplectic s diagram, we associate e1, . . . , ep(]s) to the integers to the left of ] in order. We s then associate ep(]s)+1, . . . , er0 to the positive integers to the right of ] and left of k−s } in order. Let ei1 , . . . , eil be vectors that have so far been associated to zeros. k−s Then associate fi1 , . . . , fil to the remaining zeros to the left of } in order. If there are any zeros to the left of }k−s that have not been assigned a basis vector, assign them er0+1, fr0+1, . . . , er00 , fr00 in pairs in order. Continuing this way, if there is a positive integer between }i+1 and }i associate to it the smallest index basis element eα that has not yet been assigned. Assume that the integers equal to i + 1 i+1 have been assigned the vectors ej1 , . . . , ejl . Assign to the zeros between } and i i+1 i } , the vectors fj1 , . . . , fjl . If there are any zeros between } and } that have not been assigned a vector, assign them eα+1, fα+1, . . . , eβ, fβ in pairs until the zeros are exhausted. Let Lnj be the span of the basis elements associated to the integers 52 to the left of ]j. Let Qri be the span of the basis elements associated to the integers di i to the left of } . We thus obtain a sequence (L•,Q•) whose associated symplectic diagram is D. Example 7.11. To 11]233]0000}00}0}00 we associate the sequence of vectors

e1, e2, e3, e4, e5, e6, f6, e7, f7, f4, f5, f3, f1, f2. 5 Then L2 is the span of e1, e2, L5 is the span of e1 through e5, Q9 is the span of e1 3 through e7 and f6, f7, Q11 is the span of e1 through e7 and f4 through f7. Finally, 2 3 Q12 is the span of Q11 and f3. To 22]33]0000}00}100}0 we associate the sequence of vectors

e1, e2, e3, e4, e5, f5, e6, f6, f3, f4, e7, f1, f2, f7. 4 L2 is the span of e1, e2, L4 is the span of e1 through e4, Q8 is the span of e1 2 4 1 2 through e6 and f5, f6, Q10 is the span of Q8 and f3, f4 and Q13 is the span of Q10 and e7, f1, f2. Finally, to 22]300]300}00}100}0 we associate the sequence of vectors

e1, e2, e3, e4, e5, e6, f4, f5, f3, f6, e7, f1, f2, f7. 4 Then L2 is the span of e1 and e2, L5 is the span of e1 through e5, Q8 is the span 2 4 1 of e1 through e5 and f4, f5, Q10 is the span of Q8 and f3, f6. Finally, Q13 is the span of all the vectors but f7. Remark 7.12. Notice that equivalent symplectic diagrams correspond to permu- tations of the basis elements that do not change the vector spaces in (L•,Q•).

Remark 7.13. The construction of a symplectic sequence (L•,Q•) from a symplectic diagram D is well-deﬁned. By condition (S2), the number of zeros to the left of ]s is less than or equal to the number of zeros between ]s and }k−s. Hence, we can choose vectors fi1 , . . . , fil corresponding to the vectors ei1 , . . . , eil . Similarly, if there does not exist a positive integer between }i+1 and }i, then by condition (S1), l(i+1) ≤ ρ(i+1, i). We can, therefore, associate vectors fj1 , . . . , fjl to the zeros between }i+1 and }i. If there exists a positive integer between }i+1 and }i, then there is only one positive integer between them by condition (S3). If l(i + 1) = ρ(i + 1, i), then condition (4) is violated. Hence, l(i + 1) < ρ(i + 1, i) and we can associate i+1 i vectors fj1 , . . . , fjl to the zeros between } and } . Thus the construction of the sequence makes sense. It is now straightforward to check that the sequence associated to an admissible symplectic diagram is an admissible sequence. Furthermore, the two constructions are inverses of each other. We are now ready to deﬁne symplectic restriction varieties.

Deﬁnition 7.14. Let (L•,Q•) be an admissible sequence for SG(k, n). Then the symplectic restriction variety V (L•,Q•) is the Zariski closure of the locus in SG(k, n) parameterizing {W ∈ SG(k, n) | dim(W ∩ L ) = j for 1 ≤ j ≤ s, dim(W ∩ Qri ) = k − i + 1 nj di and dim(W ∩ Ki) = xi for 1 ≤ i ≤ k − s}. Remark 7.15. The geometric reasons for imposing conditions (A1) and (A2) in Deﬁnition 3.27 are now clear. Condition (A1) is an immediate consequence of the kernel bound. If dim(Lnj ∩ Ki) = nj − 1 and a linear space of dimension k − i + 1 53 intersects nj in dimension j and Ki in dimension j − 1, then the linear space is contained in L⊥ . Hence, we need to impose condition (A1). nj The inequality d − r x ≥ k − i + 1 − i i i 2 is an easy consequence of the linear space bound. We require the k-dimensional isotropic subspaces to intersect Qri in a subspace of dimension k − i + 1 and to di intersect the singular locus of Qri in a subspace of dimension x . By the linear di i space bound, any linear space of dimension k − i + 1 has to intersect the singular di−ri locus in a subspace of dimension at least k − i + 1 − 2 , hence the inequality in condition (A2) holds. Example 7.16. The two most basic examples of symplectic restriction varieties are:

(1) A Schubert variety Σλ;µ in SG(k, n), which is the restriction variety associated to a symplectic diagram D(σλ;µ), and

(2) The intersection Σa• ∩ SG(k, n) of a general Schubert variety in G(k, n) with SG(k, n), which is the restriction variety associated to D(a•). In general, symplectic restriction varieties interpolate between these two examples. Lemma 7.17. A symplectic restriction variety corresponding to a saturated and perfectly ordered admissible sequence is a Schubert variety in SG(k, n). Conversely, every Schubert variety in SG(k, n) can be represented by such a sequence. ⊥ Proof. Let F1 ⊂ · · · ⊂ F1 ⊂ V be an isotropic flag. If Σλ,µ is a Schubert variety defined with respect to this flag, then the symplectic restriction variety defined with respect to the sequence L = F and Qri = F ⊥ is a saturated and perfectly nj λj di µk−i+1 ordered admissible sequence. Conversely, suppose that the sequence (L•,Q•) is a saturated and perfectly ordered admissible sequence. Since the sequence is saturated, we have that Qri = di K⊥. Since the sequence is in perfect order, we have that dim(L ∩ Ker(Qri )) = i nj di min(r , n ). Consequently, the set of linear spaces {L , Ker(Qri )} can be ordered i j nj di by inclusion, or equivalently, by dimension. Then the resulting partial flag can be extended to an isotropic flag. By condition (GA1) of the definition of an admissible sequence, we have that nj 6= ri + 1 for any i, j. Hence, the symplectic restriction variety defined with respect to (L•,Q•) is the Schubert variety Σλ•;µ• , where λj = nj, for 1 ≤ j ≤ s, and µi = rk−i+1, for s < i ≤ k. Remark 7.18. By Lemma 7.17, the saturated symplectic diagrams in perfect order represent Schubert varieties. Next, we show that the intersection of a general Schubert variety Σ with the symplectic Grassmannian SG(k, n) (when non-empty) is a restriction variety. Lemma 7.19. Let Σ be the Schubert variety defined with respect to a general partial

ﬂag Fa1 ⊂ · · · ⊂ Fak . Then Σ∩SG(k, n) 6= ∅ if and only if ai ≥ 2i−1 for 1 ≤ i ≤ k.

Proof. Suppose ai < 2i − 1 for some i. If [W ] ∈ Σ ∩ SG(k, n), then W ∩ Fai is an isotropic subspace of Q ∩ Fai of dimension at least i. Since Fai is general, the corank of Q ∩ Fai is 0 or 1 and equal to ai modulo 2. By the linear space bound, the largest dimensional isotropic subspace of Q ∩ Fai has dimension less than or equal to i − 1. Therefore, W cannot exist and Σ ∩ SG(k, n) = ∅. 54 ⊥ Conversely, let ai = 2i − 1 for every i. Then G1 = F1 is isotropic, G2 = F1 in F3 is the unique two-dimensional isotropic subspace of Q ∩ F3 containing G1. By ⊥ induction, we see that Gi = Gi−1 is the unique subspace of dimension i isotropic with respect to Q ∩ F2i−1 that contains Gi−1. Continuing this way, we construct a unique isotropic subspace W of dimension k contained in Σ∩SG(k, n). If ai ≥ 2i−1, the vector space W just constructed is still contained in Σ ∩ SG(k, n), hence this intersection is non-empty. Lemma 7.20. Let Σ be the Schubert variety deﬁned with respect to a general partial

ﬂag Fa1 ⊂ · · · ⊂ Fak such that ai ≥ 2i − 1. Then Σ ∩ SG(k, n) = V (D(a•)).

Proof. Let ai = 2i − 1, then since Fai is general, the restriction of Q to Fai has a one-dimensional kernel Ki. By the linear space bound, any i-dimensional isotropic subspace W contained in Fai contains Ki. For each j such that aj > 2j − 1, recall that uj is the number of i < j such that ai = 2i − 1 and vj is the number of i > j such that ai = 2i − 1. Let K be the span of one-dimensional kernels Ki for each ai = 2i − 1. Then dim(K) = u and any k-dimensional subspace W contained in Σ ∩ SG(k, n) contains K. For j such that aj > 2j − 1, let Gj+vj = ⊥ Span(Faj ,K) ∩ K . The dimension of Gj+vj is aj − uj + vj. The corank of the restriction of Q to Gj+vj is u + δ(aj), where δ(aj) = 0(1) if aj is even (odd).

Furthermore, any isotropic linear space contained in Σ ∩ SG(k, n) intersects Gj+vj in a subspace of dimension at least j + vj. From this description and the deﬁnition of V (D(a•)), it is now clear that Σ ∩ SG(k, n) = V (D(a•)).

Proposition 7.21. Let (L•,Q•) be an admissible sequence. Then V (L•,Q•) is an irreducible subvariety of SG(k, n) of dimension s k−s X X dim(V (L•,Q•)) = (nj − j) + (di − 1 − 2k + 2i + xi). (2) j=1 i=1 Proof. The proof is by induction on k. When k = 1, if the sequence consists of an isotropic linear space Ln1 , then the corresponding symplectic restriction variety is

PLn1 hence it is irreducible of dimension n1 − 1. If the sequence consists of one non-isotropic subspace Qr1 , then the corresponding symplectic restriction variety is d1 also projective space of dimension d1 −1. In both cases, the varieties are irreducible of the claimed dimension. This proves the base case of the induction. If the sequence does not contain any skew-symmetric forms, then the corresponding restriction variety is isomorphic to a Schubert variety in the ordinary Grassmannian G(k, n). In that case, it is well known that Schubert varieties are Pk irreducible and have dimension j=1(nj − j)[?]. Observe that omitting Qr1 from an admissible sequence (L ,Q ) for SG(k, n) d1 • • 0 0 gives rise to an admissible sequence (L•,Q•) for SG(k − 1, n). There is a natural 0 0 0 0 surjective morphism f : V (L•,Q•) → V (L•,Q•) that sends a vector space W to W ∩ Qr2 (or W ∩ L if s = k − 1). By induction, V (L0 ,Q0 ) is irreducible of d2 nk−1 • • Ps Pk−s dimension j=1(nj − j) + i=2 (di − 1 − 2k + 2i + xi). The fibers of the morphism f over a point W 0 correspond to choices of isotropic k-planes W that contain W 0 and are contained in Qr1 . This is a Zariski dense open subset of projective space d1 of dimension d1 − 2(k − 1) − 1 + x1. Hence, by the Theorem on the Dimension of Fibers [S, I.6.7], V (L•,Q•) is irreducible of the claimed dimension. This concludes the proof of the proposition. 55 8. The geometric explanation of the combinatorial game In this section, we will prove the combinatorial rule by interpreting it geometrically. The transformation from an admissible diagram D to Da records a one- parameter specialization of the restriction variety V (D). The algorithm describes the flat limit of this specialization. The specialization. We now explain the specialization. There are several cases depending on whether D is in order and whether l(κ) < ρ(κ, κ − 1) − 1 or not. In the previous section, given an admissible quadric diagram D, we associated an admissible sequence by defining each of the vector spaces (L•,Q•) as a union of basis elements that diagonalize the skew-symmetric form Q. All our specializations will replace exactly one of the basis elements v = eu or v = fu for some 1 ≤ u ≤ m with a vector v(t) = eu(t) or v(t) = fu(t) varying in a one-parameter family. For t 6= 0, the resulting set of vectors will be a new basis for V , but when t = 0 two of the basis elements will become equal. Since each linear space in (L•,Q•) is a union of basis elements, we get a one-parameter family of vector spaces (L•(t),Q•(t)) by replacing every occurrence of the vector v with v(t) for t 6= 0. Correspondingly, we have a one-parameter family of restriction varieties V (L•(t),Q•(t)). Since these varieties are projectively equivalent as long as t 6= 0, we obtain a flat one-parameter family. Our task is to describe the limit when t = 0. In case (1)(i), D is not in order, η is the unique integer violating the order, and ν is the leftmost integer equal to η +1. Suppose that under the translation between symplectic diagrams and sequences of vector spaces, eu is the vector associated to η and ev is the vector associated to ν. Then consider the one-parameter family obtained by changing ev to ev(t) = tev + (1 − t)eu and keeping every other vector fixed. When the set of basis elements spanning a vector space L or Qri contains nj di e , L (t) or Qri (t) is the span of the same basis elements except that e is replaced v nj di v with e (t). Otherwise, L (t) = L or Qri (t) = Qri . v nj nj di di In case (1)(ii), D is not in order, η is the unique integer violating the order, i > η does not occur in the sequence to the left of η and ν is the leftmost integer equal to i + 1. Let eu be the vector associated to η and let ev be the vector associated to ν. Consider the one-parameter family obtained by changing fv to fv(t) = tfv + (1 − t)eu. When the set of basis elements spanning a vector space Lnj or Qri contains f , L (t) or Qri (t) is the span of the same basis elements except di v nj di that f is replaced with f (t). Otherwise, L (t) = L or Qri (t) = Qri . v v nj nj di di

In case (2)(i), D is in order and l(κ) < ρ(κ, κ − 1) − 1. Suppose that ev is the vector associated to ν, the leftmost κ + 1. Let eu and fu be two vectors associated to zeros between }κ and }κ−1. These exist since l(κ) < ρ(κ, κ−1)−1. Consider the one-parameter specialization replacing fv with fv(t) = tfv +(1−t)eu. When the set of basis elements spanning a vector space L or Qri contains f , L (t) or Qri (t) nj di v nj di is obtained by replacing f with f (t). Otherwise, L (t) = L or Qri (t) = Qri . v v nj nj di di In case (2)(ii)(a), D is in order and l(κ) = ρ(κ, κ − 1) − 1. Let ν be the leftmost integer equal to κ and suppose that ev is the vector associated to ν. Let eu be κ the vector associated to the κ − 1 following } . Then let ev(t) = tev + (1 − t)eu. When the set of basis elements spanning a vector space L or Qri contains e , nj di v L (t) or Qri (t) is obtained by replacing e with e (t). Otherwise, L (t) = L or nj di v v nj nj Qri (t) = Qri . di di 56 Finally, in case (2)(ii)(b), D is in order, l(κ) = ρ(κ, κ − 1) − 1 and there does not exist an integer equal to κ to the left of κ. Let ev be the vector associated to ν, the leftmost integer equal to κ + 1 and let eu be the vector associated to κ − 1 κ to the right of } . Then let fv(t) = tfv + (1 − t)eu. When the set of basis elements spanning a vector space L or Qri contains f , L (t) or Qri (t) is obtained by nj di v nj di replacing f with f (t). Otherwise, L (t) = L or Qri (t) = Qri . v v nj nj di di The flat limits of the vector spaces are easy to describe. If L or Qri does not nj di contain the vector v, then L (t) = L and Qri (t) = Qri for all t 6= 0. Hence, the nj nj di di flat limit L (0) = L and Qri (0) = Qri . Similarly, if L or Qri contains both of nj nj di di nj di the basis elements spanning v(t), then L (t) = L and Qri (t) = Qri for all t 6= 0. nj nj di di Then in the limit L (0) = L and Qri (0) = Qri . A vector space changes under nj nj di di the specialization only when it contains the vector with coefficient t and does not contain the vector with coefficient (1 − t). In this case, in the limit t = 0, the flat limit L (0) or Qri (0) is obtained by replacing in L or Q the basis element with nj di nj di coefficient t with the basis element with coefficient (1 − t). Notice that in each of these cases, the set of limiting vector spaces is depicted by the symplectic diagram Da. In case (1)(i), if η is between }a and }a−1 and ν b b+1 s k−s is between ] and ] (respectively, between ] and } ), the vector spaces Lnj for j ≤ b (respectively, j ≤ s) and Qri for i < a are unaffected. In all the other di vector spaces, ev is replaced by eu. The effect on symplectic diagrams is to switch η and ν as in the definition of Da. In case (1)(ii), assume that η is between }i and }i−1. The linear spaces other than Qri remain unchanged under the degeneration. di In Qri the vector f is replaced by e . Note that this increases the corank of di v u the restriction of Q to Qri (0) by two since now both vectors e and e in the di u v kernel. This has the effect of changing ν to i and a zero between }i+1 and }i to η as in the definition of Da. In case (2)(i), all the vector spaces but Qrκ remain dκ unchanged. The degeneration replaces f in Qrκ by e . This increases the corank v dκ u of the restriction of Q to Qrκ (0) by two since both e and e are now contained in dκ u v the kernel of the restriction. The corresponding symplectic diagram is obtained by changing ν and a zero between }κ+1 and }κ to κ as in the definition of Da. The cases (2)(ii)(a) and (b) are analogous to the cases (1)(i) and (1)(ii), respectively. For the rest of the paper, we use the specialization just described. Example 8.1. For concreteness, consider the restriction variety associated to 200}000}00 in SG(2, 8) parameterizing isotropic subspaces that intersect A = Span(e1, e2, f2) and are contained in B = Span(ei, fi), 1 ≤ i ≤ 3. The first specialization is given by tf2 + (1 − t)e3. In the limit, A1 = A(0) = Span(e1, e2, e3) and B(0) = B. This changes the diagram to 000]000}00. The corresponding restriction variety parameterizes linear spaces that intersect A(0) and are contained in B. The next specialization is given by tf1 + (1 − t)e4. In the limit, A1(0) = A1 and B1 = B(0) = Span(e1, e2, e3, e4, f2, f3). This changes the diagram to 100]100}00. The corresponding restriction variety parameterizes linear spaces that intersect A1 and are contained in B1. The final specialization is given by te2 + (1 − t)e4. In the limit, A2 = A1(0) = Span(e1, e4, e3) and B1(0) = B1. This changes the diagram to 110]000}00. The flat limit of the restriction varieties has two components. The linear spaces may intersect Span(e1, e4), in which case we get the restriction variety associated to the diagram 11]0000}00. Otherwise, by the kernel bound, the linear ⊥ spaces have to be contained in A2 . In this case, we get the restriction variety 57 associated to the diagram 111]00}000. The reader should convince themselves that this is precisely the outcome of Algorithm 3.39. We are now ready to state and prove the main geometric theorem. Theorem 8.2. (The Geometric Branching Rule) The flat limit of the specialization S of V (D) is supported along V (Di), where V (Di) is a symplectic restriction variety associated to a diagram Di obtained by running Algorithm 3.39 on D. Furthermore, the flat limit is generically reduced along each V (Di). In particular, the equality X [V (D)] = [V (Di)] holds between the cohomology classes of symplectic restriction varieties. proof of theorem 3.43 assuming theorem 8.2. By Proposition 3.49, Algorithm 3.39 replaces each admissible symplectic diagram by one or two admissible symplectic diagrams. Hence, the algorithm can be repeated. By Proposition 3.50, after finitely many steps, the algorithm terminates leading to a collection of saturated admissible symplectic diagrams in perfect order. By Lemma 7.17, each of these diagrams represent a Schubert variety. Therefore, Theorem 3.43 is an immediate corollary of Theorem 8.2. Proof of Theorem 8.2. The proof of Theorem 8.2 has two steps. First, we interpret the algorithm as the specialization described in the beginning of this section. Let V (D) denote the initial symplectic restriction variety. Let V (D(t)) denote the one-parameter family of restriction varieties described in the specialization and let V (D(0)) be the flat limit at t = 0. We show that V (D(0)) is supported along the union of restriction varieties V (Di), where Di are the admissible symplectic diagrams derived from D via Algorithm 3.39. In the second step, we verify that the support of the flat limit contains each V (Di) and the flat limit is generically reduced along each V (Di). This suffices to prove the theorem. We now analyze the specialization to conclude that the support of V (D(0)) is the union of symplectic restriction varieties V (Di). The proof is by a dimension count. In order to restrict the possible irreducible components of V (D(0)), we find conditions that the linear spaces parameterized by V (D(0)) have to satisfy. We then observe that these conditions already cut out the symplectic varieties V (Di) and that each V (Di) has the same dimension as V (D). The following observation puts strong restrictions on the support of the flat limit.

Observation 8.3. The linear spaces parameterized by V (D(t)) intersect the linear spaces L (t) (respectively, Qri (t)) in a subspace of dimension at least j (respec- nj di tively, k −i+1). Similarly, they intersect Ker(Qri (t)) in a linear space of dimension di at least xi. Since intersecting a proper variety in at least a given dimension is a closed condition, the linear spaces parameterized by V (D(0)) have to intersect the linear spaces L (0) (respectively, Qri (0)) in a subspace of dimension at least j nj di (respectively, k − i + 1). Furthermore, they intersect Ker(Qri (0)) in a subspace of di dimension at least xi. Let Y be an irreducible component of V (D(0)). We can construct a sequence of vector spaces Fu1 ⊂ · · · ⊂ Fuk such that the locus Z parameterizing linear spaces with dim(W ∩ Fuj ) ≥ j contains Y . We have already seen that the linear spaces L (0) and Qri (0) are the linear spaces recorded by the symplectic diagram Da. nj di 58 Let z1, . . . , zn be the ordered basis of V obtained by listing the basis elements a associated to D from left to right. Let Fu be the linear space spanned by the basis elements z1, ··· , zu. Let Fu1 ⊂ · · · ⊂ Fuk be the jumping linear spaces for Y , that is the linear spaces of the form Fu such that dim(W ∩ Fu) > dim(W ∩ Fu−1) for the general isotropic space W parameterized by Y . Observation 8.3 translates to the inequalities uj ≤ nj for j ≤ s and ui ≤ dk−i+1 for s < i ≤ k. Hence, we can obtain a sequence depicting the linear spaces Fu1 ,...,Fuk by moving the braces and brackets in the diagram Da to the left one at a time. By the proof of Proposition 7.21, Equation (2) gives an upper bound on the dimension of the locus Z (note that we used the fact that the sequence is admissible in the proof only to deduce the equality).

a a We now estimate the dimension of Z. Let (L•,Q•) denote the linear spaces depicted by the diagram Da. We obtain the sequence deﬁning Z by replacing a a linear spaces in (L•,Q•) by smaller dimensional ones. • If we replace a linear space La of dimension n in (La,Qa) with a linear ni i • • space F not contained in (La,Qa) but containing La , then according ui • • ni−1 a to Equation (2) the dimension changes as follows. Let yi be the index of the smallest index linear space Qrl such that La ⊂ K . Similarly, let yu be dl ni l i

the smallest l such that Fui ⊂ Kl. The left sum in Equation (2) changes by a u a ui − ni . The quantities xl increase by one for yi ≤ l < yi . Hence, the sum a u on the right increases by yi − yi . Hence, the total change in dimension is a a u a ui −ni +yi −yi . By condition (S4) of Deﬁnition 3.24 for D and condition (A1) for D, in Da, there is at most one missing integer among the positive integers to the left of the brackets and the two integers preceding all brackets but possibly ]xν−1+1 are equal. We conclude that if we move any bracket to the left except for ]xν−1+1, we strictly decrease the dimension. Furthermore, if we move ]xν−1+1 to the left, we strictly decrease the dimension unless in xν−1+1 D we have the equality p(] ) − π(ν) − 1 = yxν−1+1 − ν, so that the decrease in the position resulting by shifting the bracket in Da is equal to the increase in the number of linear spaces Qrl containing F in their dl ui kernel. • If we replace the linear space Qri,a of dimension da in (La,Qa) with a non- di i • • isotropic linear space F of dimension du containing Qri−1,a, then, by uk−i+1 i di−1 u Equation (2), the dimension changes as follows. Let xi be the number of linear spaces that are contained in the kernel of the restriction of Q to u a a u Fuk−i+1 . Then the dimension changes by di − di − xi + xi . We have that u a a u di − di − xi + xi ≤ 0 with strict inequality unless the number of linear

spaces contained in the kernel of Fuk−i+1 increases by an amount equal to a u di − di . The latter can only happen if condition (A1) is violated for the diagram so that increasing the dimension of the kernel by one can increase the number of linear spaces contained in the kernel. • Finally, if we replace the linear space Qrk−s,a of dimension da in (La,Qa) dk−s k−s • •

with an isotropic linear space Fus+1 containing Lns , then the ﬁrst sum in Equation (2) changes by us+1 − s − 1. The second sum changes by a u a u −dk−s + ys+1 − xk−s + (2s + 1), where ys+1 denotes the number of non-

isotropic subspaces containing Fus+1 in the kernel of the restriction of Q. 59 Hence, the total change is

a a u −dk−s + us+1 − xk−s + ys+1 + s. a u If xk−s = s − j < s, then ys+1 = 0. Since by the linear space bound us+1 + j + 1 ≤ dk−s, we conclude that the dimension strictly decreases. If xk−s = s, then the change is strictly negative unless rk−s = dk−s and dk−s = us+1. The dimension count shows that V (D) and V (Da) have the same dimension. xν−1+1 b a When p(] ) − π(ν) − 1 = yxν−1+1 − ν in D, V (D ) and V (D ) have the same dimension. Furthermore, Step 2 of Algorithm 3.35 and Algorithm 3.36 preserve the dimension of the variety. By Equation (2), Step 1 of Algorithm 3.35 also preserves the dimension. If condition (A2) is violated for Da for the index i, then by Proposition 3.49, we have that 2xi = 2k − 2i − di + ri. On the other hand, the operation in Step 1 of Algorithm 3.35 changes the left sum in Equation (2) by ri + (s − xi) − s − 1 = ri − xi − 1, since it adds a new bracket of size ri and increases the positions of the brackets with index xi + 1, . . . , s. It changes the left sum by −di + 1 − xi + 2k − 2(k − s) + 2(k − s − i) since it removes the brace with index i and increases the positions and xl for the braces with indices l = i + 1, . . . , k − s. We conclude that the change in dimension is ri − 2xi − di + 2k − 2i = 0. We conclude that every variety V (Di) associated to V (D) by Algorithm 3.39 has the same dimension as V (D). We can now determine the support of the flat limit of the specialization. Since in flat families the dimension of the fibers are preserved, Y has the same dimension as V (D). Hence, our dimension calculation puts very strong restrictions on Z. First, xν−1+1 a suppose that either xν−1 = s or p(} ) − π(ν) − 1 > yxν−1+1 − ν in D. If D is admissible, then by our dimension counts, replacing an isotropic or non-isotropic a a linear space in (L•,Q•) with a smaller dimensional linear space produces a strictly smaller dimensional locus. We conclude that the general linear space parameter- a a ized by Y satisfies exactly the rank conditions imposed by (L•,Q•). Hence, Y is contained in V (Da). Since both are irreducible varieties of the same dimension, we conclude that Y = V (Da). If Da is not admissible, then it either violates condition a di−ri (A1) or (A2). If D fails condition (A2), then xi < k − i + 1 − 2 for some i. Since the linear spaces parameterized by Y have to intersect Qri in a subspace of di dimension k −i+1, by the linear space bound, we conclude that these linear spaces a have to intersect Ki in a subspace of dimension at least xi + 1. In D , there is only one integer i that is not in the beginning non-decreasing part of the sequence of integers. Geometrically, the linear spaces La or Qrj ,a either contain or are contained nj dj in Ki or intersect Ki in a codimension one linear space. Let Fa1 ⊂ Fa2 ⊂ · · · ⊂ Fal be a partial flag such that Fah intersects M in a codimension one subspace of M.

Let M = Ga0+1 ⊂ Ga1+1 ⊂ · · · ⊂ Gal+1 be the partial ﬂag where Gah+1 is the span of Fah and M for h ≥ 1. The locus of linear spaces of dimension xi + l + 1 that intersect Fah in a subspace of dimension at least xi + h and intersect M in a subspace of dimension at least xi + 1 is equivalent to the locus of linear spaces that intersect the vector spaces Gah+1 in subspaces of dimension at least xi +1+h. Notice that the diagram Dc formed in Step 1 of the Algorithm 3.35 depicts the linear spaces

L ,...,L ,K , Span(K ,L ), ··· , Span(K ,Qri+1 ),Qri−1 , ··· ,Qr1 . n1 nxi i i nxi+1 i di+1 di−1 d1 60 Hence, by the linear space bound Y must be contained in V (Dc). By Proposition 3.49, Dc is an admissible symplectic diagram. Hence, V (Dc) is an irreducible variety that has the same dimension as Y . We conclude that Y = V (Dc). On the other hand, if Da satisﬁes condition (A2) but fails condition (A1), then it fails it for the bracket with index xν−1 + 1 and the index ν. By the kernel bound, any linear space that intersects L in a subspace away from the kernel of Q restricted nxν−1+1 to Qrν−1 has to be contained in L⊥ . The latter vector space is depicted in a dν−1 nxν−1+1 symplectic diagram by changing ν to ν − 1 and shifting }ν−1 one unit to the right as in Step 2 of Algorithm 3.35. This argument applies as long as condition (A1) fails for the resulting sequence. We conclude that Y has to be contained in V (Dc). Since Y and V (Dc) are irreducible varieties of the same dimension, we conclude that Y = V (Dc).

xν−1+1 Now suppose that xν−1 < s and p(} ) − π(ν) − 1 = yxν−1+1 − ν in D. Then, by our dimension count, replacing the linear space L by a linear nxν−1+1 space F corresponding to a bracket of the form uxν−1+1 ··· a a + 1 . . . ν − 1 ν ν + 2 . . . ν + l − 1 ν + l ν + l] · · · →

··· a]a + 1 . . . ν − 1 ν ν + 2 . . . ν + l − 1 ν + l ν + l ··· produces a locus Z that has the same dimension as Y . Replacing any other linear space results in a smaller dimensional locus. However, unless F = uxν−1+1 Ker(Qrν ) ∩ L not all linear spaces parameterized by Z can be in the flat dν nxν−1+1 limit. Observe that W ⊥(t) intersects L ∩ Ker(Qra ) in a subspace of dimen- nxν−1+1 da sion at least π(a) + 1 for every W (t) ∈ V (D(t)). By upper semi-continuity, the same has to hold of the flat limit at t = 0. Hence, unless a = ν, we obtain a smaller dimensional variety. We conclude that Y ⊂ V (Db). If Db is admissible, then both varieties are irreducible of the same dimension and we conclude that Y = V (Db). If Db is not admissible, then by Proposition 3.49, Db satisfies condition (A2) but fails condition (A1). Furthermore, it fails condition (A1) only for the bracket ··· a ν] ··· . By the kernel bound, the linear spaces parameterized of dimensions k − a, k − a + 1, . . . , k − ν + 2 contained in Qra+1 ,...,Qrν−1 , respec- da+1 dν−1 tively, are contained in (L ∩ Ker(Qra ))⊥ in Qra+1 ,...,Qrν−1 . Algorithm nxν−1+1 da da+1 dν−1 3.36 replaces the linear spaces Qra+1 ,...,Qrν−1 with (L ∩ Ker(Qra ))⊥ in da+1 dν−1 nxν−1+1 da Qra+1 ,...,Qrν−1 , respectively. Hence, Y is contained in V (Dc). Finally, if during da+1 dν−1 the process two braces occupy the same position, then the resulting locus Z has strictly smaller dimension by our dimension counts so does not lead to a locus Z containing Y . Since in all other cases Y and V (Dc) are irreducible varieties of the same dimension, we conclude that Y = V (Dc). This completes the proof that the support of the flat limit of the specialization is contained in the union of V (Di), where Di are the admissible symplectic diagrams associated to D by Algorithm 3.39. Finally, there remains to check that each of the irreducible components occur with multiplicity one. This is an easy local calculation. The point here is that taking the option Da at each stage of the algorithm leads to a Schubert variety. Similarly, taking the option Db at all allowed places in the algorithm leads to a Schubert variety. The classes of these two Schubert varieties occur in the class of 61 V (D) with multiplicity one. Therefore, by intersecting V (D) with the dual of these Schubert varieties, we can tell the multiplicity of V (Da) and V (Db). First, in each of the five cases we can assume that η = 1. Let U be the Zariski open set of our family of restriction varieties parameterizing linear spaces W (t) such that dim(W (t) ∩ Qrη (t)(t)) = k − η + 1. Let Z be the family of symplectic restric- dη tion varieties obtained by applying the specialization to the admissible sequence 0 0 0 (L•,Q•) (represented by D ) obtained from (L•,Q•) by omitting the linear spaces r Qr1 ,...,Q η−1 . Then there exists a natural morphism f : U → Z sending W (t) d1 dη−1 to W (t) ∩ Qrη (t)(t), which is smooth at the generic point of each of the irreducible dη components of the fiber of Z at t = 0. The fibers f over W 0 ∈ Z is the linear spaces of dimension k that contain W 0 and satisfy the appropriate rank conditions with r respect to the linear spaces Qr1 ,...,Q η−1 . Notice that running Algorithm 3.39 d1 dη−1 on D0 results in the same outcome as running in D and removing the braces with indices i < η. Hence, we can do the multiplicity calculation for the family Z. We may, therefore, assume that η = 1. In all the cases, the argument is almost identical with very minor variations. We will give it in the hardest case, case (2)(i), and leave the minor modifications in the other cases to the reader. In case (2)(i), by a similar argument, we may further assume that κ = 1, dκ + rκ = n − 2, xκ = 0 and s ≤ 1. The most interesting case is when s = 1 and 2dk−s ≥ n. Let y1 be the minimal index l such that Ln1 is contained in Ker(Qrl ). We will check that the multiplicities are one by finding a dl cycle that intersects V (D) in one point and exactly one of the limits in one point. If Da is admissible, then consider the Schubert variety Σ defined with respect to a general isotropic flag with the following invariants

λi = n − di + 2 for κ = 1 ≤ i ≤ l − 1, λi = n − di + 1 for l ≤ i ≤ k − 1,

and µk = n − n1 + 1. a If D satisfies condition (A2) but not (A1), change the definition of λ1 so that a λ1 = n − d1 + 2. If D fails condition (A2), change the definition of Σ so that

λi = n − di+1 + 1 for 1 ≤ i ≤ l − 2, λi = n − di+1 for l − 1 ≤ i ≤ k − 2,

and µk−1 = n − n1, µk = n − rκ + 1. By Kleiman’s Transversality Theorem [Kl1], it is immediate that both Σ ∩ V (D) and Σ∩V (Da) consist of a single reduced point, whereas Σ∩V (Db) is empty. Since Σ requires the k-plane to be contained in a linear space of dimension n − n1 + 1 and V (Db) requires the linear space to intersect a linear space of dimension less than n1, these conditions cannot be simultaneously satisfied for general choices of linear spaces. Hence, Σ ∩ V (Db) is empty. On the other hand, the intersection L ∩ F ⊥ consists of a one-dimensional vector space W and Qri ∩ F consist n1 µk 1 di λi ⊥ of one-dimensional linear spaces contained in W1 when l ≤ i ≤ k − 1 and two- ⊥ dimensional linear spaces not contained in W1 when 1 ≤ i ≤ l − 1. Since any linear space contained in V (D) ∩ Σ or V (Da) ∩ Σ must intersect all these linear spaces in one-dimensional subspaces, we conclude that the k-dimensional linear space satisfying conditions imposed by V (D) and Σ or V (Da) and Σ are uniquely determined. It follows that the multiplicity of V (Da) is one. 1 b Similarly, if p(] ) − π(2) − 1 = y1 − 2, then D is admissible. Let Ω be the Schubert variety defined with respect to a general isotropic flag with the following 62 invariants:

λi = n − di + 1 for 1 ≤ i ≤ k − 1, µk = r1. By Kleiman’s Transversality Theorem [Kl1], it is immediate that both Ω ∩ V (D) and Ω ∩ V (Db) consist of a single reduced point, whereas Ω ∩ V (Da) is empty. The conditions imposed by Ω and V (Da) cannot be simultaneously satisﬁed, hence Ω ∩ V (Da) is empty. On the other hand, F ∩ Qri by construction are one- λi di dimensional subspaces that need to be contained in any W contained in Ω ∩ V (D) or Ω ∩ V (Db). These determine (k − 1)-dimensional subspace W 0 of W . Lb ∩ F ⊥ n1 µ1 is also a one-dimensinonal subspace Λ that needs to be contained in W . Since Λ ⊂ (W 0)⊥, this uniquely constructs W ∈ Ω ∩ V (Db). Similarly, L ∩ F ⊥ is n1 µ1 a y1-dimensional linear space. However, the intersection of this linear space with (W 0)⊥ is one-dimensional and must be contained in W . This uniquely constructs W in V (D) ∩ Ω. We leave the minor modiﬁcations necessary in the other cases to the reader (see [C3] for more details in the orthogonal case). This concludes the proof of the theorem.

References [Be] E. Bertini. Introduzione alla geometria proiettiva degli iperspazi Enrico Spoerri, Pisa, 1907, MR1548022, JFM 49.0484.08. [BL] S. Billey and V. Lakshmibai. Singular loci of Schubert varieties. Birkhäuser,Boston, MA, 2000, MR1782635, Zbl 0959.14032. [BH] A. Borel and A. Haefliger, La classe d’homologie fondamental d’un espace analytique, Bull. Soc. Math. France, 89 (1961), 461–513. [Br] R. Bryant. Rigidity and quasi-rigidity of extremal cycles in Hermitian symmetric spaces. Princeton University Press Ann. Math. Studies AM–153, 2005. [C1] I. Coskun, Rigid and non-smoothable Schubert classes, J. Differential Geom., 87 no. 3 (2011), 493–514. [C2] I. Coskun, Rigidity of Schubert classes in orthogonal Grassmannians, to appear Israel J. Math. [C3] I. Coskun, Restriction varieties and geometric branching rules, Adv. Math., 228 no. 4 (2011), 2441–2502. [C4] I. Coskun, Symplectic restriction varieties and geometric branching rules, to appear in Clay Proceedings. [C5] I. Coskun, Degenerations of surface scrolls and Gromov-Witten invariants of Grassmannians, J. Algebraic Geom., 15 (2006), 223–284. [D] Del Pezzo. Sulle superficie di ordine n immerse nello spazio di n + 1 dimensioni, Nap. rend., XXIV, 1885, 212–216, JFM 19.0841.01. [EH] D. Eisenbud, and J. Harris. On varieties of minimal degree (a centennial account) Algebraic geometry, Bowdoin 1985 Proc. Sympos. Pure Math. 46 no. 1, Amer. Math. Soc., Providence, RI, 1987, MR0927946, Zbl 0646.14036. [EH] David Eisenbud and Joe Harris. The geometry of schemes, volume 197 of Graduate Texts in Mathematics. Springer-Verlag, New York, 2000. [Ful1] W. Fulton. Young tableaux, volume 35 of London Mathematical Society Student Texts. Cambridge University Press, Cambridge, 1997. [Ful2] W. Fulton. Intersection theory, volume 2 of Ergebnisse der Mathematik und ihrer Grenzge- biete. 3. Folge. A Series of Modern Surveys in Mathematics. Springer-Verlag, Berlin, second edition, 1998. [GH] P. Griffiths and J. Harris. Principles of Algebraic Geometry. Wiley Interscience, 1978. [H] J. Harris. Algebraic geometry, volume 133 of Graduate Texts in Mathematics. Springer-Verlag, New York, 1995. [HM] J. Harris and I. Morrison. Moduli of curves. Springer-Verlag, 1998. [Ha] R. Hartshorne. Algebraic geometry. Springer-Verlag, New York, 1977. Graduate Texts in Mathematics, No. 52. 63 [HRT] R. Hartshorne, E. Rees, and E. Thomas. Nonsmoothing of algebraic cycles on Grassmann varieties. Bull. Amer. Math. Soc. 80 no. 5 (1974), 847–851, MR0357402, Zbl 0289.14011. [Hi] H. Hironaka. A note on algebraic geometry over ground rings. The invariance of Hilbert characteristic functions under the specialization process. Illinois J. Math. 2 (1958), 355–366, MR0102519, Zbl 0099.15701. [Ho1] J. Hong. Rigidity of smooth Schubert varieties in Hermitian symmetric spaces. Trans. Amer. Math. Soc. 359 (2007), 2361–2381, MR2276624, Zbl 1126.14010. [Ho2] J. Hong. Rigidity of singular Schubert varieties in Gr(m, n). J. Differential Geom. 71 no. 1 (2005), 1–22, MR2191767, Zbl 1137.14311. [IM] A. Iliev, and L. Manivel, The Chow ring of the Cayley plane, Compositio Math. 141 (2005), 146–160. [Kl1] S. L. Kleiman. The transversality of a general translate. Compositio Math. 28(1974), 287– 297. [Kl2] S. L. Kleiman. Problem 15: Rigorous foundation of Schubert’s enumerative calculus. In Mathematical developments arising from Hilbert problems (Proc. Sympos. Pure Math., North- ern Illinois Univ., De Kalb, Ill., 1974), pages 445–482. Proc. Sympos. Pure Math., Vol. XXVIII. Amer. Math. Soc., Providence, R. I., 1976. [KL] S. L. Kleiman and D. Laksov. Schubert calculus. Amer. Math. Monthly 79(1972), 1061–1082. [KT] A. Knutson and T. Tao. Puzzles and (equivariant) cohomology of Grassmannians. Duke Math. J. 119(2003), 221–260. [K] J. Kollár. Rational curves on algebraic varieties, volume 32 of Ergebnisse der Mathematik und ihrer Grenzgebiete. 3. Folge. A Series of Modern Surveys in Mathematics. Springer-Verlag, Berlin, 1996. [Ko] J. Kollár.Flatness criteria. J. Algebra 175 (1995), 715–727, MR1339664, Zbl 0853.14008. [LS] V. Lakshmibai and C.S. Seshadri. Singular locus of a Schubert variety. Bull. Amer. Math. Soc. vol. 11 no. 2 (1984), 363–366, MR0752799, Zbl 0549.14016. [NS] S.I. Nikolenko, and N.S. Semenov, Chow ring structure made simple, arxiv: 0606335v1.pdf [R] C. Robles, Schur flexibility of cominuscule Schubert varieties, preprint. [RT] C. Robles and D. The, Rigid Schubert varieties in compact Hermitian symmetric spaces, Selecta Math., 18 no. 2 (2012), 717–777. [S] I.R. Shafarevich, Basic Algebraic Geometry I, Springer-Verlag, 1994. [V1] R. Vakil. A geometric Littlewood-Richardson rule. to appear Ann. of Math. [V2] R. Vakil. Schubert induction. to appear Ann.of Math. [W] M. Walters. Geometry and uniqueness of some extreme subvarieties in complex Grassmanni- ans. Ph. D. thesis, University of Michigan, 1997.