COMBINATORIAL Bn-ANALOGUES of SCHUBERT POLYNOMIALS 0
Total Page:16
File Type:pdf, Size:1020Kb
TRANSACTIONS OF THE AMERICAN MATHEMATICAL SOCIETY Volume 348, Number 9, September 1996 COMBINATORIAL Bn-ANALOGUES OF SCHUBERT POLYNOMIALS SERGEY FOMIN AND ANATOL N. KIRILLOV Abstract. Combinatorial Bn-analogues of Schubert polynomials and corre- sponding symmetric functions are constructed and studied. The development is based on an exponential solution of the type B Yang-Baxter equation that involves the nilCoxeter algebra of the hyperoctahedral group. 0. Introduction This paper is devoted to the problem of constructing type B analogues of the Schubert polynomials of Lascoux and Sch¨utzenberger (see, e.g., [L2], [M] and the literature therein). We begin with reviewing the basic properties of the type A polynomials, stating them in a form that would allow subsequent generalizations to other types. Let W = Wn be the Coxeter group of type An with generators s1, ... ,sn (that is, the symmetric group Sn+1). Let x1, x2, ... be formal variables. Then W naturally acts on the polynomial ring C[x1,...,xn+1] by permuting the variables. Let IW denote the ideal generated by homogeneous non-constant W -symmetric polynomials. By a classical result [Bo], the cohomology ring H(F ) of the flag variety of type A can be canonically identified with the quotient C[x1,...,xn+1]/IW .This ring is graded by the degree and has a distinguished linear basis of homogeneous cosets Xw modulo IW , labelled by the elements w of the group. Let us state for the record that, for an element w W of length l(w), ∈ (0) Xw is a homogeneous polynomial of degree l(w); X1=1; the latter condition signifies proper normalization. As shown by Bernstein, Gelfand, and Gelfand [BGG], one can construct such a basis using the divided difference operators ∂i acting in the polynomial ring. Namely, define an operator ∂i associated with a generator si by f sif ∂if = − ,i=1,2,.... xi xi+1 − Then recursively define, for each element w W ,apolynomialXw by ∈ (1) ∂iXwsi = Xw if l(wsi)=l(w)+1. The recursion starts at the “top polynomial” Xw0 that corresponds to the element w0 W of maximal length. Choose Xw to be an arbitrary and sufficiently generic ∈ 0 Received by the editors January 6, 1994. 1991 Mathematics Subject Classification. Primary 05E15; Secondary 05E05, 14M15. Key words and phrases. Yang-Baxter equation, Schubert polynomials, symmetric functions. Partially supported by the NSF (DMS-9400914). c 1996 American Mathematical Society 3591 License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 3592 SERGEY FOMIN AND ANATOL N. KIRILLOV homogeneous polynomial of degree l(w0). Then the Xw are well defined by (1), and provide a basis for the quotient ring H(F ). The normalization condition X1 =1 (see (0)) can be ensured by multiplying all the Xw by an appropriate constant. The basis of the quotient ring thus obtained is canonical in the sense that it does not depend (modulo the ideal IW ) on the particular choice of Xw0 . One can as well replace the recursion (1) by a weaker condition (1a) ∂iXws Xw mod Iw if l(wsi)=l(w)+1 i ≡ which is exactly equivalent to saying that the Xw represent the right cohomology classes. In order to be able to calculate in the cohomology ring, one would also like to w know how the basis elements multiply, i.e., find the structure constants cuv such w that XuXv = w cuvXw mod IW . Lascoux and Sch¨utzenberger [LS] discovered n 1 2 that a particular choice of the top element (viz., Xw0 = x1 − xn 2xn 1)makes P ··· − − it possible to get rid of the unpleasant “mod Iw” provision from the last identity. More precisely, they constructed a family of polynomials Xw (called the type A Schubert polynomials and denoted Sw elsewhere in this paper) which satisfy (0)- (1) and have the following property: (2) for any u, v Wn and for a sufficiently large m, ∈ w XuXv = cuvXw . w Wm X∈ Another remarkable property of the Schubert polynomials of Lascoux and Sch¨utzenberger that also makes good geometric sense is the following: (3) the Xw are polynomials with nonnegative integer coefficients. One can give a direct combinatorial explanation of this phenomenon by providing an alternative definition of the Schubert polynomials in terms of reduced decompo- sitions and compatible sequences [BJS] or, equivalently, via noncommutative gen- erating function in the nilCoxeter algebra of W (see [FS]). This alternative descrip- tion of the Schubert polynomials that avoids the recurrence process proved to be a helpful tool in deriving their fundamental properties and dealing with their general- izations, such as the Grothendieck polynomials of Lascoux and Sch¨utzenberger [L1], [FK2]. It is transparent from the combinatorial formula — and not hard to deduce from the original definition — that the Schubert polynomials are stable with respect to a natural embedding Wn , Wm , n<m(as a parabolic subgroup with generators s0, → ... ,sn 1) and the corresponding projection pr : C[x1,...,xm+1] C[x1,...,xn+1] defined− by → pr : f(x1,...,xm+1) f(x1,...,xn+1, 0,...,0) . 7→ In other words, (n) (m) (4) for w Wn Wm , Xw =prXw ∈ ⊂ (n) where Xw denotes the Schubert polynomial for w treated as an element of Wn , and pr simply takes the last m n variables xi to zero. (Actually, the Schubert polynomials of type A are stable− in an even stronger sense, but for the other types we will only require condition (4), as stated above.) License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use COMBINATORIAL Bn-ANALOGUES OF SCHUBERT POLYNOMIALS 3593 It seems reasonable to attempt to reproduce all of the above for other classical types, and as a first step for type B where Wn = Bn is the hyperoctahedral group. There is a natural action of Bn on the polynomial ring (see Section 1); as before, the cohomology ring can be identified with the quotient C[x1,x2,...]/IW ,andits basis can be constructed via divided differences in the similar way. (There are some peculiarities related to the definition of the divided differences for type B;see Section 1.) Ideally, Bn-Schubert polynomials would be a certain family of polynomials that satisfy the verbatim B-analogues of the conditions (0)-(4) above. Unfortunately, such a family of polynomials simply does not exist. We will show in Section 10 that, already for the hyperoctahedral group B2 with two generators, one cannot find 8 polynomials satisfying all relevant instances of conditions (0)-(3), even if we replace (1) by a weaker condition (1a). Since having all of (0)-(3) is impossible, we have to sacrifice one of the basic properties. Abandoning condition (0) seems extremely unreasonable. We are then led to the problems of finding polynomials satisfying (0) and two of the properties (1)-(3): (1) and (2), (2) and (3), or (1) and (3). To sweeten the pill, let us also require that (4) be satisfied. We arrived at the following three problems whose solutions could be viewed as Bn-analogues (in the three different senses specified below) of the Schubert polynomials. Problem 0-1-2-4. (Bn-Schubert polynomials of the second kind.) Find (n) (n) a family of polynomials Xw = X = Bw (x1,...,xn), one for each element w of each group Wn = Bn , which satisfy the type B versions of conditions (0), (1), (2), and (4). In this problem, conditions (0)-(1) ensure that the Xw represent the correspond- ing cosets of the B-G-G basis, and condition (2) means that they multiply exactly as the cohomology classes do. In Section 7, we construct a solution to Problem 0- 1-2-4 by giving a simple explicit formula for the generating function of the Xw in the nilCoxeter algebra of the hyperoctahedral group. We also prove that the stable limits of our polynomials coincide with the power series introduced by Billey and Haiman [BH] whose definition involved a λ-ring substitution and Schur P -functions. This also allows us to replace a long and technical verification of (1) in [BH] by a few lines of a transparent computation. Problem 0-2-3-4. (Bn-Schubert polynomials of the first kind.) Find a (n) (n) family of polynomials Xw = X = bw (x1,...,xn) satisfying the type B versions of conditions (0), (2), (3),and(4). This problem may at first sight look unnatural since these polynomials no longer represent Schubert cycles. However, if one really wants to compute in the quotient ring H(F ), property (1) is not critical, once it is known that the structure constants are correct. On the other hand, the solution of Problem 0-2-3-4 that we suggest in Section 6 is very natural combinatorially and much easier to work with than in the previous case of the polynomials of the second kind. The formulas become much simpler; for example, the Schubert polynomial of the first kind for the element w0 B3 is given by ∈ (3) 2 3 (5) bw0 = x1x2x3(x1 + x2)(x1 + x3)(x2 + x3) License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 3594 SERGEY FOMIN AND ANATOL N. KIRILLOV whereas, for the same element w0 , the Schubert polynomial of the second kind is (6) (3) 1 6 2 5 3 5 3 4 3 2 4 2 3 7 B = 4x x2x 12x x x3 +4x x2x +4x x x 4x x x +4x x2x3 w0 512 − 1 3 − 1 2 1 3 1 2 3 − 1 2 3 1 4 4 4 4 3 2 4 2 4 3 2 3 4 2 5 2 2x x x3 2x x2x +4x x x +16x x x +4x x x +4x x x − 1 2 − 1 3 1 2 3 1 2 3 1 2 3 1 2 3 2 2 5 3 3 3 3 5 7 5 3 2 6 8x x x +8x x x +12x x x3 4x1x x3 12x1x x 8x x x3 − 1 2 3 1 2 3 1 2 − 2 − 2 3 − 1 2 6 2 6 3 7 2 8 5 4 6 3 8 5 4 8x1x x 4x x +4x x +5x x2 2x x 8x x +3x x3 6x x − 3 2 − 1 2 1 2 1 − 1 2 − 1 3 1 − 1 3 4 5 2 7 3 6 8 3 6 4 5 8 3 6 2x x 4x x 12x x +9x1x +8x x +6x x 3x1x 4x x − 1 2 − 1 2 − 1 2 2 1 3 1 3 − 3 − 2 3 2 7 4 5 8 7 2 6 3 8 7 +12x x 14x x +5x2x 4x x 4x x +7x x3 +12x1x x2 2 3 − 2 3 3 − 2 3 − 2 3 2 3 5 4 3 5 5 3 2 6 4 4 9 9 9 2x x 20x x x2 +4x1x x 4x x x2 +2x x x1 +x +5x x .