COMBINATORIAL Bn-ANALOGUES of SCHUBERT POLYNOMIALS 0

TRANSACTIONS OF THE AMERICAN MATHEMATICAL SOCIETY Volume 348, Number 9, September 1996

COMBINATORIAL Bn-ANALOGUES OF SCHUBERT POLYNOMIALS

SERGEY FOMIN AND ANATOL N. KIRILLOV

Abstract. Combinatorial Bn-analogues of Schubert polynomials and corresponding symmetric functions are constructed and studied. The development is based on an exponential solution of the type B Yang-Baxter equation that involves the nilCoxeter algebra of the hyperoctahedral group.

0. Introduction This paper is devoted to the problem of constructing type B analogues of the Schubert polynomials of Lascoux and Schützenberger (see, e.g., [L2], [M] and the literature therein). We begin with reviewing the basic properties of the type A polynomials, stating them in a form that would allow subsequent generalizations to other types. Let W = Wn be the Coxeter group of type An with generators s1, ... ,sn (that is, the symmetric group Sn+1). Let x1, x2, ... be formal variables. Then W naturally acts on the polynomial ring C[x1,...,xn+1] by permuting the variables. Let IW denote the ideal generated by homogeneous non-constant W -symmetric polynomials. By a classical result [Bo], the cohomology ring H(F ) of the flag variety of type A can be canonically identified with the quotient C[x1,...,xn+1]/IW .This ring is graded by the degree and has a distinguished linear basis of homogeneous cosets Xw modulo IW , labelled by the elements w of the group. Let us state for the record that, for an element w W of length l(w), ∈ (0) Xw is a homogeneous polynomial of degree l(w); X1=1; the latter condition signifies proper normalization. As shown by Bernstein, Gelfand, and Gelfand [BGG], one can construct such a basis using the divided difference operators ∂i acting in the polynomial ring. Namely, define an operator ∂i associated with a generator si by

f sif ∂if = − ,i=1,2,.... xi xi+1 − Then recursively deﬁne, for each element w W ,apolynomialXw by ∈

(1) ∂iXwsi = Xw if l(wsi)=l(w)+1.

The recursion starts at the “top polynomial” Xw0 that corresponds to the element w0 W of maximal length. Choose Xw to be an arbitrary and suﬃciently generic ∈ 0 Received by the editors January 6, 1994. 1991 Mathematics Subject Classiﬁcation. Primary 05E15; Secondary 05E05, 14M15. Key words and phrases. Yang-Baxter equation, Schubert polynomials, symmetric functions. Partially supported by the NSF (DMS-9400914).

c 1996 American Mathematical Society

3591

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 3592 SERGEY FOMIN AND ANATOL N. KIRILLOV

homogeneous polynomial of degree l(w0). Then the Xw are well deﬁned by (1), and provide a basis for the quotient ring H(F ). The normalization condition X1 =1 (see (0)) can be ensured by multiplying all the Xw by an appropriate constant. The basis of the quotient ring thus obtained is canonical in the sense that it does not

depend (modulo the ideal IW ) on the particular choice of Xw0 . One can as well replace the recursion (1) by a weaker condition

(1a) ∂iXws Xw mod Iw if l(wsi)=l(w)+1 i ≡ which is exactly equivalent to saying that the Xw represent the right cohomology classes. In order to be able to calculate in the cohomology ring, one would also like to w know how the basis elements multiply, i.e., ﬁnd the structure constants cuv such w that XuXv = w cuvXw mod IW . Lascoux and Sch¨utzenberger [LS] discovered n 1 2 that a particular choice of the top element (viz., Xw0 = x1 − xn 2xn 1)makes P ··· − − it possible to get rid of the unpleasant “mod Iw” provision from the last identity. More precisely, they constructed a family of polynomials Xw (called the type A Schubert polynomials and denoted Sw elsewhere in this paper) which satisfy (0)- (1) and have the following property:

(2) for any u, v Wn and for a suﬃciently large m, ∈ w XuXv = cuvXw .

w Wm X∈ Another remarkable property of the Schubert polynomials of Lascoux and Sch¨utzenberger that also makes good geometric sense is the following:

(3) the Xw are polynomials with nonnegative integer coefficients. One can give a direct combinatorial explanation of this phenomenon by providing an alternative definition of the Schubert polynomials in terms of reduced decompositions and compatible sequences [BJS] or, equivalently, via noncommutative generating function in the nilCoxeter algebra of W (see [FS]). This alternative description of the Schubert polynomials that avoids the recurrence process proved to be a helpful tool in deriving their fundamental properties and dealing with their generalizations, such as the Grothendieck polynomials of Lascoux and Schützenberger [L1], [FK2]. It is transparent from the combinatorial formula — and not hard to deduce from the original definition — that the Schubert polynomials are stable with respect to a natural embedding Wn , Wm , n

pr : f(x1,...,xm+1) f(x1,...,xn+1, 0,...,0) . 7→ In other words, (n) (m) (4) for w Wn Wm , Xw =prXw ∈ ⊂ (n) where Xw denotes the Schubert polynomial for w treated as an element of Wn , and pr simply takes the last m n variables xi to zero. (Actually, the Schubert polynomials of type A are stable− in an even stronger sense, but for the other types we will only require condition (4), as stated above.)

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use COMBINATORIAL Bn-ANALOGUES OF SCHUBERT POLYNOMIALS 3593

It seems reasonable to attempt to reproduce all of the above for other classical types, and as a first step for type B where Wn = Bn is the hyperoctahedral group. There is a natural action of Bn on the polynomial ring (see Section 1); as before, the cohomology ring can be identified with the quotient C[x1,x2,...]/IW ,andits basis can be constructed via divided differences in the similar way. (There are some peculiarities related to the definition of the divided differences for type B;see Section 1.) Ideally, Bn-Schubert polynomials would be a certain family of polynomials that satisfy the verbatim B-analogues of the conditions (0)-(4) above. Unfortunately, such a family of polynomials simply does not exist. We will show in Section 10 that, already for the hyperoctahedral group B2 with two generators, one cannot find 8 polynomials satisfying all relevant instances of conditions (0)-(3), even if we replace (1) by a weaker condition (1a). Since having all of (0)-(3) is impossible, we have to sacrifice one of the basic properties. Abandoning condition (0) seems extremely unreasonable. We are then led to the problems of finding polynomials satisfying (0) and two of the properties (1)-(3): (1) and (2), (2) and (3), or (1) and (3). To sweeten the pill, let us also require that (4) be satisfied. We arrived at the following three problems whose solutions could be viewed as Bn-analogues (in the three different senses specified below) of the Schubert polynomials.

Problem 0-1-2-4. (Bn-Schubert polynomials of the second kind.) Find (n) (n) a family of polynomials Xw = X = Bw (x1,...,xn), one for each element w of each group Wn = Bn , which satisfy the type B versions of conditions (0), (1), (2), and (4).

In this problem, conditions (0)-(1) ensure that the Xw represent the corresponding cosets of the B-G-G basis, and condition (2) means that they multiply exactly as the cohomology classes do. In Section 7, we construct a solution to Problem 0- 1-2-4 by giving a simple explicit formula for the generating function of the Xw in the nilCoxeter algebra of the hyperoctahedral group. We also prove that the stable limits of our polynomials coincide with the power series introduced by Billey and Haiman [BH] whose deﬁnition involved a λ-ring substitution and Schur P -functions. This also allows us to replace a long and technical veriﬁcation of (1) in [BH] by a few lines of a transparent computation.

Problem 0-2-3-4. (Bn-Schubert polynomials of the first kind.) Find a (n) (n) family of polynomials Xw = X = bw (x1,...,xn) satisfying the type B versions of conditions (0), (2), (3),and(4).

This problem may at ﬁrst sight look unnatural since these polynomials no longer represent Schubert cycles. However, if one really wants to compute in the quotient ring H(F ), property (1) is not critical, once it is known that the structure constants are correct. On the other hand, the solution of Problem 0-2-3-4 that we suggest in Section 6 is very natural combinatorially and much easier to work with than in the previous case of the polynomials of the second kind. The formulas become much simpler; for example, the Schubert polynomial of the ﬁrst kind for the element w0 B3 is given by ∈ (3) 2 3 (5) bw0 = x1x2x3(x1 + x2)(x1 + x3)(x2 + x3)

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 3594 SERGEY FOMIN AND ANATOL N. KIRILLOV

whereas, for the same element w0 , the Schubert polynomial of the second kind is (6) (3) 1 6 2 5 3 5 3 4 3 2 4 2 3 7 B = 4x x2x 12x x x3 +4x x2x +4x x x 4x x x +4x x2x3 w0 512 − 1 3 − 1 2 1 3 1 2 3 − 1 2 3 1 4 4 4 4 3 2 4 2 4 3 2 3 4 2 5 2 2x x x3 2x x2x +4x x x +16x x x +4x x x +4x x x − 1 2 − 1 3 1 2 3 1 2 3 1 2 3 1 2 3 2 2 5 3 3 3 3 5 7 5 3 2 6 8x x x +8x x x +12x x x3 4x1x x3 12x1x x 8x x x3 − 1 2 3 1 2 3 1 2 − 2 − 2 3 − 1 2 6 2 6 3 7 2 8 5 4 6 3 8 5 4 8x1x x 4x x +4x x +5x x2 2x x 8x x +3x x3 6x x − 3 2 − 1 2 1 2 1 − 1 2 − 1 3 1 − 1 3 4 5 2 7 3 6 8 3 6 4 5 8 3 6 2x x 4x x 12x x +9x1x +8x x +6x x 3x1x 4x x − 1 2 − 1 2 − 1 2 2 1 3 1 3 − 3 − 2 3 2 7 4 5 8 7 2 6 3 8 7 +12x x 14x x +5x2x 4x x 4x x +7x x3 +12x1x x2 2 3 − 2 3 3 − 2 3 − 2 3 2 3 5 4 3 5 5 3 2 6 4 4 9 9 9 2x x 20x x x2 +4x1x x 4x x x2 +2x x x1 +x +5x x . − 2 3 − 1 3 3 2 − 1 3 2 3 1 2 − 3 We also show (see Section 7) that, in fact, the Bn-Schubert polynomials of the first and second kinds are related to each other by a certain “change of variables.” (This explains why the structure constants are the same in both cases.) Thus one can switch between polynomials of the two kinds, if necessary. Problem 0-1-3-4. (Schubert polynomials of the third kind.) Construct a family of polynomials Xw satisfying conditions (0), (1), (3),and(4). This is simply a question of finding explicit “combinatorial” representatives for the cohomology classes. In Section 9, we conjecture1 a solution of Problem 0-1- 3-4 for the type C; the corresponding type B polynomials differ from these by a k factor of the form 2− . Recently, we discovered that our “Schubert polynomials of the third kind” can also be obtained from the polynomials in two sets of variables introduced by Fulton in [Fu], by setting y1 = y2 = =0. We now briefly describe the general framework··· of our constructions and the organization of the paper. In [FS], [FK1]-[FK3] an approach to the theory of Schubert polynomials was developed that was based on an exponential solution of the Yang-Baxter equation (YBE) in the nilCoxeter algebra of the symmetric group, the latter being the abstract algebra isomorphic to the algebra of divided differences. In this paper, we adapt this approach to the case of the hyperoctahedral group. Section 2 presents a straightforward Bn-analogue of the main geometric construction used in [FK1] and inspired by Cherednik’s work [Ch]; the role of the YBE is briefly explained. (At this point, an acquaintance with our “An-paper” [FK1] would be very helpful.) In Section 3, some exponential solutions of the Bn-YBE are given; we refer to [FK3] for details. Section 4 introduces Bn-symmetric functions (generalized Stanley symmetric functions of type B) which can be associated with any such solution. Type B Schubert expressions of the first kind are introduced in Section 5. For the nilCoxeter algebra solution of the YBE, these expressions (n) give rise to the Bn-Schubert polynomials bw of the first kind which are studied (n) in Section 6. In Section 7, we define the Schubert polynomials Bw of the second kind and relate them to the Billey-Haiman construction. In Section 8, we introduce and study the Stanley symmetric functions of type B; this study was continued in

1Note added in proof . This conjecture is now a theorem. Tao Kai Lam has proved, in November 1995, that our Schubert polynomials of the third kind do indeed satisfy property (3). Properties (0), (1), and (4) are checked in Section 9.

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use COMBINATORIAL Bn-ANALOGUES OF SCHUBERT POLYNOMIALS 3595

[TKL1], [TKL2] and [BH]. In Section 9, the Schubert polynomials of the third kind are discussed. Section 10 contains the proof that it is impossible to simultaneously satisfy conditions (0)-(3). We provide tables of the Schubert polynomials of the three kinds for the types B2 and B3 , except for the table of the B3-polynomials of the second kind, which would occupy several pages. An 18-page table of the 384 B4-Schubert polynomials of the first kind was produced by Sébastien Veigneau using his wonderful Maple package ACE; this table is available from the authors upon request. As earlier in [FK1], we intentionally use in this paper the geometric approach that allows us to derive algebraic identities in the nilCoxeter algebra by modifying, according to certain rules, corresponding configurations of labelled pseudo-lines. A typical example is Theorem 4.4 that is proved by Figure 7. A formal algebraic version of this proof would be a straightforward (albeit messy and unreadable) translation from the geometric language. The combinatorial constructions of this paper can be adapted to describe the Schubert polynomials of types C and D, reproducing, in particular, the corresponding results in [BH]. A more or less straightforward modification of these constructions leads to combinatorial formulas for the Grothendieck polynomials of types BCD, in the spirit of [FK2], and also to the double Schubert polynomials of respective types. We plan to discuss these generalizations in a separate publication. Acknowledgements The authors are grateful to Tao Kai Lam, Peter Magyar, Victor Reiner, Richard Stanley, John Stembridge, and Andrei Zelevinsky for helpful comments. This paper originally appeared in June 1993 as a preprint PAR-LPTHE 93/33 of Université Paris VI. That earlier version did not contain the construction of the polynomials of the second and third kind; neither could it refer to the later work of Billey and Haiman [BH]. We thank the referee whose legitimate request compelled us to develop the corresponding theory presented in Sections 7 and 9. 1. The hyperoctahedral group: definitions and conventions

The hyperoctahedral group Bn is the group of symmetries of the n-dimensional cube. We deﬁne Bn formally as the group with generators s0, ... ,sn 1 satisfying the relations −

sisj = sjsi , i j 2; | − |≥ 2 si =1; sisi+1si = si+1sisi+1 ,i 1; ≥ s0s1s0s1 =s1s0s1s0 .

The indexing in this definition differs from the usual one (namely, what we call s0, ... ,sn 1 would have to be denoted by sn, ... ,s1, respectively; cf. [B]). However, we find− this labelling more convenient since it is respected by the natural embedding Bn , Bn+1. → The elements of Bn can be thought of as signed permutations:ageneratorsi, i>0, swaps the i’th and (i + 1)’st entries and the generator s0 changes the sign of the first entry. As in any Coxeter group, the length l(w)ofanelementwis the minimal number of generators whose product is w. Such a factorization of minimal length (or the corresponding sequence of indices) is called a reduced decomposition

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 3596 SERGEY FOMIN AND ANATOL N. KIRILLOV

12 w0=u1 u0 u1 u0

u0 u1 u0 21 12u1u0u1 u1

u0 u1 21 21uu10 u0

u0 12 21u1

1 2 1

Figure A. The weak order of the hyperoctahedral group B2

1 2 3

13221 3 1 23

31221312332231 1

321 321 213132123 23 1 21 3

31221 332321 1132132 123 23 1

213123 321 1 32 3 12321 1 32 321

213223 1 123 31 2312 1 321 3

231 21 3 1 32 2 31 3 12 u 12

123123132 u 1

1 2 3 u0

Figure B. The weak order of the hyperoctahedral group B3

of w.Theweak order on the group is deﬁned as the transitive closure of the covering relation wsi m w l ( wsi) >l(w). ⇐⇒ The weak orders of the hyperoctahedral groups B2 and B3 are given in Figures A and B. To represent signed permutations, we circle their negative entries. The hyperoctahedral group Bn acts on the polynomial ring C[x1,...,xn]inthe natural way. Namely, si interchanges xi and xi+1 ,fori=1,...,n 1, and the − special generator s0 acts by

s0f(x1,x2,...)=f( x1,x2,...) . −

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use COMBINATORIAL Bn-ANALOGUES OF SCHUBERT POLYNOMIALS 3597

The reversal of the indexing of generators has an annoying drawback: we have to change the sign in the usual deﬁnition of the divided diﬀerences. Rather than doing that (and creating a lot of confusion), we simply change the recurrence rule (1) into

Xw if l(wsi)=l(w)+1, (10) ∂iXwsi = − ( 0 otherwise.

To include the special case i =0in(10), we also deﬁne

f s0f ∂0f = − ; x1 − the negative sign in the denominator compensates for the one in (10). Thus one should replace condition (1) by (10) in the formulations of the Prob- lems 0-1-2-4, 0-2-3-4, and 0-1-3-4, while treating the Bn case.

2. Generalized configurations and the Yang-Baxter equation The notion of a generalized configuration was introduced in [FK1]. It is a configuration of contiguous lines which cross a given vertical strip from left to right; each line is subdivided into “segments”; each segment has an associated variable. A configuration is assumed to be generic in the following sense: (i) no three lines intersect at the same point; (ii) no two lines intersect at an endpoint of any segment; (ii) no two intersection points lie on the same vertical line. In the Bn case, this notion assumes an additional flavor. Namely, configurations are contained in a semi-strip bounded from below by a bottom mirror (cf. [Ch]). The lines of a configuration are allowed to touch the bottom; corresponding points are called points of reflection. Whenever this happens, an associated variable changes its sign. An example of a Bn-configuration is given on Figure 1. Intersection points and points of reflection will be of a particular interest to us. Each intersection point has a level number which indicates how many lines there are below this point (the point itself contributes 1). For example, the intersection points on Figure 1 have level numbers (from left to right) 1, 1, 2, and 1. By definition, the level number of a point of reflection is 0. Let be a configuration of the described type. Order its intersection and reflection pointsC altogether from left to right; then write down their level numbers. The resulting sequence of integers a( )=a1a2... is called a word associated with . In our running example, a( ) = 101201.C Now it is time to bring the variables intoC C the picture. Assume that is an associative algebra and hi(x):i=0,1,... is a family of elements of A whichdependonaformalvariable{ x(we always as-} sume that the main field containsA all participating formal variables as independent transcendentals). Then the associated expression for a configuration is C Φ( )=ha (z1)ha (z2) ... C 1 2 where, as before, a1a2 ... is an associated word and zi is one of the following: if ai =0,thenzi is the variable related to the corresponding point of reflection (to the left of it); if ai > 0, then zi = xi yi where xi and yi are the variables for − the segments intersecting at the corresponding point, xi being a variable for the segment which is above to the left of this point. In the example of Figure 1,

Φ( )=Φ( ;x, y, z, u, v)=h1(y u)h0(y)h1(v+y)h2(x+y)h0(v)h1(x+v). C C −

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 3598 SERGEY FOMIN AND ANATOL N. KIRILLOV

z -y x

u v -v y

11010 2

Figure 1

Figure 2

Informally, the variables associated with the segments are their “slopes”; an argu- ment of each factor in Φ( ) is the corresponding “angle of intersection”. The Yang-Baxter equationsC (see, e.g., [Ch] and references therein) are certain conditions on the hi(x) which allow us to transform conﬁgurations without changing their associated expression. The type B YBE are

(2.1) hi(x)hj (y)=hj(y)hi(x)if i j 2; |−|≥

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use COMBINATORIAL Bn-ANALOGUES OF SCHUBERT POLYNOMIALS 3599

x y = x+y

ii i

Figure 3

(2.2) hi(x)hi+1(x + y)hi(y)=hi+1(y)hi(x + y)hi+1(x)ifi 1; ≥

(2.3) h1(x y)h0(x)h1(x + y)h0(y)=h0(y)h1(x+y)h0(x)h1(x y). − − Each of these equations has its pictorial interpretation; see Figure 2. Following [FK3], we introduce the additional condition

(2.4) hi(x)hi(y)=hi(x+y),hi(0) = 1 for i 1 ≥ (cf. Figure 3) which means that we are interested in exponential solutions of the YBE. Relations (2.1)-(2.4) have various nice implications which can be derived by braid manipulation that replaces cumbersome algebraic computations. Informally, algebraic identities can be proved by moving lines according to the rules of Figures 2 and 3. 3. Examples of solutions There is a natural approach to constructing solutions of the equations (2.1)-(2.4). Assume that is a local associative algebra (in the sense of [V]) with generators A u0, u1, u2, ... which means that

(3.1) uiuj = uj ui if i j 2. | − |≥ Deﬁne hi(x)by

(3.2) hi(x)=exp(xui) . Then (2.1) and (2.4) are guaranteed, and we only need the Yang-Baxter equations (2.2)-(2.3) to be satisﬁed. Rewrite (2.2)-(2.3) as (3.3) exui e(x+y)ui+1 eyui = eyui+1 e(x+y)ui exui+1 and (3.4) [exu0 exu1 exu0 ,eyu0 eyu1 eyu0 ]=0 where [X, Y ] denotes a commutator XY YX. These equations were studied in [FK3] where the following solutions were suggested.− 3.1 Example. The nilCoxeter algebra of the hyperoctahedral group. Thisisthe algebra deﬁned by

uiuj = ujui , i j 2; | − |≥ 2 ui =0; uiui+1ui = ui+1uiui+1 ,i 1; ≥ u0u1u0u1=u1u0u1u0 .

These relations are satisfied by the divided differences ∂i ; thus one can think of the nilCoxeter algebra as of the algebra of divided difference operators.

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 3600 SERGEY FOMIN AND ANATOL N. KIRILLOV

2 Example 3.1 will be the main one in this paper. The relation ui = 0 implies that (3.2) can be rewritten as hi(x)=1+xui . Checking that these hi(x)satisfy the conditions (2.1)-(2.4) is straightforward. The nilCoxeter algebra (of any Coxeter group W ) has the following alternative description. For w W , take any reduced decomposition w = si si and identify ∈ 1 ··· l w with the element ui1 uil of the nilCoxeter algebra. These elements form a linear basis of the nilCoxeter··· algebra, and the multiplication rule is usual product wv if l(w)+l(v)=l(wv), w v = · (0 , otherwise . In this paper, we will frequently make use of this description, while expanding various expressions in the nilCoxeter algebra of Bn in the basis of group elements.

3.2 Example. Universal enveloping algebra of U+(so (2n + 1)). This algebra can be deﬁned as the local algebra with generators u1, u2, ... subject to Serre relations

[ui, [ui,ui 1]] = 0 ,i 1; ± ≥ [u0,[u0,u1]] = 0 ;

[u1, [u1, [u1,u0]]] = 0 .

As shown in [FK3], this universal enveloping algebra provides an exponential solution to the Yang-Baxter equation, that is, (3.3) and (3.4) are satisfied. 4. Symmetric expressions By analogy with [FK1], we will show now that the basic relations (2.1)-(2.4) (or (3.1)-(3.4)) imply that certain configurations produce symmetric expressions in the corresponding variables. In what follows we assume that (2.1)-(2.4) are satisfied. 4.1 Theorem. For the configuration of Figure 4, C Φ( ; x, y)=Φ( ;y,x) . C C In other words, Φ( ) is symmetric in x and y. C This statement has the following straightforward reformulation. 4.2 Proposition. Let

(4.1) B(x)=hn 1(x) h1(x)h0(x)h1(x) hn 1(x) . − ··· ··· − Then B(x) and B(y) commute.

Special case: n =2.ThenB(x)=h1(x)h0(x)h1(x). Now use (2.3) and (2.4) to show that

B(x)B(y)=h1(x)h0(x)h1(x)h1(y)h0(y)h1(y)

=h1(x)h0(x)h1(x+y)h0(y)h1(y)

=h1(x)h0(x)h1(x+y)h0(y)h1(y x)h1(x) − =h1(x)h1(y x)h0(y)h1(x+y)h0(x)h1(x) − =h1(y)h0(y)h1(y+x)h0(x)h1(x) =B(y)B(x).

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use COMBINATORIAL Bn-ANALOGUES OF SCHUBERT POLYNOMIALS 3601

x-xy -y 0 0 0 0

Figure 4

x x -x y -y 00 0 = -x y -y

0 -x 0 x-x-y = -y = x y

y 0 0 ==x-x -y x-x y-y

Figure 5

The same proof can be performed in the language of conﬁgurations — see Fig- ure 5. Moreover, the geometric proof has the advantage of being easily adjustable for the general case of an arbitrary n.

Proof of Theorem 4.1 (and Proposition 4.2). Same transformations as in Figure 5, with additional horizontal lines added near the bottom mirror.

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 3602 SERGEY FOMIN AND ANATOL N. KIRILLOV

4.3 Corollary. Let

(n) (4.2) H (x1 ,x2,...)=B(x1)B(x2) ··· where B(x) is deﬁned by (4.1),andthehi(x)satisfy the relations (2.1)-(2.4). Then (n) (n) the expression H (x1,x2,...) is symmetric in the xi. Moreover, H obeys the following cancellation rule:

(n) (n) (4.3) H (x1, x1,x2,,x3,...)=H (x2,x3,...) . − Proof. The symmetry is an immediate consequence of Proposition 4.2. The cancellation rule follows from the identity B( x)B(x)=1. − Corollary 4.3 suggests that one can construct non-trivial examples of symmetric functions by taking any solution of (2.1)-(2.4), then any representation of the (n) corresponding algebra , then applying the operator representing H (x1,x2,...) to any vector and, finally,A taking any coordinate of the image. Not only will those functions be symmetric; by a theorem of Pragacz [P], the cancellation rule (4.3) implies that they will belong to the subring Ω(x1,x2,...) of the ring Λ(x1,x2,...) of symmetric functions that is generated by odd power sums; equivalently, any such function is a linear combination of Schur P -functions. (n) Surprisingly enough, the expression H (x1,x2,...) can be alternatively defined by a quite different configuration. 4.4 Theorem. Let be the configuration defined by Figure 6. Then C (n) Φ( ; x1,...,xn)=H (x1,...,xn) . C Proof. See Figure 7.

(m) Remark. If the number of generators is m>n, then, to get H (x1,...,xn), one only needs to add m n horizontal lines near the bottom mirror in the configuration of Figure 6. − Theorem 4.4 allows us to relate the type B and type A constructions to each other. Note that the configuration of Figure 6 coincides with one of [FK1, Fig- ure 14], up to renumbering the variables xi in the opposite order (this is not essen- tial since the expression is symmetric in the xi), setting yi = xi, and attaching the bottom mirror. Since Figure 14 of [FK1] defines the ordinary− (i.e, type A) double stable Schubert expression G(x1,...,xn;y1,...,yn), the above observation has the following precise formulation.

4.5 Theorem. Let hi(x):i=1,...,n 1 be any solution of (2.1), (2.2),and { − } (2.4); in other words, let hi(x) be an exponential solution of the YBE of type { } (n) An 1. Deﬁne h0(x)=1. Then (2.3) obviously holds and so H is well-deﬁned. Moreover,− in this case

(n) H (x1,...,xn)=G(x1,...,xn; x1,..., xn) − − where G(...) is the double stable Schubert expression (see [FK1]).

In the special case of the nilCoxeter solution of Example 3.1 we obtain the Bn- analogues of the Stanley symmetric functions [S], or stable Schubert polynomials. These functions are studied in Section 8.

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use COMBINATORIAL Bn-ANALOGUES OF SCHUBERT POLYNOMIALS 3603

xn -x 1 0 -x 0 xn-1 2 0 0 x -x n 0 1 0 0 0

Figure 6

-x x1 n =

= =

Figure 7

5. Schubert expressions

Deﬁne the Bn-analogue of the generalized Schubert expression by

(n) (n) (5.1) b (x1,...,xn)=H (x1,...,xn)S( x1,..., xn 1) − − −

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 3604 SERGEY FOMIN AND ANATOL N. KIRILLOV

where S(x1,...,xn 1)istheAn 1-Schubert expression as deﬁned in [FS], [FK1]. In other words, − − (n) (5.2) b (x1,...,xn)=B(x1) B(xn)A1( x1) An 1( xn 1) ··· − ··· − − − where

(5.3) Ai(x)=hn 1(x)hn 2(x) hi(x) − − ··· (recall that B(x) is deﬁned by (4.1)). The formula (5.2) can be simpliﬁed.

(n) 5.1 Theorem. b (x1,...,xn) is equal to the expression deﬁned by Figure 8.In other words, (5.4) n 1 n i 1 (n) − − − b (x1,...,xn)=S(xn,xn 1,...,x2) h0(xn i) hj (xn i j + xn i) − − − − − i=0 j=1 ! Y Y where, as before, S(xn,xn 1,...,x2)=A1(xn) An 1(x2) and in the products the factors are multiplied− left-to-right, according··· − to the increase of i and j, respectively.··· Q 2 Note that the total number of factors h...(...) in (5.4) is n , the length of the longest element w0 of the hyperoctahedral group Bn with n generators. More- over, it can be immediately seen from Figure 8 that these factors are in a natural order-respecting bijection with the entries of the lexicographically maximal reduced decomposition of w0: n 1,n 2, ..., 2, 1, 0,n 1,n 2, ..., 2, 1, 0, ... , − − − − n 1,n 2, ..., 2, 1, 0 . − − 5.2 Examples. (1) b (x1)=h0(x1), (2) b (x1,x2)=h1(x2)h0(x2)h1(x1+x2)h0(x1), (3) b (x1,x2,x3)=h2(x3)h1(x3)h2(x2)h0(x3)h1(x2+x3)

h2(x1 +x3)h0(x2)h1(x1+x2)h0(x1). × Proof of Theorem 5.1. Let Φ6 and Φ8 be the expressions deﬁned by conﬁgurations of Figures 6 and 8, respectively. Then

Φ6 =Φ8A˜n 1(xn 1) A˜2(x2)A˜1(x1) − − ··· where

A˜i(x)=hi(x)hi+1(x) hn 1(x). ··· − ˜ 1 Since (2.4) implies that (Ai(x))− = Ai( x), it follows from Theorem 4.4 that − Φ8 =Φ6A1( x1)A2( x2) An 1( xn 1) − − ··· − − − (n) =H (x1,...,xn)S( x1,..., xn 1) − − − (n) =b (x1,...,xn).

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use COMBINATORIAL Bn-ANALOGUES OF SCHUBERT POLYNOMIALS 3605

x n -x n 0 xn-1 0 -x n-1

0 x1 0

-x 1

Figure 8

6. Schubert polynomials of the first kind In the rest of this paper we study the main example of solution of (2.1)-(2.4), namely, the one related to the nilCoxeter algebra of the hyperoctahedral group (Example 3.1). In this example, hi(x)=1+xui where ui is the i’th generator. By analogy with the case of the symmetric group (cf. [FK1]), we deﬁne the type B Schubert polynomials of the ﬁrst kind by expanding the corresponding expression in the nilCoxeter algebra in the basis of group elements: (n) (n) (6.1) b (x1,...,xn)= bw (x1,...,xn) w.

w Bn X∈ 6.1 Examples. (Cf. Examples 5.2.) In B1 , (1) b (x1)=1+x1u0

(1) (1) and therefore b1 =1 and bu0 = x1 . In B2 , (2) b (x1,x2)=(1+x2u1)(1 + x2u0)(1 + (x1 + x2)u1)(1 + x1u0) .

Expanding in the basis of group elements, we obtain the B2-Schubert polynomials of the ﬁrst kind

(2) w bw 1 1 u0 x1 + x2 u1 x1 +2x2 u0u1 x2(x1 +x2) 2 u1u0 (x1 +x2) u0u1u0 x1x2(x1 +x2) 2 u1u0u1 x2(x1 +x2) 2 w0 x1x2(x1 +x2)

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 3606 SERGEY FOMIN AND ANATOL N. KIRILLOV

In B3 , (3) b (x1,x2,x3)=(1+x3u2)(1 + x3u1)(1 + x2u2)(1 + x3u0)(1 + (x2 + x3)u1)

(1 + (x1 + x3)u2)(1 + x2u0)(1 + (x1 + x2)u1)(1 + x1u0). × (3) Expanding the right-hand side, we obtain the table of the polynomials bw given in Figure 9. It immediately follows from the deﬁnitions that, in general, the top polynomial of the ﬁrst kind is given by n (n) k bw0 (x1,...,xn)= (xk) (xi + xj ) k=1 1 i

(2) 2 and thus, e.g., Hw0 (x1,x2)=x1x2(x1+x2). (Note that, in general, k and n in (6.2) need not be equal.) As shown in Section 4 (see Corollary 4.3 and the paragraph immediately following its proof), these functions are indeed symmetric, and the following result holds. (n) 6.2 Lemma. The Stanley polynomials Hw of type B belong to the ring Ω generated by odd power sums. They are integer linear combinations of Schur P - functions. We discuss these polynomials in greater detail in Section 8. The deﬁnitions (6.1) and (6.2) can be straightforwardly restated in terms of reduced decompositions and “compatible sequences”. Use (4.1)-(4.2) to rewrite (6.2) as (n) γ(a,b) (6.3) H (x1,...,xk)= 2 xb xb xb w 1 2 ··· l a1,...,al R(w) 1 b1 bl k X∈ aiai≤···≤X+2 = ≤bi

γ(a, b)=# bi # i : ai =0 { }− { } (here # bi denotes the number of diﬀerent entries in the sequence b1, ... ,bl). Correspondingly,{ } (5.1) can be presented as (n) (n) (6.4) bw (x1,...,xn)= Hu (x1,...,xn)Sv( x1,..., xn 1) uv=w − − − l(u)+Xl(v)=l(w) v An 1 ∈ − where Sv is the ordinary Schubert polynomial for the symmetric group An 1 = Sn. It is also possible to entirely rewrite the deﬁnition of Theorem 5.1 in− terms of reduced decompositions and compatible sequences. We avoid doing so since the

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use COMBINATORIAL Bn-ANALOGUES OF SCHUBERT POLYNOMIALS 3607

(3) w R(w) bw | | 123 1 1 1 123 u0 1 x1 + x2 + x3 213 u1 1 x1 +2x2 +2x3 132 u2 1 x1 + x2 +2x3 213 u0u1 1 (x2 + x3)(x1 + x2 + x3) 132 u0u2 2 (x1 + x2 +2x3)(x1 + x2 + x3) 2 213 u1u0 1 (x1 + x2 + x3) 2 231 u1u2 1 x1x2 +2x1x3 +2x2x3 +2x3 2 2 312 u2u1 1 (x1 + x2 + x3) +(x2 +x3) 213 u0u1u0 1 (x1 + x2)(x1 + x3)(x2 + x3) 231 u0u1u2 1 x3(x1 + x3)(x2 + x3) 2 312 u0u2u1 2 (x2 + x3)((x1 + x2 + x3) + x2x3) 2 2 123 u1u0u1 1 (x2 + x3)(x1x2 + x1x3 + x2 + x2x3 + x3) 2 2 2 2 231 u1u0u2 2 x1x2 + x1x2 +2x1x3 +4x1x2x3 +3x1x3 2 2 3 +2x2x3 +3x2x3 +x3 2 2 2 2 2 321 u1u2u1 2 x1x2 +2x1x2 +2x1x3 +4x1x3 +4x2x3 2 3 +6x2x3 +2x3 +6x1x2x3 2 2 2 312 u2u1u0 1 (x1 + x2 + x3)(x1 + x2 + x3 + x1x2 + x1x3 + x2x3) +x1x2x3 2 123 u0u1u0u1 2 (x1 + x2)(x1 + x3)(x2 + x3) 231 u0u1u0u2 2 x3(x1 + x2)(x1 + x3)(x2 + x3) 321 u0u1u2u1 3 x3(x2 + x3)(x1 + x3)(x1 +2x2 +x3) 312 u0u2u1u0 2 (x1 + x2)(x1 + x3)(x2 + x3)(x1 + x2 + x3) 2 132 u1u0u1u2 1 x3(x2 + x3)(x1 + x3) 2 2 2 2 321 u1u0u2u1 2 (x2 + x3)(x1x2 + x1x2 + x1x3 + x1x3 2 2 +2x2x3 +2x2x3 +3x1x2x3) 3 3 4 3 2 2 2 321 u1u2u1u0 3 3x1x3 +3x2x3 +x3 +x1x2 +2x1x2 +6x1x2x3 3 2 2 2 2 3 +2x1x3 +4x1x3 +6x1x2x3 +8x1x2x3 +x1x2 3 2 2 +2x2x3 +4x2x3 2 2 132 u2u1u0u1 1 (x2 + x3)((x2 + x3)(x1 + x2 + x3)+x1x2x3) 2 132 u0u1u0u1u2 3 x3(x1 + x2)(x1 + x3)(x2 + x3) 321 u0u1u0u2u1 2 x2x3(x1 + x2)(x1 + x3)(x2 + x3) 321 u0u1u2u1u0 5 x3(x1 + x2)(x1 + x3)(x2 + x3)(x1 + x2 + x3) 2 2 132 u0u2u1u0u1 3 (x1 + x2)(x1 + x3)(x2 + x3)(x2 + x2x3 + x3) 2 312 u1u0u1u2u1 3 x3(x1 +2x2)(x1 + x3)(x2 + x3) 321 u1u0u2u1u0 2 (x1 + x2)(x1 + x3)(x2 + x3)(x1x2 + x1x3 + x2x3) 2 2 3 2 2 3 4 3 231 u1u2u1u0u1 3 6x1x2x3 +4x2x3 +4x2x3 +2x2x3 +4x1x2x3 3 4 2 3 2 2 2 2 +4x1x2x3 +2x2x3 +x1x2 +2x1x2x3 +2x1x2x3 2 3 4 4 +x1x3 +x1x3 +x1x2 3 123 u2u1u0u1u2 1 x3(x2 + x3)(x1 + x3) 2 312 u0u1u0u1u2u1 5 x2x3(x1 + x2)(x1 + x3)(x2 + x3) 321 u0u1u0u2u1u0 2 x1x2x3(x1 + x2)(x1 + x3)(x2 + x3) 2 231 u0u1u2u1u0u1 5 x2x3(x1 + x2)(x1 + x3)(x2 + x3) 3 123 u0u2u1u0u1u2 4 x3(x1 + x2)(x1 + x3)(x2 + x3) 2 2 312 u1u0u1u2u1u0 5 x3(x1 + x2) (x1 + x3)(x2 + x3) 2 2 2 2 231 u1u0u2u1u0u1 5 (x1x2 + x1x3 + x1x2x3 + x2x3 + x2x3) (x1 + x2)(x1 + x3)(x2 + x3) 3 213 u1u2u1u0u1u2 4 x3(x1 +2x2)(x1 + x3)(x2 + x3) 2 312 u0u1u0u1u2u1u0 7 x1x2x3(x1 + x2)(x1 + x3)(x2 + x3) 2 231 u0u1u0u2u1u0u1 7 x1x2x3(x1 + x2)(x1 + x3)(x2 + x3) 3 213 u0u1u2u1u0u1u2 9 x2x3(x1 + x2)(x1 + x3)(x2 + x3) 2 2 132 u1u0u1u2u1u0u1 5 x2x3(x1 + x2)(x1 + x3)(x2 + x3) 3 2 213 u1u0u2u1u0u1u2 9 x3(x1 + x2) (x1 + x3)(x2 + x3) 2 2 132 u0u1u0u1u2u1u0u1 12 x1x2x3(x1 + x2)(x1 + x3)(x2 + x3) 3 213 u0u1u0u2u1u0u1u2 16 x1x2x3(x1 + x2)(x1 + x3)(x2 + x3) 2 3 123 u1u0u1u2u1u0u1u2 14 x2x3(x1 + x2)(x1 + x3)(x2 + x3) 2 3 123 u0u1u0u1u2u1u0u1u2 42 x1x2x3(x1 + x2)(x1 + x3)(x2 + x3)

Figure 9. Type B3 Schubert polynomials of the ﬁrst kind

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 3608 SERGEY FOMIN AND ANATOL N. KIRILLOV

resulting formulas are rather messy; we also think that the following geometric approach (cf. [FK1, Section 6]) is more natural. (n) (n) Both bw and Hw have a direct combinatorial interpretation in terms of “resolved configurations”; this interpretation can actually be applied to a family of polynomials which come from any configuration . Take all the intersection points of and “resolve” each of them either as orC as . Then take all points of reflectionC and resolve each of them either as×^ or as ; the latter corresponds to changing a “sign”, or a “spin”, of the corresponding string.∨ If a configuration has N intersection and reflection points altogether, then there are 2N ways of producing such a resolution. Each of the 2N resolved configurations is a “signed braid” which naturally gives an element w of the hyperoctahedral group. Reading the -and -points from left to right produces a decomposition of w into a product of× gener- ∨ ators. Let w,foragivenw, denote the set of resolved configurations which give w C and for which this decomposition is reduced. Then the polynomials Φw associated with ,thatis, C

Φ( ; x1,x2,...)= Φw(x1,x2,...) w, C w X can be expressed as

(6.5) Φw(x1,x2,...)= (xi xj ) xk − c w X∈C Y Y where the ﬁrst product is taken over all intersections in and the second one — over all “change-of-sign” (i.e., -) points. C ∨ (n) (n) This interpretation enables us to prove the stability of bw and Hw .

6.3 Theorem. Let Bn and Bm, n

(n) (m) (6.7) Hw (x1,...,xk)=Hw (x1,...,xk) .

(n) In other words, the Hw do not depend on the superscript n (so we may drop it), (n) and the bw are stable in the weaker sense of (4): the coeﬃcient of any monomial in (n) bw (x1,...,xn) stabilizes as n . Thus we can introduce a well-deﬁned formal power series →∞

(6.8) (n) (n) bw(x1,x2,...) = lim bw (x1,...,xn)=H (x1,x2,...)S( x1,..., xn 1) n − →∞ − − which could be viewed as a limiting form of the type B Schubert polynomial of (n) the ﬁrst kind. Here H (x1,x2,...) is a symmetric expression in inﬁnitely many variables x1 ,x2, ... .

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use COMBINATORIAL Bn-ANALOGUES OF SCHUBERT POLYNOMIALS 3609

Proof of Theorem 6.3. Let I>n denote the two-sided ideal in the nilCoxeter algebra that is generated by un+1,un+2, ... Then (6.6) and (6.7) can be restated, respectively, as (n) (m) b (x1,...,xn) b (x1,...,xn,0,...,0)modI>n ≡ m n − and | {z } (n) (m) H (x1,...,xk) H (x1,...,xk)modI>n ≡ for m>n. The latter congruence is immediate from the deﬁnition of H(n) (one can also interpret it geometrically; cf. Figure 4). As to the former one, note that the stability of the ordinary (type A) Schubert polynomials can be reformulated as (n) (m) S (x1,...,xn) S (x1,...,xm)modI>n , ≡ implying

(n) (n) (n) b (x1,...,xn)=H (x1,...,xn)S ( x1,..., xn 1) − − − (m) (m) H (x1,...,xn)S ( x1,..., xn 1,0,...,0) mod I>n ≡ − − − (m) (m) = H (x1,...,xn,0,...,0)S ( x1,..., xn 1,0,...,0) − − − (m) = b (x1,...,xn,0,...,0).

Recall that the divided diﬀerence operator ∂i is deﬁned by

f(x1,...) f(...,xi+1,xi,...) (6.9) ∂if(x1,...)= − . xi xi+1 − We will show now that the Schubert polynomials of the ﬁrst kind satisfy the divided diﬀerence recurrences (10) for all i 1. ≥ 6.4 Theorem. For any w Bn and i 1, ∈ ≥ (n) (n) bw if l(wsi)=l(w)+1 (6.10) ∂i bwsi = − (0otherwise. (n) (Unfortunately, (6.10) is false for i = 0. Otherwise the polynomials bw would satisfy the conditions (0)-(4) of the introduction, which is impossible.)

Proof. As before, let ui be the generators of the nilCoxeter algebra. Then the theorem is equivalent to the identity (n) (n) (6.11) ∂i bwsi (wsi)= bw wui , − w · w X X where in the left-hand side w is interpreted as an element of the symmetric group, and in the right-hand side w is identiﬁed with the basis element of the nilCoxeter algebra. In turn, (6.11) can be rewritten as (n) (n) ∂ib = b ui . − (n) Now recall that H is symmetric in the xi and therefore

(n) (n) (n) (n) ∂ib = H (x1,...)∂iS( x1,...)=H (x1,...)S( x1,...)ui =b ui . − − − −

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 3610 SERGEY FOMIN AND ANATOL N. KIRILLOV

In this computation, we used (5.1) and the identity ∂iS = Sui (see [FS, Lemma 3.5]) which is just another way of stating the divided-diﬀerences recurrence for the ordinary Schubert polynomials.

By taking limits, one can obtain an analogue of Theorem 6.4 for the power series bw deﬁned in (6.8). A proof of property (2) for the Schubert polynomials of the ﬁrst kind will be giveninSection7.

7. Schubert polynomials of the second kind The Schubert expression of the second kind is deﬁned in the nilCoxeter algebra of the hyperoctahedral group by the formula

(n) (n) (7.1) B (x1,...,xn)= H (x1,...,xn) S( x1,..., xn 1) − − − q (n) (cf. (5.1)). In this formula, H (x1,...,xn) has constant term 1, so we interpret α α2 the square root via the expansion √1+α=1+2 8 +... . Note that in our case this expansion is ﬁnite since every noncommutative− monomial of degree >n2 in the nilCoxeter algebra of Bn vanishes. We then deﬁne the Schubert polynomials of the second kind by expanding B(n) in the basis of the nilCoxeter algebra formed by the group elements:

(n) (n) (7.2) B (x1,...,xn)= Bw (x1,...,xn) w ;

w Bn X∈

(cf. (6.1)). For example, the B2-Schubert polynomials of the second kind can be obtained by expanding the expression

(2) B (x1,x2)= (1 + x2u1)(1 + x2u0)(1 + (x1 + x2)u1)(1 + x1u0)(1 + x1u1)

(1 x1u1) . p· − 7.1 Theorem. The Schubert polynomials of the second kind satisfy the recurrence relations (10):

(n) (n) Bw if l(wsi)=l(w)+1, (7.3) ∂iBwsi = − ( 0 otherwise

for i =0,1,2,....

Proof. The veriﬁcation of (7.3) for i 1isexactlythesameasintheproofof ≥ Theorem 6.4, only replacing H(n) by √H (n).Astoi= 0, we obtain, using Propo- sition 4.2 and (5.3),

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use COMBINATORIAL Bn-ANALOGUES OF SCHUBERT POLYNOMIALS 3611

(n) 1 ∂0B = B(x1)B(x2) S( x1, x2,...) − x1 ··· − − p B( x1)B(x2) S(x1, x2,...) − − ··· − 1 p = B(x1)B(x2)B(x3) A1( x1) B( x1)A1(x1) A2( x2)A3( x3) x1 ··· − − − − − ··· 1 p = B(x1)B(x2) A1( x1) 1 h0( x1) A2( x2)A3( x3) x1 ··· − − − − − ··· 1 p = B(x1)B(x2) A1( x1)x1 u0A2( x2)A3( x3) x1 ··· − − − ··· p = B(x1)B(x2) A1( x1)A2( x2)A3( x3) u0 ··· − − − ··· (n) =pB u0 .

7.2 Theorem. The Schubert polynomials of the second kind satisfy the stability condition (4):

(n) (m) (7.4) Bw (x1,...,xn)=Bw (x1,...,xn,0,...,0) m n − for i =0,1,2,.... | {z }

Proof. The proof duplicates that of Theorem 6.3, with √H(n) instead of H(n).

Analogously to (6.8), the stability of the Schubert polynomials of the second kind allows us to introduce their stable limits, i.e., the power series

(n) Bw(x1,x2,...) = lim B (x1,...,xn) n w (7.5) →∞ = H(x1,x2,...) S( x1,..., xn 1) − − − p in infinitely many variables x1,x2, ... . These power series were first defined (in a different way) by Billey and Haiman [BH]. We will soon demonstrate the equiva- lence of the two definitions of the Bw . The next theorem directly relates the symmetric power series √H and H to each other; this relation will enable us to establish a connection between the Schubert polynomials of the two kinds.

7.3 Theorem. Assume that the variables x1,x2, ... and t1,t2, ... are related by

p (x ,x ,...) (7.6) k 1 2 =p (t ,t ,...),k=1,3,5,..., 2 k 1 2

k k where pk(x1,x2,...)= xi and pk(t1,t2,...)= ti are (odd) power sums. Then P P

(7.7) H(x1,x2,...)=H(t1,t2,...) . p

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 3612 SERGEY FOMIN AND ANATOL N. KIRILLOV

Proof. 1 1 log H(x ,x ,...) = log H(x ,x ,...) = log B(x ) 1 2 2 1 2 2 i i Y p 1 = log B(x ) = log B(t ) 2 i i i i X X =log B(ti) =log H(t1,t2,...) . i Y Note that, in view of Lemma 6.1, the relation (7.6) can be regarded as a “change of variables”. A λ-ring substitution essentially equivalent to (7.6) was first used by Billey and Haiman [BH] in their alternative definition of the power series Bw .In [BH], the Bw are defined, in our current notation, by

(7.8) Bw(x1,x2,...) w = H(t1,t2,...)S( x1, x2,...) w − − X where the xi and the tj are related to each other via (7.6). By virtue of (7.7), this definition is equivalent to our formula (7.5). (To be precise, the definition of BH Bw in [BH] differs from ours in sign. Denoting their polynomials by Bw to avoid confusion, we get BH l(w) (7.9) B =( 1) Bw . w − We chose our definition to make it consistent with the recurrence relations (10) whereas the definition of [BH] respects (1).)

7.4 Relations between the type B Schubert polynomials of the two kinds. One can directly compute the type B Schubert polynomials of the first kind from their counterparts of the second kind, and vice versa, using the following application of the substitution (7.6). Suppose we know a polynomial bw(t1,t2,...) of the first kind. Expand it in the basis of type An 1 Schubert polynomials, the coefficients − being symmetric functions in the ti :

(7.10) bw(t1,t2,...)= αv(t1,t2,...)Sv(t1,t2,...) .

v Sn X∈ (Such an expansion is unique and can be found, e.g., by a repeated use of divided diﬀerences.) The symmetric functions αv will necessarily belong to the ring Ω generated by the odd power sums pk.Expresseachαv(t1,t2,...)intermsof the pk(t1,t2,...) and apply the substitution (7.6), thus obtaining the functions γv(x1,x2,...)=αv(t1,t2,...). The polynomial of the second kind is now given by

(7.11) Bw(x1,x2,...)= γv(x1,x2,...)Sv(x1,x2,...) . v X To compute bw from Bw , one can use the same algorithm with the inverse substitution.

To give an example, let us compute Bw0 in B2 . We know (see Example 6.1) (2) 2 that bw0 (t1,t2)=t1t2(t1+t2). This can be uniquely expanded in the ordinary A1-Schubert polynomials, which are 1 and t1, with symmetric coeﬃcients. The expansion is (2) 2 2 b (t1,t2)=t1t (t1+t2)=t1t2(t1+t2) 1 t1t2(t1+t2) t1 . w0 2 · − ·

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use COMBINATORIAL Bn-ANALOGUES OF SCHUBERT POLYNOMIALS 3613

3 3 Denoting p1 = t1 + t2 and p3 = t1 + t2 ,weobtain 2 3 α1(t1,t2)=t1t2(t1+t2) =p1(p p3)/3 1− and 3 αu (t1,t2)= t1t2(t1+t2)=( p +p3)/3. 1 − − 1 3 3 Now plug in p1 =(x1 +x2)/2andp3=(x1+x2)/2 (cf. (7.6)) to get

3 3 3 1 x1+x2 x1+x2 x1+x2 1 2 2 γ1(x1,x2)= = (x1 x2) (x1 + x2) 3· 2 · 2 − 2 −16 − ! and 3 3 3 1 x1+x2 x1+x2 1 2 γu (x1,x2)= + = (x1 x2) (x1 +x2). 1 3· − 2 2 8 − ! Finally, use (7.11) to compute

(2) 2 2 2 B (x1,x2)= (x1 x2) (x1+x2) /16 + x1 (x1 x2) (x1 + x2)/8 w0 − − · − 3 =(x1 x2) (x1+x2)/16 . − The rest of the B2-Schubert polynomials of the second kind can be computed from the divided diﬀerence recurrence relations (10), producing the following table:

(2) w Bw 1 1 u0 (x1 + x2)/2 u1 x2 u0u1 (x1 x2)(x1 + x2)/4 − − 2 u1u0 (x1 + x2) /4 2 u0u1u0 (x1 x2) (x1 + x2)/8 − − u1u0u1 x2(x1 x2)(x1 + x2)/4 − −3 w0 (x1 x2) (x1 + x2)/16 −

The above algorithm allows us to avoid calculations based on the expansion of the square root in (7.1). However, the λ-ring substitution (7.6) becomes progres- sively harder to compute, as n increases, so the computational advantages of this approach are questionable. Another problem that we hid while treating the B2 case (fortunately, it did not affect our computations) is that the very definition of the substitution (7.6) is ambiguous in the case of finitely many variables, since the pk are not algebraically independent. To resolve this problem, one can use the stability property (7.4): increase the number of variables n (thus going to a hyperoctahedral group Bn of a larger order) while keeping w fixed. The maximal degree of a coeffi- cient αv in (7.10) is l = l(w). Therefore a representation of αv as a polynomial in l+1 l+1 the pk may only involve first 2 odd power sums. If n 2 or, equivalently, 2n l, then these power sums are algebraically independent.≥ Thus an expression ≥ for each αv is uniquely determined, and the above algorithm works. (In fact, the inequality 2n l can be strengthened.) For w0 B2,wehadn=2andl=4,so the problem did≥ not come up. ∈

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 3614 SERGEY FOMIN AND ANATOL N. KIRILLOV

(3) Type B3 Schubert polynomials Bw of the second kind can be obtained by applying the divided difference operators to the top polynomial (3) Bw0 (x1,x2,x3) 6 3 2 2 3 =((P 5P P3+9P1P5 5P )( x x2 +(P P3)/3 1 − 1 − 3 − 1 1 − 2 2 P x1 +P1(x +x1x2))/45 − 1 1 8 5 3 (P 7P P3 +14P P5 +7P3P5 15P1P7)x2/105 − 1 − 1 1 − k k k where Pk =(x1 +x2 +x3)/2fork=1,3,5,7; see (6) for the expansion of this polynomial in the variables x1, x2,andx3. 7.5 Multiplication of the Schubert polynomials. We are now going to show that the Schubert polynomials of the two kinds satisfy condition (2). First note that, in view of the stability property (4) which was proved in Theorems 6.3 and 7.2, it suffices to prove (2) for the power series bw and Bw defined by (6.8) and (7.5). In view of Theorem 7.3 (see also Subsection 7.4), the structure constants for the bw and the Bw are the same, since they are not affected by the substitution (7.6). Thus it sufices to prove property (2) for the power series Bw of the second kind: w (7.12) BuBv = cuvBw . w X A two-line proof of (7.12) (based on definition (7.8)) was given in [BH]. We reproduce it here to make the paper self-contained. Recurrences (10) imply that (7.12) holds modulo the ideal IW of B-symmetric functions. Since both sides of (7.12) belong to the ring R = C[x1,x2,...;p1,p3,...], and the only element in the intersection R IW is 0, (7.12) follows. ∩ 8. Stanley symmetric functions of type B This section is devoted to studying the basic properties of the type B Stanley symmetric functions Hw defined by (6.2), (4.2), and (4.1). In the nilCoxeter case, Theorem 4.5 immediately allows us to establish the following connection between the Hw and the ordinary (type A) Stanley symmetric functions Gw.

8.1 Corollary. Let x =(x1,...,xn).Letwbe an element of the parabolic subgroup of type An 1 of the hyperoctahedral group Bn that is generated by s1, ... , sn 1. Then − − (n) super Hw (x)=Gw (x, x) super where Gw is the super-symmetric function that canonically corresponds to the super stable Schubert polynomial Gw . (In the λ-ring notation, it means that Gw (x, y) = Gw(x+y).)

The last formula implies that, for such w, Hw is a nonnegative integer linear combination of Schur P -functions. (Recall that, by Lemma 6.1, any Hw is an integer linear combination of Schur P -functions.) Tao Kai Lam [TKL1], [TKL2] has recently found a proof that, in fact, Hw is always a nonnegative integer combination of P -functions (see also [BH]). It can be shown that the [skew] P -functions themselves are a special case of the Hw. To do that, we generalize, in a more or less straightforward way, the corresponding Sn-statement about 321-avoiding permutations (see [BJS]).

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use COMBINATORIAL Bn-ANALOGUES OF SCHUBERT POLYNOMIALS 3615

8.2 Theorem. Let σ be a skew shifted shape presented in a standard “English” notation (see, e.g., [SS]). Define a “content” of each cell of σ to be the difference between the number of column and the number of row which this cell is in. (For example, the content of a cell lying on the main diagonal is 0.) Read the contents of the cells of σ column by column, from top to bottom; this gives a sequence a1, ... , al. Define an element wσ of the hyperoctahedral group Bn by wσ = sa sa 1 ··· l (in fact, a1, ... , al is a reduced decomposition of wσ ). Then Hwσ = Pσ where Pσ is the skew Schur P -function corresponding to the shape σ.

One could also ask: which elements w Bn can be represented as wσ (see Theorem 8.2)? The answer (informal though∈ unambigous) is: those w which avoid the following patterns: 321 321 321 3211212 where i denotes the element i of a signed permutation that has changed its sign. Technical proofs of the last statement and of Theorem 8.2 are omitted. (n) Similarly to the type A case, the symmetric functions Hw can be obtained as (m) some kind of limit of bw as m . →∞ 8.3 Theorem. For w Bn , ∈ (n+N) (n) lim bw (0,...,0,x1,...,xn)=Hw (x1,...,xn) . N →∞ N Furthermore: if N n 1,then ≥ − | {z } (n+N) (n) bw (0,...,0,x1,...,xn)=Hw (x1,...,xn) . Proof. Similar to the proof of Theorem 6.3.

Here is another useful property of the polynomials Hw . (n) (n) 8.4 Lemma. Hw = H 1 . w− Proof. Follows from the symmetry of the deﬁning conﬁguration for H(n) (see Fig- ure 7).

One can also study the Stanley symmetric functions “of the second kind” deﬁned by expanding the symmetric expression √H(n) (cf. Section 7) in the basis of group elements. These symmetric functions can be viewed as stable Schubert polynomials (of the second kind); they clearly are linear combinations of Schur P -functions with rational coeﬃcients.

9. On Schubert polynomials of the third kind

(n) In this section, we introduce a family of polynomials Cw which we call the type C Schubert polynomials of the third kind. For these polynomials, the respective versions of properties (0) and (10) are immediate from their definition; we will also prove property (4). Unfortunately, we were unable to prove (3), which would provide a solution to Problem 0-1-3-4 of the Introduction2. However, we found significant computational evidence that condition (3) is indeed satisfied by these

2This was recently proved by Tao Kai Lam.

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 3616 SERGEY FOMIN AND ANATOL N. KIRILLOV

(n) polynomials. The type B Schubert polynomials of the third kind Bw are then defined by (n) σ(w) (n) (9.1) Bw =2− Cw where σ(w) denotes the number of sign changes in w. It will immediately follow that the Bw satisfy the type B conditions (0), (10), and (4), and, under the conjecture l(w) stated above, have nonnegative coefficients which are multiples of 2− . First, let us make clear what are the type C divided differences. For i 1, they ≥ are the same ∂i as before. For i = 0, define

C ∂0f f(x1,x2,...) f( x1,x2,...) (9.2) ∂0 f = = − − 2 2x1 − which explains (9.1). Now set n C(n) (n) k (9.3) w0 = bw0 = (xk) (xi + xj ) k=1 1 i

(2) 2 9.1 Example. For n =2,wehaveCw0 = x1x2(x1 + x2). Applying the divided C diﬀerences ∂ and ∂1 (cf. (10)), we obtain the following table: − 0 − (2) w Cw 1 1 u0 x1 + x2 u1 x2 2 u0u1 x2 2 2 u1u0 x1 + x1x2 + x2 u0u1u0 x1x2(x1 + x2) 3 u1u0u1 x2 2 w0 x1x2(x1 + x2)

Using (9.1), we then compute the Schubert polynomials of the third kind of type B2:

(2) w Bw 1 1 u0 (x1 + x2)/2 u1 x2 2 u0u1 x2/2 2 2 u1u0 (x1 + x1x2 + x2)/2 u0u1u0 x1x2(x1 + x2)/4 3 u1u0u1 x2/2 2 w0 x1x2(x1 + x2)/4

The table of the type C3 Schubert polynomials of the third kind is given in Figure 10.

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use COMBINATORIAL Bn-ANALOGUES OF SCHUBERT POLYNOMIALS 3617

(3) w Cw 123 1 123 x1 + x2 + x3 213 x2 + x3 132 x3 2 2 213 x3 + x2 + x2x3 2 2 2 132 x1 + x3x1 +2x3 +x2 +x2x3 2 2 2 213 x1 + x2x1 + x3x1 + x3 + x2 + x2x3 2 231 x3 2 2 2 312 x1 + x3 + x2 + x2x3 2 2 2 2 2 2 213 x1x2 + x1x3 + x3x1 + x1x2 + x2x3x1 + x3x2 + x2x3 3 231 x3 2 2 2 312 (x2 + x3)(x1 + x3 + x2 + x2x3) 3 2 3 2 123 x3 + x3x2 + x2 + x2x3 2 2 2 231 x3(x1 + x3x1 + x3 + x2 + x2x3) 2 2 2 321 x3(x1 + x3 + x2 + x2x3) 3 2 2 2 2 3 2 3 2 312 x1 + x1x2 + x1x3 + x3x1 + x1x2 + x2x3x1 + x3 + x3x2 + x2 + x2x3 3 3 2 2 2 2 2 3 2 2 2 2 3 123 x3x1 + x2x3 + x1x3 +2x2x3 +x3x2x1 +x2x3 +x2x1x3 +x1x2x3 +x1x2 +x1x2 2 2 2 231 x3(x1 + x3x1 + x2 + x2x3) 2 2 2 2 321 x3(x1 + x3 + x2 + x2x3) 2 2 2 2 2 2 312 (x1 + x2 + x3)(x3x1 + x3x2 + x1x3 + x2x3 + x1x2 + x1x2) 4 132 x3 3 2 2 2 2 3 2 2 2 321 x2x3 + x1x3 +2x2x3 +x2x3 +x2x1x3 +x1x2 3 2 2 2 2 2 2 3 2 4 2 2 3 3 321 x1x3 +2x1x3 +x1x2 +x2x1x3 +x3x2x1 +x3x1 +x1x2x3 +x3 +2x2x3 +x2x3 +x2x3 3 3 4 4 2 2 132 x2x3 + x2x3 + x2 + x3 + x2x3 3 2 2 132 x3(x1 + x3x1 + x2 + x2x3) 2 321 x2x3(x2 + x3)(x1 + x2x3) 3 2 2 2 2 2 2 3 2 2 2 3 3 321 x3(x1x3 +2x1x3 +x1x2 +x2x1x3 +x3x2x1 +x3x1 +x1x2x3 +2x2x3 +x2x3 +x2x3) 4 4 3 2 2 3 3 2 2 3 2 132 x3x1 + x3x2 + x3x1 +2x2x3 +x3x2x1 +x2x1x3 +2x2x3+ 2 2 4 2 2 3 2 3 4 x3x1x2 +x2x3 +x1x2x3 +x1x2x3 +x1x2 +x1x2 3 2 2 312 x3(x1 + x2x3 + x2) 2 3 2 3 2 3 2 2 3 3 2 2 3 321 x1x2x3(x1 + x2 + x3) + x1x3 + x1x2 + x3x1 + x1x2 + x2x3 + x2x3 4 3 2 2 3 2 2 3 2 4 2 2 2 3 231 x3x2 + x3x1 +2x2x3 +x2x1x3 +2x2x3 +x2x3 +x1x2x3 +x1x2 5 123 x3 2 2 312 x2x3(x2 + x3)(x1 + x2x3) 321 x1x2x3(x1 + x2)(x2 + x3)(x1 + x3) 2 2 231 x2x3(x2 + x3) (x1 + x2x3) 2 2 4 123 (x1 + x3x1 + x2 + x2x3)x3 3 2 2 2 2 2 2 2 2 2 3 2 312 (x1x3 + x1x3 + x1x2 + x2x1x3 + x3x2x1 + x1x2x3 + x2x3 + x2x3)x3 2 4 4 2 4 3 3 3 3 2 3 2 3 2 2 2 4 2 231 x1x3 + x1x2x3 + x2x3 + x1x3 +2x2x3 +2x1x2x3 +2x1x2x3 +3x1x2x3 +x2x3+ 3 2 2 3 2 3 4 3 2 2 4 3 3 x1x3x2 +2x3x1x2 +2x1x2x3 +x1x2x3 +x1x2x3 +x1x2 +x1x2 4 2 2 213 x3(x1 + x2x3 + x2) 2 312 x1x2x3(x1 + x2)(x2 + x3)(x1 + x3) 2 231 x1x2x3(x2 + x3) (x1 + x2)(x1 + x3) 3 2 213 x2x3(x2 + x3)(x1 + x2x3) 2 2 2 132 x2x3(x2 + x3)(x1 + x2x3) 3 3 2 2 2 2 2 2 2 2 2 3 213 x3(x1x3 + x1x3 + x1x2 + x2x1x3 + x3x2x1 + x1x2x3 + x2x3 + x2x3) 2 2 132 x1x2x3(x1 + x2)(x2 + x3)(x1 + x3) 3 213 x1x2x3(x1 + x2)(x2 + x3)(x1 + x3) 2 3 2 123 x2x3(x2 + x3)(x1 + x2x3) 2 3 123 x1x2x3(x1 + x2)(x2 + x3)(x1 + x3)

Figure 10. Type C3 Schubert polynomials of the third kind

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 3618 SERGEY FOMIN AND ANATOL N. KIRILLOV

(n) (n) The polynomials Cw and Bw are obviously homogeneous of degree l(w) and, by deﬁnition, they satisfy the respective recurrences (10). We are now going to (n) (n) prove the stability property (4); this will also imply that C1 = B1 =1,since we have already checked it for n =2. To prove stability, we need to demonstrate that the rule (9.3) is consistent with the divided diﬀerence recurrences. This is shown in the following theorem.

(n 1) 9.2 Theorem. Let w = w0 − be the element of maximal length in the parabolic subgroup Bn 1 Bn generated by s1, ... ,sn 2. Then − ⊂ − (n) (n 1) (9.4) Cw (x1,...,xn 1,0) = Cw − (x1,...,xn 1) . − −

Proof. Let us observe that the elements of maximal length in Bn 1 and Bn are related by −

(n) (n 1) (9.5) w0 = w0 − sn 1sn 2 s1s0s1 sn 2sn 1 . − − ··· ··· − − This allows to restate (9.4) as

C (9.6) ( ∂n 1 ∂1∂0 ∂1 ∂n 1fn)(x1,...,xn 1,0) = fn 1(x1,...,xn 1) − − ··· ··· − − − − whereweusedthenotation n k (9.7) fn(x1,...,xn)= (xk) (xi + xj ) k=1 1 i

n 1 2 n 1 ( 1) − ∂1 ∂n 1fn =x1x2x3 xn− (xi + xj )=x1F − ··· − ··· 1 i

2 n 1 (9.8) F = x2x x − (xi + xj ) . 3 ··· n 1 i

n C C (9.9) ( 1) ∂0 ∂1 ∂n 1fn =F x1∂0 F. − ··· − − Thus the claim (9.6) can be reformulated as

C (9.10) Dn 1(F x1∂0 F ) = fn 1(x1,...,xn 1) − − xn=0 − −

whereweusedthenotation k Dk =( 1) ∂k ∂1 . − ··· To prove (9.10), we will need the following lemma.

9.3 Lemma.

2 n 1 2 n 2 (9.11) Dn 1(x2x3 xn− ) x =0 = x2x3 xn−1 . − ··· n ··· −

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use COMBINATORIAL Bn-ANALOGUES OF SCHUBERT POLYNOMIALS 3619

Proof. Induction on n.Forn= 2, we check that, indeed, ∂1x2 = 1. Suppose 2 n 2 2 n 3 − we know that Dn 2(x2x3 xn−1)=x2x3 xn−2+xn 1f for some polynomial f. Then − ··· − ··· − − 2 n 1 Dn 1(x2x3 xn− ) − ··· n 1 2 n 2 = ∂n 1 xn− Dn 2(x2x3 xn−1) − − − ··· − 2 n 3 n 1 n 1 = ∂n 1 x2x3 xn−2xn− +xn− xn 1f − − ··· − − 2 n 3 n 2 n 3 n 2 n 2 = x2x3 xn−2 xn−1 +xn−1xn + +xn− xnxn 1∂n 1 xn− f . ··· − − − ··· − − − Setting xn = 0, we obtain (9.11), as desired. We continue the proof of Theorem 9.2. From (9.8) we get

C 2 n 1 C ∂ F = x2x x − ∂ (xi + xj ) 0 3 ··· n 0 1 i

C Dn 1(F x1∂0 F ) − − xn=0 = Dn 1F by (9.12) − xn=0 2 n 1 = (xi + xj )Dn 1(x2x3 xn− ) by (9.8) − ··· xn=0 1 i

proving (9.10) and hence Theorem 9.2.

10. A negative result

10.1 Lemma. Let W = B2 be the hyperoctahedral group with two generators. As- sume that Xw(x1,x2):w W is a family of polynomials satisfying conditions (0) and{ (1a) (see Introduction∈ }) and the following instances of condition (2): 2 Xs0 = Xs1s0 , Xs0 Xs1 = Xs1s0 + Xs0s1 . Then (a) for some w W , the polynomial Xw has both positive and negative coeﬃcients; ∈ (b) for some w W , the polynomial Xw has non-integer coeﬃcients. ∈ The same statement is true with condition (1) replaced by (10). 1 Proof. Conditions (0)-(1) imply Xs = (x1 + x2)andXs = x2 .Wethenuse 0 − 2 1 − (2) to compute X = X X X2 = 1 (x2 x2), which proves both (a) and s0s1 s0 s1 s0 4 2 1 (b). − −

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 3620 SERGEY FOMIN AND ANATOL N. KIRILLOV

The second version of the lemma is equivalent to the ﬁrst one, under the trans- formation (7.9). References

[BGG] I. N. Bernstein, I. M. Gelfand, and S. I. Gelfand, Schubert cells and cohomology of the spaces G/P , Russian Math. Surveys 28 (1973), 1–26. MR 55:2941 [B] N. Bourbaki, Groupes et Algèbres de Lie, Ch. VI, Hermann, Paris, 1968. MR 39:1590 [Bo] A. Borel, Sur la cohomologie des espaces fibrés principaux et des espaces homogènes des groupes de Lie compacts 5,, Ann. Math. 57 (1953), 115–207. MR 14:490e [BH] S. C. Billey and M. Haiman, Schubert polynomials for the classical groups,manuscript, September 1993. [BJS] S. C. Billey, W. Jockusch, and R. P. Stanley, Some combinatorial properties of Schubert polynomials, J. Algebr. Combinatorics 2 (1993), 345–374. MR 94m:05197 [Ch] I. Cherednik, Notes on affine Hecke algebras. I, Max-Planck-Institut Preprint MPI/91-14, 1991. [FK1] S. Fomin and A. N. Kirillov, The Yang-Baxter equation, symmetric functions, and Schu- bert polynomials, Proceedings of the 5th International Conference on Formal Power Series and Algebraic Combinatorics, Firenze, 1993, pp. 215–229, to appear in Discrete Mathe- matics. [FK2] , Grothendieck polynomials and the Yang-Baxter equation, Proceedings of the 6th International Conference on Formal Power Series and Algebraic Combinatorics, DIMACS, 1994, pp. 183–190. [FK3] , Universal exponential solution of the Yang-Baxter equation, preprint PAR- LPTHE 93/37, Université Paris VI, June 1993, to appear in Letters in Math. Physics. [FS] S. Fomin and R. P. Stanley, Schubert polynomials and the nil-Coxeter algebra, Advances in Math. 103 (1994), 196–207. MR 95f:05115 [Fu] W. Fulton, Schubert varieties in flag bundles for the classical groups,preprint. [L1] A. Lascoux, Anneau de Grothendieck de la variete de drapeaux, The Grothendieck Fest- schrift, vol. III, Birkhäuser, 1990, pp. 1–34. MR 92j:14064 [L2] , Polynômes de Schubert. Une approche historique,Séries formelles et combinatoire algébrique (P. Leroux and C. Reutenauer, eds.), Montréal, LACIM, UQAM, 1992, pp. 283–296. [LS] A.Lascoux, M.P.Schützenberger, Polynômes de Schubert,C.R.Ac.Sci.294 (1982), 447. [TKL1] Tao Kai Lam, Bn Stanley symmetric functions,Séries formelles et combinatoire algébri- que, Proceedings of the 6th International Conference on Formal Power Series and Alge- braic Combinatorics, DIMACS, 1994, pp. 315–324. [TKL2] , B and D analogues of stable Schubert polynomials and related insertion algorithms, Ph. D. thesis, M.I.T., 1994. [M] I. G. Macdonald, Notes on Schubert polynomials, Laboratoire de combinatoire et d’infor- matique mathématique (LACIM), UniversitéduQuébec àMontréal, Montréal, 1991. [P] P. Pragacz, Algebro-geometric applications of Schur S-andQ-polynomials,“Topicsin Invariant Theory” (M.-P. Malliavin, ed.), pp. 130–191, Lecture Notes in Math., vol. 1478, Springer-Verlag, Berlin, 1991. MR 93h:05190 [SS] B. E. Sagan and R. P. Stanley, Robinson-Schensted algorithms for skew tableaux,J.Com- bin. Theory, Ser. A 55 (1990), 161–193. MR 86i:05011 [S] R.P.Stanley,On the number of reduced decompositions of elements of Coxeter groups, European J. Combin. 5 (1984), 359–372. MR 86i:05011 [V] A. M. Vershik, Local stationary algebras, Amer. Math. Soc. Transl. (2) 148 (1991), 1–13. MR 92b:16060

Department of Mathematics, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139-4307 E-mail address: [email protected] Department of Mathematical Sciences, University of Tokyo, Komaba, Meguro-ku, Tokyo 153, Japan E-mail address: [email protected]

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use