Journal of Combinatorial Theory, Series A TA2842 Journal of Combinatorial Theory, Series A 82, 74111 (1998) Article No. TA972842 A Random q, t-Hook Walk and a Sum of Pieri Coefficients A. M. Garsia* and M. Haiman* Department of Mathematics, University of California, San Diego, La Jolla, California 92093-0001 Communicated by the Managing Editors Received September 18, 1996 This work deals with the identity B+(q, t)=& Ä + c+&(q, t), where B+(q, t) denotes a$(s) l$(s) the biexponent generator of a partition +. That is, B+(q, t)=s # + q t , with a$(s) and l$(s) the co-arm and co-leg of the lattice square s in +. The coefficients c+&(q, t) are closely related to certain rational functions occuring in one of the Pieri rules for the Macdonald polynomials and the symbol & Ä + is used to indicate that the sum is over partitions & which immediately precede + in the Young lattice. This identity has an indirect manipulatorial proof involving a number of deep identities established by Macdonald. We show here that it may be given an elementary probabilistic proof by a mechanism which emulates the GreeneNijehuisWilf proof of the hook formula. 1998 Academic Press INTRODUCTION Given a partition + we shall represent it as customary by a Ferrers diagram. We shall use the French convention here and, given that the parts of + are +1+2}}}+k>0, we let the corresponding Ferrer's diagram have +i lattice squares in the ith row (counting from the bottom up). We shall also adopt the Macdonald convention of calling the arm, leg, co-arm, and co-leg of a lattice square s the parameters a(s), l(s), a$(s) and l$(s), giving the number of cells of + that are respectively strictly East, North, West, and South of s in +. We recall that Macdonald in [13] defines the symmetric function basis [P+(x; q, t)]+ as the unique family of polynomials satisfying the following conditions (a) P*=S*+ +<* S+ !+*(q, t) (b) (P* , P+) q, t=0 for *{+, * Work by the authors was carried out under NSF grant support. 74 0097-3165Â98 25.00 Copyright 1998 by Academic Press All rights of reproduction in any form reserved. File: DISTL2 284201 . By:CV . Date:16:03:98 . Time:08:24 LOP8M. V8.B. Page 01:01 Codes: 3885 Signs: 2052 . Length: 50 pic 3 pts, 212 mm RANDOM q, t-HOOK WALK 75 where ( , ) q, t denotes the scalar product of symmetric polynomials defined by setting for the power basis [p\] \i 1&q (1) (2) z\ ` if \ =\ =\ \i (p\ (1) , p\ (2)) q, t= i 1&t {0 otherwise, where z\ is the integer that makes n!Âz\ the number of permutations with cycle structure \. Macdonald shows that the basis [Q*(x; q, t)]+, dual to [P+(x; q, t)]+ with respect to this scalar product, is given by the formula Q*(x; q, t)=d*(q, t) P*(x; q, t), where h*(q, t) d*(q, t)= h$*(q, t) and a*(s) l*(s)+1 a*(s)+1 l*(s) h*(q, t)=` (1&q t ), h$*(q, t)=` (1&q t ) (I.1) s # * s # * Macdonald sets J+(x; q, t)=h+(q, t) P+(x; q, t)=h$+(q, t) Q+(x; q, t), (I.2) and then defines his q, t-analogues of the Kostka coefficients by means of an expansion that in *-ring notation may be written as J+(x; q, t)=: S*[X(1&t)] K*+(q, t). (I.3) * This note is concerned with the modified basis [H +(x; q, t)]+ defined by setting H +(x; q, t)=: S*(x) K *+(q, t), (I.4) * where we have set n(+) K *+(q, t)=K*+(q,1Ât) t (I.5) with n(+)= : l+(s). s # + File: DISTL2 284202 . By:CV . Date:16:03:98 . Time:08:24 LOP8M. V8.B. Page 01:01 Codes: 2319 Signs: 933 . Length: 45 pic 0 pts, 190 mm 76 GARSIA AND HAIMAN To this date it is still an open problem to prove that the K*+(q, t) (and the K *+(q, t) as well) are polynomials with positive integer coefficients. In [2] we have conjectured that H +(x; q, t) is in fact (for a given + &| n) the bivariate Frobenius characteristic of a certain Sn -module H+ yielding a bigraded version of the left regular representation of Sn . In particular this would imply that the expression F+(q, t)=: f* K *+(q, t) * should be the Hilbert series of H+ . Here, f* denotes the number of standard tableaux of shape *. Since Macdonald proved that K*+(1, 1)=f* , (I.6) we see that we must necessarily have 2 F+(1, 1)=: f *=n! (I.7) * According to our conjectures in [2] the polynomial H (x; q, t) p1 + should give the Frobenius characteristic of the action of Sn&1 on H+ . Using the fact that the operator is in a sense1 dual to multiplication by the p1 elementary symmetric function e1 , we can transform one of the Pieri rules given by Macdonald in [14] into the expansion of H (x; q, t) in terms p1 + of the polynomials H &(x; q, t) whose index & immediately precedes + in the Young partial order. More precisely, we obtain H (x; q, t)= : c (q, t) H (x; q, t) (I.8) p1 + +& & & Ä + with tl+(s)&qa+(s)+1 qa+(s)&tl+(s)+1 c+&(q, t)= ` ` , (I.9) tl&(s)&qa&(s)+1 qa&(s)&tl&(s)+1 s # R+Â& s # C+Â& where R+Â& (resp. C+Â&) denotes the set of lattice squares of & that are in the same row (resp. same column) as the square we must remove from + to 1 This will be made precise in section 1. File: DISTL2 284203 . By:CV . Date:16:03:98 . Time:08:24 LOP8M. V8.B. Page 01:01 Codes: 2786 Signs: 1448 . Length: 45 pic 0 pts, 190 mm RANDOM q, t-HOOK WALK 77 obtain &. This given, an application of n&1 to both sides of (I.8) yields the p1 recursion F+(q, t)= : c+&(q, t) F&(q, t), (I.10) & Ä + which together with the initial condition F(1)(q, t)=1 permits the computa- tion of extensive tables of F+(q, t). Of course, all the data so obtained not only confirm the polynomiality and positive integrality of the coefficients of F+(q, t) but exhibit some truly remarkable symmetries under various trans- formations of the variables +, q, and t. The temptation is strong to try and deduce some of these properties directly from the recursion in (I.10). In particular, we should want to construct a pair of statistics :+(_), ;+(_)on permutations _ # Sn yielding :+(_) ;+(_) F+(q, t)= : q t . (I.11) _ # Sn Unfortunately, the complexity of the coefficients c+&(q, t) turns this into an arduous task. The present work results from a systematic effort to understand as much as possible about the mechanism which results in the positive polynomiality of F+(q, t) in spite of the intricate rationality of the recursion. The idea that a ``hook walk'' of sorts is involved here stems from noting what takes place if we successively make the substitutions t Ä 1Ât and then t Ä q. To this end, setting n(+) G+(q)=(F+(q,1Ât) t )|t Ä q , routine manipulations yield that the recursion in (I.10) becomes G+(q)= : #+&(q) G&(q), (I.12) & Ä + with 1&qh+(s) 1&qh+(s) #+&(q)= ` ` 1&qh&(s) 1&qh&(s) s # R+Â& s # C+Â& where h+(s)=l+(s)+a+(s)+1 and h&(s)=l&(s)+a&(s)+1. However, now these coefficients #+&(q) may be given a very revealing form. Indeed, since when s is not in R+Â& or C+Â& we have h+(s)=l+(s)+a+(s)+1=l&(s)+a&(s)+1=h&(s), File: DISTL2 284204 . By:CV . Date:16:03:98 . Time:08:24 LOP8M. V8.B. Page 01:01 Codes: 2766 Signs: 1602 . Length: 45 pic 0 pts, 190 mm 78 GARSIA AND HAIMAN we may write h (s) 1&qh+(s) 1 > (1&q + ) ` = s # + 1&qh&(s) 1&q > (1&qh&(s)) s # R+Â&+C+Â& s # & where the divisor 1&q compensates for the fact that + differs from & by a corner square (of hook length =1). Using the notation 1&qm [m] =1+q+}}}+qm&1= , q 1&q we can finally rewrite the recursion in (I.12) in the form G (q) G (q) + = : & . >s # + [h+(s)]q & Ä + >s # & [h&(s)]q This means that the expression G+(q)Â>s # + [h+(s)]q satisfies the same recursion as the number of of standard tableaux f+ . Since the initial condition is G(1)=1, we deduce that for all partitions + we must have G+(q)=f+ ` [h+(s)]q . (I.14) s # + This identity, which was noted by Macdonald in [14], points out the order of difficulty of finding a pair of statistics yielding (I.11). Indeed, once that is done, the specialization that sends F+(q, t)toG+(q) would deliver a q-analogue of the hook formula. The derivation of (I.14) suggests that the coefficient c+&(q) is some sort of q, t-analogue of the ratio h+ Âh& , where h+ and h& denote the hook products for + and & respectively. This given, the recursion in (I.10) may be viewed as a q, t-analogue of the identity h n!= : + (n&1)!.