Arxiv:2006.02025V2 [Math.QA] 5 Jun 2020 Polynomials

arXiv:2006.02025v2 [math.QA] 5 Jun 2020 polynomials. eea.I atclr ti ayt e htdet that see to easy is it particular, In general. (1.1) httehpreemnn sa nain ne h cino action the under invariant an is hyperdeterminant the that A A eea,i sacalnigpolmt oeu ihrelative GL with of up action come to (non-diagonal) problem the challenging a is it general, alysscn yedtriat[ hyperdeterminant second Cayley’s nrdcdi h otx fqatmgop [ groups quantum of context the in introduced determinant: and rd yefruafrMcoadplnmast generalize to hyperdet polynomials g new Macdonald we the for application, formula that an show type As Trudi will hyperdeterminant. We mat Cayley’s manner. sign deforms nontrivial alternating and with ral permutations replacing by tries) )( srpae by replaced is i e od n phrases. Jing. Naihuan author: and *Corresponding words Key n[ In 2010 o 2 a For A nti ae,w nrdc a introduce we paper, this In 1 q eomto fCye’ hyperdeterminants Cayley’s of Deformation , aao ftehpreemnn o o-omtn matrice non-commuting for hyperdeterminant the of -analog · · · q 7 ucin frcaglrsae eeaiigMatsumoto’s generalizing for functions. shapes formula Jack rectangular Jacobi-Trudi the of appli of an functions generalization As a hypermatrices. formulate even-dimensional for terminant Abstract. aao akldtriathsbe tde n[ in studied been has determinant Hankel -analog ahmtc ujc Classiﬁcation. Subject Mathematics rpriso yedtriat n iciiat r st are discriminants and hyperdeterminants of properties ] det i , 2 m [2 m m = ) ] iesoa hypermatrix dimensional ( A P B = ) om uigCiadNiunJing Naihuan and Cai Wuxing Tommy eitoueadfraino alysscn hyperde- second Cayley’s of deformation a introduce We j n ◦ =1 k n 1 A ! B σ hr h cini endfreape ( example, for deﬁned is action the where , i 1 1 ,...,σ ,j A .Introduction 1. X Hyperdeterminants, 2 ( m ,i j, n ∈ S × · · · × 1 , n λ · · · 5 Y i hpreemnn o omtn en- commuting (of -hyperdeterminant 2 1 =1 of ] m rmr:0E5 eodr:1B9 05E10. 17B69, Secondary: 05E05; Primary: i , sgn( 2 GL A A m o any for ) σ sagnrlzto fteusual the of generalization a is ( = n i . ) 9 Y i ]. =1 n A λ [2 ( A dtriat,Macdonald -determinants, m i 1 ( ] , B ( σ · · · A 1 ( ∈ sivratwhen invariant is ) i ) i , GL oml for formula σ , . . . , Macdonald ain we cation, 8 nainsunder invariants 2 ie nanatu- a in rices m seas [ also (see ] n )) hsmeans This . GL f v Jacobi- a ive h previous the 1 ≤ 2 a been has s m i erminant 1 de in udied ( n ⊗ ,...,i i 2 )) m 2 . B In . m 1 ≤ ◦ ]) n 1 2 TOMMYWUXINGCAIANDNAIHUANJING formula of Matsumoto for Jack polynomials of rectangular shapes. It would be interesting to uncover the relation with q-Hankel determinants. Also the relationship with q-deformations remains mysterious.

2. λ-hyperdeterminants To introduce the λ-hyperdeterminant, we ﬁrst consider generalization of the permutation sign in the usual determinant, and then generalize Cayley’s hyperdeterminants. A permutation matrix is any square matrix obtained by permuting the rows (or columns) of the identity matrix, therefore its determinant is equal to the sign of the corresponding permutation. Each row or column of a ﬁxed permutation matrix has only one nonzero entry along each row and each column. Generalizing the notion of permutation matrices, an alternating sign matrix is a square matrix with entries 0’s, 1’s and (−1)’s such that the sum of each row and column is 1 and the nonzero entries in each row and column alternate in sign. Therefore, each permutation matrix is an alternating sign matrix. However, there are more alternating sign matrices than permutation matrices of the same size. For example, the unique alternating sign 3 × 3 matrix which is not a permutation matrix is

0 1 0 (2.1) Q = 1 −1 1 . 0 1 0 Clearly the nonzero entries in each row or column of an alternating sign matrix must start with 1. We denote by Altn the set of n by n alternating sign matrices and Pn the subset of permutation matrices of size n. The set Pn, endowed with the usual matrix multiplication, is a group isomorphic to the symmetric group Sn. As n increases, Altn contains more and more non-permutation matrices. In fact, it is known [17, 10] that Altn has a cardinality of n−1 (3i+1)! . Qi=0 (n+i)! For X = (xij) ∈ Altn, we deﬁne the inversion number of X by [14]

i(X)= x x . X ij rs r>i,s

If P = (δi,πi ) is the permutation matrix associated to the permutation π ∈ Sn, Then i(P ) counts the number of pairs (i, j) such that i < j but πi >πj, i.e., i(P ) is the inversion number of π, thus explains the name for alternating sign matrices. Let n(X) be the number of negative entries in X. For any n × n matrix X xij xij A = (aij), we write A = a . As usual if xij = 0, a = 1. In general Qi,j ij ij AX is a quotient of two monomials and the degree of the denominator being n(X). DEFORMATION OF CAYLEY’S HYPERDETERMINANTS 3

Let λ be a parameter and A an n × n matrix. Following Robbins and Rumsey [15], we introduce the λ-determinant as follows. det (A)= (−λ)i(X)(1 − λ−1)n(X)AX λ X X∈Altn (2.2) = (−λ)i(π)a · · · a + (−λ)i(X)(1 − λ−1)n(X)AX . X 1π1 nπn X π∈Sn X∈Altn\Pn Note that the second equality holds even for λ =1 due to 00 = 1. We remark that the original λ-determinant [15] corresponds to a sign change of ours, while our deﬁnition deforms the usual determinant at λ = 1. In fact when λ = 1, the second summand vanishes, therefore det1(A) reduces to the usual determinant. In general, detλ(A) is a rational function in the variable ±1 aij or rather a polynomial function in the variable aij . For any 3×3 matrix A = (aij) 3 detλ(A)= a11a22a33 − λa12a21a33 − λ a13a22a31 − λa11a23a32 2 2 a12a21a23a32 + λ a12a23a31 + λ a13a21a32 + λ(1 − λ) , a22 where the last term is due to the alternating sign matrix Q in (2.1). Another example is a λ-deformation of the Vandermonde determinant [15]: For commutating variables xi, 1 ≤ i ≤ n, one has that

(2.3) det (xj−1)= (x − λx ), λ i Y j i 1≤i

Suppose X = (xij) is an alternating sign matrix of size n, we claim that X sends the vector (1, 2, · · · ,n)t to the vector (X(1), X(2),...,X(n))t by 4 TOMMYWUXINGCAIANDNAIHUANJING right matrix multiplication and 1 ≤ X(i) ≤ n. So for each alternating sign matrix X, we deﬁne the generalized permutation (X(1), X(2),...,X(n)) by 1 X(1)     2 X(2) (2.6) X  .  :=  .  ,  .   .      n X(n) where X(i) may not be distinct (cf. (2.5)). Note that (X(1), X(2),...,X(n)) descends to the permutation (π(1),π(2), · · · ,π(n)) when X is the permutation matrix corresponding to π ∈ Sn. For example, (1, 2, 3)Q = (2, 2, 2) for Q in (2.1). Suppose there is xij = −1, then there are odd number of nonzero entries along the ith row and they are alternatively ±1 located at the j1, · · · , j2k+1th columns such that 1 ≤ j1 < j2 < · · · < j2k+1 ≤ n and k ≥ 1. Therefore n X(i)= jx = j − j + j − j + ···− j + j X ij 1 2 3 4 2k 2k+1 j=1

= j1 + (j3 − j2)+ · · · + (j2k+1 − j2k) ≥ k + 1 and n jx = (j − j )+ · · · + (j − j )+ j < j ≤ n. X ij 1 2 2k−1 2k 2k+1 2k+1 j=1 n Hence one always has that 1 ≤ X(i) = jxij ≤ n for any alternating Pj=1 sign matrix X = (xij). For ﬁxed real numbers q and a, we use the conventional q-analogue of a: n−1 (2.7) (a; q)n = (1 − a)(1 − aq) · · · (1 − aq ). With this preparation, we now introduce the notion of λ-hyperdeterminants.

Definition 2.1. Let M = (M(i1, i2,...,i2m))1≤i1,i2,...,i2m≤n be a 2m dimensional hypermatrix, the λ-hyperdeterminant of M is deﬁned by n [2m] (1 − λ) (2.8) detλ (M)= φλ(X1, X2,...,X2m) (λ; λ)n X X1,X2,...,X2m∈Altn n × M(X (i), X (i),...,X (i)), Y 1 2 2m i=1 where the generalized sign factor φλ(X1,...,X2m) is the following expression m (2.9) (−λr−1)i(Xr)(1 − λr−1)n(Xr)(−λr)i(Xm+r )(1 − λr)n(Xm+r), Y r=1 and the generalized permutation (X(1),...,X(n)) for X ∈ Altn was deﬁned in (2.6). DEFORMATION OF CAYLEY’S HYPERDETERMINANTS 5

Note that the λ-hyperdeterminant is different from the q-hyperdeterminant defined on quantum linear group [9]. We also remark that alternating sign matrices with n(X1) > 0 do not contribute to the sum, the first summation variable X1 is therefore always taken as a permutation matrix. Moreover, one can even fix the first indices to be 1, 2,...,n and drop the quantum factor. We now explain how the λ-hyperdeterminant deforms the Cayley hyperdeterminant. Proposition 2.2. For any 2m-dimensional hypermatrix

A = (ai1,...,i2m )1≤i1,...,i2m≤n one has that [2m] [2m] (2.10) lim detλ (A) = det (A). λ→1 Proof. When λ = 1 the summands in the right side of (2.8) (omitting the factor) vanish if one of Xi’s has at least one negative entry. Thus only permutation matrices contribute to the sum, so we can let Xr = Pσr = ∈ S · · · n (δjσr(i)) n for r = 1, , 2m. Subsequently Xr(i) = j=1 jXr(i, j) = n P jδjσ i = σr(i), and we see that the right side of (2.8) matches exactly Pj=1 r ( ) with that of (1.1).

3. Hyperdeterminant formula for Macdonald functions We now generalize Matsumoto’s hyperdeterminant formula for Jack polynomials [13] as an application of the λ-hyperdeterminant. Recall that the classical Jacobi-Trudi formula expresses the Schur function associated to partition µ as a determinant of simpler Schur functions of row shapes:

(3.1) sµ(x) = det(sµi−i+j(x)). In [13], Matsumoto gave a simple formula for Jack functions of rectangular shapes using the hyperdeterminant (see also [1]). Proposition 3.1. [13] Let k,s,m be positive integers. The rectangular −1 Jack functions Q(ks)(m ) can be expressed compactly as follows. (sm)! −1 [2m] m (3.2) Q ks (m )= s!det (Qk i i )1≤i1,...,i2m≤s. (m!)s ( ) +Pj=1( m+j − j )

As the Schur function sµ is the specialization Qµ(1) of the Jack function, one sees immediately that this formula specializes to the Jacobi-Trudi formula for rectangular Schur functions. Macdonald symmetric functions Qµ(q,t) [12] are a family of orthogonal symmetric functions with two parameters q,t and labeled by partition µ. In this paper we consider the case that t = qm, m any positive integer. When m q approaches to 1, Qµ(q,q ) specializes to the Jack symmetric function −1 Qµ(m ) [12] which has many applications (eg. [2]). When the partition 6 TOMMYWUXINGCAIANDNAIHUANJING

s −1 µ = (k ), the symmetric function Qµ(q,t) or Qµ(m ) is referred as the rectangular (shaped) Macdonald function. We now state our main result.

Theorem 3.2. Let k,s,m be positive integers. Up to a scalar factor, m the rectangular Macdonald function Q(ks)(q,q ) is a 2m-dimensional λ- hyperdeterminant:

(q; q)sm m s Q(ks)(q,q ) (q; q)m (q; q)s [2m] m m (3.3) = det (Qk i i (q; q ))1≤i1,...,i2m≤s, (1 − q)s q +Pr=1( m+r − r)

[2m] where detq is the λ-hyperdeterminant with λ = q.

Remark 3.3. When q = 1, one recovers the hyperdeterminant formula (3.2) for Jack functions of rectangular shapes. The formula also gives another expression of the general formula of Lassalle-Schlosser [11] (see also [16]) in the special shape.

To prepare its proof, we ﬁrst recall the following result from [4].

Proposition 3.4. Let ρ = (ks) be an rectangular partition with k,s > (q;q)sm m 0. Let m be a positive integer. Then s Qρ(q,q ) is the coeﬃcient of (q;q)m k k k z1 z2 · · · zs in the Laurent polynomial F G, where

−1 −1 (3.4) F = zizj ; q qzi zj; q Y m m 1≤i

We can now prove Theorem 3.2.

Proof. Recall that the q-Dyson Laurent polynomial (3.4) is a generalization of the Vandemonde polynomial. Using (2.3), we can rewrite the DEFORMATION OF CAYLEY’S HYPERDETERMINANTS 7 q-Dyson Laurent polynomial in the zi using λ-determinants.

−1 −1 F = zizj ; q qzjzi ; q Y m m 1≤i

×  (−qr)i(X)(1 − qr)n(X) z(1−j)X(i,j) X Y i X∈Alts i,j  m = (−qr−1)i(Xr )(−qr)i(Xm+r )(1 − qr−1)n(Xr)(1 − qr)n(Xm+r) X Y X1,...,X2m∈Alts r=1

s s m P P j(X (i,j)−X + (i,j)) × z j=1 r=1 r m r Y i i=1 s s m P P j(Xr(i,j)−X + (i,j)) = φ (X , · · · , X ) z j=1 r=1 m r X q 1 2m Y i X1,...,X2m∈Alts i=1 where in the last equality we have used the fact that the sum of entries in each row of an alternating sign matrix is one. By Proposition 3.4, (q;q)sm m k k k s Q ks (q,q ) is the coeﬃcient of z z · · · z in (q;q)m ( ) 1 2 s

s s m P P j(Xr(i,j)−X + (i,j)) F G = [φ (X , · · · , X ) z j=1 r=1 m r X q 1 2m Y i X1,...,X2m∈Alts i=1 s ∞  Q (q,qm)zj], Y X j i i=1 j=0  and this coeﬃcient is seen as s m φq(X1, · · · , X2m) Qk s m j X i,j X i,j (q,q ). X Y −Pj=1 Pr=1 ( r( )− m+r( )) X1,...,X2m∈Alts i=1

We now show formula (3.3) holds. The entries of matrix M in Deﬁni- m tion 2.1 is now given by M(i , i ,...,i )= Q m (q,q ). Then 1 2 2m k+Pr=1(im+r−ir) 8 TOMMYWUXINGCAIANDNAIHUANJING we have that s M (X (i), X (i),...,X (i)) Y 1 2 2m i=1 s s s = M  jX (i, j), · · · , jX (i, j) Y X 1 X 2m i=1 j=1 j=1  s m = Q m s s (q,q ) Y k+Pr=1(Pj=1 jXm+r(i,j)−Pj=1 Xr(i,j)) i=1 s m = Qk m s j X i,j X i,j (q,q ), Y +Pr=1 Pj=1 ( m+r( )− r( )) i=1 which then implies the formula (3.3).

Acknowledgments NJ gratefully acknowledges the partial support of Simons Foundation grant no. 523868, Humbolt Foundation, NSFC grant no. 11531004 and MPI for Mathematics in the Sciences in Leipzig during this work.

References [1] H. Belbachir, A. Boussicault, J.-G. Luque, Hankel hyperdeterminants, rectangular Jack polynomial and even powers of the Vandermonde, J. Algebra 320 (2008), 3911– 3925. [2] W. Cai, N. Jing, On vertex operator realizations of Jack functions, J. Alg. Combin. 32 (2010), 579–595. [3] T. W. Cai, N. Jing, Jack vertex operators and realization of Jack functions, J. Alg. Combin. 39 (2014), 53–74. [4] T. W. Cai, Macdonald symmetric functions of rectangular shapes, J. Combin. Th. A 128 (2014), 162–179. [5] A. Cayley, On the theory of determinants, Collected Papers, vol. 1, Cambridge Univ. Press, Cambridge, 1889, pp. 63–80. [6] F. J. Dyson, Statistical theory of the energy levels of complex systems I, J. Math. Phys. 3 (1962), 140–156. [7] I. M. Gelfand, M. M. Kapranov, A. V. Zelevinsky, Discriminants, resultants and multidimensional determinants, Birkhäuser, Boston, 1994. [8] M. Ishikawa, H. Tagawa, J. Zeng, A q-analogue of Catalan Hankel determinants. New trends in combinatorial representation theory, 19–41, RIMS Kôkyûroku Bessatsu, B11, Res. Inst. Math. Sci. (RIMS), Kyoto, 2009. [9] N. Jing, J. Zhang, Quantum hyperdeterminants and hyper-Pfaffians, Math. Zeit. 287 (2017), No. 3-4, 897-914. [10] G. Kuperberg, Another proof of the alternating-sign matrix conjecture, Internat. Math. Res. Notices (1996) No. 3, 139–150. [11] M. Lassalle, M. J. Schlosser, Inversion of the Pieri formula for Macdonald polynomials, Adv. Math. 202 (2006), 289–325. [12] I. G. Macdonald, Symmetric functions and Hall polynomials, 2nd ed., Oxford Uni- versity Press, New York, 1995. [13] S. Matsumoto, Hyperdeterminantal expressions for Jack functions of rectangular shapes, J. Algebra 320 (2008), 612–632. DEFORMATION OF CAYLEY’S HYPERDETERMINANTS 9

[14] W. H. Mills, D. Robbins, H. Rumsey, Alternating sign matrices and descending plane partitions, J. Combin. Thoery A, Ser. A 34 (1983), 340-359. [15] D. Robbins, H. Rumsey, Determinants and alternating sign matrices, Adv. Math. 62 (1986), 169–184. [16] E.-A. Williams, A formula for N-row Macdonald polynomials, J. Alg. Combin. 21 (2005), 111–130. [17] D. Zeilberger, Proof of the alternating sign matrix conjecture, Electron. J. Combin. 3 (1996), research paper 13, 84 pp.

Department of Mathematics, University of Manitoba, Winnipeg, MB R3T 2N2 Canada E-mail address: [email protected]

Department of Mathematics, North Carolina State University, Raleigh, NC 27695-8205, USA E-mail address: [email protected]