Howe Duality for Lie Superalgebras

Compositio Mathematica 128: 55^94, 2001. 55 # 2001 Kluwer Academic Publishers. Printed in the Netherlands.

SHUN-JEN CHENG1* and WEIQIANG WANG2 1Department of Mathematics, National Taiwan University, Taipei, Taiwan. e-mail: [email protected] 2Department of Mathematics, North Carolina State University, Raleigh, NC 27695-8205, U.S.A. e-mail: [email protected] (Received: 23 November 1999; accepted: 21 July 2000)

Abstract. Westudya dual pairofgeneral linear Lie superalgebras in the sense of R. Howe.Wegive an explicit multiplicity-free decomposition of a symmetric and skew-symmetric algebra (in the super sense) under the action of the dual pair and present explicit formulas for the highest-weight vectors in each isotypic subspace of the symmetric algebra.We give an explicit multiplicity-free decomposition into irreducible gl mjn-modules of the symmetric and skew-symmetric algebras of the symmetric square of the natural representation of gl mjn. In the former case, we also ¢nd explicit formulas for the highest-weight vectors. Our work uni¢es and generalizes the classical results in symmetric and skew-symmetric models and admits several applications. Mathematics Subject Classi¢cation (2000). 17 B 67.

Key words. Lie superalgebra, Howe duality, highest-weight vectors.

1. Introduction Howe duality is a way of relating representation theory of a pair of reductive Lie groups/algebras [H1], [H2]. It has found many applications to invariant theory, real and complex reductive groups, p-adic groups and in¢nite-dimensional Lie algebras etc. As an example we consider one of the fundamental cases ^ the gl m; gl n Howe duality. The symmetric algebra S Cm Cn and the skew-symmetric algebra L Cm Cn admit remarkable multiplicity-free decompositions under the natural actions of gl mÂgl n. The highest-weight vectors of gl mÂgl n inside the symmetric algebra are given by products of certain determinants (see (2.2)) and form a free Abelian semi-group while those inside the skew-symmetric algebra are given by Grassmann monomials, cf. [H2], [KV], [GW]. Our present paper is devoted to the study of Howe duality for Lie superalgebras and its applications. It is by now a well-established fact that one should put the Grassmann variables on the same footing as Cartesian variables and hence it is natural to consider the supersymmetric algebra, which is a mixed tensor of symmetric and skew-symmetric algebras. In this paper we give a complete description of the Howe

*Partially supported by NSC-grant 89-2115-M-006-002 of the R.O.C

Downloaded from https://www.cambridge.org/core. IP address: 170.106.35.76, on 02 Oct 2021 at 11:02:20, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1023/A:1017594504827 56 SHUN-JEN CHENG AND WEIQIANG WANG

duality in a symmetric* algebra under the action of a dual pair of general linear Lie superalgebras and ¢nd explicit formulas for the highest-weight vectors inside our symmetric model. A dual pair consisting of a general linear Lie superalgebra and a general linear Lie algebra was discussed in [H1]. We also study in detail some other multiplicity-free actions of the general linear Lie superalgebras as speci¢ed below. Our motivation is manifold. Firstly, our work is motivated by an attempt to unify the Howe duality in the symmetric and skew-symmetric models [H2] which have many differences and similarities. Specialization of our results gives rise to the Howe duality for general linear Lie algebras in both symmetric and skew-symmetric models. Secondly, we are motivated by our study of the duality in the in¢nite- dimensional setup (see the review [W] and references therein) and our work in progress on its generalization to the superalgebra case. We realize that we have to understand the ¢nite-dimensional picture better ¢rst in order to have a more complete description of the in¢nite-dimensional picture. Thirdly, there exists a new type of Howe duality which is of pure superalgebra phenomenon which is treated in [CW]. Let us discuss the contents of the paper. The generalization of Schur duality for the superspace was given by Sergeev in [Se]. For lack of an analog in the super setup of the criterion of multiplicity-free action in terms of the existence of a dense open orbit of a Borel subgroup (cf. [V] and [H2]), we use Sergeev's result to derive the decomposition of the symmetric algebra S Cpjq Cmjn with respect to the action of the sum of two general linear Lie superalgebras gl pjqÂgl mjn.Weseethata representation of gl pjq is paired with a representation of gl mjn parameterized by the same Young diagram. On the other hand one can show that our Howe duality for superalgebras implies Sergeev's Schur duality as well. We also obtain an explicit multiplicity-free decomposition of a skew-symmetric algebra L Cpjq Cmjn as gl pjqÂgl mjn-modules. In particular it follows that gl pjq and gl mjn when acting on S Cpjq Cmjn and respectively on L Cpjq Cmjn are mutual (super)centralizers. A remarkable phenomenon is the complete reducibility of the symmetric model under the action of the dual pair, which is quite unusual for Lie superalgebras. In a purely combinatorial way, Brini, Palareti and Teolis [BPT] were indeed the ¢rst to obtain an explicit decomposition of S Cpjq Cmjn under the action of gl pjqÂgl mjn. In addition, their combinatorial approach exhibits explicit bases parameterized by so-called left (or right) symmetrized bitableuax between two `standard Young diagrams' (see [BPT] for de¢nition). However, Brini et al. did not identify the highest weights for these gl pjqÂgl mjn-modules inside S Cpjq Cmjn. We also obtain an explicit decomposition into irreducible gl mjn-modules of the symmetric algebra S S2Cmjn and respectively skew-symmetric algebra L S2Cmjn

*In this paper we will freely suppress the term super. So in the case when a superspace is in- volved, the terms symmetric, commute, etc., mean supersymmetric, supercommute, etc., unless otherwise specified.

of the symmetric square of the natural representation of gl mjn. These results uni¢es and generalizes several classical results and they can be proved in an analogous way as in the classical case [H2], [GW]. Associated to the Howe duality and the above gl mjn-module decompositions, we obtain, by taking characters, various combinatorial identities involving the so-called hook Schur functions. Being generalization of Schur functions, these hook Schur functions have been studied in [BR]. The decompositions mentioned above in turn provide the representation theoretic realization of the corresponding combinatorial identities. For example, the gl mjn; gl pjq-duality gives rise to a combinatorial identity for the hook Schur functions which generalizes the Cauchy identity for Schur functions, cf. [H2]. Specializations and variations of these combinatorial identities are well known and other proofs can be found in [M]. However it is a much more dif¢cult problem to ¢nd explicit formulas for the highest-weight vectors of gl pjqÂgl mjn-modules inside the symmetric algebra. We ¢rst ¢nd formulas for the highest-weight vectors in the case for q 0(and so for n 0 by symmetry). A main ingredient in the formulas for the highest-weight vectors is given by the determinant of a matrix which involves both Cartesian i i variables xj's and Grassmann variables Z 's of the form: 0 1 1 2 r x1 x1 ÁÁÁ x1 B 1 2 r C B x2 x2 ÁÁÁ x2 C B C B . . . C B . . ÁÁÁ . C B 1 2 r C: B xm xm ÁÁÁ xm C B 1 2 r C B Z Z ÁÁÁ Z C B . . . C @ . . ÁÁÁ . A Z1 Z2 ÁÁÁ Zr

We remark that the rows involving Grassmann variables are the same but the determinant is nonzero (one needs to overcome some psychological barriers). Note that when m r the Grassmann variables disappear and the above determinant reduces to those mentioned earlier which occur in the formulas for highest-weight vectors in the symmetric algebra case of the classical Howe duality. When m 0, the Cartesian variables disappear and the above determinant is equal to (up to a scalar multiple) a Grassmann monomial which shows up in the formulas for highest-weight vectors in the classical skew-symmetric algebra case. We show that the gl pjqÂgl mjn highest-weight vectors form an Abelian semigroup in the case when p m. However, in contrast to the Lie algebra case this semigroup is not free in general. We ¢nd that the generators of the semigroup are given by highest-weight vectors associated to rectangular Young diagrams of length not exceeding m 1. This way we are able to ¢nd explicit formulas for all highest-weight vectors in the case when q 0(orm 0), or p m. In the general case the highest-weight vectors no longer form a semigroup. We ¢nd a nice way to overcome this dif¢culty by introducing some extra variables which,

roughly speaking, help us to reduce the general case to the case p m.Thenweusea simple method to get rid of the extra variables to obtain the genuine highest-weight vectors we are looking for. In contrast to the Howe duality in the Lie algebra setup, it is dif¢cult to check directly the highest-weight condition of the vectors we have obtained. We use instead the multiplicity-free decomposition of the symmetric algebra to get around this dif¢culty. As highest-weight constraint we obtain interesting non-trivial polynomial identities typically involving various minors of a matrix. We also ¢nd explicit formulas for the highest-weight vectors appearing in the gl mjn-module decomposition of S S2Cmjn. These highest-weight vector formulas, which constitute a mixture of determinants and Pfaf¢ans, have somewhat similar features as those found in the Howe duality for the general linear Lie superalgebras. A formula for highest-weight vectors in the decomposition of S Cpjq Cmjn as gl pjqÂgl mjn modules may also be obtained in principle using the combinatorial approach of Brini et al. [BPT], and in this way the highest-weights for these gl pjqÂgl mjn modules can be identi¢ed. However this way one can neither expect to obtain formulas as explicit as ours, nor can one see the semigroup structure of the set of the highest-weight vectors which is the guiding principle for us to ¢nd these vectors. It is also interesting to see whether our results concerning the decomposition of S S2Cmjn (and respectively L S2Cmjn) and the highest-weight vectors in these models may also be obtained with extra insights from the combinatorial approach in [BPT] as well. The plan of the paper goes as follows. In Section 2 we review the classical dual pairs of general linear Lie algebras and Schur duality. In Section 3 we present various multiplicity-free actions for Lie superalgebras and obtain the corresponding symmetric function identities. Section 4 is devoted to the construction of the gl pjqÂgl mjn highest-weight vectors inside S Cpjq Cmjn. More precisely, in Section 4.1, Section 4.2, and Section 4.3, we ¢nd explicit formulas of highest-weight vectors in the case q 0, p m, and the general case, respectively. Finally in Section 5 we construct the gl mjn highest-weight vectors inside S S2Cmjn.

2. The Classical Picture In this section we will review some classical multiplicity-free actions of the general linear Lie algebra. We begin with the classical gl mÂgl n-duality, cf. Howe [H2].

Let l l1; l2; ...; ll be a partition of the integer jljl1 ... ll,where l1 X ... X ll > 0. The integer jlj is called the size, l is called the length (denoted 0 by l l), and l1 is called the width of the partition l.Letl denote the Young diagram obtained from l by transposing. We will often denote l1 by t and write 0 0 0 0 l l1; l2; ...; lt. For example, the Young diagram

2:1

stands for the partition 5; 3; 2; 1 and its transpose is the partition 4; 3; 2; 1; 1.

Given a partition l l1; l2; ...; ll satisfying l W m, we may regard l as a highest-weight of gl m by identifying l with the m-tuple l1; l2; ...; ll; 0; ...; 0 by adding m À l zeros to l. We denote the irreducible ¢nite-dimensional l highest-weight module of gl m by Vm. Consider the natural action of the complex general linear Lie groups GL m and m n m n GL n on the space C C . If we identify C C with Mmn,thespaceofall m Â n matrices, then the actions of GL m and GL n are given by left and right multiplications:

t À1 À1 g1; g2 T g1 Tg2 g1 2 GL m; g2 2 GL n; T 2 Mmn: The Lie algebras gl m and gl n act on Cm Cn accordingly. Denoting by S Cm Cn the symmetric tensor algebra of Cm Cn with an induced action of gl mÂgl n, we have the following multiplicity-free decomposition of S Cm Cn as a gl mÂgl n-module: X m n l l S C C Vm Vn ; l

where the sum above is over Young diagrams l of length not exceeding min m; n. One can ¢nd an explicit formula for the gl mÂgl n highest-weight vectors in this m n decomposition. Let us denote a basis of C by x1; x2; ...; xm and a basis of C by 1 2 n j j x ; x ; ...; x . Then the vectors xi: xi x ,fori 1; ...; m and j 1; ...; n form a basis for Cm Cn so that we may identify S Cm Cn with 1 n 1 n Cx1; ...; x1; ...; xm; ...; xm. Using this identi¢cation the standard Borel subalgebra of gl m is a sum of the Cartan subalgebra generated by

Xn @ xj ; 1 W i W m; i j j1 @xi

and the nilpotent radical generated by

Xn @ xj ; 2 W i W m: iÀ1 j j1 @xi

Similarly the Borel subalgebra of gl n is the sum of the Cartan subalgebra generated

Xm j @ xi j ; 1 W j W n; i1 @xi and the nilpotent radical generated by

Xm @ xjÀ1 ; 2 W j W n: i j i1 @xi

Let ei for i 1; ...; m (respectively e~j for j 1; ...; n) be the fundamental weights corresponding to the Cartan subalgebra of gl m (respectively gl n) above. For 1 W r W min m; n de¢ne 0 1 1 1 1 x1 x2 ÁÁÁ xr B 2 2 2 C B x1 x2 ÁÁÁ xr C Dr: detB . . . . C: 2:2 @ . . . . A r r r x1 x2 ÁÁÁ xr

It is easy to see that Dr is a highest-weight vector for both gl m and gl n and its Pr Pr weights are, respectively, i1 ei and i1 e~i. This weight corresponds to the Young diagram

r m r n m n That is, Dr is the highest-weight vector for L C L C inside S C C ,the tensor product of the rth fundamental representations of gl m and gl n. Let l be a Young diagram as in (2.1) with length not exceeding min m; n.Theset of highest-weight vectors in S Cm Cn form an Abelian semigroup, and the prod-

uct D 0 D 0 ÁÁÁD 0 is a highest-weight vector for the irreducible representation in l1 l2 lt S Cm Cn corresponding to the Young diagram l. On the other hand, the skew-symmetric algebra L Cm Cn admits an induced gl mÂgl n action. Following Howe [H2], we have the multiplicity-free decomposition X m n l l0 L C C Vm Vn ; l where the summation runs over Young diagrams l of length not exceeding m and of width not exceeding n.

j m n Denote by Zi; 1 W i W m; 1 W j W n the standard basis for C C in the consider- ation of skew-symmetric algebra. The highest-weight vector for the gl mÂgl n- l l0 m n module Vm Vn inside L C C is given by

1 2 l1 1 2 l2 1 2 ll Z1Z1 ÁÁÁZ1 Z2Z2 ÁÁÁZ2 ...Zl Zl ÁÁÁZl where l is the length of l. Intimately related to the Howe duality is the Schur duality, which we review below. Consider the standard representation of GL m on Cm. It induces an k m action on the kth tensor power C . Now the symmetric group Sk in k letters acts on kCm in a natural way. These two actions commute and we may thus decompose kCm into a direct sum of irreducible

GL mÂSk-module. Recalling that the irreducible representations of symmetric group Sk admit parameterization by Young diagrams of weight k, Schur duality states that X k m l l C Vm Mk; l where the summation is over Young diagrams l of size k and of length not l exceeding m.HereMk is the irreducible representation of Sk corresponding to the Young diagram l. Further well known examples of a multiplicity-free action of gl m that are of interest to us are as follows: consider the action of gl m on the symmetric square S2Cm and skew-symmetric square L2Cm. We have an induced action on their respective symmetric algebras S S2Cm and S L2Cm. Explicitly, the decomposition of these spaces as gl m-modules is as follows (cf. [H2], [GW]): X 2 m 2l S S C Vm ; 2:3 l l W m

X 2 m 2l0 S L C Vm : 2:4 m l l W 2 Explicit formulas for the highest-weight vectors in either cases are well known (cf.[H2])andaregiveninRemark5.1. One may also consider the decompositions of the skew-symmetric algebra of S2Cm and L2Cm. In order to describe the highest-weights that appear in these decompositions we need a few terminology. The Young diagram associated to the partition l k 1; 1;;...; 1 of length k X 1iscalleda k 1; k-hook. We will sometimes also call this k 1; k-hook a hook of shape k 1; k. Assuming that k > l we may form a new Young diagram by `nesting' the l 1; l-hook inside the k 1; k-hook. The resulting partition of length k is k 1; l 2; 2; ...; 2; 1; ...; 1, where 2 appears l À 1 times and 1 appears k À l À 1 times. Simi-

larly a sequence of hooks of shapes k1 1; k1; ...; ks 1; ks with ki > ki1 for i 1; ...; s À 1 may be nested, and the resulting partition has length k1. In consist- ency with the terminology used we call the partition k; 1; ...; 1 of length k 1 a hook of shape k; k 1 or a k; k 1-hook. Nesting of hooks of shapes

k1; k1 1; ...; ks; ks 1 with ki > ki1 for i 1; ...; s À 1 is done in an analogous fashion. Now we can state the following multiplicity-free decompositions of gl m-modules (cf. [H2], [GW]): X 2 m l L S C Vm; 2:5 l X 2 m m L L C Vm; 2:6 m where l (respectively m) is over all partitions with l l W m (respectively l m W m such that l (respectively m) is obtained by nesting a sequence of k 1; k-hooks (respectively of k; k 1-hooks).

3. Multiplicity-Free Actions of the General Linear Lie Superalgebra Let Cmjn denote the superspace of superdimension mjn. Recall that this means mjn that C is a Z2-graded space, where the even subspace has dimension m and the odd subspace has dimension n.ThespaceoflinearmapsfromCmjn toitselfcanberegardedasthespaceof m nÂ m n matrices with an

induced Z2-gradation, which gives it a natural structure as a Lie superalgebra, denoted by gl mjn. We have a triangular decomposition

gl mjngl mjnÀ1 gl mjn0 gl mjn1,wheregl mjnÆ1 denote the set of strictly upper and lower triangular matrices and gl mjn0 denotes the set of diagonal matrices. Given an m n tuple of complex numbers a1; ...; am; b1; ...; bn, we associate an irreducible gl mjn-module Vmjn of highest weight a1; ...; am; b1; ...; bn (with respect to the standard Borel subalgebra gl mjn0 gl mjn1). It is well known (cf. e.g. [K]) that the module Vmjn is ¢nite-dimensional if and only if a1; ...; am; b1; ...; bn satis¢es the conditions ai À ai1; bj À bj1 2 Z,foralli 1; ...; m À 1andj 1; ...; n À 1. Let Cpjq and Cmjn denote complex superspaces of superdimensions pjq and mjn, respectively. We will now describe a duality between the Lie superalgebras gl pjq and gl mjn. Our starting point is Schur duality for Lie superalgebra gl mjn. Schur duality for the Lie superalgebra gl mjn was studied in [Se]. Below we will recall the main result for the convenience of the reader. Let Cmjn denote the standard gl mjn-module. We may, as in the classical case, consider the kth tensor power kCmjn which admits a natural action of the Lie superalgebra gl mjn.

k mjn On the other hand, the symmetric group Sk acts naturally on C by permutations with appropriate signs (corresponding to the permutations of odd mjn elements in C ). It is easy to check that the actions of gl mjn and Sk commute with each other, cf. [Se] (also see [BR] for a more detailed study).

THEOREM 3.1. (Sergeev). As a gl mjnÂSk-module we have X k mjn l l C Vmjn Mk; l

l where l is summed over Young diagrams of size k such that lm1 W n, Mk is the l irreducible Sk-module parameterized by l, and Vmjn denotes the irreducible 0 0 gl mjn-module with highest weight l1; l2; ...; lm; hl1 À mi; ...; hln À mi,where we denote hlilforl2 Z and hli0,otherwise.

The symmetric algebra S Cpjq Cmjn is by de¢nition equal to the tensor product of the symmetric algebra of the even part of Cpjq Cmjn and the skew-symmetric algebra of the odd part of Cpjq Cmjn. It admits a natural gradation X S Cpjq Cmjn Sk Cpjq Cmjn k X 0

by letting the degree of the basis elements of Cpjq Cmjn be 1. The natural actions of gl pjq on Cpjq and gl mjn on Cmjn induce commuting actions on the kth symmetric algebra Sk Cpjq Cmjn.Indeedgl pjq and gl mjn are mutual centralizers in gl Cpjq Cmjn. We obtain the following theorem by an analogous argument as in [H2].

THEOREM 3.2. The symmetric algebra S Cpjq Cmjn is multiplicity-free as a module over gl pjqÂgl mjn. More explicitly, we have the following decomposition X pjq mjn l l S C C Vpjq Vmjn; l

where the sum is over Young diagrams l satisfying lp1 W q and lm1 W n. Here the l l highest weight of the module Vpjq (respectively Vmjn)isgivenby l1; ...; lp; 0 0 0 0 hl1 À pi; ...; hlq À pi (resp. l1; ...; lm; hl1 À mi; ...; hln À mi). Proof. By the de¢nition of the kth supersymmetric algebra we have

Sk Cmjn Cpjq kCmjn kCpjqDk ;

where Dk is the diagonal subgroup of Sk Â Sk. By Theorem 3.1 we have therefore ! !! X1 X X Dk mjn pjq l l m m S C C Vmjn Mk Vpjq Mk k0 jljk jmjk X1 X l m l m Dk Vmjn Vpjq Mk Mk k0 jljjmjk X1 X l l Vmjn Vpjq Xk0 jljk l l Vmjn Vpjq; l

where l in the previous line is summed over all Young diagrams satisfying the con-

ditions lm1 W n and lp1 W q. The second to last equality follows from the l well-known fact that Mk is a self-contragredient module. &

Remarks 3.1. (1) This theorem (except the explicit formula for the highest weights) was ¢rst obtained in [BPT] in a combinatorial approach. It is also obtained independently recently by Sergeev. (2) When n q 0, we recover the gl p; gl m-duality in the symmetric algebra case. When q m 0 we recover the gl p; gl n-duality in the skew-symmetric algebra case.

(3) One can easily show that the gl mjn; gl k-duality implies the gl mjn; Sk Schur duality (Theorem 3.1), using an argument of Howe (cf. 2.4, [H2]). The next corollary is immediate from Theorem 3.2.

COROLLARY 3.1. The image of the action of the universal enveloping algebras of gl pjq and gl mjn on Sk Cpjq Cmjn are double commutants.

THEOREM 3.3. The skew-symmetric algebra L Cpjq Cmjn is multiplicity-free as a module over gl pjqÂgl mjn. More explicitly, we have the following decomposition X pjq mjn l l0 L C C Vpjq Vmjn; l

0 where the sum is over Young diagrams l satisfying lp1 W q and lm1 W n. Here the l l0 highest weight of the module Vpjq (respectively Vmjn)isgivenby l1; ...; lp; 0 0 0 0 hl1 À pi; ...; hlq À pi (resp. l1; ...; lm; hl1 À mi; ...; hln À mi). Proof. By the de¢nition of the kth skew-symmetric algebra we have

Lk Cpjq Cmjn kCpjq kCmjnDk;

k mjn Dk where Dk is the diagonal subgroup of Sk Â Sk and C isthesubspaceof k mjn C that transforms according to the sign character of Dk. By Theorem 3.1

we have therefore

! !! X1 X X Dk pjq mjn l l m m L C C Vpjq Mk Vmjn Mk k0 jljk jmjk X1 X l m l m Dk Vpjq Vmjn Mk Mk k0 jljjmjk X1 X l l0 Vpjq Vmjn Xk0 jljk l l0 Vpjq Vmjn; l

where l in the previous line is summed over all Young diagrams satisfying the con- 0 ditions lp1 W q and lm1 W n. The second to last equality follows from the following l l well-known facts: Mk is a self-contragredient module and tensoring the module Mk l0 with the sign character yields the module Mk . &

Remark 3.2. Of course it follows from Theorem 3.3 that the image of the action of the universal enveloping algebras of gl pjq and gl mjn on Lk Cpjq Cmjn are also double commutants.

The following corollary turns out to be very useful later on in order to check that a given vector is indeed a highest-weight vector inside S Cpjq Cmjn.

COROLLARY 3.2. Assume a vector v 2 S Cpjq Cmjn has the weight l with respect

to gl pjqÂgl mjn associated to a Young diagram l satisfying lp1 W q and lm1 W n. If v is a highest-weight vector for gl pjq,thenitisforgl mjn as well. Proof. Since v is a highest-weight vector for gl pjq with weight l, it belongs to the subspace W of S Cpjq Cmjn which consists of vectors with weight l annihilated by l the standard Borel in gl pjq. By Theorem 3.2, W is isomorphic to Vmjn as a gl mjn- l module. There exists a unique vector (up to scalar multiple) in Vmjn which has weight l, which is the highest-weight vector. By assumption v has weight l as a gl mjn- module, so it is a highest-weight vector for gl mjn. &

The description of highest-weight vectors of the irreducible gl pjqÂgl mjn- modules in the symmetric algebra turns out to be much more subtle than in the classical Howe duality case and we will deal with this question in Section 4. Next consider the symmetric square S2Cmjn of the natural representation of gl mjn. The following theorem can be proved by an analogous argument as in [H2]. This result was also obtained independently recently by Sergeev. We omit the proof since it is in any case parallel to the proof of Theorem 3.5 below.

THEOREM 3.4. The symmetric algebra of the symmetric square of the natural representation Cmjn of the Lie superalgebra gl mjn is a completely reducible multiplicity- free gl mjn-module. More precisely we have the following decomposition X k 2 mjn l S S C Vmjn; l

where the summation is over all partitions l into even parts of size 2kandlm1 W n.

Now S2Cmjn reduces to S2Cm inthecasewhenn 0, and to L2Cn inthecasewhen m 0, the symmetric and skew-symmetric square of the natural representation of gl m and gl n, respectively. Thus one obtains as a corollary the classical multiplicity-free decompositions of their respective symmetric algebras, namely (2.3) and (2.4). Again the question of obtaining explicit formulas for the highest-weight vectors inside S S2Cmjn is substantially more subtle than in the nonsuper case. We will give these in Section 5.

THEOREM 3.5. The skew-symmetric algebra of the symmetric square of the natural representation Cmjn of the Lie superalgebra gl mjn is a completely reducible multiplicity-free gl mjn-module. More precisely we have the following decomposition X k 2 mjn l L S C Vmjn; l

where the summation is over all partitions l of size 2k, which are obtained by nesting

l 1; l-hooks with lm1 W n. Proof. Our argument follows closely the one given in the proof of Theorem 4.4.2 in

[H2] with Theorem 3.1 replacing the classical Schur duality. Let Dk denote the the subgroup of S2k, which preserves the partition ff1; 2g; f3; 4g; ...; f2k À 1; 2kgg of k 2k. Note that Dk is isomorphic to a semidirect product of Sk and Z2 ,where Z2 acts by interchanging 2j À 1with2j and Sk acts by permuting the pairs. Let k sign denote the character on Dk which is trivial on Z2 , but transforms by the sign character on Sk. We observe that

O2k L2k S2Cmjn CmjnDk;sign:

Thus using Theorem 3.1, we obtain X X 2k 2 mjn l l Dk;sign l l Dk;sign L S C Vmjn M2k Vmjn M2k : jlj2k jlj2k

l Dk;sign Now by Theorem A1.4 of [H2] the space M2k is nonzero if and only if l is constructed from nesting hooks of types l 1; l, in which case it is one- dimensional. &

Similarly we obtain as a corollary the classical multiplicity-free decompositions (2.5) and (2.6).

l Remark 3.3. The character of Vmjn is de¢ned as the trace of the action of the diag- l onal matrix diag x1; ...; xm; y1; ...; yn in gl mjn on Vmjn and according to [BR] is given by so-called hook Schur functions HSl x; y (see [BR] for de¢nition). Thus, comparing the characters of both sides of Theorem 3.2 and Theorem 3.3,

respectively, with x x1; ...; xp, y y1; ...; yq, u u1; ...; um and v v1; ...; vn we obtain the following combinatorial identities:

X Y À1 À1 HSl x; yHSl u; v 1 À xiuk 1 À yjvl 1 xivl 1 yjuk; Xl i;Yj;k;l À1 À1 HSl x; yHSl0 u; v 1 xiuk 1 yjvl 1 À xivl 1 À yjuk ; l i;j;k;l

where 1 W i W p,1W j W q,1W k W m and 1 W l W n with summation in the ¢rst

identity over l such that lp1 W q and lm1 W n and in the second one over l such 0 that lp1 W q and lm1 W n.Nowputtingy v 0 in the ¢rst identity we obtain the classical Cauchy identity, while putting respectively y u 0 the dual Cauchy identity (see, e.g., [M]):

X Y À1 sl xsl y 1 À xiyj ; Xl Yi;j sm xsl0 y 1 xiyj: m i;j

Remark 3.4. Similarly Theorem 3.4 and Theorem 3.5 give rise to the following

combinatorial identities (x x1; ...; xm and y y1; ...; yn):

X Y Y À1 À1 HSl x; y 1 À xixi0 1 À yjyj0 1 xiyj; l i W i0;j

where in the ¢rst identity the sum is over all partitions l with even rows such that

lm1 W n and in the second over all partitions m that can be obtained by nesting 0 0 k 1; k-hooks such that mm1 W n and 1 W i; i W m,1W j; j W n. Putting either x 0ory 0 in these two identities we obtain the following classical Schur function identities (see e.g. [M]), which correspond to the decompositions in (2.3), (2.4), (2.5)

and (2.6), respectively: X Y Y 2 À1 À1 s2l 1 À xi 1 À xixj ; l l W m 1 W i W m 1 W i

where r (respectively p) above is summed over all nested sequences of hooks of shape k 1; k with k W m (respectively of hooks of shape k; k 1 with k W m À 1).

4. Construction of Highest-Weight Vectors in S Cpjq Cmjn This section is devoted to the construction of the highest-weight vectors of gl pjqÂgl mjn inside the symmetric algebra of Cpjq Cmjn. We will divide this section into several cases. Before we embark on this task we will set the notation to be used throughout this section. We let e1; ...; ep; f 1; ...; f q denote the standard homogeneous basis for the standard gl pjq-module. Here ei are even, while f j are odd basis elements. Similarly

we let e1; ...; em; f1; ...; fn denote the standard homogeneous basis for the standard i j ~ gl mjn-module. The weights of e ; f ; el and fk are denoted by e~i, dj, el and dk, for 1 W i W p,1W j W q,1W l W m and 1 W k W n, respectively. We set

i i j j i i j j xl: el e ; xl: el f ; Zk: fk e ; yk: fk f : 4:1 We will denote by Cx; x; Z; y the polynomial superalgebra generated by (4.1). The commuting pair of gl pjq and gl mjn may be realized as ¢rst-order differential operators as follows (1 W i; i0 W p; 1 W l; l0 W q and 1 W s; s0 W m; 1 W k; k0 W n):

Xm @ Xn @ Xm @ Xn @ xi Zi ; xl yl ; j @xi0 j @Zi0 j l0 j @yl0 j1 j j1 j j1 @xj j1 j 4:2 Xm @ Xn @ Xm @ Xn @ xi Zi ; xl yl ; j l0 j @yl0 j @xi0 j @Zi0 j1 @xj j1 j j1 j j1 j

Xp Xq Xp Xq j @ j @ j @ j @ xs j xs j ; Zk0 j yk0 j ; j1 @xs0 j1 @xs0 j1 @Z j1 @y k k 4:3 Xp @ Xq @ Xp @ Xq @ xj À xj ; Zj À yj : s j s j k j k @xj j1 @Zk j1 @yk j1 @xs j1 s (4.2) spans a copy of gl pjq, while (4.3) spans a copy of gl mjn.

Our Cartan subalgebras of gl pjq and gl mjn are spanned, respectively, by

Xm @ Xn @ Xm @ Xn @ xi Zi ; xl yl j @xi j @Zi j l j @yl j1 j j1 j j1 @xj j1 j and

Xp @ Xq @ Xp @ Xq @ xj xj ; Zj yj ; s j s @xj k j k j j1 @xs j1 s j1 @Zk j1 @yk while the nilpotent radicals are respectively generated by the simple root vectors

Xm @ Xn @ Xm @ Xn @ xiÀ1 ZiÀ1 ; xlÀ1 ylÀ1 ; j @xi j @Zi j l j @yl j1 j j1 j j1 @xj j1 j 4:4 Xm @ Xn @ xp Zp ; 1 < i W p; 1 < l W q j 1 j @y1 j1 @xj j1 j and

Xp Xq Xp Xq j @ j @ j @ j @ xsÀ1 j xsÀ1 j ; ZkÀ1 j ykÀ1 j ; j1 @xs j1 @xs j1 @Z j1 @y k k 4:5 Xp @ Xq @ xj À xj ; 1 < s W m; 1 < k W n: m j m j j1 @Z1 j1 @y1

With these conventions, we may thus identify S Cpjq Cmjn with the polynomial superalgebra Cx; x; Z; y (as gl pjqÂgl mjn-modules).

4.1. HIGHEST-WEIGHT VECTORS: THE CASE q 0 In this section we will describe the highest-weight vectors for gl pÂgl mjn in the symmetric algebra S Cp Cmjn,i.e.q 0 case. The space S Cp Cmjn is identi¢ed with Cx; Z, and (4.4) and (4.5) reduce to

Xm @ Xn @ xiÀ1 ZiÀ1 ; 4:6 j @xi j @Zi j1 j j1 j

Xp Xp Xp j @ j @ j @ xsÀ1 j ; ZkÀ1 j ; xm j ; 4:7 j1 @xs j1 @Zk j1 @Z1

l l respectively. Now by Theorem 3.2 a highest weight representation Vp Vmjn of gl pÂgl mjn appears in the decomposition of Sk Cp Cmjn if and only if l is

of size k and of length at most p such that lm1 W n. We will consider two cases separately, namely m X p and m < p.

Webeginwiththecaseofm X p. Here the condition lm1 W n is an empty condition. So we are looking for homogeneous polynomials of degree k in Cx; Z, annihilated by all vectors of (4.6) and (4.7), and having gl p-andgl mjn-weight 0 l of length not exceeding p.Ifl is such a weight, then li W p, where we recall that 0 0 0 l l1; ...; lt denotes the transpose of l. It is easy to see that the product D 0 ÁÁÁD 0 is annihilated by all vectors of (4.6) and (4.7), where we recall that Dr l1 lt is de¢ned in (2.2). It is straightforward to check that its weight is exactly l.

THEOREM 4.1. InthecasewhenmX p, all gl pÂgl mjn highest-weight vectors in

Cx; Z form an Abelian semigroup generated by Dr,forr 1; ...; p. The highest-weight vector associated to the weight l is given by the product D 0 ÁÁÁD 0 . l1 lt

We now consider the case p > m. In this case the condition lm1 W n is no longer an empty condition. Obviously the highest-weight vectors associated to Young dia-

grams l with lm1 0 can be obtained just as in the previous case. 0 0 0 Now suppose l is a diagram of length exceeding m.Letl1; l2; ...; lt denote its 0 0 0 0 column lengths as usual. We have p X l1 X l2 ... X lt and m X ln1.For m W r W p, the following determinant of an r Â r matrix plays a fundamental role in this paper:

0 1 2 r 1 x1 x1 ÁÁÁ x1 B x1 x2 ÁÁÁ xr C B 2 2 2 C B . . . C B . . . C B . . ÁÁÁ . C B 1 2 r C B xm xm ÁÁÁ xm C Dk;r: detB 1 2 r C; k 1; ...; n: 4:8 B Zk Zk ÁÁÁ Zk C B 1 2 r C B Zk Zk ÁÁÁ Zk C B . . . C @ . . ÁÁÁ . A 1 2 r Zk Zk ÁÁÁ Zk

1 r That is, the ¢rst m rows are ¢lled by the vectors xj ; ...; xj ,forj 1; ...; m,in increasing order and the last r À m rows are ¢lled with the same vector 1 r i Zk; ...; Zk. Since the matrix entries involve Grassmann variables Zk,wemust specify what we mean by the determinant. By the determinant of a matrix

0 1 2 r 1 a1 a1 ÁÁÁ a1 B a1 a2 ÁÁÁ ar C B 2 2 2 C A: B . . . C; @ . . ÁÁÁ . A 1 2 r ar ap ÁÁÁ ar

whose matrix entries involve Grassmann variables Zi ,wewillalwaysmeanthe P k expression À1p sas 1as 2 ÁÁÁas r,wherep s is the length of s in the sym- s2Sr 1 2 r t metric group Sr. In general it is not true that detA detA .

Remark 4.1. The determinant (4.8) is always nonzero. It reduces to (2.2) when 1 r m r, and reduces to (up to a scalar multiple) Zk ÁÁÁZk when m 0.

Now let l be a diagram of length at most p such that lm1 W n. It is thus of the following shape:

0 0 where r is de¢ned by lr > m and lr1 W m. We can divide such a diagram into two diagrams, namely

4:9

Now the second diagram in (4.9) has length not exceeding m, so its associated

highest-weight vector is given by the product D 0 ÁÁÁD 0 . A formula for the lr1 lt highest-weight vector associated to the ¢rst diagram in (4.9) is given by the Qr following proposition. We will denote by D 0 the (ordered) product k1 k;lk D 0 D 0 ...D 0 . 1;l1 2;l2 r;lr

0 0 0 Qr PROPOSITION 4.1. Let p X l X l X ... X l > m. Then D 0 is a highest- 1 2 r k1 k;lk weight vector associated to the ¢rst Young diagram in (4.9). Qr Proof. Observe that k1 Dk;l0 has the same gl pÂgl mjn-weight as the ¢rst Qk r Young diagram of (4.9). Clearly k1 Dk;l0 is nonzero. It is straightforward to verify Qr k that k1 Dk;l0 is annihilated by the operators in (4.6). It follows from Corollary 3.2 Qr k that D 0 is also a highest-weight vector for gl mjn. & k1 k;lk

Our next theorem follows by observing that the product of the highest-weight vectors corresponding to the two Young diagrams in (4.9) is nonzero and is a highest-weight vector associated to the Young diagram l.

THEOREM 4.2. Suppose that m < p. An irreducible highest-weight module l l Vp Vmjn appearing in Cx; Z if and only if l corresponds to a Young diagram l of length not exceeding p and lm1 W n. Furthermore a highest-weight vector associated to such a l is given by

Yr Yt D 0 D 0 ; k;lk lj k1 jr1

0 0 where r is de¢ned by lr > m and lr1 W m.

As a corollary we obtain the following useful combinatorial identity, which will play an important role later on.

i COROLLARY 4.1. Let xl be even variables for i 1; ...; pandl 1; ...; mwith i i p X q > m. Let Z1 and Z2 be odd variables for i 1; ...; p. Then 0 1 1 2 p x1 x1 ÁÁÁ x1 0 q 1 p 1 2 B 1 2 C x1 x1 ÁÁÁ x1 B x2 x2 ÁÁÁ x2 C q B C B x1 x2 ÁÁÁ x C B . . . C B 2 2 2 C B . . ÁÁÁ . C B . . . C B 1 2 p C B . . ÁÁÁ . C B xm xm ÁÁÁ xm C B C p B 1 2 q C B 1 2 C B xm xm ÁÁÁ xm C detB Z1 Z1 ÁÁÁ Z1 C Á det q 0: B p C B Z1 Z2 ÁÁÁ Z C B Z1 Z2 ÁÁÁ Z C B 1 1 1 C B 1 1 1 C B Z1 Z2 ÁÁÁ Zq C B . . . C B 2 2 2 C B . . ÁÁÁ . C B . . . C B . . . C @ . . ÁÁÁ . A @ . . . A . . ÁÁÁ . Z1 Z2 ÁÁÁ Zq 1 2 p 2 2 2 Z1 Z1 ÁÁÁ Z1

Proof. Consider D1;p and D2;q where p X q. By Theorem 4.2 the product D1;pD2;q is a highest-weight vector and thus is annihilated by all operators in (4.7). In particular Pp j j applying the operator j1 Z1 @=@Z2 to D1;pD2;q and dividing by q À m we obtain the desired identity. &

Remark 4.2. The above corollary gives rise to identities involving minors in even variables xs by looking at the coef¢cient of a ¢xed Grassmann monomial involving Zs. We do not know of other direct proof of these identities.

It is well known (cf. [OV]) that as a gl p-module Si Cp Lj Cp,forj 1; ...; p, decomposes into a direct sum two irreducible components of highest weights Pj Pj1 ie1 k1 ek and ie1 k2 ek, respectively. We can also get this result from Theorem 4.2 and in addition obtain explicit formulas of the highest-weight vectors. P k p 1j1 k pjp i pj0 j 0jp To do so consider S C C S C ijk S C L C .Now according to Theorem 4.2 all the gl pÂgl 1j1 highest-weight vectors inside k p 1j1 1 i S C C are given by x1 D1;j,wherej 1; ...; p and i j k. These vectors are of course gl p highest-weight vectors. Now a simple calculation shows that

1 i applying the negative root vector of gl 1j1 to x1 D1;j we obtain a nonzero multiple 1 i 1 2 j of x1 Z1Z1 ...Z1, while applying the negative root vector again gives zero. Thus the vectors x1iD and x1iZ1Z2 ...Zj exhaust all gl p highest-weight vectors inside P 1 1;j 1 1 1 1 the space Si Cp Lj Cp. To conclude the proof we observe that the vectors ijk P 1 iÀ1 1 i 1 2 j i p j p j x1 D1;j1 and x1 Z1Z1 ...Z1 lie in S C L C ,withweightsie1 k1 ek Pj1 and ie1 k2 ek, respectively.

4.2. HIGHEST-WEIGHT VECTORS: THE CASE p m In this section we shall ¢nd gl mjqÂgl mjn highest-weight vectors that appear in the decomposition of Cx; x; Z; y. By Theorem 3.2 we need to construct a vector in Cx; x; Z; y annihilated by all operators in (4.4) and (4.5) of weight corresponding to the Young diagram l

4:10

where r W min q; n (which we will always assume for this section). First we remark that if l has length less than or equal to m then it is easy to check that a formula for the corresponding highest-weight vector is given by

D 0 ÁÁÁD 0 . So we may assume that the length of l exceeds m. l1 lt As before we cut up this Young diagram into two diagrams, namely

4:11

Denoting the second diagram by m and v a highest-weight vector associated to the

¢rst diagram, it is easy to see that the product vDm0 ÁÁÁDm0 is a highest-weight vector 1 tÀr for the diagram l. Thus our task reduces to ¢nding a highest-weight vector associated to the ¢rst diagram in (4.11). We claim that a highest-weight vector associated to the ¢rst diagram in (4.11) can be essentially obtained by taking a product of those associated to s diagrams of

rectangular shape

4:12

and dividing by a suitable power of Dm. Indeed taking the product of two highest-weight vectors for the Young diagram of shape (4.12) of widths lmi and lmi1 respectively produces a highest-weight vector for the Young diagram

lmi1 Once we verify that the product is nonzero, we may divide it by Dm and the resulting vector is a highest-weight vector for the diagram

Similarly by taking a product of s such vectors associated to the s diagrams of the lm2...lms form (4.12) of widths lm1; ...; lms, respectively, and dividing by Dm we obtain a highest-weight vector associated to the ¢rst diagram of (4.11). So our task now is to ¢nd a formula for a highest-weight vector corresponding to a Young diagram of shape (4.12). (From the explicit formula it will follow immediately that a product of s vectors of such type is nonzero.)

Let us put r lmi in (4.12). We de¢ne the r Â r matrix Y and the m Â m matrix X as follows: 0 1 0 1 1 2 r 1 2 m y1 y1 ÁÁÁ y1 x1 x1 ÁÁÁ x1 B 1 2 r C B 1 2 m C B y2 y2 ÁÁÁ y2 C B x2 x2 ÁÁÁ x2 C Y: B . . . C; X: B . . . C: 4:13 @ . . ÁÁÁ . A @ . . ÁÁÁ . A 1 2 r 1 2 m yr yr ÁÁÁ yr xm xm ÁÁÁ xm

Given a Young diagram l of rectangular shape (see (4.12)) consisting of m rows and r columns, we consider marked diagrams D obtained by marking the boxes in l subject to the restriction that each column can contain no more than one marked box. For example the following is a marked diagram in the case r 6andm 4:

: 4:14

To each such a marked diagram D we may associate an r Â r matrix YD obtained from Y as follows. For each marked box, say in the ith column and jth row, we 1 2 r replace the ith row of the matrix Y by the vector xj ; xj ; ...; xj . The resulting matrix will be denoted by YD. For instance in our example (4.14) the matrix YD is 0 1 1 2 3 4 5 6 x1 x1 x1 x1 x1 x1 B 1 2 3 4 5 6 C B y y y y y y C B 1 1 1 1 1 1 C B x1 x2 x3 x4 x5 x6 C Y B 3 3 3 3 3 3 C: D B x1 x2 x3 x4 x5 x6 C B 2 2 2 2 2 2 C @ 1 2 3 4 5 6 A x2 x2 x2 x2 x2 x2 1 2 3 4 5 6 x3 x3 x3 x3 x3 x3

To each such diagram D we may also associate rmÂ m matrices Xi i 1; ...; r obtained from X as follows. If the ith column of D is not marked, then Xi X. If the ith column is marked at the jth row, then Xi is the matrix obtained from 1 2 m X by replacing its jth row by the vector Zi ; Zi ; ...; Zi . As an illustration, the diagram in our example (4.14) gives rise to the matrices 0 1 0 1 1 2 3 4 1 2 3 4 Z1 Z1 Z1 Z1 x1 x1 x1 x1 B 1 2 3 4 C B 1 2 3 4 C B x2 x2 x2 x2 C B x2 x2 x2 x2 C X1 @ 1 2 3 4 A; X2 X; X3 @ 1 2 3 4 A etc: x3 x3 x3 x3 Z3 Z3 Z3 Z3 1 2 3 4 1 2 3 4 x4 x4 x4 x4 x4 x4 x4 x4

Let jDj denote the total number of marked boxes in the diagram D.Set

DD: detXDdetYD;

Qr where by detXD we mean i1 detXi arranged in increasing order. We can now state the following theorem.

P 1 2jDj jDjÀ1 THEOREM 4.3. The vector D À1 DD is a gl mjqÂgl mjn highest-weight vector in Cx; x; Z; y corresponding to the rectangular Young diagram of length m 1 and width r, where the summation over D ranges over all possible marked m Â rdia- grams. P 1 2jDj jDjÀ1 Proof. We ¢rst show that D À1 DD indeed has the correct weight. First note that diagram (4.12) corresponds to the gl mjqÂgl mjn-weight P P P P m m r r ~ ~ i1 rei i1 re~i j1 dj j1 dj.LetDi, j 1; ...; m, denote the m disjoint ~ subsets of f1; ...; rg de¢ned by the condition that j 2 Di if and only if D contains a marked box at its jth column and ith row. Put D~ [m D~ and D~ c P P i1Pi f1; ...; rgÀD~ . The weight of detY is d m jD~ je r d~ .Now P P D j2PD~ c j iP1 i i j1 j detX has weight r m e À m jD~ je d r m e~ . Hence, each D Pi1 i Pi1 i i P j2D~ j P i1 i m m r r ~ detYDdetXD has weight i1 rei i1 re~i j1 dj j1 dj, as required. Hence by Corollary 3.2 it is suf¢cient to show that (4.5) annihilates it, namely ! ! Xm Xq X j @ j @ 1jDj jDjÀ1 x x À12 D 0; 4:15 sÀ1 j sÀ1 j D j1 @xs j1 @xs D

! ! Xm Xq X j @ j @ 1 2jDj jDjÀ1 ZsÀ1 j ysÀ1 j À1 DD 0; 4:16 j1 @Zs j1 @ys D

! ! Xm Xq X @ @ 1 j j 2jDj jDjÀ1 xm j À xm j À1 DD 0: 4:17 j1 @Z1 j1 @y1 D

We will ¢rst establish (4.15). Note that the simple root vector

Xm @ Xq @ xj xj sÀ1 j sÀ1 j j1 @xs j1 @xs

1 m 1 m 1 q 1 q maps the vectors xs ; ...; xs to xsÀ1; ...; xsÀ1 and xs ; ...; xs to xsÀ1; ...; xsÀ1. For a diagram D, let us denote by D sÀ1 a diagram obtained from D by moving each "s marked box in its sth row to the box above it in the s À 1st row. Analogously

we de¢ne D sÀ1 a diagram obtained from D by moving each marked box in the #s s À 1st row to the box below it in the sth row. It is easy to check ! Xm Xq j @ j @ xsÀ1 j xsÀ1 j DD @xs @xs j1 X j1 X detX detY sÀ1 À detX detY : D D s D sÀ1 D " #s D sÀ1 D sÀ1 "s #s

Thus, we have ! ! Xm @ Xq @ X xj xj D sÀ1 j sÀ1 j D j1 @xs j1 @xs jDjk 0 1 4:18 X X X @ detX detY À detX detY A: D D sÀ1 D sÀ1 D "s #s jDjk D sÀ1 D sÀ1 "s #s But evidently X X detX detY detX detY D sÀ1 D D D sÀ1 #s "s jDjk jDjk

thanks to the equality D sÀ1 sÀ1 D. Hence the right-hand side of (4.18) is zero, "s #s proving (4.15). Ournextstepistoprove(4.16).Inthiscase

Xm Xq j @ j @ ZsÀ1 j ysÀ1 j j1 @Zs j1 @ys

1 m 1 m 1 q 1 q maps the vectors Zs ; ...; Zs to ZsÀ1; ...; ZsÀ1 and ys ; ...; ys to ysÀ1; ...; ysÀ1. ~ ~ For a diagram D such that j 2 Di and l 62 Di we denote by Dj!l the diagram obtained ~ ~ from D by removing j from Di and adding l to Di.Fora¢xedk we write 0 1 X X X X @ A DD DD DD : jDjk jDjk s;sÀ12D~ s62D~ or sÀ162D~ First observe that !0 1 Xm Xq X X j @ j @ @ A Z y DD sÀ1 @Zj sÀ1 @yj j1 s j1 s jDjk s;sÀ12D~ !! X X Xm Xq j @ j @ Z y detXD detYD sÀ1 @Zj sÀ1 @yj jDjk s;sÀ12D~ j1 s j1 s 0: ~ This is because if s; s À 1 2 Di,forsomei, then the term ! Xm Xq j @ j @ ZsÀ1 j ysÀ1 j detXD j1 @Zs j1 @ys

ÁÁÁdetXsÀ1 idetXsÀ1 iÁÁÁ 0;

where in general Xb a is the matrix obtained from X by replacing the ath row with 1 m the vector Zb; ...; Zb .

~ ~ Now if s 2 Di and s À 1 2 Dl,then ! Xm Xq j @ j @ ZsÀ1 j ysÀ1 j detXDÁÁÁdetXsÀ1 ldetXsÀ1 i ...: j1 @Zs j1 @ys

0 ~ 0 ~ 0 Let D be the same diagram as D, except s 2 D l and s À 1 2 D i. Then we have ! Xm Xq j @ j @ Z y detX 0 ÁÁÁdetX idetX l ...: sÀ1 j sÀ1 j D sÀ1 sÀ1 j1 @Zs j1 @ys

Of course YD YD0 and detXsÀ1 i anticommutes with detXsÀ1 l,so ! Xm Xq j @ j @ Z y detX detY detX 0 detY 0 0: sÀ1 j sÀ1 j D D D D j1 @Zs j1 @ys

Next we observe that if D is a diagram such that s; s À 1 62 D~ ,then ! Xm @ Xq @ Zj yj detX detY 0; sÀ1 j sÀ1 j D D j1 @Zs j1 @ys

so that our task of proving (4.16) reduces to proving that 0 1 ! Xm Xq X X j @ j @ B C Z y @ detXDdetYD detXDdetYDA; sÀ1 @Zj sÀ1 @yj 4:19 j1 s j1 s s2D~ sÀ12D~ sÀ12=D~ s2=D~ 0;

where the sum is over all diagrams D with jDjk. But the left-hand side of (4.19) is equal to X X detX detY À detX detY 0: Ds!sÀ1 D D DsÀ1!s s2D~ ;sÀ162D~ sÀ12D~ ;s2=D~

To complete the proof we now need to verify (4.17). The odd simple root vector

Xm @ Xq @ xj À xj m j m j j1 @Z1 j1 @y1

1 m 1 m 1 q has the effect of changing the vectors Z1; ...; Z1 to xm; ...; xm and y1; ...; y1 to 1 q À xm; ...; xm.IfD is a diagram such that 1 2 Dj with j 6 m,then ! Xm Xq j @ j @ xm j À xm j Á DD 0: j1 @Z1 j1 @y1

Thus ! ! Xm Xq X j @ j @ 1jDj jDjÀ1 x À x À12 D m j m j D j1 @Z1 j1 @y1 D

! Xm @ Xq @ xj À xj m j m j 4:20 j1 @Z1 j1 @y1

0 1 X X @ 1jDj jDjÀ1 1jDj jDjÀ1 A Â À12 DD À12 DD : ~ ~ D;12Dm D;12=D

~ ~ À For a diagram D with 1 62 D (resp. with 1 2 Dm)wedenotebyD (resp. D )the ~ ~ diagram obtained from D by adding 1 to Dm (resp. by removing 1 from D). Then (4.20) becomes X X 1jDj jDjÀ1 1jDj jDjÀ1jDj À12 detXDÀ detYD À À12 detXDdetYD : 4:21 ~ ~ D;12Dm D;162D Setting D0 DÀ in the ¢rst sum of (4.21) we may rewrite (4.21) as X 1 0 0 2jD j jD j1 0 À1 detXDdetYD0 À D0;12=D~ 0 X 1jDj jDjÀ1jDj À À12 detXDdetYD 0: & D;12=D~

P 1 2jDj jDjÀ1 We will denote the vector D À1 DD by Gr. It is clear that a product of Grs (not necessary for the same value r) remains nonzero. Thus a highest-weight vector for an arbitrary Young diagram of shape (4.10) can be constructed using such vectors, as described earlier in this section. We summarize the results in this section in the following theorem.

l l THEOREM 4.4. An irreducible representation Vmjq Vmjn of gl mjqÂgl mjn appears in the decomposition of Cx; x; Z; y if and only if l is associated to a Young 0 diagram with lm1 W min q; n. Let t be the length of l .Then

(1) ifthe length of ldoes not exceed m, then a highest-weight vector is given by D 0 ÁÁÁD 0. l lt 0 1 (2) if the length of l is m s, s X 1,let0 W r W min q; n be such that lr > mand 0 lr1 W m, then a highest-weight vector corresponding to l is given by

À lm2...lms Dm Gl Gl ÁÁÁGl D 0 ÁÁÁD 0 : 4:22 m1 m2 ms lr1 lt

We will obtain a more explicit formula for (4.22) in the next section.

4.3. HIGHEST-WEIGHT VECTORS: THE GENERAL CASE We consider now the general case. Without loss of generality we may assume p X m. l l According to Theorem 3.2 an irreducible gl pjqÂgl mjn-module Vpjq Vmjn appears in the decomposition of Cx; x; Z; y if and only if lm1 W n and lp1 W q. If the length of the Young diagram l is less than or equal to m,then 0 D 0 ÁÁÁD 0 is the desired highest-weight vector, where t is the length of l . If the length l1 lt of l exceeds m, but is less than or equal to p, then we see that the vector given in Theorem 4.2 provides a formula for the highest-weight vector in this case as well. Thus it remains to study the case when the length of l exceeds p. So we are to consider a Young diagram of the form:

4:23

Inthecasewhenq W n, the numbers r; r0 satisfying the conditions 0 W r W q, 0 0 0 0 0 0 W r W n and r W r are determined as follows: lr > p and lr1 W p, lr0 > m and 0 0 0 lr01 W m. In the case when q X n, the numbers r; r satisfying 0 W r W r W n are de¢ned in exactly the same way. In either case we may split (4.23) into three diagrams:

4:24

We associate the vectors Dr1;l0 ÁÁÁDr0;l0 and Dl0 ÁÁÁDl0 to the second and third r1 r0 r01 t diagrams in (4.24) respectively. Below we will construct a highest-weight vector for the ¢rst diagram in (4.24). From the formula it will be easy to see that the product of these three vectors is a highest-weight vector for the Young diagram (4.23).

The above discussion thus reduces our question to ¢nding a gl pjqÂgl mjn- highest-weight vector corresponding to a Young diagram of type:

4:25

where r W min q; n. The dif¢culty of ¢nding a highest-weight vector associated to such a diagram lies in the fact that the highest-weight vectors in Cx; x; Z; y no longer form a semigroup in general. We will now outline our strategy. We need to ¢nd highest-weight vectors in Cx; x; Z; y, annihilated by (4.5) and having weight corresponding to the Young i diagram in (4.25). Recall that x denotes the set of even variables fxlj1 W i W p; i 1 W l W mg. We introduce a new set of even variables xl; 1 W i W p; m < l W p, 0 i and denote by x fxlj1 W i; l W pg, the union of our old set with this new set. We shall construct certain vectors in Cx0; x; Z; y, which can been shown, using our results in the previous section, that they are annihilated by (4.5). A priori these vectors lie in Cx0; x; Z; y so that such vectors do not make sense. However, we will show that these vectors, after dividing by a suitable power of the determinant of i i the p Â p matrix xl, are in fact independent of the variables fxlj1 W i W p; m 1 W l W pg,andthuslieinCx; x; Z; y. Consider a marked diagram having m rows and r columns with at most r marked boxes subject to the constraint that at most one marked box appears on each column.

To such a diagram D we have associated in the previous section a matrix YD,whichis obtained from the r Â r matrix Y (see (4.13)) by suitably replacing its rows. To each

such diagram we now associate p Â p matrices Xi,for1W i W r, similar to the ones in the previous section: Let X denote the p Â p matrix:

0 1 2 p 1 x1 x1 ÁÁÁ x1 B x1 x2 ÁÁÁ xp C B 2 2 2 C B . . . C B . . . C B . . ÁÁÁ . C B x1 2 p C X: B m xm ÁÁÁ xm C: 4:26 B x1 x2 ÁÁÁ xp C B m1 m1 m1 C B . . . C @ . . ÁÁÁ . A 1 2 p xp x1 ÁÁÁ xp

If the ith column of D is not marked, then Xi X.Iftheith column is marked at the jth row, then Xi is the matrix obtained from X by replacing its jth row by the vector

1 2 p Zi ; Zi ; ...; Zi . Note that none of its m 1st to pth rows are replaced. As before we Qr de¢ne detXD: i1 detXi (arranged in increasing order) and de¢ne Gr P 1 2jDj DjÀ1 D À1 detXDdetYD. The proof of Theorem 4.3 carries over word for word to prove

PROPOSITION 4.2. The vector Gr is annihilated by (4.5).

Inthecasewhenp m a highest-weight vector for the ¢rst Young diagram in (4.11) is obtained essentially by taking the product of highest-weight vectors for the Young diagram of type (4.12). In the case when p > m one can verify that this procedure cannot be carried out, as such products are necessarily zero. Hence in this case we will need to ¢nd a general formula for the Young diagram l of shape (4.25). To do so we will ¢rst consider matrices that will play the same role in

thecaseofp X m as the Xisplayinthecasep m. As we will generalize diagrams to include those that allow more than one marked box on each column, we are led to study combinatorial identities of determinants of matrices obtained from X that have more than one row replaced by an odd vector. This leads us to de¢ne the following types of determinants. 1 p Let X be the p Â p matrix as in (4.26) and let Zj ; ...; Zj be an odd vector. Let I be a subset of f1; ...; pg and de¢ne Xj I to be the matrix obtained from X by replacing 1 p its ith row by the vector Zj ; ...; Zj ,foralli 2 I.IfI fi1; ...; ilg,wewrite Xj IXj i1; ...; il as well.

LEMMA 4.1. We have

1 detX 1detX 2ÁÁÁdetX p detXpÀ1detX 1; ...; p: 4:27 1 1 1 p! 1

Proof. Denoting by R X and L X the right-hand side and the left-hand side of (4.27), respectively, we may regard R and L as functions of X.Sincethegroup GL p acts on X,thespaceofp Â p matrices, by left multiplication, it acts on functions of X.TobemorepreciseifA 2 GL p,then A Á L X: L AÀ1X and A Á R X: R AÀ1X. We want to study the effect of this action on R and L. In order to do so, consider ¢rst the action of the three kinds of elementary matrices on them. Namely, those that interchanges any two rows, that multiplies a row by a scalar, and those that add a scalar multiple of a row to another. It is subject to a direct veri¢cation that if A is any of the three types of elementary matrices, we have

A Á R X detA1ÀpR X; A Á L X detA1ÀpL X: 4:28

Since every element in GL p is a product of elementary matrices, we conclude that

(4.28) holds for every A 2 GL p as well. Putting X 1p, the identity p Â p matrix, we see that R 1pL 1p so that R AL A by (4.28) for all A 2 GL p.As

GL p is a Zariski open set in the space of p Â p matrices we have R XL X for any p Â p matrix X. &

Remark 4.3. An alternative proof of the above lemma can be given as follows. Let pÀ1 Xij denote the i; j-th minor of X. It is known that det XijdetX which follows directly from a form of the Cramer's formula X Xij detX1p.Theabovelemma follows from this identity by expanding each determinant on the left-hand side 1 p of (4.27) by the row Z1; ...; Z1 and noting that detX1 1; ...; p is equal to 1 2 p p! Z1Z1 ÁÁÁZ1.

COROLLARY 4.2. Let I fi1; ...; ijIjg and J fj1; ...; jjJjg be two subsets of f1; ...; pg arranged in increasing order. Then

(i) detX1 IdetX1 J0 if and only if I \ J 6;. 1 jIjÀ1 (ii) detX1 i1detX1 i2ÁÁÁdetX1 ijIjjIj! detX detX1 I: (iii) For I \ J ;we have

jIj!jJj! detX IdetX Je detXdetX I [ J; 1 1 IJ jIjjJj! 1

where eIJ is the sign of the permutation that arranges the ordered tuple i1; ...; ijIj; j1; ...; jjJj in increasing order. Proof. (i) is an obvious consequence of Lemma 4.1. c For (ii) let I fk1; ...; kjI cjg denote the complementary subset of I in f1; ...; pg put in increasing order. We apply successively the differential operators

Xp @ Xp @ Xp @ xj ; xj ; ...; xj k1 j k2 j kjIcj j j1 @Z1 j1 @Z1 j1 @Z1

to (4.27) and ¢nd that

1 detXpÀjIjdetX i ÁÁÁdetX i detXpÀ1detX I: 1 1 1 jIj jIj! 1

Dividing by detXpÀjIj we obtain (ii). By (ii) we have

1 1 detXjIjÀ1 detXjJjÀ1detX IdetX J jIj! jJj! 1 1

detX1 i1ÁÁÁdetX1 ijIjdetX1 j1ÁÁÁdetX1 jjJj:

Thus if eIJ is the permutation arranging I [ J in increasing order, then

1 e detXjIjjJjÀ2detX IdetX J IJ detXjIjjJjÀ1detX I [ J: jIj!jJj! 1 1 jIjjJj! 1

Now (iii) follows from dividing the above equation by detXjIjjJjÀ2 and multiplying by jIj!jJj!. &

Returning to the problem of ¢nding the highest-weight vector associated to the Young diagram (4.25), our ¢rst task is to present a more explicit expression for a product of the form G ÁÁÁG .WeassociatetosuchaYoungdiagramacol- lp1 lps lection D of s marked diagrams Di, i 1; ...; s,withDi having lpi columns and m rows. We arrange these Di in the form:

4:29

Marked boxes are put into D subject to the following constraint: in each Di a column has at most one marked box. If a diagram Di contains a marked box in its kth row and sthcolumn,thennootherDj j 6 i contains a marked box in its kth row and sth column. From now on D will denote such a collection of marked diagrams.

Now suppose that D is a collection of diagrams Di, i 1; ...; s.ToeachDi we may associate a l Â l matrix Y as in the previous section. We let pi pi Di detY : detY detY ÁÁÁdetY . Now to each column j of D 1 W j W l ,we D D1 D2 Ds p1 may associate a p Â p matrix Xj Ij obtained from the matrix X as follows. Let Ij be the subset of f1; ...; pg consisting of the numbers of the marked rows on the column j. We de¢ne Xj Ij to be the matrix obtained from X by replacing 1 p the rows of X corresponding to Ij by the vector Zj ; ...; Zj . We then de¢ne X : X I X I ...X I . D 1 1 2 2 lp1 lp1 Suppose we have a marked box in Di appearing in its kth row and sth column. We ks ks associate an odd indeterminate ai . Consider the product of all ai arranged in

increasing order following the lexicographical ordering of i; s; k.Nowwemayalso consider the product arranged in increasing order following the lexicographical ordering s; k; i. These two products differ by a sign, and this sign is denoted by

eD.Furthermoreweletdi jDij, the number of marked boxes in Di,andej be the number of marked boxes in the jth column of D.

PROPOSITION 4.3. With notation as above we have P Gl ÁÁÁGl X 1 e p1 ps 2 didj ÀjDj D À1 i;j detXDdetYD: lp2...lps e ! ÁÁÁe ! detX D 1 lp1

Proof. Given diagram Di with m rows and lpi columns, for i 1; ...; s,wewant to know how to simplify the expression

Ps 1 d d À1 À12 i1 i i detX detY ÁÁÁdetX detY : D1 D1 Ds Ds We move all detX to the left and get Di

Ps 1 d d À1jDj À12 i;j1 i j 2 detX ÁÁÁdetX detY ÁÁÁdetY : 4:30 D1 Ds D1 Ds Now each detX is a product of detX a. We arrange all X a together so that Di b 1 X1 a1 appears to the left of X1 a2 if and only if a1 < a2 and call the resulting Z1 expression XD andmoveittotheleft.WedothesamethingtoX2 a and move Z2 Z1 XD to the right of XD etc. Then (4.30) becomes

Ps 1 d d À1jDj Z Z Zl À12 i;j1 i j 2 e X 1 X 2 ÁÁÁX p1 detY ÁÁÁdetY : 4:31 D D D D D1 Ds We apply now Corollary 4.2 to (4.31) and obtain P 1 s 1 0 0 d d À jDj eD l Àp... l ÀpÀlp1 À12 i;j1 i j 2 detX p1 lp1 detX detY : e ! ÁÁÁe ! D D 1 lp1 P P s lp1 0 Since i1 lpi i1 li À p, the proposition follows. &

It follows immediately from Corollary 4.1 that

PROPOSITION 4.4. The vector

Z: detX m 1; ...; p detX m 1; ...; pÁÁÁdetX m 1; ...; p 1 2 lp1

is annihilated by (4.5).

PROPOSITION 4.5. The expression X P 1 d d ÀjDj eD À12 i;j i j detX detY detZ e ! ÁÁÁe ! D D D 1 lp1

is divisible by detXlp1 . Furthermore the resulting expression is independent of the i variables xl l m 1; ...; p and is annihilated by (4.5). Proof. Since for a subset I of f1; ...; mg, detXi IdetXi m 1; ...; p is a scalar multiple of detXdetXi I [fm 1; ...; pg by Corollary 4.2(iii), it follows that the lp1 i expression is divisible by detX and independent of xl,forl m 1; ...; p, after division. It is clear that it is annihilated by (4.5). &

The vector P X 1 didj ÀjDj À1 Àl À12 i;j e ! ÁÁÁe ! e detX p1 detZdetX detY 1 lp1 D D D D

depends only on the s-tuple lp1; ...; lps and thus we will denote this vector by G lp1; ...; lps.

PROPOSITION 4.6. The vector G lp1; ...; lps has weight corresponding to the Young diagram (4.25).

Proof. Let fj, j 1; ...; m denote the number of marked boxes in D that appear in the jth row of some diagram Di.ThendetYD has weight

Xlp1 Xm Xlp1 Xlp1 0 0 ~ lj À pdj fjej À ejdj lj À pdj; j1 j1 j1 j1

Àlp1 while the expression detX detXDdetZ has weight

Xm Xlp1 Xlp1 Xm Xp lp1 ej p À m dj ejdj À fjej lp1 e~j: j1 j1 j1 j1 j1

So the combined weight is

Xm Xp Xlp1 Xlp1 0 0 ~ lp1 ej lp1 e~j lj À mdj lj À pdj; j1 j1 j1 j1

which of course is the weight of the Young diagram (4.25). &

Combining our results in this section we have proved.

THEOREM 4.5. InthecasewhenpX m an irreducible gl pjqÂgl mjn module l l Vpjq Vmjn appears in Cx; x; Z; y if and only if lm1 W nandlp1 W q. The following are highest-weight vectors corresponding to such a Young diagram l (t is the length of l0):

(i) In the case when lm1 0 it is given by Yt D 0 : li i1

(ii) In the case when lm1 > 0 and lp1 0 it is given by Yr Yt D 0 D 0 ; i;li li i1 ir1

0 0 where 0 W r W nisde¢nedbylr > mandlr1 W m. (iii) In the case when lp1 > 0 it is given by

Yr0 Yt G lp1; ...; lps D 0 D 0 ; i;li li ir1 ir01

where r W r0 arede¢nedasin(4.23)andp s is the length of l. Proof. The only thing that remains to prove is that the vector in (iii) is indeed killed

by (4.5). But this is because of the presence of Z in the formula of G lp1; ...; lps and so is an immediate consequence of Corollary 4.1. &

5. Construction of Highest-Weight Vectors in S S2Cmjn In this section we will give an explicit formula for a highest-weight vector of each irreducible gl mjn-module that appear in the symmetric algebra of the symmetric square of the natural gl mjn-module. According to Theorem 3.4 we have the following decomposition of S S2Cmjn as a gl mjn-module: X 2 mjn l S S C Vmjn; l

where the summation is over all partitions l with even rows and lm1 W n. mjn We let fx1; ...; xm; x1; ...; xng be the standard basis for C ,withxi denoting even, and xj odd vectors. Regarding xi as even and xj as odd variables the Lie superalgebra gl mjn has a natural identi¢cation with the space of ¢rst order

differential operators over Cxi; xj. The Cartan subalgebra of diagonal matrices is then spanned by its standard basis xi @=@xi and xj @=@xj,fori 1; ...; m and j 1; ...; n. The nilpotent radical is generated by the simple root vectors @ @ @ xi ; xj ; xm ; i 1; ...; m À 1; j 1; ...; n À 1: 5:1 @xi1 @xj1 @x1 2 mjn S C then is spanned by the vectors xij xji xixj, ykl Àylk xkxl and Zki xkxi,where1W i; j W m and 1 W k; l W n.Thisallowsustoidentify 2 mjn S S C with the polynomial algebra over C in the even variables xij and ykl

and odd variables Zki,with1W i W j W m and 1 W k < l W n, which we denote by Cx; y; Z. A convention of notation we will use throughout this section is the following: By

xi x1; x2; ...; xm we will mean the row vector xi1; xi2; ...; xim. So by the expression 0 1 x1 x1; x2; ...; xm B C B x2 x1; x2; ...; xm C B . C @ . A xm x1; x2; ...; xm

we mean the matrix whose ith row entries equals to xi1; xi2; ...; xim,i.e.thematrix 0 1 x11 x12 ÁÁÁx1m B x x ÁÁÁx C B 21 22 2m C X: @ . . . A: . . ÁÁÁ. xm1 xm2 ÁÁÁxmm Similarly by an expression of the form 0 1 x1 x1; x2; ...; xm B . C B . C B . C B C B xiÀ1 x1; x2; ...; xm C B C Xi xj: B xj x1; x2; ...; xm C B C B xi1 x1; x2; ...; xm C B . C @ . A xm x1; x2; ...; xm

we mean to replace the ith row of the matrix X by the vector Zj1; Zj2; ...; Zjm.In these forms the action of (5.1) will be more transparent.

Consider the ¢rst r Â r minor Dr of the m Â m matrix X,for1W r W m.Itiseasily Pr seen to be a highest-weight vector in Cx; y; Z of highest weight 2 i1 ei,where as before we use ei and dk to denote the fundamental weights of gl mjn. Hence if l is a Young diagram with even rows of length not exceeding m, then its corre-

sponding highest-weight vector is a product of Drs. To be explicit note that since l has even rows, l1 2t is an even number. Furthermore we also have 0 0 Qt l l ,foralli 1; ...; t. Then the highest-weight vector is given by D 0 . 2iÀ1 2i i1 l2i Consider now a diagram of the form

5:2

The product of highest-weight vectors of two such diagrams, if nonzero, gives a highest-weight vector of a diagram of the form

Dividing by the determinant detX l2 we obtain a highest-weight vector for the diagram

Thus it is enough to construct vectors associated to the Young diagrams of the form (5.2). To do so we ¢rst consider the case when l 1 in (5.2). Consider the expression

D x ; x : À detX x x detX x x x 1 2 1 2 1 1 2 1 5:3 detX2 x1 x2x2ÁÁÁ detXm x1 x2xm;

where by x1x2 and x2xi we mean y12 and Z2i, respectively. The following lemma will be useful later on.

LEMMA 5.1. Let A aij be a complex symmetric m Â m matrix and y1; y2; ...; ym be odd variables. Then 0 1 0 y1 ÁÁÁym B y C B 1 C det@ . A 0: . A ym

Proof. It is enough to restrict ourselves to real symmetric m Â m matrices A.LetU be an orthogonal m Â m matrix such that U tAU D,whereD is a diagonal matrix.

We compute 0 10 10 1 0 1 10ÁÁÁ0 0 y1 ÁÁÁym 10ÁÁÁ0 0 z1 ÁÁÁzm B 0 CB y CB 0 C B z C B CB 1 CB C B 1 C @ . A@ . A@ . A B . C; 5:4 . U t . A . U @ . D A 0 ym 0 zm

Pm where zk j1 ujkyj and U uij. But the determinant of the matrix on the right-hand side of (5.4) is zero. &

The next lemma is straightforward.

Pm LEMMA 5.2. D x1; x2 has weight 2 i1 eid1 d2 and, hence, its weight corresponds to the weight of the Young diagram (5.2) with l 1.

LEMMA 5.3. D x1; x2 is annihilated by all operators in (5.1) and, hence, is a highest-weight vector in S S2Cmjn corresponding to the Young diagram (5.2) with l 1.

Proof. First consider the action of the operator xiÀ1 @=@xi,fori 2; ...; m,on D x1; x2 givenasin(5.3).CertainlyxiÀ1 @=@xi annihilates the ¢rst summand of (5.3), and furthermore it takes the summand Xj x1 x2xj for j 6 i À 1; i to 0 1 x1 x1; ...; xm B . C 0 1 B . C x1 x1; ...; xiÀ1; xiÀ1; ...; xm B C B . C B xjÀ1 x1; ...; xm C B . C B C B . C B x x ; ...; x C B C B 1 1 m C B xjÀ1 x1; ...; xiÀ1; xiÀ1; ...; xm C B x x ; ...; x C C B j1 1 m C detB x x1; ...; xiÀ1; xiÀ1; ...; xm x x detB . C x x ; B 1 C 2 j B . C 2 j B x x ; ...; x ; x ; ...; x C B . C B j1 1 iÀ1 iÀ1 m C B C B . C B xiÀ1 x1; ...; xm C @ . A B C B xiÀ1 x1; ...; xm C xm x1; ...; xiÀ1; xiÀ1; ...; xm B . C @ . A xm x1; ...; xm

which is zero. xiÀ1 @=@xi takes Xi x1 x2xi to 0 1 0 1 x1 x1; ...; xiÀ1; xiÀ1; ...; xm x1 x1; ...; xm B . C B . C B . C B . C B . C B . C B C B C B xiÀi x1; ...; xiÀ1; xiÀ1; ...; xm C B xiÀ1 x1; ...; xm C B C B C detB x1 x1; ...; xiÀ1; xiÀ1; ...; xm C x2xidetB x1 x1; ...; xm C x2xiÀ1: B C B C B xi1 x1; ...; xiÀ1; xiÀ1; ...; xm C B xi1 x1; ...; xm C B . C B . C @ . A @ . A xm x1; ...; xiÀ1; xiÀ1; ...; xm xm x1; ...; xm The ¢rst summand is zero, while the second summand remains. Now we verify simi-

larly that xiÀ1 @=@xi takes XiÀ1 x1 x2xiÀ1 to the identical expression as the second summand above with the difference that the i À 1st and ith rows are interchanged.

Thus xiÀ1 @=@xi D x1; x2 0. Consider now the action of xm @=@x1 on D x1; x2.Notethatxm @=@x1 kills every term in (5.3) except for the ¢rst and the last. The contribution from the ¢rst

summand is ÀdetX x2xm, while that from the last summand is detX x2xm,and hence xm @=@x1 D x1; x2 0. Finally we consider the action of x1 @=@x2 on D x1; x2 as in (5.3). x1 @=@x2 kills the ¢rst term in (5.3) and the resulting vector is

Xm detXi x1 x1xi; 5:5 i1

which can be in a consistent form with our earlier notation written as D x1; x1. Expanding along the ¢rst row we see that (5.5) is the same as 0 1 0 x1x1ÁÁÁ x1xm B C B x1x1 C detB . C; @ . X A x1xm

which is zero by Lemma 5.1. &

The proof of the above theorem gives us certain identities that will be used later on. We will collect them here for the convenience of the reader:

@ xiÀ1 D xk; xl 0; 5:6 @xi

@ xjÀ1 D xj; xl D xjÀ1;xl; 5:7 @xj

@ xjÀ1 D xs; xj D xs; xjÀ1; 5:8 @xj

D xj; xj0: 5:9

We now turn our attention to the general case of a Young diagram of the form (5.2) with general l. Of course, we have the restriction that 2l W n.

Let s f i1; i2; i3; i4; ...; i2lÀ1; i2lg be a partition of the set f1; 2; ...; 2lg. Assuming that we have arranged s in the form so that i1 < i2; i3 < i4; ...; i2lÀ1 < i2l, we may de¢ne es to be the sign of the permutation taking k to i for all 1 W k W 2l. We may associate to s a vector D x ; x ÁÁÁD x ; x in k i1 i2 i2lÀ1 i2l

S S2Cmjn and de¢ne X G 2l e D x ; x ÁÁÁD x ; x ; s i1 i2 i2lÀ1 i2l s

where the sum is taken over all partition s f i1; i2; i3; i4; ...; i2lÀ1; i2lg of the set f1; 2; ...; 2lg arranged in the form that i1 < i2; i3 < i4; ...; i2lÀ1 < i2l. The following lemma is again a straightforward computation.

Pm P2l LEMMA 5.4. The weight of G 2l is 2l i1 ei j1 dj and, hence, corresponds to the weight of the Young diagram (5.2).

LEMMA 5.5. G 2l is annihilated by (5.1) and, hence, a highest-weight vector in S S2Cmjn.

Proof. The fact that G 2l is annihilated by xiÀ1 @=@xi for i 2; ...; m is a consequence of (5.6). Now the proof of Lemma 5.3 shows that xm @=@x1 D x1; xs 0, for every s 1; ...; n. Thus xm @=@x1 annihilates G 2l as well. So it remains to show that xjÀ1 @=@xj kills G 2l. Given a summand in G 2l of the form es ...D xjÀ1; xl ...D xj; xi ...there exists a summand of the form es0 ...D xjÀ1; xi ...D xj; xl ..., which is identical to it except at these two places. Now xjÀ1 @=@xj takes the ¢rst of the two summands above to

es ...D xjÀ1; xl ...D xjÀ1; xi ...; and the second summand to

es0 ...D xjÀ1; xi ...D xjÀ1; xl ... :

0 But s and s differ by a transposition i; l and hence es Àes0 andsothesetwoterms cancel.

Consider a summand in G 2l of the form es ...D xl; xjÀ1 ...D xj; xi ....Butin G 2l we also have a summand of the form es0 ...D xl; xj ...D xjÀ1; xi .... Applying xjÀ1 @=@xj to these two terms, we again see that they cancel by the same reasoning as before.

Now we look at a term of the form es ...D xj; xl ...D xi; xjÀ1 .... We also have a term of the form es0 ...D xjÀ1; xl ...D xi; xj .... Again they will cancel each other after applying xjÀ1 @=@xj. Finallyatermoftheformes0 ...D xjÀ1; xj ...is killed by xjÀ1 @=@xj by (5.9). This completes the proof. &

It is clear that a product of G 2ls (not necessarily the same l) is nonzero, which therefore allows us to construct all other highest-weight vectors, as discussed in the beginning of this section. Below we summarize the results of this section.

THEOREM 5.1. The gl mjn-highest-weight vectors of S S2Cmjn form an Abelian

semigroup generated by G 2; ...; G 2n=2 and D1; ...; Dm,wheren=2 denotes

the largest integer not exceeding n=2. Furthermore this semigroup is free if and only if n 0; 1. More precisely a highest-weight vector associated to an even partition

l l1; ...; ll with lm1 W n is given by

Pl Yl Yl 1 À li detX im2 G li D 0 2; lj m1 jr1

0 0 where the nonnegative integer r is de¢ned by lr > mandlr1 W m.

Remark 5.1. From Theorem 5.1 we may recover the highest-weight vectors in S S2Cm and S L2Cn by putting n 0andm 0, respectively. Namely, identifying S2Cm (respectively, L2Cn) with the space of symmetric m Â m (respectively skew-symmetric n Â n) matrices, we see that in the ¢rst case the highest-weight vectors are generated by the leading minors of the determinant of the typical element of S2Cm, while in the second case they are generated by the Pfaf¢ans of the leading 2l Â 2l minors of the typical element of L2Cn,where2l W n.(cf.[H2]).

Acknowledgements Our work is greatly in£uenced by the beautiful article [H2] of R. Howe to whom we are grateful. The results in this paper and and its sequel [CW] are based on our two preprints under the same title (part one and two) as the current paper. After we submitted part one and ¢nished part two, we came across two preprints of Sergeev: `An analog of the classical invariant theory for Lie superalgebras', I, II, math.RT/9810113 and math.RT/9904079, which have overlaps with our work. More precisely, Sergeev obtained independently the gl pjqÂgl mjn-module decomposition of the symmetric algebra S Cpjq Cmjn (i.e. our Theorem 3.2), and the gl mjn-module decomposition of S S2Cmjn (i.e.ourTheorem3.4).Finally, we also like to thank the referee for bringing the paper [BPT], which is discussed in the introduction, to our attention. Therefore we have made corresponding changes on our preprints which result in the current version of this paper and [CW].

References [BPT] Brini, A., Palareti, A. and Teolis, A.: Gordan^Capelli series in superalgebras, Proc. Nat. Acad. Sci. USA 85 (1988), 1330^1333. [BR] Berele, A. and Regev, A.: Hook Young diagrams with applications to combinatorics and to representations of Lie superalgebras, Adv. in Math. 64 (1987), 118^175. [CW] Cheng, S.-J. and Wang, W.: Remarks on the Schur^Howe^Sergeev duality, Lett. Math. Phys. 52 (2000), 143^153. [GW] Goodman, R. and Wallach, N.: Representations and Invariants of the Classical Groups, Cambridge University Press, Cambridge, 1998. [H1] Howe, R.: Remarks on classical invariant theory, Trans. Amer. Math. Soc. 313 (1989), 539^570.

[H2] Howe, R.: Perspectives on invariant theory: Schur duality, multiplicity-free actions and beyond, The Schur Lectures, Israel Math. Conf. Proc. 8, Tel Aviv (1992), pp. 1^182. [K] Kac, V. G.: Lie superalgebras, Adv. Math. 26 (1977), 8^96. [KV] Kashiwara, M. and Vergne, M.: On the Segal^Shale^Weil representation and har- monic polynomials, Invent. Math. 44 (1978), 1^47. [M] MacDonald, I. G.: Symmetric Functions and Hall Polynomials, Oxford Math. Monogr., Clarendon Press, Oxford, 1995. [OV] Onishchik A. L. and Vinberg, E. B.: Lie Groups and Algebraic Groups,SpringerSer. Soviet Math., Springer-Verlag, Berlin, 1990. [Se] Sergeev, A. N.: The tensor algebra of the identity representation as a module over the Lie superalgebras gl n; m and Q n, Math. USSR Sbornik 51 (1985), 419^427. [V] Vinberg, E. B.: Complexity of actions of reducitive Lie groups, Funct. Anal. Appl. 20 (1986), 1^11. [W] Wang, W.: Dual pairs and in¢nite dimensional Lie algebras, In: N. Jing and K. C. Misra (eds), Recent Developments in Quantum Af¢ne Algebras and Related Topics, Contemp. Math. 248, Amer. Math. Soc., Providence, 1999, pp. 453^469.