TRANSACTIONS OF THE AMERICAN MATHEMATICAL SOCIETY Volume 349, Number 9, September 1997, Pages 3517{3534 S 0002-9947(97)01823-0 MONOID HECKE ALGEBRAS MOHAN S. PUTCHA Abstract. This paper concerns the monoid Hecke algebras introduced by Louis Solomon. We determine explicitly the unities of the orbitH algebras asso- ciated with the two-sided action of the Weyl group W . We use this to: 1. find a description of the irreducible representations of , 2. find an explicit isomorphism between and the monoidH algebra of the H Renner monoid R, 3. extend the Kazhdan-Lusztig involution and basis to ,and H 4. prove, for a W W orbit of R, the existence (conjectured by Renner) of generalized Kazhdan-Lusztig× polynomials. Introduction A monoid analogue of the Iwahori-Hecke algebra [11] was obtained by Solomon [27]–[29]. In an earlier paper [19] the author studied Solomon’s monoid Hecke al- gebras by studying the associated orbit algebras. These orbit algebras arise from the two-sided action of the Weyl group W on the Renner monoid R. In particular, the coefficients of the unity of the empty level orbit algebra were shown to be Rx;y, where Rx;y are polynomials introduced by Kazhdan and Lusztig [12]. The other orbit algebras were also shown to have unities, but their coefficients were only im- plicitly given. In this paper we give an explicit formula for the unities of all the orbit algebras, thereby obtaining a description of the irreducible representations of monoid Hecke algebras. We also obtain an explicit, but very complicated, isomor- phism between the monoid Hecke algebra and the monoid algebra of R,solvinga problem posed by Solomon [28]. We go on to extend to the monoid Hecke algebra the Kazhdan-Lusztig involution and basis for the (group) Iwahori-Hecke algebra. This then immediately yields polynomials Pθ,σ for θ, σ in the same W W orbit of R, partially solving a problem posed by Renner [26]. These polynomials× are still mysterious; however, in the simplest case they are products of relative Kazhdan- Lusztig polynomials introduced by Deodhar [6]. 1. Reductive monoids and monoids of Lie type Consider the general linear group G = GLn(F ) over an algebraically closed field F . It is the unit group of the multiplicative monoid M = Mn(F )ofalln nmatrices over F . This monoid has the following structure. The diagonal idempotents× form a Boolean lattice with respect to the natural order of idempotents: f e if ef = fe = f: ≤ Received by the editors December 3, 1993. 1991 Mathematics Subject Classification. Primary 20G40, 20G05, 20M30. Research partially supported by NSF Grant DMS9200077. c 1997 American Mathematical Society 3517 License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 3518 MOHAN S. PUTCHA The G G orbits have a cross-section Λ of idempotents × I 0 e = r : r 00 These idempotents and hence the G G orbits are linearly ordered according to × rank. The orbit Jr = GerG consists of all rank r matrices and gives rise to the orbit semigroup J 0 = Ge G 0 ; r r ∪{ } where for a; b J , ∈ r ab if ab J ; a b = ∈ r · (0otherwise: We also have the orbit monoid M(J )=G J0: r ∪ r The idempotents in Jr are all conjugate, and hence M(Jr) is completely determined by the parabolic subgroups AB P = g G ge = e ge = A GL ; r { ∈ | r r r} 0 C|∈ r A 0 P − = g G e g = e ge = A GL ; r { ∈ | r r r} BC|∈ r and the natural homomorphisms δ+ : Pr GLr and δ : Pr− GLr.Wewrite → − → M(Jr)asM(G; Pr;Pr−;GLr) with δ+,δ being understood to be part of the data. There are two deficiencies with the monoid− M. First, only maximal parabolics arise as Pr, and second, the orbit monoids are not submonoids of M.Bothofthese drawbacks can be handled by considering the representation θ of M given by θ(a)= 1(a) 2(a) n(a); ∧ ⊗∧ ⊗···⊗∧ the tensor product of all exterior powers of a. The representation θ has the effect of killing the singular part of M, because θ(M)=θ(G) 0 . However, taking ∪{ } the Zariski closure of θ(M) yields an amazing monoid M. Again there is a G G cross-section Λ of idempotents, but this time Λ 0 is a Boolean lattice isomorphic× \{ } to the power set of the Dynkin diagram of G. Moreover,f any parabolic subgroup is associated with a unique orbit monoid, which now is a submonoid of M.Further, the lattice of diagonal idempotents is the dual of the Coxeter complex. Both M and M are examples of reductive monoids. The theoryf of reductive monoids has been well developed by Renner and the author; cf. [17]. In particular the author [16] hasf proved the existence of a cross-section lattice Λ and has shown that the orbit monoids are of the form M(G; P; P −;L=K) for some pair of opposite parabolics P; P −, K/L=P P−. Renner [24] has shown the existence of the finite inverse monoid ∩ R = W; Λ ; h i where W is the Weyl group of G such that M = BσB; σ R G∈ where B is the Borel subgroup G.WecallRthe Renner monoid of M. License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use MONOID HECKE ALGEBRAS 3519 The theory of monoids of Lie type is an attempt to accomplish the above for finite groups of Lie type. Let G be a finite group of Lie type with Weyl group W and Borel subgroup B. The most convenient definition of a monoid M of Lie type (with unit group G) is also the most general, and is due to the author [18]. The definition is simply that all the orbit monoids are of the form M(G; P; P −;L=K); where P; P − are opposite parabolic subgroups, L = P P −, K/L. These monoids have been classified by the author [18], [20] using the∩ theory of buildings [30] and the ideas of Renner and the author [21] for reductive monoids. Renner and the author [22] obtained an analogue of the canonical monoid M for any finite group G of Lie type. Let M be a monoid of Lie type with unit group G. Theref is again a cross- section lattice Λ for the G G orbits. There is also an analogue of Renner monoid R = W; Λ such that × h i M = BσB: σ R G∈ We refer to [18] for details. We will let and ` denote the Bruhat order and the length function, respectively, on the Weyl≤ group W ;cf.[10].Fore Λ, let ∈ W (e)= x W xe = ex ; { ∈ | } W = x W xe = e /W(e): e { ∈ | } Then both W (e)andWe are parabolic subgroups of W ,andW(e) is the direct product of We and the Weyl group of the unit group of eMe.Let D(e)= x W xis of minimum length in xW (e) ; { ∈ | } D = x W x is of minimum length in xW : e { ∈ | e} Now R = WeW; e Λ G∈ and each element σ WeW can be uniquely written as ∈ 1 σ = xey− ;xD;y D(e): ∈e ∈ We call this the standard form of σ.Ifw0;v0 are respectively the longest elements of W and W (e), then w0v0 is the longest element of D(e). Following Solomon [27], [29] and Renner [25], we let `(e)=`(w0v0) 1 and, for σ = xey− in standard form, `(σ)=`(x)+`(e) `(y): − It is shown in [25] that `(σ) = 0 if and only if σB = Bσ: 1 1 If σ = xey− and θ = sft− in standard form, then define (1) σ θ if e f and x sw; tw y for some w W (f)W : ≤ ≤ ≤ ≤ ∈ e License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 3520 MOHAN S. PUTCHA We have shown in [15] that in the analogous situation for reductive monoids, σ θ if and only if BσB is contained in the Zariski closure of BθB. For this reason≤ we 1 say that σ is triangular if σ 1. Thus, if σ = xey− in standard form, ≤ (2) σ is triangular if and only if x y: ≤ If σ WeW and θ WeW,then ∈ ∈ σ θ implies `(σ) `(θ): ≤ ≤ 2. Hecke algebras Let G be a Chevalley group defined over Fq, B;B− opposite Borel subgroups of G, T = B B−, B = UT, B− = U−T,andWthe Weyl group of G with generating set S of simple∩ reflections. If x W ,let˙xdenote a coset representative in G.Let ∈ 1 = b C[G]: B ∈ b B | |X∈ The Iwahori-Hecke algebra HC(G)=HC(G; B)=C[G] is a semisimple algebra that is isomorphic to C[W ] by a theorem of Tits (cf. [1], [4]). Clearly HC(G; B) has a basis A = x, x W: x ∈ This basis is normalized as T = q`(x)A ;xW: x x ∈ With respect to this basis, Iwahori [11] showed that the structure constants are integer polynomials in q, depending only on W . In particular, (3) Txy = TxTy;Axy = AxAy if `(xy)=`(x)+`(y): One can therefore consider the generic Hecke algebra (W ) with basis Tx (x W ) 1=2 1=2 H ∈ over Z[q ;q− ], where q is now being treated as an indeterminate. Specifying q and tensoring with C, one recovers HC(G; B). Kazhdan and Lusztig [12] introduced an involution on (W )givenby H 1=2 1=2 1 `(y) (4) q =q− ; Tx =T−1 = q− Ry;xTy; x− y W X∈ where Ry;x = Ry;x(q) Z[q].