Japan. J. Math. Vol. 19, No. 1, 1993 Finite dimensional representations of the quantum group GLq(n;C) and the zonal spherical functions on Uq(n-1)•_Uq(n) By Masatoshi NOUMI, Hirofumi YAMADA and Katsuhisa MIMACHI (Received March 16, 1992) •˜ 0. Introduction Representation theory of quantum groups is now studied in connection with various fields of mathematics. In this paper, we discuss representations of quantum groups and their homogeneous spaces from the viewpoint of spherical functions. In this direction, the spherical functions on the quantum group SUq(2) are studied by Vaksman-Soibelman [VS1], Koornwinder [K1, 2], Masuda et al. [MM1, 2] and Noumi-Mimachi [NM1, 2]. This paper, a detailed version of the announcement [NYM], deals with the quantum general linear group GLq(n;C) and its "compact" real form Uq(n), the quantum unitary group; the objects of our study are their algebras of functions A(GLq(n;C)) and A(Uq(n)). The main points we will discuss are the following: (1) An analogue of the Borel-Weil construction of finite dimensional irre ducible representations of GLq(n;C). (2) Irreducible decomposition of A(GLq(n;C)) as a two-sided A(GLq(n;C)) - comodule and the orthogonal decomposition of A(Uq(n)). (3) Invariant functional and the zonal spherical functions on the quantum homogeneous space Uq(n-1)•_Uq(n). We will give here a brief description of the contents without precise definition of the terminology. (1) For each dominant integral weight A, we realize the finite dimensional irreducible representation of GLq(n;C) of highest weight A as a space of relative invariants in A(GLq(n;C)) with respect to the Borel subgroup Bn , and also its basis consisting of standard monomials (Theorem 2.4 and 2.5). This gives a quantum analogue of the space of sections of the associated line bundle on the flag manifold. (2) We give the irreducible decomposition of A(GLq(n;C)) as a two-sided A(GLq(n;C))-comodule (Theorem 2.11), by using the complete reducibility the orem of finite dimensional Uq(si(n))-modules, due to Rosso [R] and Lusztig [L]. 32 M. NOUMI, H. YAMADA and K. MIMACHI This leads to the existence of an invariant functional on A(GLq(n;C)) and the parametrization of all finite dimensional irreducible A(GLq(n;C))-comodules by means of the dominant integral weights (Theorem 2.12). The coordinate ring A(Uq(n)) is defined as a Hopf *-algebra (A(GLq(n;C)),*). We introduce two Her mitian forms on A(Uq(n)) by means of the invariant functional. The irreducible decomposition of A(Uq(n)) is orthogonal with respect to these Hermitian forms (Theorem 3.4). This statement on Uq(n) is an algebraic reformulation of results of Woronowicz [W]. (3) The quantum homogeneous space Uq(n-1)•_Uq(n) is a quantum (2n-1) - sphere. The algebra of functions on this quantum homogeneous space is defined as the *-subalgebra of all left Uq(n-1)-invariants in A(Uq(n)) . We also give its irre ducible decomposition as a right A(Uq(n))-comodule (Proposition 4.3). Furthermore we determine the zonal spherical functions on Uq(n-1)•_Uq(n) explicitly in terms of the little q-Jacobi polynomials (Theorem 4.7). It is remarkable that the algebra of functions on the quotient space Uq(n-1)•_Uq(n)/H by the diagonal subgroup H of Uq(n) falls into a commutative algebra. On this commutative subalgebra, the invariant functional on Uq(n-1)•_Uq(n) is expressed by an iterated Jackson integral (Theorem 4.6). After we completed this work, we received preprints by Taft-Towber [TT], Hashimoto-Hayashi [HH] and Vaksman-Soibelman [VS2]. The first two, [TT] and [HH], also deal with the standard monomial bases for finite dimensional irreducible representations of GLq(n;C). Our arguments are different from theirs in the sense that we give a characterization of relative B-n-invariants (Theorem 2.2). The last one [VS2] is also devoted to a harmonic analysis on the odd dimensional quan tum spheres Uq(n-1)•_Uq(n). In our formulation, the zonal spherical functions are realized as elements of the algebra of functions. Throughout this paper we always assume that the parameter q •¸ E C•_{0} is not a root of unity. When the real form Uq(n) is concerned, we assume that q is a real number with 0<|q|<1. Contents •˜ 1. Quantum group GLq(n;C). 1.1. Quantum semigroup Matq(n;C). 1.2. Quantum minor determinants. 1.3. Quantum group GLq(n;C). 1.4. Quantum universal enveloping algebra Uq(g.?(n;C)). 1.5. A monomial basis for A(Matq(m,n;C)). •˜ 2. Finite dimensional representations of GLq(n;C). 2.1. Quantum G-spaces and relative invariants. 2.2. Representations of GLq(n;C). Quantum group GLq(n;C) and zonal spherical functions 33 2.3. Irreducibility of the representations A(B 1 GLq (n; C); zA). 2.4. Irreducible decomposition of A(GLq(n;C)). •˜ 3. Quantum group Uq(n) and the invariant functional. 3.1. Quantum group Uq(n) and its unitary representations. 3.2. Invariant functional.•˜ 4. Zonal spherical functions on Uq(n-1)•_Uq(n). 4.1. Quantum homogeneous space Uq(n-1)•_Uq(n). 4.2. Invariant functional on A(Uq(n-1)•_Uq(n)). 4.3. Zonal spherical functions on Uq(n-1)•_Uq(n). •˜ 1. Quantum group GLq(n;C) 1.1. Quantum semigroup Matq(n;C) For any positive integers m and n, we denote by Mat(m,n,C) the space of all m•~n complex matrices and by Mat(n;C) the space of all n•~n square matrices. We introduce the quantum semigroups Matq (n;C) and their action on the quantum spaces Matq(m,n;C). Hereafter we fix a nonzero complex number q. By definition, the coordinate ring A(Matq(m,n;C)) is the C-algebra generated by mn letters x2j (1 < i < m, 1 < j < n) with fundamental relations (1.1.a) (1.1.b) (1.1.c) When m=n, we write simply Matq(n;C)=Matq(n,n;C). One can directly verify that there exist unique C-algebra homomorphisms L : A(Matq(rn, n, C)) -~ A(Matq(m, r, C)) ® A(Matq(r, n, C)) and : A(Matq(n, C)) --~ C satisfying (1.2) It is clear that each A(Matq(n;C)) becomes a bialgebra with coproduct •¢ and counit ƒÃ, and that each A(Matq(m,n;C)) has the natural structure of two-sided (A(Matq(m;C)), A(Matq(n;C)))-comodule. We remark that the commutation relations (1.1) of the algebra A(Matq(n;C)) are related to the so-called the constant R-matrix of type An-1(see[RTF]). Define the matrix R •¸ Mat(n;C) ® Mat(n;C) by (1.3) 34 M. NOUMI,H. YAMADAand K. MIMACHI where Eij are the matrix units. It is well-knownthat this matrix R satisfies the Yang-Baxter equation (see[J1]). With the matrix notation the commutation relations (1.1) are simply expressed in the form (1.4) where X ® X = xzaxkQE2~ ® Ei As preliminaries to the representation theory of the quantum group GLq(n;C), we investigate some fundamental comodules over the bialgebra A(Matq(n;C)). In the following, we will sometimes write Mn for (Matq(n;C). Let S be the C-algebra generated by n letters x1, ..., xn with fundamental relations Then one can directly show that there exists a unique C-algebra homomorphism RMn: S --* S ® A(Matq(n, C)) such that (1.5) Then the algebra S=C [x1,..., xn] becomes a right A(Matq (n;C))-comodule with this homomorphism RM. Note that the algebra S has a graduation S = ~r>o Sr, where Sr denotes the subspace of S of all homogeneous elements of degree r. It is clear that the subspaces Sr(r•¸N) are right A(Matq(n;C))-subcomodules of S. We call the right comodule Sr the symmetric tensor representation of the quantum semigroup (Matq(n;C) of degree r. The comodule S1 = Cx1® ... Cxn is also called the vector representation. Let E be the C-algebra generated by n letters y1,...,yn with (anti-) commuta tion relations Then one can directly show that there exists a unique C-algebra homomorphism RMn : E --~E ® A(Matq(n; C)) satisfying (1.6) and that E becomes a right A(Matq(n;C))-comodule with this coaction. Similarly to the case of S, the right A(Matq(n;C))-comodule E decomposes into a direct Quantumgroup GLq(n;C) and zonalspherical functions 35 sum E = ® o Er of subcomodules according to the homogeneous degree. The right A(Matq(n;C))-comodule Er is called the alternating tensor representation of (Matq(n;C) of degree r; the comoduleE1 is isomorphic to the vector representation S1. Note that the subspace Er of degree r has a C-basis {Yji• • • yjr;1 <j1 <.<•• jr <n}: We will see later that the representations Sr(r•¸N) and Er (0 < r < n) are irreducible. REMARK. The "exterior algebra" E=C[y1,...,yn] also has a structure of left A(Matq(n;C))-comodule whose coaction is given by the C-algebra homomor phism LM : E -~ A(Matq(n; C)) ® E such that Then the subspaces Er (0 < r < n) are again left subcomodules of E. •¡ 1.2. Quantum minor determinants We now investigate the matrix elements of the right comodule Er (0 < r < n).