<<

Proc. Indian Acad. Sci. (Math. Sci.), Vol. 104, No. l, February 1994, pp. 99-135. Printed in India.

Zeta functions of prehomogeneous vector spaces with coefficients related to periods of automorphic forms

FUMIHIRO SATO Department of Mathematics, Rikkyo University, Nishi-Ikehukuro, Toshimaku, Tokyo 171, Japan

Dedicated to the memory of Professor K G Ramanathan

Abstract. The theory of zeta functions associated with prehomogeneousvector spaces (p.v. for short) provides us a unified approach to functional equations of a large class of zeta functions. Howeverthe general theory does not includezeta functionsrelated to automorphic forms such as the Hecke L-functions and the standard L-functions of automorphic forms on GL(n), even though they can naturally be considered to be associated with p.v.'s. Our aim is to generalize the theory to zeta functions whose coefficients involve periods of automorphic forms, which include the zeta functions mentioned above. In this paper, we generalize the theory to p.v.'s with symmetric structure of K=-typeand prove the functional equation of zeta functions attached to automorphic forms with generic infinitesimal character. In another paper, we have studied the case where automorphic forms are given by matrix coefficientsof irreducible unitary representationsof compact groups.

Keywords. Zeta function;prehomogeneous vector space; functional equation; automorphic form.

Introduction

The theory of zeta functions associated with prehomogeneous vector spaces was developed by Sato and Shintani in [S] and [SS] and generalized later in [S1] to multivariable zeta functions. The theory provides us a unified approach to functional equations of a large class of zeta functions. However the general theory does not include zeta functions related to automorphic forms, even though they can naturally be considered to be associated with prehomogeneous vector spaces. Among those are the Hecke L-functions with Gr6ssencharacters (cf. [Heel], [Hec2]), the Epstein zeta functions with spherical functions (cf. [El), the Maass zeta functions attached to quadratic forms (cf. [M1]-[M4]), and the standard L-functions of automorphic forms on GL(n) (cf. [GJ]). Our ultimate purpose is to generalize the theory of zeta functions associated with prehomogeneous vector spaces ([SS], IS 1]) to zeta functions whose coeff~-'ients involve periods of automerphic forms, which include the zeta functions mentioned above. In [$6], we have studied the case where automorphic forms are given by matrix coefficients of irreducible unitary representations of compact groups. The simplest example in this case is the Epstein zeta functions with spherical functions (cf. w2.2, [$4]). In the present paper, we are concerned mainly with prehomogeneous vector spaces with symmetric structure and prove the functional equations and the analytic 99 100 FumihiroSato

continuations of zeta functions with automorphic forms in the case where the infinitesimal character of an automorphic form is generic and the prehomogeneous vector space in question has a symmetric structure of K,-type. Our results can be applied, for example, to zeta ,functions considered in [M3] and [Hej] and some special cases of zeta functions in [M 1, 2, 4] as well as several new zeta functions (cf. w6). Recall that, in the theory of prehomogeneous vector spaces developed in [SS] and [S1], the proof of the existence of analytic continuations and functional equations of zeta functions, is based on the following four properties: 1. Integral representation of the product of zeta functions and local zeta functions at the infinite prime (= the zeta integral); 2. Analytic continuation and the functional equation of the zeta integral; 3. Functional equations satisfied by local zeta functions; 4. The existence of b-functions (the Bernstein-Sato polynomials), which control the singularities of zeta functions and the gamma-factor of functional equations. Moreover, by using the b-functions, one can eliminate the troublesome contribution of rational points in the singular set to the zeta integral. We extend these four properties to zeta functions with automorphic forms. In w1, we introduce zeta functions with automorphic forms and give their integral representation (the zeta integral) in a rather general setting. In w2, as examples of generalized zeta functions defined in w1, we examine zeta functions with Gr6ssencharacters and the Epstein zeta functions with spherical functions from our point of view. The easiest part of the generalization is the functional equation of the zeta integral, which we describe in w3. In w4, we define the notion of symmetric structure of prehomogeneous vector spaces and establish some elementary properties. In w5, we prove the functional equation of local zeta functions (Theorem 5.3) and that of global zeta functions (Theorem 5.4) under the condition that the infinitesimal character of an automorphic form is generic and its symmetric structure is of K,-type. In w6, we present explicit formulas for the functional equations of zeta functions for some concrete examples of prehomogeneous vector spaces with symmetric structure.

Notation. As usual, 7/, Q, • and C stand for the ring of rational integers, the field of rational numbers, the field of real numbers and the field of complex numbers, respectively. For a complex number z, Re(z) is the real part of z. Let X be an affine algebraic variety defined over a field K. Then the set of K-rational points of X is denoted by Xr. For a linear G defined over a field K, G O= the identity component of G with respect to the Zariski topology, Ru(G) = the unipotent radical of G, X(G) = the group of rational characters of G, X(G) r = the subgroup of X(G) of rational characters defined over K. For a real G, GO = the identity component of G with respect to the usual Hausdorfftopology. Let X be a C~~ Then C~(X) stands for the space of smooth functions on X with compact support. Let Va be a finite-dimensional real vector space. Then ff'(Va) stands for the space of rapidly decreasing functions on Va. Let V o be a finite-dimensional Q-vector space. A C-valued function f on V o is called a Schwartz- Zeta functions with automorphic forms 101

Bruhat function, if there exist lattices L t, L 2 in V~ such that the support of f is contained in L~ and the value of f(x) is determined by the residue class x (mod L2). We denote by 6e(V~) the space of Schwartz-Bruhat functions on V~, For m,n >i !, we put M(m, n) = the space of m by n complex matrices, M(n) = M(n, n) = the ring of complex square matrices of size n, I,eM(n) = the identity matrix of size n, 0"~'"~ = the zero matrix in M(m, n). For a subring R of C, denote by M(m, n; R) the subset of M(m, n) of matrices with entries in R. For a real 9, we denote by U(9) the universal enveloping algebra of the complexification 9c = g | ~ C.

1 Definition of zeta functions and their integral representations

1.1 Let (G,p,V) be a prehomogeneous vector spaces defined over a field K of characteristic 0 and denote its singular set by S. Then, by definition, Vg-Sg is a single Gg-orbit, where/( is the algebraic closure of K. Let St ..... S, be the K-irreducible hypersurfaces contained in S and take K-irreducible polynomials P t .... , Pn defining S~ ..... Sn, respectively. It is known that the polynomial Pi is unique up to a non-zero constant factor in K. For each i = 1,..., n, there exists a K-rational character Xi satisfying

Pi(p(g)x)= ~i(g)Pi(x) (geG, xeV).

We call P~ ..... P, as the basic relative invariants over K. Any relative invariant of (G,p, V) with coefficients in K can be expressed as a product of Pt ..... Pn, negative power being allowed. Denote by Xp(G)r the subgroup of X(G)K generated by Zt,..., Z,, which is a free abelian group of rank n. (For basic definitions and properties in the theory of prehomogeneous vector spaces, refer to [S1, w 1] and [SK].)

1.2 In the following, we consider a prehomogeneous vector space (G, p, V) defined over the rational number field Q. For an x~V, put

G~ -" {g~GIp(g)x = x}.

We assume that (A- 1) for any x E V o - So, the isotropy subgroup Gx is reductive and X ((Gx)~ = { 1}; (A-2) G has a semidirect product decomposition G = LU, where L is a connected reductive Q-subgroup and U is a connected normal Q-subgroup with X(U) = {1). The group G always has a semi-direct product decomposition satisfying (A-2). Namely G is a semi-direct product of U = R~(G), the unipotent radical, and a Levi subgroup L. In the following we fix a decomposition G = LU satisfying (A-2) once for all, which may not be the Levi decomposition (for concrete examples, see w2, w4.1 and w6). 102 Fumihiro Sato

One of the consequences of assumption (A-l) is the following:

Lemma I. I The singular set S is a hypersurface.

Proof Since Gx is reductive, the open orbit V- S is an affine variety. This implies that S is a hypersurface.

Let P1 ..... P. be the basic relative invariants over Q and X1 ..... X. the rational characters of G corresponding to Pa ..... P,, respectively. Let Go be the identity component of (~ kerx~ with respect to the Zariski topology. Put Lo = LnGo. Then i=1 Lo is connected and we have Go = Lo U (semi-direct product).

Lemma 1.2 The oroups X(Lo) ~ and X(Go)~ are trivial.

Proof By (A-l), (A-2) and [S1, Lemma 4.1], we have

rank Xp(G)Q = rank X(G)Q = rank X(L) o .

This implies that

rank X(Go)o = rank X(Lo)o = 0.

Since Go and Lo are connected, the groups X(Lo)o and X(Go)o are trivial.

Let T be the largest Q-split torus of the identity component of the center Z(L) of L. Then dim T = rank X(G)Q = rank Xp(G)o and L is an almost direct product of T and L o.

1.3 Let G +, Go, T + , L o+ and U be the identity components of the real Lie groups Ga, Go,a, Tn, Lo,a and Ua, respectively. Then we have

G +_-T +L oU +, G o=L oU + and the decomposition

9= thu (9~G +,tET +,h~L o,u~U +)

is unique. By (A-2) and Lemma 1.2, the groups L o and U + are unimodular. Let dt, dh and du be (bi-invariant) Haar measures on T +, L o are U +, respectively. Let dro be a right invariant measure on G + and let A:G § ~R+ be the module of d,o. Then we can normalize these measures so that

d,9 = d,(thu) = A(t)dt dh du.

As proved in IS1, w the assumption (A-l) ensures the existence of 6 = (61 ..... 6.)eQ", for which

f~(x) = fi JP~(x)]-n'dx, dx = the Lebesgue measure on VR i=1 gives a relatively G +-invariant measure on V a -Sn with multiplier A. Zeta functions with automorphicforms 103

Let VR-S. = Vl u...uV, be the decomposition into connected components. Each connected component V~ is a single G+-orbit. For an x~V~, put G + = G x r G + . By (A-l), the group G~+ is a unimodular Lie group. We normalize a (bi-invariant) Haar measure d#x on G Jr§ such that

f~. F(o)d,g= fvtl(p(~))x)f61 F(Oh)dl~(h) (FeL'(G+,d,g)). (1.1)

1.4 Let ~:L o -~ W be a function on L~ with values in a finite-dimensional complex vector space W, which is invariant under the right multiplication of some arithmetic subgroup of Lo. Q c~ L o . Later we shall assume that r is an automorphic form on Lo; however at the moment we do not assume it. Now let us associate to r a linear form Zr on 5~(Va)| with complex parameter s in C n, which we call the zeta integral attached to ~ (for the definition of ~(Ya) and ~(Vo), see Notation. See also [$5, w Consider the canonical surjection p:Go-~Lo = Go/U. The map p induces a real analytic mapping

p:Oo Lo = G /u +.

For an arithmetic subgroup F of Go. o c~ G~', put FL = p(F)c L~'. Then F L i8 an arithmetic subgroup of Lo. o c~ L o (of. [Bo, Theorem 6]). For f~ | fo ~ ~v (V n) | ~ (V o), take an arithmetic subgroup F of G o,o c~ G~ such that fo is F-invariant and ~ is FL-invariant. Then we define the zeta integral attached to ~ by setting

z~cs)(fo~| = z~o~ ..... s.)(f~ | Z fo(X)Lo(p(thu)x)dhdu, = ~('r)f r+ ,O z,(tr'a(Odt f~urr xe V o - So (1.2)

(, where v(F)= | dhdu, which is finite by Lemma 1.2 and [BH, Theorem 9-4]. J6 Ur Note that the integral Z~ (s) is independent of the choice of F. In the following, we assume that (A-3) for any fo0|174 the integral Zc,(s)(f|174 is absolutely convergent, when Re(st) ..... Re(s,) are sufficiently large. In case ~ is a constant function, the integral Zr174 gives an integral representation of the usual zeta functions' associated with (G, p, V) (see [S1, w4], [$5, w [SS, w In this case some sufficient conditions for (A-3) are known by [$2, Theorem 1] and [SS, Lemmas 2.2, 2.5]. For example, we have the following criterion of convergence of Z, (s): 104 Fumihiro Sato

PROPOSITION 1.3 Assume that Xp(G)o = Xp(G)c. If Go.. = Go ta G.(xEV - S) is a connected semisimple algebraic group and dp:Lo --* W is bounded, then Zr | | fo)(f| | fo~Ae(Vn)| 5r is absolutely convergent for Re(s 1) > t$1 ..... Re(s.) > t$..

Proof. Proposition is an immediate consequence of [$2, Theorem 1] and the recent results of Kottwitz [K] and Chernousov I-C].

COROLLARY1.4 Assume that Xp(G) o = Xp(G) c. If Go., = Go n G~(xeV - S) is a connected semisimple algebraic group and d~ is a cusp form on Lo , then Z ~ (f ~o | f o ) (f ~ | f o ~ Ac (V n ) | Ae(Vo) ) is absolutely convergent for Re(s t) > Jl ..... Re(s.) > t~.,

1.5 What we must do first is to find a good condition under which the integral Z~,(f~ | can be decomposed into product of Dirichlet series (related only to fo) and local zeta functions (related only to f| as in the case where ~b is a constant function (cf. [SS, Lemma 2.5], IS1, Lemma 4.2]). For an xeV o - S o, put F~ = Fn G + . By (A-l) and [BH, Theorem 9-4], the volume t* #(x) = / d#, is finite (for d#,, see (1.I)). Also put

L (.)+ = ~'~-,G+,,c-+, . J~ ~o J, r'(.) = p(r.)(c L (.)),+ U+= G x+ nU + , Fv,x=F, nU +.

Here we note that G + c G~. We normalize Haar measures dr. and dz~ on L(.)+ and Ux, respectively by

fdv.=landf d~, = #(x). L,:,/r,x, v:/rv~

Then we have d#. = dv.dz, on G~+ . For each connected component V~ of VR - Sn, we fix a representative x~ and put

Xi=L+/L0 / (xD"+ For each x~Vi, choose t.~T +, h.~L o and u.~U + such that x = p(th.u.)x~. Define a mapping - : V~--,X i by x~--,~ = h~ L(.,)~+ X i- The point,is independent of the choice of hx and the mapping - defines a real analytic mapping equivariant under the action of L o . For x~V o n V~ and y~ Vi, set

Jg~)~(~) = f dp(h h/~ r/)dv~0/), (1.3) d which we call the mean value of ~ at x. We consider .~)~ as a function on X~. Now it is easy to check the following lemma:

Lemma 1.5. If Re(s1), .... Re(s.) are sufficiently large to ensure the absolute convergence Zeta functions with automorphic forms 105 of Zc~(s)(f| | then

Z,(s)(f~ | fo) = ,(r),E [I le,fy)r,f= :~r\vo~v, =:I~lipa(x)l, ~ v,/=l j=l

1.6 From now on, we assume that ~ is an automorphic form on L o with respect to some arithmetic subgroup. To be precise, let K be a maximal compact subgroup of L~ and rr an irreducible unitary representation of K on a finite dimensional Hilbert space W~. Denote by .~(L~ ) the algebra of bi-invariant differential operators on L~'. Let Z:.~(Lo)--* C be an infinitesimal character. Then we call a function ~b:L o --, W an automorphic form of type (Z, n) with respect to F~ if it satisfies the conditions

D~ = z(O)~b (D~(L~)), ~(khT) = n(k)dp(h) (k~K,h~L o,7~FL), is slowly increasing.

We denote by ~r Z, n) the space of automorphic forms of type (X,n) with respect to F L. It is known that the dimension of ~t(L~/FL;z,n) is finite ([BJ, Theorem 1.71 [H, Theorem 1]). Any element D~s induces an L~-invariant differential operator on the homogeneous space X~ =L o/L~+ + o, which we denote by /). We call a function ~b:X~--, W, a spherical function of type ~,n) if it satisfies the conditions

/)~, = z(D)~b (De~(L~)), ~(k~) = n(k)~b(~) " (keK, xeXt). We denote by s Z, n) the space of spherical functions of type (Z, n) on X i.

Lemma 1.6. Let dp be an automorphic form in M(L~/I"I.;Z,n). If the integral (1.3) converges absolutely, then the mean value ~r162dp at x is in 8(X~; Z, n). Our final assumption in this section is the following: (A-4) the dimension of 8(X~;z, n) (1 ~< i ~< v) is finite. Put m, = dimS(X: X, n) (1 <~ i ~< v) and take a basis {r ..... r } of g(X,; X, n). By l.emma 1.6, we can express ./r as a linear combination of ~,~",.... r

ml .Act~'q~ = ~ c~"(~b;x)~,~0. (1.4) I=I

The coefficients c~O(O;x) can be viewed as functions of x on F\Vo r~ Ft. We define (global) zeta functions ~~ s) and local zeta functions O]~174 n, X, s) 106 Fumihiro Sato by

s) -- 1r E o( )x~r\vo~v, i~i iPj(x)lS~ ' j=l

Ol')(f~; re, Z, s) = f I-I IPj(Y)IS~bI~ (1 <~ i <<.v, 1 <~ ! <<.ms). gi j=l

The zeta functions (l~ s) are independent of the choice of F. By Lemma 1.5 and the identity (1.4), we easily obtain the following:

PROPOSITION 1.7 Assume that (G,p,V) satisfies (A-I)-(A-4). Then the following identity holds for sufficiently large Re(s1),..., Re(sn):

gtt; Zc~(s)(f~o| = ~ E r (') I', n, z, s). i=1 1=1

Remark. The coefficients cl~ ) can be expressed as a linear combination of functions of x of the form (~r e), where {y,} are a finite number of points in X~, {es} is an orthonormal basis of W and (,) is the inner product on W,. Thus the coefficients of our zeta functions are, roughly speaking, mean values (or periods) of automorphic forms. The simplest case where the assumption (A-4) is satisfied is the following: The case of Gr6ssencharacters - ~ is a quasi-character of L~-. It is known that (A-4) holds also in the following two cases: Compact case - L o is a compact Lie group (by the theorem of Peter-Weyl); Symmetric case - X~ (1 ~< i ~< v) are reductive symmetric spaces (by a theorem of van den Ban, see.lB1, Cor. 3.10], [B2, Lemma 2.1]).

2. Examples of zeta functions

In this section we give some examples of zeta functions introduced in w 1 for the ease of Gr6ssencharacters, and for a certain compact ease. Examples in symmetric case will be given in w6 after establishing a general theory for symmetric case. We keep the notation in w 1.

2.1 The case of quasi-characters Let us consider the case where the automorphic form ~b is a quasi-character ofL~. Put

rl = 2rankXp(G)R - rankXp(G)c and r 2 = rank Xp(G)c - rankXp(G)R.

Let Q~ ..... Q,~ +2,2 be the basic relative invariants over C. We may assume that

Q1 ..... Q,,ER[v], Q,,+i=Q,,+,2+~(l <~i<~r2) Zeta functions with automorphic forms 107

and the basic relative invariants over R are given by

QI ..... Q,,,Q,t+IQ,,+,2+I ..... Q,,+,~Q,,+2,2. Let ~1 ..... ~,,+2,~ be the rational characters of G corresponding to Q1 ..... Q,,+2,~, respectively.

Lemma 2.1. /f r is a quasi-character of L o and the zeta inteoral Z4,(s)(f oo | fo) does not vanish, then dp is of the form

rt + 2r2 ~b(h)= H ~,(h)', i=l

= ,=,fl ~'(h)" 1-1fi 1~" +J(h'P"+J+~' ..... J'(~1)' a''+'- a...... ~(h~Lff),

where ~tI ..... %, +2,~eC and or,, +j =- ot,, +r2 +~(mod Z) (1 ~

Proof. Since r is a quasi-character, we have

.4l(~) ch(~) = dp(hy)ch(hx)- t f ~b(r/)dvx(F/). d L~)/F(x) Therefore, if Z~(s)(foo | does not vanish, the quasi-character ~b is trivial on L(+). We extend ~b to a quasi-character of G + by ~b(t) = ~b(u) = 1 for t~T + and u~U*. Thenker~bcontainsG + [L o+ , L 0+ ] U +. Denote by 1" the identity component of the center of L. Let 7"+ be the identity component of the real Lie group Ta. Put

"i'(=)= T c~ G [L, L] U and I"(+~)= 7"+ n G~ [L, L] U.

quasicharacterNote that 7"~)=7" of ~+/~-~ +ha +I-L L o + ,-ot + ] U+. Hence the quasi-character tk induces a - (x)- By [SK, Proposition 19], we have an isomorphism

rt + 2r2

T/T(~) ~ GL(1) x ... x GL(1)

t~"-*(~l(t) ..... ~j,, + 2r,(t)) and, for the group of real points, we have

rl r2

(T/T(~))R -~ R ~ x ... • R • • C ~ • ... • C ~ tr--+ (~1 (t),..., ~,, +,~(t)). Since r! r2

~_(~) ..+x...xN+xC x.-.xC , we see that the quasi-character tk is of the form given in Lemma. 108 Fumihiro Sato

We decompose the set of the indices {1,2 ..... r t + 2r2} as follows:

N {1,2 ..... rl+2r2}= LJIj, Zj=~i. j = 1 i~lj

Since L o is contained in the kernel of every character in Xp(G)a, we may choose the exponents ~ ,..., ~,t + 2,~ so that

~i=O (I <~j<~n). i~lj Put / O + .(xl \~<,,+s-~' ...... , O'(x)= rI iQ,(x)l', ~ IQ,,+j'~ txll="+s+= #l ...... s.| x-,i-j,", / ,:~ J:, \lQ,,+j(x)i) (xEVR --Sa).

By Lemma 2.1 above and (1.4), it is easy to check the following lemma.

Lemma 2.2. We have

..~)~(.~) = Q~(y)/Q~(x) for xeV o c~ Vi and yeVi.

In particular, mi = 1 and c(i)(~b; x) = Q~(x)- 1, ~k(i)(f)= Q~(y).

Put 1 l,(x)fo(x) ~(')(~,fo; s) = -- Z v(F) x,r\vo~v, Q=(x) ~ IPi(x)]'~ j=l and r o~,s) = f (-I IPs(y)l'~Q~'tv)footy)tl(y). V, j=l

Then the following proposition is an immediate consequence of Lemma 2.2.

PROPOSITION 2.3 Assume that (G, p, V) satisfies (A-1)-(A-3) and d~ is a quasi-character. Then the followin# identity holds for sufficiently larae Re(s I )..... Re(s.):

Z,(s)(f~ | = ~ ~")(~,fo; s)r174 ~, s). 1=I

Remarks. (1) Let K be an algebraic number field and V(l) the 1-dimensional vector space over K. Consider the prehomogeneous vector space

(G,p, V) = RK/o(GL(1),pl , V(1)), where Rx/o is Well's functor of restriction of scalars and Pl is the standard Zeta functions with automorphic forms 109 representation of GL(1) on V(1). Then we have

n = rankXo(G) ~ = 1, r t + 2r2 = [K:Q], rt = the number of real places, r2 = the number of complex places, and the zeta functions (~ s)are the Hecke L-functions with Gr6ssencharacters of K (cf. [Heel], [Hec2]). (2) As we remarked in the introduction, the functional equation of ~")(~,fo; s) is a consequence of the functional equations of Z, (s) and O")(f~o; ~, s). In the present case, the functional equation of *")(f~;rt, s) is a quite simple special case of [$6, Theorem 3.1]. The functional equation of Z,(s) will be proved in w Combining these results, one can prove the analytic continuation and the functional equation of ((i)(~,fo; s).

2.2 Epstein zeta functions with spherical functions

In compact case, a general theory of the zeta functions (~~ s) have been developed in [$6]. Therefore, we give here only a simplest example of zeta functions of compact case, namely, the Epstein zeta function. Let SO(n)(n~>3) be the special orthogonal group for the quadratic form q(v) = v 2 +... + v 2 and Po the vector representation of SO(n) on the n-dimensional vector space V. We identify V with the vector space of column vectors with n entries. Put G = GL(1) x SO(n) and let p be the rational representation of G on V defined by

p(t,k)v = tpo(k)v (teGL(1), keSO(n), v~V).

Then (G, p, V) is a regular prehomogeneous vector space. We consider the natural Q-structure on (G,p, V). We put L= G, U= {1}, and F= {1}. Then Lo =SO(n). Since Va - Sa = Va - {0}, we have v = 1 and

X 1 ~ SO(n)/SO(n - 1) '-' {v~Va[q(v) = 1}.

Here we identify the Lie group SO(n- 1) with the isotropy subgroup of SO(n) at Xo = t(0, .... 0, I). For v~Va- SR, the element g in X1 is given by ~ = q(v)-1/2v. Let n be an irreducible unitary representation of SO(n)= SO(n)a and d(SO(n), n) the space of functions q~ on SO(n) with values in W~ satisfying c~(kh)= n(k)c~(h) (h, keSO(n)). Then the space .at(SO(n), 7r) is isomorphic to W~ by the mapping q~* ~(I). Since .~(SO(n)) acts on ~/(SO(n), It) as scalar multiplication, we need not specify the infinitesimal character. Recall that n is said to be of class one (with respect to SO(n - 1)) if there exists a non-zero vector in W, fixed under SO(n - 1). It is not hard to see that Zr vanishes unless 7r is of class one (cf. [$4, Lemma 2.1]). Therefore, in the following, we assume that 7r is of class one. Take a non-zero vector wo~W~ with norm 1 fixed under SO(n - 1). By the irreducibility of n, such a w o is unique up to a constant factor with absolute value 1. Then the space #(Xl,n) of functions ~, on X1 satisfying ~,(ks = n(k)~(s is 1-dimensional and is spanned by the function ~o(k~o) = n(k)wo. Hence, for any ~e~/(SO(n),n) and any xeVo- {0}, there exists a constant c(dAx) such that J/~)~b(35)--c(~b,X)~o(D. It is obvious that c(q~,x)= (Jg~)~(~o),Wo). On 110 Fumihiro Sato the other hand J/~t)O(~o)= (~(1), ~o(~/))Wo. Hence we have

c(O, x) = (~(1), ~o(~)) and ~l)O(y-) _ (~(1), ~o(~))~o(~).

The zeta function and the local zeta function are defined as follows:.

(6,fo;s) = E x~vQ -{o} q(x)" '

(f~;~,s) = q(y)'-(n/2)~o(~) f ~{y)dy. d v~ - {0} Moreover the integral representation of the zeta function given in Proposition 1.7 reads

t2S-l dt dp(k) ~ fo(x)f~o(tpo(k)x)dk 0 O(n) Vo -So = r s). It is known that any representation ~ of class one can be realized on the space Hd of harmonic polynomials of homogeneous degree d; d is determined by re. For w~ W,, put P~(v) = d(~)l/2 q(v)a/2 (w, ~o(V-)), where d(~) = dim W~. Then the mapping w~-,P~ defines a linear isomorphism of W~ onto Ha. Thus we have

~(~,fo; s) = d(~)- 1/2 ~ #(x)fo(x)P§ ~vo-{o} q(x)a+(d/2)

The right hand side of the identity above is the Epstein zeta function with spherical function (cf. [E] and [Si]). In [$4], we proved the local functional equation of ~(f| and the global functional equation of ~(O,fo; s) (for fo = the characteristic function of a lattice in V o) from the same point of view as in the present paper. A generalization of this example is found in [$7].

3. Functional equation of the zeta integral

3.1 We keep the notation in w 1 and assume the conditions (A-l), (A-2) and (A-3). It is not necessary in the present section to assume (A-4). Instead we assume that (A-5) (G, p, V) is decomposed over Q into direct product as

(G, p, V) = (G, px ~ p2,E~ F) and the invariant subspace F is a regular subspace. For the definition and elementary properties of regular subspaces, we refer to IS1, w Note that, in IS1], we introduced the notion of k-regularity, where k is the field Zeta functions with automorphic forms 111 of definition. However the k'-regularity implies the k-regularity (cf. l-$6, w2.1]). Hence in the assumption (A-5), F is necessarily a Q-regular subspace. Let F* be the vector space dual to F and p~ the rational representation of G on F* contragredient to P2. Set (G, p*, V*) = (G, p~ ~ p~, E 0) F*). The assumption (A-5) implies that (G, p*, V*) is also a prehomogeneous vector space defined over Q and F* is its regular subspace. By Lemma 2.4 in [S1], the assumption (A-I) holds also for (G,p*,V*). Let S* be the singular set of (G,p*,V*). Let P*,..., P*~ be the basic relative invariants of (G, p*, V*) over Q. Note that the number of basic relative invariants of (G,p*,V*) is equal to n, the number of basic relative invariants of (G, p, V). Let X~' be the Q-rational character of G corresponding to P*: x'/(g)e*(x*)

Let Xp,(G)Q be the subgroup of X(G) o generated by Xt,...,X,.* * Since Xp(G)o = Xp,(G) o, there exists an n by n unimodular matrix U = (uo~.~: ~ such that

;Ci = I~ Xi *'. (1 i n). (3.1)

Let 2 = (2t,..., 2,) be an n-tuple of half-integers such that

(detp2(o))2 = l~i Z~(ff)2~, (3.2)

(for the existence of 2, see IS1, Lemma 2.5]). Let 0:L~/FL--. W be the same as in w1.4. Then, as in (1.2), we can define the zeta integral attached to 0 also for (G, p*,V*):

Z* (s)(f *~ Of| * --~ Z~(sl,'",* s,)(f |* | f~) = v( r) f r+ x?(tr'a(t)dtf 4'(h)

~,f~(x*)f*~(p*(thu)x*)dhdu x*~V~- S~ (fooOfo* *~ 6e(V~)|* if' 0/o)).* Now let us recall the Poisson summation formula for functions in S~(VR) | S~(V0 ). For f| E~(Vo) and x*eF o, take a lattice Le in F 0 such that the value of fo(Xl, x 2) (x~Eo,x2~Fo) is determined by the coset of x 2 modulo ff and x* is contained in the dual lattice

.~* = {x 2* ~Fol(Xz,* * ~) Z}. Put fo(Xt,X'~)=v(.Y) -~ ~ fo(xt,xz)exp[2ni(x2,x*)],

r where v(.s = ] dx2. Then f| (x 1, x*) is independent of the choice of ~ and defines dr m/.~' 112 Fumihiro SaW a function in 6e(V~). The function fo is called the partial Fourier transform of fo with respect to F. We define the partial Fourier transform f~ ~6~(V~) of f~ ~(Va) with respect to F by setting

37| (x~, x*) = ~r foo (x~, xz)exp( - 2ni(x 2, xl ))dx 2 . II Then the partial Fourier transforms

: Y'(V o ) --. ~ (V~) and ~: ~ (V R) --. ~ (V~) are linear isomorphisms and the following Poisson summation formula holds:

Y, (xt,x2)~Vo

= detp2(g) -1 ~fo(X~,X~)f| (3.3) (xl.x*)cv~ (f ~ | foEY'(Vn)| ). Let B (resp. B*) be the domain in C" on which Zr174 @fo) (resp. Z~(s)(f| @fo)) converges absolutely. Denote by D(resp. D*) be the convex hull of (B'U-1 + 2)uB (resp. (B - 2) U u B*) in C'. Then it is clear that (D - 2) U = D*.

PROPOSITION 3.1 Let f| be a function satisfyino that f| and f ~ vanish identically on S n and S~, respectively. Then Z r (s)(f | | f o) and Z~(s)(f | | f o) have analytic continuations to holoraorphic functions on D and D*, respectively, and satisfy the functional equation Z~((s - 2) U)4(f~o | fo) = Z~(s)(f~o | fo) (seD). The proof of Proposition 3.1, which is based on (3.3), is quite similar to that of [$1, Lemma 6.1] and we do not repeat it here. For later convenience, we recall the construction of functions f| satisfying the assumption in Proposition 3.1. Let r ffin-rankXp,(G)o. Then, among the basic relative invariants Pl ..... P,, of (G, p, V) (resp. P*,..., P* of (G, p*, V*)) over Q, there exist precisely n- r relative invariants which are constant as functions of x 2 on F (resp. x* on F*). Hence we may assume that

Pl(Xl,X2) = P~(xl,x~) = Pl(Xl) (i---r + 1..... n). These PdXl) (1" + 1 ~< i ~< n) are the basic relative invariants of (G, p~, E) ov~ Q. We put

P~(xi,x2)= fl P~(xa,x2) and P*(xl,x~)= fl P*(Xl,X* ). lffil lffil

Lemma 3.2. (IS1, Lemma 6.2]) (i) For an Jo0~Or174 ~,'R -S~), put

f| = Zeta functions with automorphic forms 113

Then f ~ and f ~o vanish on S a and S~, respectively. (ii) For an f~EC~(V a -Sa), put

Then f~ and f| vanish on S a and S~, respectively.

4. Prehomogeneous vector spaces with symmetric structure

4.1 In this section, we keep the notation in w 1 and assume the conditions (A-l) and (A-2). As in w 1.4, let p:Go--* Lo be the canonical surjection and put L(~) = p(G~ r~ Go) for x~V - S. We say that the semi-direct product decomposition G = LU determines a symmetric structure on (G, p, V) over Q if, for any x~V o -So, there exists an involution (= an automorphism of order 2) r defined over Q such that

L~: = {heLol~(h) = h} = L(x) ~ (L~)~ (4.1)

Then, for any xeVn - Sa, there exists an involution a of Lo defined over R satisfying (4.1). The involution a induces an in'r of L~, which we denote also by a, satisfying

(L~")" = L o c~ L(x) = L (x)+ =((Lo)~) o. Therefore the homogeneous spaces X~(1 <<.i<<.v) defined in 31.5 are reductive symmetric spaces (symmetric case in w 1.6) and the construction of zeta functions given in w1 can be applied to (G, p, V) with symmetric structure.

Lemma 4.1 Suppose that (G, p, V) satisfies the condition (A-5) in w2, namely, V contains a regular subspace F. Then the decomposition G = LU determines a symmetric structure also on (G, p*, V*), the prehomogeneous vector space dual to (G, p, V) with respect to F.

Proof. By (A-5), one can find a relative invariant P of (G, p, V) with coefficients in Q for which the rational mapping ~be:V- S ~V* defined by

~be(x 1, x2) = (Xl, gradx2(log P(x,, x2)) ) gives rise to a G-equivariant biregular mapping of V - S onto V* - S* defined over Q. For x~V o - S o, put x* = ~e(x)r - S~. Then we have G~ = G x. and L(~) = Loe) (cf. iS1, Lemma 2.4]). Now the assertion is obvious.

Examples. (1) Let (G, p, V) = (GL(n), p, Sym(n)), where Sym(n) is the space of symmetric matrices of size n and the representation p is given by p(0) x = 0x'0 (ff~ GL(n), x ~Sym(n)). Then the trivial decomposition G = LU with L = GL(n) and U = {1} determines a symmetric structure. In fact, we have Lo = SL(n) and, if we define ~(h) = x'h- Ix- 1(hELo), then L~=L(x)~-SO(n) for xeV o-S o. The corresponding symmetric space is SL(n)/SO(n). 114 F umihiro Sato

(2) More generally, if (G, p, V) is an irreducible regular prehomogeneous vector space of commutative parabolic type, then the trivial decomposition G = LU with L = G and U = { 1 } determines a symmetric structure (cf. I-Ru] and [BR]). (3) Let (G,p,V)=(GL(1)xSO(m)xSL(n),p,M(m,n)) (m~>n~>l), where the representation p is given by

p(t, k, g)x = tkxg- 1 (t~GL(1), g~SL(n), k~SO(m), x~M(m, n)).

Then one can define two symmetric structures on (G, p, V). In fact, the following two decompositions

G = L x U,L = GL(1) x SO(m), U = SL(n) and G = L x U, L = GL(1) x SL(n), U = SO(m) determine different symmetric structures. The corresponding symmetric spaces are

SO(m)/(SO(n) x SO(m - n)) in the former case, SL(n)/SO(n) in the latter case.

4.2 Let P~. be a parabolic subgroup of L and put P = P,U. We denote the restriction of the representation p to P by the same symbol p. We do not assume that PL is defined over Q. In fact, in w5, we need to consider a parabolic subgroup defined over R.

Lemma 4.2. Suppose that G = LU determines a symmetric structure of (G, p, V). Then (1) (P, p, V) is also a prehomogeneous vector space. (2) If (G, p, V) is regular, so is (P, p, V).

Proof. Let V 1 = {x~VlPl(x ) ..... P,(x) = 1}. Then V 1 is a single p(Go)-orbit (cf. [$6, Lemma 1.1]). Fix a point XOEVI. The mapping ~:V1--,Lo/L~xo) defined by [3(p(hu)xo)=h.L,~o)(heLo,u~U) is clearly Lo-equivariant. Since Lo/L,~o)is a symmetric space and PLo = PLnLo is a para~bolic subgroup of L o, there exists a Zariski-open PLo-orbit [2o in Lo/L,~,o ) (see [V, w Then Q=p(T)~-l(f~o) is a Zariski-open P-orbit in V. Hence 0P, p, V) is a prehomogeneous vector space. The second assertion is obvious.

We denote by Se the singular set of (P, p, V). It is obvious that Sv ~ S. Recall that the parabolic subgroup PLo = PnLo of L o is called o-anisotropic for an involution o of L o ff PLon~r(PLo ) is a Levi subgroup of PLo (cf. IV. w 1]).

Lemma 4.3. Suppose that G = LU determines a symmetric structure of (G, p, V) and PLo = PL~Lo is o-anisotropic for the involution a corresponding to some xoeV- S. Then

(1) the point x o is in V- S e. (2) For xeV-Se, the isotropy subgroup Px= {pEPIp(p)x=x} is (not necessarily connected) reductive. (3) The singular set S e is a hypersurface. Zeta functions with automorphic forms 115

Proof. We use the notation in the proof of Lemma 4.2. By replacing x o by p(t)Xo(t~T ) if necessary, we may assume that xoeV1. By IV, Theorem 1] and the assumption that PLo is r we see that//(Xo) is in fl o. This implies the first assertion. To prove the second assertion, it is sufficient to consider the case where x = Xo. Since the identity component of Pxo coincides with that of (Pt.o?U)xo= Pxo n(PLo'U), we prove that (PLo.U)xo is reductive. By (A-I), Gxo is reductive; hence its normal subgroup U~o--UraG~o is reductive. Put L o = P~nor(PLo ). By the assumption, L o is a Levi subgroup of PLo" The identity component of PLo n L.xo~ coincides with that of (Lo)~. Hence the group Pt.o~L~o~ is reductive (cf. IV, w1~). Since (P~-U)~,o/U~o----.PLoc~ Lr this proves the second part. The third assertion is an immediate consequence of the second. II

Remarks. (1) The prehomogeneous vector space (P, p,V) does not necessarily satisfy the latter half of (A-I), even if it is defined over Q. (2) Under the assumption in Lemma 4.3, we put

L -'o - PLonCr(PLo), L' =TL o, U' = R,(PL)U. As the following example illustrates, the semi-direct product decomposition P = L'U' determines a symmetric structure of (P, p, V). Example. (1)' Let (G,p,V) be as in Example 1 and put O(o'n- P)~ 6

GL(m)I a, CL(p), 03 M(n- p,p),g,~GL(n- p)}.

Then PLo = P c~SL(n) is cr-anisotropic, where cr is the involution given by ~r(0) = tg-1. We have

glcGL(p),g4~GL(n- p ~ GL(p) x GL(n- p) ' g4~ ,t

The basic relative invariants of (P, p, V) are given by

PI(x) = det(xl), X1(0)-- det(gl) 2, Pz(x) = det (x), X2(g)= det(g)e, where xl is the upper left p by p block of x. We have

sp -- {xeVl P (x) = o} {x VIP2(x) = o}. For xo = 1,eV- Sv, the corresponding symmetric space is

(SL(p) x SL(n- p))/(SO(p) x SO(n- p))

= (SL(p)/SO(p)) x (SL(n - p)/SO(n - p)). 116 Fumihiro Sato

4.3 Let the assumption be as in Lemma 4.3. Take a field k such that PLo and the involution tr are defined over k, and xoeVk- S~. We examine the group Xp(P)k of k-rational characters corresponding to relative invariants of (P, p, V). For simplicity, we assume that (A-6) the basic relative invariants P1,...,P, of (G,p,V) over Q are absolutely irreducible. This is equivalent to the condition

Xp(G)o = Xp(G)c.

Let T o be the identity component of the center of L o. The central torus T O is a-stable. Hence we get a separable isogeny To. + x T o_ ~ To, where

To+ = {t~Tolo(t ) = t} ~ and T o_ = {t~Tola(t ) = t-a} ~

We consider the following commutative diagram of the natural mappings:

x(P), ~ x~),~xff'o), T ~ restriction to T x T~_ Xp(P)~ ~ X(T)~xCr'o_)t.

Here note that X(T)k = X(T)o , since T is a Q-split torus.

Lemma 4.4. The homomorphism ~ :Xp(P)k-~ XOF)k ~ X(T o_ )k is injective and of finite cokernel.

Proof. Any character X in X,(P)k is trivial on P,oU. The group P,oU contains (P~,oC~L(xo~)U and hence ((Lo)r176 Since To+ is a subgroup of ((Lo)~)~ g is trivial on To+. This implies that ~ is injective. As we have already seen in w 1.2, ~(Xp(G)~)(= ~(Xp(G)o) by. (A-6)) is of finite index in X(T)k(= X(T)o). Let g be a k-rational character of To. Then, for some integer e~, g" can be extended to a k-rational character of P such that ker X"1 contains TI" o + D(Lo)U', where D(Lo) is the derived group of L o. Since (p)o is contained in Tro+ D(Lo)U', there exists an integer e such that ~" is trivial on P,. This implies that x'~Xp(P)k. Therefore ~(Xp(P)k ) is of finite index in X(T)k~)X( T'0-)k"

Let P1,...,P,,P,+I ..... P,+i be the basic relative invariants of (P,p,V) over k, where P1 ..... P, are the basic relative invariants of(G, p, V). We have l = rank X(T o_)k by [.emma 4.4. Let l,+ ~..... X,+l be the k-rational characters corresponding to P,+, ..... P,+~, respectively. Take a positive integer e such that (x~lr) (n+ 1 <<.i<~n+l) are in ~(Xp(G)k). Then one can find mijEe-tZ(1 ~< i ~< n, 1 ~

X,+~" ~ ~m,j_= 1 on T. (4.2)

These m o will play a role in the algebraic construction of the Poisson kernel in w5. Zeta functions with automorphic forms 117

5 Functional equations--The case of symmetric structure of K~-type

Let (G, p, V) be a prehomogeneoos vector space with symmetric structure G = L'U satisfying the assumptions (A-l), (A-3), (A-5) and (A-6). The assumption (A-2) is automatically satisfied. In this section we prove the functional equation satisfied by zeta functions attached to automorphic forms under the following assumption: (A-7) Lo is semisimple and the symmetric spaces Xi = Lo/L(+,) (1 ~< i ~ v) are K,-spaces in the sense of [OS].

5.1 Let K be a maximal compact subgroup of L~ and 0 the corresponding Cartan involution. Let Po be a minimal parabolic subgroup of L~ with Langlands decomposi- tion Po = MAN with respect to 0. Denote by ~o, m and a the Lie algebras of L~', M and A, respectively. Let Y~(c a*) be the set of restricted roots and Y-+ the set of positive restricted roots corresponding to Po. Put 9~o = {X~P.oI[H,X ] =~t(H)X, H~a} for ~t~Z. Following [OSJ, we call a mapping 8:E ~ { • 1} a signature of roots if it satisfies the condition

8(,,) -- 8(- ~) (~:), 8(a + r) = 8(~)8(f) if ~, five and a + fl~Z.

For a signature of roots 8, define an involution 0~ of s by

O (X) = 8(- ~)O(X) x~.~o,~Y., O,(X) = O(X) X~m + a.

Then a precise formulation of the condition (A-7) is as follows: (A-7)' for each i = 1..... v, there exists a representative xiE V i and a signature of roots 8i such that L+(x,)= M. K,~ where K ~ is the connected analytic subgroup of L~ with Lie algebra

t,, = {x~olo,,(x)= x}. In this case, one can apply the results in [OS] to the homogeneous spaces + + X i = L o/L(x,). Let W= NK(A)/Zx(A) be the Weyl group. Note the M = Zr(A). Define a subgroup W ") of W by W (i)= (L(+)c~Nx(A))/M. Put r I = [W: W (i)] and fix a complete system {~(1) w(i)~ of representatives of W/W (i). Then, by [OS, Proposition 1.10] (or by "'| ' " " " ' "'rl J [Mat]), the set

rt rt U ANw(]'L(+o = (,J PoW~)L(+~,) (disjoint union) (5.1) jr1 j=l is an open dense subset of L~. Let PLo be a minimal R-parabolic subgroup of L o such that P~,a r~Lo = Po- The parabolic subgroup PLo is 0 -anisotropic. Put

P = TPLoU and P+ = T+Po U+. 118 Fumihiro Sate

Note that P+ is not necessarily connected. Using the notation in w we have T o=To_andTo.+={l}. As in w let Se be the singular set of (P,p,V). Then it follows from (5.1) that the P+-orbit decomposition of Va -Se, a is given by

v. - s,.. = i=1 jft

Let P1 ..... P,,P.+I ..... P.+~ be the basic relative invariants of (P,p,V) over R. As in w P1 ..... Pn are the basic relative invariants of (G, p, V). We have != dim A in the present case. Let mo (1 ~< i ~< n, 1 ~

Ip~(x)[ = Ie,+~(x)l/,ffilfi Ie'(x)l"~ (i <~j <~ 1) on Va - Sa satisfies that ]p j(p( tmanu)x)] = X. +~(a) lp ~(x)l (t~ T +, m~M, a6A, n6N, ur U +, x~VR - Sa).

This implies that IPfl defines a function I/~jl on X,:

IpJI -\ /,,, Xi By Lemma 4.4, {Xn+t, .... X.+z} gives a basis of X(T;).| We can identify X(To)a| with a t--a*| by X(To)agX~--~log(xeexp)r For gear, write = 2~= 1~'~l~ +j o exp) and put

I Ip(x)l ~= I-I Ipj(x)l xJ (x~Vn-Sa) ./ffil and ip(x)l~= tlP(0)l~ ifx~V~, otherwise. The function Ip(x)l~ is well-defined for Re(21)..... Re(2z) > 0 and we define [p(x)]~ for arbitrary 2~a~ by analytic continuation. We denote by [p(~)[~ the function on Xl induced by Ip(x)l~. Then ]/~(~)l~ 0i -- h~,heLo) (1 ~j ~< r~) coincide with the functions exp{ - 2(H~ ~(h- 1)) } defined by lOS, (3.33)].

5.2 Now we examine the space 8(Xi;~,X) of spherical functions of type (~,X) introduced in w1.5. Let~(X~) = ~(L o/L(x,)+ + ) be the algebra ell o+ -invariant differential operators on X I. Denote by .~(X~) the subring of ~(X~) consisting of the restrictions D of bi-invariant differential operators D in .~(L~ ). It is known that .~(X~) = ~(X~) if L~ is of classical type and ~(X~) is a finite ~(X~)-module in general ([Hell, w7], [Hel4]). Zeta functions with automorphic forms 119

Let

§ § --- "~i:~(Xi)( '~ U(~o)L,~,,/(U(~.o)L,~,,,r U(.~o)([e,)) ) ~ U(o) 1r be the standard isomorphism of ~(X~) onto the ring U(a)w of the Weyl group invariants (cf. lOS, w2.3], [Hel3, Chap. II, w4, w5]). For/~ea~, we obtain an algebra homomorphism of U(a)w into C by extending it to U(a)rr which we denote by the same symbol. Put

X,: = # o ~: ~(X~) --, C. (5.2)

Let n be an irreducible unitary representation of K on a finite-dimensional Hilbert space W~ and X:~(Lo)--*C an infinitesimal character. It is obvious that

~(Xi; n, ~) = {0} unless X:.~(L o ) --, C factors through .~(X~) = ~tL 0+/t. /~(x0 + ~"r

X:a (Lo) ; c

In the following, we assume that s n, X) v~ {0} and denote the character of .~(X~) induced by X also by the same symbol. For #ea~, put

8o(X,;n,Z~,)= {r W~[ ~'(k~)=n(k)r (keK,~eX,) O~= x,(O)r (Oe~(X,)) .,"

Since ~(X~)D .~(Xi), we have

8o(X~; n, X,) ~ ~(Xi; n, x~l:z~x,~). On the other hand, since ~(X~) is a commutative algebra, the ring ~(X~) acts on s n, X) (Z~Hom(~(L~), C)). We assume that the action of ~(X~) on s n, Z) is semisimple, namely, (A-8) there exist a finite number of #1,...,/~a~ such that

d ~'(Xi; n, Z) = (~ co(X,; n, X,j)- (5.3) j=l Remark. The assumption always holds unless the symmetric space X i contains an irreducible symmetric space of type EVII or EIX ([Och]). Even in this exceptional case, the assumption holds for generic Z. By I-as, Lemma 2.241,/~1 ..... ]~d do not depend on i = 1,..., v. If G is of classical type, then ~(X~) = .~(Xi) and d = 1 for any X. In some exceptional cases, it may occur that d >/2; however, for a generic Z, we may take d = 1 ([Hel4]). Now we define End(W~)-valued 'spherical functions W~(~) (~eXi) by analytic continuation (as a function of ~) of the integral

W~,)~(~):---f I~(k-t.~)l~+Pn(k)dk (Izea~, 1 ~ i<. v, 1 <~j <~ r,), (5.4) K 120 Fumihiro Sato where p -xZ - z ~z+ a and dk is the normalized Haar measure on K. (We have used p to denote a prehomogeneous representation of G. This double use of the symbol will not cause any confusion, since the meaning is clear from the context.) Then, using the Poisson integral representation of eigenfunctions on X~ of invariant differential operators ([OS, Theorem 5.1]), we immediately obtain the following proposition:

PROPOSITION 5.1 If #~a~ satisfies 2(#,a)/(a,~)r for all aeZ, then ~P~.)~'(~)is holomorphic at # and the linear mapping

~,.:(W~) ~" ~ r ~, X~) rl

j=l is an isomorphism, where W M, = {veW~]~(m)v = v(meM)}.

Thus we have constructed a basis of 8(X~; ~, X) for genetic #ea t. Let #~ ..... #d be the elements in a t appearing in the right hand side of the decomposition (5.3). We assume in the following that 2(# z, 0c)/(~, 0c)r for all aeZ and 1 <~l~d. Then, for Or and xeVanV~, one can find elements v~l,~,(~b;x)~ W~ such that

d rf (0 - I=I J=l We put .__1 ~. l.t(X)fo(X)V.t.~,(O;(1) X ) r (~,_(~), fn ; s) = v'(1) ,=r\vo~ v, I'I~=,IP,(x)l" and

= ,O1 Now Proposition 1.7 can be formulated as follows:

PROPOSITION 5.2 When Re(sl)..... Re(s.) are sufficiently large, we have

i=l j=l I=1

Note that ~l~(O,fo;S ) are Dirichlet series with values in W~ and O~~174 are local zeta f'unctions with values in End (W~). Hence the products of JO(/)(f| ;r, #z, S) and ~lJ(0,fo;S) in Proposition 5.2 are well-defined.

5.3 Let (G, p, V) -- (G, p~ (D P2, E ~ F) and assume that F is a regular subspace. Denote by (G, p*, V*) the prehomogeneous vector space dual to (G, p, V) with respect to F. Zeta functions with automorphic forms 121

In the following we indicate with the superscript * the notation for (G, p*, V*). For example, we denote by Pt,...,P,** the basic relative invariants of (G, p*, V*). Take a relative invariant P of (G, p, V) with coefficients in Q such that ~be defined in the proof of Lamina 4.1 gives a biregular mapping of V - S onto V* - S*. Since Se is defined over Q and G-equivariant, we have a one-to-one correspondence of

P+-orbits in V R - Sj, , a and those in V~ - S*P,R" Hence we have

= o, v~=r247 i=l j=l

Since L~,)= L(+x,)forx * = ~p(X,), we may identify X i = I~/L(+)with X*=/~+/L(+,) and the assumption (A-7) holds also for (G, p*, V*). Moreover we have the commutative diagram

Xt For x* = r E), it is easy to check the following identity:

i~*(~*)1~ +0 = i~(~)1~ +0.

If x is in V~c~Vo (and hence x* is in Vec~V~), than we have .a~.r(i) - _ (~x,), v~,(,; x*) = v~';,(r x).

The zeta functions and the local zeta functions associated with (G,p*,V*) arc defined as follows:

~*",(~,f*;s)= 1 F~ #(x*)f~(x*)v~"~'(4';x*) ~.~ u v(r) x*~r\v~v,, l~ IP*(x*)l" '

(I)~*(O(f*~;x,gt,s) = ~ l~I [P,. (Y) Is, Vi,j~.m (Y)f|-* * *)f~*(y* ). Jv* t=!

Theorem 5.3. For any f~(VR) and f~e~(Vn),* * the integrals ~i)(f~;Tt, p,s), ~.(i)tf,. ~ , s) (0z, s)ea~ x C ~) converge absolutely, when Re((# + p, 0t>)< 0 for all ~A and Re(st),...,Re(s,) are sufficiently large. Moreover they have analytic continuations to meromorphic functions of (p, s) in a~ x C' and satisfy the functional equation

F~, r (p, s)~j. (f| 7t, #, (s - it) U), i*=1 j*---I where F(j~'je)~,s) are meromorphic functions independent of f~ and ~ having an elementary expression in terms of the gamma function and the exponential function. 122 FumihiroSato

Remark. In the case where (G, p, V) is of the form (G, p, V) = Rc/n (Gt, p~, Vl), where (G t, P t, V t) is an irreducible regular prehomogeneous vector space of commutative parabolic type, Bopp and Rebenthaler ([BR, Th6or6m 3.2]) obtained a functional equation that is essentially equivalent to the local functional equation in the theorem above.

Proof From (5.4), we have

Since P/s are K-invariant, we obtain

(5.5) Similarly we obtain

(1)*(i)[f* "X #l $~

:IilP*(y*)l"'[p*(y*)l'+P{fKf*(p*(k)y*)x(k)dk}fl*(y* ), (5.6) = fv~ t= Putting t~ = Y~ffit/~j log(x,+~oexp) and p = Z~= 1P)~176 we have

leI te,(y)lS,-6,.im)l.+p = ) IPn§ t=l i=1 j=l

If A~X(T0)a is a dominant weight, then, for some positive integer m, - mA corresponds to a relative invariant that is regular on V-S (cf. IV, w This implies that, if Re((/~ + p,~>)< 0 for all a~A, then Re~j + p j)> 0(j = 1,...,l). Hence the integral (5.5) is absolutely convergent, when Re((p+p,a>)<0 for all ~EA and Re(s1),..., Re(s,) are sufficiently large. The proof of the convergence of the integral (5.6) is quite similar. Since any matrix coefficient of

is a rapidly decreasing function on V a (resp. V~), the integrals O~)(resp. O~(~ can be viewed as local zeta functions associated with (P, p, V) of the type considered in [$1, w5] and have analytic continuations to meromorphic functions on a~ x C" (d. [BG], [KK], [Sab]). We note further that, for u,vr

Applying IS1, Theorem 1] to (P,p,V), we can see that there exist meromorphic Zeta functions with automorphic forms 123 functions F~.~.~)Oz,s) on a~ x C" such that the functional equation

P=I j*=l holds for all u, vr Pc',. This proves the theorem.

For (p,s)~a~ x C ", put

. H P,(r)- H P,+j(Y~ , l= l j'= l where/~j and pj are the same as in the proof of Theorem 5.3 above. Let P* be the relative invariant of (G, p*, V*) introduced just before Lcmma 3.2 and X~ the rational character of G corresponding to P~. Then, by [$1, w3], there exists a polynomial br(s, #), the b-function of (P, p, V) with respect to F, satisfying

p* ( y l, ~---~z)p .,(y) = b~(s,g)P ,.,+,(y), (5.7) where a=(al,...,a,) is defined by X~=~'...~". We can similarly define the b-function b*(s, lO of (P,p*,V*) with respect to F*. Now we are in a position to prove the functional equation of the zeta functions ~e.~(O,fo;S) and (~f)(O,f*;s).

Theorem 5.4. Assume that 2 /(~t,~t>r for all ~tr and 1 <~l <~d. Then (1) the zeta functions (~.[(~,fo; s) and ~f)(~,f~; s) can be extended to meromorphic functions of s in D and D*, respectively (j'or the definition of D and D*, see w3). (2) The functions br(s, lzt)(~(dp,fo;s) and b*(s,iQ(Tf)(dp, f~;s) are holomorphic functions of s in D and D*, respectively. (3) The followino functwnal equation holds for any for

l~l j=l

Proof. (1) and (2): Let the notation be as in w For an ]~oe o t d, put f~ = P~(xl, t~/t~xz)f'(xt, x2). Then, by Lcmma 3.2, we can apply Proposition 3.1 to f| and we see that the function

rt d Z~(s)(f~ | fo)-- ~ ~. ~)(f~;~,gl,s)~(q~,fo;S) j:l i:1 is a holomorphic function of s in D. On the other hand ~O(f| ; 7r, #,, s)

ijt:l LdK \ if the integral in the right hand side of the identity is absolutely convergent. Since 124 Fumihiro SaW

P* is K-invariant, we have

Hence, integrating by parts, we obtain

r oo; ~, #,, s) = +_ be(s, #t)~'(f~; lr, #r s + a), where we use the identity (5.7). This identity holds for any s and /zt by analytic continuation. Thus we see that

r~ d Z~,(s)(foo| E E +-be(s, gt)r176174 (5.8) j=l I=1 is a holomorphic function in D. Now we need the following lemma, whose proof is not hard and is omitted.

Lemma 5.5. Let V and W be finite-dimensional C-vector spaces. Let 9X --* Hem ( W, V) be a Hem(W, V)-valued continuous function on a domain X in R N. Take a basis {w 1..... w~} (n = dim W) of W and let ~Fl:X--* V be the function defined by ~Fl(x) = ~d(x)(wi). Assume that the functions ~Ft ..... ~F, are linearly independent over C. Then there exist f l ..... f, eC o| ( ) such that the linear mappith9 of W into V ~" defined by w w (fx'etx)(w)f, tx)dx..... fx(x)(w)f, is injective. When 2(#~,~)/(~, ~)r ~

rl d 9 (x)(v)=l~I IP,(x)l~'+~'-~' ~ ~ ~t'~.)"(~).vj, k=l j=l I=I

(v = (v~z)~ e(WM)~"). = 1,...,rt f = l,...,d

Hence, by (5.8), we see that the functions be(s,#z)~i~(dAfo;S) are holomorphie in D. The holomorphy of b*tsr, ,~-l,*j.z,, ~r*(i)t~ ,~-,Jo, r s~, can be shown quite similarly.^ (3): Now we take j|162176*~..~o , t', and put f| ee(xl,x2)'f~(xt,x2). Then we can apply Proposition 3.1 to f~o and get the functional equation

Z$((s- 2) U)(f~ | fo) = Z~(s)(f| | fo) (seD). By Proposition 5.2 and Theorem 5.3, we have

ri* d E E (s - (s - u) j*=l I*=l Zeta functions with automorphic forms 125

d = ~P} (f~, ~, #v, s)[j, (~P'fo, s) i=1 J=l l=l d ~- j,j* ~1~1*' ! j* ~Joo' 'l~i*'~ "'l~l~jl~1'dO,~ i=1 j=l 1=1 Therefore r|* d E E v) j*=l 1=1

By an argument based upon Lcmma 5.5, we see that the functional equation

= r,. r ~z,s)~(~,fo,S) iffil Jffil holds for any s~D.

Remarks. (1) As we mentioned in the introduction, the functional equation of zeta functions are based on local functional equations, the existence of b-functions and the functional equations of the zeta integrals. In the case considered above, the local functional equation (Theorem 5.3) and the b-function (5.7) are reduced to the usual local functional equations and the b-functions of the prehomogeneous vector space (P, p, V). (2) By the results of Oshima [O], Proposition 5.1 can be extended to the case where the symmetric spaces X~ = L o/L~x,)+ + are not necessarily of K -type. By a similar argument based on the extended version of Proposition 5.1, we can extend the functional equations of zeta functions attached to automorphic forms to more general prehomogeneous vector spaces with symmetric structure not of K~-type. In the general case, P is not necessarily minimal parabolic, and the functional equations axe reduced to the local functional equations of local zeta functions attached to representations of the levi part of P, which belong to the class of local zeta functions studied in [$6, w3].

6. Examples of functional equations

In this section, we present some examples of zeta functions, to which we can apply the results in the previous section. We retain the notation in w5.

6.1 Zeta functions of Maass attached to positive definite quadratic forms

First we consider (a special case of) the zeta functions studied by Maass in [M1], [M2] and [M4]. Let SO(m) be the special orthogonal group of the quadratic form v~ +... + v~. The prchomogeneous vector space to be considered is (G, p, V) = (SO(m) x GL(n), p, M(m, n)) (m >1 n, m 1> 3), where the representation p is given by p(k, g)x = kxg- 1 (k~SO(m), g~GL(n), x~M(m, n)). The singular set S is given by

S = {xeVldet(x~x) = O} 126 Fumihiro Sato

and P(x) = det(x'x) is the basic relative invariant. We can naturally regard (G, p, V) as a prehomogeneous vector space defined over Q. Then Vn-Sa = {xeM(m,n;R) Irankx = n} and this is a single G+-orbit. Put Xo = (,.)ly,,_n.n) Consider the symmetric structure defined by putting L ffi GL(n) and U = SO(m) (el. w Example 3). Then L~ = SL(n): = SL(n)n, L(+~o)=SO(n): = SO(n)n and v = 1. The corresponding symmetric space is X: = SL(n)/SO(n). We identify X1 with the space of positive definite symmetric matrices of size n with determinant 1. Let a be the set of real diagonal matrices with trace 0. The complexification ac is a Cartan subalgebra(i of ~oc.o)) We define Ai~a ~ (1 ~< i ~< n - 1) by At (, ". ffi at + "'" + at. \ an

Then {A 1..... An-1} forms a basis of a t (the fundamental dominant weights). We put 2=ZT-~21Ai~a~(21~C). We denote by X=X~ the infinitesimal character of ~'(Xx) = ~(X1) defined by (5.2). The character X canonically induces a character of ~(L~), which we denote by the same symbol X. Let n be an irreducible unitary representation of K = SO(n) and W= its representation space. Put

~F~(~) = n-~H dt('kfk)-(~'+ l)/2n(k) dk, .v = (det'yy)X/n' (6.1) fsO(n) I=1 ,yy

where di(A) is the determinant of the upper left i by i block of a square matrix A. In the present case, Proposition 5.1 is the integral representation of K-finite eigenfunctions on X due to Helgason ([Hel2, Corollary 7.4]) and any element in 8(X1; n,X) is of the form W~(p)v (veWU~). Put F ffi SL(n)z. Since an automorphic form 4~ealt(L~/T'; n, X) is slowly increasing and X: is a riemannian symmetric space, the condition (A-3) is satisfied. By (6.1), for each xeV o -S o, one can find a unique v(~, x)e W~ such that

.$r f 4J(hykh~l)dk = Wx(~)v(4J,x). ds O(n) In particular, if n is the trivial representation % and 4~ is SO(n)-invariant, then W~(~)=oJ~), the zonal spherical function of SL(n). Any automorphic form 4~eM(L~ /r; no, X) can be viewed as a r-invariant function on X 1 and we have v(~, x) = ~(~- 1). Now return to the case of general n. For a r-invariant foeLC(Vo) and an f| the zeta function and the local zeta function arc given as follows:

~(~b,fo;s)__ ~ fo(x)v(4~;x) x~ve/r (det txx~'' rnnkxfn

O(f:o; 7t, 2, s) = ~ (det'xx)'-'/2 ~P~(~)f| (x) dx. Jv a-Sa Zeta functions with automorphic forms 127

By the proof of Theorem 5.3 (or by the integral representation (6.1)), the local functional equation satisfied by ~(f| n, 4, s) can be reduced to the functional equation of

~(~)(f~; S, 4) = I-I di(x) -('h + 1)/2(det'xx)s-m/zf~(x) dx, R__SR /:1 which is the local zeta function of the prehomogeneous vector space (SO(m)x B(n), p, M(m, n)), where B(n) is the subgroup of upper triangular matrices in GL(n). Using the functional equation of ~(| 4) calculated in IS, pp. 155-1561 we obtain the following explicit functional equation, which connects ~(~b,fo;S) to ~(~,fo; s), where ~(h) = ~('h - l). Note that ~ ~r (L~-/F; n, Xx) where ~ = Z,="-', 4, _ tA,.

Theorem 6,1. (I) The zeta function ((O, fo;s) has an analytic continuation to a meromorphic function of s in C. (2) Set

/ 1"-1" -1"~ x4~ n-1).s

Then m ) A(~,j?o;S) = A ,fo;-~- s .

Remarks. (1) In [M1], [M2], and [M4], Maass considered the case where n is the trivial representation no of SO(m). (2) The zeta function ((dp,fo,s) is closely related to the Eisenstein series of SL(n) corresponding to the standard parabolic subgroup defined by the partition (n, m - n) (cf. IT, Chapter IV, w (3) If n, m - n ~ 2, then the results in this section can be generalized to arbitrary Q-forms of(SO(m) x GL(n), p, V) including the case where the quadratic form defining SO(m) is indefinite.

6.2 Zeta functions with Maass cusp forms

Let G = GL(2), V = Sym(2)= {xEM(2)l'x = x}, and define a rational representation p of G on V by p(g)x = 0x'0. Then, (G,p,V) is a prehomogeneous vector space with singular set S={xeVIdetx=O}. The prehomogeneous vector space admits a symmetric structure defined by L = G and U = {1} (of. w Example 1). Let ~ be the upper half plane and O:~--.C be a Maass cusp form with respect to 1~ = SL(2)z. Then has the following properties:

(r.z) = ~(z) (vrED; (6.2)

a~(z) = 4(4 - 1)~(z), A = y2 -t- ; '(6.3) 128 FumihiroSato

for any r > 0, O(z)= O(e-Y) as y--, oo. (6.4) Here we note that the estimate in (6.4) is uniform with respect to x = Re(z). For gcL~ = SL(2): = SL(2)R, we put ~(O) = O(O- t.~_ 1). Let

Then $(fl) satisfies the following conditions:

6(koa~) = r (0ER, r~F), e~b = 2(1 - A)~b, where ~ is the Casimir operator of L~. We consider the natural Q-structure on (G, p, V). Take a lattice L in V o = Sym(2, Q) that is stable under F. Let fL be the characteristic function of L. For f| ~ (Vn), set

|174 = f $(g) X f=(gx'g)dg. JL~/F x~L - S Then we have

Z~,(s)(f~| = t 2, O(•, L 'f•)t'' dr where f~(x)= f| Though the prehomogeneous vector space (G, p, V) does not satisfy the latter half of the condition (A-l), it follows from the estimate (6.4) that the zeta integral Z,(s)(f|174 is absolutely convergent for Re(s)>{. Note that the convergence of the zeta integral is sufficient for the application of the results in w5. Decompose Vn -Sn as follows: V~ - Sn = V+ ~ V_, V+ = {xCVnldetx > 0}, V_ -- {xCVnldetx < 0}.

For I2 = (10 01) and J2 = (~ :),putH+=SO(12), H-=SO(J2):

H+ - $0(2) = K = {k010 ~< 0 < 2~},

a_~ z aERX } . We normalize the Haar measures dg on L~ and d#• on H+ by d0 dx dy dg = y2 ' g = koayn*' Zeta functions with automorphic forms 129

dO dy d/~+(k0)=T, du-(+a,)=~-yy

Then we have the following integration formulas:

dt f(t, g)dg = idet x13/2 f(tx, g~k)d/a + (k) f0~T L~ f v,~ n+

v5 idetxl3/2 +f(t~,g~k)d~+(k)

v_ [detx[3/2 _ f(t~,o~h)d~_(h), (6.5)

where V~ (resp. V~ ) is the set of positive (resp. negative) definite symmetric matrices in V+. Here tx and g~ are defined as follows. Let X+ = {xsVP+ldetx= 1} and X_ = {xeV_ldetx= - 1}. For xsVR-SR we take txeN~ and geL o such that (t~(gj2%) ifxEV~, x =4- t~(g~I2'g~) ifxeV%, [tx(gxJ2'gx ) ifxe V_.

We put ~ = O~I2tg~ or gxJetgx according as detx > 0 or detx < O. For an xeL+ = Ln V+ and a yeV+, consider the mean value

-/(x~(37) = f q~(gyhgf t)d#• (h). dn :t/gf t F~g. Then Jt'x qb07) is absolutely convergent and belongs to the space 8(X+ ; 2) of functions ~b satisfying ~b(k~)= ~(,2) (k~r,~eX• ),

a+_ ~ = 2(1 - 2)r

where A+ is the L~-invariant differential operator on X_+ induced by the Casimir operator cr The space s ;2) can be determined explicitly as follows (cf. [Ro, Theorems 6 and 10]. See also [Se]). In this case, we can remove the restriction on 2 in Proposition 5.1.

Lemma 6.2. (1) Assume that ~ is in X +. Let e = and e -~ be the eioenvalues of 2 (~eR). Then any spherical function in 8(X+;2) is a constant multiple of

~(2) = P_ ~(cosh(~)), where P_x(z) is the Legendre function and is defined by the Gauss hypergeometric function as follows:

P-~(z)=F(2,1-2;1;~-~). 130 Fumihiro Sato

(2) Assume that ~ is in X_. Let e ~ (resp. -e -~) be the positive (resp. neoative) eioenvalue of ~ (~ E R). Then any spherical function in $'(X_ ; 2) is a linear combination of qjto)(~) = cosh(- ot)-l/2Pa-(t/2)ftanh/-__t/2,--" ~)), ~F~I)(~) = cosh (~) - t/2 P~-t~t,/2)(tanh (*t)),

where ---Pa-"/2)t,~ 1/2 ',-t is the associated Legendre function and is defined by the Gauss hyperoeometric function as follows:

pa-(x/2,,.~_ 1 (1 +z)(2a-1)/aF(l13 1 z) -'~ '-'-r(~-~)\~-:~/ ~,~,~;~ ,z;--~- .

(3) The functions ~ ~(~), Wto)(~) and W~I)(~) have the followin# inte#ral representa- tions, which are special cases of (5.4):

V~(~) = 2~lfo " (ko'ff)~-?dO, I(ko"~)~ ~1; adO, V~~ = v/~r(1' - ~) /o'"

vT(g ) - x/~r(l - ~) I(k~ I-adO, 1 /o" where A~x denotes the (1, 1)-entry of the matrix A, It[+ = t or 0 accordin9 as t > 0 or t.<

P_a(1) = 1, Pa-(1/2)(O) = 2a-(1/2)X//~ (6.6) -1/2

Lemma 6.3. (1) If x~V+. then

7~ mx4,(Y) = *(z~)'va07), ~(x) where e(x) = #(Fx) and zx = #~" ~--1. (2) If xe V_, then

~xd'(Y)+ ./r F(1 _~)2 2 2a +(1/2)X/f~

Proof. (1) By Lemma 6.2 (1), J/~$(y) is a constant multiple of q~a(~). Since tFa(I2) = 1 (see (6.6)), We have ~x4,(~ = dcx 4,(12)'e~(y). Zeta functions with automorphic forms 131

By the definition of Jt'~b07), we obtain

~r = 4~(kO~-1)dkt + (k) = ~,(x)- 1q~(9~ -a ) e(x)O(z~). § ~Fxox (2) By (1) and Lemrna 6.2 (2), Jt'xq~07) can be written as

Note that g_~ = g~w, where w = (o- 1 ;).o=

,t[ _ ~ q~ ( -- y) = f , , dp (g, whw- 1 g-~ )d/~_ (h) ,~ H_/w- O~ F~o~w

= fH_/0;1 rxg~ dp(gyhg; 1)d/g_ (h) = .,r162q~ 07).

This implies that c~~ x)= c~)(~b; -x) and

~t ~b(~) + ~_ ~ 4,03 = (c~~ x) + c"~(4,; x))(~'~~ + ~'(~)).

Putting y = J2, we have by (6.6)

~'~4~(J~) + ~'-~4~(J:) _ 2~-"mx/~ 2 r(1 _~)2(c(~

Since ~-~ 4'(J2) = ~'~(- J2) = .~'~ 4,(w.S~) = ~r we have 2 2 F(1-~) ~r ~b(J) c~~ x) + c~(~b; x) = 2~_~1:2)x/~ x ~

This proves the lemma.

Remark. Since ~b is K-invariant, the representation of SL(2) generated by ~b does not belong to the discrete series. Hence 2 cannot be a positive integer and F(1 - 2/2) # oo. Now we can define zeta functions (• (q~,L;s) and local zeta functions O• (f;2,s) attached to tl) as follows:

y ~(zx) , +(dp, L; s) = x~L,+e(x)-~etx I"

~_ (O,L;s) = ~ x~r\z_ idetxl s

O+ (f~o;2,s) = f Idetxr-a/2w~ (~)f(x)dx. ,/ v• 132 Fumihiro Sato

Here we put w; (~) = %(~) (xe v+), ~P2(~) = w~~ + wT(~) (x~v_). The following proposition is an explicit version of the integral representation given by Proposition. 1.7.

PROPOSITION 6.4 For Re(s)> 3, we have

Zr (s)(f~ | fL) = 1_. ~ + (~, L; s) t} + (f~; 2, s) 7t

+ _ (~, L; s)~_ (f~; 2, s). 21+~1/2)x/~

We identify the dual vector space V~ with V a via the inner product (x,x*)= tr(xwx* w- 1), where w = (010)- 1 " By the cuspidal property of ~ and the fact that q~(,g- 1) = ck(wgw- 1) = q~(g), we have

v(L)O(dA L,f ~) = O(dp, L*,f ~), where L* is the lattice dual to L and

/- foo(x*) = J vRfoo(x)exp(2nx/:- 1

Using this theta transformation formula, we can derive the following proposition.

PROPOSITION 6.5 The zeta integral Z~(s)(f o~ | f L) has an analytic continuation to an entire function of s in C and satisfies the functional equation v(L)Z,(s)(f~ | = Z,(~- s)(f~ | where v(L)= ~ dx. dv niL

Remarks. By the cuspidal property of ~b, we can prove the functional equation of the zeta integral without the vanishing of f~ and on f~o on the singular sets (compare to Proposition 3.1). This implies that the zeta functions have analytic continuations to entire functions. Zeta functions with automorphic forms 133

PROPOSITION 6.6

2 x( cos(~s) r(1 - 2) cos(~2)]

/ ~ sin (n2) sin (ns) J k*_,f| s))" \F(1- 2) \ 2 ] As we can see from the proof of Theorem 5.3, the proposition above is reduced to the functional equation of

eV)(f| s)= f [x,al-~ldetxl'+(~-3)/2f(x)d x, d It+

C| 2, s) = Jfv_ Ix, ~I- ~ldet xl s +~a- 3)/2f(x)dx,

which are the local zeta functions of the prehomogeneous vector space obtained by restricting the representation p to the of GL(2). In fact, by the integral representations of W~ (2) given in Lemrna 6.2, we obtain the following relation between ~• (f~; 2, s) and ~(~)(f~,o; 2, s). 1 ~+ (Too;2, s) = ~" ~+oo)(f| ;2, s),

1 - (f~o; 2, s) = " ~)(f~o,o; ~', s), - where f~,o(X) = f~(ko.x)dO. The functional equation of ~| 2, s) is given in [Sh, Lemma [ (i)] and Proposition 6.6 is its immediate consequence. Summing up the results above, we obtain the following explicit form of the functional equation given in Theorem 5.4.

Theorem 6.7 The zeta functions ~_+(O,L;s) have analytic continuations to entire functions of s and satisfy the functional equation -2s.,/,-2.r( s

cos( s) 2aF(1 - 2) 2XF(1-2) cos(~) .... ~--2 sin(ns) 134 Fumihiro Sato

Remarks on further examples. (1) The results in w apply to Example 1 in w even if n >/3. In this case, the functional equation can be reduced to the functional equation of the local zeta functions attached to the prehomogeneous vector space (B(n),p, Sym(n)), whose explicit formula is found in [$5, Theorem 3.2] (cf. [$3, Ttieorem 8]). Here we denote by B(n) the group of upper triangular matrices of size n, the Borel subgroup of GL(n). We also note that the zeta functions studied in [$3, w can be viewed as the zeta functions attached to ~ = the Eisenstein series for a minimal parabolic subgroup. (2) The zeta functions (s (~b,L; s) admit an interpretation as the Mellin transform of the modular form of weight obtained from ~b by the Maass correspondence (cf. [KS], [Hej]). From this point of view, the zeta functions studied in [M3] are a generalization of (+ (~b,L; s) to the zeta functions attached to automorphic forms on orthogonal groups with trivial K-type. Though the symmetric structure corresponding to the Maass case is in general not of K~ type, the necessary harmonic analysis was developed by Rossman [Ro] (or Sekiguchi [Se]) and we can generalize Maass' results further to automorphic forms with non-trivial K-type, as Hejhal did for SL(2) in [Hej].

Acknowledgements

The main part of this paper was written during the author's stay in Strasbourg in the spring of 1990. The author would like to express his sincere gratitude to D6partment de Math6matiques de Universit6 Louis Pasteur, in particular to Professors H Rubenthaler and G Schiffmann, for their hospitality. Thanks are also due to Egami and Arakawa. Discussions with them in 1983 on the works of Maass ([M3]) and Hejhal ([Hej]) were the starting point of the present investigation.

References

[B1] van den Ban E P, Invariam differential operators on a semisimple symmetric space and finite multiplicities in a Plancherel formula, Ark. Mat. 25 (1987) 175-187 [B21 van den Ban E P, Asymptotic behaviour of matrix coefficients related to reductive symmetric spaces, Proc. K. Ned. Akad. Wet. Ag0 (1987) 225-249 [BG] Bernstein I N and Gelfand S I, Meromorphic properties of the functions P~, Funct. Anal. Appl. 3 (1969) 68-69 [BR] Bopp N and Rubenthaler H, Fonction z~ta associ6e ~i la s6rie principle sph6rique de certaines espaces sym6triques, C. R. Acad. Sci. Parist. 310 (1990) 505-508 [So] Borel A, Density and maximality of arithmetic groups, J. Reine Angew. Math. 224 (1966) 78-89 [BH3 Borel A and Harish-Chandra, Arithmetic subgroups of algebraic groups, Ann. Math. 75 (1962) 485-535 [BJ] Borel A and Jacquet H, Automorphic forms and automorphic representations, Proc. Syrup. pure Math. 33 (1979) part I, 189-202 [C] Chernousov V, On the Hasse principle for groups of type E8, Soy. Math. Dokl. 39 (1989) 592-596 [El Epstein P, Zur Theorie aUgemeiner Zetafunktionen, I, II, Math. Ann. 56 (1903) 615-644; 63 (1907) 205-216 [GJ] Godement R and Jacquet H, Zeta functions of simple algebras, Lect. notes in Math. No. 260, Springer-Verlag (1972) [H] Harish-Chandra, Automorphic forms on semisimple Lie groups, Lect. notes in Math. No. 68, Springer-Verlag (1968) [Hej] Hejhal D, Some Dirichlet series with coefficients related to periods of automorphic eigen forms, Proc. Jpn Acad. 58 (1982) 413-417 Zeta functions with automorphic forms 135

[Hecl] Hecke E, Eine neue art von Zetafunktionen und ihre Beziehungen zur Verteilung der Primzahlen, Erste Mitteilung, Math. Z. 1 (1918) 357-376 [Hec2] Hecke E, ibid, Zweite Mitteilung, Math. Z. 6 (1920) I 1-51 [Hell] Helgason S, Fundamental solutions of invariant differential operators on a symmetric space, Am. J. Math. 1t6 (1964) 565-601 [nel2] Helgason S, A duality for symmetric spaces with application to group representations II, Adv. Math. 22 0976) 187-219 [Hel3] Helgason S, Groups and geometric analysis, Academic Press, 1984 [HeI4] Helgason S, Some results on invariant differential operators on symmetric spaces, Am. J. Math. 114 (1992) 789-811 [KS] Katok S and Sarnak P, Heegner points, cycles and Maass forms. Preprint, 1990 [KK] Kawai T and Kashiwara M, On holonomic systems for Fl~= 1(fl + ~ - ~0)~, Publ. RIMS, Kyoto Univ. 15 (1979) 551-575 [K] Kottwitz R E, On Tamagawa numbers, Ann. Math. 127 (1988) 629-646 [MI] Maass H, Spherical functions and quadratic forms, J. Ind. Math. Soc. 20 (1956) 117-162 [M2] Maass H, Zetafunktionen mit Gr61~encharakteren und Kugelfunktionen, Math. Ann. 132 (1957) 1-32 [M3] Maass H, Ober die r/iumliche Verteilung der Punkte in Gittern mit indefiniter Metrik, Math. Ann. 138 (1959) 287-315 [M4] Maass H, Siegel's modular forms and Dirichlet series, Letc. notes in Math. No. 216, Springer Verlag (1971) [Mat] Matsuki T, The orbits of affine symmetric spaces under the action of minimal parabolic subgroups, J. Math. Soc. Jpn 12 (1982) 307-320 [Och] Ochiai H, A remark on invariant eigenfunctions on some exceptional noncompact Riemannian symmetric spaces. Preprint, 1990 [O] Oshima T, Poisson transformations on affine symmetric spaces, Proc. Jpn Acad. 55 (1979) 323-327 [os] Oshima T and Sekiguchi J, Eigenspaces of invariant differential operators on an affine symmetric space, Invent. Math. 57 (1980) 1-81 [Ro] Rossmann W, Analysis on real hyperbolic spaces, J. Funct. Anal. 30 (1978) 448-477 [Ru] Rubenthaler H, Espaces pr6homogen6nes de type parabolique. Th6se, Universit6 de Strasbourg, 1982 [Sab] Sabbah.C, Proximit6 6vanescente II, Compos. Math. 64 (1987) 213-241 [Sl] Sato F, Zeta functions in several variables associated with prehomogeneous vector spaces I, Functional equations, T6hoku Math. J. 34 (1982) 437-483 [$2] Sato F, ibid. II, A convergence criterion, T6hoku Math. J. 35 (1983) 77-99 [s3] Sato F, ibid. III, Eisenstein series for indefinite quadratic forms, Ann. Math. 116 (1982) 177-212 [$4] Sato F, The Hamburger theorem for Epstein zeta functions, Algebraic Analysis, Vol. II, Academic Press (1989) 789-807 [ss] Sato F, On functional equations of zeta distributions, Adv. Studies in Pure Math. 15 (1989) 465-508 [s6] Sato F, Zeta functions with polynomial coefficients associated with prehomogeneous vector spaces. Preprint, 1989 [$7] Sato F, The Maass zeta functions attached to positive definite quadratic forms, Adv. Studies in pure Math. 21 (1992) 409-443 [S] Sato M, Theory of prehomogeneous vector spaces (Notes by T. Shintani in Japanese), Sugaku no Ayumi 15-1 (1970) 85-157 [SK] Sato M and Kimura T, A classification of irreducible prehomogeneous vector spaces and their invariants, Nagoya Math. J. 65 (1977) 1-155 [ss] Sato M and Shintani T, On zeta functions associated with prehomogeneous vector spaces, Ann. Math. 100 (1974) 131-170 [Se] Sckiguchi J, Eigenspaces of the Laplace-Beltrami operator of a hyperboloid, Nagoya Math. J. 79 (1980) 151-185 [Sh] Shintani T, On zeta functions associated with the vector space of quadratic forms, J. Fac. Sci. Univ. Tokyo 22 (1975) 25-65 [Si] Siegel C, Advanced analytic number theory, Tata Studies in Math. 9, Tata Institute of Fundamental Research, Bombay, 1980 IT] Tetras A, Harmonic analysis on symmetric spaces and applications II, Springer Verlag, 1985 [V] Vust T, Op6ration de groupes reductifs darts un type de c6ne presque homog6ne, Bull. Soc. Math. France 102 (1974) 317-334