Invariant Hilbert Subspaces of the Oscillator Representation Aparicio, S

Invariant Hilbert subspaces of the oscillator representation Aparicio, S.

Citation Aparicio, S. (2005, October 31). Invariant Hilbert subspaces of the oscillator representation. Retrieved from https://hdl.handle.net/1887/3507

Version: Corrected Publisher’s Version Licence agreement concerning inclusion of doctoral thesis License: in the Institutional Repository of the University of Leiden Downloaded from: https://hdl.handle.net/1887/3507

Note: To cite this publication please use the final published version (if applicable). Invariant Hilbert subspaces of the oscillator representation

Proefschrift

ter verkrijging van de graad van Doctor aan de Universiteit Leiden, op gezag van de Rector Magnicus Dr. D. D. Breimer, hoogleraar in de faculteit der Wiskunde en Natuurwetenschappen en die der Geneeskunde, volgens besluit van het College voor Promoties te verdedigen op maandag 31 oktober 2005 te klokke 15.15 uur

door

Sofa Aparicio Secanellas

geboren te Zaragoza (Spanje) op 22 juni 1977 Samenstelling van de promotiecommissie:

promotor: Prof. dr. G. van Dijk

referent: Dr. E. P. van den Ban (Universiteit Utrecht)

overige leden: Prof. dr. E. G. F. Thomas (Rijksuniversiteit Groningen) Dr. M. F. E. de Jeu Prof. dr. M. N. Spijker Prof. dr. S. M. Verduyn Lunel A mi familia y a Guillermo Thomas Stieltjes Institute for Mathematics Contents

Publication history 9

1 Introduction 11 1.1 Some basic denitions ...... 11 1.2 Connection between representation theory and quantum mechanics 13 1.3 Overview of this thesis ...... 14 1.3.1 Representations of SL(2, IR) and SL(2, C) ...... 14 1.3.2 The metaplectic representation ...... 14 1.3.3 Theory of invariant Hilbert subspaces ...... 14 1.3.4 The oscillator representation of SL(2, IR) O(2n) ...... 15 1.3.5 The oscillator representation of SL(2, IR) O(p, q) . . . . . 15 1.3.6 The oscillator representation of SL(2, C) SO(n, C) . . . . 15 1.3.7 Additional results ...... 16

2 Representations of SL(2, IR) 17 2.1 The principal (non-unitary) series ...... 17 2.1.1 A ‘non-compact’ model ...... 17 2.1.2 A ‘compact’ model ...... 18 2.2 Irreducibility ...... 19 2.3 Intertwining operators ...... 20 2.4 Invariant Hermitian forms and unitarity ...... 22 2.5 The ‘non-compact’ models ...... 25 2.5.1 The continuous series: , , = 0, iIR and 0,+ . . . . 25 (1) 6 (2) ∈ 2.5.2 The representation: 0, and 0, ...... 25 2.5.3 The complementary series: ,+ (0 < < 1) ...... 25 2.5.4 The analytic discrete series ...... 26 2.5.5 The anti-analytic discrete series ...... 26 2.6 The analytic discrete series: realization on the complex upper half plane ...... 27 2.7 The limit of the analytic discrete series ...... 28 2.8 Explicit intertwining operator ...... 28

5 6 Contents

3 Representations of SL(2, C) 31 3.1 The principal (non-unitary) series ...... 31 3.1.1 A ‘non-compact’ model ...... 31 3.1.2 A ‘compact’ model ...... 32 3.2 Irreducibility ...... 33 3.3 Intertwining operators ...... 35 3.4 Invariant Hermitian forms and unitarity ...... 39 3.5 The ‘non-compact’ models ...... 40 3.5.1 The continuous series: , iIR ...... 40 ,l ∈ 3.5.2 The complementary series: ,0 (0 < < 2) ...... 40 3.6 Finite-dimensional irreducible representations ...... 41

4 The metaplectic representation 43 4.1 The Heisenberg group ...... 43 4.1.1 Denition ...... 43 4.1.2 The Schrodinger representation ...... 45 4.1.3 The Fock-Bargmann representation ...... 45 4.2 The metaplectic representation ...... 46 4.2.1 Symplectic linear algebra and symplectic group ...... 46 4.2.2 Construction of the metaplectic representation ...... 47

5 Theory of invariant Hilbert subspaces 51 5.1 Kernels and Hilbert subspaces ...... 51 5.2 Invariant Hilbert subspaces ...... 52 5.3 Multiplicity free decomposition ...... 53 5.4 Representations ...... 53 5.5 Schwartz’s kernel theorem for tempered distributions ...... 54

6 The oscillator representation of SL(2, IR) O(2n) 55 6.1 The denition of the oscillator representation ...... 55 2n 6.2 Some minimal invariant Hilbert subspaces of 0(IR ) ...... 56 S 6.3 Decomposition of L2(IR2n) ...... 59 2n 6.4 Classication of all minimal invariant Hilbert subspaces of 0(IR ) 59 S 7 Invariant Hilbert subspaces of the oscillator representation of SL(2, IR) O(p, q) 65 7.1 The denition of the oscillator representation ...... 65 n 7.2 Invariant Hilbert subspaces of 0(IR ) ...... 66 7.3 Multiplicity free decompositionSof the oscillator representation . . 70

8 Decomposition of the oscillator representation of SL(2, C) SO(n, C) 73 8.1 The case n = 1 ...... 73 8.1.1 The denition of the oscillator representation ...... 73 8.1.2 Some minimal invariant Hilbert subspaces of 0(C) . . . . . 74 8.1.3 Decomposition of L2(C) ...... S...... 78 8.2 The case n = 2 ...... 78 8.2.1 The denition of the oscillator representation ...... 78 2 8.2.2 Some minimal invariant Hilbert subspaces of 0(C ) . . . . 78 S Contents 7

8.2.3 Decomposition of L2(C2) ...... 83 8.3 The case n 3 ...... 83 8.3.1 Plancherel formula for SO(n, C)/SO(n 1, C), n 3 . . . . 83 8.3.2 Decomposition of L2(Cn) ...... ...... ...... 95

9 Invariant Hilbert subspaces of the oscillator representation of SL(2, C) SO(n, C) 105 9.1 The cases n = 1 and n = 2 ...... 105 9.1.1 The denition of the oscillator representation ...... 105 2 9.1.2 Invariant Hilbert subspaces of 0(C ) ...... 106 9.1.3 Multiplicity free decompositionS ...... 109 9.2 The case n 3 ...... 109 9.2.1 Complex generalized Gelfand pairs ...... 109 9.2.2 MN-invariant distributions on the cone ...... 113 9.2.3 Invariant Hilbert subspaces of the oscillator representation . 121

A Conical distributions of SO(n, C), n 3 127 A.1 The cone = G/MN ...... 127 A.2 Denition of conical distribution ...... 128 A.3 The function ...... 129 M A.4 Conical distributions with support on 0 ...... 131 A.5 Conical distributions ...... 135

B Irreducibility and unitarity 143 B.1 The standard minimal parabolic subgroup ...... 143 B.2 Irreducibility ...... 146 B.3 Unitarity ...... 150

Bibliography 153

Samenvatting 157

Curriculum Vitae 159 8 Contents Publication history

Chapter 7 has been published in“Indagationes Mathematicae”, New Series 14 (3,4), December 2003, in a series of papers dedicated to Tom Koornwinder. Section 8.3.1 was released as report MI 2004-03 (February 2004) of the Ma- thematical Institute, University of Leiden. It has been accepted for publication in “Acta Applicandae Mathematicae”. Section 9.2.1 was released as report MI 2004-05 (March 2004) of the Mathe- matical Institute, University of Leiden, and submitted for publication.

9 10 Publication history CHAPTER 1

Introduction

The metaplectic representation -also called the oscillator representation, harmonic representation, or Segal-Shale-Weil representation- is a double-valued unitary representation of the symplectic group Sp(n, IR) (or, if one prefers, a unitary representation of the double cover of Sp(n, IR)) on L2(IRn). It appears implicitly in a number of contexts going back at least as far as Fresnel’s work on optics around 1820. However, it was rst rigorously constructed on the Lie algebra level -as a representation of sp(n, IR) by essentially skew-adjoint operators on a common invariant domain- by van Hove in 1950, and on the group level by Segal and Shale a decade later. These authors were motivated by quantum mechanics. At about the same time, Weil developed analogues of the metaplectic representation over arbitrary local elds, with a view to applications in number theory. Since then the metaplectic representation has attracted the attention of many people. In this thesis we study the decomposition of the oscillator representation for some subgroups of the symplectic group. By oscillator representation of these subgroups we mean the restriction of the metaplectic representation of the symplectic group to these subgroups. The main results of this thesis are the Plancherel formula of these representations and the multiplicity free decomposition of every invariant Hilbert subspace of the space of tempered distributions. In Section 1.1 we introduce the common concepts required for a proper un- derstanding of this work. In Section 1.2 we explain the connection between representation theory and quantum mechanics. In Section 1.3 we provide a brief description of each chapter.

1.1 Some basic denitions

Let F be either the eld IR of real numbers, or the eld C of complex numbers. A Lie group G over F is a group G endowed with an analytic structure such that 1 the group operations g g and (g , g ) g g are analytic operations. → 1 2 → 1 2 In particular, a Lie group G over F is a locally compact group, and admits a

11 12 Chapter 1. Introduction positive left-invariant measure dg, which is unique up to a positive scalar, i.e.

f(xg)dg = f(g)dg, ZG ZG for all x G and all continuous complex valued functions f on G which vanish outside a∈compact subset. Such a measure is called a Haar measure of G. From here on we shall assume that G is unimodular, i.e. the measure dg is both left- and right-invariant. Let H be a closed unimodular subgroup of G, and let X denote the associated quotient space G/H. The group G acts on X in a natural way. It is well known that, as G and H are unimodular, there exists a positive G-invariant measure dx on X, which is unique up to a positive scalar. Let G denote a Lie group over F , and V a complex topological vector space. By a representation of G on V , which will often be denoted by (, V ) we shall mean a homomorphism from G into the group GL(V ) of invertible continuous endomorphisms of V such that for each v V the map ∈ g (g)v, g G, 7→ ∈ is continuous on G. A subspace W of V is called invariant under if (g)W W for each g G. The representation (, V ) is called irreducible if the onlyclosed invariant∈subspaces of V are the trivial ones: (0) and V itself. Two representations (, V ) and (0, V 0) are called equivalent if there is a continuous linear isomorphism A : V V 0 such that → A(g) = 0(g)A, g G. ∈ Let denote a Hilbert space. A representation (, ) of G is called unitary if each Hendomorphism (g), g G, of is unitary, i.e.H (g) is surjective and preserves the Hilbert norm. ∈ H By Schur’s lemma the representation of G on is irreducible if and only if the only bounded linear operators A on , for whichHA(g) = (g)A for all g G, are of the form A = I for some C.H ∈ One of the main goals of harmonic∈ analysic on the space X is to nd a decomposition of the Hilbert space L2(X, dx) into minimal subspaces which are invariant under the left regular action of G, this action being dened by 1 2 (g)f(x) = f(g x), x X, f L (X, dx), g G. ∈ ∈ ∈ This decomposition is known as the Plancherel formula of X. The continuous part that occurs in the Plancherel formula of X is called the principal or continuous series of X and the discrete part is called the discrete series of X. Additional irreducible unitary representations which play no role in the Plancherel formula of X are called complementary or supplementary series of X. As an example, let G be the circle-group T = z C : z = 1 . Let dt denoted the Haar measure on T normalized so that v{ol(∈T ) = 1.| It| is kno} wn that the Hilbert space L2(T, dt) has a decomposition

2 L (T, dt) = Cn, n M∈ 1.2. Connection between representation theory and quantum mechanics 13

n where for each integer n the function n on T is dened by n(t) = t , t T . ∈ ∈ Observe that each function n (n ) denes a unitary character of the group T , i.e. a homomorphism from the group∈ T into T itsef, the range being considered 2 as a subgroup of the multiplicative group C. Each function f L (T, dt) has an expansion ∈

f = cnn, (1.1) n X∈ where 2 1 i in c = f(t) (t)dt = f(e )e d, n , n n 2 ∈ ZT Z0 the sum on the right-hand side of (1.1) being convergent in the Hilbert norm of 2 the space L (T, dt). One recognizes the numbers cn, n , as the classical Fourier coecients of the function f, and (1.1) as the inversion∈ formula for the classical Fourier transform on the space of periodic functions on the interval [0, 2]. The correspondig Plancherel formula, which is in this case also called the Parseval equality, is written as follows:

2 2 f(t) dt = cn .

| | | | T n Z X∈ An important source of motivation for this aspect of harmonic analysis, which can be viewed as a generalization of the classical Fourier theory, lies in this example.

1.2 Connection between representation theory and quantum mechanics

The elds of representation theory and quantum mechanics are the result of crea- tive interaction between mathematics and physics. Ever since the foundation of both disciplines, some 75 years ago, they have continuously inuenced each other, and nowadays they are well established theories of key importance for the development of other elds of research. In mathematics, unitary representation theory generalises Fourier analysis, turns out to be relevant to practically all elds of mathematics ranging from probability theory to number theory, and in addition has created a eld of its own. In quantum physics, unitary representations come in whenever symmetry plays a role, from solid state physics to elementary particles and quantum eld theory. As an example of its multiple applications, many of the known subatomic particles were predicted by the standard model of quantum mechanics more than 30 years before their detection. The contribution of quantum mechanics to representation theory is two-fold: Firstly, it enriches mathematics with new constructions of representations, and secondly, it provides examples and applications of the theory developped by ma- thematicians. A brief look at some key dates in the history of physics and mathematics shows that quantum mechanics and unitary representation theory were born together. In fact, this is no coincidence, as the concept of a unitary representation on a possibly 14 Chapter 1. Introduction innite-dimensional Hilbert space was directly inspired by quantum mechanics. Hermann Weyl asked himself which type of innite-dimensional topological vector space would give him the best possible generalization of the decomposition theory of the regular representation of a nite group to the compact case, and found the answer in John von Neumann’s brand new concept of abstract Hilbert space. This concept, in turn, was directly inspired by the recent development of quantum mechanics, with which Weyl was thoroughly familiar.

1.3 Overview of this thesis 1.3.1 Representations of SL(2, IR) and SL(2, C) In Chapter 2 and Chapter 3, the irreducible unitary representations of SL(2, IR) and SL(2, C), respectively, are given in the ‘non-compact’ model. By ‘non-compact’ model we mean, in the case of SL(2, IR), the realization of the representations on a space of functions on IR, and in the case of SL(2, C), on C. In order to give a good analysis we require a ‘compact’ model. By ‘compact’ model we mean in Chapter 2 the realization of the representations on a space of functions on the unit circle, and in Chapter 3 on the unit sphere. The representations of SL(2, IR) and SL(2, C) are certainly well known but our treatment is a little dierent from the usual ones. It is also well-suited for the discussion of the oscillator representations in the next chapters.

1.3.2 The metaplectic representation In Chapter 4 we give an overview of the construction of this representation and we examine it from several viewpoints. The contents of this chapter are adapted from [10]. First we introduce the denition of the Heisenberg group Hn. After 2 n that we dene an irreducible unitary representations of Hn on L (IR ), we call this the Schrodinger representation, and then we give another realization on the Fock space, the Fock-Bargmann representation. Since the symplectic group, which is invariant under the symplectic form, acts on the Heisenberg group, we get another representation of the Heisenberg group. The theorem of Stone-von Neumann cla- ssies all irreducible unitary representations of Hn. Applying this theorem we get our double-valued unitary representation of the symplectic group, and we call it the metaplectic representation. We consider two models for the metaplectic representation. In the Schrodinger model we can not give an explicit formula. For this purpose we use the Fock model, where we can express our representation by means of integral operators.

1.3.3 Theory of invariant Hilbert subspaces In Chapter 5 we recall an important tool for the representation theory of Lie groups, namely the theory of Hilbert subspaces invariant under a group of automorphisms. This theory is due to L. Schwartz [27], Thomas [34] and Pestman [25]. A main subject is the study of the theory of the reproducing kernels, associated with Hilbert subspaces. We give a criterion (due to Thomas) which assures multiplicity 1.3. Overview of this thesis 15 free decomposition of representations. This criterion will be applied in the next chapters.

1.3.4 The oscillator representation of SL(2, IR) O(2n) In the rst part of Chapter 6 we compute the Plancherel formula of the oscillator representation ω2n of SL(2, IR) O(2n) for n > 1. The main tool we use is a Fourier integral operator, whichwas introduced by M. Kashiwara and M. Vergne, see [18]. After that we can conclude that any minimal invariant Hilbert subspace 2n 2 2n of 0(IR ) occurs in the decomposition of L (IR ). S Finally, we study the oscillator representation ω2n in the context of the theory of invariant Hilbert subspaces. The oscillator representation acts on the Hilbert 2 2n 2n space L (IR ). It is well-known that (IR ) is ω2n(G)-stable, so, by duality, 2n 2 2Sn ω2n acts on 0(IR ) as well, and L (IR ) can thus be considered as an invariant S 2n Hilbert subspace of 0(IR ). Our main result is that any ω2n(G)-stable Hilbert 2n S subspace of 0(IR ) decomposes multiplicity free. The caseSn = 1 is treated in a similar way. Here a non-discrete series representation occurs in the decomposition of L2(IR2).

1.3.5 The oscillator representation of SL(2, IR) O(p, q)

The explicit decomposition of the oscillator representation ωp,q for the dual pair G = SL(2, IR) O(p, q) was given by B. rsted and G. Zhang in [23]. In Chapter 7 we only study the multiplicity free decomposition of any ωp,q(G)-stable Hilbert p+q subspace of 0(IR ). The oscillator representation acts on the Hilbert space L2(IRp+q). ItS is well-known that (IRp+q), the space of Schwartz functions on p+q S p+q IR , is ω (G)-stable. Thus, by duality it follows that ω acts on 0(IR ), p,q p,q S the space of tempered distributions on IRp+q, as well, and L2(IRp+q) can thus be p+q considered as an invariant Hilbert subspace of 0(IR ). According to Howe [16], L2(IRp+q) decompSoses multiplicity free into minimal p+q invariant Hilbert subspaces of 0(IR ). This is a special case of our result, S p+q which states that any ωp,q(G)-stable Hilbert subspace of 0(IR ) decomposes multiplicity free. S We restrict to the case p + q even for simplicity of the presentation of the main results. In addition we have to assume p 1, q 2, since we apply results from [9] where this condition is imposed. Our result ishowever true in general. The contents of this chapter have appeared in [39].

1.3.6 The oscillator representation of SL(2,C) SO(n,C) In Chapter 8 we determine the explicit decomposition of the oscillator representation for the groups SL(2, C) O(1, C) and SL(2, C) SO(n, C) with n 2. For this, we again make use of a Fourier integral operatorintroduced by M. Kashiw ara and M. Vergne for real matrix groups, see [18]. For the group SL(2, C) O(1, C) we construct two intertwining operators. One from the space of even Schwartz functions to the space of functions where the complementary series of SL(2, C) are dened, and the other one, from the space 16 Chapter 1. Introduction of odd Schwartz functions to L2(C). So in this case a complementary series of SL(2, C) occurs in the decomposition of L2(C). For the group SL(2, C) SO(n, C) with n 2, given that SO(2, C) is an abelian group, we split the proof into the cases n = 2and n 3. The computation of the Plancherel measure for SO(n, C)/SO(n 1, C) is required in order to compute the explicit decomposition for the case n 3. For this purpose we follow the approach of Van den Ban [36]. However, the computation of the Plancherel measure is not so obvious, as it involves a good deal of non-trivial steps. In Chapter 9 we study the oscillator representation ωn for the groups SL(2, C) O(1, C), SL(2, C) O(2, C) and SL(2, C) SO(n, C) with n 3 in the context of the theory invariant Hilbert subspaces. Our main result is that any ωn(G)-stable n Hilbert subspace of 0(C ) decomposes multiplicity free. S In case n = 1 is easy to prove that every invariant Hilbert subspace of 0(C) decomposes multiplicity free. We leave to the reader to check this result Susing techniques similar to those applied for other cases. The case n = 2 is slightly dierent from the other cases. The cone consists in two disjoint pieces and we have to study the O(2, C)-invariant distributions on for each of the four components. For the case n 3 we rst need to prove that (SO(n, C), SO(n 1, C)) are generalized Gelfandpairs, and to compute the MN-invariant distributions on the cone. To prove that (SO(n, C), SO(n 1, C)) are generalized Gelfand pairs, for n 2 we rst introduce a brief resume ofthe theory of generalized Gelfand pairs, whic h is connected with the theory of invariant Hilbert subspaces presented in Chapter 5. We also introduce a criterion that was given by Thomas to determine generalized Gelfand pairs. We will use this criterion for our own aim. To compute the MN-invariant distributions on the cone for n 3 we follow the same method as in [9]. Since every distribution T on the cone invariant under MN can be written as T = 0S + T , where S is a continuous linear form on , M 1 J is the average map and T1 is a singular MN-invariant distribution, see Section M9.2.2 for denitions, we need to compute the singular MN-invariant distributions on the cone. To do this we have to split in two cases n = 3 and n > 3, since for n = 3 the group M is equal to the identity.

1.3.7 Additional results At the end of this thesis we include two appendixes, A and B. In Appendix A we compute the conical distributions associated with the orthogonal complex group SO(n, C) with n 3. The group SO(n, C) acts transitively on the isotropic cone of the quadratic form associated with it. The action of SO(n, C) in the space of homogeneous functions on this cone denes a family of representations of SO(n, C). Their study leads to the conical distributions. We follow the same method as in [9]. In Appendix B we study the irreducibility and unitarity of the representations of SO(n, C) with n 3 induced by a maximal parabolic subgroup and dened in Section 8.3.1 in more detail. We follow the same method as in [41]. CHAPTER 2 Representations of SL(2, IR)

In this chapter we give the irreducible unitary representations of SL(2, IR) in the ‘non-compact’ model. By ‘non-compact’ model we mean the realization of the representations on a space of functions on IR. In order to give a good analysis we require a ‘compact’ model. By ‘compact’ model we mean the realization of the representations on a space of functions on the unit circle S. The representations of SL(2, IR) are well known, of course, but our treatment is a little dierent from the usual ones. It is also well-suited for the discussion of the oscillator representations in the next chapters.

2.1 The principal (non-unitary) series 2.1.1 A ‘non-compact’ model Set t 0 P = MAN = 1 : t IR, x IR x t ∈ ∈ ½µ ¶ ¾ where M = I , { } t 0 1 0 A = 1 : t > 0 and N = : x IR . 0 t x 1 ∈ ½µ ¶ ¾ ½µ ¶ ¾ 1 y cos sin Put N = : y IR and K = : 0 < 2 . Then 0 1 ∈ sin cos ½µ ¶ ¾ ½µ ¶ ¾ NP is open, dense in G = SL(2, IR) and its complement has Haar measure zero. a b 1 y t 0 Any g = in G can be written in the form np = 1 with c d 0 1 x t µ ¶ µ ¶ µ ¶ t = 1/d, y = b/d and x = c, provided d = 0. 6 The principal series , ( C, Mˆ ) acts on the space V of ∞-functions f with ∈ ∈ C +1 f(gmatn) = t (m)f(g)

17 18 Chapter 2. Representations of SL(2, IR) with inner product f(k) 2dk = f 2. | | || ||2 ZK , is given by 1 ,(g0)f(g) = f(g0 g).

For this inner product, ,(g0) is a bounded transformation and the representation , is continuous. Identifying V with a space of functions on N IR, we can write ,(g0) in these terms. The inner product becomes '

∞ f 2 = f(y) 2(1 + y2)Re dy. || ||2 | | Z∞

1 a b If g = then, 0 c d µ ¶

1 1 y a ay + b 1 y0 t 0 g0 = = 1 0 1 c cy + d 0 1 x t µ ¶ µ ¶ µ ¶ µ ¶ ay+b 1 with y0 = cy+d and t = (cy + d) , so that

1 2 (1 ) (+1) cy + d ay + b (g )f(y) = cy + d f . , 0 | | cy + d cy + d µ| |¶ µ ¶ The representations , are unitary for imaginary. The converse is also true, see [32].

2.1.2 A ‘compact’ model This model enables us more easily to answer questions about irreducibility, equivalence, unitarity, etc. G = SL(2, IR) acts on the unit circle S = (s , s ) IR2 : s2 + s2 = 1 by 1 2 ∈ 1 2 g(s©) ª g s = . g(s) || || The stabilizer of e2 = (0, 1) is AN. So , can be realized (depending on ) on V , the space of ∞-functions ϕ with C 1 ϕ(m s) = (m)ϕ(s) with m M, s S and ∈ ∈ 1 1 (+1) (g)ϕ(s) = ϕ(g s) g (s) . , || || 1 1 1 1 [ If g k = k0a n, then g s = k0 e and g (s) = t ]. We shall also t 2 || || write ,+ if 1 and , if ( I) = 1. 2.2. Irreducibility 19

Observe that , K ind . | ' M K ↑

l V+ is spanned by the functions (s1 + is2) with l even, V by the same functions with l odd. Let (ϕ, ) be the usual inner product on L2(S)

(ϕ, ) = ϕ(s)(s)ds (2.1) ZS where ds is the normalized measure on S. The measure ds is transformed by g G 2 ∈ as follows: ds˜ = g(s) ds if s˜ = g s. It implies that the inner product (ϕ, ) is || || invariant with respect to (,, ,), so that , is unitary for iIR. ∈ Now we want to study for , the following questions: irreducibility, compo- sition series, intertwining operators, unitarity.

2.2 Irreducibility

1 0 Since G is generated by K and the subgroup A = exp(tZ ) with Z = , { 0 } 0 0 1 µ ¶ in order to study the irreducibility of , we have to know how the latter subgroup transforms the functions (s) = (s + is )l, l . An easy computation shows: l 1 2 ∈ 1 1 ,(Z0)l = ( + 1 + l)l+2 + ( + 1 l)l 2. (2.2) 2 2 This immediately leads to a complete analysis of the reducibility properties of the representations , . The results are as follows.

Theorem 2.1.

a) If , ,+ and , are irreducible. 6∈ b) If is an even integer, ,+ is irreducible while , is not irreducible. For , the decomposition is as follows: (1) (2) (i) = 0. In this case, V , spanned by 1, 3, . . . , and V , spanned by 1, 3, . . . , are invariant. No other closed invariant subspaces of V exist. (1) (ii) = 2k, k 1. In this case, V , spanned by 2k 1, 2k 3, . . . , (2) (1) (2) and V , spanned by 2k+1, 2k+3, . . . , are invariant. V , V and V (1) V (2) are the only proper closed invariant subspaces. V /V (1) V (2) is nite-dimensional and denes the irreducible representation with highest weight 2k-1.

(1) (iii) = 2k, k 1. In this case, V , spanned by 2k 1, 2k 3, . . . , and (2) V , spanned by (2k 1), (2k 3), . . . , are invariant; and these, along 20 Chapter 2. Representations of SL(2, IR)

with V (1) V (2) are all the proper closed invariant subspaces. V (1) ∩ ∩ V (2) is nite-dimensional and denes the irreducible representation with highest weight 2k-1.

c) If is an odd integer, , is irreducible while ,+ is reducible. For ,+ the splitting is as follows:

(1) (2) (i) = 2k + 1, k 0. V+ , spanned by 2k 2, 2k 4, . . . , and V+ , spanned by 2k+2, 2k+4, . . . , are invariant. These and their direct (1) (2) sum are the only proper closed invariant subspaces. V+/V+ V+ is nite-dimensional and denes the irreducible representation with highest weight 2k.

(1) (2) (ii) = 2k 1, k 0. V+ , spanned by 2k, 2k 2, . . . , and V+ , spanned by 2k, 2k+2, . . . , are invariant; these, together with their intersec- (1) (2) tion, exhaust all proper closed invariant subspaces. V+ V+ is nite- dimensional and denes the irreducible representation of∩highest weight 2k.

(1) (2) For the reducible case, the restriction of , to V and V are denoted by (1) (2) (1) (2) , and , respectively. Furthermore, V and V depend on . We shall (1) (2) occasionally write therefore V, and V, .

2.3 Intertwining operators

Now we want to nd (non zero) continuous linear operators A : V V in- → 1 tertwining the representations , and 1,1 (and their subrepresentations and subquotiens), i.e. A (g) = (g)A (g G). , 1,1 ∈ Theorem 2.2. A non-zero non-trivial intertwining operator as above exists if and only if = 1, 1 = . Such as operator is unique up to a factor. For the (1) (2) reducible case, this operator vanishes on V, V, ( = 2k or = 2k + 1) and (1) (2) (1) (2) gives rise to an isomorphism of V /V, V, onto V , V , . Restricting ∩ A to V (1) (or V (2) ) gives an isomorphism onto V /V (2) (or V /V (1) ). 2k, 2k, 2k, 2k, Similarly for = 2k + 1:

(1) (2) (2) (1) V2k+1,+ V+/V 2k 1,+ and V2k+1,+ V+/V 2k 1,+. ' ' 2.3. Intertwining operators 21

Also the ‘dual’ isomorphisms are true:

(1) (2) V 2k, V /V2k, , ' (2) (1) V 2k, V /V2k, , ' (1) (2) V 2k 1,+ V+/V2k+1,+, ' (2) (1) V 2k 1,+ V+/V2k+1,+. ' Proof. Restricting to K we obtain = and since is multiplicity free, 1 ,|K l is an eigenvector of A with eigenvalue, say al. These numbers depend on , 1 and . They should satisfy the system of equations:

( + 1 + l)al+2 = (1 + 1 + l)al

( + 1 l)al 2 = (1 + 1 l)al Here we applied (2.2). Combining these equations gives

( + 1 + l)( 1 l) = ( + 1 + l)( 1 l), 1 1 so 1 = . If = 1, then all al coincide, so A is an scalar operator. In the second case ( = ) we get 1 ( + 1 + l)a = ( + 1 + l)a . (2.3) l+2 l For the irreducible case, equation (2.3) has a, up to a factor, unique solution, which can be written in one of the following three forms: ( 1)l/2 al = c1 +1+l +1 l (2.4) ( 2 )( 2 ) + 1 + l + 1 l = c ( 1)l/2( )( ) (2.5) 2 2 2 +1 l ( 2 ) = c3 +1 l . (2.6) ( 2 ) In the reducible cases, there is also a unique solution, but one has to start at another base value, e.g. l = 2k 1 for V (1) . This is proven in the same way. 2k, The formulae obtained for the solutionsof (2.3) show, when the solution is dened on an unbounded set, that it is of polynomial growth at innity, see [6]:

Re a const. l ( l ). l | | | | → ∞ This implies that the operator A having al as eigenvalues is continuous in the ∞-topology of V .¤ C Let us produce the intertwining operator in integral form. Dene the operator A, on V by the formula

1, A, ϕ(s) = [s, t] ϕ(t)dt ZS 22 Chapter 2. Representations of SL(2, IR)

, u where = 0 if = 1, = 1 if = 1 and u = u u if u IR, | | | | ∈ u = 0. Furthermore, [s, t] = s1t2 s2t1 if s = (s1, s2), t = (t1,³t2).´Observe that [g(6 s), g(t)] = [s, t] for all g G. This integral converges for Re > 0 and can be extended analytically on the∈ whole complex -plane to a meromorphic function. Clearly A, carries V into itself. It is easily checked that A, is an intertwining operator:

A, ,(g) = ,(g)A, (g G). ∈

For the eigenvalues al(, ) we have an explicit expression:

il +1 ()e 2 al(, ) = 2 +1+l +1 l . ( 2 )( 2 ) This formula is proven in the following way. We start from:

A, l(s) = al(, )l(s).

Taking s = e1, we obtain:

2 1 1, il a (, ) = (sin ) e d. l 2 Z0 This last integral is computed with the help of [12].

As a function of the operator A, has poles of the rst order at 2IN ( = 0) and 1 2IN ( = 1), so 2IN. Let be reducible,∈= 0; ∈ ∈ , 6 If > 0, so = 0 = 2k ( = 1) or 0 = 2k + 1 ( = 0), then the operator

A0, has not a pole at these points. Moreover A0, vanishes on the irreducible subspaces. On each of the irreducible subspaces V it has a zero of the rst order. Its derivative ∂A, intertwines the restriction of to V and the factor ∂ |=0 0, representation , on V = V/V ⊥. 0 With the Hermitian form (2.1) the operator A, interacts as follows:

(A, ϕ, ) = (ϕ, A, ).

2.4 Invariant Hermitian forms and unitarity

In this section we determine all invariant Hermitian forms on V and its subfactors with respect to the representations , and determine which of these forms are positive (negative) denite, so that the corresponding representations are unitarizable. A continuous Hermitian form H(ϕ, ) on V is called invariant with respect to , if 1 H(,(g)ϕ, ) = H(ϕ, ,(g )) 2.4. Invariant Hermitian forms and unitarity 23 for all g G. Any such a form can be written in the form ∈ H(ϕ, ) = (Aϕ, ) with A an operator on V and the inner product (2.1) on the right-hand side. It implies that A intertwines , and , . So by Theorem 2.2, there are two po- ssibilities: = and = . In the rst case, we have Re = 0 and, provided = 0, the operator A is equal to cE, so that H is c times (2.1). In the second 6 case, A intertwines , and ,. The case when H is dened on a subfactor is treated in a similar way. So weget:

Theorem 2.3. A non-zero invariant Hermitian form H(ϕ, ) on V exists only if: (a) Re = 0, or (b) Im = 0. In case (a) the form is proportional to the 2 L -inner product (2.1). In case (b) the form H(ϕ, ) on V has the form H(ϕ, ) = (Aϕ, ) where A is an intertwining operator between , and ,. On an irreducible subfactor V/W the form H looks the same with A an operator V W ⊥/V ⊥ → vanishing on W and intertwining the subfactors of , on V/W and of , on W ⊥/V ⊥. In particular, the Hermitian form

(A, ϕ, ) (2.7) with IR, where A is the operator, dened in Section 2.3, dened on V and ∈ , invariant with respect to , . At singular points one has to take residues of (2.7). If , is reducible, then (2.7) vanishes on each irreducible invariant subspace V . Its derivative with respect to on the subspace V is an invariant Hermitian form on V . In this way we obtain Hermitian forms on all irreducible subfactors. Now we determine when the Hermitian forms above are positive or negative denite.

Theorem 2.4. The unitarizable irreducible representations , or their irreducible subfactors belong to the following series. (i) , with = 0, Re = 0 and 0,+: the continuous series. (1) (2)6 (ii) 0, and 0, . (iii) The complementary series consisting of the representations ,+ with 0 < < 1. (1) (2) (iv) The trivial representation, acting on V 1,+ V 1,+ and also on the factor (1) (2) ∩ space V1,+/V1,+ V1,+. (1) (v) The analytic discrete series, consisting of the subrepresentations , on the (1) (1) (1) subspaces V, ( = 2k, k = 1, 2, . . . ) and ,+ on the subspaces V,+ ( = 2k + 1, k = 0, 1, 2, . . .). These representations are equivalent to the factor representations (2) (2) , on V /V ( = 2k, k = 1, 2, . . . ) and ,+ on V+/V ( = (2k + 1), , ,+ k = 0, 1, 2, . . . ). 24 Chapter 2. Representations of SL(2, IR)

(2) (vi) The anti-analytic discrete series, consisting of the subrepresentations , (2) (2) (2) on the subspaces V, ( = 2k, k = 1, 2, . . . ) and ,+ on the subspaces V,+ ( = 2k + 1, k = 0, 1, 2, . . . ). These representations are equivalent to the factor (1) (1) representations , on V /V ( = 2k, k = 1, 2, . . . ) and ,+ on V+/V , ,+ ( = (2k + 1), k = 0, 1, 2, . . . ).

Proof. In the basis l an invariant Hermitian form H(ϕ, ) has a diagonal matrix, with real scalars hl on the diagonal. The hl have the same expression as al. We have to determine when the numbers al have the same sign for all l L, where L is the set of weights of our representation. This leads to the representations∈ in the theorem. For example, when 0 < < 1 we easily see that (2.4) is positive for all l, if c1 > 0, l even. For odd l this is not the case, since a l = al. ¤ Let us indicate the invariant inner products for these series of representations. (1) (2) For the continuous series, 0, and 0, , the inner product is just (2.1). For the trivial representation the inner productis clear. For the complementary series it is:

(A, ϕ, ).

For the representations (2) ( = 2k, k = 1, 2, . . . ) and (2) ( = 2k + 1, 0, 0 0,+ 0 k = 0, 1, 2, . . . ) the inner product is:

∂ (A ϕ, ) (2.8) ∂ , ¯=0 ¯ ¯ on V , respectively V+. If we take¯the square norm of l in the sense of (2.8) for the lowest weight l0 = 0 + 1 is equal to 1, then the square norms of the other l, l = + 1 + 2m, m IN are equal to 0 ∈ m! [m] (2.9) (0 + 1) where we used the notation a[m] = a(a + 1) (a + m 1). For the representations (1) ( = 2k, k = 1, 2, . . . ) and (1) ( = 2k + 1, 0, 0 0,+ 0 k = 0, 1, 2, . . . ) the inner product is the same as (2.8). The normalization by 1 at the highest weight l0 = 0 1 gives the same formula (2.9) for all other square norms, for the weights l = l 2m. 0 Let us denote the unitary completions of the representations by the same sym- bols. These unitary completions exhaust all irreducible unitary representations of G up to equivalence, by Harish-Chandra’s subquotient theorem [13]. 2.5. The ‘non-compact’ models 25

2.5 The ‘non-compact’ models of the irreducible unitary representations of SL(2, IR)

B, is dened by x + i 1 B ϕ(x) = ϕ x i . , x + i | | µ| |¶ Observe that the operator is an intertwining operator between the ‘compact’ model and the ‘non-compact’ model. (Here S is identied with the complex numbers of absolute value one.) So it is possible to describe the spaces V, in the ‘non- compact’ model by the functions B,l.

2.5.1 The continuous series: λ,§, =6 0, ∈ iIR and 0,+ We just refer to Section 2.1.1. The space is L2(IR) with the usual inner product and 1 2 (1 ) 1 cy + d ay + b (g)f(y) = cy + d f (2.10) , | | cy + d cy + d µ| |¶ µ ¶ a b if g 1 = . c d The spaceµ can¶ also be described as follows. Calling

k k x + i 1, = (x i) , , x i µ ¶ k k V,+ and V, are spanned by ,0 = B,+2k and ,1 = B, 2k 1 with k respectively. ∈

(1) (2) 2.5.2 The representation: 0,− and 0,−

(1) (2) 2 These representations act on the closed subspaces V0, and V0, of L (IR) by (1) (2) formula (2.10). As before we can see that the spaces V0, and V0, are spanned k k by the functions 0,1 with k IN and 0,1 with k IN, k = 0 respectively. The representations on these∈ spaces are given b∈y 6

(i) 1 ay + b 0, (g)f(y) = (cy + d) f cy + d µ ¶ a b for i=1,2; g 1 = . c d µ ¶

2.5.3 The complementary series: λ,+ (0 < < 1) The inner product is given in the ‘compact model’ by

1 (ϕ, ) = [s, t] ϕ(t)(s)dsdt | | ZS ZS 26 Chapter 2. Representations of SL(2, IR)

(ϕ, V+). The spaces are dened as in Section 2.5.1 and the representation is again∈given by (2.10):

1 ay + b (g)f(y) = cy + d f ,+ | | cy + d µ ¶ a b if g 1 = . c d µ ¶ Let us rewrite (ϕ, ) in terms of functions on IR. This is easily seen to be:

∞ ∞ 1 (f, g) = x y f(x)g(y)dxdy. | | Z∞ Z∞ 2.5.4 The analytic discrete series Let us dene the following functions by

n x i m (x) = (x + i) n x + i µ ¶ where n IN and m IN, m 2. The spaces can be described by these functions. (1) ∈ (1) ∈ V2k, and V2k+1,+ are spanned by j = Bm 1, m 2j with m 1 = 2k, j IN ∈ and j = Bm 1,+ m 2j with m 1 = 2k + 1, j IN respectively. + ∈ + (1) We call these spaces as V where 0 = m 1, V = V , if 0 is even and 0 0 0 V + = V (1) if is odd . 0 0,+ 0 (1) (1) (1) The representations with 0 = 1, 2, 3, . . . , where = , if 0 is even 0 0 0 and (1) = (1) if is odd, act by means of the formula 0 0,+ 0

(1) 1 ay + b 1 a b (g)f(y) = (cy + d) 0 f g = . 0 cy + d c d µ ¶ µ ¶ The inner product in this model is given by

∂ ∞ ∞ 1, x y f(x)g(y)dxdy ( ) ∂ = | | ¯ 0 Z∞ Z∞ ¯ ¯ ( = 1 if 0 even, ¯ = 0 if 0 odd). So ∞ ∞ 1, ( ) = x y 0 log x y f(x)g(y)dxdy. (2.11) | | | | Z∞ Z∞ 2.5.5 The anti-analytic discrete series This is similar to the Section 2.5.4. Only V + is dierent. To describe these spaces 0 we have to take n. 2.6. The analytic discrete series: realization on the complex upper half plane 27

Remark 2.5. Extending the results of Kashiwara and Vergne [18] we can dene the spaces V as Sobolev spaces. For example for the analytic discrete series: 0

+ 2 2 0 V = f L (IR, (1 + x ) dx) : fˆ has support in IR+ 0 ∈ n o and for the anti-analytic discrete series:

2 2 V = f L (IR, (1 + x ) 0 dx) : fˆ has support in IR . 0 ∈ n o 2.6 The analytic discrete series: realization on the complex upper half plane

[See e.g. [21]]. Let m be an integer 2. On C+ = z C : Im z > 0 we have the usual fractional linear action of G =SL(2, IR): { ∈ }

az + b a b g z = if g = cz + d c d µ ¶ dxdy and, if z = x + iy, y2 is a G-invariant measure on C. Let be the Hilbert space of holomorphic functions f on C+ with satisfying Hm

∞ ∞ 2 m 2 f(z) y dxdy < . 0 | | ∞ Z Z∞ G acts in by Hm m az + b (g)f(z) = (cz + d) f (2.12) m cz + d µ ¶ a b if g 1 = , as a continuous unitary representation. c d Let nµbe an¶integer 0 and n z i m (z) = (z + i) . n z + i µ ¶ Then for all n. n ∈ Hm Theorem 2.6. (See [21]). The representation on is irreducible. Let V m Hm m+2n be the one-dimensional subspace generated by n. Then Vm+2n is an eigenspace of K, with weight m 2n and = V Hm m+2n n 0 M is an orthogonal decomposition with highest weight vector of weight m. 0 28 Chapter 2. Representations of SL(2, IR)

Computation of 2 in gives c n! , for all n. We conclude: the map || n|| Hm m m[n] (z) (x) (x IR) i i → i i ∈ i i X X extends to a unitary equivalence between m and the analytic discrete series repre- (1) sentation with 0 = m 1. 0 The inverse map is given by (2.13).

2.7 The limit of the analytic discrete series

For ‘m = 1’, we consider the space of holomorphic functions f on C+ satisfying: H1 2 1 ∞ ∞ 2 1+ f = lim f(z) y dxdy < . || || 0 () 0 | | ∞ ↓ Z Z∞ This is a Hilbert space and G acts on it by (2.12), with m = 1. Theorem 2.6 (1) holds with m=1. Moreover, 1 is unitarily equivalent to 0, , as above. (2) It is clear that the anti-analytic discrete series and 0, can be treated in a similar way.

2.8 Explicit intertwining operator

(1) We shall now describe the explicit form of the unitary equivalence of 1 and 0, from Section 2.7. (1) Observe that V0, is spanned by the functions n x i 1 (x) = (x + i) ; n = 0, 1, 2, . . . n x + i µ ¶ 2y For n = 0, we have ˆ0(y) = 2iY (y)e where Y is the Heaviside function, so all clearly have Fourier transform in [0, ), so that obviously (see Remark 2.5) n ∞ (1) 2 ˆ V0, = f L (IR) : f has support in [0, ) . ∈ ∞ n o Theorem 2.7. Let f L2(IR) be such that fˆ has support in [0, ). Dene ∈ ∞ ∞ F (z) = fˆ(y)e2iyzdy (2.13) Z0 for z C, Im z > 0. Then F is holomorphic on the upper half plane and satises ∈ a. lim F (x + iy) = f(x) in L2- sense y 0 ↓ 2 1 ∞ ∞ 2 1+ b. f 2 = lim F (z) y dxdy. || || 0 () 0 | | ↓ Z Z∞ 2.8. Explicit intertwining operator 29

Proof. F (z) is well-dened for Im z > 0 and clearly holomorphic there. To show a, let g ∞(IR). Then we have: ∈ Cc

∞ ∞ ∞ 2iuy 2vy [F (u + iv) f(u)]g(u)du = fˆ(y)e e g(u)dydu 0 Z∞ Z∞ Z ∞ f(u)g(u)du Z∞ ∞ 2vy = fˆ(y)gˆ(y)e dy Z0 ∞ f(u)g(u)du Z∞ ∞ 2vy = fˆ(y)gˆ(y)[e 1]dy. Z0 1/2 ∞ Select M > 0 such that fˆ(y) 2dy < /4. Splitting the integral into M | | M µZ ¶ ∞ ∞ = + gives: Z0 Z0 ZM

∞ [F (u + iv) f(u)]g(u)du g , || ||2 ¯Z∞ ¯ ¯ ¯ ¯ ¯ independent of g. This¯ proves a. ¯ Now we show that F satises b. One has:

2 1+ ∞ ∞ 2 4vy 1+ F (u + iv) v dudv = fˆ(y) e v dydv + | | | | Z Z0 Z0 Now, ∞ 4vy 1+ ∞ t 1 e v dv = (4y) e t dt = (4y) (). Z0 Z0 Hence:

1 ∞ ∞ 2 1+ ∞ 2 2 lim F (z) y dxdy = lim fˆ(y) (4y) dy = f .¤ 0 () 0 | | 0 0 | | || || ↓ Z Z∞ ↓ Z (1) The map f F is 1-1 from V0, into 1. It is even onto, since n(x) clearly → H corresponds to (z), since ˆ L1. Moreover, the inverse map, restricted to n n ∈ linear combination of n(z) is just the restriction to IR. This map commutes with (1) the respective representations, so f F is an intertwining operator for 0, and → 1. The inverse map is given by a. 30 Chapter 2. Representations of SL(2, IR) CHAPTER 3 Representations of SL(2, C)

In this chapter we give the irreducible unitary representations of SL(2, C) in the ‘non-compact’ model. By ‘non-compact’ model we mean the realization of the representations on a space of functions on C. In order to give a smooth analysis we apply again a ‘compact’ model. Compare, for appreciating our approach, Knapp’s treatment in chapter XVI of his book [19].

3.1 The principal (non-unitary) series 3.1.1 A ‘non-compact’ model Set t 0 P = MAN = 1 : t C, x C x t ∈ ∈ ½µ ¶ ¾ where ei 0 t 0 M = i : IR , A = 1 : t > 0, t IR 0 e ∈ 0 t ∈ ½µ ¶ ¾ ½µ ¶ ¾ and 1 0 N = : w C . w 1 ∈ ½µ ¶ ¾ 1 z Put N = : z C . Then NP is open, dense in G = SL(2, C) and its 0 1 ∈ ½µ ¶ ¾ a b complement has Haar measure zero. Any g = in G can be written in the c d µ ¶ 1 z t 0 form np = 1 with t = 1/d, z = b/d and w = c, provided d = 0. 0 1 w t 6 µ ¶ µ ¶ The principal series ( C, l ) acts on the space V of ∞-functions f with ,l ∈ ∈ C +2 f(gmatn) = t l(m)f(g) +2 il = t e f(g)

31 32 Chapter 3. Representations of SL(2, C)

with l Mˆ ∈ ei 0 (m ) = = eil l l 0 e i µ ¶ and with inner product f(k) 2dk = f 2, | | || ||2 ZK/M where K = SU(2). ,l is given by

1 ,l(g0)f(g) = f(g0 g) with g , g SL(2, C). 0 ∈ Identifying V with a space of functions on N C, we can write ,l(g0) in these terms. The inner product becomes: '

2 2 2 Re f 2 = f(z) (1 + z ) dz. || || | | | | Z

1 a b If g = then, 0 c d µ ¶

1 1 z a az + b 1 z0 t 0 g0 = = 1 0 1 c cz + d 0 1 w t µ ¶ µ ¶ µ ¶ µ ¶ az+b 1 with z0 = cz+d and t = (cz + d) , so that l (+2) cz + d az + b (g )f(z) = cz + d f ,l 0 | | cz + d cz + d µ| |¶ µ ¶ with f L2(C), z C and g SL(2, C). ∈ ∈ 0 ∈ The representations ,l are unitary for imaginary. The converse is also true.

3.1.2 A ‘compact’ model This model enables us more easily to answer questions about irreducibility, equivalence, unitarity, etc. G = SL(2, C) acts on the unit sphere S = (s , s ) C2 : s 2 + s 2 = 1 by 1 2 ∈ | 1| | 2| g(s)© ª g s = g(s) || || transitively. The stabilizer of e2 = (0, 1) is AN. So ,l can be realized (depending on l) on V , the space of ∞-functions ϕ on S satisfying l C ϕ(s) = lϕ(s) with C, = 1, s S and ∈ | | ∈ 1 1 (+2) (g)ϕ(s) = ϕ(g s) g (s) . ,l || || 3.2. Irreducibility 33

Observe that ,l K ind l. | ' M K ↑ Let (ϕ, ) be the usual inner product on L2(S):

(ϕ, ) = ϕ(s)(s)ds (3.1) ZS where ds is the normalized measure on S. The measure ds is transformed by 4 g G as follows: d(g s) = g(s) ds. It implies that the inner product (ϕ, ) is ∈ || || invariant with respect to (,l, ,l), so that ,l is unitary for iIR. ∈ Now we want to study for ,l the following questions: irreducibility, composi- tion series, intertwining operators, unitarity.

3.2 Irreducibility

If l 0, V is spanned by harmonic polynomials homogeneous of degree l + j in l s1, s2 and degree j in s1, s2, i.e.

V = . l Hl+j,j j 0 M

Since dim l+j,j = l + 2j + 1 this K-splitting of Vl is multiplicity free. Moreover, since H ,l K ind l, | ' M K ↑ we have by Frobenius reciprocity any occurring in the decomposition of V Hl+j,j l contains an element l+j,j with

1 (m s) = (m) (s), l+j,j l l+j,j so iϕ iϕ ilϕ l+j,j (e s1, e s2) = e l+j,j (s1, s2).

[l+j,j is unique up to scalars]. It is easily seen that l+j,j (s1, s2) depends only on s2. More precisely, one even has (s , s ) = sl F ( s 2). l+j,j 1 2 2 l+j,j | 2| Then: F (z) = F ( j, l + j + 1; l + 1; z ) l+j,j 2 1 | | see [22]. Since G is generated by K and the subgroup A = exp(tZ ) with { 0 } 1 0 Z = , 0 0 1 µ ¶ in order to study the irreducibility of ,l we have to know how the latter subgroup transforms the functions l+j,j . 34 Chapter 3. Representations of SL(2, C)

l 2 On functions of the form (s1, s2) = s2f( s2 ), ,l(Z0) acts as an ordinary dierential operator : | | L (Z )(s , s ) = sl f( s 2) ,l 0 1 2 2L | 2| with df (f)(u) = 4u(1 u) (u) + (( + 2 + l)(1 2u) + l) f(u). L du

This follows by an easy computation. Applying it to 2F1( j, l + j + 1; l + 1; u), we obtain:

F ( j, l + j + 1; l + 1; u) = c (j, l; ) F ( j, l + j + 1; l + 1; u) + L2 1 0 2 1 c (j, l; )2F1( j + 1, l + j; l + 1; u) + c (j, l; ) F ( j 1, l + j + 2; l + 1; u) + 2 1 using relations between Gauss hypergeometric functions (see [6]). This implies also

,l(Z0)l+j,j = c0(j, l; )l+j,j + c (j, l; )l+j 1,j 1 +c+(j, l; )l+j+1,j+1 If l < 0 we can do the same,

Vl = j, l +j H | | j 0 M and we obtain

,l(Z0)j, l +j = c0(j, l; )j, l +j + c (j, l; )j 1, l +j 1 | | | | | | +c+(j, l; )j+1, l +j+1 | | with l2 c (j, l; ) = , 0 ( l + 2j)( l + 2j + 2) | | | | 2j2 c (j, l; ) = ( l 2j) and ( l + 2j + 1)( l + 2j) | | | | | | 2( l + j + 1)2 c (j, l; ) = | | (2j + + 2 + l ). + ( l + 2j + 1)( l + 2j + 2) | | | | | | This immediately leads to a complete analysis of the reducibility properties of the representations ,l. The results are as follows:

Theorem 3.1.

a) If , l is irreducible. 6∈ ∈ ,l b) If , l are integer and = l + 2j and = 2j 2 l for all j IN, 6 | | 6 | | ∈ ,l is irreducible. For = l + 2j for some j IN the decomposition of ,l is as follows: | | ∈ 3.3. Intertwining operators 35

(i) l 0. In this case, ∞ V = ,l Hl+i,i i= l M2 is an irreducible innite dimensional subspace of Vl and Vl/V,l is irreducible and nite dimensional. (ii) l < 0. In this case, ∞ V,l = i, l+i H i= +l M2 is an irreducible innite dimensional subspace of Vl and Vl/V,l is irreducible and nite dimensional. For = 2j 2 l for some j IN the decomposition of is as follows: | | ∈ ,l (i) l 0. In this case, 2l 2 V = ,l Hl+i,i i=0 M is an irreducible nite dimensional subspace of Vl and Vl/V,l is irreducible and innite dimensional. (ii) l < 0. In this case, 2+l 2 V,l = i, l+i H i=0 M is an irreducible nite dimensional subspace of Vl and Vl/V,l is irreducible and innite dimensional.

3.3 Intertwining operators

Now we want to nd (non zero) continuous linear operators A : V V intert- l → l1 wining the representations ,l and 1,l1 , i.e. A (g) = (g)A (g G). ,l 1,l1 ∈ Theorem 3.2. A non-zero non-trivial intertwining operator as above exists if and only if l = l , = . 1 1 Proof. (a) We suppose that l, l 0. 1 A : V = V = l Hl+j,j → l1 Hl1+j,j j 0 j 0 M M

is a continuous linear operator. Then l = l1, because dim l+j,j = 2j +l +1, dim = 2j + l + 1 and for j = 0 we have the sameH dimension, so Hl1+j,j 1 l = l1. 36 Chapter 3. Representations of SL(2, C)

A = a I on each for some complex constant a by Schur. Moreover, j Hl+j,j j A (X) = (X) A ,l 1,l for all X g. By specializing to X = Z and by letting act the left and ∈ 0 right-hand side on l+j,j , we obtain:

A,l(Z0)l+j,j = 1,l(Z0)Al+j,j .

If l = 0, 6 a = a j 1 j ( l 2j)aj 1 = (1 l 2j)aj ( + l + 2j + 2)aj+1 = (1 + l + 2j + 2)aj .

Then = 1 and A is an scalar operator.

If l = l1 = 0,

( 2j)aj 1 = (1 2j)aj ( + 2j + 2)aj+1 = (1 + 2j + 2)aj .

Combining these equations gives, (2j + + 2)( 2j 2) = (2j + + 2)( 2j 2) 1 1 so = . 1 If = , then all a coincide, so A is an scalar operator. If = we get: 1 j 1 (2j + + 2)a = (2j + 2)a . (3.2) j+1 j For the irreducible case, the equation has, up to a factor, a unique solution which can be written in one of the following three forms:

(j 2 + 1) aj = c1 (3.3) (j + 2 + 1) = c ( 1)j ( + 1 + j)( + 1 j) (3.4) 2 2 2 ( 1)j = c3 . (3.5) ( + 1 + j)( + 1 j) 2 2 In the reducible cases, there is also a unique solution, but one has to start at another base value. This is proven in the same way. The formulae obtained for the solutions of (3.2) show, when the solution is dened on an unbounded set, that it is of polynomial growth at innity, Re a const. j ( j ). j | | | | → ∞ This implies that the operator A having aj as eigenvalues is continuous in ( the ∞-topology of V . C l 3.3. Intertwining operators 37

(b) We suppose that l, l1 < 0.

A : Vl = j, l +j Vl1 = j, l +j H | | → H | 1| j 0 j 0 M M

is an intertwining operator between ,l and 1,l1 if and only if l = l1 and = 1. The proof is the same as l, l1 > 0.

A : Vl = l+j,j Vl1 = j, l +j H → H | 1| j 0 j 0 M M is an intertwining operator between and . Then l = l , because ,l 1,l1 | 1| dim l+j,j = l + 2j + 1, dim j, l1 +j = l1 + 2j + 1. For j = 0 they have to be theH same, l + 1 = l + 1 henceH | | l = l| .| We want to see that = . | 1| | 1| 1 Let

A0 : Vl V l → ϕ(s) ϕ(s) 7→ be an isomorphism and

1 1 2 A (g)ϕ(s) = ϕ(g s) g (s) 0 ,l || || = , l(g)ϕ(s) = , l(g)A0ϕ(s). Hence A0,l(g) = , l(g)A0

t 1 1 0 1 and as g = wgw with w = we have, if A0 = , l(w)A0, 1 0 µ ¶ t 1 A0,l(g) = , l( g )A0.

Notice that A0 is given by A0 ϕ(s , s ) = ϕ(s , s ) if ϕ V . Let us 0 0 1 2 2 1 ∈ l 2, l dene A, lϕ(s) = [s, t] ϕ(t)dt where [s, t] = s1t1 + s2t2. This is an S Z t 1 intertwining operator between , l( g ) and , l(g) (see Lemma 3.3), i.e. t 1 A, l, l( g ) = , l(g)A, l (g G). ∈ Then A, lA0 is an intertwining operator between ,l(g) and , l(g) since t 1 A, lA0,l(g) = A, l, l( g )A0 = , l(g)A, lA0. 38 Chapter 3. Representations of SL(2, C)

We have proved that ,l , l with l > 0. Suppose that ,l , l 1 then , l , l and this can only happen if 1 = . So the only 1 possibility is l1 = l and 1 = . Observe that A, lA0 is given by 2, l (A, lA0)ϕ(s) = (s2t1 s1t2) ϕ(t)dt. ZS

(d) We suppose that l < 0 and l1 > 0. We obtain the same as in (c), l1 = l and = .¤ 1 Lemma 3.3. The integral operator on Vl

2,l A,lϕ(s) = [s, t] ϕ(t)dt ZS t 1 is an intertwining operator between ,l(g) and ,l( g ). Proof. A,l is not dened for all . It is dened for Re > 1 and there is holomorphic. It can be meromorphically continued to the whole complex plane. First, we prove that A,l is an intertwining operator between ,l and ,l the induced representation from a b P = 1 0 a µ ¶ given by 1 1 2 ,l(g)ϕ(s) = ϕ((g ) s) (g )(s) || || 1 t with (g) = (g) , the Cartan involution and g = g. We shall show that A,l,l(g) = ,l(g)A,l. Indeed,

2,l A,l,l(g)ϕ(s) = [s, t] ,l(g)ϕ(t)dt ZS 2,l 1 1 2 = [s, t] ϕ(g t) g (t) dt. || || ZS On the other hand, 1 1 2 ,l(g)Aϕ(s) = (Aϕ)((g ) s) (g )(s) || || 1 2 1 2,l = (g )(s) [(g ) s, t] ϕ(t)dt || || ZS 2,l = [s, gt] ϕ(t)dt ZS 2,l 1 1 1 4 = [s, t˜] ϕ(g t˜) g (t˜) dt˜ g 1(t˜) 2 || || ZS || || 2,l 1 1 2 = [s, t˜] ϕ(g t˜) g (t˜) dt.˜ || || ZS 3.4. Invariant Hermitian forms and unitarity 39

d c a b because being (g) = (g ) 1 = (tg) 1 = if g = and (g 1) = b a c d µ ¶ µ ¶ a c , it is obtained b d µ ¶ 1 1 1 1 [(g ) s, t] = [(g )(s), t] = [s, gt]. (g 1)(s) (g 1)(s) || || || || 1 1 4 Doing a change of variable t = g t˜ and dt = g (t˜) dt˜. || || t 1 So A,l,l(g) = ,l(g)A,l and ,l(g) = ,l( g ). Hence t 1 A,l,l(g) = ,l( g )A,l.¤ 3.4 Invariant Hermitian forms and unitarity

In this section we determine all invariant Hermitian forms on Vl with respect to the representations ,l and determine which of these forms are positive (negative) denite, so that the corresponding representations are unitarizable. A continuous Hermitian form H(ϕ, ) on Vl is called invariant with respect to ,l if 1 H(,l(g)ϕ, ) = H(ϕ, ,l(g )) for all g G. Any such a form can be written in the form ∈ H(ϕ, ) = (Aϕ, ) with A an operator on Vl and the inner product (3.1) on the right-hand side. It implies that A intertwines ,l and ,l. So by Theorem 3.2, there are two po- ssibilities: = and = . In the rst case, we have Re = 0 and, provided = 0, the operator A is equal to cE, so that H is c times (3.1). In the second 6 case, we must have l = 0 and A intertwines ,0 and ,0. So we get:

Theorem 3.4. A non-zero invariant Hermitian form H(ϕ, ) on Vl exists only if: (a) Re = 0, or (b) Im = 0 and l = 0. In case (a) the form is proportional 2 to the L -inner product (3.1). In case (b) the form H(ϕ, ) on Vl has the form H(ϕ, ) = (Aϕ, ) where A is an intertwining operator between ,0 and ,0. Now we determine when the Hermitian forms above are positive or negative denite.

Theorem 3.5. The unitarizable irreducible representation ,l belong to the following series. (i) the unitary principal series: ,l with Re = 0. (ii) the complementary series consisting of the representations ,0 with 0 < < 2. (iii) the trivial representation. 40 Chapter 3. Representations of SL(2, C)

Proof. Like in case SL(2, IR). In the basis l+j,j an invariant Hermitian form H(ϕ, ) has a diagonal matrix, with real scalars hj on the diagonal. The hj have the same expression as aj . We have to determine when the numbers aj have the same sign for all j J, where J is the set of weights of our representations. This leads to the represen∈ tations in the theorem. For example, when 0 < < 2 we easily see that (3.4) is positive for all j, if c2 > 0. ¤ Let us denote the unitary completions of the representations by the same sym- bols. These unitary completions exhaust all irreducible unitary representations of G up to equivalence, see [13].

3.5 The ‘non-compact’ models of the irreducible unitary representations of SL(2, C)

3.5.1 The continuous series: λ,l, ∈ iIR

We just refer to Section 3.1.1. The space is L2(C) with the usual inner product and l (+2) cz + d az + b (g)f(z) = cz + d f (3.6) ,l | | cz + d cz + d µ| |¶ µ ¶ a b if g 1 = . c d µ ¶

3.5.2 The complementary series: λ,0 (0 < < 2) The inner product is given in the ‘compact’ model by

2 (ϕ, ) = s t s t ϕ(t)(s)dsdt | 2 1 1 2| ZS ZS (ϕ, V ). ∈ 0 The representation in the ‘non-compact’ model is again given by (3.6):

(+2) az + b (g)f(z) = cz + d f ,0 | | cz + d µ ¶ a b if g 1 = . c d µ ¶ Let us rewrite (ϕ, ) in terms of functions on C. This is easily seen to be:

(f, g) = z w f(z)g(w)dzdw.

| | Z Z 3.6. Finite-dimensional irreducible representations 41

3.6 Finite-dimensional irreducible representations

Choose = l 2 in the ‘non-compact’ picture, span(1, z, . . . , zl) is an irreducible invariant subspace of Vl under ,l. These spaces exhaust all nite-dimensional irreducible representations of SL(2, C). In the ‘compact’ picture the spaces are . Hl,0 42 Chapter 3. Representations of SL(2, C) CHAPTER 4 The metaplectic representation

The metaplectic representation -also called the oscillator representation, harmonic representation, or Segal-Shale-Weil representation- is a double-valued unitary representation of the symplectic group Sp(n, IR) on L2(IRn). In this chapter we give an overview of the construction of this representation and we examine it from several viewpoints. The contents of this chapter are adapted from [10].

4.1 The Heisenberg group 4.1.1 Denition First we introduce some notations. Let us denote the product of two vectors in IRn or Cn by simple juxtaposition

n xy = x y (x, y IRn or Cn). j j ∈ 1 X Thus, the Hermitian inner product of z, w Cn is zw. We also set ∈ n x2 = xx = x2 (x IRn or Cn), j ∈ 1 Xn z 2 = zz = z 2 (z Cn). | | | j | ∈ 1 X When linear mappings intervene in such products we denote by

xAy = y tAx = x A y (x, y Cn, A M (C)). j jk k ∈ ∈ n X We consider IR2n+1 with coordinates

(p1, . . . , pn, q1, . . . , qn, t) = (p, q, t),

43 44 Chapter 4. The metaplectic representation and we dene a Lie bracket on IR2n+1 by

[(p, q, t), (p0, q0, t0)] = (0, 0, pq0 qp0). (4.1) It is easily veried that the bracket (4.1) makes IR2n+1 into a Lie algebra, called the Heisenberg Lie algebra and denoted by hn. In order to identify the Lie group corresponding to hn, it is convenient to use a matrix representation. Given (p, q, t) IR2n+1, we dene the matrix m(p, q, t) ∈ ∈ Mn+2(IR) by 0 p p t 1 n 0 0 0 q1 . . . . .  m(p, q, t) = ......   0 0 0 qn 0 0 0 0    Moreover, we dene   M(p, q, t) = I + m(p, q, t). It is easily veried that

m(p, q, t)m(p0, q0, t0) = m(0, 0, pq0), (4.2)

M(p, q, t)M(p0, q0, t0) = M(p + p0, q + q0, t + t0 + pq0). (4.3) From (4.2) it follows that

[m(p, q, t), m(p0, q0, t0)] = m(0, 0, pq0 qp0), where the bracket now denotes the commutator. Hence the correspondence X 2n+1 → m(X) is a Lie algebra isomorphism from hn to m(X) : X IR and to obtain the corresponding Lie group we can simply apply the matrix∈ exponential map. So, © ª 1 em(p,q,t) = M(p, q, t + pq). 2 Thus the exponential map is a bijection from m(X) : X IR2n+1 to M(X) : { ∈ } { X IR2n+1 , and the latter is a group with group law (4.3). We could take this ∈ } to be the Lie group corresponding to hn, but we prefer to use a dierent model. It is easily veried that 1 exp m(p, q, t) exp m(p0, q0, t0) = exp m(p + p0, q + q0, t + t0 + (pq0 qp0)). 2 Therefore, if we identify X IR2n+1 with the matrix em(X), we make IR2n+1 into a group with group law ∈ 1 (p, q, t)(p0, q0, t0) = (p + p0, q + q0, t + t0 + (pq0 qp0)). 2

We call this group the Heisenberg group and denote it by Hn. We observe that = (0, 0, t) : t IR Z { ∈ } is the center of Hn. 4.1. The Heisenberg group 45

4.1.2 The Schrodinger representation n Let Xj and Dj be the dierential operators on IR dened by 1 ∂f (Xj f)(x) = xjf(x), Djf = . 2i ∂xj We may regard these operators as continuous operators on the Schwartz space n (IR ). The map dh from the Heisenberg algebra hn to the set of skew-Hermitian opS erators on (IRn) dened by S dh(p, q, t) = 2i(hpD + qX + tI) is a Lie algebra homomorphism. We exponentiate this representation of hn to obtain a unitary representation of the Heisenberg group Hn. The map dened by

2iht 2i(hpD+qX) h(p, q, t) = e e that is, 2iht+2iqx+ihpq h(p, q, t)f(x) = e f(x + hp) 2 n is a unitary representation of Hn on L (IR ), for any real number h. Moreover, h and 0 are inequivalent for h = h0. is irreducible for h = 0. h 6 h 6 We call h the Schrodinger representation of Hn with parameter h. Generally we shall take h = 1 and restrict attention to the representation = 1. Since the central variable t always acts in a simple-minded way, as multiplication by the scalar e2it, it is often convenient to disregard it entirely; we therefore dene

(p, q) = (p, q, 0) = e2i(pD+qX).

4.1.3 The Fock-Bargmann representation There is a particularly interesting realization of the innite-dimensional irreducible unitary representations of Hn in a Hilbert space of entire functions. Let us dene the Fock Space as

n 2 2 z 2 n = F : F is entire on C and F = F (z) e | | dz < F || ||F | | ∞ ½ Z ¾ and for z Cn ∈ n/4 2xz x2 (/2)z2 Bf(z) = 2 f(x)e dx. Z Bf is called the Bargmann transform of f and it is an isometry from L2(IRn) into the Fock Space. The Schrodinger representation can be transferred via the Bargmann transform to a representation of Hn on n. To describe this representation, it will be convenient to identify the underlyingF manifold of H with Cn IR: n (p, q, t) (p + iq, t). → 46 Chapter 4. The metaplectic representation

In this parametrization of Hn the group law is given by 1 (z, t)(z0, t0) = (z + z0, t + t0 + Im zz0). 2

The group Hn can also be seen inside U(1, n) as the subgroup N, see [8]. The transferred representation is then dened by

(p + iq, t)B = B(p, q, t), in other words,

(/2) w 2 zw+2it (w, t)F (z) = e | | F (z + w).

This is called the Fock-Bargmann representation.

4.2 The metaplectic representation 4.2.1 Symplectic linear algebra and symplectic group In this section we shall be working with 2n 2n matrices, which we shall frequently write in block form: A B = A C D µ ¶ where A, B, C and D are n n matrices. Let be the matrix J 0 I = J I 0 µ ¶ which describes the symplectic form on IR2n:

[w , w ] = w w . 1 2 1J 2 The symplectic group Sp(n, IR) is the group of all 2n 2n real matrices which, as operators on IR2n, preserve the symplectic form:

Sp(n, IR) [ w , w ] = [w , w ] for all w , w IR2n. A ∈ ⇐⇒ A 1 A 2 1 2 1 2 ∈

The symplectic Lie algebra sp(n, IR) is the set of all M2n(IR) such that t A ∈ e A Sp(n, IR) for all t IR. When the dimension n is xed, we shall abbreviate ∈ ∈ Sp = Sp(n, IR) , sp = sp(n, IR).

The group Sp is generated by D N and also by D N with ∪ ∪ {J } ∪ ∪ {J } I A I 0 N = : A = tA , N = : A = tA 0 I A I ½µ ¶ ¾ ½µ ¶ ¾ 4.2. The metaplectic representation 47 and A 0 D = t 1 : A GL(n, IR) . 0 A ∈ ½µ ¶ ¾ In connection with the Fock model we want to use complex coordinates z = p + iq corresponding to the real coordinates p, q on IR2n. Since the action of Sp on IR2n is not complex linear, however, it will be more appropriate to map IR2n into C2n by W (p, q) = (p + iq, p iq). 0 Under this mapping, any linear transformation T on IR2n turns into the linear 1 2n transformation Tc = W0T W0 on C , so on the level of matrices we have the map M (IR) M (C) A ∈ 2n → Ac ∈ 2n given by 1 1 = = , (4.4) Ac W0AW0 WAW where I iI 1 1 I iI 0 = , = 0 = . W I iI W √2W √2 I iI µ ¶ µ ¶ We denote the images of M2n(IR) and Sp under the map (4.4) by M2n(IR)c and Spc: M (IR) = : M (IR) , Sp = : S . 2n c {Ac A ∈ 2n } c {Ac A ∈ p} One can easily show that

P Q M (IR) = : P, Q M (C) . 2n c Q P ∈ n ½µ ¶ ¾ 4.2.2 Construction of the metaplectic representation 4.2.2.1 The Schrodinger model

The symplectic group Sp acts on the Heisenberg group Hn by

(p, q, t) = ( (p, q), t) with Sp and (p, q, t) H . A A A ∈ ∈ n Composing with the Schrodinger representation we have a new representation 2 n 2it of Hn on L (IR ) such that (0, 0, t) = e I. By the Stone-von Neumann theorem,A and are equivalent:Athere exists a unitary operator ( ) on L2(IRn) such that A A 1 (X) = ( )(X)( ) , X H . (4.5) A A A ∈ n Moreover, by Schur’s lemma, ( ) is determined up to a scalar factor of modulus one. It follows that A

( ) = c , ( )( ), c , = 1, AB A B A B | A B| so that is a projective unitary representation of Sp in L2(IRn). We shall prove later that the scalar factors of modulus one can be chosen in one and only one way 48 Chapter 4. The metaplectic representation up to factors of 1 so that becomes a double-valued unitary representation of Sp, i.e., ( ) = ( )( ). AB A B With this choice of the scalar factors of modulus one, is called the metaplectic representation of Sp. If we pass to the double covering group Sp2, denes a unitary representation of Sp2. The explicit calculation of ( ) for a general Sp is rather complicated, but it is easy to use (4.5) to nd A( ) up to a scalarA ∈factor of modulus one when belongs to certain subgroups of SAp. Doing this we obtain, A

A 0 1/2 1 t 1 f(x) = det A f(A x), (4.6) 0 A | | ·µ ¶¸ I 0 f(x) = e ixCxf(x), (4.7) C I ·µ ¶¸ 1 ( ) = (4.8) J F 2ixy where is the Fourier transform dened by f(y) = e f(x)dx. F F As Sp is generated by matrices of these three typZes, we have in some sense computed the metaplectic representation up to scalar factors of modulus one.

4.2.2.2 The Fock model The easiest way to construct the metaplectic representation globally is to move it over to Fock space. We recall that on the Fock Space we have the representation Fn of Hn obtained from the Schrodinger representation by conjugation with the Bargmann transform B:

1 (/2) w 2 zw (p + iq) = B(p, q)B , (w)F (z) = e | | F (z + w). Since we use the complex coordinates w = p + iq for describing , we also use the complex form Spc of Sp. We adopt the following notational convention: if Spc and w Cn, we dene w to be the vector z Cn such that (w, w) =A ∈(z, z). That is,∈ A ∈ A P Q w = z z = P w + Qw where = . A ⇐⇒ A Q P µ ¶ The Fock-metaplectic representation of Spc is then dened up to scalar factors of modulus one by the condition

1 ( w) = ( )(w)( ) . (4.9) A A A The advantage of the Fock space is that the operators ( ) are integral operators whose kernels are rather easily computable. We have A

w 2 ( )F (z) = K (z, w)F (w)e | | dw (4.10) A A Z 4.2. The metaplectic representation 49

zw where K (z, w) =< ( )Ew, Ez > and Ew(z) = e . We proceed to calculate K . A A F A P Q Theorem 4.1. For = Sp , dene the operator ( ) (modulo 1) on A Q P ∈ c A given by (4.10), µ ¶ Fn

1/2 1 1 1 1 K (z, w) = (det P ) exp (zQP z + 2wP z wP Qw) . A 2 ½ ¾ Then for all , Sp and w Cn, A B ∈ c ∈ a) ( ) is unitary, A 1 b) ( )(w)( ) = ( w), A A A c) ( )( ) = ( ). A B AB Moreover, if 0( ) : Sp is any family of operators satisfying these three { A A ∈ } conditions, then 0( ) = ( ) for all Sp. A A A ∈ Remark 4.2. The uniqueness part of this theorem shows, in particular, that can not be made into a single-valued representation of Spc, as it is impossible to dene 1/2 a continuous single-valued branch of det P on Spc. We can now go back to the Schrodinger picture and dene the metaplectic representation by

1 ( ) = B ( )B, as in (4.4). A Ac Ac Theorem 4.1 translates into a corresponding result for . Proposition 4.3. With the correct scalar factors of modulus one the equation (4.6), (4.7) and (4.8) are equal

A 0 1/2 1 t 1 f(x) = (det A) f(A x), 0 A ·µ ¶¸ I 0 ixCx f(x) = e f(x), C I ·µ ¶¸ n/2 1 ( ) = i . J F 50 Chapter 4. The metaplectic representation CHAPTER 5 Theory of invariant Hilbert subspaces

In this chapter we recall an important tool for the representation theory of Lie groups, namely the theory of Hilbert subspaces invariant under a group of automorphisms. This theory is due to L. Schwartz [27], Thomas [34] and Pestman [25]. A main subject is the study of the theory of the reproducing kernels, associated with Hilbert subspaces. We give a criterion (due to Thomas) which assures multiplicity free decomposition of representations. This criterion will be applied in the next chapters.

5.1 Kernels and Hilbert subspaces

n Let E be a quasi-complete locally convex space over C, e.g. 0(IR ), the space of n S tempered distributions on IR , and E the anti-dual of E, being the linear space of continuous, anti-linear forms on E provided with the strong topology. Let H be a Hilbert space. It is called a Hilbert subspace of E if H E and the linear inclusion j : H E is continuous. Note that the image of j : E H is a dense subspace of H. → → For e E, E we set e, for the value of at e. The inner product of H will be∈denoted∈ by ( ). h i | Let K = jj. Then K is a continuous linear operator from E to E, called the reproducing kernel of H. So, for , E one has ∈

K, = (j j), (5.1) h i | which shows that K is a Hermitian kernel:

K, = K, . h i h i Putting = in (5.1), we see that K is even a positive-denite kernel. Let be the convex cone of positive-denite kernels. One has (L. Schwartz [27]):

51 52 Chapter 5. Theory of invariant Hilbert subspaces

Lemma 5.1. To each K there corresponds a unique Hilbert subspace H of E of which K is the reproducing∈ kernel.

5.2 Invariant Hilbert subspaces

Let u be a continuous automorphism of E. We say that a Hilbert subspace H of E is u-invariant if u(H) = H and u is unitary. An equivalent statement is : |H uKu = K, if K is the reproducing kernel of H, so

Ku, u = K, (, E). h i h i ∈ We call K a u-invariant kernel. Let GL(E) be the group of all continuous automorphisms of E and let G be a subgroup of GL(E). A Hilbert subspace H of E is said to be G-invariant if H is u-invariant for all u G. Similarly, a kernel K is said to be G-invariant if K is u-invariant for all u ∈G. It is clear that the set of all G-invariant reproducing kernels is a closed conv∈ex cone in . We will denote this cone by G. Proposition 5.2. A Hilbert subspace H is a minimal (irreducible) G-invariant Hilbert subspace of E if and only if K, the reproducing kernel of H, lies on an extremal ray of G. If E is a conuclear space, i.e. the strong dual of a barrelled nuclear space ([35] Ch. 33), then G is a well-capped cone (in the sense of Choquet [4]), so in this case G is the closed convex hull of its extremal rays ([4]). We then always have minimal G-invariant Hilbert subspace provided G = 0 . A ne example for E n 6 { } is the space E = 0(IR ). S Denote by ext(G) the set of extremal rays of G. Let So be a section of ext(G). That is a set, not containing 0, having exactly one point on each extremal ray. A section is said to be admissible if the function equal to 1 on So and homogeneous of degree 1, is universally measurable. One can prove that such sections always exist. An admissible parameterization of ext( ) is a topological Hausdor space S with a continuous one-to-one map G P from S to ext(G), : s Ks ext(G), such that the image is an admissible section and the inverseP map→ is univ∈ ersally measurable. Such parameterizations exist ([34]). We have ([27], [34]): Theorem 5.3. Let E be a conuclear space, G a subgroup of GL(E) and s K an → s admissible parameterization of ext(G). Then for every K G there is a positive Radon measure m on S such that ∈

K = Ksdm(s). ZS The next theorem describes the conditions under which the decomposition in Theorem 5.3 is unique ([25], [34]). Theorem 5.4. Let E be a quasi-complete locally convex space, G a subgroup of GL(E) and S an admissible parameterization of ext(G). Then the following statements are equivalent: 5.3. Multiplicity free decomposition 53

1. For every G-invariant Hilbert subspace H of E the commutant of G in (H), the algebra of continuous linear operators on H, is commutative. L 2. The cone G is a lattice (i.e. any two elements in the cone have a smallest common majorant in the cone). If, in addition, E is conuclear, then the above statements are each equivalent with: 3. If H1 and H2 are minimal G-invariant Hilbert subspaces of E which are not proportional then the irreducible representations of G on H1 and H2 are inequivalent. 4. For every K G there is a unique positive Radon measure m on S such that ∈ K = Ksdm(s). ZS 5.3 Multiplicity free decomposition

Denition 5.5. We say that the action of G on E is multiplicity free if one of the conditions 1.,2.,3. of Theorem 5.4 is satised. Criterion 5.6. (Thomas). Let J : E E be an anti-automorphism. If JH = H → (i.e. J H is anti-unitary) for every G-invariant Hilbert subspace H of E, then G acts multiplicity| free on E. Let K be the reproducing kernel of H. Then the condition JH = H is equivalent with K = JKJ , so it suces to show that

KJ , J = K, h i h i for all , E. For the ∈proof of the criterion, we refer to [25], Theorem I.5.4.

5.4 Representations

Assume that G (instead of a subgroup of GL(E)) is a locally compact topological group. Let E, for the moment, be an arbitrary locally convex space. A map : G GL(E) is said to be a representation of G on E if: → (i) (g1g2) = (g1)(g2) (g1, g2 G), (ii) the map (g, e) (g)e from G∈ E to E is continuous. If E is a barrel→led space then condition (ii) is already satised if the map g (g)e is continuous for every e E. → ∈ Denition 5.7. If the map : G GL(E) is a representation of G on E, we say that a Hilbert subspace H is G-in→variant if H is invariant under (G). Of course this notion of G-invariance depends upon the representation . This will give no problems to us because mostly there is only one representation under consideration. 54 Chapter 5. Theory of invariant Hilbert subspaces

Proposition 5.8. Suppose : G GL(E) is a representation of G on E and H is → a G-invariant Hilbert subspace of E. Then the map g (g) H is a continuous unitary representation of G. → | For a proof, see [25], Prop. I.5.4. Clearly (g) = (g) H is irreducible if and only if H is a minimal G-invariant subspace. | Denote Ec (Eb) the space E equipped with the topology of uniform convergence on compact (bounded) sets in E. If : G GL(E) is a represen- 1 → tation of G on E, then the map g (g ) is a representation of G on Ec. This representation is said to be the c→ontragradient representation of if the map 1 (g, e) (g )e is a continuous map G E E. This map is in general not → b → b continuous. For a Montel space E (see [35] for denition), the spaces Eb and Ec are equal however. In particular it is true for E = (IRn), the space of Schwartz n n S functions on IR , and E = 0(IR ). S 5.5 Schwartz’s kernel theorem for tempered distributions

Let (E, E) denote the space of continuous linear maps E E, where E is a L → locally convex space and E its dual, provided with the strong topology. According to [35], Theorem 51.7 we have the following canonical isomorphism. Theorem 5.9. The space of tempered distributions on IRm+n is canonically isomor- m n m n phic to ( (IR ), 0(IR )). So given K ( (IR ), 0(IR )), there is a unique LmS+n S ∈ L S S T 0(IR ) such that ∈ S K, = T, ( (IRm), (IRn)). h i h i ∈ S ∈ S where ( ) (x, y) = (x)(y) (x IRm, y IRn). ∈ ∈ CHAPTER 6 The oscillator representation of SL(2, IR) O(2n)

In the rst part of this chapter we compute the Plancherel formula of the oscillator representation ω2n of SL(2, IR) O(2n). The main tool we use is a Fourier integral operator which was introducedby M. Kashiwara and M. Vergne, see [18]. After 2n that we can conclude that any minimal invariant Hilbert subspace of 0(IR ) S occurs in the decomposition of L2(IR2n). Finally we study the oscillator representation ω2n in the context of the theory of invariant Hilbert subspaces. The oscillator representation acts on the Hilbert 2 2n 2n space L (IR ). It is well-known that (IR ) is ω2n(G)-stable, so, by duality, 2n 2 2Sn ω2n acts on 0(IR ) as well, and L (IR ) can thus be considered as an invariant S 2n Hilbert subspace of 0(IR ). Our main result is that any ω2n(G)-stable Hilbert 2n S subspace of 0(IR ) decomposes multiplicity free. S 6.1 The denition of the oscillator representation

Let G be the group SL(2, IR) O(2n) with n > 1 and let ω be the unitary 2n representation of G on H = L2(IR2n) dened by:

1 ω (g)f(x) = f(g x), g O(2n) 2n ∈ n a 0 ω2n(g(a))f(x) = a f(ax), g(a) = 1 0 a µ ¶ 2 1 b ω (t(b))f(x) = e ib x f(x), t(b) = 2n || || 0 1 µ ¶ n 2i[x,y] 0 1 ω2n()f(x) = i e f(y)dy, = 2n 1 0 Z µ ¶ where [x, y] = x y + x y + + x y for x = (x , x , . . . , x ), y = (y , y , . . . , 1 1 2 2 2n 2n 1 2 2n 1 2 y2n).

55 56 Chapter 6. The oscillator representation of SL(2, IR) O(2n)

This is a well-dened representation for 2n. If we consider SL(2, IR) O(n) with n odd, this is a unitary representation of SL(2, IR) O(n) with SL(2, IR) a double covering of SL(2, IR) [10],[29],[30],[44]. We call ω2n the oscillator representation of Gf. More precisely, it is thef natural extension of the restriction of the metaplectic representation of the double covering of the group Sp(2n, IR) to SL(2, IR) SO(2n), dened by Shale, Segal and Weil. Let (IR2n) be the space of Schwartz functions on IR2n. Note that (IR2n) is S 2n S stable under the action of ω (G), so is the space 0(IR ) of tempered distributions 2n S on IR2n.

6.2 Some minimal invariant Hilbert subspaces of S0(IR2n)

2 2n 2n We consider L (IR ) as a Hilbert subspace of 0(IR ) and we will try to decompose it into minimal invariant Hilbert subspaces.S Irreducible unitary representations of G are of the form where is an irreducible representation of SL(2, IR) and one of O(2n). We will now construct intertwining operators from (IR2n) to the Hilbert space of such representations. Not all combinations (, ) occur.S Here are some of these operators. Let denote the space of harmonic polynomials on IR2n, homogeneous of Hl degree l. This space has a reproducing kernel, say Kl(x, y). For any f l one has: ∈ H 2n f(x) = f(s)Kl(s, x)ds (x IR ) (6.1) 2n1 ∈ ZS 2n 1 2n 2 2 2 with S = x IR : x = x + +x = 1 . The reproducing kernel K is { ∈ || || 1 2n } l known to be real-valued and is positive-denite. It satises Kl(gx, gy) = Kl(x, y) 2l for all g O(2n), Kl(x, y) = Kl(x, y), Kl(x, ) and Kl( , y) are in l. The group∈ O(2n) acts irreducibly on by H Hl 1 (g)f(x) = f(g x) l for all g O(2n) and f l. Let C∈+ denote the upp∈ erH half plane z = x+iy : y > 0 with invariant measure (under SL(2, IR)): { } dxdy . y2 2 + + Let Lhol,m(C ) be the space of holomorphic functions on C satisfying,

2 m 2 f(z) y dxdy < + | | ∞ Z with m an integer 2 (in Chapter 2 this space is called m). The group SL(2, IR) acts irreducibly onthis space by H

1 m (g)f(z) = f(g z)(cz + d) m 2n 6.2. Some minimal invariant Hilbert subspaces of 0(IR ) 57 S

1 a b 1 az+b with g = , g z = . ( belongs to the so-called analytic discrete c d cz+d m series, see Chapterµ ¶2) Let us dene an operator from (IR2n) into the Hilbert space L2 (C+) Fl S hol,l+n ˆ as follows: 2Hl i y 2 ( lϕ)(, x) = e || || ϕ(y)Kl(y, x)dy F 2n Z This integral exists for all x IR2n and C+. We shall show: ∈ ∈ 1) The operator is well dened, i.e. ϕ L2 (C+) ˆ . Fl Fl ∈ hol,l+n 2Hl 2) is continuous as operator from (IR2n) to L2 (C+) ˆ . Fl S hol,l+n 2Hl 3) For l = 0, 1, 2, . . . the operator intertwines the action of ω and . Fl 2n l+n l Let us assume for the moment property 1) and 2), and let us show property 3). It is clear that l intertwines the O(2n)-action. The intertwining relations for the elements g(aF) and t(b) are easy to check. We now check the -intertwining relation: ω ()ϕ(, x) = ()( ϕ)(, x) (6.2) Fl 2n l+n Fl for ϕ (IR2n). By denition, ∈ S n i y 2 lω2n()ϕ(, x) = i e || || ϕˆ( y)Kl(y, x)dy F 2n Z 2i[x,y] with ϕˆ being the Fourier transform dened by ϕˆ(x) = ϕ(y)e dy. 2n We apply the following lemma, see [26], PropositionZ5-1. n Lemma 6.1. For P l, a harmonic polynomial of degree l, and ϕ (IR ) one has ∈ H ∈ S i x 2 l n/2 n/2 i x 2/

e || || P (x)ϕˆ(x)dx = i e || || P (x)ϕ(x)dx

n n Z Z with C+ . ∈

The proof is straightforward by induction on l, recalling that l is spanned by l n H2 2 the polynomials of the form [y, z] with z C satisfying [z, z] = z1 + + zn = 0. For l = 0 the result is classical and is related∈ to the theory of the Fourier transform of quadratic characters. The lemma easily implies (6.2).

Now we prove property 1) and 2). Let us compute the square norm of ϕ in Fl L2 (C+) ˆ for ϕ (IR2n). We have: hol,l+n 2Hl ∈ S 2 2 n+l 2 lϕ = ( lϕ)(, s) (Im ) dsd ||F || + 2n1 | F | Z ZS i y 2 i z 2 n+l 2

= e || || || || ϕ(y)ϕ(z)Kl(y, z)(Im ) dzdyd

+ 2n 2n Z Z Z 58 Chapter 6. The oscillator representation of SL(2, IR) O(2n)

here we have used the reproducing property (6.1) of Kl(y, ). 2n 1 Writing ϕ(y) = ϕ(rs), ϕ(z) = ϕ(ts0) with r = y , t = z , s, s0 S , = u + iv and u IR, v > 0, we get, || || || || ∈ ∈

2 ∞ ∞ ∞ ∞ iu(r2 t2) v(r2+t2) lϕ = e e ϕ(rs)ϕ(ts0)Kl(s, s0) ||F || 0 0 0 S2n1 S2n1 Z∞ Z Z Z Z Z l+2n 1 l+2n 1 n+l 2 r t v dsds0drdtdvdu ∞ ∞ 2vr2 2l+4n 3 n+l 2 = e ϕ(rs)ϕ(rs0)Kl(s, s0)r v dsds0drdv 2n1 2n1 Z0 Z0 ZS ZS ∞ ∞ 2vr2 n+l 2 = e v dv ϕ(rs)ϕ(rs0)Kl(s, s0)dsds0 2n1 2n1 Z0 µZ0 ¶ µZS ZS ¶ 2l+4n 3 r dr (6.3) Setting 2r2v = w we obtain

∞ 2vr2 n+l 2 (n + l 1) e v dv = . (2r2)n+l 1 Z0 Dening

(Plϕ)(rs0) = ϕ(rs)Kl(s, s0)ds, 2n1 ZS we get

2 ϕ(rs)ϕ(rs0)Kl(s, s0)dsds0 = (Plϕ)(rs0) ds0 2n1 2n1 2n1 | | ZS ZS ZS ∞ since P ϕ(r ) is in and ϕ(rs) = (P ϕ)(rs). By substitution we obtain from l Hl l l=0 (6.3), X 2 (n + l 1) ∞ 2 2n 1 lϕ = n+l 1 (Plϕ)(rs0) r ds0dr ||F || (2) 2n1 | | Z0 ZS By Parseval’s theorem,

2 2 (Plϕ)(rs0) ds0 = ϕ(rs0) ds0 2n1 | | 2n1 | | l 0 S S X Z Z so nally we obtain

n+l 1 ∞ (2) 2 ∞ 2 2n 1 2 lϕ = ϕ(rs0) r ds0dr = ϕ 2 (6.4) (n + l 1) ||F || 0 S2n1 | | || || Xl=0 Z Z This proves property 1) and also property 2) since (6.4) implies the inequality

2 (n + l 1) 2 lϕ n+l 1 ϕ 2, ||F || (2) || || 6.3. Decomposition of L2(IR2n) 59 which actually shows that is continuous in the L2-topology on (IR2n), so on Fl S (IR2n) itself, since the embedding of (IR2n) into L2(IR2n) is continuous. S So we have shown: S

Theorem 6.2. For l = 0, 1, 2, . . . the operator l is a continuous intertwining 2n F 2 + operator for the action of ω on (IR ) and on Lhol (C ) ˆ . 2n S l+n l ,l+n 2Hl 6.3 Decomposition of L2(IR2n)

Clearly the proof of Theorem 6.2 in Section 6.2 also gives the decomposition of L2(IR2n) into minimal invariant subspaces. The decomposition is multiplicity free (see [16]) and the irreducible unitary representations which occur are given by (n+l 1) . Observe that the factor d = is the formal degree of . In l+n l l (2)n+l1 n+l conclusion we have, Corollary 6.3. Let ϕ be a function in (IR2n). We have the following Plancherel formula S ∞ (2)n+l 1 ϕ 2 = ϕ 2. || ||2 (n + l 1) ||Fl || Xl=0 As representations of SL(2, IR) O(2n), we have

∞ L2(IR2n) = . l+n l Xl=0 6.4 Classication of all minimal invariant Hilbert subspaces of S0(IR2n)

2n Instead of classifying the minimal ω2n(G)-invariant Hilbert subspaces of 0(IR ) we shall classify the extremal reproducing kernels. S Let us rst compute the reproducing kernel of l+n l. It is given by the distribution T on IR2n IR2n, satisfying: l

l+n 2 < Tl, ϕ >= ( lϕ, l) = lϕ(, s) l(, s)(Im ) dsd. F F + 2n1 F F Z ZS

Tl is equal to the distribution:

2 2 2n+2 T (x, y) = d ( x y ) x K (s, s0) l l || || || || || || l 2n 2n 2 on IR IR (0, 0). This easily follows from the computation of lϕ in Section 6.2. \ ||F || 2n Let now H be a Hilbert subspace of 0(IR ), invariant under ω2n(G), and let K be its reproducing kernel. By Schwartz’sS kernel theorem we can associate to it a unique tempered distribution T on IR2n IR2n. This reproducing distribution satises the following conditions: 60 Chapter 6. The oscillator representation of SL(2, IR) O(2n)

1. T is a positive-denite kernel, in particular a Hermitian kernel, i.e.

2. T (x, y) = T (y, x).

3. T is ω2n(G)-invariant: (ω2n(g) ω2n(g)) T = T for all g G. The latter property implies, in more detail, ∈

3a. T (g x, g y) = T (x, y) for all g O(2n). ∈ 3b. If a IR, then ∈ 2n T (ax, ay) = a T (x, y). | | 3c. If b IR, then ∈ ib( x 2 y 2) e || || || || T (x, y) = T (x, y).

3d. Tˆ(x, y) = T (x, y).

By condition 3c, we obtain

Supp T = (x, y) IR2n IR2n : x = y . 2n { ∈ || || || ||} 2n 2n Let 20 n = 2n (0, 0), being a cone in IR IR . In a neighborhood of 20 n in 2n 2n \ 2 2 IR IR we take as coordinates s = x y and w 20 n. So we can write locally there || || || || ∈ T = S (w) (i)(s) i i I X∈ where I is some nite subset of IN and the Si are distributions on 20 n. Applying condition 3c again, we get that only the term with i = 0 survives, so

T (x, y) = ( x 2 y 2)R(w) || || || || outside (x, y) = (0, 0). It remains to study the distributions R on 20 n. We get by properties 3a and 3b, 2n+2 R(x, y) = x (s, s0) || || 2n 1 2n 1 where is a positive-denite distribution on S S which is O(2n)- invariant. Now we have the following theorems. Let Kl be, as before, the reproducing kernel of , the space of harmonic polynomials on IRn. Hl n 1 n 1 Theorem 6.4. Let U be a positive denite distribution on S = S S O(n)- invariant. Then U = mlKl l 0 X with m 0 and m of polynomial growth in l. l l 2n 6.4. Classication of all minimal invariant Hilbert subspaces of 0(IR ) 61 S

Proof. First we shall prove that C∞(S) has a topology also given by semi-norms 1/2 m j f 2 with m an integer 0 and the Laplacian on S. Let be  || S ||2 S H j=0 X equal to L2(S), the continuous representation of O(n) on by left translations H and the space of C∞ vectors for . We know that C∞(S) . By [5], TheoremH∞ 3.3 we even have that H∞

C∞(S) = . (6.5) H∞ Applying a result of Goodman (see [2], Theorem 1.2) and (6.5) we obtain that

1/2 m 2 j 2 C∞(S) = f L (S) : f < { ∈  || S ||2 ∞} j=0 X   and thus, by Banach’s Theorem, the topology of C ∞(S) is also given by the semi- 1/2 m norms j f 2 .  || S ||2 j=0 X By [7], Theorem Bo chner-Schwartz page 375, U = m K with m 0 and l l l l 0 X < U, Kl >= ml. Since U is O(n)-invariant

0 1 0 1 0 U(s, s0) = U(s, ks ) = U(k s, s ) = mlKl(k s, s ) l 0 X with s0 = (1, 0, . . . , 0) and k O(n). So there exists a constant C such that, ∈ m = < U( , s0), K ( , s0) > l l 1/2 m C j K ( , s0) 2  || S l ||2 j=0 X   1/2 m = C lj (l + n 2)j K ( , s0) 2  || l ||2 j=0 X   1/2 m = C(dim )1/2 l2j (l + n 2)2j Hl   j=0 X n 1   Since dim l , m has polynomial growth.¤ Hl l

Theorem 6.5. i) The extremal positive-denite ω2n(G)-invariant reproducing distributions are given by Tl(x, y) up to a positive constant. The associated irreducible unitary representations are where l = 0, 1, 2, . . . n+l l 62 Chapter 6. The oscillator representation of SL(2, IR) O(2n)

2n ii) Any ω2n(G)-invariant Hilbert subspace of 0(IR ) decomposes multiplicity S 2n free into minimal invariant Hilbert subspaces of 0(IR ). S

Proof. i) Let T be an extremal positive-denite ω2n(G)-invariant reproducing distribution. We saw before that a reproducing distribution outside of (0, 0) is equal to 2 2 2n+2 T (x, y) = ( x y ) x (s, s0) || || || || || || 2n 1 2n 1 where is a positive denite distribution on S S which is O(2n)- invariant. One has (s, s0) = mlKl(s, s0) l 0 X with ml 0, ml of polynomial growth in l by the last theorem. ml Let us now formally consider = Tl. This is a convergent series and T dl T l 0 X a tempered distribution satisfying the conditions a, b, c, d. Indeed, let S be the 2n 1 Laplacian on S . Then we have:

ϕ SP ds = Sϕ P ds 2n1 2n1 ZS ZS 2n 1 for all ϕ (S ) and P l (all l). Notice that S commutes with the ∈ D 2n∈ 1H orthogonal projection of (S ) on . Moreover D Hl P = l(l + 2n 2)P S for all P . By the computations in Section 6.2 we now get: ∈ Hl < , ϕ ϕ > c k ϕ 2 | T | || S ||2 for some constant c > 0 and some integer k 0, where

k 2 ∞ k 2 2n 1 Sϕ 2 = Sϕ(rs) r drds. || || 2n1 | | Z0 ZS

So is tempered and clearly satises a, b, c, d since all Tl do. TFrom property 3c we conclude that

( x 2 y 2)k (T ) = 0 || || || || T for k = 1, 2, . . . , hence, combining it with property 3d, we have

( )k (T ) = 0 x y T for k = 1, 2, . . ., in particular ( ) (T ) = 0. Here and denote x y T x y ∂2 ∂2 x = 2 + + 2 , ∂x1 ∂x2n 2n 6.4. Classication of all minimal invariant Hilbert subspaces of 0(IR ) 63 S and ∂2 ∂2 y = 2 + + 2 . ∂y1 ∂y2n The dierence T is easily seen to be zero since this dierence has support in (0, 0) and satises(T )(T ) = 0. So T = . So, if T is minimal, T is a x y T T positive scalar multiple of some Tl. 2n Observe that any minimal invariant Hilbert subspace of 0(IR ) occurs in the S decomposition of L2(IR2n). Moreover, given l, or , the SL(2, IR) component of Hl the irreducible representation l is determined, namely = n+l. This is the well-known Howe-correspondence. ii) As preparation for the multiplicity free decomposition of the oscillator representation, we shall show that any of the above reproducing distributions T is symmetric: T (x, y) = T (y, x). We have seen that

2 2 2n+2 T (x, y) = ( x y ) x (s, s0) || || || || || || 2n 1 2n 1 outside (x, y) = (0, 0) where is a positive-denite distribution on S S which is O(2n)-invariant. Observe that is symmetric, i.e. (s, s0) = (s0, s) since 2n 1 O(2n) acts doubly transitivily on S , hence

T (x, y) = T (y, x) on IR2n IR2n (0, 0). We easily sho\ w that T (x, y) T (y, x), having support in (0, 0) and satisfying (x y)(T (x, y) T (y, x)) = 0 as before, is actually zero. Hence T (x, y) = T (y,x), i.e. T is symmetric. The multiplicity free decomposition of the oscillator representation now easily follows from Criterion 5.6 with JT = T . ¤ Remark 6.6. The case n = 1 is treated in a similar way. The contribution for l = 0 is given by 1 id, where 1 is the limit of the analytic discrete series representation, discussed in Chapter 2. So here a non-discrete series representation occurs in the decomposition of L2(IR2). 64 Chapter 6. The oscillator representation of SL(2, IR) O(2n) CHAPTER 7 Invariant Hilbert subspaces of the oscillator representation of SL(2, IR) O(p, q)

The explicit decomposition of the oscillator representation ωp,q for the dual pair G = SL(2, IR) O(p, q) was given by B. rsted and G. Zhang in [23]. In this chapter we only study the multiplicity free decomposition of any ωp,q(G)-stable p+q Hilbert subspace of 0(IR ). The oscillator representation acts on the Hilbert space L2(IRp+q). It isS well-known that (IRp+q), the space of Schwartz functions p+q S p+q on IR is ω (G)-stable, so, by duality, ω acts on 0(IR ), the space of p,q p,q S tempered distributions on IRp+q, as well, and L2(IRp+q) can thus be considered as p+q an invariant Hilbert subspace of 0(IR ). According to Howe [16], L2(IRSp+q) decomposes multiplicity free into minimal p+q invariant Hilbert subspaces of 0(IR ). We restrict to the case p + qSeven for simplicity of the presentation of the main results. In addition we have to assume p 1, q 2, since we apply results from [9] where this condition is imposed. Our result ishowever true in general. The contents of this chapter have appeared in [39].

7.1 The denition of the oscillator representation

Let G = SL(2, IR) O(p, q) and let ω be the unitary representation of G on p,q H = L2(IRp+q) dened by:

1 ω (g)f(x) = f(g x), g O(p, q), p,q ∈ p+q pq a 0 ωp,q(g(a))f(x) = a 2 sgn 2 (a)f(ax), g(a) = 1 | | 0 a µ ¶ 1 b ω (t(b))f(x) = e ib[x,x]f(x), t(b) = p,q 0 1 µ ¶

65 66 Chapter 7. Invariant Hilbert subspaces of the oscillator representation

pq 2i[x,y] 0 1 ωp,q()f(x) = i 2 e f(y)dy, = , p+q 1 0 Z µ ¶ where [x, y] = x y + ... + x y x y ... x y for x = (x , ..., x ), y = 1 1 p p p+1 p+1 n n 1 n (y1, ..., yn) and n = p + q. This is a well-dened representation for p + q even. If p + q is odd, this is a unitary representation of SL(2, IR) O(p, q) with SL(2, IR) a double covering of SL(2, IR) [10],[29],[30],[44]. We call ωp,q the oscillatorf representation of G. Moref precisely, it is the natural extension of the restriction of the metaplectic representation of the double covering of the group Sp(n, IR) to SL(2, IR) SO(p, q), dened by Shale, Segal and Weil. We shall assume from now on that n = p+q is even. Let (IRn) be the space of n n S Schwartz functions on IR . Note that (IR ) is stable under the action of ωp,q(G), n S n so is the space 0(IR ) of tempered distributions on IR . S 7.2 Invariant Hilbert subspaces of S 0(IRn)

Given a function f (IRn), we dene its Fourier transform fˆ by ∈ S

2i[x,y] fˆ(x) = f(y)e dy. n Z n This denition naturally gives rise to a Fourier transform on 0(IR ). We denote S n Tˆ this Fourier transform of a tempered distribution T 0(IR ). Observe that n ∈ S E = 0(IR ) is a quasi-complete, barrelled, locally convex, conuclear space and S ωp,q(G) is a group of continuous automorphisms of E. So the theory of Chapter 5 n applies in its full strength. Moreover ωp,q is a representation of G on (IR ) and n S its contragradient is a representation of G on 0(IR ). The rst statement follows n S from [10]: the space (IR ) is precisely the space of C∞-vectors for the oscillator representation of theSdouble cover of Sp(n, IR) on L2(IRn). The second statement follows since (IRn) is a Montel space. S n Let H be a Hilbert subspace of 0(IR ), invariant under ωp,q(G), and let K be its reproducing kernel. By Schwartz’sS kernel theorem (see Theorem 5.9) we can associate to it a unique tempered distribution T on IRn IRn. This distribution satises the following conditions: 1. T is a positive-denite kernel, in particular a Hermitian kernel, i.e. 2. T (x, y) = T(y, x). 3. T is ωp,q(G)-invariant : (ωp,q(g) ωp,q(g)) T = T for all g G. The latter property implies, in more detail, ∈ 3a. T (g x, g y) = T (x, y) for all g O(p, q). ∈ 3b. If a IR, then ∈ n T (ax, ay) = a T (x, y). | | 3c. If b IR, then ∈ ib([x,x] [y,y]) e T (x, y) = T (x, y). n 7.2. Invariant Hilbert subspaces of 0(IR ) 67 S

3d. Tˆ(x, y) = T (x, y). A straightforward example is given by T (x, y) = (x y), the reproducing distribution of H = L2(IRn). As preparation for our main result, the multiplicity free decomposition of the oscillator representation, we shall show that any of the above distributions T is symmetric: T (x, y) = T (y, x). We shall do this in several steps. Observe that T (y, x) satises the same conditions as T (x, y). Step I By condition 3c, we obtain

Supp T = (x, y) IRn IRn : [x, x] [y, y] = 0 . n,n { ∈ } Step II n n Let n,n0 = n,n (0, 0), being an isotropic cone in IR IR . In a neighborhood n n\ of n,n0 in IR IR we take as coordinates s = [x, x] [y, y] and ω n,n0 . So we can write locally there ∈ T = S (ω) (i)(s) i i I X∈ where I is some nite subset of IN and the Si are distributions on n,n0 . Applying condition 3c again, we get that only the term with i = 0 survives, so

T (x, y) = ([x, x] [y, y]) S (ω) 0 outside (x, y) = (0, 0). It remains to study the distribution S0 on n,n0 . Step III

On the open subset of n,n0 given by [x, x] = 0, we get by properties 3a and 3b, 1 6 with = 2 (n 2), S0(x, y) = [x, x] 0(ω1, ω2) + + where 0 is an O(p, q)-invariant distribution on or , with = n X X X X X x IR [x, x] = 1 . It is known that 0 is symmetric: 0(ω1, ω2) = 0(ω2, ω1) (see{ ∈[38]),| hence } T (x, y) = T (y, x) on the open subset of IRn IRn dened by [x, x] = 0 (or, what is the same, [y, y] = 0). 6 So6 S(x, y) = T (x, y) T (y, x) has support contained in [x, x] = [y, y] = 0. Step IV n Set n = x IR [x, x] = 0 . The distribution S has support in n n. With the coordinates{ ∈ s| = [x, x],}s = [y, y] near , which can be tak en, 1 2 n n provided x = 0 and y = 0, so on 0 0 (in obvious notation), we get 6 6 n n S(x, y) = ([x, x], [y, y]) U( , ) ( , 0 ) 1 2 1 2 ∈ n where U is an O(p, q)-invariant distribution on n0 n0 , homogeneous of degree 2 + 2. Here we applied 3c again. Since S(x, y) = S(y, x) it easily follows that U( , ) = U( , ). 1 2 2 1 68 Chapter 7. Invariant Hilbert subspaces of the oscillator representation

Before we continue the preparation, we will recall now some structure theory of O(p, q) and some results of [9] about distributions on n0 . We therefore assume (as in [9]) p 1, q 2. Let P be the stabilizer in O(p, q) of the line generated by 0 = (1, 0, . . . , 0,1). P is a maximal parabolic subgroup and has Langlands decomposition P = MAN. The subgroup M consists of matrices of the form

1 0 0 0 m 0 0 0 1   where m is a matrix of the group O(p 1, q 1). The group A is the one-parameter subgroup of matrices 1 1 1 (u + ) 0 1 (u ) 2 u 2 u au =  0 1 0  1 1 1 1  (u ) 0 (u + )  2 u 2 u    where u IR. The subgroup∈ N consists of matrices of the form

1 1 1 Q() Q() 2 2 n = 1  1 1  Q() 1 + Q() 2 2   n 2 where IR . If = (2, . . . , n 1), then ∈ t = Ip 1,q 1, with Ip 1 0 Ip 1,q 1 = 0 Iq 1 µ ¶ 2 2 2 2 and Q() = 2 + + p p+1 p+q 1. n 2 The group N is isomorphic to IR . Moreover:

1 au n au = nu. Step V

Since O(p, q) acts transitively on n0 , we can conclude that to U corresponds a MN-invariant distribution on n0 . Call it V . We recall some results of [9] about such distributions, see ([9], III). 1 Dene u() = ( ) on 0 and set for t IR, = u() = t . 2 1 n n ∈ t { | } For f (0 ), the space of C∞-functions on 0 , one can dene the function ∈ D n c n f on IR by M f(t) = f() (u() t) . M Z n 7.2. Invariant Hilbert subspaces of 0(IR ) 69 S

maps (0 ) continuously onto some topological vector space of functions M D n F on IR with singularities at t = 0 (see also [30]). And if W is a continuous linear form on , the distribution V dened on 0 by F n

V (f) = W ( f) (f (0 )), M ∈ D n i.e., V = 0W , is invariant under MN. 0 is not surjective. One has ([9], M M Proposition III.1): if V is a distribution on n0 invariant under MN, then

V = 0W + V M 1 where W is a continuous linear form on and V is a distribution on 0 , MN- F 1 n invariant and with support contained in 0. The structure of the distributions V1 is as follows. We use the local chart in a neighborhood of given by the map from A N to 0 , 0 n (a , n ) a n 0, u → u where N consists of the matrices

1 1 1 2 Q() 2 Q() n 2 n = 1 , IR .  1 1  ∈ 2 Q() 1 + 2 Q()   We denote by the dierential operator in this chart given by

∂2 ∂2 ∂2 ∂2 = 2 + + 2 2 2 . ∂2 ∂p ∂p+1 ∂p+q 1 We quote [9], Theoreme III.2 here:

Let V1 be a MN-invariant distribution on n0 with support in 0. Then there n exist a distribution T0 on IR and constants Ak, Bk, k = 1, 2, . . . , m such that, in the above chart,

m du V = T + (A + B sgn(u)) u +k k 1 0 k k | | u kX=1 | | where is the Dirac measure at = 0. Step VI We return to our distribution V from Step V, and write it in the form V = 0W + V , as above. By abuse of notation we have M 1 V () = V (g0) = U(g0, 0).

0 1 0 0 0 1 0 Because u() = u(g ) = 2 [g , ] satises u(g ) = u(g ), we easily get that

0 1 0 0W (g ) = 0W (g ). M M 70 Chapter 7. Invariant Hilbert subspaces of the oscillator representation

0 1 0 0 0 1 0 Since V (g ) = V (g ), we see that 2V (g ) = V1(g ) V1(g ). Now 1 0 0 0 V (au0 gau0 ) = U(gau0 , au0 ) = 0 0 2+2 0 U(u g , u ) = u V (g ). 0 0 | 0| 0 0 1 0 Let us apply this property to the distribution V2(g ) = V1(g ) V1(g ), 0 supported by 0. We get with g = aun , that

1 1 au0 aunau0 = auau0 nau0 = aunu0, hence 0 n+2 V (a n ) = u T () 2 u u0 | 0| 0 m n+2 2k +k du k + u (A + B sgn (u)) u . | 0| k k | | u kX=1 | | 0 n+4 0 Since V (a n ) = u V (a n ), we get V = 0. So V = 0, and hence 2 u u0 | 0| 2 u 2 S = 0 in a neighborhood of n0 n0 , and therefore Supp S (0) n n (0) . Step VII { }∪{ } From property 3c we conclude that

([x, x] [y, y])k S = 0 for k = 1, 2, . . . , hence, combining it with property 3d, we have

k S = 0 for k = 1, 2, . . ., in particular S = 0. Here denotes the d’Alembertian

∂2 ∂2 ∂2 ∂2 ∂2 ∂2 ∂2 ∂2 ( 2 + + 2 2 2 ) ( 2 + + 2 2 2 ). ∂x1 ∂xp ∂xp+1 ∂xn ∂y1 ∂yp ∂yp+1 ∂yn

If S has support in (0) n n (0) , and S = 0, it easily follows, by recalling the local structure{ of} suc∪ {h distributions, } being basically a nite linear combination of tensor products of distributions supported by n and the origin, that S = 0 (see [28],Ch.III, 10). So nally we have shown that T (x, y) = T (y, x). § 7.3 Multiplicity free decomposition of the oscillator representation

The multiplicity free decomposition of the oscillator representation, i.e. the multiplicity free decomposition into irreducible invariant Hilbert subspaces of any n ωp,q(G)-invariant Hilbert subspace of 0(IR ), is now easily proved by applying S n Criterion 5.6 with JT = T. If H is any invariant Hilbert subspace of 0(IR ) with n n S reproducing kernel T 0(IR IR ), then T is the kernel of the space J H and the property J H = H comes∈ S down to T(x, y) = T (x, y). Since T is positive-denite, this is equivalent with T (x, y) = T (y, x) which immediately follows from Section 7.2. So we have the following result. 7.3. Multiplicity free decomposition of the oscillator representation 71

n Theorem 7.1. Any ωp,q(G)-invariant Hilbert subspace of 0(IR ) decomposes mul- S n tiplicity free into irreducible invariant Hilbert subspaces of 0(IR ). S 2 n n Theorem 7.1 is well-known for H = L (IR ) 0(IR ), see [16]. We have shown S n that the result extends to any ω (G)-invariant Hilbert subspace of 0(IR ). p,q S 72 Chapter 7. Invariant Hilbert subspaces of the oscillator representation CHAPTER 8

Decomposition of the oscillator representation of SL(2, C) SO(n, C)

In this chapter we determine the explicit decomposition of the oscillator representation for the groups SL(2, C) O(1, C) and SL(2, C) SO(n, C) with n 2. The main tool we use is a Fourier integral operator introduced by M. Kashiwara and M. Vergne for real matrix groups, see [18]. For the group SL(2, C) SO(n, C) with n 2, given that SO(2, C) is an abelian group, we split theproof into the cases n = 2 and n 3. The computation of the Plancherel measure for SO(n, C)/SO(n 1, C) is required in order to compute the explicit decomposition for the case n 3.

8.1 The case n = 1

8.1.1 The denition of the oscillator representation

Let G be the group SL(2, C) O(1, C) and let ω1 be the unitary representation of G on H = L2(C) dened by:

1 ω (g)f(z) = f(g z), g O(1, C) 1 ∈ a 0 ω1(g(a))f(z) = a f(az), g(a) = 1 a C | | 0 a ∈ µ ¶ iRe (bz2) 1 b ω (t(b))f(z) = e f(z), t(b) = b C 1 0 1 ∈ µ ¶ 2iRe (zw) 0 1 ω1()f(z) = e f(w)dw, = 1 0 Z µ ¶

73 74 Chapter 8. Decomposition of the oscillator representation where the Fourier transform is dened by

2iRe (zw)

fˆ(z) = e f(w)dw.

We call ω1 the oscillator representation of G. More precisely, it is the natural extension of the restriction of the metaplectic representation of the group Sp(1, C) Sp(2, IR) to SL(2, C) SO(1, C). Let (C) be the space of Schwartz functions on C. Note that (C) is stable S S under the action of ω , so is the space 0(C) of tempered distributions on C. 1 S 8.1.2 Some minimal invariant Hilbert subspaces of S 0(C) 2 We consider L (C) as a Hilbert subspace of 0(C). This space can be decom- 2 2 2 2 S 2 posed as L (C) = L (C)+ L (C) with L (C)+ = f L (C) : f is even and L2(C) = f L2(C) : f isodd . Irreducible unitary{represen∈ tations of G are} of the form { ∈ where is an irreducible} representation of SL(2, C) and one of O(1, C). We will now construct intertwining operators from (C) to the Hilbert space of such representations. Not all combinations (, ) occur.S Here are some of these operators. Let s be the irreducible representation of O(1, C) dened by,

s s(g) = g with s = 0, 1 and g O(1, C). { } ∈ Let Vm,0 be the space of functions on C satisfying,

f(z) f(w)

| || |dzdw <

z w 2 m ∞ Z Z | | with 0 < m < 2, m IR. The group SL(2, C) acts irreducibly on this space by ∈

m 2 az + b (g)f(z) = cz + d f m,0 | | cz + d µ ¶

1 a b with g = , g SL(2, C) and 0 < m < 2. ( belongs to the so-called c d ∈ m,0 complementaryµ series,¶ see Chapter 3) Let us now dene an operator 1,0 from (C)+ into the Hilbert space of V1,0 as follows: F S iRe (zy2) 2 ( 1,0ϕ)(z) = e y ϕ(y)dy F | | Z This integral exists for all z C. The group SL(2, C) acts ∈irreducibly on L2(C) by

l 2 cz + d az + b (g)f(z) = cz + d f 0,l | | cz + d cz + d µ| |¶ µ ¶ 8.1. The case n = 1 75

1 a b with g = , g SL(2, C). ( belongs to the so-called principal series, c d ∈ 0,l see Chapter µ3) ¶ 2 Let us now dene an operator 0,1 from (C) into the Hilbert space of L (C) as follows: F S iRe (zy2) ( 0,1ϕ)(z) = e yϕ(y)dy F Z This integral exists for all z C. We shall show: ∈

1) The operators 1,0, 0,1 are well dened, i.e. 1,0ϕ V1,0 and 0,1ϕ L2(C). F F F ∈ F ∈

2 2) 1,0, 0,1 are continuous as operators from (C)+, (C) to V1,0, L (C) FrespectivF ely. S S

3) The operator 1,0 intertwines the action of ω1 and 1,0 id. 0,1 intertwines the action of ωF and . F 1 0,1 1 So we have:

Theorem 8.1. The operator 1,0 on (C)+ into V1,0 intertwines the action ω1 with F S 2 1,0 id of SL(2, C) O(1, C). 0,1 on (C) into L (C) intertwines the action ω with F S 1 0,1 1 Proof. Let us assume 1) and 2), we show 3). It is clear that 1,0, 0,1 intertwine the O(1, C)-action. The intertwining relations for the elemenF ts Fg(a) and t(b) are easy to check. We now check the - intertwining relation:

ω ()ϕ(z) = ()( ϕ)(z) F1,0 1 1,0 F1,0 for ϕ (C) . By denition, ∈ S + iRe (zy2) 2 1,0ω1()ϕ(z) = e y ϕˆ( y)dy. F | | Z Applying Proposition 6-1 from [26] we obtain,

3 iRe (y2/z) 2 1,0ω1()ϕ(z) = z e y ϕ(y)dy F | | | | Z = 1,0()ϕ(z). Doing the same for we have, F0,1

iRe (zy2) 0,1ω1()ϕ(z) = e yϕˆ( y)dy F Z 1 1 iRe (y2/z) = z z e yϕ(y)dy | | Z = 0,1()ϕ(z). 76 Chapter 8. Decomposition of the oscillator representation

Then we have proved that 1,0 intertwines ω1 and 1,0 id and 0,1 intertwines ω and . ¤ F F 1 0,1 1 2 2 Now we prove 1) and 2). Let us compute 1,0ϕ for ϕ (C)+ and 0,1ϕ for ϕ (C) . First we need the following ||Flemma|| ∈ S ||F || ∈ S Lemma 8.2. Let ϕ be an even function in (C) and a continuous bounded funtion S 1 dz

(z2)ϕ(z)dz = (z)ϕ(√z)

2 z Z Z | | Proof. As ϕ is an even function

(z2)ϕ(z)dz = 2 (z2)ϕ(z)dz Re z 0 Z Z Doing the change of variables z2 = w we obtain

1 dw = 2 (w)ϕ(√w) 4 w Z | | and we have the result. ¤

Now we compute ϕ 2 for ϕ (C) ||F1,0 || ∈ S +

2 ( 1,0ϕ)(z)( 1,0ϕ)(w) 1,0ϕ = F F dzdw ||F || 2 z w Z | | ( ϕ)(z + w)( ϕ)(w) = F1,0 F1,0 dzdw 2 z Z | | Applying the last lemma

iRe ((z+w)y wx) 1 e = ϕ(√y)ϕ(√x)dxdydzdw 4 4 z Z | | iRe (zy) 1 e iRe ((y x)w)

= ϕ(√y) e ϕ(√x)dx dydzdw

4 3 z Z | | µZ ¶ eiRe (zy) = ϕ(√y)ϕ(√y)dzdy 2 z Z | | Writing z = rei, y = sei with r, s 0, , [0, 2]. We then get, ∈

2 2 ∞ ∞ = eirs cos(+)ϕ(√sei)ϕ(√sei)sdddrds 0 0 0 0 Z Z 2 Z Z 2 ∞ ∞ = ϕ(√sei)ϕ(√sei)s eirs cos(+)d dr dds Z0 Z0 µZ0 µZ0 ¶ ¶ 8.1. The case n = 1 77

Using the following properties of Bessel functions

2 eirs cos(+)d = 2 (rs) J0 Z0 and ∞ (r)dr = 1 J0 Z0 we have

2 ∞ ∞ ϕ 2 = 2 ϕ(√sei)ϕ(√sei)s (rs)dr dds ||F1,0 || J0 Z0 Z0 µZ0 ¶ 2 ∞ = 2 ϕ(√sei)ϕ(√sei)dds Z0 Z0 dz = 2 ϕ(√z)ϕ(√z) z Z | |

= 4 ϕ(z)ϕ(z)dz

Z = 4 ϕ 2. || || 2 Now we compute 0,1ϕ for ϕ (C) ||F || ∈ S

2 0,1ϕ = ( 0,1ϕ)(z)( 0,1ϕ)(z)dz ||F || F F Z iRe (zy2 zx2) = e yxϕ(y)ϕ(x)dxdydz 3 Z 1 √yϕ(√y) iRe ((y x)z) √xϕ(√x)

= e dx dydz

4 2 y x Z | | ÃZ | | ! ϕ(√y)ϕ(√y) = dy y Z | |

= 2 ϕ(y)ϕ(y)dy

Z = 2 ϕ 2. || ||2

This proves 1). Moreover we have 2) since actually 1,0, 0,1 are continuous in the L2-topology on (C), so on (C) itself, since the emF beddingF of (C) into L2(C) is continuous. S S S 2 2 2 By the Parseval’s theorem as L (C) = L (C)+ L (C) ,

2 2 2 1 2 1 2 ϕ = ϕ+ + ϕ = 1,0ϕ+ + 0,1ϕ . || || || || || || 4||F || 2||F || 78 Chapter 8. Decomposition of the oscillator representation

8.1.3 Decomposition of L2(C) Clearly the subsection 8.1.2 also gives the decomposition of L2(C) into a minimal invariant suspaces. The decomposition is multiplicity free (see next chapter) and the irreducible unitary representations which occur are given by 1,0 id and . In conclusion we have, 0,1 1 Corollary 8.3. Let ϕ be a function in (C). We have the following Plancherel formula S 2 2 2 1 2 1 2 ϕ 2 = ϕ+ + ϕ = 1,0ϕ+ + 0,1ϕ . || || || || || || 4||F || 2||F || As representations of SL(2, C) O(1, C), we have L2(C) = id + . 1,0 0,1 1 8.2 The case n = 2 8.2.1 The denition of the oscillator representation Let G be the group SL(2, C) SO(2, C) and let ω be the unitary representation 2 of G on H = L2(C2) dened by:

1 ω (g)f(z) = f(g z), g SO(2, C) 2 ∈ 2 a 0 ω2(g(a))f(z) = a f(az), g(a) = 1 a C | | 0 a ∈ µ ¶ iRe (b[z,z]) 1 b ω (t(b))f(z) = e f(z), t(b) = b C 2 0 1 ∈ µ ¶ 2iRe ([z,w]) 0 1 ω2()f(z) = e f(w)dw, = 2 1 0 Z µ ¶ where [z, w] = z1w1 + z2w2 if z = (z1, z2), w = (w1, w2). The Fourier transform is dened by 2iRe ([z,w]) fˆ(z) = e f(w)dw. 2 Z We call ω2 the oscillator representation of G. More precisely, it is the restriction of the metaplectic representation of the group Sp(2, C) Sp(4, IR) to G. Let (C2) be the space of Schwartz functions on C2. Note that (C2) is stable S 2 S 2 under the action of ω , so is the space 0(C ) of tempered distributions on C . 2 S 8.2.2 Some minimal invariant Hilbert subspaces of S 0(C2)

2 2 2 We consider L (C ) as a Hilbert subspace of 0(C ). Irreducible unitary representations of G are of the form where S is an irreducible representation of SL(2, C) and one of SO(2, C). We will now construct intertwining operators from (C2) to the Hilbert space of such representations. Not all combinations (, )Soccur. Some of these operators are given below. 8.2. The case n = 2 79

Let is, be the irreducible unitary representation of SO(2, C) dened by,

is, is,(g) = (a + ib)

a b with s IR, , g SO(2, C), g = . We set zis, = z is z for ∈ ∈ ∈ b a | | z µ ¶ | | z C. ³ ´ ∈The group SL(2, C) acts irreducibly on L2(C) by

m 2 cz + d az + b (g)f(z) = cz + d f ,m | | cz + d cz + d µ| |¶ µ ¶

1 a b with g = , g SL(2, C), iIR, m . ( belongs to the so-called c d ∈ ∈ ∈ ,m principal series,µ see¶Chapter 3) 2 2 Let us now dene an operator is, from (C ) into the Hilbert space L (C) as follows: F S

iRe (z(y2+y2)) is, ( is,ϕ)(z) = e 1 2 (y1 + iy2) ϕ(y1, y2)dy1dy2 F 2 Z with s IR and . This integral exists for all z C. We ∈will show that:∈ ∈ 1) The operator is well dened, i.e. ϕ L2(C), Fis, Fis, ∈ 2) is continuous as operator from (C2) to L2(C). Fis, S 3) The operator intertwines the action of ω and . Fis, 2 is, is, First we want to prove 1) and 2). We have:

2 Proposition 8.4. For all s IR and the operator is, maps (C ) into L2(C) and is continuous.∈ ∈ F S Fis, 2 2 Proof. Observe that w1 + w2 = 1 can be parametrized by the two real parameters t IR, IR, 0 < 2 as follows: any g SO(2, C) can be uniquely written as ∈ ∈ ∈

g = htk

cosh t i sinh t cos sin where h = , k = . t i sinh t cosh t sin cos µ ¶ µ ¶ 1 Hence w2 + w2 = 1 is precisely h k x0 t IR, 0 < 2 where x0 = . 1 2 { t | ∈ } 0 2 2 µ ¶ Then dtd is both an invariant measure on SO(2, C) and on w1 +w2 = 1. More precisely, if dg is the thus dened invariant measure on G = SO(2, C), then

2 ∞ f(g)dg = f(htk)ddt (f c(G)). G 0 ∈ C Z Z∞ Z 80 Chapter 8. Decomposition of the oscillator representation

Then it follows:

2 ∞ 2

f(z)dz = f(wt,) dtdd (8.1)

2 0 | | Z Z Z Z∞ 0 where wt, = htkx . Let ϕ (C2). Applying (8.1) we have ∈ S 2 2 1 ∞ 2

ϕ(z1, z2) dz = ϕ(√wt,) ddtd

2 | | 4 0 | | Z Z Z∞ Z Applying (8.1) to the intertwining operator we obtain

iRe (z(y2+y2)) is, is,ϕ(z) = e 1 2 (y1 + iy2) ϕ(y1, y2)dy1dy2 F 2 Z

= eiRe (z)F (, is, )d (8.2)

Z 2 1 is/2,/2 ∞ is, i where F (, is, ) = (cosh t + sinh t) e ϕ(√wt,)ddt. 4 0 Since ϕ is a Schwartz classZfunction∞ Z

2 1 ∞ F (, is, ) ϕ(√wt,) dtd | | 4 0 | | Z Z∞ ∞ 2t N C (1 + e ) dt | | Z0 ∞ N 1 C (1 + u) u du Z| | N C(1 + ) 2 ln | | | | by an easy calculation with N > 4. Thus F (, is, ) as a function of is in L2(C) L1(C) and therefore (8.2) is everywhere dened and ϕ is in L2(C). ∩ Fis, We can see that the value of the constant C is equal to C 0 ϕ where || ||,0 2N | | 2 X ϕ ,0 = sup ( z ϕ(z) ) are norms in (C ) and C0 is a constant. Then we have || || 2 | | S z C ∈

ϕ C00 ϕ ||Fis, || || ||,0 2N | X| 2 for all ϕ (C ) where C00 is another constant. Therefore we have proved that is con∈ tinS uous. ¤ Fis, 2 2 Theorem 8.5. The operator is, from (C ) into L (C) intertwines the action of ω with of SL(2,FC) SO(2S, C). 2 is, is, 8.2. The case n = 2 81

Proof. First we prove the intertwining relation for SO(2, C). Let g be an element a b of this group g = , then b a µ ¶

iRe (z(y2+y2)) is, 1 is,ω2(g)ϕ(z) = e 1 2 (y1 + iy2) ϕ(g (y1, y2))dy1dy2. F 2 Z 1 Making the change of variables g y = w we obtain is, iRe (z(w2+w2)) is, = (a + ib) e 1 2 (w1 + iw2) ϕ(w1, w2)dw1dw2 2 Z = (g) ϕ(z) is, Fis, The intertwining relations for the elements g(a) and t(b) are easy to check. We now check the -intertwining relation: ω ()ϕ(z) = ()( ϕ)(z) Fis, 2 is, Fs, for ϕ (C2). By denition, ∈ S iRe (z(y2+y2)) is, is,ω2()ϕ(z) = e 1 2 (y1 + iy2) ϕˆ( y1, y2)dy. F 2 Z Here we insert the following lemma: Lemma 8.6. For every function ϕ (C2) one has ∈ S iRe (z(y2+y2)) is, e 1 2 (y1 + iy2) ϕˆ(y1, y2)dy1dy2 2 Z is, 2 iRe ((y2+y2)/z) is, = z z e 1 2 (y1 + iy2) ϕ(y1, y2)dy1dy2 | | 2 Z with s IR, and the Fourier transform is dened by ∈ ∈ 2iRe ([z,w]) fˆ(z) = e f(w)dw. 2 Z Proof. We consider the auxiliary function

2 2 1 y1+iy2 2 y1 iy2 1,2 (y1, y2) = e | | | | with 1, 2 > 0. By denition of the Fourier transform and the dominated convergence theorem

iRe (z(y2+y2)) is, e 1 2 (y1 + iy2) ϕˆ(y1, y2)dy 2 Z 2 2 iRe (z(y +y )) is, 2iRe (y1w1+y2w2)

= e 1 2 (y1 + iy2) e ϕ(w1, w2)dw dy

2 2 Z µZ ¶ 2 2 iRe (z(y1 +y2 )) is, = lim lim e (y1 + iy2) 1,2 (y1, y2) 2 0 1 0 2 → → Z 2iRe (y1w1+y2w2) e ϕ(w1, w2)dw dy 2 µZ ¶ 82 Chapter 8. Decomposition of the oscillator representation

Using Fubini’s theorem we get that the integral above is equal to

2 2 iRe (z(y1 +y2 ) 2(y1w1+y2w2)) is,

ϕ(w1, w2) e (y1 + iy2) 1,2 (y1, y2)dy dw

2 2 Z µZ ¶ Let us perform the change of variables y = Su, w = Sv, 1 1 where S = 1 . The above integral then equals 2 i i µ ¶

1 iRe ((zu u ) (u v +u v )) is, u 2 u 2

ϕ(Sv) e 1 2 1 2 2 1 u e 1| 1| 2| 2| du dv (8.3)

2 1 4 2 2 Z µZ ¶ The inner integral becomes

2 2 is, iRe (u1v2) 1 u1 iRe ((zu1 v1)u2) 2 u2

u e e | | e e | | du2 du1

Z µZ 2 2 ¶ 2 |zu1v1| is, iRe (u1v2) 1 u1 4 = u e e | | e 2 du1. 1 2 Z Taking the limit 1 0, by the dominated convergence theorem, we see that (8.3) converges to →

2 2 |zu1v1| is, iRe (u1v2) 4

ϕ(Sv) u e e 2 du1 dv

2 1 4 2 2 Z µZ ¶ 2 u v 2 1 | 1 1| u1v2 2 4 iRe ( ) 1 is,

= z e 2 ϕ(Sv)e z (z u1) dv2 du1 dv1

4| | 4 Z · 2 Z µZ ¶ ¸ 1 Now the term in the inner parenthesis, as a function of v1 is in L (C) and taking the limit 0 we know that the term in the outer parenthesis tends to this term: 2 → iRe ( v1v2 ) 1 is,

ϕ(Sv)e z (z v1) dv2

Z 1 in the space L (C, dv1). Thus the integral becomes

1 2 iRe ( v1v2 ) 1 is, z ϕ(Sv)e z (z v1) dv 4| | 2 Z is, 2 iRe ((y2+y2)/z) is, = z z e 1 2 (y1 + iy2) ϕ(y1, y2)dy | | 2 Z We have thus proved the lemma. ¤ Applying this lemma it follows

is, 2 1 ω ()ϕ(z) = ( 1) z z ϕ( ) Fis, 2 | | Fis, z = () ϕ(z) is, Fis, and Theorem 8.5 is proven. ¤ 8.3. The case n 3 83

8.2.3 Decomposition of L2(C2) Let us compute ϕ 2. Making use of Plancherel’s theorem on the abelian group SO(2, C) and (8.2)|| it|| follows

2 2 1 ∞ 2

ϕ(z1, z2) dz = ϕ(√wt,) ddtd

2 | | 4 0 | | Z Z Z∞ Z 4

= F (, is, ) 2ds d

2 | | Ã ! Z X∈ Z 4

= F (, is, ) 2dds

2 | | X∈ Z Z 1 2 = is,ϕ ds

2 ||F || X∈ Z The above result also gives the decomposition of L2(C2) into minimal invariant subspaces. The decomposition is multiplicity free and the irreducible unitary representations which occur are given by . In conclusion, we have is, is, Corollary 8.7. Let ϕ be a function in (C2). We have the following Plancherel formula S 2 1 2 ϕ 2 = is,ϕ ds. 2 || || ||F || X∈ Z As representations of SL(2, C) SO(2, C), we have

2 2 L (C ) = is, is,ds. X∈ Z 8.3 The case n 3

To determine the explicit decomposition of the oscillator representation for the dual pair SL(2, C) SO(n, C) with n 3 we compute rst the Plancherel formula for SO(n, C)/SO(n 1, C), n 3. 8.3.1 Plancherel formula for SO(n, C)/SO(n 1, C), n 3 8.3.1.1 The symmetric spaces SO(n, C)/SO(n 1, C) for n 3 Let G be the group SO(n, C). G acts on Cn in the usual manner. Let [, ] be the G-invariant bilinear form on Cn, given by,

H = SO(n 1, C). The space X is a complex manifold, homeomorphic to G/H. Set 1 0 0 0 1 0 J =  . . .  . . . ..    0 0 1   J is not in G, but J is as soon as n is odd. Dene the involutive automorphism of G by (g) = JgJ. Then H is of index 2 in the stabilizer H of . So G/H is a complex symmetric space. Observe that is an inner automorphism if n is odd. We will pass now to the Lie algebra g of G. Let g = h + q be the decomposition of g into eigenspaces for the eigenvalues +1 and 1 for . Then we have: 0 0 . h = . Y  : Y complex, antisymmetric ,  0        0 z z 2 n z2 q = .  : z2, . . . , zn C . . ∈  . —0      zn      0 1   Write L = 1 0 q. This element actually belongs to q k where  —0  ∈ ∩ k = so(n, IR), the real antisymmetric matrices. The Lie algebra k belongs to K = SO(n, IR). Clearly g = k + ik. Call p = ik. Then iL q p. The Lie algebra a = CL is a maximal abelian subspace of∈q; a∩is a Cartan subspace of q with respect to . Let m be the centralizer of a in h. Then

0 0 0 0 0 0 m = . .  : Y so(n 2, C) . . . ∈  . . Y     0 0      Let g be the eigenspaces of ad(iL) for the eigenvalues 1. Then 0 0 tp t n 2 g = X(p) = 0 0 i p : p C   p ip 0  ∈      and g = (g+). We have the following decomposition of g into eigenspaces of ad(iL):

g = g m a g+. 8.3. The case n 3 85

Set n = g . Then n is a nilpotent subalgebra of g. The connected subgroup A and N of G corresp onding to the Lie subalgebras a and n are given by:

cos z sin z 0 A = a = exp zL = sin z cos z 0 : z C  z   ∈  0 0 In 2     and   1 1 t 1 + 2 [p, p] 2 i[p, p] i p 1 1 t n 2 N = n(p) = i[p, p] 1 [p, p] p : p C .   2 2  ∈  ip p In 2   0   0 Let be the vector (1, i, 0, . . . , 0). Dene P0 : X C by P0(x) = [x, ].  0 iz →  Observe that P (na hx ) = e for n N, a A, h H. 0 z ∈ z ∈ ∈ Proposition 8.8. (i) The set NAH is open in G and dense. One has: g NAH if and only if P (gx0) = 0. ∈ 0 6 (ii) The map (n, a, h) nah is a submersion and an immersion. → (iii) nah = n0a0h0 if and only if n = n0, a = a0, h = h0.

8.3.1.2 The cone = G/MN and the Poisson kernel The centralizer M of a in H consists of the elements of G of the form: 1 0 0 0 0 1 0 0   0 0 , with h SO(n 2, C). . . ∈ . .  . . h    0 0    n Furthermore ZG(a) = MA, M A = e . Put = z C : [z, z] = 0, z = 0 , the isotropic cone. The group G acts∩ transitiv{ } ely on {and∈Stab(0) = MN,6 so } can be identied with G/MN. The stabilizer of 0 : C is equal to P = MAN, a parabolic subgroup of G. ∈ Let K = SO(n, IR) as before, a maximal©compact subgroupª of G. Then G = KP , K P = (K M) exp(IRL). So G 0 = K exp(iIRL) 0. Dene for x X and ∩, ∩ ∈ ∈ P (x, ) = [x, ]. P is called the Poisson kernel and assumes all complex values. The following properties are immediate: (i) P (gx, g) = P (x, ) for g G, ∈ 0 (ii) P (x, ) = P0(x).

(iii) Dene P : C by P (, 0) = [, 0]. Then one has: → 0 t 0 P (g , ) = 2 lim e P (ga itx , ) (g G). t →∞ ∈ 86 Chapter 8. Decomposition of the oscillator representation

8.3.1.3 The representations δ,s

Write A = A AI with A = exp(IRL), AI = exp(iIRL). The subgroup P = MAN is a maximal parabolic subgroup of G with Langlands decomposition

P = (MA )AI N.

Observe that A is compact. Dene ix sy ,s(maxaiyn) = e e with , s C a representation of P and dene ∈ ∈

,s = ind ,s. P G ↑

The space of ,s is called E,s and consists of complex-valued ∞-functions f on G satisfying C ix (s )y f(gma n) = e f(g) (g G) x+iy ∈ where = n 2. The representation ,s is then given by

1 (g)f(x) = f(g x) (x, g G; f E ). ,s ∈ ∈ ,s The following lemma is standard. Lemma 8.9. Let f (G). Then, with suitable normalization of Haar measures, ∈ D

2Im z f(g)dg = f(kmazn)e dkdmdzdn. ZG ZK ZM ZA ZN It follows that the non-degenerate sesquilinear form <, > on E,s E, s dened by < f, h >= f(k)h(k)dk = f(b)h(b)db K B=K/(M K) Z Z ∩ is G-invariant. Hence is pre-unitary for s iIR. ,s ∈ 8.3.1.4 Realization on the cone

Since the functions in E,s are right MN-invariant they can be viewed as functions 0 0 iz 0 on : to f E,s we associate f() = f(g) if = g . Since az = e , we easily get the∈ precise properties of the functions on :

s f() = f() ( , C). (8.4) | | ∈ ∈ µ| |¶ 1 G acts on E,s() by ,s(g)f() = f(g ). According to Theorem 8.3 in [36], (s iIR) is irreducible and pre-unitary for s = 0. ,s ∈ 6 We will not make a more detailed study of the representations ,s concerning irreducibility, equivalence, unitarity etc. 8.3. The case n 3 87

The cases n = 3, 4 are well-known through the isomorphism

SO(3, C) SL(2, C)/ 1 and SO(4, C) SL(2, C) SL(2, C). ' { } ' But the cases n > 4 should be treated, and we should consider SO(3, C) and SO(4, C) as part of these cases. A major point in the analysis of the representations ,s would be the description of their K-types. We will give it only for reference here.

8.3.1.5 The K-types of δ,s

Clearly any function in E,s is completely determined by its restriction to K. Actually we get ∞(K), the space of ∞-functions on K satisfying: C C ix - f(kax) = e f(k) - f(km) = f(k) if m M K = SO(n 2, IR). ∈ ∩ 0 If we set B = K , then B K/(M K). So, E,s ∞(B), the space of ' ∩ ' C ∞-functions on B satisfying f(b) = f(b) ( C, = 1). As a representation C ∈ | | of K (so restricting ,s to K), this space is just the space of

ix ind e .

(K M)A K ∩ ↑ Clearly 2 2 L (B) = L(B) (orthogonal direct sum). M∈ 2 It is well-known that L(B) splits multiplicity free into irreducible representations of K. We need a detailed description of the K-types. We refer therefore to [31]. We introduce the following notation. Let j(z) be the space of harmonic polynomials on Cn, homogeneous of degree j; harmonicH meaning:

∂2 ∂2 ( 2 + + 2 )p(z) = 0 if p j (z) ∂z1 ∂zn ∈ H n with p(z) = p(z1, . . . , zn). So j(z) is the complexication of j(IR ), the harmonic polynomials on IRn, homogeneousH of degree j. These polynomialsH are com- n 1 n 2 2 pletely determined by their values on S = x IR : x + + x = 1 . The { ∈ 1 n } group K acts irreducibly on j, as well as G. So we can speak of highest and lowest weight vectors (for the HG-action). Clearly p(z) = [z, 0]j is a highest weight jt vector with weight e . We also consider the spaces (z) (complex conjugates of elements in (z)). Hj Hj The K representation is “the same”. Set m1,m2 = m1 (z) m2 (z). The space B can be seen as the variety [z, z] = 0,H [z, z] =H2 in Cn. HLet 0 as before and 1 = (0, 0, 1, i, 0, , 0) is n 4. According to [31] we have the following. The irreducible represen tations ofSO(n), which occur in the decomposition of L2(SO(n)/SO(n 2)) are of the form R with m = (m , m , 0, . . . , 0) highest m1,m2 1 2 88 Chapter 8. Decomposition of the oscillator representation

weight, m1 m2 0 (n > 4). The representationR occurs m m + 1 times and has a unique represen- m1,m2 1 2 tative in l1+m2,l2+m2 with l1 + l2 = m1 m2, l1 0, l2 0. The highest weight in these spacesH is: m [z, 0]l1 [z, 0]l2 [z, 0][z, 1] [z, 1][z, 0] 2 Clearly, if l l = , we are in L¡2(B); this occurs once. So¢we get for n > 4: 1 2 Proposition 8.10. If n > 4, then

L2(B) = R if 0 2l++m,m l 0,m 0 M = R2l +m,m if < 0. l 0,m 0 M The highest weight vectors are: m Y (z) = [z, 0]l+[z, 0]l [z, 0][z, 1] [z, 1][z, 0] ( 0) l,m and ¡ ¢ 0 l 0 l 0 1 1 0 m Y (z) = [z, ] [z, ] [z, ][z, ] [z, ][z, ] ( < 0) l,m respectively. ¡ ¢ 1 Remark 8.11. For n = 3 we have to replace by e3 = (0, 0, 1) and only take 1 m2 = 0 or 1; for n = 4 we have to take in addition the vectors where is replaced 1 by . We refer to [31], for more details. Let us denote the representations R2l+ +m,m just by l,m; observe that l,m | | ' , the identication is given by the map ϕ(z) ϕ(z). l,m → 8.3.1.6 Casimir operators Let (X) be the algebra of invariant dierential operators on X. Observe that the (real) rank of X = G/H is equal 2. So (X) has two generators. We shall present now these generators. Both belong to the center of U(g). We shall also study the eect of their action on E,s. The Lie algebra g is complex with Killing form F (X, Y ) = (n 2) tr XY . This form F is non-degenerate, Ad(G)-invariant and complex bilinear. This gives rise, by well-known techniques, to a holomorphic and an anti-holomorphic invariant dierential operator; call them 1 and 2. Then = 1 + 2 is the Casimir operator. According to [20] we have: Proposition 8.12. The holomorphic invariant operator is equal to the following

n 2 m 1 = (iL)2 (iL) + 4 (N )N + Z2 1 2(n 2) j j j j j j=1 j=1 ¡ ¢ X X where Y g corresponds to (Y f)(g) = ∂ f(g exp zY ) , f holomorphic. For ∈ ∂z z=0 notation see [20]. ¯ ¯ 8.3. The case n 3 89

Let f E . Then ∈ ,s 1 ∂2 ∂ ( f)(g) = f(ga ) 1 2(n 2) ∂z2 ∂z iz z=0 µ ¶ 2 ¯ 1 ∂ ∂ (s )x¯ iy = e e f(g) 2(n 2) ∂z2 ∂z z=0 µ ¶ 1 ¯ = (s + )2 2 f(g) ¯ 8(n 2) ¡ ¢ where ∂ = 1 ∂ i ∂ . ∂z 2 ∂x ∂y Similarly, ³ ´ 1 ( f)(g) = (s )2 2 f(g). 2 8(n 2) Let be the Casimir operator of the real¡Lie algebra g¢. Then 1 (f)(g) = s2 + 2 2 f(g) for f E . 4(n 2) ∈ ,s ¡ ¢ 8.3.1.7 Construction of the spherical distributions Dene P : C by P () = [x0, ] = . If f E (), dene 1 → 1 1 ∈ ,s

s , Z,s(f) = P1(b) f(b)db ZB l ,l y with y = y y . | | | | We want to extend³ ´ the analytic distribution-valued function s Z,s, dened on s Re s < + 1 to a meromorphic distribution-valued function→ on C. { | } Proposition 8.13. If f E,s(), the integral dening Z,s(f) is absolutely convergent for Re (s) < ∈+ 1 which has a meromorphic continuation with at most simple poles at + + 2, + + 4, + + 6, . . . . { | | | | | | } Proof. We use the identication of B with = = (p, q) p, q S(IRn), (p, q) = 0 . The integral Z (f) can be written as { | ∈ } ,s

s , Z,s(f) = P1(b) f(b)db ZB s , = (p1 + iq1) f(p1 + iq1, . . . , pn + iqn)d(p, q) Z where d(p, q) is a measure on . By (8.4) a function f is a function on with the property s C, = 0 f() = f(). (8.5) ∀ ∈ 6 | | µ| |¶ Then we use a very important lemma. 90 Chapter 8. Decomposition of the oscillator representation

Lemma 8.14. The following equality holds for x C, Re > 1 ∈ x 1 e(, ) x ( ) = Re (ux) sgn (Re (ux)) (u)du, | | x | | | | ZU(1, )

where e(, ) = 2 (1+) . Here (ei) = ei for some . ( +2 )( ++2 ) 2 2 ∈ The proof is given in [3], Appendix B, section B.2. This lemma permits to write down the integral Z,s(f) in the form

s ++2 s +2 ( 2 )( 2 ) s , Z,s(f) = [Re ((p1 + iq1)u)] u 2s+(1 s ) ZU(1, ) Z f(p1 + iq1, . . . , pn + iqn)d(p, q)du l ,l since for a IR, a sgn a = a . We change the variables pj + iqj (pj + 1 ∈ | | → iqj )u with j = 1, . . . , n. By (8.5) one has

s ++2 s +2 ( )( ) Z (f) = 2 2 ,s 2s+(1 s ) s , p f(p1 + iq1, . . . , pn + iqn)d(p, q) du 1 ZU(1, ) ·Z ¸ Using classical results for the distributions x and x sgn x ([11]), we see that the integral in square brackets can be analytically| | contin| |ued to the entire s-plane with at most simple poles at s = + 1, + 3, + 5, . . . if is even; s = + 2, + 4, + 6, . . . if is odd. The Gamma function () is known to have simple poles at = 0, 1, 2, . . . Thus, Z,s(f) admits a meromorphic continuation with at most simple poles at s = + + 2, + + 4, + + 6, . . . If we change and consider Z ,s(f) we obtain the same answer for 7→ Z ,s(f). So, in fact, Z,s(f) admits a meromorphic continuation with at most simple poles at s = + + 2, + + 4, + + 6, . . . ¤ Let us write for f E| |() | | | | ∈ ,s 1 u,s(f) = s + +2 Z,s(f). ( 2 | | )

Then by the above proposition, the function u,s(f) is an entire function of s C. Furthermore: ∈

u E ()0 (s C) ,s ∈ ,s ∈ ∞(h)u = u for all h H. ,s ,s ,s ∈ If ϕ (G) we dene: ∈ D ,s(ϕ) =< u, s, ∞(ϕ)u,s > . ,s 8.3. The case n 3 91

Proposition 8.15. The distribution ,s is an H-bi-invariant eigendistribution of (X):

1 = (s + )2 2 1 ,s 8(n 2) ,s 1 ¡ ¢ = (s )2 2 . 2 ,s 8(n 2) ,s ¡ ¢ 8.3.1.8 Intertwining operators Let f E (). Dene ∈ ,s

s , (A,sf) () = P (, b) f(b)db. ZB This is a meromorphic function of s C. ∈ Lemma 8.16. (i) If f E,s, then A,sf E , s. ∈ ∈ (ii) A,s : E,s E , s is continuous. → (iii) A,s intertwines the action of G:

A,s ,s(g) = , s(g) A,s for all g G. ∈ Let w K be the element ∈ 0 1 0 1 0 0 . 0 0 1   I   1 1 Then wLw = L and wNw = N. An easy calculation shows that

0 s 0 (A,sf) (g ) = 2 i f(gwn )dn ZN provided dn is suitably normalized. This is the standard intertwining operator, see [19].

8.3.1.9 The Fourier transform We start with the Cartan-Berger decomposition of X.

+ + Lemma 8.17. (i) There holds G = KAI H, with AI = a it = exp itL : t > 0 . More precisely: to any x G corresponds a unique{ t 0such that } ∈ x Ka itH. ∈ (ii) The set ∞(K G/H) is (by restriction to AI ) in bijective correspondence C even\ with ∞(IR) . C 92 Chapter 8. Decomposition of the oscillator representation

(iii) The map K/M IR+ G/H dened by (k, t) ka itH, is a dieomorphism onto an open subset→ of G/H. 7→

Choose from now on the G-invariant measure dx on X such that:

∞ 0 ϕ(x)dx = ϕ(ka itx )A(t)dkdt, ZX ZK/M Z0 n 2 where dk is the Haar measure of K with total volume 1 and A(t) = (sinh 2t) .

Denition 8.18. If ϕ (X), then the Fourier transform ϕ is given by ∈ D F,s

s , ( ϕ) () = P (x, ) ϕ(x)dx (Re (s) > ). F,s ZX If ϕ (G), then ∈ D ( ϕ) () = ( ϕ ) () F,s F,s 0 where 0 ϕ0(x) = ϕ(gh)dh (x = gx ), ZH where dh is the Haar measure of H such as dg = dhdx.

We have:

(i) For xed , the map s (ϕ) is meromorphic in s. → F,s (ii) (ϕ) E . F,s ∈ ,s (iii) commutes with the G-action. F,s (iv)

1 ( ϕ) = (s + )2 2 (ϕ) F,s 1 8(n 2) F,s 1 ¡ ¢ ( ϕ) = (s )2 2 (ϕ) F,s 2 8(n 2) F,s ¡ ¢ Remark 8.19. The spherical distributions are related to the Fourier transform. If ϕ is a function in (G), then we have the following relation: D

1 2 ,is(ϕ˜ ? ϕ) = is + +2 is + +2 ,isϕ(b) db ( | | )( | | ) B |F | 2 2 Z with ϕ˜(g) = ϕ(g 1) and s IR. ∈ 8.3. The case n 3 93

8.3.1.10 The Plancherel formula In this section we will formulate the Plancherel theorem. For this we use the same method as Van den Ban. In our case only one parabolic subgroup occurs. To dene the Plancherel measure, we need, as Van den Ban has done, to construct a specic intertwining operator T,s between ,s(g) and , s(g). Composing T, s with T,s we have an intertwining operator from E,s() to E,s(). Then using the Schur’s lemma we arrive to the conclussion that this must be a constant. For more details see [36]. We begin with the construction of T,s. Let A0 : E,s() E ,s() be given → by ϕ() ϕ(). Then 7→ A0 ,s(g) = ,s(g) A0 where g is the complex conjugate of g. Observe that K is xed under this conjugation. A0 A,s is an intertwining operator between ,s(g) and , s(g) with A,s s+ as before. Calling A0 = A A , the specic operator T is equal to 2 A0 . ,s 0 ,s ,s ,s We have T, s T,s is an intertwining operator between E,s() and E,s(). By 0 0 Schur’s lemma T, s T,s = I. Taking = , f(b) = [ , b] if 0 and 0 0 = , f(b) = [ , b] if < 0. Applying Schur’s lemma for ,s restricted to K it follows T,sf() = c(, s)f() and c(, s) = c(0, s ) for all . Therefore = c(, s)c(, s). | | We compute the value of c(0, s). Taking = 0 and f = 1, we obtain

0 c(0, s) = (T0,sf)( ) s+ 0 = 2 (A0,sf)( )

0 0 s Let (t, s) be equal to P (ka itx , ) dk. Using property (iv) of the K | | Fourier transform, we obtainZ that u = (. , s) is a solution of 1 d du A(t) = (s2 2)u. A(t) dt dt

We can solve this equation by rst deriving a dierential equation for v(t) = ( s+)/2 2 (cosh 2t) u(t), and then substituting tanh 2t as a new variable in the equation thus found, to obtain a hypergeometric dierential equation. Solving the latter 94 Chapter 8. Decomposition of the oscillator representation we nd (using the fact that (t, s) is regular for t = 0)

(s )/2 s s + 2 1 + 2 u(t) = (s)(cosh 2t) F ( , ; ; tanh 2t). 4 4 2 We now determine (s)

0 0 s (s) = u(0) = P (kx , ) dk | | ZK 2 2 (s )/2 = s1 + s2 ds Sn1 | | Z 2 s +1 s +n 2 = . . . (sin 2) . . . (sin n 1) d2 . . . dn 1. vol(Sn 1) Z0 Z0 ( 1+ ) Using sin tdt = √ 2 , we obtain ( 2+ ) Z0 2 s +2 2 n 2 ( 2 ) (s) = (√) vol(Sn 1) s +n ( 2 ) n s +2 ( 2 )( 2 ) = s +n . ( 2 )

Substituting this value in (8.6) and using relations between hypergeometric functions see [6], we get

(s+)t ( s )/2 + s + s + 2 1 + 2 c(0, s) = lim e ( s)(cosh 2t) F ( , ; ; tanh 2t) t →∞ 4 4 2 1+ s (s+)/2 ( 2 )( 2 ) = ( s)2 s 2+ s ( 4 )( 4 ) n s +2 1+ s (s+)/2 ( 2 )( 2 )( 2 )( 2 ) = 2 s +n s 2+ s ( 2 )( 4 )( 4 ) n 1+ s +2 s 1 ( 2 )( 2 )( 2 )( 2 ) = 2 s +n s+ . √( 2 )( 2 )

Remark 8.20. During the calculation we had to assume Re (s) < , but the result is valid for all s, because both sides of the equality are meromorphic functions of s. Now we can compute the value of c(, s)

n 1+ s+ +2 s+ 1 ( 2 )( 2 )( | 2| )( 2 | | ) c(, s) = 2 s+ +n s+ + . √( | 2| )( 2| | ) 8.3. The case n 3 95

Theorem 8.21. Let f (X). Then it follows ∈ D

2 1 2 f = C ,isf ds

2 || || c(, is) ||F || X∈ Z | | with C a positive constant.

Remark 8.22. Taking the same normalization of the measures as Van den Ban, the value of the constant C is equal to . 1 Remark 8.23. For n = 3 the value of c(, s) is and we obtain the Plancherel s | | 4 formula of SL(2, C)/SO(2, C). For n = 4 we get c(, s) = and we obtain (s )2 the Plancherel formula of SL(2, C) SL(2, C)/D, where D stands | | for the diagonal of SL(2, C) SL(2, C). 8.3.2 Decomposition of L2(Cn) 8.3.2.1 The denition of the oscillator representation

Let G be the group SL(2, C) SO(n, C) with n 3 and let ωn be the unitary representation of G on H = L2(Cn) dened by:

1 ω (g)f(z) = f(g z), g SO(n, C) n ∈ n a 0 ωn(g(a))f(z) = a f(az), g(a) = 1 , a C | | 0 a ∈ µ ¶ iRe (b[z,z]) 1 b ω (t(b))f(z) = e f(z), t(b) = , b C n 0 1 ∈ µ ¶ 2iRe ([z,w]) 0 1 ωn()f(z) = e f(w)dw, = n 1 0 Z µ ¶ where [z, w] = z w +z w + +z w if z = (z , z , . . . , z ), w = (w , w , . . . , w ). 1 1 2 2 n n 1 2 n 1 2 n We call ωn the oscillator representation of G. More precisely, it is the restriction of the metaplectic representation of the group Sp(n, C) Sp(2n, IR) to G. Let (Cn) be the space of Schwartz functions on Cn. Note that (Cn) is stable S n S n under the action of ω , so is the space 0(C ) of tempered distributions on C . n S

8.3.2.2 Some minimal invariant Hilbert subspaces of S 0(Cn)

2 n n We consider L (C ) as a Hilbert subspace of 0(C ) and we will try to decompose it into minimal invariant Hilbert subspaces. IrreducibleS unitary representations of G are of the form where is an irreducible representation of SL(2, C) and one of SO(n, C). Wewill now construct intertwining operators from (Cn) to the Hilbert space of such representations. Not all combinations (, ) occur.S Here are some of these operators. 96 Chapter 8. Decomposition of the oscillator representation

n Let E,s() be the space of ∞-functions on = z C : [z, z] = 0, z = 0 such that: C { ∈ 6 } s f() = f() ( , C) | | ∈ ∈ µ| |¶ with , s C. ∈ ∈ The group SO(n, C) acts irreducibly on E,s() by

1 (g)f(z) = f(g z) ,s for s = 0, with g SO(n, C) and f E,s(). The representation is pre-unitary for s 6 iIR, see Section∈ 8.3.1. Let us∈denote the unitary completion of this space ∈ by ,s. HThe group SL(2, C) acts irreducibly on L2(C) by

m 2 cz + d az + b (g)f(z) = cz + d f ,m | | cz + d cz + d µ| |¶ µ ¶

1 a b with g = , g SL(2, C), iIR, m . ( belongs to the so-called c d ∈ ∈ ∈ ,m principal series,µ see¶Chapter 3). Let us now dene an operator from (Cn) into the Hilbert space L2(C) Fis, S ˆ as follows: 2H,is

iRe (z[w,w]) is , ( is,ϕ)(z, ) = e [w, ] ϕ(w)dw (8.7) F n Z with s IR, , z C and . We ∈will sho∈w that:∈ ∈ 1) The operator is well dened, i.e. ϕ L2(C) ˆ . Fis, Fis, ∈ 2H,is 2) is continuous as operator from (Cn) to L2(C) ˆ . Fis, S 2H,is 3) The operator intertwines the action of ω and . Fis, n is, is, First, we want to prove 1) and 2). We have:

Proposition 8.24. For all s IR and the operator maps (Cn) into ∈ ∈ Fis, S L2(C) ˆ and is continuous. 2H,is Fis, Proof. We take = 0 = (1, i, 0, . . . , 0) in (8.7). To get the result we divide the proof in several steps. 1. Consider the following map:

(F ϕ)(z) = eiRe (zw )ϕ(w)dw

Z 8.3. The case n 3 97

for z C, ϕ (C). We will make some estimations for (F ϕ)(z) , which we need∈ later∈on.S | | We may assume z IR. Indeed, writing z = z ei we obtain: ∈ | | (F ϕ)(z) = (F ϕ )( z ) 1 | | i/2 with ϕ1(w) = ϕ(e w). So assume z IR. Then ∈ iz(x2 y2) (F ϕ)(z) = e ϕ(x, y)dxdy. 2 Z Set x + y = u, x y = v. Then 1 u + v u v (F ϕ)(z) = eizuvϕ , dudv. 2 2 2 2 Z µ ¶ 1 u + v u v Write g(t, v) = eituϕ , du. This is again a Schwartz 2 2 2 Z µ ¶ function in (IR2), and S

(F ϕ)(z) = g(zv, v)dv.

Z So we have:

2 2 N 2 2 N (F ϕ)(z) sup (1 + t + r ) g(t, r) (1 + zv + v ) dv | | | | | | t,r Z £ ¤ with N a positive integer. The Schwartz-norm

g = sup (1 + t2 + r2)N g(t, r) || || t,r | | £ ¤ u+v u v is bounded by a Schwartz-norm of ϕ1 2 , 2 , hence by a Schwartz-norm of ϕ1, hence by a Schwartz-norm of ϕ, since all operations which occur are continuous (IR2) (IR2). ¡ ¢ S → S So we get for z C: ∈ N (F ϕ)(z) C ϕ 1 + (1 + z 2)v2 dv | | || || | | Z ¡ 2 1/2 ¢ 2 N C ϕ (1 + z ) (1 + v ) dv || || | | Z 2 1/2 C0 ϕ (1 + z ) || || | |

with C and C0 constants. 98 Chapter 8. Decomposition of the oscillator representation

2. Consider now

iRe (z(w2+w2)) is , ( is,ϕ)(z) = e 1 2 (w1 + iw2) ϕ(w1, w2)dw1dw2 F 2 Z for is C, IN, , z C, ϕ (C2). ∈ ∈ ∈ ∈ ∈ S The integral exists anyway for Re (is) . For other is we use analytic continuation.

To estimate ( is,ϕ)(z) also for these is, we need an explicit description of this analytic conF tinuation. We make use of [24], p. 138,139. Perform the change of variables y = w + iw , y = w iw . Then we get 1 1 2 2 1 2

1 iRe (zy1y2) is , y1 y2 ( is,ϕ)(z) = e y ϕ y1 + y2, dy1dy2

1 F 4 2 i Z µ ¶ is y1 = y1 g(zy1, y1)dy1 (8.8) | | y Z µ| 1|¶

1 iRe (y2u) y1 y2 where g(u, y1) = e ϕ y1 + y2, dy2. Now g is again 4 i µ ¶ Z y y in (C2) and the operation ϕ y + y , 1 2 g(u, y ) is continuous S 1 2 i → 1 µ ¶ (C2) (C2). S → S i We write this integral using polar coordinates y1 = re ,

∞ is +1 ( ϕ)(z) = r h(z, r)dr Fis, Z0 2 where h(z, r) = g(zrei, rei)eid. Z0 Now the meromorphic continuation reads:

1 1 ( 1) ∞ ∂ ( ϕ)(z) = ris h(z, r)dr. Fis, (is + 2) . . . (is 1)is ∂r 1 Z0 One has:

2 ∂ i ∂ i i i ∂ i i h(z, r) = ze g(zre , re ) + ze g(zre , re ) ∂r ∂y ∂y Z0 1 1 i£ ∂ i i i ∂ i i i + e g(zre , re ) + e g(zre , re ) e d. ∂y2 ∂y2 ¤ Hence, for some s IR, ∈

2 1 ∞ 2 2 N ( ϕ)(z) C g (1 + z ) 2 1 + r (1 + z ) dr | Fis, | || || | | | | Z0 ¡ ¢ 8.3. The case n 3 99

for some Schwartz-norm of g, depending on a positive integer N and with C a constant. This gives:

2 1 2 1/2 ( ϕ)(z) C0 g (1 + z ) 2 (1 + z ) | Fis, | || || | | | | 2 /2 1 C0 g (1 + z ) || || | | 2 /2 1 C00 ϕ (1 + z ) || || | | for some Schwartz-norm of ϕ and C, C0, C00 constants. ∂k Remark 8.25. It follows that h(z, 0) = 0 for k = 0, 1, . . . , 1. So (8.8) ∂rk | | is the actual analytic continuation to Re (is) > 2. | | Dene by analytic continuation

0 iRe (z[w,w]) is , ( is,ϕ)(z, ) = e (w1 + iw2) ϕ(w)dw F n Z with s IR, = n 2, , z C, ϕ (Cn). Applying∈ 1. and2. step∈ by step∈ we get:∈ S

0 2 /2 1 2 n2 ( ϕ)(z, ) C ϕ (1 + z ) (1 + z ) 2 | Fis, | || || | | | | 2 1 C ϕ (1 + z ) || || | | for some Schwartz-norm of ϕ and with C a constant. Dene by analytic continuation:

iRe (z[w,w]) is , ( is,ϕ)(z, ) = e [w, ] ϕ(w)dw F n Z 0 B = K , K = SO(n, IR). This is a ∞-function of (z, ). Then clearly, writing∈ = k0 for some k K, we get: C ∈ ( ϕ)(z, ) = ( (L ϕ))(z, 0), Fis, Fis, k so 2 1 ( ϕ)(z, ) C ϕ (1 + z ) | Fis, | || || | | for all , for some Schwartz-norm of ϕ and with C a constant. Hence ( is,ϕ)(z, ) 2 n 2 F L (C B), and ϕ is,ϕ is continuous (C ) L (C B). ¤ ∈ In order to prove the→ Fintertwining relationSwe intro→duce thefollowing lemma. Lemma 8.26. For every function ϕ (Cn) one has ∈ S eiRe (z[w,w])[w, ]is,ϕˆ(w)dw n Z is, n iRe ([w,w]/z) is, = z z e [w, ] ϕ(w)dw | | n Z 100 Chapter 8. Decomposition of the oscillator representation with , s IR, and the Fourier transform is dened by ∈ ∈ ∈ 2iRe ([z,w]) fˆ(z) = e f(w)dw. n Z Proof. By denition

eiRe (z[w,w])[w, ]is,ϕˆ(w)dw n Z i Re (z[w,w]) 2Re ([w,x]) is,

= ϕ(x) e [w, ] dw dx

n n Z µZ ¡ ¢ ¶ We claim that the inner integral is equal to

i Re (z[w,w]) 2Re ([w,x]) is, e [w, ] dw n Z ¡ ¢ is, n is, iRe ([x,x]/z) = z z [x, ] e (8.9) | | By the SO(n, C)-invariance of the equality it suces to prove it for =(1, i, 0, . . . , 0) n 2 . In that case, writing w = (w , w , w) in C C C , we have ∈ 1 2

i Re (z[w,w]) 2Re ([w,x]) is, e [w, ] dw n Z ¡ ¢ 2 2 i Re (z(w1 +w2 )) 2Re (w1x1+w2x2) is, = e (w1 + iw2) dw1dw2 2 µZ ¡ ¢ ¶ i Re (z[w,w]) 2Re ([w,x]) e dw n2 µZ ¡ ¢ ¶ It follows from Lemma 8.6 in Section 8.2 that the rst integral in the above is:

2 2 i Re (z(w1 +w2 )) 2Re (w1x1+w2x2) is, e (w1 + iw2) dw1dw2 2 Z ¡ ¢ 2 2 is, 2 is, iRe ((x1+x2)/z) = z z (x + ix ) e . | | 1 2 The second integral, by the Bochner formula (see Lemma 4.5 in [17]) is:

i Re (z[w,w]) 2Re ([w,x]) n+2 iRe ([x, x]/z) e dw = z e . n2 | | Z ¡ ¢ Multiplying these two we have thus proved the equality (8.9) and thus the lemma. ¤ Remark 8.27. We note here that we can, as in the proof of Lemma 8.6 in Section 8.2, introduce the Gaussian kernel and prove the above lemma rigorously.

So we have:

Theorem 8.28. For s IR, the operator is, intertwines the action ωn with of SL(2, C∈) SO(∈n, C). F is, ,is 8.3. The case n 3 101

Proof. It is clear that is, intertwines the SO(n, C)-action. The intertwining relations for the elemenFts g(a) and t(b) are easy to check. We now check the -intertwining relation:

ω ()ϕ(z, ) = ()( ϕ)(z, ) (8.10) Fis, n is, Fis, for ϕ (Cn). By denition, ∈ S

iRe (z[w,w]) is , is,ωn()ϕ(z, ) = e [w, ] ϕˆ( w)dw. F n Z Lemma 8.26 easily implies (8.10) and thus the theorem. ¤

8.3.2.3 Decomposition of L2(Cn) First we introduce the following lemma.

Lemma 8.29. Let f be a function in (Cn). Then the relation S

2n 2

f(z)dz = f(w) dd(w)

n w2+ +w2 =1 | | Z Z Z 1 n holds where d(w) is a suitably normalized SO(n, C)-invariant measure on X = w Cn w2 + + w2 = 1 . { ∈ | 1 n } n 2 2 Proof. Let T be the map from C z1 + + zn = 0 into C X where X = w Cn : w2 + + w2 = \1{ such that for every} z we have T (z) = { ∈ 1 n } ( z , z2 + + z2 ). Then T is a dieomorphism and we can apply the √z2+ +z2 1 n 1 n theorem of changep of variables. Therefore we obtain

f(z)dz = f(w)J(, w)d(w)d (8.11)

n X Z Z Now we want to compute the value of this function J(, w). First we see that the function J does not depend on w, i.e. J(, w) = J(). Let g be an element of SO(n, C). Then we get for all functions f

f(z)dz = f(g z)dz

n n Z Z = f(g (w))J(, w)d(w)d X Z = f((g w))J(, w)d(w)d X Z 1 = f(w)J(, g w)d(w)d X Z 102 Chapter 8. Decomposition of the oscillator representation since d(w) is an SO(n, C)-invariant measure on X. Comparing this relation with (8.11) we see that J(, w) = J(, g w) for all g SO(n, C). This means J(, w) = J(). Therefore it is proven that ∈

f(z)dz = f(w)J()d(w)d

n X Z Z

Now we will compute the value of J(). Let C. Then we have ∈

f(z)dz = f(w)J()d(w)d

n X Z Z 1 2 = f(w)J( ) d(w)d X | | Z On the other hand we can write the above integral as follows

f(z)dz = f(z)dz

n | | n Z Z 2n = f(w)J()d(w)d | | X Z We can conclude that 1 2n+2 J( ) = J(). | | 2n 2 If = 1 then we have J() = c with c a positive constant and the result is proven. ¤ | | Let us compute ϕ 2. Let ϕ (Cn). Applying Lemma 8.29 we have || || ∈ S

2 2 n 2

ϕ(z1, . . . , zn) dz = C ϕ(√x1, . . . , √xn) d(x)d

n | | x2+ +x2 =1 | | | | Z Z Z 1 n with C a constant. Applying the lemma to the intertwining operator and by analytic continuation we obtain

iRe (z[w,w]) is , is,ϕ(z, ) = e [w, ] ϕ(w)dw F n Z

= C eiRe (z)F (, is, , )d (8.12)

(is )/2+n 2,/2 is , where F (, is, , ) = [x, ] ϕ(√x)d(x). x2+ +x2 =1 Z 1 n Making use of Plancherel’s theorem on SO(n, C)/SO(n 1, C) see Theorem 8.3. The case n 3 103

8.21 in Section 8.3.1 and (8.12) it follows

2 2 n 2

ϕ(z) dz = C ϕ(√x) d(x)d

n | | x2+ +x2 =1 | | | | Z Z Z 1 n

1 2

= C0 F (, is, , ) ds d

2 c(, is) || || Ã ! Z X∈ Z | | 1 2

= C0 F (, is, , ) d ds

2 c(, is) || || µ ¶ X∈ Z | | Z 1 2 = C00 is,ϕ ds

2 c(, is) ||F || X∈ Z | | n 1+ is+ +2 is+ 1 ( 2 )( 2 )( |2| )( 2 | | ) where c(, is) = 2 is+ +n is+ + and C, C0, C00 are cons- √( | 2| )( 2| | ) tants. The above result also gives the decomposition of L2(Cn) into a minimal invariant subspaces. The decomposition is multiplicity free (see next chapter) and the irreducible unitary representations which occur are given by is, is,. In conclusion, we have Corollary 8.30. Let ϕ be a function in (Cn). We have the following Plancherel formula S 2 ∞ 1 2 ϕ 2 = D 2 is,ϕ ds || || c(, is) ||F || 0 X∈ Z | | with D a constant. As representations of SL(2, C) SO(n, C), we have

2 n ∞ L (C ) = is, is,ds. 0 X∈ Z 104 Chapter 8. Decomposition of the oscillator representation CHAPTER 9 Invariant Hilbert subspaces of the oscillator representation of SL(2, C) SO(n, C)

In this chapter we study the oscillator representation ωn for the groups SL(2, C) O(1, C), SL(2, C) O(2, C) and SL(2, C) SO(n, C) with n 3 in the context of the theory invariant Hilbert subspaces. Our main result is that any ωn(G)-stable n Hilbert subspace of 0(C ) decomposes multiplicity free. For this purpose, we rst need to prove, in caseS n 3, that (SO(n, C), SO(n 1, C)) are generalized Gelfand pairs and to compute the MN-invariant distributions on the cone. We do not know if the result holds for the cases SL(2, C) SO(1, C) and SL(2, C) SO(2, C). 9.1 The cases n = 1 and n = 2

If G is equal to the group SL(2, C) O(1, C) then it is easy to prove that every invariant Hilbert subspace of 0(C) decomposes multiplicity free. We leave this to the reader to check by similarStechniques as applied in this section. We are going to prove here the case n = 2.

9.1.1 The denition of the oscillator representation Let G be the group SL(2, C) O(2, C) and let ω be the unitary representation of 2 G on H = L2(C2) dened by: 1 ω (g)f(z) = f(g z), g O(2, C) 2 ∈ 2 a 0 ω2(g(a))f(z) = a f(az), g(a) = 1 a C | | 0 a ∈ µ ¶ iRe (b[z,z]) 1 b ω (t(b))f(z) = e f(z), t(b) = b C 2 0 1 ∈ µ ¶

105 106 Chapter 9. Invariant Hilbert subspaces of the oscillator representation

2iRe ([z,w]) 0 1 ω2()f(z) = e f(w)dw, = 2 1 0 Z µ ¶ where [z, w] = z1w1 + z2w2 if z = (z1, z2), w = (w1, w2). The Fourier transform is dened by 2iRe ([z,w]) fˆ(z) = e f(w)dw. 2 Z We call ω2 the oscillator representation of G. More precisely, it is the natural extension of the restriction of the metaplectic representation of the group Sp(2, C) Sp(4, IR) to SL(2, C) SO(2, C). Let (C2) be the space of Schwartz functions on C2. Note that (C2) is stable S 2 S 2 under the action of ω , so is the space 0(C ) of tempered distributions on C . 2 S 9.1.2 Invariant Hilbert subspaces of S 0(C2) The denition of the Fourier transform of a function on (C2) naturally gives 2 S rise to a Fourier transform on 0(C ). We denote Tˆ this Fourier transform of a S2 tempered distribution T 0(C ). ∈ S 2 Let H be a Hilbert subspace of 0(C ), invariant under ω2(G), and let K be its reproducing kernel. By Schwartz’s Skernel theorem we can associate to it a unique tempered distribution T on C2 C2. This distribution satises the following conditions: 1. T is a positive-denite kernel, in particular a Hermitian kernel, i.e. 2. T (x, y) = T(y, x). 3. T is ω2(G)-invariant : (ω2(g) ω2(g)) T = T for all g G. The latter property implies, in more detail, ∈ 3a. T (g x, g y) = T (x, y) for all g O(2, C). ∈ 3b. If a C, then ∈ 4 T (ax, ay) = a T (x, y). | | 3c. If b C, then ∈ iRe (b([x,x] [y,y])) e T (x, y) = T (x, y).

3d. Tˆ(x, y) = T (x, y).

As preparation for our main result, the multiplicty free decomposition of the oscillator representation, we shall show that any of the above distributions T is symmetric: T (x, y) = T (y, x). We shall do this in several steps. Observe that T (y, x) satises the same conditions as T (x, y). Step I By condition 3c, we obtain

Supp T = (x, y) C2 C2 : [x, x] [y, y] = 0 . 2,2 { ∈ } Step II 9.1. The cases n = 1 and n = 2 107

2 2 Let 20 ,2 = 2,2 (0, 0), being an isotropic cone in C C . In a neighborhood 2 2 \ of 20 ,2 in C C we take as coordinates s = [x, x] [y, y] and ω 20 ,2. So we can write locally there ∈ T = S (ω) (i)(s) i i I X∈ where I is some nite subset of IN and the Si are distributions on 20 ,2. Applying condition 3c again, we get that only the term with i = 0 survives, so

T (x, y) = ([x, x] [y, y]) S (ω) 0 outside (x, y) = (0, 0). It remains to study the distribution S0 on 20 ,2. Step III On the open subset of 0 given by [x, x] = 0, we get by properties 3a and 3b, 2,2 6

S0(x, y) = 0(ω1, ω2)

2 where 0 is an O(2, C)-invariant distribution on , with = x C [x, x] = 1 . The distribution is symmetric: (ω , Xω ) =X (ω ,Xω ). {We∈are|going } 0 0 1 2 0 2 1 1 0 a b to prove this. Observe that SO(2, C). Let g = , g = , X ' 0 0 1 b a µ ¶ µ ¶ with a2 + b2 = 1, then 0 1 0 g (g x ) = g x 0 with x0 = (1, 0). So

(ω , ω ) = (g x0, h x0) = (g (g x0), g (h x0)) 0 1 2 0 0 0 0 = (h x0, g x0) = (ω , ω ) 0 0 2 1 with g, h SO(2, C). Hence∈ T (x, y) = T (y, x) on the open subset of C2 C2 dened by [x, x] = 0 (or, what is the same, [y, y] = 0). So S(x, y) = T (x, y) T (y, x) has support6 contained in [x, x] = [y, y] = 0.6 Step IV 2 Set 2 = x C : [x, x] = 0 . The distribution S has support in 2 2. With the coordinates{ ∈ s = [x, x], }s = [y, y] near , which can be tak en, 1 2 2 2 provided x = 0 and y = 0, so on 0 0 (in obvious notation), we get 6 6 2 2

S(x, y) = ([x, x], [y, y]) U( , ) ( , 0 ) 1 2 1 2 ∈ 2 where U is an O(2, C)-invariant distribution on 0 0 , homogeneous of degree 2 2 4. Since S(x, y) = S(y, x) it easily follows that U(1, 2) = U(2, 1). Step V 2 2 2 The cone 20 = x C : x1 + x2 = 0, x = 0 consists of two disjoint pieces: 0 0 { 0∈ 6 } C and C with = (1, i). 108 Chapter 9. Invariant Hilbert subspaces of the oscillator representation

So the O(2, C)-invariant distribution U on 0 0 has 4 components: 2 2 U(0, 0), U(0, 0), U(0, 0), U(0, 0) with , C. By homogeneit∈ y and O(2, C)-invariance, the rst two components are zero, i.e. U(t0, t0) = t 4U(0, 0) = U(0, 0) | | for all t = 0. Consider6 the third component. By O(2, C)-invariance we have

0 1 0 0 0 U(t , t ) = U( , ) for all t = 0. By homogeneity, 6 0 1 0 4 0 2 0 0 0 U(t , t ) = t U( , t ) = U( , ) | | for all t = 0. So U(0, t0) = t 2U(0, 0). Hence U(0, 0) = d V (). 6 | | Similarly we get U(t20, 0) = t 4U(0, 0). Hence V (t) = t 2V (), so V () = d. | | | | By consequence, U(0, 0) = U(0, 0). 1 0 If we apply g = O(2, C), we get 0 0 1 ∈ µ ¶ U(0, 0) = U(0, 0). We have seen before that the distribution U also satises U( , ) = U( , ). 1 2 2 1 Then we obtain that also the third component is zero. We can do the same for the fourth component. Then we obtain that the O(2, C)-invariant distribution U on 0 0 is equal 2 2 to zero. Hence S = 0 in a neighborhood of 0 0 , and therefore Supp S 2 2 (0) 2 2 (0) . { Step VI} ∪I { } From property 3c we conclude that ([x, x] [y, y])k S = 0 for k = 1, 2, . . . , hence, combining it with property 3d, we have ¤k S = 0 for k = 1, 2, . . ., in particular ¤ S = 0. Here ¤ denotes the d’Alembertian ∂2 ∂2 ∂2 ∂2 2 + 2 2 2 . ∂x1 ∂x2 ∂y1 ∂y2

If S has support in (0) 2 2 (0) , and ¤ S = 0, it easily follows, by recalling the local structure{ of} suc∪ {h distributions, } being basically a nite linear combination of tensor products of distributions supported by 2 and the origin, that S = 0 (see [28],Ch.III, 10). So nally we have shown that T (x, y) = T (y, x). § 9.2. The case n 3 109

9.1.3 Multiplicity free decomposition The multiplicity free decomposition of the oscillator representation, i.e. the multiplicity free decomposition into irreducible invariant Hilbert subspaces of any ω2(G)- 2 invariant Hilbert subspace of 0(C ), is now easily proved by applying Criterion 5.6 S 2 with JT = T. If H is any invariant Hilbert subspace of 0(C ) with reproducing 2 2 S kernel T 0(C C ), then T is the kernel of the space J H and the property J H = H∈comesS down to T(x, y) = T (x, y). Since T is positive-denite, this is equivalent with T (x, y) = T (y, x) which immediately follows from the last section. So we have the following result.

2 Theorem 9.1. Any ω2(G)-invariant Hilbert subspace of 0(C ) decomposes multi- S 2 plicity free into irreducible invariant Hilbert subspaces of 0(C ). S

9.2 The case n 3 9.2.1 Complex generalized Gelfand pairs In this section we show that the pairs (SO(n, C), SO(n 1, C)) are generalized Gelfand pairs for n 2.

9.2.1.1 Denition of generalized Gelfand pairs We shall give a brief summary of the theory of generalized Gelfand pairs which is connected with the theory of invariant Hilbert subspaces that we have introduced in Chapter 5, for more details see [40]. Let G be a Lie group and H a closed subgroup of G. We shall assume both G and H to be unimodular. Denote by (G), (G/H) D D the space of ∞-functions with compact support on G and G/H respectively, C endowed with the usual topology. Let 0(G), 0(G/H) be the topological anti- dual of (G) and (G/H) respectively,DprovidedD with the strong topology. A conDtinuous unitaryD representation of G on a Hilbert space is said to be H realizable on G/H if there is a continuous linear injection j : 0(G/H) such that H → D j(g) = Lgj for all g G (L denotes left translation by g). The space j( ) is called an ∈ g H invariant Hilbert subspace of 0(G/H). We shall take all scalar products anti- linear in the rst and linear inDthe second factor.

Denition 9.2. The pair (G, H) is called a generalized Gelfand pair if for each continuous unitary representation on a Hilbert space , which can be realized on G/H, the commutant of (G) in the algebra End( H) of all continuous linear operators of into itself, is abelian. H H For equivalent denitions we refer to [37] and [33]. A large class of examples is given by the Riemannian semisimple symmetric pairs and by the nilpotent symmetric pairs [37], [1]. 110 Chapter 9. Invariant Hilbert subspaces of the oscillator representation

A usefull criterion for determining generalized Gelfand pairs was given by Thomas ([33], Theorem E). We shall apply it throughout this section. Its proof is easy and straightforward (1.c.). Denote by 0(G, H) the space of right H-invariant distributions on G pro- D vided with the relative topology of 0(G). It is well-known that 0(G, H) can be D D identied with 0(G/H). D Criterion 9.3. Let J : 0(G, H) 0(G, H) be an anti-automorphism. If J = (i.e. (J ) anti-unitary)D foral→l DG-invariant or minimal G-invariant HilbHert H |H subspaces of 0(G, H), then (G, H) is a generalized Gelfand pair. D We shall apply it in the following form. Criterion 9.4. Let be an involutive automorphism of G which leaves H stable. Dene JT = T for all T 0(G, H). If JT = T for all bi-H-invariant positive- denite (or extremal positive-denite)∈ D distributions on G, then (G, H) is a generalized Gelfand pair. Remark 9.5. T is dened by < T , f >=< T, f > (f (G)) and f (g) = ∈ D f((g)) (g G). T is dened by < T, f >=< T, f > (f (G)). ∈ ∈ D An important consequence of being a generalized Gelfand pair is the multiplicity -free desintegration of the left regular representation of G on L2(G/H). So one could, more or less without ambiguity, call this the Plancherel formula for G/H. If a xed parametrization is used for the set of irreducible unitary representations realized on G/H, there is no ambiguity at all. Let Z denote the algebra of all analytic dierential operators on G which com- mute with left and right translations by elements of G. Any bi-H-invariant common eigendistribution of all elements of Z is called a spherical distribution. It is a well-known consequence of Schur’s Lemma that any bi-H-invariant extremal positive-denite distribution on G is spherical. Spherical distributions play an important role in the harmonic analysis on G/H.

9.2.1.2 Morse’s lemma Let X be a complex analytic manifold of dimension n (n IN), and f : X C an analytic function on X. The tangent space of X at a p∈oint x0 will be denoted→ 0 by T X 0 . A point x X is called a critical point of f if the induced map x ∈ f : T Xx0 T Cf(x0) is zero. If we choose a local coordinate system (x1, . . . , xn) → in a neighborhood U of x0 this means that ∂f ∂f (x0) = = (x0) = 0. ∂x1 ∂xn

A critical point x0 is called non-degenerate if and only if the matrix

∂2f (x0) ∂x ∂x µ i j ¶ is non-singular. 9.2. The case n 3 111

Theorem 9.6. (Morse’s lemma) Let f : X C be an analytic function from a complex manifold X into C and let x0 be a→non-degenerate critical point of f. 0 0 There are local coordinates (x1, . . . , xn) at x with (0, . . . , 0) corresponding to x such that f can be written as

f(x , . . . , x ) = f(x0) + x2 + + x2 . 1 n 1 n The proof of the theorem is similar to the proof of Morse’s lemma in [15] p.146.

9.2.1.3 The pairs (SO(n, C), SO(n 1, C)) Assume n 3. Let G and H be the groups SO(n, C) and SO(n 1, C) respectively. The space X = G/H can be clearly identied with the set of all points x = (x1, . . . , xn) in n 2 2 C satisfying x1 + + xn = 1. We consider the follo wing function Q on the space X which parametrizes the H-orbits on X: Q(x) = x1. Q is an H-invariant complex analytic function on X with Q(x0) = 1 where x0 = (1, 0, . . . , 0). Dene X(z) = x X : Q(x) = z for z C. Now the H-orbit structure on X is as follows: { ∈ } ∈ Lemma 9.7. a) Let z C, z = 1, 1. Then X(z) is a H-orbit. ∈ 6 0 0 0 b) X(1) consits of two H-orbits: x and = x + v X : v x ⊥ 0 . { } 1 { ∈ ∈ { } \ } 0 0 c) X( 1) consits of two H-orbits: x and 1 = x + v X : v 0 { } { ∈ ∈ x ⊥ 0 . { } \ } In order to treat the sets X(1) and X( 1) separately, we choose open H- invariant sets X 1 and X1 such that X( 1) X 1, X(1) X 1, X(1) X1, 6 X( 1) X1 and X 1 X1 = X. These sets clearly exist. The6critical points of∪Q are x0 and x0. Both critical points are non-degenerate. We examine Q in the neighborhood of a critical point. Firstly, near x0 there exists a coordinate system w1, . . . , wn 1 such that { } 2 2 Q(w1, . . . , wn 1) = 1 + w1 + + wn 1, x0 corresponding to (0, . . . , 0). Secondly near x0 there exists a coordinate system w1, . . . , wn 1 such that { } 2 2 Q(w1, . . . , wn 1) = 1 + w1 + + wn 1, x0 corresponding to (0, . . . , 0). This is due to Morse’s lemma. From the properties of Q, we deduce applying [26] the existence of a linear map , which assigns to every f (X) a function f on C such that M ∈ D M F (Q(x))f(x)dx = F (z) f(z)dz M ZX Z 112 Chapter 9. Invariant Hilbert subspaces of the oscillator representation for all F (C). Here dx is an invariant measure on X, dz = dxdy (z = x + iy). f(z) giv∈ esD the mean of f over the set X(z). Let = ( (X)) and = M H M D Hi ( (Xi)) (i = 1, 1). Using the nature of the critical points of Q and the results ofM[26],D 6 we get: § = + + : , , (C) H { 0 0 1 1 0 1 ∈ D } 1 = 0 + 00 : 0, 0 (Q(X 1) H { ∈ D } = + : , (Q(X ) , H1 { 1 1 1 1 1 ∈ D 1 } where n 2 z + 1 if n is even 0(z) = | |n 2 z + 1 Log z + 1 if n is odd ½ | | | | and n 2 z 1 if n is even 1(z) = | |n 2 . z 1 Log z 1 if n is odd ½ | | | |

If we topologize , 1 and 1 as in [26] we have for i = 1, 1: H H H a) : (X ) is continuous. M D i → Hi

b) The image of the transpose map 0 : 0 0(X ) is the space of H- M Hi → D i invariant distributions on X . 0 is injective on 0 , because is surjective. i M Hi M Similar properties hold for : (X) . M D → H So, given an H-invariant distribution on X, there exists an element S 0 such that ∈ H < T, >=< S, > (9.1) M for all (X) . Fix Haar measures dg on G and dh on H in such a way that dg = dxdh∈,Dsymbolically. For f (G) put ∈ D

f ](x) = f(gh)dh (x = gH). ZH

Given a bi-H-invariant distribution T0 on G, there is an unique H-invariant ] distribution T on X satisfying < T0, f >=< T, f > (f (G)). This is a well-known fact. ∈ D We are now prepared to prove that (G,H) is a generalized Gelfand pair. We apply Criterion 9.4 with JT = T (T 0(G, H)). We have to show that T = T ∈ D for all bi-H-invariant positive-denite distributions T on G. Since T = T for such T , we shall show the following: for any bi-H-invariant distribution T on G one 1 has T = T. Here < T, f >=< T, f >, f(g) = f(g ) (g G, f (G)). In view of the relation between bi-H-invariant distributions on∈ G and∈HD-invariant distributions on X, and because of (9.1), this amounts to the relation

[(f)]] = (f ]) M M 9.2. The case n 3 113 for all f (G). For all F (C) one has ∈ D ∈ D

F (z) [(f)]](z)dz = F (Q(x))(f)](x)dx M Z ZX = F (Q(g))f(g)dg ZG 1 = F (Q(g ))f(g)dg. ZG 1 Since Q(g) = Q(g ) (g G) we get the result. So we have shown: ∈ Theorem 9.8. The pairs (SO(n, C), SO(n 1, C)) are generalized Gelfand pairs for n 3. The case n = 2 is easily seen to provide a generalized Gelfand pair too, since SO(2, C) is an abelian group.

9.2.2 MN-invariant distributions on the cone 9.2.2.1 The symmetric spaces SO(n, C)/SO(n 1, C) for n 3 We repeat here some facts from Section 8.3.1. Let G be the group SO(n, C). G acts on Cn in the usual manner. Let [, ] be the G-invariant bilinear form on Cn, given by,

[z, z0] = z z0 + + z z0 1 1 n n 2 if z = (z1, . . . , zn), z0 = (z10 , . . . , zn0 ). The (algebraic) manifold [z, z] = z1 + + 2 zn = 1, called X, is invariant under this action, and G acts transitively on X. Let x0 be the vector (1, 0, . . . , 0) in X. The stabilizer of x0 in G is equal to H = SO(n 1, C). The space X is a complex manifold, homeomorphic to G/H. Set 1 0 0 0 1 0 J =  . . .  . . . ..    0 0 1   J is not in G, but J is as soon as n is odd. Dene the involutive automorphism of G by (g) = JgJ. Then H is of index 2 in the stabilizer H of . So G/H is a complex symmetric space. Observe that is an inner automorphism if n is odd. We will pass now to the Lie algebra g of G. Let g = h + q be the decomposition of g into eigenspaces for the eigenvalues +1 and 1 for . Then we have: 0 0 . h = . Y  : Y complex, antisymmetric ,  0        114 Chapter 9. Invariant Hilbert subspaces of the oscillator representation

0 z z 2 n z2 q = .  : z2, . . . , zn C . . ∈  . —0   z    n     0 1   Write L = 1 0 q, actually in q k where k = so(n, IR), the real  —0  ∈ ∩ antisymmetric matrices. The Lie algebra k belongs to K = SO(n, IR). Clearly g = k + ik. Call p = ik. Then iL q p. The Lie algebra a = CL is a maximal∈ ∩abelian subspace of q; a is a Cartan subspace of q with respect to . Let m be the centralizer of a in h. Then 0 0 0 0 0 0 m = . .  : Y so(n 2, C) . . . ∈  . . Y     0 0     Let g be the eigenspaces of ad(iL) for the eigenvalues 1. Then 0 0 tp t n 2 g = X(p) = 0 0 i p : p C   p ip 0  ∈      and g = (g+). We have the following decomposition of g into eigenspaces of ad(iL): g = g m a g+. Set n = g . Then n is a nilpotent subalgebra of g. The connected subgroup A and N of G corresp onding to the Lie subalgebras a and n are given by: cos z sin z 0 A = a = exp zL = sin z cos z 0 : z C  z   ∈  0 0 In 2     and  1 1 t  1 + 2 [p, p] 2 i[p, p] i p 1 1 t n 2 N = n(p) = i[p, p] 1 [p, p] p : p C .   2 2  ∈  ip p In 2     9.2.2.2 The cone = G/MN  The centralizer M of a in H consists of the elements of G of the form: 1 0 0 0 0 1 0 0   0 0 , with h SO(n 2, C). . . ∈ . .  . . h    0 0    9.2. The case n 3 115

n Furthermore ZG(a) = MA, M A = e . Put = z C : [z, z] = 0, z = 0 , the isotropic cone. The group G∩acts transitiv{ } ely on {and∈ Stab(0) = MN,6 so } can be identied with G/MN. Let 0 be the vector (1, i, 0, . . . , 0). The stabilizer of 0 : C is equal to P = MAN, a parabolic subgroup of G. The group∈ MN is unimodular, so has a G-invariant measure which we shall © ª 0 denote by d. Let u be the map from to C dened by u() = [, ] = 1 + i2, u is MN-invariant and holomorphic, since is a complex subvariety of Cn. 0 Proposition 9.9. The MN-orbits on are: 0, = with C, t = { 0} ∈ { ∈ : u() = t for t C and = : u() = 0, / C if n 3. } ∈ 0,0 { ∈ } The open set = = : u() = 0 is dense in . The open set of t { 6 } t=0 [6 regular points for u is given by 0 = . So the singular points are the ∪ 0,0 points of 0 = 0,. =0 [6 9.2.2.3 MN-invariant distributions on

Denote by 0 () the space of distributions on , invariant with respect to MN. DMN Recall that for f () we can dene the function f on C by : ∈ D M f(t) = f()(u() t). M Z With the help of the results of Rallis-Schimann we want to describe the image of . For this we introduce the following space. M Denition 9.10. Let be the space of functions from C to C such that J

ϕ(t) = ϕ1(t) + (t)ϕ2(t) with ϕ , ϕ (C) and 1 2 ∈ D n 4 t if n is odd (t) = | |n 4 t Log t if n is even. ½ | | | | If we topologize as in [26]. is a map from () to a space , which is continuous and surjectivJ e. If S is aMcontinuous linearDform on , the distributionJ T dened by J T (f) = S( f) (f ()), M ∈ D so T = 0S, is invariant under MN. We shall see later on that 0 is not surjective.M M Proposition 9.11. Let T be a distribution on invariant under MN then

T = 0S + T M 1 where S is a continuous linear form on and T1 is a MN-invariant distribution on with Supp T . J 1 0 116 Chapter 9. Invariant Hilbert subspaces of the oscillator representation

Let us assume n 3 (as usual). Then clearly, if the support of f () is ∈ D contained in 0, the complement of , we have f (C). The restriction 0 M ∈ D M0 of to (0) has a image (C), and the transposed map 0 : 0(C) 0(0) M D D M0 D → D has as image 0 (0). Let T be a distribution on , invariant under MN. By the DMN above, the restriction T 0 to 0 is of the form T 0 = 0S0, where S0 is a distribution on C. But every distribution on C can be extendedMto a continuous linear form on . Let S be such an extension of S , and consider the distribution T dened by J 0 1 T = T 0S. 1 M

Since the distributions T and 0S coincide on 0, the support of T is contained M 1 in 0, so we get T = 0S + T . M 1 We are now going to study the structure of the MN-invariant distributions on with support in 0.

9.2.2.4 Singular MN-invariant distributions on

We use local coordinates in a neighborhood of 0 provided by the map:

A N → (a, n) an0 7→ We have: 1 1 1 1 2 (u + u ) 2i (u u ) 0 1 1 1 1 au = (u ) (u + ) 0 , u C  2i u 2 u  ∈ 0 0 In 2 iz   (if u = e , then we get the usual expression for az)

1 i t 1 + 2 [, ] 2 [, ] i i 1 t n 2 n = [, ] 1 [, ] , C .  2 2  ∈ i In 2   So we have: 1 u + u [, ] 0 1 aun = u u [, ] i .  2i  ¡ ¢ We denote by the dierential operator which is given by (in these coordinates):

∂2 ∂2 = 2 + + 2 . ∂3 ∂n Let T be a distribution on which is MN-invariant and whose support is contained in 0. Consider a function f in () which can be written, on the above local coordinates, as D

0 f(aun ) = (u, u)(, ) 9.2. The case n 3 117

n 2 where (C) and (C ), f = . For ∈xed,D the map ∈ D T ( ) → n 2 is a distribution on C , supported by the origin and SO(n 2, C)-invariant. Let us now assume that n > 3. (The case n = 3 has to be treated separately since in this case M = 1). We may then conclude that there are constants Ck,l such that m k k l l T ( ) = C (0) k,l kX=0 Xl=0 where the constants depend on : Ck,l = Tk,l(), and Tk,l are distributions on C: m k k l l T ( ) = T () (0), k,l kX=0 Xl=0 so we have: m k k l l T (u, ) = T (u) (). k,l kX=0 Xl=0 We are now going to use that T is also N-invariant. So let 1 1 t 1 + 2 [x, x] 2 i[x, x] i x 1 1 t n 2 n(x) = i[x, x] 1 [x, x] x , x C .  2 2  ∈ ix x In 2 For x small we get:   0 0 nxaun = au0 n0 . A simple calculation gives: x 0 = [, ] u 1 u0 = u 2[, x] + [, ][x, x]. u The distribution T is invariant under N if and only if ∂ ∂ < T, f n >= < T, f n >= 0 ∂x x ∂x x j ¯x=0 j ¯x=0 ¯ ¯ for j = 3, 4, . . . , n. ¯ ¯ ¯ ¯ Lemma 9.12. Let f be a ∞-function on . Set C 0 0 f(aun , aun ) = F (u, u, , ). We have: ∂ ∂F 1 ∂F f(n , n ) = 2 (u, u, , ) [, ] (u, u, , ) ∂x x x j ∂u u ∂ j ¯x=0 j ¯ ∂ ¯ ∂F 1 ∂F f(nx, nx)¯ = 2j (u, u, , ) [, ] (u, u, , ) ∂x ∂u u ∂ j ¯x=0 j ¯ ¯ ¯ 118 Chapter 9. Invariant Hilbert subspaces of the oscillator representation where = a n 0 and 3 j n. u So, using this lemma, we get: ∂F 1 ∂F < T, 2j [, ] > = 0 ∂u u ∂j ∂F 1 ∂F < T, 2j [, ] > = 0 ∂u u ∂j Suppose F is of the form F (u, u, , ) = (u, u)(, ). Then we get:

m k ∂ k l l 2 < T , > ( )(0) k,l ∂u j kX=0 Xl=0 ³ k l l ∂ + < Tk,l, > [, ] (0) = 0 u ∂j ¡ ¢ ´ and m k ∂ k l l 2 < T , > ( )(0) k,l ∂u j kX=0 Xl=0 ³ k l l ∂ + < Tk,l, > [, ] (0) = 0. u ∂j ¡ ¢ ´ Applying the relations:

k ∂ k 1 j = 2k ∂j k k 1 [, ] = 2k(2k + n 4) and similar relations for , we conclude:

m k ∂ ∂ l k l 1 4(k l) < Tk,l, > (0) ∂u ∂j kX=0 Xl=0 ³ ∂ l k l 1 + 2(k l) 2(k l) + n 4 < Tk,l, > (0) = 0 u ∂j ¡ ¢ ´ and m k ∂ ∂ l 1 k l 4l < Tk,l, > (0) ∂u ∂j kX=0 Xl=0 ³ ∂ l 1 k l + 2l(2l + n 4) < Tk,l, > (0) = 0. u ∂j ´ Therefore, we obtain the following two equations ∂T n 4 T k,l = k l + k,l for k > l ∂u 2 u µ ¶ 9.2. The case n 3 119 and ∂T n 4 T k,l = l + k,l for l 1, k. ∂u 2 u ∀ µ ¶ So we have:

n4 +k l n4 +l T = A u 2 u 2 if k > l, l 1 k,l k,l n4 +k = S (u) u 2 if k = l, l 1 k,k n4 +k = u 2 S (u) if l = 0, k > l k,0 where Sk,k, Sk,0 are distributions on C depending on u, u only respectively and Ak,l are constants. Now we consider the case n = 3. In this case M = 1 then we can not use the M-invariant as we did before. Let T be a distribution on which is MN-invariant and whose support is contained in . Consider a function f in () which can be written as 0 D 0 f(aun ) = (u, u)(, ) where (C) and (C), f = . For ∈xed,D the map ∈ D T ( ) → is a distribution on C, supported by the origin. We may then conclude that there are constants Ck,l such that

m k k l l k ∂ ∂ T ( ) = ( 1) Ck,l k l l (0) ∂ ∂ kX=0 Xl=0 where the constants depend on : Ck,l = Tk,l(), and Tk,l are distributions on C: m k k l l k ∂ ∂ T ( ) = ( 1) Tk,l k l l (0), ∂ ∂ kX=0 Xl=0 so we have: m k k l l ∂ ∂ T (u, ) = Tk,l(u) k l l (). ∂ ∂ kX=0 Xl=0 We are now going to use that T is also N-invariant. Using as before the last lemma, we get:

∂F 1 ∂F < T, 2 2 > = 0 ∂u u ∂ ∂F 1 ∂F < T, 2 2 > = 0 ∂u u ∂ 120 Chapter 9. Invariant Hilbert subspaces of the oscillator representation

Suppose F is of the form F (u, u, , ) = (u, u)(, ). Then we get:

m k k l l k ∂ ∂ ∂ ( 1) 2 < Tk,l, > k l l ()(0) ∂u ∂ ∂ kX=0 Xl=0 ³ k l l ∂ ∂ 2 ∂ + < Tk,l, > k l l (0) = 0 u ∂ ∂ ∂ ¡ ¢ ´ and

m k k l l k ∂ ∂ ∂ ( 1) 2 < Tk,l, > k l l ()(0) ∂u ∂ ∂ kX=0 Xl=0 ³ k l l ∂ ∂ 2 ∂ + < Tk,l, > k l l (0) = 0. u ∂ ∂ ∂ ¡ ¢ ´ Applying the relations:

k k 1 ∂ ∂ k = k k 1 ∂ ∂ k k 2 2 ∂ ∂ k = k(k 1) k 2 ∂ ∂ ∂k and similar relations for , we conclude: ∂k

m k k l 1 l k ∂ ∂ ∂ ( 1) 2(k l) < Tk,l, > k l 1 l (0) ∂u ∂ ∂ kX=0 Xl=0 ³ k l 1 l ∂ ∂ + (k l)(k l 1) < Tk,l, > k l 1 l (0) = 0 u ∂ ∂ ´ and

m k k l l 1 k ∂ ∂ ∂ ( 1) 2l < Tk,l, > k l l 1 (0) ∂u ∂ ∂ kX=0 Xl=0 ³ k l l 1 ∂ ∂ + l(l 1) < Tk,l, > k l l 1 (0) = 0. u ∂ ∂ ´ Therefore, we obtain the following two equations

∂T k l 1 T k,l = k,l for k > l ∂u 2 u µ ¶ and ∂T l 1 T k,l = k,l for l 1, k. ∂u 2 u ∀ µ ¶ 9.2. The case n 3 121

So we have:

kl1 l1 Tk,l = Ak,lu 2 u 2 if k > l, l 1 k1 = Sk,k(u) u 2 if k = l, l 1 k1 = u 2 S (u) if l = 0, k > l k,0 where Sk,k, Sk,0 are distributions on C depending on u, u only respectively and Ak,l are constants. Then we get the following theorem: Theorem 9.13. Let T be a MN-invariant distribution on with Supp T . 0 Then there exist distributions T0,0 = T0,0(u, u), Sk,k(u, u) = Sk,k(u), Sk,0(u, u) = Sk,0(u) on C and constants Ak,l such that

m k m k k1 ∂ k1 ∂ T = T0,0 + u 2 Sk,0(u) + Sk,k(u) u 2 ∂k ∂k k=1 k=1 X k X m k 1 2 l k l l k u du ∂ ∂ 2 +1 + Ak,l u 2 k l l for n = 3 | | u u ∂ ∂ kX=1 Xl=1 µ| |¶ | | and

m m 2 +k k 2 +k k T = T + u 2 S (u) + S (u) u 2 0,0 k,0 k,k kX=1 kX=1 m k 1 k 2l k+ u du k l l + A u for n > 3. k,l| | u u 2 kX=1 Xl=1 µ| |¶ | | 9.2.3 Invariant Hilbert subspaces of the oscillator representation 9.2.3.1 The oscillator representation

Let G = SL(2, C) SO(n, C) with n 3 and let ωn be the unitary representation of G on H = L2(Cn) dened by:

1 ω (g)f(z) = f(g z), g SO(n, C), n ∈ n a 0 ωn(g(a))f(z) = a f(az), g(a) = 1 , a C | | 0 a ∈ µ ¶ iRe (b[z,z]) 1 b ω (t(b))f(z) = e f(z), t(b) = , b C n 0 1 ∈ µ ¶ 2iRe ([z,w]) 0 1 ωn()f(z) = e f(w)dw, = , n 1 0 ZC µ ¶ where [z, w] = z1w1 + ... + znwn for z = (z1, ..., zn), w = (w1, ..., wn). We call ωn the oscillator representation of G. 122 Chapter 9. Invariant Hilbert subspaces of the oscillator representation

9.2.3.2 Invariant Hilbert subspaces of S0(Cn) Given a function f (Cn), we dene its Fourier transform fˆ by ∈ S

2iRe ([z,w]) fˆ(z) = f(w)e dw. n Z n This denition naturally gives rise to a Fourier transform on 0(C ). We denote nS Tˆ this Fourier transform of a tempered distribution T 0(C ). n ∈ S Let H be a Hilbert subspace of 0(C ), invariant under ωn(G), and let K be its reproducing kernel. By Schwartz’sSkernel theorem we can associate to it a unique tempered distribution T on Cn Cn. This distribution satises the following conditions: 1. T is a positive-denite kernel, in particular a Hermitian kernel, i.e. 2. T (x, y) = T(y, x). 3. T is ωn(G)-invariant : (ωn(g) ωn(g)) T = T for all g G. The latter property implies, in more detail, ∈ 3a. T (g x, g y) = T (x, y) for all g SO(n, C). ∈ 3b. If a C, then ∈ 2n T (ax, ay) = a T (x, y). | | 3c. If b C, then ∈ iRe (b([x,x] [y,y])) e T (x, y) = T (x, y).

3d. Tˆ(x, y) = T (x, y).

Supp T = (x, y) Cn Cn : [x, x] [y, y] = 0 . n,n { ∈ } Step II n n Let n,n0 = n,n (0, 0), being an isotropic cone in C C . In a neighborhood n n \ of n,n0 in C C we take as coordinates s = [x, x] [y, y] and ω n,n0 . So we can write locally there ∈ T = S (ω) (i)(s) i i I X∈ where I is some nite subset of IN and the Si are distributions on n,n0 . Applying condition 3c again, we get that only the term with i = 0 survives, so

T (x, y) = ([x, x] [y, y]) S (ω) 0 9.2. The case n 3 123

outside (x, y) = (0, 0). It remains to study the distribution S0 on n,n0 . Step III

On the open subset of n,n0 given by [x, x] = 0, we get by properties 3a and 3b, with = n 2, 6 S (x, y) = [x, x] (ω , ω ) 0 | | 0 1 2 where is an SO(n, C)-invariant distribution on , with = x Cn [x, x] 0 X X X { ∈ | = 1 . It is known that 0 is symmetric: 0(ω1, ω2) = 0(ω2, ω1) (see Section 9.2.1),} hence T (x, y) = T (y, x) on the open subset of Cn Cn dened by [x, x] = 0 (or, what is the same, [y, y] = 0). So S(x, y) = T (x, y) T (y, x) has support6 contained in [x, x] = [y, y] = 0.6 Step IV n Set n = x C : [x, x] = 0 . The distribution S has support in n n. With the coordinates{ ∈ s = [x, x], }s = [y, y] near , which can be tak en, 1 2 n n provided x = 0 and y = 0, so on 0 0 (in obvious notation), we get 6 6 n n S(x, y) = ([x, x], [y, y]) U( , ) ( , 0 ) 1 2 1 2 ∈ n where U is an SO(n, C)-invariant distribution on n0 n0 , homogeneous of degree 2 + 4. Since S(x, y) = S(y, x) it easily follows that U( , ) = U( , ). 1 2 2 1 Before we continue the preparation, we will recall now some structure theory of SO(n, C) and some results of Section 9.2.2 about distributions on n0 . We therefore assume (as in Section 9.2.2) n 3. Let P be the stabilizer in SO(n, C) of the line generated by 0 = (1, i, 0, . . . , 0). P = MAN is a maximal parabolic subgroup. The subgroup M consists of matrices of the form

1 0 0 0 1 0 0 0 m   where m is a matrix of the group SO(n 2, C). The group A is the one-parameter subgroup of matrices 1 1 1 (u + ) 1 (u ) 0 2 u 2i u a = 1 1 u  1 (u ) 1 (u + ) 0 2i u 2 u    0 0 1   where u C. The subgroup∈ N consists of matrices of the form

1 i t 1 + 2 [, ] 2 [, ] i i 1 t n = 2 [, ] 1 2 [, ]  i 1    n 2 where C . ∈ 124 Chapter 9. Invariant Hilbert subspaces of the oscillator representation

n 2 The group N is isomorphic to C . Moreover:

1 au n au = nu. Step V

Since SO(n, C) acts transitively on n0 , we can conclude that to U corresponds a MN-invariant distribution on n0 . Call it V . We recall some results of Section 9.2.2 about such distributions. 0 Dene u() = 1 + i2 on n0 and t = : u() = t for t C, 0, = 0 { } ∈ { } with C, = : u() = 0, / C . ∈ 0,0 { ∈ } The open set = = : u() = 0 is dense in 0. The open set of t { 6 } t=0 [6 regular points for u is given by 0 = . So the singular points are the ∪ 0,0 points of 0 = 0,. =0 [6 For f (0 ), the space of C∞-functions on 0 , one can dene the function ∈ D n c n f on C by M f(t) = f() (u() t) . M Z maps (0 ) continuously onto some topological vector space of functions on M D n J C with singularities at t = 0, see Section 9.2.2. And if W is a continuous linear form on , the distribution V dened on 0 by J n

V (f) = W ( f) (f (0 )), M ∈ D n i.e., V = 0W , is invariant under MN. 0 is not surjective. One has (Section M M 9.2.2, Proposition 9.11): if V is a distribution on n0 invariant under MN, then

V = 0W + V M 1 where W is a continuous linear form on and V is a MN-invariant distribution J 1 on n0 with support contained in 0. The structure of the distributions V1 is as follows. We use the local chart in a neighborhood of given by the map from A N to 0 , 0 n (a , n ) a n 0, u → u where N consists of the matrices

1 i t 1 + 2 [, ] 2 [, ] i i 1 t n 2 n = 2 [, ] 1 2 [, ] , C .  i 1  ∈   We denote by the dierential operator in this chart given by

∂2 ∂2 = 2 + + 2 . ∂3 ∂n 9.2. The case n 3 125

We quote 9.2.2, Theorem 9.13 here:

Let V1 be a MN-invariant distribution on n0 with support in 0. Then there exist distributions T0,0, Sk,k, Sk,0 on C and constants Ak,l such that, in the above chart,

m k m k k1 ∂ k1 ∂ V1 = T0,0 + u 2 Sk,0(u) + Sk,k(u) u 2 ∂k ∂k k=1 k=1 X k X m k 1 2 l k l l k u du ∂ ∂ 2 +1 + Ak,l u 2 k l l for n = 3 | | u u ∂ ∂ kX=1 Xl=1 µ| |¶ | | and m m 2 +k k 2 +k k V = T + u 2 S (u) + S (u) u 2 1 0,0 k,0 k,k kX=1 kX=1 m k 1 k 2l +k u du k l l + A u for n > 3 k,l | | u u 2 kX=1 Xl=1 µ| |¶ | | where is the Dirac measure at = 0. Step VI We return to our distribution V from Step V, and write it in the form V = 0W + V , as above. By abuse of notation we have M 1 V () = V (g0) = U(g0, 0). 0 0 0 0 1 0 Because u() = u(g ) = [g , ] satises u(g ) = u(g ), we easily get that 0 1 0 0W (g ) = 0W (g ). M M 0 1 0 0 0 1 0 Since V (g ) = V (g ), we see that 2V (g ) = V1(g ) V1(g ). Now 1 0 0 0 V (au0 gau0 ) = U(gau0 , au0 ) = 0 0 2+4 0 U(u0g , u0 ) = u0 V (g ). | | 0 0 1 0 Let us apply this property to the distribution V2(g ) = V1(g ) V1(g ), 0 supported by 0. We get with g = aun , that 1 1 au0 aunau0 = auau0 nau0 = aunu0, hence m k 0 2 2 k k1 ∂ V (a n ) = u T + u u u 2 S (u) 2 u u0 | 0| 0,0 | 0| 0 k,0 ∂k kX=1 m k 2 k k1 ∂ + u0 u Sk,k(u) u 2 | | 0 ∂k k=1 X k m k 1 2 l k l l 2 (k l) l k +1 u du ∂ ∂ 2 + u0 u0 u0 Ak,l u 2 k l l for n = 3 | | | | u u ∂ ∂ kX=1 Xl=1 µ| |¶ | | 126 Chapter 9. Invariant Hilbert subspaces of the oscillator representation and

m 0 2 2 2k 2 +k k V (a n ) = u T + u u u 2 S (u) 2 u u0 | 0| 0,0 | 0| 0 k,0 k=1 m X 2 2k 2 +k k + u u S (u) u 2 | 0| 0 k,k kX=1 m k 1 k 2l 2 2(k l) 2l +k u du k l l + u u u A u for n > 3. | 0| 0 0 k,l | | u u 2 kX=1 Xl=1 µ| |¶ | | 0 2+4 0 Since V (a n ) = u V (a n ), we get V = 0. So V = 0, and hence 2 u u0 | 0| 2 u 2 S = 0 in a neighborhood of n0 n0 , and therefore Supp S (0) n n (0) . Step VII { }∪{ } From property 3c we conclude that

([x, x] [y, y])k S = 0 for k = 1, 2, . . . , hence, combining it with property 3d, we have

¤k S = 0 for k = 1, 2, . . ., in particular ¤ S = 0. Here ¤ denotes the d’Alembertian

∂2 ∂2 ∂2 ∂2 2 + + 2 2 2 . ∂x1 ∂xn ∂y1 ∂yn

If S has support in (0) n n (0) , and ¤ S = 0, it easily follows, by recalling the local structure{ of} suc∪ {h distributions, } being basically a nite linear combination of tensor products of distributions supported by n and the origin, that S = 0 (see [28],Ch.III, 10). So nally we have shown that T (x, y) = T (y, x). § 9.2.3.3 Multiplicity free decomposition of the oscillator representation The multiplicity free decomposition of the oscillator representation, i.e. the multiplicity free decomposition into irreducible invariant Hilbert subspaces of any n ωn(G)-invariant Hilbert subspace of 0(C ), is now easily proved by applying Cri- S n terion 5.6 with JT = T. If H is any invariant Hilbert subspace of 0(C ) with n n S reproducing kernel T 0(C C ), then T is the kernel of the space J H and the property J H = H comes∈ S down to T(x, y) = T (x, y). Since T is positive-denite, this is equivalent with T (x, y) = T (y, x) which immediately follows from the last section. So we have the following result. n Theorem 9.14. Any ωn(G)-invariant Hilbert subspace of 0(C ) decomposes mul- S n tiplicity free into irreducible invariant Hilbert subspaces of 0(C ). S APPENDIX A Conical distributions associated with the orthogonal complex group SO(n, C), n 3

The group G acts transitively on the isotropic cone of the quadratic form associated with it. The action of G in the space of homogeneous functions on this cone denes a family of representations of G. Their study leads to the conical distributions. In this appendix we compute the conical distributions in the case of the group SO(n, C) with n 3. We follow the same method as in [9]. A.1 The cone = G/MN

Let G be the orthogonal group SO(n, C) of the quadratic form [z, z] = z2 + + z2 . 1 n Let P be the stabilizer of 0 : C with 0 = (1, i, 0, . . . , 0). The subgroup P is a maximal parabolic subgroup and∈ is equal to P = MAN where M is the subgroup of matrices of the form© ª 1 0 0 0 0 1 0 0 0 0  . . . .  . . h    0 0    with h SO(n 2, C), ∈ cos z sin z 0 A = a = sin z cos z 0 : with z C  z   ∈  0 0 In 2       127 128 Appendix A. Conical distributions of SO(n, C), n 3 and 1 1 t 1 + 2 [p, p] 2 i[p, p] i p 1 1 t n 2 N = np = i[p, p] 1 [p, p] p : p C .   2 2  ∈  ip p In 2     Let s, be the character of C dened by 

s,(z) = z with , s C. The representation s, of G induced by s, can be realized in a space∈of homogeneous∈ functions dened on the cone:

= Cn : = 0, [, ] = 0 . { ∈ 6 } G acts transitively on and Stab 0 = MN, so can be identied to G/MN. The space of s, is called Es, and consists of complex-valued ∞-functions f on satisfying C

1 (g)f() = f(g ) ( , g G; f E ). s, ∈ ∈ ∈ s, The group MN is unimodular, so has a G-invariant measure which we shall 0 denote by d. Let u be the map from to C dened by u() = [, ] = 1 + i2, u is MN-invariant and holomorphic, since is a complex subvariety of Cn. 0 Proposition A.1. The MN-orbits on are: 0, = with C, t = { 0} ∈ { ∈ : u() = t for t C and = : u() = 0, / C if n 3. } ∈ 0,0 { ∈ }

The open set = = : u() = 0 is dense in . The open set of t { 6 } t=0 [6 regular points for u is given by 0 = . So the singular points are the ∪ 0,0 points of 0 = 0,. =0 [6 A.2 Denition of conical distribution

Let s, be the character of C dened by

(z) = zs, with , s C. s, ∈ ∈ A function f dened on is -homogeneous if

f(z) = (z) z f(). s, | | A.3. The function 129 M

Let f be a continuous function on , -homogeneous. We can dene the distribution T on by T() = f()()d Z where d is the invariant measure on . Calling () = ( ) z z we have d(z) = z 2d and | | T( ) = z 2 f(z)()d = z (z)T(). z | | | | s, Z Denition A.2. A distribution T on is called -homogeneous if T( ) = (z) z T(). z s, | | Denition A.3. A distribution T on is called -conical if (1) T is -homogeneous (2) T is MN-invariant.

We can associate to 0 the -conical distribution ,0 dened as follows

0 du < ,0, f > = f(u )(u) u . | | u 2 ZC | | We call k 2 l k l l k +1 u du ∂ ∂ Rk,l = u 2 | | u u 2 ∂k l ∂l µ| |¶ | | which is a ( k , k + l)-conical distribution with support on if n = 3 and 2 2 0 k 2l +k u du k l l L = u k,l | | u u 2 µ| |¶ | | which is ( k, k + 2l)-conical distribution with support on if n > 3. 0 A.3 The function M

We can dene the integral of a function f () on as follows: exists a ∈ D t continuous function f on C, integrable on C, such that for all continuous function F on C, M F (u())f()d = F (t) f(t)dt M Z ZC with f(t) = f()(u() t). M Z 130 Appendix A. Conical distributions of SO(n, C), n 3

The function f ∞(C) and has a singularity in t = 0. This is due to the critical pointsMof the∈ functionC u() which are the points of . → 0 We will use the following chart in a neighborhood of 0.

A N → (a, n) an0 → We have:

1 1 1 1 2 (u + u ) 2i (u u ) 0 1 1 1 1 au = (u ) (u + ) 0 , u C  2i u 2 u  ∈ 0 0 In 2   iz (if u = e , then we get the usual expression for az)

1 i t 1 + 2 [, ] 2 [, ] i i 1 t n 2 n = [, ] 1 [, ] , C .  2 2  ∈ i In 2   So we have: 1 u + u [, ] 0 1 aun = u u [, ] i  2i  ¡ ¢ and   2 u(a n 0) = [, ]. u u In this chart the invariant measure is

2(n 1) du d = 2 d. u 2 | | Let f be a function in () whith the support contains in AN0. Calling D f(u, ) = f(a n 0)([, ] ) N u Z such that, if F is a continuous function on C

2(n 1) 2 0 du F (u())f()d =2 F ( [, ])f(aun ) 2 d A N u u Z Z | | tu = F (t) f(u, )dudt C C N 2 Z and tu f(t) = f(u, )du. M N 2 ZC We summarize here some results of Rallis-Schimann [26]. A.4. Conical distributions with support on 0 131

Let Q be a non-degenerate quadratic form

Q(z) = z2 + + z2 1 n and let f be a function of (Cn). There exists a function f on C such that for all continuous function F onD C N

F (Q(z))f(z)dz = F (t) f(t)dt. n N ZC ZC Denition A.4. Let be the space of functions from C to C such that J

ϕ(t) = ϕ1(t) + (t)ϕ2(t) with ϕ , ϕ (C) and 1 2 ∈ D n 2 t if n is odd (t) = | |n 2 t Log t if n is even. ½ | | | | Calling ∂u+v Bu,v(ϕ) = ϕ2(0) ∂tu∂tv v if u, v = 0, 1, 2, 3, . . . , we obtain that B ( f) is a multiple of u . u,v N

A.4 Conical distributions with support on 0 Proposition A.5. A. The case n=3. (1) If s = k , with k = 1, 2, . . . . 2 a) If s and satisfy s + = k and s = k with k = 3, 4, . . . the space of -conical distributions with support on 0 has dimension 4. These distributions are as follows

k1 1 k 1+ k k1 C1,0 + C2u 2 u 2 4 + C3u 2 4 u 2 + C4R k . k,+ 2 b) If s and satisfy s + = k and s = k with k = 3, 4, . . . the 6 space of -conical distributions with support on 0 has dimension 3. These distributions are as follows

k1 1 k C1,0 + C2u 2 u 2 4 + C3R k . k,+ 2 c) If s and satisfy s + = k and s = k with k = 3, 4, . . . the 6 space of -conical distributions with support on 0 has dimension 3. These distributions are as follows

1+ k k1 C1,0 + C2u 2 4 u 2 + C3R k . k,+ 2 132 Appendix A. Conical distributions of SO(n, C), n 3

d) If s and satisfy s + = k and s = k with k = 3, 4, . . . the 6 6 space of -conical distributions with support on 0 has dimension 2. These distributions are as follows

C1,0 + C2R k . k,+ 2 (2) If s = k , with k = 1, 2, . . . . 6 2 a) If s and satisfy s + = k with k = 3, 4, . . . the space of - conical distributions with support on 0 has dimension 2. These distributions are as follows

k1 1+s C1,0 + C2u 2 u 2 . b) If s and satisfy s = k with k = 3, 4, . . . the space of - conical distributions with support on 0 has dimension 2. These distributions are as follows

s+1+ k1 C1,0 + C2u 2 u 2 .

c) Otherwise the space of -conical distributions with support on 0 has dimension 1. These distributions are as follows

C1,0. B. The case n > 3. (1) If s = k, with k = 1, 2, . . . . a) If s and satisfy s+ = 2k and s = 2k with k = 2, 3, . . . the space of -conical distributions with support on 0 has dimension 4. These distributions are as follows

2 +k k k++ 2 +k C1,0 + C2u 2 u 2 + C3u 2 u 2 + C4L +k . k, 2 b) If s and satisfy s+ = 2k and s = 2k with k = 2, 3, . . . the 6 space of -conical distributions with support on 0 has dimension 3. These distributions are as follows

2 +k k C1,0 + C2u 2 u 2 + C3L +k . k, 2 c) If s and satisfy s+ = 2k and s = 2k with k = 2, 3, . . . the 6 space of -conical distributions with support on 0 has dimension 3. These distributions are as follows

k++ 2 +k C1,0 + C2u 2 u 2 + C3L +k . k, 2 c) If s and satisfy s+ = 2k and s = 2k with k = 2, 3, . . . the 6 6 space of -conical distributions with support on 0 has dimension 2. These distributions are as follows

C1,0 + C2L +k . k, 2 A.4. Conical distributions with support on 0 133

(2) If s = k with k = 1, 2, . . . . 6 a) If s and satisfy s + = 2k with k = 2, 3, . . . the space of - conical distributions with support on 0 has dimension 2. These distributions are as follows

2 +k k C1,0 + C2u 2 u 2 .

b) If s and satisfy s = 2k with k = 2, 3, . . . the space of - conical distributions with support on 0 has dimension 2. These distributions are as follows

k++ 2 +k C1,0 + C2u 2 u 2 .

c) Otherwise the space of -conical distributions with support on 0 has dimension 1. These distributions are as follows

C1,0.

Proof. Let T be a -conical distribution with support on 0. Since T is MN- invariant, Theorem 9.13 of Section 9.2.2 can be applied and we have:

m k m k k1 ∂ k1 ∂ T = T0,0 + u 2 Sk,0(u) + Sk,k(u) u 2 ∂k ∂k k=1 k=1 X k X m k 1 2 l k l l k u du ∂ ∂ 2 +1 + Ak,l u 2 k l l for n = 3 | | u u ∂ ∂ kX=1 Xl=1 µ| |¶ | | and

m m 2 +k k 2 +k k T = T + u 2 S (u) + S (u) u 2 0,0 k,0 k,k kX=1 kX=1 m k 1 k 2l k+ u du k l l + A u for n > 3 k,l| | u u 2 kX=1 Xl=1 µ| |¶ | | where T0,0 = T0,0(u, u), Sk,k(u, u) = Sk,k(u), Sk,0(u, u) = Sk,0(u) are distributions on C and Ak,l constants. If f is a function of () such that D 0 f() = f(aun ) = (u)()

n 2 where (C) and (C ), then ∈ D ∈ D u f () = f = = (u) (). µ ¶ ³ ´ ³ ´ 134 Appendix A. Conical distributions of SO(n, C), n 3

If follows that

< T , >= +s < T , > (A.5) 0,0 | | 0,0 µ| |¶ 2 +k +s 2k 2 +k = (A.6) k,0 | | k,0 µ| |¶ 2 +k +s 2k 2 +k < S (u) u 2 , >= < S (u) u 2 , > (A.7) k,k | | k,k µ| |¶ A.5. Conical distributions 135

k+2l k +s A = A . (A.8) k,l| | k,l| | µ| |¶ µ| |¶

Let us rst assume that n = 3. Then we get from equation (A.1) that T0,0 = C,0. Equation (A.2) leads to s + = k 2 and Sk,0 is a distribution on C s+1 homogeneous of degree 2 , i.e.

s+1 Sk,0 = Cu 2 where C is a constant. From equation (A.3) one nds that s = k 2 and Sk,k is a distribution on s+1+ C homogeneous of degree 2 , i.e.

s+1+ Sk,k = Cu 2 where C is a constant. Finally, from equation (A.4) we derive that A = 0 if (k, l) = ( 2s, s). k,l 6 For the case n > 3 we have from equation (A.5) that T0,0 = C,0. Equation (A.6) leads to s + = 2k 2 and that Sk,0 is a distribution on C homogeneous s+ of degree 2 , i.e. s+ Sk,0 = Cu 2 where C is a constant. Equation (A.7) shows that s = 2k 2 and that Sk,k is a distribution on C s++ homogeneous of degree 2 , i.e.

s++ Sk,k = Cu 2 where C is a constant. s Finally, equation (A.8) results in Ak,l = 0 if (k, l) = ( s, 2 ). Therefore we have proven the proposition.¤ 6

A.5 Conical distributions

Let be a function of and Re s > 1 then we dene J 1 s z Z () = z (z)dz. s + +2 C | | z 2| | Z µ| |¶ ³ ´ If (C), the function s Z () can be extended to an entire function on C. The ∈distributionD Z is (s ,→)-homogeneous. All distributions which are (s , )- homogeneous on C are proportional to Z . If , the function s Z () can be extended to a meromorphic function on C with∈ Jsimple poles in s =→ 2k 2 with k = 0, 1, 2, . . . . | | 136 Appendix A. Conical distributions of SO(n, C), n 3

Let be a function in , can be written as follows J

∞ j k ∞ j k (z) = aj,k()z z + (z, z) bj,k()z z j=0 jX,k=0 X where n 2 z if n is odd (z, z) = | |n 2 z Log z if n is even. ½ | | | | First I assume that n is odd. If > 0 the residue of Z in s = 2k 2 for k = 0, 1, 2, . . . is | | 4i Res (Z , s = 2k 2) = bk,k+. 2k 2 ³ ´ If < 0 the residue of Z in s = 2k 2 for k = 0, 1, 2, . . . is | | 4i Res (Z , s = 2k + 2) = bk ,k. 2k 2 ³ ´ Finally, if n is even. If > 0 the residue of Z in s = 2k 2 for k = 0, 1, 2, . . . is | | 1 0 2k Res (Z , s = 2k 2) = 4i b . 2 k,k+ µ ¶ µ ¶ If < 0 the residue of Z in s = 2k 2 for k = 0, 1, 2, . . . is | | 1 0 2k Res (Z , s = 2k + 2) = 4i bk ,k. 2 µ ¶ µ ¶ For the above computations see [11]. For Re s > 1 we have 1 s u() < 0Z , f >= u() f()d. s + +2 M | | u() 2| | Z µ| |¶ ³ ´ Calling ,1 = 0Z . Therefore, the function s ,1 can be meromorphically extended to C withM poles at s = 2k 2 for→k = 0, 1, 2, . . . . The residues in these poles are | |

Res ( , 2k 2) = 0 (Res (Z , 2k 2)) . ,1 | | M | | These residues are equal to

Res ( , 2k 2) = L if 0 ,1 k, 2k+,k+ and Res (,1, 2k + 2) = k,L2k ,k if < 0 A.5. Conical distributions 137

where k, is a constant dierent of 0. If s = 2 is a pole of then ,1 Res ( , 2) = L ,1 0,0 0,0 where 0,0 is a constant dierent of 0. The distribution ˜ dened by

0,0 ˜ = lim ,1 ,0 s 2 s + 2 → µ ¶ is ( 2, 0)-conical. Proposition A.6. A. The case n=3. (1) If s = k , with k = 1, 2, . . . . 2 a) If s is not a pole of ,1. i) If s and satisfy s + = k and s = k with k = 3, 4, . . . the space of -conical distributions has dimension 5. These distributions are as follows

k1 1 k 1+ k k1 C1,1 +C2,0 +C3u 2 u 2 4 +C4u 2 4 u 2 +C5R k . k,+ 2 ii) If s and satisfy s + = k and s = k with k = 3, 4, . . . the space of -conical distributions has6 dimension 4. These distributions are as follows

k1 1 k C1,1 + C2,0 + C3u 2 u 2 4 + C4R k . k,+ 2 iii) If s and satisfy s + = k and s = k with k = 3, 4, . . . the space of -conical 6distributions has dimension 4. These distributions are as follows

1+ k k1 C1,1 + C2,0 + C3u 2 4 u 2 + C4R k . k,+ 2

iv) If s and satisfy s + = k and s = k with k = 3, 4, . . . the space of -conical 6distributions has6 dimension 3. These distributions are as follows

C1,1 + C2,0 + C3R k . k,+ 2

b) If s is a pole of ,1. i) If s = 2 the space of -conical distributions has dimension 2. These distributions are as follows

= C1,1 + C2˜. 138 Appendix A. Conical distributions of SO(n, C), n 3

ii) If s = 2. 6 ii.1) If s and satisfy s+ = k and s = k with k = 3, 4, . . . the space of -conical distributions has dimension 4. These distributions are as follows

k1 1 k 1+ k k1 C1,0 + C2u 2 u 2 4 + C3u 2 4 u 2 + C4R k . k,+ 2

ii.2) If s and satisfy s+ = k and s = k with k = 3, 4, . . . the space of -conical has dimension 63.These distributions are as follows

k1 1 k C1,0 + C2u 2 u 2 4 + C3R k . k,+ 2

ii.3) If s and satisfy s+ = k and s = k with k = 3, 4, . . . the space of -conical 6 distributions has dimension 3. These distributions are as follows

1+ k k1 C1,0 + C2u 2 4 u 2 + C3R k . k,+ 2

ii.4) If s and satisfy s+ = k and s = k with k = 3, 4, . . . the space of -conical 6 distributions has6 dimension 2. These distributions are as follows

C1,0 + C2R k . k,+ 2

(2) If s = k , with k = 1, 2, . . . . 6 2 a) If s and satisfy s+ = k with k = 3, 4, . . . the space of -conical distributions has dimension 3. These distributions are as follows

k1 1+s C1,1 + C2,0 + C3u 2 u 2 .

b) If s and satisfy s = k with k = 3, 4, . . . the space of -conical distributions has dimension 3. These distributions are as follows

s+1+ k1 C1,1 + C2,0 + C3u 2 u 2 .

c) Otherwise the space of -conical distributions has dimension 2. These distributions are as follows

C1,1 + C2,0.

B. The case n > 3.

(1) If s = k, with k = 1, 2, . . . . a) If s is not pole of ,1. A.5. Conical distributions 139

i) If s and satisfy s+ = 2k and s = 2k with k = 2, 3, . . . the space of -conical distributions has dimension 5. These distributions are as follows

2 +k k k++ 2 +k C1,1 + C2,0 + C3u 2 u 2 + C4u 2 u 2 +

C5L +k . k, 2 ii) If s and satisfy s+ = 2k and s = 2k with k = 2, 3, . . . the space of -conical distributions has6 dimension 4. These distributions are as follows

2 +k k C1,1 + C2,0 + C3u 2 u 2 + C4L +k . k, 2 iii) If s and satisfy s+ = 2k and s = 2k with k = 2, 3, . . . the space of -conical6 distributions has dimension 4. These distributions are as follows

k++ 2 +k C1,1 + C2,0 + C3u 2 u 2 + C4L +k . k, 2 iv) If s and satisfy s+ = 2k and s = 2k with k = 2, 3, . . . the space of -conical6 distributions has6 dimension 3. These distributions are as follows

C1,1 + C2,0 + C3L +k . k, 2

b) If s is a pole of ,1. i) If s = 2 the space of -conical distributions has dimension 2 and these distributions are as follows

= C1,1 + C2˜. ii) If s = 2. 6 ii.1) If s and satisfy s + = 2k and s = 2k with k = 2, 3, . . . the space of -conical distributions hasdimension 4. These distributions are as follows

2 +k k k++ 2 +k C1,0 + C2u 2 u 2 + C3u 2 u 2 + C5L +k . k, 2 ii.2) If s and satisfy s + = 2k and s = 2k with k = 2, 3, . . . the space of -conical distributions 6hasdimension 3. These distributions are as follows

2 +k k C1,0 + C2u 2 u 2 + C3L +k . k, 2 ii.3) If s and satisfy s + = 2k and s = 2k with k = 2, 3, . . . the space of -conic6 al distributions hasdimension 3. These distributions are as follows

k++ 2 +k C1,0 + C2u 2 u 2 + C3L +k . k, 2 140 Appendix A. Conical distributions of SO(n, C), n 3

ii.4) If s and satisfy s + = 2k and s = 2k with k = 2, 3, . . . the space of -conic6 al distributions 6hasdimension 2. These distributions are as follows

C1,0 + C2L +k . k, 2 (2) If s = k with k = 1, 2, . . . . 6 a) If s and satisfy s + = 2k with k = 2, 3, . . . the space of - conical distributions has dimension 3. These distributions are as follows 2 +k k C1,1 + C2,0 + C3u 2 u 2 . b) If s and satisfy s = 2k with k = 2, 3, . . . the space of - conical distributionshas dimension 3. These distributions are as follows k++ 2 +k C1,1 + C2,0 + C3u 2 u 2 . c) Otherwise the space of -conical distributions has dimension 2. These distributions are as follows

C1,1 + C2,0.

Proof. Let be equal to (s, ) and let be a -conical distribution. We call 0 the restriction to 0. The distribution 0 is MN-invariant and -homogeneous. There exists a distribution S0 on C which is (s , )-homogeneous such that 0 = 0(S0). M The distribution S0 is proportional to Z , i.e. S0 = C1Z . a) If s is not a pole of then the distribution C is -conical and ,1 1 ,1 with support in 0. We only have to apply last proposition to get the result. 0 b) If s is a pole of ,1. Then s = 2k 2 for k = 0, 1, . . . and let Z be equal to | | 0 d Z = (s + 2k + + 2)Z s= 2k 2. ds | | | | ¯ t Z0 is not an homogeneous linear form. Calling¯ f (t) = f its follows µ ¶ s +2 Z (f ) = Z (f) | | µ| |¶ and 0 2k 0 Z (f ) = | | Z (f) | | µ| |¶ 2k + | | Log Res (Z (f), 2k 2). | | | | | | µ| |¶ A.5. Conical distributions 141

0 0 Calling = 0(Z ) we get ,1 M 0 2k 2+ 0 (f ) = | | (f) ,1 | | ,1 µ| |¶ 2k 2+ + | | Log Res ( (f), 2k 2). | | | | ,1 | | µ| |¶ The distribution = C 0 is MN-invariant and satises 1 1 ,1 (f ) () (f) = C ()Log Res ( (f), 2k 2). 1 | | 1 1| | | | ,1 | | Proposition A.7. Let be equal to ( 2k 2, ) with k = 0, 1, . . . . Let T be a distribution on such that | | (1) T is MN-invariant

(2) T has support in 0 (3) T satises

T (f) ()T (f) = C ()Log L2k ,k(f) if < 0. | | | | | | If k = 0 or = 0 then C = 0 and T is -conical. If k = 0 and = 0 then 6 d 6 T C is -conical. ds ,0 ¯s= 2 ¯ For the pro¯of we follow the same method as in [9] using Theorem 9.13 of Section ¯ 9.2.2. From this proposition we get the result. If k = 0 or = 0 then C = 0 and 6 6 1 is a conical distribution with support on 0. If k = 0 and = 0 we obtain,

d C 0 = C + C 1 ,1 1 0,0 ds ,0 0 ,1 ¯s= 2 ¯ ¯ and ¯

0,0 ˜ = lim ,1 ,0 s 2 s + 2 → µ ¶ d = 0 .¤ ,1 0,0 ds ,0 ¯s= 2 ¯ ¯ ¯ 142 Appendix A. Conical distributions of SO(n, C), n 3 APPENDIX B Irreducibility and unitarity

In this appendix we study the irreducibility and unitarity of the representations of SO(n, C) induced by a maximal parabolic subgroup and dened in Section 8.3.1 in more detail. We follow the same method as in [41].

B.1 The standard minimal parabolic subgroup

We refer to [19]. Let G be the group SO(n, C) with n 3 and let g be the Lie algebra with Cartan decomposition g = k ik. Let a0 be the maximal abelian subalgebra of ik where

0 ix1 ix1 0  0 ix    2   ix2 0     a0 =  .  , xi IR  .  ∈   .       0 ixk   ixk 0       0        if n = 2k + 1and 

0 ix1 ix 0 1  0 ix2     a =  ix2 0  , x IR 0   i ∈   ..    .    0 ixk     ixk 0         if n = 2k.  

143 144 Appendix B. Irreducibility and unitarity

Let m0 be the centralizer of a0 in k, then it is equal to

0 y1 y1 0  0 y    2   y2 0     m0 =  .  , yi IR  .  ∈   .       0 yk   yk 0       0        if n = 2k + 1 and 

0 y1 y 0 1  0 y2     m =  y2 0  , y IR 0   i ∈   ..    .    0 yk     yk 0         if n = 2k.   Let hj be the function such that hj(H) = xj, 1 j k where H a0. The set of roots is ∈

= h h with i = j h if n = 2k + 1 { i j 6 } ∪ { l} and = h h with i = j if n = 2k. { i j 6 } The root space decomposition is

g = a m g 0 0 X∈ where g = CH . To dene H , rst let i < j and = h h . Then H is zero i j except in the entries corresponding to the ith and jth pairs of indices, where it is equal to

i j

0 X H = t X 0 µ ¶ with 1 i 1 i Xhi hj = , Xhi+hj = , i 1 i 1 µ ¶ µ ¶ 1 i 1 i X hi+hj = , X hi hj = . i 1 i 1 µ ¶ µ ¶ B.1. The standard minimal parabolic subgroup 145

To dene H for = h in case n = 2k + 1, write l pair entry l 2k + 1

0 X H = t X 0 µ ¶ with zeros elsewhere and with 1 1 Xhl = and X hl = . i i µ ¶ µ ¶ The set of positive roots is

+ = h h with i < j h if n = 2k + 1 { i j } ∪ { l} and + = h h with i < j if n = 2k. { i j } We dene n0 = g and p0 = m0 + a0 + n0. We call P0 = M0A0N0 the + X∈ standard minimal parabolic subgroup where A0 = exp a0, N0 = exp n0 and M0 is the centralizer of a0 in K = SO(n, IR). The set of simple roots is

= h1 h2, h2 h3, . . . , hk 1 hk, hk if n = 2k + 1 { } and = h1 h2, h2 h3, . . . , hk 1 hk, hk 1 + hk if n = 2k. { } For any subset F let PF = MF AF NF be the Langlands decomposition of the parabolic subgroup P associated to F , then P P , M M , A A F 0 F 0 F F 0 and NF N0. For details on parabolic subgroups see [42]. Taking F = h1 h we get if n = 2k or n = 2k + 1 and n > 4 { 2} 0 ix a = ix 0 with x IR F   ∈   0    and if n = 4   0 ix 0 0 ix 0 0 0 aF = with x IR .  0 0 0 ix ∈     0 0 ix 0    If n = 3 then PF = P0.   We dene = IN F , in our case = F , then F ∩ F

m1F = m0 + a0 + g, nF = g + F F X∈ ∈ X\ 146 Appendix B. Irreducibility and unitarity and pF = m1F + nF .

Let mF be the orthogonal complement of aF in m1F with respect the inner product < X, Y >= Re F (X, Y ), where F is the Killing form dened in Section 8.3.1.6, then m1F = mF + aF .

Let M1F be the centralizer of aF in G, NF = exp nF , AF = exp aF , M1F = MF AF and PF = MF AF NF . We can see that PF = P where P = MAN is a parabolic subgroup with M, A and N dened in Section 8.3.1. Put = n 2 and dene for s C, ∈ ∈ ix (s )y E = f ∞(G) : f(gma n) = e e f(g), ,s { ∈ C x+iy for all (g, m, a , n) G M A N . x+iy ∈ } B.2 Irreducibility

To prove the irreducibility we are going to follow the same method as in [41]. First we will study the irreducibility of E0,s, by determining sucient conditions under which the, up to scalars, unique K-xed vector is cyclic in E0,s. Put s = sh1, = (n 2)h then it is easily seen that 1 1

1 (s+ ) log a E = f ∞(G) : f(gman) = (m )e 1 f(g), ,s { ∈ C for all (g, m, a, n) G M A N . ∈ F F F } where cos x sin x ix (m) = sin x cos x = e  t   where t SO(n 2, C). ∈ We have endowed E,s with a pre-Hilbert space structure with norm

f 2 = f(k) 2dk || || | | ZK where K = SO(n, IR) and dk the normalized Haar measure on K. Furthermore the sesqui-linear form < f, h >= f(k)h(k)dk (B.1) ZK denes a non-degenerate G-invariant pairing between E,s and E, s with s C, . ∈ ∈ Now dene for a arbitrary ∈ 0c 1 (+ ) log a E (G/P , ) = f ∞(G) : f(gman) = (m )e 0 f(g), 0 { ∈ C for all (g, m, a, n) G M A N . ∈ 0 0 0} B.2. Irreducibility 147

Dene the following special elements of a0c:

k 1 = 2 (k i)h if n = 2k, 0 i i=1 X k 1 = (2(k i) + 1) h + h if n = 2k + 1, 0 i k i=1 X (s) = + (s C) if n = 2k or n = 2k + 1. s 1 0 ∈ These elements can also be written as

hi = ei, (B.2) (s) = (s, 2k + 4, 2k + 6, . . . , 2, 0) if n = 2k (s) = (s, 2k + 3, 2k + 5, . . . , 3, 1) if n = 2k + 1, k = 1 6 (s) = (s) if n = 3 whith respect to e : i = 1, . . . , k , the standard basis of Ck. { i } Lemma B.1. Let s C, then E E (G/P , (s)), . ∈ ,s 0 ∈

By the duality (B.1) we deduce

Corollary B.2. Let s C, then E, s is a quotient of E(G/P0, (s)) for . ∈ ∈ For a dene ∈ 0c

1 (, ) 1 (, ) e() = 1 + + , 2(, ) 2 2(, ) + µ ¶ µ ¶ Y∈ where ( , ) is the usual invariant bilinear form for SO(n, C). e() is the denominator of Harish-Chandra’s c-function, see [14]. The Killing form is up to a constant equal to the invariant bilinear form ( , ) for SO(n, C), but this formula is independent of that constant. The Iwasawa decomposition of G is given by G = KA0N0. Dene

(+ ) log a 1 (kan) = e 0 (k K, a A , n N ), ∈ ∈ 0 ∈ 0 then 1 E (G/P , ). ∈ 0 0 Theorem B.3. (Helgason [14]) 1 is a cyclic vector in E0(G/P0, ) if and only if e() = 0.

Let us call even the set of even integers and odd the set of odd integers. Lemma B.4. e( ( s)) = 0 for 6 a) for all s C ( ( , 0)) when n = 3, 5

∈ \ ∩ ∞

b) for all s C ( even ( , 0)) when n = 4 ∈ \ ∩ ∞ 148 Appendix B. Irreducibility and unitarity

c) for all s C ( ( , n 6]) when n is even > 4

∈ \ even ∩ ∞

d) for all s C with s / odd ( , n 6] and s / even ( , 0) when n is odd > ∈5. ∈ ∩ ∞ ∈ ∩ ∞

Proof. Using the identication (B.2) one easily gets for n = 2k

j i + 2k i = 1, j = k s 6 6 (( s), ei + ej ) 2 i = 1, j = k =  s+2(k j) , (e + e , e + e ) i = 1, j = k i j i j  2  k i i = 1, j =6 k 6  j i i = 1, j = k  s 6 6 (( s), ei ej ) 2 i = 1, j = k =  s 2(k j) , (e e , e e ) i = 1, j = k i j i j  2  k i i = 1, j =6 k 6  for n = 2k + 1 with k = 1  6 i j + 2k + 1 i = 1, j = k s+1 6 6 (( s), ei + ej) 2 i = 1, j = k =  s+2(k j)+1 , (e + e , e + e ) i = 1, j = k i j i j  2  k i + 1 i = 1, j =6 k 6  j i i = 1, j = k  s 1 6 6 (( s), ei ej) 2 i = 1, j = k =  s 2(k j) 1 , (e e , e e ) i = 1, j = k i j i j  2  k i i = 1, j =6 k 6 2(k i) + 1 i = 1, k (( s), e )  i = s i 6= 1 , (ei, ei)   1 i = k and for n = 3  (( s), e ) 1 = s, (e1, e1) and we obtain the result. ¤ (s )y Since also G = KMAN, the function 1 (kma n) = e (k K, m s x+iy ∈ ∈ M, y IR and n N) is well-dened and is in E0,s. Corollary B.2 combined with the ab∈ove results∈gives

Lemma B.5. 1 is cyclic in E as soon as e( ( s)) = 0. s 0,s 6

Corollary B.6. E0,s is irreducible for

a) for all s C when n is even > 4 and for all s C when n = 4

∈ \ even ∈ \ even

b) for all s C when n is odd 3. ∈ \ B.2. Irreducibility 149

Proof. This follows easily from Lemma B.5 and the observation that 0,s is irreducible if and only if 1s is cyclic in E0,s and 1 s is cyclic in E0, s. ¤ Next we are going to study the irreducibility of E1,s, this is done by applying the method of “translation of parameters” introduced by Wallach [43]. Consider the action of SO(n, C) on Cn. Put

v = (1, i, 0, . . . , 0), w = (1, i, 0, . . . , 0). 0 0

iz a v = e v z 0 0 a w = eizw (z C) z 0 0 ∈ m v = v 0 0 m w = w (m M) 0 0 ∈ n v = v 0 0 n w = w (n N, n N). 0 0 ∈ ∈ Dene F (g) = (g v , w ), (g G) 0 0 0 ∈ whith ( , ) dened as before. For any v Cn dene the matrix coecient ∈ c (g) = (g v , v), (g G). v 0 ∈ n The map v cv is a G-equivariant injective linear map from C into E 1, 1 . →

Lemma B.7. (Wallach [43], Ch. 8.13.9) F0 1s is cyclic in E 1,s 1 as soon as 1s is cyclic in E0,s.

Proof. Since G = KMAN the G-invariant subspace generated by F 1 clearly 0 s contains [o,s(G)1s] F0. Hence if f E 1,s 1 is orthogonal to 1,s 1(G)(F0 1s) ∈ then f F0 = 0, so f = 0.¤ To obtain = n in the above lemma we have to do the same as before but n n taking F0(g) and if we want = n we take F0(g) . Calling

F (g)n if = n 0 F(g) = .  n  F0(g) if = n We obtain 

Lemma B.8. F 1 is cyclic in E as soon as 1 is cyclic in E . s ,s+ s 0,s For the main corollary we need another lemma.

Lemma B.9. E,s is irreducible if F 1s is cyclic in E,s and F 1 s is cyclic in E, s. 150 Appendix B. Irreducibility and unitarity

Proof. We assume that = n, and consider an invariant subspace V E . ,s Any invariant subspace under G, is invariant under K. Recall that F, in this n case, is equal to F0(g) = (g (v0 v0 v0), (w0 w0 w0)), and we have span K translates of F span K (w w w ) Cn Cn. { } ' { 0 0 0 } We will see that the space span K translates of F is irreducible and occurs { } once. If we prove that there is, up to scalars, only one vector v in span K (w0 w w ) such that { 0 0 } cos sin 0 sin cos 0 v = einv (B.3)  0 0 h   with h SO(n 2, IR), we have proven, applying Frobenius reciprocity theorem, ∈ both statements. The vector w0 w0 w0 is, up to scalars, the only vector that satises (B.3). Therefore span K translates of F is either in V or in { } E,s/V , hence by the assumptions of the lemma V = E,s or V = (0), so E,s is irreducible. For = n we have to do the same. ¤ Corollary B.10. Let us consider s C and . ∈ ∈ a) If n 3 and n odd, E is irreducible when s / . ,s ∈ b) If n 4 and n even, E is irreducible when s / . ,s ∈ even For the proof we have to apply the two last lemmas.

B.3 Unitarity

When are our representations ,s unitary? We know that our representations are unitary if s iIR and we want to compute when 0,s with s IR is unitary. To do this we need∈to construct an inner product and to see when∈it is positive denite. Let A0,s be the intertwining operator between E0,s and E0, s dened in Section 8.3.1. We have A0,s1s = c(0, s)1 s ( n )( 1+ )( s+2 ) where c(0, s) = 2 2 2 2 . Let us call √( s+ )s 2 A = 0,s , A0,s c(0, s) then 0,s is well dened for s < . WAe take n odd then we know that = I on E since is an irreducible A0,0 0,0 0,0 representation on E0,0. On each K-type we have that

= A0,s A0,s 0, s 0,s = I for 1 < s < 1, A A B.3. Unitarity 151 what means that is Hermitian and invertible on each K-type. To obtain the A0,s second equality we apply that 0,s is irreducible for 1 < s < 1 see Corollary B.6. This is the reason for requiring the bound being equal to 1. Now we need a lemma. Lemma B.11. (Knapp [19], Lemma 16.1) Let F (x) be a continuous function from a topological space X to the vector space of n-by-n Hermitian matrices such that F (x0) is positive denite for some x0 and such that det F (x) is nonvanishing for x in a dense connected subset Y . Then F (x) is positive denite for x in Y , and F (x) is positive semidenite for all x in X. Applying the latter lemma with n odd, X = s IR : 1 < s < 1 , Y = X { ∈ } and F (x) = 0,s K type we obtain that < 0,sϕ, > is positive denite for 1 < s < 1. WAe ha|veproven that is unitaryA for 1 < s < 1 if n is odd. 0,s We can do the same for the case n = 4 since the representation 0,0 and 0,s for 2 < s < 2 are irreducible, see Corollary B.6. We obtain that 0,s is unitary for 2 < s < 2 if n = 4. 152 Appendix B. Irreducibility and unitarity Bibliography

[1] Y. Benoist. Espaces symetriques exponentiels. PhD thesis, Universite Paris VII, 1983. [2] P. Cartier. Vecteurs dierentiables dans les representations unitaires des groupes de Lie. In Seminaire Bourbakt (1974/1975), Exp. No. 454. Lecture Notes in Mathematics, Vol. 514, pages 20–34, Springer, Berlin, 1976. [3] I. Charchov. Harmonic Analysis on Line Bundles over Complex Hyperbolic Spaces. PhD thesis, Leiden University, 2000. [4] G. Choquet. Lectures on analysis. Vol. II: Representation theory. Edited by J. Marsden, T. Lance and S. Gelbart. W. A. Benjamin, Inc., New York- Amsterdam, 1969. [5] J. Dixmier and P. Malliavin. Factorisations de fonctions et de vecteurs in- deniment dierentiables. Bull. Sci. Math. (2), 102(4):307–330, 1978. [6] A. Erdelyi, W. Magnus, F. Oberhettinger, and F. G. Tricomi. Higher tran- scendental functions. Vol. I, McGraw-Hill, New York, 1953. [7] J. Faraut. Distributions spheriques sur les espaces hyperboliques. J. Math. Pures Appl. (9), 58(4):369–444, 1979. [8] J. Faraut. Analyse harmonique sur les espaces riemanniens symetriques de rang un. Ecole d’ete “Analyse Harmonique”, Universite de Nancy I, 1980. [9] J. Faraut and K. Harzallah. Distributions coniques associees au groupe orthogonal O(p, q). J. Math. Pures Appl. (9), 63(1):81–109, 1984. [10] G. B. Folland. Harmonic analysis in phase space. Annals of Mathematics Studies, Vol. 122, Princeton University Press, Princeton, NJ, 1989. [11] I. M. Gelfand and G. E. Shilov. Generalized functions. Vol. 1, Academic Press, New York, 1964. [12] I. S. Gradshteyn and I. M. Ryzhik. Table of integrals, series, and products. Academic Press Inc., San Diego, CA, sixth edition, 2000. [13] Harish-Chandra. Representations of semisimple Lie groups. II. Trans. Amer. Math. Soc., 76:26–65, 1954.

153 154 Bibliography

[14] S. Helgason. Geometric analysis on symmetric spaces. Mathematical Surveys and Monographs, Vol. 39, American Mathematical Society, Providence, RI, 1994.

[15] M. W. Hirsch. Dierential topology. Graduate Texts in Mathematics, Vol. 33, Springer-Verlag, New York, 1976.

[16] R. Howe. On some results of Strichartz and Rallis and Schiman. J. Funct. Anal., 32(3):297–303, 1979.

[17] M. Kashiwara and M. Vergne. On the Segal-Shale-Weil representations and harmonic polynomials. Invent. Math., 44(1):1–47, 1978.

[18] M. Kashiwara and M. Vergne. Functions on the Shilov boundary of the generalized half plane. In Noncommutative harmonic analysis (Proc. Third Col- loq., Marseille-Luminy, 1978). Lecture Notes in Mathematics, Vol. 728, pages 136–176, Springer, Berlin, 1979.

[19] A. W. Knapp. Representation theory of semisimple groups. An overview based on examples. Princeton Mathematical Series, Vol. 36, Princeton University Press, Princeton, NJ, 1986.

[20] M. T. Kosters. Spherical distributions on rank one symmetric spaces. PhD thesis, Leiden University, 1983.

[21] S. Lang. SL2(R). Graduate Texts in Mathematics, Vol. 105, Springer-Verlag, New York, 1985.

[22] V. F. Molchanov and G. van Dijk. Berezin forms on line bundles over complex hyperbolic spaces. Integral Equations Operator Theory, 45(2):177–230, 2003.

[23] B. rsted and G. Zhang. L2-versions of the Howe correspondence. II. J. Math. Pures Appl. (9), 74(2):165–183, 1995.

[24] B. rsted and G. Zhang. L2-versions of the Howe correspondence. I. Math. Scand., 80(1):125–160, 1997.

[25] W. R. Pestman. Group representations on Hilbert subspaces of distributions. PhD thesis, University of Groningen, 1985.

[26] S. Rallis and G. Schimann. Distributions invariantes par le groupe orthogonal. In Analyse harmonique sur les groupes de Lie (Sem., Nancy-Strasbourg, 1973–75). Lecture Notes in Mathematics, Vol. 497, pages 494–642, Springer, Berlin, 1975.

[27] L. Schwartz. Sous-espaces hilbertiens d’espaces vectoriels topologiques et noyaux associes (noyaux reproduisants). J. Analyse Math., 13:115–256, 1964.

[28] L. Schwartz. Theorie des distributions. Publications de l’Institut de Mathe- matique de l’Universite de Strasbourg, No. IX-X., Hermann, Paris, 1966. Bibliography 155

[29] I. E. Segal. Foundations of the theory of dynamical systems of innitely many degrees of freedom. I. Mat.-Fys. Medd. Danske Vid. Selsk., 31(12):39 pp. (1959), 1959. [30] D. Shale. Linear symmetries of free boson elds. Trans. Amer. Math. Soc., 103:149–167, 1962. [31] R. S. Strichartz. The explicit Fourier decomposition of L2(SO(n)/SO(n m)). Canad. J. Math., 27:294–310, 1975. [32] M. Sugiura. Unitary representations and harmonic analysis. An introduction. Kodansha Ltd., Tokyo, 1975. [33] E. G. F. Thomas. The theorem of Bochner-Schwartz-Godement for generalized Gelfand pairs. In Functional analysis: surveys and recent results, III (Paderborn, 1983). North-Holland Math. Stud., Vol. 90, pages 291–304, North-Holland, Amsterdam, 1984. [34] E. G. F. Thomas. Integral representations in conuclear cones. J. Convex Anal., 1(2):225–258 (1995), 1994. [35] F. Treves. Topological vector spaces, distributions and kernels. Academic Press, New York, 1967. [36] E. van den Ban. The Plancherel theorem for a reductive symmetric space. Lectures for the European School of Group Theory, SDU-Odense University, August 14-26, 2000. [37] G. van Dijk. On generalized Gelfand pairs. Proc. Japan Acad. Ser. A Math. Sci., 60(1):30–34, 1984. [38] G. van Dijk. On a class of generalized Gelfand pairs. Math. Z., 193(4):581–593, 1986. [39] G. van Dijk, M. Pevzner, and S. Aparicio. Invariant Hilbert subspaces of the oscillator representation. Indag. Math. (N.S.), 14(3-4):309–318, 2003. [40] G. van Dijk and M. Poel. The Plancherel formula for the pseudo-Riemannian space SL(n, IR)/GL(n 1, IR). Compositio Math., 58(3):371–397, 1986. [41] G. van Dijk and M. Poel. The irreducible unitary GL(n 1, IR)-spherical representations of SL(n, IR). Compositio Math., 73(1):1–30, 1990. [42] V. S. Varadarajan. Harmonic analysis on real reductive groups. Lecture Notes in Mathematics, Vol. 576, Springer-Verlag, Berlin, 1977. [43] N. R. Wallach. Harmonic analysis on homogeneous spaces. Pure and Applied Mathematics, Vol. 19, Marcel Dekker Inc., New York, 1973. [44] A. Weil. Sur certains groupes d’operateurs unitaires. Acta Math., 111:143–211, 1964. 156 Bibliography Samenvatting

De resultaten die we in dit proefschrift presenteren behoren tot het gebied van de harmonische analyse en zijn sterk beinvloed door ontwikkelingen in de na- tuurkunde, vooral in de quantummechanica. We houden ons vooral bezig met eigenschappen van een bijzondere unitaire representatie, de metaplectische representatie. De metaplectische representatie –ook bekend als oscillator representatie, harmonische representatie, oftewel Segal-Shale-Weil representatie– is een dubbel-waardige unitaire representatie van de symplectische groep Sp(n, IR) op L2(IRn). Is, in het algemeen G een Lie groep en een Hilbert ruimte waar G unitair op werkt, dan verstaan we onder een unitaire represenH tatie van G een homomorphisme van G in de groep van unitaire operatoren op de Hilbert ruimte zodat de afbeelding g (g)x continu is van G naar voor elke x in . Een vanHde centrale onderwerpen→in de harmonische analyse is hetH ontbinden vanH de representatie van G op in irreducibele representaties: het vinden van de zogenaamde Plancherel formuleH voor de gegeven G-actie. In dit proefschrift beschouwen we de ontbinding van de een-waardige oscillator representatie voor enkele ondergroepen van de symplectische groep, namelijk SL(2, IR) O(2n), SL(2, IR) O(p, q) en SL(2, C) SO(n, C). De belangrijkste resultaten zijn de Plancherel form ule van deze representaties en de multipliciteitsvrije ontbinding van elke invariante Hilbert deelruimte van de ruimte van getempereerde distributies. De ontbinding van een invariante Hilbert deelruimte van de ruimte van getempereerde distributies heet multipliciteitsvrij als voor elk paar minimale invariante Hilbert deelruimten en , die voorkomen in de ontbinding en niet evenredig H1 H2 zijn, de irreducibele representaties op 1 en 2 niet equivalent zijn. Een essentieel gereedschap dat wordtH gebruiktH voor de berekening van de Plan- cherel formule is een Fourier integraal operator, voor het eerst geintroduceerd door M. Kashiwara en M. Vergne. Om de multipliciteitsvrije ontbinding te bewijzen hebben we gebruik gemaakt van een criterium van Thomas.

157 158 Samenvatting Curriculum Vitae

Sofa Aparicio Secanellas was born in June 22nd, 1977, in Zaragoza, Spain. She completed primary and started secondary studies in Escatron, a small village in the province of Zaragoza facing the Ebro river. She nished secondary studies in the city of Zaragoza, where she started her studies in mathematics. During the last year of her studies she got an Erasmus scholarship from the European Union to go to Naples, Italy. She graduated in Applied Mathematics at the University of Zaragoza in 2001. From January 2002 until October 2005, she did her PhD research under supervision of prof. dr. Gerrit van Dijk at the Mathematical Institute of the University of Leiden, The Netherlands, which resulted in this thesis.

159 S’ha feito de nuey

S’ha feito de nuey Dicen qu’en querer tu ma guardas ya ye de dos no mas lo peito me brinca y que ye mas facil tornate a besar. ferlo caminar cuando l’uno caye Lo nuestro querer l’otro a devantar, no se crebara cuando l’uno caye aunque charren muito l’otro a devantar. y te fagan plorar.

S’ha feito de nuey Yo no quiero vier tu m’a guarda ya guel los de cristal lo peito me brinca mullaus por glarimas te quiero besar. que culpa no hay.

Escuita muller disxa de plorar yo siempre e’stau tuyo tu mia has d’estar.

—Jota en fabla, antigua lengua aragonesa, cantada por mis primas Maria Luisa y Gloria.