The C*-Algebras of Certain Lie Groups Janne-Kathrin Günther

The C*-algebras of certain Lie groups Janne-Kathrin Günther

To cite this version:

Janne-Kathrin Günther. The C*-algebras of certain Lie groups. Algebraic Topology [math.AT]. Université de Lorraine, 2016. English. NNT : 2016LORR0118. tel-01446979

HAL Id: tel-01446979 https://tel.archives-ouvertes.fr/tel-01446979 Submitted on 26 Jan 2017

HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés.

AVERTISSEMENT

Ce document est le fruit d'un long travail approuvé par le jury de soutenance et mis à disposition de l'ensemble de la communauté universitaire élargie.

Il est soumis à la propriété intellectuelle de l'auteur. Ceci implique une obligation de citation et de référencement lors de l’utilisation de ce document.

D'autre part, toute contrefaçon, plagiat, reproduction illicite encourt une poursuite pénale.

Contact : [email protected]

LIENS

Code de la Propriété Intellectuelle. articles L 122. 4 Code de la Propriété Intellectuelle. articles L 335.2- L 335.10 http://www.cfcopies.com/V2/leg/leg_droi.php http://www.culture.gouv.fr/culture/infos-pratiques/droits/protection.htm PhD-FSTC-2016-41 Facult´edes Sciences et Technologies The Faculty of Sciences, Technology and Institut Elie Cartan de Lorraine Communication Ecole Doctorale IAEM Lorraine

DISSERTATION

Defense held on 22/09/2016 in Luxembourg to obtain the degree of

DOCTEUR DE L’UNIVERSITE´ DU LUXEMBOURG EN MATHEMATIQUES´ AND

DOCTEUR DE L’UNIVERSITE´ DE LORRAINE EN MATHEMATIQUES´

by Janne-Kathrin GUNTHER¨ Born on 4 March 1985 in Preetz (Germany)

THE C∗-ALGEBRAS OF CERTAIN LIE GROUPS

Dissertation defense committee Dr Martin OLBRICH, dissertation supervisor Professor, Universitédu Luxembourg Dr Jean LUDWIG, dissertation supervisor Professor, Universitéde Lorraine Dr Ingrid BELTITA, reporter Professor, Romanian Academy Dr Joachim HILGERT, reporter Professor, Universität Paderborn Dr Anton THALMAIER, Chairman Professor, Universitédu Luxembourg Dr Angela PASQUALE, Vice Chairman Professor, Universitéde Lorraine

Contents

1 Introduction 1

2 General deﬁnitions and results 6 2.1 Nilpotent Lie groups ...... 6 2.1.1 Preliminaries ...... 6 2.1.2 Orbitmethod...... 7 2.1.3 The C∗-algebra C∗(G/U, χℓ) ...... 8 2.2 Generalresults ...... 9

3 The C∗-algebras of connected real two-step nilpotent Lie groups 11 3.1 Preliminaries ...... 11 3.1.1 Two-step nilpotent Lie groups ...... 11 3.1.2 Induced representations ...... 11 3.1.3 Orbitmethod...... 12 3.2 Conditions 1, 2 and 3(a) ...... 13 3.3 Condition 3(b) ...... 17 3.3.1 Introductiontothesetting ...... 17 3.3.2 Changing the Jordan-Hölder basis ...... 17 3.3.3 Definitions ...... 19 3.3.4 Formula for πk ...... 20 3.4 Condition 3(b) – First case ...... 22 3.5 Condition 3(b) – Second case ...... 23 3.5.1 Convergence of (πk)k N in G ...... 23 ∈ 3.5.2 Definition of νk ...... 25 3.5.3 Theorem – Second Case . .b ...... 29 V 3.5.4 Transition to πk k N ...... 37 3.6 Condition 3(b) – Third case∈ ...... 38 3.6.1 Convergence of (πk)k N in G ...... 38 ∈ 3.6.2 Definition of νk ...... 41 3.6.3 Theorem – Third Case . . .b ...... 46 V 3.6.4 Transition to πk k N ...... 50 3.7 Result for the connected real∈ two-step nilpotent Lie groups ...... 51 3.8 Example: The free two-step nilpotent Lie groups of 3 and 4 generators . . . . 51 3.8.1 The case n =3...... 52 3.8.2 The case n =4...... 56

4 The C∗-algebra of SL(2, R) 64 4.1 Preliminaries ...... 64 4.2 The spectrum of SL(2, R)...... 67 4.2.1 Introduction of the operator Ku ...... 67 4.2.2 Description of the irreducible unitary representations ...... 76 4.2.3 The SL(2, R)-representations applied to the Casimir operator . . . . . 79 4.2.4 The topology on SL(2, R)...... 81 4.2.5 Deﬁnition of subsets Γi ofthespectrum ...... 86 4.3 Condition 3(a) ...... d ...... 87 4.4 Condition 3(b) ...... 90 4.5 Result for the Lie group SL(2, R)...... 95

5 On the dual topology of the groups U(n) ⋉Hn 100 5.1 Preliminaries ...... 100 5.1.1 The coadjoint orbits of Gn ...... 101 5.1.2 The spectrum of U(n)...... 103 5.1.3 The spectrum and the admissible coadjoint orbits of Gn ...... 104 5.2 Convergence in the quotient space gn‡ /Gn ...... 107 5.3 The topology of the spectrum of Gn ...... 121 5.3.1 The representation π(µ,r) ...... 121 5.3.2 The representation π(λ,α) ...... 123 5.3.3 The representation τλ ...... 143 5.4 Results...... 143

6 Résuméétendu de cette thèse en fran¸cais 146 6.1 Introduction...... 146 6.2 Définitionsetrésultatsgénéraux ...... 150 6.2.1 Les groupes de Lie nilpotents – Préliminaires ...... 150 6.2.2 Les groupes de Lie nilpotents – La méthode des orbites ...... 151 6.2.3 Les groupes de Lie nilpotents – La C∗-algèbre C∗(G/U, χℓ) ...... 152 6.2.4 Résultats généraux ...... 153 6.3 Les C∗-algèbres des groupes de Lie connexes réels nilpotents de pas deux . . 154 6.3.1 Préliminaires – Les groupes de Lie nilpotents de pas deux ...... 154 6.3.2 Préliminaires – La méthode des orbites ...... 154 6.3.3 Les Conditions 1, 2 et 3(a) ...... 156 6.3.4 Condition 3(b) – Introduction de la situation ...... 156 6.3.5 Condition 3(b) – Changement de la base de Jordan-Hölder ...... 157 6.3.6 Condition 3(b) – Premier cas ...... 159 6.3.7 Condition 3(b) – Deuxième cas ...... 160 6.3.8 Condition 3(b) – Troisième cas ...... 162 6.3.9 Résultat pour les groupes de Lie connexes réels nilpotents de pas deux 162 6.4 La C∗-algèbre du groupe de Lie SL(2, R) ...... 163 6.4.1 Préliminaires ...... 163 6.4.2 Le spectre de SL(2, R) – Introduction de l’opérateur Ku ...... 165 6.4.3 Le spectre de SL(2, R) – Description des représentations irréductibles unitaires...... 168 6.4.4 Le spectre de SL(2, R) – La topologie sur SL(2, R) ...... 170 6.4.5 Les conditions limites duales sous contrôle normique ...... 171 6.4.6 Résultat pour le groupe de Lie SL(2, R) . . .d ...... 172 6.5 Sur la topologie duale des groupes U(n) ⋉Hn ...... 173 6.5.1 Préliminaires ...... 173 6.5.2 Le spectre et les orbites coadjointes admissibles de Gn ...... 174 6.5.3 Convergence dans l’espace quotient gn‡ /Gn et la topologie du spectre de Gn ...... 175 6.5.4 Résultats ...... 177 7 Appendix 178

8 Acknowledgements 186

9 References 187 Abstract In this doctoral thesis, the C∗-algebras of the connected real two-step nilpotent Lie groups and the Lie group SL(2, R) are characterized. Furthermore, as a preparation for an analysis of its C∗-algebra, the topology of the spectrum of the semidirect product U(n)⋉Hn is described, where Hn denotes the Heisenberg Lie group and U(n) the unitary ∗ group acting by automorphisms on Hn. For the determination of the group C -algebras, the operator valued Fourier transform is used in order to map the respective C∗-algebra into the algebra of all bounded operator fields over its spectrum. One has to find the conditions that are satisfied by the image of this C∗-algebra under the Fourier transform and the aim is to characterize it through these conditions. In the present thesis, it is proved that both the C∗-algebras of the connected real two-step nilpotent Lie groups and the C∗-algebra of SL(2, R) fulfill the same conditions, namely the “norm controlled dual limit” conditions. Thereby, these C∗-algebras are described in this work and the “norm controlled dual limit” conditions are explicitly computed in both cases. The methods used for the two-step nilpotent Lie groups and the group SL(2, R) are completely different from each other. For the two-step nilpotent Lie groups, one regards their coadjoint orbits and uses the Kirillov theory, while for the group SL(2, R) one can accomplish the calculations more directly.

Résumé Dans la présente thèse de doctorat, les C∗-algèbres des groupes de Lie connexes réels nilpotents de pas deux et du groupe de Lie SL(2, R) sont caractérisées. En outre, comme préparation àune analyse de sa C∗-algèbre, la topologie du spectre du produit semi-direct U(n) ⋉Hn est décrite, où Hn dénote le groupe de Lie de Heisenberg et U(n) le groupe ∗ unitaire qui agit sur Hn par automorphismes. Pour la détermination des C -algèbres de groupes, la transformation de Fourier àvaleurs opérationnelles est utilisée pour appliquer chaque C∗-algèbre dans l’algèbre de tous les champs d’opérateurs bornés sur son spectre. On doit trouver les conditions que satisfait l’image de cette C∗-algèbre sous la transformation de Fourier et l’objectif est de la caractériser par ces conditions. Dans cette thèse, il est démontréque les C∗-algèbres des groupes de Lie connexes réels nilpotents de pas deux et la C∗-algèbre de SL(2, R) satisfont les mêmes conditions, des conditions appelées “limites duales sous contrôle normique”. De cette manière, ces C∗-algèbres sont décrites dans ce travail et les conditions “limites duales sous contrôle normique” sont explicitement calculées dans les deux cas. Les méthodes utilisées pour les groupes de Lie nilpotents de pas deux et pour le groupe SL(2, R) sont très différentes l’une de l’autre. Pour les groupes de Lie nilpotents de pas deux, on regarde leurs orbites coadjointes et on utilise la théorie de Kirillov, alors que pour le groupe SL(2, R), on peut mener les calculs plus directement. Zusammenfassung In dieser Doktorarbeit werden die C∗-Algebren der zusammenhängenden reellen zweistufig nilpotenten Lie-Gruppen und der Lie-Gruppe SL(2, R) charakterisiert. Außer- dem wird die Topologie des Spektrums des semidirekten Produktes U(n) ⋉Hn in Vor- ∗ bereitung auf eine Analyse der zugehörigen C -Algebra beschrieben, wobei Hn die Heisenberg-Lie-Gruppe und U(n) die unitäre Gruppe, die durch Automorphismen auf ∗ Hn wirkt, bezeichnen. Für die Bestimmung der Gruppen-C -Algebren wird die opera- torwertige Fouriertransformation benutzt, um die jeweilige C∗-Algebra in die Algebra der beschränkten Operatorenfelder über ihrem Spektrum abzubilden. Das Ziel ist, die Bedingungen zu finden, die das Bild dieser C∗-Algebra unter der Fouriertransforma- tion erfüllt und sie durch eben diese zu charakterisieren. In der vorliegenden Arbeit wird bewiesen, dass sowohl die C∗-Algebren der zusammenhängenden reellen zweistufig nilpotenten Lie-Gruppen, als auch die C∗-Algebra von SL(2, R) dieselben Bedingungen erfüllen, nämlich die “normkontrollierter dualer Limes”-Bedingungen. Dadurch werden diese C∗-Algebren beschrieben und die “normkontrollierter dualer Limes”-Bedingungen werden in beiden Fällen explizit nachgewiesen. Die Methoden, die für die zweistufig nilpotenten Lie-Gruppen und die Gruppe SL(2, R) verwendet werden, sind dabei komplett unterschiedlich. Für die zweistufig nilpotenten Lie-Gruppen betrachtet man deren koad- jungierte Bahnen und benutzt die Kirillov-Theorie, während man für die Gruppe SL(2, R) die Berechnungen direkter durchführen kann.

1 Introduction

In this doctoral thesis, the structure of the C∗-algebras of connected real two-step nilpotent Lie groups and the structure of the C∗-algebra of the Lie group SL(2, R) will be analyzed. Moreover, as a preparative work for an examination of its C∗-algebra, the topology of the spectrum of the semidirect product U(n) ⋉ Hn will be described, where Hn denotes the Heisenberg Lie group and U(n) the unitary group acting by automorphisms on Hn.

The research of C∗-algebras – an abstraction of algebras of bounded linear operators on Hilbert spaces – began in the 1930s due to a need in quantum mechanics and more precisely in order to serve as mathematical models for algebras of physical observables. Quantum systems are described with the help of self-adjoint operators on Hilbert spaces. Hence, algebras of bounded operators on these spaces were regarded. The term “C∗-algebra” was first introduced in the 1940s by I.Segal for describing norm-closed subalgebras of the algebra of bounded linear operators on a Hilbert space. By now, C∗-algebras represent an important tool in the theory of unitary representations of locally compact groups and in the mathematical description of quantum mechanics. Lie groups were introduced in the 1870s by S.Lie in the framework of the Lie theory in order to examine continuous symmetries in differential equations and their theory has been developed further in the 20th century. Today Lie groups are used in many mathematical fields and in theoretical physics, as for example in particle physics. As Lie groups and other symmetry groups hold an important position in physics, the examination of their group C∗-algebras – C∗-algebras that are constructed by building a completion of the space of all L1-functions on the respective groups – has been continued and advanced. This task represents the mission of this thesis. The method of analyzing a group C∗-algebra employed in this work has been initiated by G.M.Fell in the 1960s. It uses the non-abelian Fourier transform in order to write the C∗-algebras as algebras of operator fields and to study their elements in that way. A requirement to make use of this method and thus to understand these C∗-algebras is the knowledge of their spectrum and its topology which are unknown for many groups. Hence, this represents a major problem of Fell’s procedure. The advancement of this method that is adopted in this thesis has been elaborated in the last years (see [24] and [26]) and makes the study of a large class of group C∗-algebras possible.

In order to be able to understand C∗-algebras, the Fourier transform is an important tool. Denoting by the unitary dual or spectrum A of a C∗-algebra A the set of all equivalence classes of the irreducible unitary representations of A, the Fourier transform (a) =a ˆ of an F element a A is defined in the following way: Oneb chooses in every equivalence class γ A ∈ ∈ a representation (π , ) and defines γ Hγ b (a)(γ) := π (a) ( ), F γ ∈B Hγ where ( ) denotes the C -algebra of bounded linear operators on . Then, (a) is con- B Hγ ∗ Hγ F tained in the algebra of all bounded operator fields over A

l∞(A)= φ = φ(πγ) ( γ) φ :=b sup φ(πγ) op < γ A ∞ ∈B H ∈ b k k γ A k k ∞ n ∈ b o b

1 and the mapping : A l∞(A), a aˆ F → 7→ is an isometric -homomorphism. ∗ b Denoting by dx the Haar measure of the locally compact group G, one can deﬁne on L1(G) := L1(G, C) the convolution product in the following way ∗

1 1 f f ′(g) := f(x)f ′ x− g dx f,f ′ L (G) g G ∗ ∀ ∈ ∀ ∈ Z G and obtains a Banach algebra L1(G), . Moreover, one gets an isometric involution on this ∗ algebra by letting 1 1 f ∗(g):=∆ (g)− f g g G, G − ∀ ∈ ∆G being the modular function which is deﬁned to be the positive function fulﬁlling

1 f tg− dt =∆ (g) f(t)dt f C (G) g G, G ∀ ∈ 0 ∀ ∈ Z Z G G where C0(G) is the space of all compactly supported continuous functions. For every irreducible unitary representation (π, ) of G, a representation (˜π, ) of L1(G) can H H be obtained as follows π˜(f) := f(g)π(g)dg f L1(G). ∀ ∈ GZ This representation turns out to be irreducible and unitary. 1 Now, the C∗-algebra of G is deﬁned as the completion of the convolution algebra L (G) with 1 respect to the C∗-norm of L (G), i.e.

1 C∗(G) C∗(G) := L (G,dx)k·k with f C∗(G) := sup π(f) op, k k [π] G k k ∈ b where G is the spectrum of G, deﬁned as above as the set of all equivalence classes of the irreducible unitary representations of G. Furthermore,b every irreducible unitary representation (˜π, ) of L1(G) can be uniquely written H as an integral in the above shown way and hence, one obtains an irreducible unitary representation (π, ) of G. One therefore gets the following well-known result, that can be found in H [9], Chapter 13.9, and states that the spectrum of C∗(G) coincides with the spectrum of G: \ C∗(G)= G.

b The structure of the group C∗-algebras is already known for certain classes of Lie groups, such as the Heisenberg and the thread-like Lie groups (see [26]) and the ax + b-like groups (see [24]). Furthermore, the C∗-algebras of the 5-dimensional nilpotent Lie groups have been determined in [31], while those of all 6-dimensional nilpotent Lie groups have been characterized by H.Regeiba in [30]. In this thesis, a result shown by H.Regeiba and J.Ludwig (see [31], Theorem 3.5) shall be utilized for the characterization of the C∗-algebras mentioned at the beginning.

2 For the C∗-algebras of the two-step nilpotent Lie groups, the approach will partly be similar but more complex, to the one used for the characterization of the C∗-algebra of the Heisenberg Lie group (see [26]), which is also two-step nilpotent and thus serves as an example. In the case of the Heisenberg Lie group, the situation is a lot easier than for general two-step nilpotent Lie groups, as in this special case the appearing coadjoint orbits can only have two different dimensions, while in the general case, there are many different dimensions that can appear. Consequently, much more complicated cases emerge. The 5- and 6-dimensional nilpotent Lie groups, in turn, have a more complicated structure than the two-step nilpotent Lie groups. Here, non-flat coadjoint orbits appear, whereas in the two-step nilpotent case, one only has to deal with flat coadjoint orbits. In these cases of 5- and 6-dimensional nilpotent Lie groups, one therefore examines every group by itself, while for the two-step nilpotent Lie groups, one covers this whole class of groups without knowing their exact structure. As mentioned above, the approach for the group SL(2, R) will be completely different. Since one treats one single group which is explicitly given, in this case, one can carry out more concrete computations. For semisimple Lie groups in general, there is no explicit description of their group C∗-algebras given in literature. However, for those semisimple Lie groups whose spectrum is classified, the procedure of the determination of the group C∗-algebra used in this work might be successfully applied in a similar way. A description of the C∗-algebra of the Lie group SL(2, C) is given in [13], Theorem 5.3 and Theorem 5.4, and a characterization of reduced group C∗-algebras of semisimple Lie groups can be found in [34]. In the present doctoral thesis, the group C∗-algebras of connected real two-step nilpotent Lie groups and of SL(2, R) shall be described very explicitly. It will be shown that they are characterized by specific conditions that are called “norm controlled dual limit” conditions which will be given below. The main objective is to concretely compute these conditions and to construct the required “norm controls”. In an abstract existence result (see [2], Theorem 4.6), these conditions are shown to hold true for all simply connected connected nilpotent Lie groups. They are explicitly checked for all 5- and 6-dimensional nilpotent Lie groups (see [31]), for the Heisenberg Lie groups and the thread-like Lie groups (see [26]). The results about the group C∗-algebras of the connected real two-step nilpotent Lie groups have been published in the “Revista Matemática Complutense” as a joint article with J.Ludwig (see [17]). A further article about the group C∗-algebra of SL(2, R) has been submitted to the “Journal of Lie Theory” and can be found on arXiv (see [16]). Since for the determination of a group C∗-algebra the spectrum of the group and its topology are necessary, in this doctoral thesis, the topology of the spectrum of the groups G = U(n) ⋉H for n N = N 0 shall be analyzed as well. This work was started by n n ∈ ∗ \{ } M.Elloumi and J.Ludwig (see [11], Chapter 3) and their preprint will be elaborated in this thesis. Like for several other classes of Lie groups, for example the exponential solvable Lie groups and the Euclidean motion groups, the spectrum Gn can be parametrized by the space of all admissible coadjoint orbits in the dual gn∗ of the Lie algebra gn of Gn. The aim is to show that the bijection between thec space of all admissible coadjoint orbits equipped with the quotient topology and the spectrum Gn of Gn is a homeomorphism. This has already been shown for the classical motion groups SO(n) ⋉Rn for n 2 in [12]. ≥ c

3 For the groups G = U(n) ⋉H , this claim has been proved for n = 1. For general n N , it n n ∈ ∗ could be shown that the mapping between the spectrum Gn and the space of all admissible coadjoint orbits of Gn is continuous. The remaining parts of the proof of the claim are work in progress. c

In this section, the definition of a C∗-algebra with “norm controlled dual limits” will be given. The conditions required by this definition characterize group C∗-algebras. There- after, in Section 3, they will be shown to hold true for the two-step nilpotent Lie groups. After having completed this proof, an example of a two-step nilpotent Lie group will be examined. In Section 4, these conditions will then be proved in the case of the Lie group SL(2, R), in a very different approach than the one used for the two-step nilpotent Lie groups.

Deﬁnition 1.1. A C∗-algebra A is called C∗-algebra with norm controlled dual limits if it fulﬁlls the following conditions:

- Condition 1: Stratiﬁcation of the spectrum:

(a) There is a finite increasing family S S ... S = A of closed subsets of the 0 ⊂ 1 ⊂ ⊂ r spectrum A of A in such a way that for i 1, ,r the subsets Γ := S and ∈ { ··· } 0 0 Γi := Si Si 1 are Hausdorff in their relative topologies andb such that S0 consists of \ − all the charactersb of A. (b) For every i 0, ,r , there is a Hilbert space and for every γ Γ , there is a ∈ { ··· } Hi ∈ i concrete realization (π , ) of γ on the Hilbert space . γ Hi Hi - Condition 2: CCR C∗-algebra: A is a separable CCR (or liminal) C∗-algebra, i.e. a separable C∗-algebra such that the image of every irreducible representation (π, ) of A is contained in the algebra of compact H operators ( ) (which implies that the image equals ( )). K H K H - Condition 3: Changing of layers: Let a A. ∈ (a) The mappings γ (a)(γ) are norm continuous on the different sets Γ . 7→ F i (b) For any i 0, ,r and for any converging sequence contained in Γ with limit set ∈ { ··· } i outside Γi (thus in Si 1), there is a properly converging subsequence γ = (γk)k N, − as well as a constant C > 0 and for every k N an involutive linear mapping∈ ∈ ν˜k =ν ˜γ,k : CB(Si 1) ( i), which is bounded by C Si (uniformly in k), such − →B H k · k −1 that

lim (a)(γk) ν˜k (a) Si = 0. k F − F | −1 op →∞ Here, CB(Si 1) is the -algebra of all the uniformly bounded ﬁelds of operators − ∗ ψ(γ) ( l) , which are operator norm continuous on the subsets Γl ∈B H γ Γl,l=0, ,i 1 for every l 0, ∈ ,i ···1 −, provided with the inﬁnity-norm ∈ { ··· − }

ψ Si−1 := sup ψ(γ) op. k k γ Si k k ∈ −1

4 Theorem 1.2 (Main result). The C∗-algebras of the connected real two-step nilpotent Lie groups G and of the Lie group G = SL(2, R) have norm controlled dual limits. \ In order to prove Theorem 1.2, concrete subsets Γi and Si of C∗(G) ∼= G will be deﬁned and the mappings ν˜k k N will be constructed. ∈ b The norm controlled dual limit conditions completely characterize the structure of a C -algebra in the following sense: Taking the number r, the Hilbert spaces , the sets Γ ∗ Hi i and S for i 0,...,r and the mappings ν˜ required in the above deﬁnition, by [31], i ∈ { } k k N Theorem 3.5, one gets the result below for the∈ C -algebras of the two-step nilpotent Lie ∗ groups G and of G = SL(2, R).

Theorem. The C∗-algebra C∗(G) of a connected real two-step nilpotent Lie group G and the C∗-algebra of the Lie group G = SL(2, R), respectively, are isomorphic (under the Fourier transform) to the set of all operator ﬁelds ϕ deﬁned over the spectrum G of the respective group such that:

1. ϕ(γ) ( ) for every i 1,...,r and every γ Γ . ∈K Hi ∈ { } ∈ b i 2. ϕ l (G). ∈ ∞ 3. The mappings γ ϕ(γ) are norm continuous on the diﬀerent sets Γ . b 7→ i 4. For any sequence (γk)k N G going to inﬁnity, lim ϕ(γk) op = 0. ∈ ⊂ k k k →∞ 5. For i 1,...,r and anyb properly converging sequence γ = (γk)k N Γi whose ∈ { } ∈ ⊂ limit set is contained in Si 1 (taking a subsequence if necessary) and for the mappings − ν˜k =ν ˜γ,k : CB(Si 1) ( i), one has − →B H

lim ϕ(γk) ν˜k ϕ Si = 0. k − | −1 op →∞

In the case of the Lie group G = SL(2, R), this result can still be simpliﬁed and more concretely described in Theorem 4.21.

5 2 General deﬁnitions and results

2.1 Nilpotent Lie groups 2.1.1 Preliminaries Let g be a real nilpotent Lie algebra. Fix a scalar product , on g and take on g the Campbell-Baker-Hausdorﬀ multiplication h· ·i 1 1 1 u v = u + v + [u,v]+ u, [u,v] + v, [v,u] + ... u,v g, · 2 12 12 ∀ ∈ which is a ﬁnite sum, as g is nilpotent. This gives the simply connected connected Lie group G = (g, ) with Lie algebra g. The · exponential mapping exp : g G =(g, ) is in this case the identity mapping. → · The Haar measure of this group is a Lebesgue measure which is denoted by dx.

Convention 2.1. Throughout this work, all function spaces are spaces of complex valued functions.

For a vector space V , denote by the Schwartz space (V ) the space of all rapidly decreasing S smooth functions on V . Then, as G =(g, ), one can define (G) := (g). · 1 S S Define on g and thus on G a norm by := , 2 . If X ,...,X represents an orthonormal k·k |h· ·i| { 1 n} basis of g with respect to , and α = (α ,...,α ) Nn, define Xα := Xα1 Xαn in the h· ·i 1 n ∈ 1 ··· n universal enveloping algebra (g). Moreover, denote by f the derivative of f (G) in the U α′ ∈S direction of Xα for every α Nn. Then, the semi-norms on (G) can be given by ∈ S

2 N f = 1+ g f ′ (g) dg f (G) N N. k k(N) k k α ∀ ∈S ∀ ∈ α N Z | X|≤ G

The space (G) is dense in L1(G) and in turn, for a vector space V , a well-known result S states that the space f (V ) fˆ is compactly supported is dense in (V ). Hence, ∈ S | S f (G) fˆ is compactly supported is dense in (G) as well. ∈S | S Now, for a linear functional ℓ of g, consider the skew-bilinear form

B (X,Y ) := ℓ, [X,Y ] ℓ h i on g. Moreover, let

g(ℓ) := X g ℓ, [X, g] = 0 ∈ | h i { } be the radical of Bℓ and the stabilizer of the linear functional ℓ. Deﬁnition 2.2 (Polarization). A subalgebra p of g, that is subordinated to ℓ (i.e. that fulﬁlls ℓ, [p, p] = 0 ) and that has h i { } the dimension 1 dim(p)= dim(g)+ dim(g(ℓ)) , 2 which means that p is maximal isotropic for Bℓ, is called a polarization of ℓ.

6 If p g is any subalgebra of g which is subordinated to ℓ, the linear functional ℓ defines a ⊂ unitary character χℓ of P := exp(p): 2πi ℓ,log(x) 2πi ℓ,x χ (x) := e− h i = e− h i x P. ℓ ∀ ∈ Definition 2.3 (The coadjoint action). Define for all g G the mappings ∈ 1 α : G G, x gxg− and Ad (g) := d(α ) : g g. g → 7→ g e → Then, Ad : G GL(g) is a group representation. The coadjoint action Ad ∗ can be defined → as follows:

1 Ad ∗(g): g∗ g∗ g G, Ad ∗(g)ℓ(X)= ℓ Ad g− X ℓ g∗ X g. → ∀ ∈ ∀ ∈ ∀ ∈ Ad ∗ : G GL(g ) is another group representation. → ∗ The space of coadjoint orbits Ad ∗(G)ℓ ℓ g is denoted by g /G. { | ∈ ∗} ∗ Furthermore, deﬁne the mapping

ad ∗(X): g∗ g∗ X g, ad ∗(X)ℓ(Y )= ℓ([Y,X]). → ∀ ∈ ad ∗ : g GL(g ) is a representation of the Lie algebra g. → ∗ Deﬁnition 2.4 (Induced representation). Let H be a closed subgroup of G and deﬁne

L2(G/H, χ ) := ξ : G C ξ measurable, ξ(gh)= χ (h)ξ(g) g G h H, ℓ → ℓ ∀ ∈ ∀ ∈

ξ 2 := ξ(g) 2dg˙ < , k k2 | | ∞ G/HZ where dg˙ is an invariant measure on G/H which always exists for nilpotent G. 2 L (G/H, χℓ) is a Hilbert space and one can deﬁne the induced representation G 1 2 ind χ (g)ξ(u) := ξ g− u g,u G ξ L (G/H, χ ). H ℓ ∀ ∈ ∀ ∈ ℓ This is a unitary representation. If the associated Lie algebra h of H is a polarization of ℓ, this representation is also irreducible.

2.1.2 Orbit method By the Kirillov theory (see [8], Section 2.2), for every representation class γ G, there exists ∈ an element ℓ g and a polarization p of ℓ in g such that γ = indGχ , where P := exp(p). ∈ ∗ P ℓ Moreover, if ℓ,ℓ′ g∗ are located in the same coadjoint orbit g∗/G band p and p′ ∈ O ∈ G G are polarizations in ℓ and ℓ′, respectively, the induced representations indP χℓ and indP ′ χℓ′ are equivalent and the Kirillov map which goes from the coadjoint orbit space g∗/G to the spectrum G of G G K : g∗/G G, Ad ∗(G)ℓ ind χ → 7→ P ℓ b is a homeomorphism (see [6] or [23], Chapter 3). Therefore, b g∗/G ∼= G as topological spaces. b

7 Deﬁnition 2.5 (Properly converging). Let T be a second countable topological space and suppose that T is not Hausdorﬀ, which means that converging sequences can have many limit points. Denote by L((tk)k N) the collection of ∈ all the limit points of a sequence (tk)k N in T . A sequence (tk)k N is called properly converging ∈ ∈ if (tk)k N has limit points and if every subsequence of (tk)k N has the same limit set as (tk)k N. ∈ ∈ ∈ It is well-known that every converging sequence in T admits a properly converging subsequence.

2.1.3 The C∗-algebra C∗(G/U, χℓ) Let u g be an ideal of g containing [g, g], U := exp(u) and let ℓ g such that ℓ, [g, u] = 0 ⊂ ∈ ∗ h i { } and such that u g(ℓ). Then, the character χℓ of the group U = exp(u) is G-invariant. One ⊂ 1 can thus deﬁne the involutive Banach algebra L (G/U, χℓ) as

1 1 L (G/U, χ ) := f : G C f measurable, f(gu)= χ (u− )f(g) g G u U, ℓ → ℓ ∀ ∈ ∀ ∈

f := f(g) dg˙ < . k k1 | | ∞ G/UZ

The convolution

1 f f ′(g) := f(x)f ′(x− g)dx˙ g G ∗ ∀ ∈ G/UZ

and the involution

1 f ∗(g) := f(g ) g G − ∀ ∈ are well-deﬁned for f,f L1(G/U, χ ) and ′ ∈ ℓ

f f ′ f f ′ . k ∗ k1 ≤ k k1 k k1 Moreover, the linear mapping

p : L1(G) L1(G/U, χ ), p (F )(g) := F (gu)χ (u)du F L1(G) g G G/U → ℓ G/U ℓ ∀ ∈ ∀ ∈ UZ is a surjective -homomorphism between the algebras L1(G) and L1(G/U, χ ). ∗ ℓ Let I Gu,ℓ := [π] G π U = χℓ U π . ∈ | | H n o Then, Gu,ℓ is a closed subsetb of G, which canb be identiﬁed with the spectrum of the algebra L1(G/U, χ ). Indeed, it is easy to see that every irreducible unitary representation (π, ) ℓ Hπ whose equivalenceb class is locatedb in Gu,ℓ deﬁnes an irreducible representation (˜π, π) of the 1 H algebra L (G/U, χℓ) as follows: b π˜ p (F ) := π(F ) F L1(G). G/U ∀ ∈ 8 Similarly, if (˜π, ) is an irreducible unitary representation of L1(G/U, χ ), then [π] given by Hπ˜ ℓ π :=π ˜ p ◦ G/U

defines an element of Gu,ℓ. Let s g be a subspace of g such that g = g(ℓ) s. Since u contains [g, g], one has ⊂ ⊕ b G = χ π q (u + s)⊥ , u,ℓ q ⊗ ℓ | ∈ G letting πℓ := indP χℓ for a polarizationb p of ℓ and P := exp(p). 1 Denote by C∗(G/U, χℓ) the C∗-algebra of L (G/U, χℓ), whose spectrum can also be identified with Gu,ℓ. For π := indGχ , the Fourier transform defined by ℓ+q P ℓ+q F b (a)(q) := π (a) q (u + s)⊥ F ℓ+q ∀ ∈ then maps the C∗-algebra C∗(G/U, χℓ) onto the algebra C (u + s)⊥, ( π ) of the ∞ K H ℓ continuous mappings ϕ :(u + s) ( ) vanishing at infinity with values in the algebra ⊥ →K Hπℓ of compact operators on the Hilbert space of the representation π . Hπℓ ℓ If one restricts p to the Fréchet algebra (G) L1(G), its image is the Fréchet algebra G/U S ⊂ 1 (G/U, χ ) = f L (G/U, χ ) f smooth and for every subspace s′ g with g = s′ u S ℓ ∈ ℓ | ⊂ ⊕ and for S′ = exp(s′), f S′ (S′) . | ∈S

2.2 General results In this section, some general results shall be listed which are later needed in several proofs. Theorem 2.6 (Schur’s lemma). A unitary representation π of a topological group G on a Hilbert space is irreducible if and H only if the only bounded linear operators on commuting with π(g) for all g G are the H ∈ scalar operators. For the proof of this theorem see [20], Chapter I.3. Lemma 2.7. Let G be a Lie group and g the corresponding Lie algebra. Deﬁne the canonical projection pG going from g∗ to the space of coadjoint orbits g∗/G and equip the space g∗/G with the quotient 1 topology, i.e. a subset U in g∗/G is open if and only if pG− (U) is open in g∗. Then, a sequence ( k)k N of elements in g∗/G converges to the orbit g∗/G if and only if for any ℓ O ∈ O ∈ ∈ O there exists ℓk k for every k N such that ℓ = lim ℓk. ∈O ∈ k →∞ For the proof of this Lemma, see [23], Chapter 3.1.

Now, let G be a second countable locally compact group and (π, ) an irreducible Hπ unitary representation on the Hilbert space . A function of positive type of π is deﬁned Hπ to be a linear functional π C Cξ : G , g π(g)ξ,ξ π , −→ 7−→ h iH where ξ is a cyclic vector in . Hπ

9 The spectrum G has a natural topology which can be characterized in the following way: Theorem 2.8 (Topology of the spectrum). b Let (πk, πk )k N be a family of irreducible unitary representations of G. Then, (πk)k N con- H ∈ ∈ verges to (π, π) in G if and only if for some non-zero (respectively for every) vector ξ in π, H π H for every k N there exists ξ such that the sequence C k = π ( )ξ ,ξ k πk ξk k N k k k π k N ∈ ∈H ∈ · H k ∈ converges uniformlyb on compacta to Cπ = π( )ξ,ξ . ξ π · H The topology of G can also be expressed by the weak convergence of the coeﬃcient functions: Theorem 2.9. b Let πk, πk k N be a sequence of irreducible unitary representations of G. Then, πk, πk k N H ∈ H ∈ converges to(π, π) in G if and only if for some (respectively for every) unit vector ξ in π, H N H there is for every k a vector ξk π with ξk π 1 such that the sequence of linear ∈ ∈H k k kH k ≤ functionals Cπk convergesb weakly on some dense subspace of C (G) to Cπ. ξk k N ∗ ξ ∈ The proofs of the Theorems 2.8 and 2.9 can be found in [9], Theorem 13.5.2.

For the following corollary, let G be a Lie group, g its Lie algebra and (g) the uni- U versal enveloping algebra of g. Furthermore, for a unitary representation (π, ) of G, let Hπ := f G g π(g)f is smooth , the subspace of of smooth vectors. Hπ∞ ∈Hπ| ∋ 7→ ∈Hπ Hπ Corollary 2.10. Let πk, πk k N be a sequence of irreducible unitary representations of the Lie group G. If H ∈ πk, πk k N converges to (π, π) in G, then for each unit vector ξ in π∞, there exists for H ∈N H H every k a vector ξk ∞ such that the sequence dπk(D)ξk,ξk N converges to πk π k ∈ ∈ H b h H k ∈ dπ(D)ξ,ξ π for each D (g). h iH ∈U The proof below of this corollary originates from [11].

Proof: Let ξ a unit vector. It follows from [10], Theorem 3.3, that there are functions f ,...,f ∈Hπ∞ 1 s in C0∞(G), the space of all compactly supported C∞(G)-functions, and linearly independent vectors ξ ,...,ξ , such that ξ = π(f )ξ + ... + π(f )ξ . Since π is irreducible, for any 1 s ∈ Hπ 1 1 s s non-zero vector η π and every j 1...,s , there exist elements qj in the C∗-algebra of ∈ H s∈ { } G fulﬁlling ξj = π(qj)η. Hence, ξ = π(fj)π(qj)η. j=1 Now, choose for every k N vectorsPη such that Cπk converges weakly to Cπ. k πk ηk k N η ∈ s ∈ H ∈ Furthermore, for k N let ξk := πk(fj)πk(qj)ηk. Then, for D (g) it follows that ∈ j=1 ∈U P s s lim dπk(D)ξk,ξk π = lim πk(D fj)πk(qj)ηk, πk(fi)πk(qi)ηk k h iH k k ∗ →∞ →∞ π Xj=1 Xi=1 H k s s

= lim πk qi∗ fi∗ D fj qj ηk,ηk k ∗ ∗ ∗ ∗ π →∞ k Xi=1 Xj=1 D EH s s

= π qi∗ fi∗ D fj qj η,η = dπ(D)ξ,ξ π . ∗ ∗ ∗ ∗ π h iH i=1 j=1 H X X D E

10 3 The C∗-algebras of connected real two-step nilpotent Lie groups

In this section, the connected real two-step nilpotent Lie groups will be examined. In its first part, some preliminaries about two-step nilpotent Lie groups will be given which are needed in order to understand the setting and to prepare the proof of the norm controlled dual limit conditions listed in Definition 1.1. In Section 3.2, the Conditions 1, 2 and 3(a) will be verified and in Section 3.3, Condition 3(b) will be examined. Its proof is divided into three cases which are treated in the three following subsections. Subsequently, a result describing the C∗-algebras of connected real two-step nilpotent Lie groups will be given. After having finished the characterization of general connected real two-step nilpotent Lie groups, the example of the free two-step nilpotent Lie groups will be regarded.

The proof of Condition 1 of Definition 1.1 consists in the construction of the different layers of the spectrum of G and is based on a method of construction of a polarization developed in [27]. It is rather technical but not very long. Condition 2 follows directly from a result in [8] and Condition 3(a) is due to the construction of the above-mentioned polarization. The main work of the examination of the C∗-algebras of two-step nilpotent Lie groups consists in the proof of Property 3(b) of Definition 1.1 and in particular in the construction of the mappings (˜νk)k N. ∈ 3.1 Preliminaries 3.1.1 Two-step nilpotent Lie groups Let g be a real Lie algebra which is nilpotent of step two. This means that

[g, g] := span [X,Y ] X,Y g | ∈ is contained in the center of g. Fix a scalar product , on g like for general nilpotent Lie algebras. The Campbell-Baker- h· ·i Hausdorﬀ multiplication is then of the form 1 u v = u + v + [u,v] u,v g · 2 ∀ ∈ and one gets again the simply connected connected Lie group G =(g, ) with Lie algebra g. · As g is two-step nilpotent, [g, g] g(ℓ) and thus g(ℓ) is an ideal of g. ⊂ Furthermore, for ℓ g , every maximal isotropic subspace p of g for B containing [g, g] is a ∈ ∗ ℓ polarization of ℓ.

3.1.2 Induced representations G The induced representation σℓ,p = indP χℓ for a polarization p of ℓ and P := exp(p) can be described in the following way: Since p contains [g, g] and even the center z of g, one can write g = s p and p = t z for two ⊕ ⊕ subspaces t and s of g. The quotient space G/P is then homeomorphic to s and the Lebesgue measure ds on s deﬁnes an invariant Borel measure dg˙ on G/P . The group G acts by the left 2 translation σℓ,p on the Hilbert space L (G/P, χℓ).

11 Now, if one uses the coordinates G = s p, one can identify the Hilbert spaces L2(G/P, χ ) · ℓ and L2(s)= L2(s,ds): Let U : L2(s,ds) L2(G/P, χ ) be deﬁned by ℓ → ℓ U (ϕ)(S Y ) := χ ( Y )ϕ(S) Y p S s ϕ L2(s). ℓ · ℓ − ∀ ∈ ∀ ∈ ∀ ∈

Then, Uℓ is a unitary operator and one can transform the representation σℓ,p into a represen- 2 tation πℓ,p on the space L (s):

π := U ∗ σ U . (1) ℓ,p ℓ ◦ ℓ,p ◦ ℓ

Furthermore, one can express the representation σℓ,p in the following way:

1 1 σ (S Y )ξ(R) = ξ Y − S− R ℓ,p · 1 1 = ξ (R S) Y + [R,S] [R S,Y ] − · − 2 − 2 − 2πi ℓ, Y + 1 [R,S] 1 [R S,Y ] 2 = e h − 2 − 2 − iξ(R S) R,S s Y p ξ L (G/P, χ ) − ∀ ∈ ∀ ∈ ∀ ∈ ℓ and hence,

2πi ℓ, Y + 1 [R,S] 1 [R S,Y ] 2 π (S Y )ϕ(R) = e h − 2 − 2 − iϕ(R S) R,S s Y p ϕ L (s). (2) ℓ,p · − ∀ ∈ ∀ ∈ ∀ ∈ 3.1.3 Orbit method In the case of two-step nilpotent Lie groups, for every ℓ g and for every g G =(g, ), the ∈ ∗ ∈ · element Ad ∗(g)ℓ can be computed to be

Ad ∗(g)ℓ = Ig∗ + ad ∗(g) ℓ ℓ + g(ℓ)⊥. ∈ Therefore, as ad ∗(g)ℓ = g(ℓ)⊥,

:= Ad ∗(G)ℓ = Ad ∗(g)ℓ g G = ℓ + g(ℓ)⊥ ℓ g∗, (3) Oℓ { | ∈ } ∀ ∈ i.e. only ﬂat orbits appear. Hence,

g∗/G = Ad ∗(G)ℓ ℓ g∗ = ℓ + g(ℓ)⊥ ℓ g∗ . { | ∈ } | ∈ Thus, by the Kirillov theory, one gets

G = g∗/G = ℓ + g(ℓ)⊥ ℓ g∗ ∼ | ∈ as topological spaces. b

Now, let [πk] k N G be a properly converging sequence in G with limit set L [πk] k N . ∈ ⊂ ∈ Let g∗/Gbe the coadjoint orbit of some [π] L [πk] k N , k the coadjoint orbit of O ∈ b ∈ ∈b O [πk] for every k N and let ℓ . Then, by Lemma 2.7, there exists for every k N an ∈ ∈ O ∈ element ℓk k, such that lim ℓk = ℓ in g∗. One can assume that, passing to a subsequence ∈O k →∞ if necessary, the sequence (g(ℓk))k N converges in the topology of the Grassmannian to a ∈

12 subalgebra u of g(ℓ) and that there exists a number d N, such that dim( ) = 2d for every ∈ Ok k N. Then, it follows from (3) that ∈ L(( k)k N)= lim ℓk + g(ℓk)⊥ = ℓ + u⊥ = ℓ + u⊥ g∗/G. (4) O ∈ k O ⊂ →∞ Since in the two-step nilpotent case g(ℓ ) contains [g, g] for every k N, the subspace u also k ∈ contains [g, g]. Hence, as the Kirillov mapping is a homeomorphism and u⊥ consists only of characters of g, the limit set L [πk] k N of the sequence [πk] k N in G is the “aﬃne” subset ∈ ∈ G L [πk] k N = χq indP χℓ q u⊥ b ∈ ⊗ | ∈ for a polarization p of ℓ and P := exp( p).

The observations above lead to the following proposition. Proposition 3.1. There are three diﬀerent types of possible limit sets of the sequence ( k)k N of coadjoint orbits: O ∈ 1. dim( ℓ) = 2d: In this case, the limit set L(( k)k N) is the singleton ℓ = ℓ + g(ℓ)⊥, O O ∈ O i.e. u = g(ℓ).

N 2. dim( ℓ) = 0: Here, L(( k)k ) = ℓ + u⊥ and the limit set L [πk] k N consists only O O ∈ ∈ of characters, i.e. q [g, g] = 0 for all q L(( k)k N). { } O ∈ O ∈ 3. The dimension of the orbit is strictly larger than 0 and strictly smaller than 2d. In Oℓ this case, 0 < dim( ) < 2d for every L(( k)k N) and O O∈ O ∈ G N L(( k)k )= q + ℓ, i.e. L [πk] k N = χq indP χℓ O ∈ O ∈ ⊗ q u⊥ q u⊥ ∈[ [∈ for a polarization p of ℓ and P := exp(p).

3.2 Conditions 1, 2 and 3(a) Now, to start with the proof of the conditions listed in Definition 1.1, the families of sets (Si)i 0,...,r and (Γi)i 0,...,r are going to be defined and the Properties 1, 2 and 3(a) of Definition∈{ 1.1} are going∈{ to be} verified. In order to be able to define the required families (Γi)i 0,...,r and (Si)i 0,...,r , one needs to ∈{ } ∈{ } construct a polarization pV of ℓ g in the following way: ℓ ∈ ∗ Let H ,...,H be a Jordan-Hölder basis of g, in such a way that g := span H ,...,H { 1 n} i { i n} for i 0,...,n is an ideal in g. Since g is two-step nilpotent, one can first choose a basis ∈ { } Hn˜, ,Hn of [g, g] and then add the vectors H1, ,Hn˜ 1 to obtain a basis of g. Let { ··· } ··· − IPuk := i n g(ℓ) g = g(ℓ) g ℓ { ≤ | ∩ i ∩ i+1} be the Pukanszky index set for ℓ g . The number of elements IPuk of IPuk is the dimension ∈ ∗ ℓ ℓ of the orbit of ℓ. Oℓ Moreover, if one denotes by gi(ℓ gi ) the stabilizer of ℓ gi in gi, | | n V pℓ := gi(ℓ gi ) | Xi=1 13 is the Vergne polarization of ℓ in g. Its construction will now be analyzed by a method developed in [27], Section 1. So, let ℓ g . Then, choose the largest index j (ℓ) 1,...,n such that H g(ℓ) and let ∈ ∗ 1 ∈ { } j1(ℓ) 6∈ Y V,ℓ := H . Moreover, choose the index k (ℓ) 1,...,n such that ℓ, [H ,H ] = 0 1 j1(ℓ) 1 ∈ { } k1(ℓ) j1(ℓ) 6 and ℓ, [H ,H ] = 0 for all i>k (ℓ) and let XV,ℓ := H . i j1(ℓ) 1 1 k1(ℓ) Next, let g1,ℓ := U g ℓ, U, Y V,ℓ = 0 . Then, g1,ℓ is an ideal in g which does not contain ∈ | 1 XV,ℓ, and g = RXV,ℓ g1,ℓ. Now, the Jordan-Hölder basis will be changed, taking out H . 1 1 k1(ℓ) ⊕ 1,ℓ 1,ℓ 1,ℓ 1,ℓ 1,ℓ Consider the Jordan-Hölder basis H ,...,H ,H ,...,Hn of g , where 1 k1(ℓ) 1 k1(ℓ)+1 − V,ℓ V,ℓ ℓ, Hi,Y X H1,ℓ := H i>k (ℓ) and H1,ℓ := H 1 1 i

Then, choose the largest index j2(ℓ) 1,...,k1(ℓ) 1,k1(ℓ) + 1,...,n in such a way that 1,ℓ 1,ℓ V,ℓ 1,ℓ∈ { − } Hj (ℓ) g (ℓ g1,ℓ ) and let Y2 := Hj (ℓ). Again, choose k2(ℓ) 1,...,k1(ℓ) 1,k1(ℓ)+1,...,n 2 6∈ | 2 ∈ { − } 1,ℓ 1,ℓ 1,ℓ 1,ℓ in such a way that ℓ, [H ,H ] = 0 and ℓ, [H ,H ] = 0 for all i>k2(ℓ) and set k2(ℓ) j2(ℓ) 6 i j2(ℓ) XV,ℓ := H1,ℓ . 2 k2(ℓ) Iterating this procedure, one gets sets Y V,ℓ,...,Y V,ℓ and XV,ℓ,...,XV,ℓ for d 0,..., [ n ] 1 d 1 d ∈ { 2 } with the properties pV = span Y V,ℓ,...,Y V,ℓ g(ℓ) ℓ 1 d ⊕ and ℓ, XV,ℓ,Y V,ℓ = 0, ℓ, XV,ℓ,Y V,ℓ = 0 i = j 1,...,d and i i 6 i j ∀ 6 ∈ { } ℓ, Y V,ℓ ,YV,ℓ = 0 i,j 1,...,d . i j ∀ ∈ { } Now, let

J(ℓ) := j (ℓ), ,j (ℓ) and K(ℓ) := k (ℓ), ,k (ℓ) . { 1 ··· d } { 1 ··· d } Then,

IPuk = J(ℓ) ˙ K(ℓ) and j (ℓ) >...>j (ℓ). ℓ ∪ 1 d Puk The index sets Iℓ , J(ℓ) and K(ℓ) are the same on every coadjoint orbit (see [27]) and can therefore also be denoted by IPuk, J( ) and K( ) if ℓ is located in the coadjoint orbit . O O O O Now, for the parametrization of g∗/G and thus of G and for the choice of the concrete realization of a representation required in Property 1(b) of Deﬁnition 1.1, let g /G.A O ∈ ∗ theorem of L.Pukanszky (see [29], Part II, Chapter I.3b or [27], Corollary 1.2.5) states that Puk there exists one unique ℓ such that ℓ (Hi) = 0 for all i I . Choose this ℓ , let O ∈ O O ∈ O P V := exp(pV ) and deﬁne the irreducible unitary representation O ℓO ℓO

V G π := ind V χ ℓO P ℓO ℓO

2 V 2 d associated to the orbit and acting on L G/P ,χℓ = L (R ). O ℓO O ∼

14 Next, one has to construct the demanded sets Γ for i 0,...,r . i ∈ { } For this, deﬁne for a pair of sets (J, K) such that J, K 1, ,n , J = K and J K = ⊂ { ··· } | | | | ∩ ∅ the subset (g∗/G)(J,K) of g∗/G by

(g∗/G) := g∗/G (J, K)=(J( ),K( )) . (J,K) {O ∈ | O O } Moreover, let

:= (J, K) J, K 1,...,n , J K = , J = K , (g∗/G) = M { | ⊂ { } ∩ ∅ | | | | (J,K) 6 ∅} and Puk (g∗/G)2d := g∗/G I = 2d . O∈ O n o Then, (g∗/G)2d = (g∗/G)(J,K) (J,K): J,K 1,...,n , J = K =[d,⊂{ J K=} | | | | ∩ ∅ and g∗/G = (g∗/G)2d = (g∗/G)(J,K). d 0,...,[ n ] (J,K) ∈{ [ 2 } [∈M Now, an order on the set shall be introduced. M First, if J = K = d, J = K = d and d

V Γ := π (g∗/G) and iJK ℓO O∈ (J,K) n o

SiJK := Γi.

i 0,...,iJK ∈{ [ }

Then, obviously, the family (Si)i 0,...,r is an increasing family in G. Furthermore, the set S is closed∈{ for every} i 0,...,r . This can easily be deduced from the i ∈ { } definition of the index sets J(ℓ) and K(ℓ). The indices jm(ℓ) andbkm(ℓ) for m 1,...,d m∈1,ℓ { } are chosen such that they are the largest to fulfill a condition of the type ℓ, [H − , ] = 0 jm(ℓ) · 6 m 1,ℓ or ℓ, [H − , ] = 0, respectively. For a more detailed proof of the closure of the sets km(ℓ) · 6 (Si)i 0,...,r , see Lemma 7.1. ∈{ } In addition, the sets Γi are Hausdorff. For this, let i = iJK for (J, K) and ( k)k N ∈ M O ∈ in (g /G) be a sequence of orbits such that the sequence πV converges in Γ , ∗ (J,K) ℓO k N i k ∈ i.e. ( k)k N converges in (g∗/G)(J,K) and thus has a limit point in (g∗/G)(J,K). If now ∈ O k O k ℓk →∞ ℓ , then by (4), it follows that the limit u of the sequence (g(ℓk))k N is equal O ∋ −→ ∈O ∈

15 to g(ℓ). Therefore, the sequence ( ) N and thus also the sequence πV have unique k k ℓO k N O ∈ k ∈ limits and hence, Γi is Hausdorff. Moreover, one can still observe that for d = 0 the choice J = K = represents the only ∅ possibility to get J = K = d. So, the pair ( , ) is the first element in the above defined | | | | ∅ ∅ order and therefore corresponds to 0. Thus,

V Puk Γ0 = πℓ I = , O O ∅ n o which is equivalent to the fact that g(ℓ )= g which again is equivalent to the fact that every V O π Γ0 is a character. Hence, S0 = Γ0 is the set of all characters of G, as demanded. ℓO ∈ Since one can identify G/P V with Rd by means of the subspace s = span XV,ℓO ,...,XV,ℓO , ℓO ℓO 1 d 2 V V one can thus identify the Hilbert space L G/P ,χℓ , that the representation π acts on, ℓO O ℓO with the Hilbert space L2(Rd) for every (g /G) as in (1). Therefore, one can take for O ∈ ∗ 2d all J, K 1,...,n with J = K = d the Hilbert space L2(Rd) as the desired Hilbert space ∈ { V 2} d | | | | i and π ,L (R ) thus represents the demanded concrete realization. H JK ℓO Hence, the first condition of Definition 1.1 is fulfilled. For the proof of the Properties 2 and 3(a), a proposition will be shown.

Proposition 3.2. For every a C (G) and every (J, K) with J = K = d 0,..., [ n ] , the mapping ∈ ∗ ∈M | | | | ∈ { 2 } Γ L2(Rd), γ (a)(γ) iJK → 7→ F is norm continuous and the operator (a)(γ) is compact for all γ Γ . F ∈ iJK Proof: The compactness follows directly from a general theorem which can be found in [8], Chapter 4.2 or [29], Part II, Chapter II.5 and states that the C∗-algebra C∗(G) of every connected nilpotent Lie group G fulﬁlls the CCR condition, i.e. the image of every irreducible representation of C∗(G) is a compact operator.

n Next, let d 0,..., [ 2 ] and (J, K) such that J = K = d. ∈ { } ∈M | | V | | First, one has to observe that the polarization pℓ is continuous in ℓ on the set ℓ ′ ′ (g∗/G)(J,K) with respect to the topology of the Grassmannian. This can O | O ∈ V,ℓ V,ℓ be seen by the construction of the vectors Y1 ,...,Yd . Now, let ( k)k N be a sequence of orbits in (g∗/G)(J,K) and (g∗/G)(J,K) such that ∈ k O k O ∈ V →∞ V →∞ πℓ πℓ and let a C∗(G). Then, ℓ k ℓ and by the observation above, Ok −→ O ∈ O −→ O the associated sequence of polarizations pV converges to the polarization pV . By the ℓO k N ℓO k ∈ V k V proof of Theorem 2.3 in [31], thus, πℓ (a) →∞ πℓ (a) in the operator norm. Ok −→ O

Since C∗(G) is obviously separable, this proposition proves the desired Properties 2 and 3(a) of Deﬁnition 1.1 and hence, it remains to show Property 3(b).

16 3.3 Condition 3(b) 3.3.1 Introduction to the setting For simplicity, in the following, the representations will be identiﬁed with their equivalence classes.

Let d 0,..., [ n ] and (J, K) with J = K = d. Furthermore, fix i = i 0,...,r . ∈ { 2 } ∈M | | | | JK ∈ { } V V Let π N = π N be a sequence in Γi whose limit set is located outside Γi. k k ℓOk k Since every∈ converging sequence∈ has a properly converging subsequence, it will be assumed V that πk k N is properly converging and the transition to a subsequence will be omitted. ∈ Hence, the sequence πV takes on the role of the subsequence (γ ) N in Condition 3(b) k k N k k of Definition 1.1. Therefore,∈ for every k N, one has to construct∈ an involutive linear ∈ mappingν ˜k : CB(Si 1) ( i), bounded by C Si and fulfilling − →B H k · k −1 V lim (a) πk ν˜k (a) Si−1 = 0, k F − F | op →∞ with a constant C > 0 independent of k. In order to do so, regard the sequence of coadjoint orbits ( k)k N corresponding to the V O ∈ sequence πk k N. It is contained in (g∗/G)(J,K) and in particular, every k has the same ∈ O dimension 2d. Moreover, it converges properly to a set of orbits L(( k)k N). V O ∈ V In addition, since Si is closed, the limit set L πk k N of the sequence πk k N is contained ∈ ∈ in Si 1 = Γj and therefore, for every element L(( k)k N) there exists a pair − j 0,...,i 1 O ∈ O ∈ ∈{ S − } V (J ,K ) < (J, K) such that πℓ ΓiJ K or equivalently, (g∗/G)(JO,KO). O O O ∈ O O O∈ Now, the Lie algebra g will be examined and divided into different parts. With their help, a new sequence of representations (πk)k N which are equivalent to the representations V ∈ πk k N will be defined. Then, in the second and third case mentioned in Proposition 3.1, ∈ (πk)k N will be analyzed and for this new sequence, mappings (νk)k N with certain properties ∈ ∈ similar to the ones required in Condition 3(b) of Definition 1.1 will be constructed. At the end, the requested convergence will be deduced from the convergence of (πk)k N together V ∈ with the equivalence of the representations πk and πk .

3.3.2 Changing the Jordan-H¨older basis

˜ ˜ N ˜ N Let ℓ L(( k)k ). Then, there exists a sequence ℓk k N in ( k)k such that ∈ O ∈ O ∈ ∈ O ∈ ℓ˜= lim ℓ˜k. k →∞ ˜ ˜ ⊥ ˜ Since one is interested in the orbits k = ℓk + g ℓk , one can change the sequence ℓk k N O ∈ to a sequence (ℓk)k N by letting ℓk(A) = 0 for every A g ℓ˜k ⊥ = g(ℓk)⊥. ∈ ∈ Thus, one obtains another converging sequence (ℓk)k N in ( k)k N whose limit ℓ is located ∈ O ∈ in an orbit L(( k)k N). O∈ O ∈ In the following, this sequence (ℓk)k N will be investigated and with its help, the above- mentioned splitting of g will be accomplished.∈

17 As above, one can suppose that the subalgebras (g(ℓk))k N converge to a subalgebra u, whose corresponding Lie group exp(u) is denoted by U. These∈ subalgebras g(ℓ ) for k N can be k ∈ written as g(ℓ )=[g, g] s , k ⊕ k where sk [g, g]⊥. In addition, let nk,0 be the kernel of ℓk [g,g] and sk,0 the kernel of ℓk s for ⊂ | | k all k N. One can assume that s = s and choose T s of length 1 orthogonal to s . ∈ k,0 6 k k ∈ k k,0 The case s = s for k N, being easier, will be omitted. k,0 k ∈ Similarly, choose Zk [g, g] of length 1 orthogonal to nk,0. One can easily see that such a Zk ∈ V exists: If ℓk [g,g] = 0, then πℓ is a character and thus contained in S0 = Γ0. This happens | Ok only in the case i = 0 and then the whole sequence πV is contained in S . But S = Γ ℓO k N 0 0 0 k ∈ is closed and thus, πV cannot have a limit set outside Γ , as regarded in the setting ℓO k N 0 k ∈ of Condition 3(b) of Deﬁnition 1.1.

Furthermore, let r = g(ℓ ) g. k k ⊥ ⊂ One can assume that, passing to a subsequence if necessary, lim Zk =: Z, lim Tk =: T and k k →∞ →∞ lim rk =: r exist. k →∞ Now, new polarizations pk of ℓk are going to be constructed in order to deﬁne the represenations (πk)k N with their help. ∈ The restriction to rk of the skew-form Bk deﬁned by B (V,W ) := ℓ , [V,W ] , V,W g k h k i ∀ ∈ is non-degenerate on rk and there exists an invertible endomorphism Sk of rk such that

x,S (x′) = B (x,x′) x,x′ r . h k i k ∀ ∈ k S is skew-symmetric, i.e. St = S , and with the help of Lemma 7.2, one can decompose r k k − k k into an orthogonal direct sum d j rk = Vk Xj=1 of two-dimensional S -invariant subspaces. Choose an orthonormal basis Xk,Y k of V j. k { j j } k Then,

[Xk,Xk] n i,j 1,...,d , i j ∈ k,0 ∀ ∈ { } [Y k,Y k] n i,j 1,...,d and i j ∈ k,0 ∀ ∈ { } [Xk,Y k]= δ ckZ mod n i,j 1,...,d , i j i,j j k k,0 ∀ ∈ { } k R k where 0 = cj and sup cj < for every j 1,...,d . 6 ∈ k N ∞ ∈ { } ∈ k Again, by passing to a subsequence if necessary, the sequence (cj )k N converges for every ∈ j 1,...,d to some cj R. ∈ { k }k ∈ k k Since Xj ,Yj rk and ℓk(A) = 0 for every A rk, ℓk(Xj )= ℓk(Yj ) = 0 for all j 1,...,d . ∈ ∈ k k ∈ { } Furthermore, one can suppose that the sequences (Xj )k N, (Yj )k N converge in g to vectors ∈ ∈ Xj,Yj which form a basis modulo u in g.

18 It follows that k k k ℓk, [Xj ,Yj ] = cj λk, where k λ = ℓ ,Z →∞ ℓ,Z =: λ. k h k ki −→ h i As Z was chosen orthogonal to n , λ = 0 for every k N. k k,0 k 6 ∈ Now, let p := span Y k, ,Y k, g(ℓ ) k { 1 ··· d k } and Pk := exp(pk). Then, pk is a polarization of ℓk. Furthermore, deﬁne the representation πk as G πk := indPk χℓk . V Since πk as well as πk are induced representations of polarizations and of the characters χℓk and χℓ , where ℓk and ℓ lie in the same coadjoint orbit k, the two representations are Ok Ok O equivalent, as observed in Section 2.1.2.

Let ak := nk,0 + sk,0. Then, ak is an ideal of g on which ℓk is 0. Therefore, the normal subgroup exp(ak) is contained in the kernel of the representation πk. Moreover, let a := lim ak. k →∞ N k k In addition, let p ,p ˜ 1,...,p and let A1, ,Ap˜ denote an orthonormal basis of ∈ ∈ { } { ··· k} k nk,0, the part of ak which is located inside [g, g], and Ap˜+1, ,Ap an orthonormal basis k k { ··· } of sk,0, the part of ak outside [g, g]. Then, A1, ,Ap is an orthonormal basis of ak and as k { ··· } above, one can assume that lim Aj = Aj exists for all j 1,...,p . k ∈ { } Now, for every k N, one can→∞ take as an orthonormal basis for g the set of vectors ∈ Xk, ,Xk,Y k, ,Y k,T ,Z ,Ak, ..., Ak { 1 ··· d 1 ··· d k k 1 p} as well as the set X , ,X ,Y , ,Y ,T,Z,A , ..., A . { 1 ··· d 1 ··· d 1 p} This gives the following Lie brackets: k k k [Xi ,Yj ] = δi,jcj Zk mod ak, k k [Xi ,Xj ] =0 mod ak and k k [Yi ,Yj ] =0 mod ak.

The vectors Zk and Tk are central modulo ak.

3.3.3 Deﬁnitions

Before starting the analysis of (πk)k N, some notations have to be introduced. ∈

Choose for j 1,...,d the Schwartz functions η (R) such that η 2 R = 1 and ∈ { } j ∈ S k jkL ( ) η ∞ R 1. k jkL ( ) ≤ Furthermore, for x ,...,x ,y ,...,y ,t,z,a , ,a R, write 1 d 1 d 1 ··· p ∈ d d k k (x)k := (x1,...,xd)k := xjXj , (y)k := (y1,...,yd)k := yjYj , (t)k := tTk, Xj=1 Xj=1 19 p˜ p k k (z)k := zZk, (˙a)k := (a1,...,ap˜)k := ajAj , (¨a)k := (ap˜+1,...,ap)k := ajAj Xj=1 j=˜Xp+1 p k and (a)k := (˙a, a¨)k =(a1,...,ap)k = ajAj , Xj=1 where ( ,..., ) is deﬁned to be the d-,p ˜-, (p p˜)- or the p-tuple with respect to the bases · · k − Xk,...,Xk , Y k,...,Y k , Ak, ..., Ak , Ak , ..., Ak and Ak, ..., Ak , respectively, and let { 1 d } { 1 d } { 1 p˜} { p˜+1 p} { 1 p}

(g)k := (x1,...,xd,y1,...,yd,t,z,a1,...,ap)k := ((x)k, (y)k, (t)k, (z)k, (˙a)k, (¨a)k)

= ((x)k, (h)k) d d p k k k = xjXj + yjYj + tTk + zZk + ajAj , Xj=1 Xj=1 Xj=1 where (h) is in the polarization p and the (2d+2+p)-tuple ( ,..., ) is regarded with respect k k · · k to the basis Xk,...,Xk,Y k,...,Y k,T ,Z ,Ak, ..., Ak . { 1 d 1 d k k 1 p} Moreover, deﬁne the limits

d d (x) := (x1,...,xd) := lim (x)k = xjXj, (y) := (y1,...,yd) := lim (y)k = yjYj, ∞ ∞ k ∞ ∞ k →∞ →∞ Xj=1 Xj=1 p˜

(t) := lim (t)k = tT, (z) := lim (z)k = zZ, (˙a) := (a1,...,ap˜) := lim (˙a)k = ajAj, ∞ k ∞ k ∞ ∞ k →∞ →∞ →∞ Xj=1 p

(¨a) := (ap˜+1,...,ap) := lim (¨a)k = ajAj, ∞ ∞ k →∞ j=˜Xp+1 p

(a) := (˙a, a¨) =(a1,...,ap) := lim (a)k = ajAj and ∞ ∞ ∞ k →∞ Xj=1 d d p (g) := (x,y,t,z, a,˙ a¨) := lim (g)k = xjXj + yjYj + tT + zZ + ajAj. ∞ ∞ k →∞ Xj=1 Xj=1 Xj=1

3.3.4 Formula for πk

Now, the representations (πk)k N can be computed. Let f L1(G). ∈ ∈ d k k k R k With ρk := ℓk,Tk , c := (c1,...,cd) and for s1,...,sd and (s)k := (s1,...,sd)k = sjXj , h i ∈ j=1 k k P where again ( ,..., )k is the d-tuple with respect to the basis X1 ,...,Xd , as in (2), the · · 2 { } representation πk acts on L (G/Pk,χℓk ) in the following way:

20 1 π (g) ξ (s) = ξ (g)− (s) k k k k · k 1 = ξ (s) (x) (h) + [ (x) (h) , (s) ] k − k − k 2 − k − k k 1 = ξ (s) (x) (h) + [(s) , (h) ] [(x) , (h) +(s) ] k − k · − k k k − 2 k k k 2πi ℓ , (h) +[(s) ,(h) ] 1 [(x) ,(h) +(s) ] = e h k − k k k − 2 k k k iξ (s x) − k 1 1 2πi ℓk, (y)k (t)k (z)k (˙a)k (¨a)k+[(s)k (x)k,(y)k] [(x)k,(s)k] = e h − − − − − − 2 − 2 iξ (s x) − k d k 1 2πi tρk zλk+ P λkcj (sj 2 xj )yj − − j=1 − = e ξ (s x)k (5) k 1 − 2πi( tρk zλk+λkc ((s)k (x)k)(y)k) = e − − − 2 ξ (s x)k , − k since ℓk(Y ) = 0 for all j 1,...,d . j ∈ { } Rd Rd R R Rp˜ Rp p˜ R2d+2+p 2 Rd Rd Now, identify G with − ∼= , let ξ L ( ) and s . × × × × × 2 d ∈ ∈ Moreover, identify πk with a representation acting on L (R ) which will also be called πk. To stress the dependence on k of the function f L1(G) ﬁxed above, denote by f L1(R2d+2+p) ∈ k ∈ the function f applied to an element in the k-basis: d d p k k k fk(g) := f((g)k)= f xjXj + yjYj + tTk + zZk + ajAj . ! Xj=1 Xj=1 Xj=1 Denote by f for f (G) and α N2d+2+p the derivative of f in the direction of α,k′ ∈ S ∈ (Xk)α1 (Xk)αd (Y k)αd+1 (Y k)α2d (T )α2d+1 (Z )α2d+2 (Ak)α2d+3 (Ak)α2d+2+p . Then, 1 ··· d 1 ··· d k k 1 ··· p 2 N f = 1+ g f ′ (g) dg. k kk(N) k k α,k α N 2dZ+2+p | X|≤ R

Since the k-basis of G converges to the -basis, one can estimate the expressions f (g) ∞ α,k′ for g R2d+2+p and α N by constants independent of k and therefore gets an estimate ∈ | | ≤ for fk (N) that does not depend on k. k k ˆ2,3,4,5,6 Now, denoting by fk the Fourier transform in the 2nd, 3rd, 4th, 5th and 6th variable,

πk(f)ξ(s)

= fk(g)πk(g)ξ(s)dg

Rd Rd R ZR Rp˜ Rp−p˜ × × × × × 2πi tρ zλ +λ ck(s 1 x)y = f (x,y,t,z, a,˙ a¨)e − k− k k − 2 ξ(s x)d(x,y,t,z, a,˙ a¨) k − Rd Rd R ZR Rp˜ Rp−p˜ × × × × × 2πi tρ zλ + 1 λ ck(s+x)y = f (s x,y,t,z, a,˙ a¨)e − k− k 2 k ξ(x)d(x,y,t,z, a,˙ a¨) k − Rd Rd R ZR Rp˜ Rp−p˜ × × × × × λ ck = fˆ2,3,4,5,6 s x, k (s + x), ρ ,λ , 0, 0 ξ(x)dx. (6) k − − 2 k k RZd

21 3.4 Condition 3(b) – First case

First, consider the case that L(( k)k N) consists of one single limit point , the first case O ∈ O described in Proposition 3.1. In this case, for every k N, ∈ 2d = dim( ) = dim( ). Ok O Thus, the regarded situation occurs if and only if λ = 0 and c = 0 for every j 1,...,d . 6 j 6 ∈ { } The first case is the easiest case. After a few observations, here the operators (˜νk)k N can be defined immediately. Moreover, the claims of Condition 3(b) of Definition 1.1 can∈ be shown easily. In this first case, the transition to the representations (πk)k N and the definitions and computations accomplished in the Sections 3.3.3 and 3.3.4 are not∈ needed. Consider again the sequence (ℓk)k N chosen above which converges to ℓ . As the dimen- ∈ ∈ O sions of the orbits k and are the same, there exists a subsequence of (ℓk)k N (which will O O V ∈ also be denoted by (ℓk)k N for simplicity) such that p := lim pℓ is a polarization of ℓ, but ∈ k k →∞ V not necessarily the Vergne polarization. Moreover, define P := exp(p)= lim Pℓ and let k k →∞ G π := indP χℓ.

V G Now, if one identiﬁes the Hilbert spaces V and π of the representations π = ind V χℓ πℓ ℓk P k H k H ℓk and π with L2(Rd), from [31], Theorem 2.3, one can conclude that

V G G k π (a) π(a) = ind V χ (a) ind χ (a) →∞ 0 ℓk op P ℓk P ℓ − ℓk − op −→

for every a C∗(G ). ∈ V G Since π and π = ind V χℓ are both induced representations of polarizations and of the same ℓ Pℓ character χℓ, they are equivalent and hence, there exists a unitary intertwining operator 2 Rd 2 Rd V F : πV = L ( ) π = L ( ) such that F πℓ (a)= π(a) F a C∗(G). H ℓ ∼ →H ∼ ◦ ◦ ∀ ∈ V V G V G Furthermore, the two representations π = π = ind V χℓO and π = ind V χℓk are k ℓOk Pℓ k ℓk Pℓ Ok k equivalent for every k N because ℓ and ℓk are located in the same coadjoint orbit k and ∈ Ok O pV and pV are polarizations. Thus, there exist further unitary intertwining operators ℓOk ℓk

2 d 2 d V V V R V R Fk : π = L ( ) π = L ( ) such that Fk πk (a)= πℓk (a) Fk a C∗(G). H k ∼ →H ℓk ∼ ◦ ◦ ∀ ∈ Now, deﬁne the required operatorsν ˜ for every k N as k ∈ V ν˜k(ϕ) := Fk∗ F ϕ πℓ F ∗ Fk ϕ CB(Si 1), ◦ ◦ ◦ ◦ ∀ ∈ − V V which is reasonable since πℓ is a limit point of the sequence πk k N and hence contained ∈ in Si 1, as seen in Section 3.3.1. − As ϕ πV (L2(Rd)) and F and F are intertwining operators and thus bounded, the ℓ ∈ B k image ofν ˜ is contained in (L2(Rd)), as requested. k B

22 Next, it needs to be shown thatν ˜ is bounded. By the deﬁnition of , one has for every k k · kSi−1 ϕ CB(Si 1), ∈ − V ν˜ (ϕ) = F ∗ F ϕ π F ∗ F k k kop k ◦ ◦ ℓ ◦ ◦ k op V F ∗ op F op ϕ π F ∗ op F op ≤ k k k k k ℓ op k k k kk = ϕ πV ℓ op ϕSi−1. ≤ k k

In addition, one can easily observe thatν ˜k is involutive. For every ϕ CB(Si 1), ∈ − V V ν˜ (ϕ)∗ = F ∗ F ϕ π F ∗ F ∗ = F ∗ F ϕ∗ π F ∗ F =ν ˜ (ϕ∗). k k ◦ ◦ ℓ ◦ ◦ k k ◦ ◦ ℓ ◦ ◦ k k Now, the last thing to check is the required convergence of Condition 3(b). For all a C (G), ∈ ∗ V V V πk (a) ν˜k (a) Si = πk (a) Fk∗ F (a) Si πℓ F ∗ Fk − F | −1 op − ◦ ◦ F | −1 ◦ ◦ op V V = π (a) F ∗ F π (a) F∗ F k − k ◦ ◦ ℓ ◦ ◦ k op V = F ∗ π (a) F F ∗ π(a) F k ◦ ℓk ◦ k − k ◦ ◦ k op V = F ∗ π π (a) F k ◦ ℓk − ◦ k op k πV (a) π(a) →∞ 0. ≤ ℓk − op −→

V N Therefore, the representations πk k N and the constructed (˜νk)k fulfill Condition 3(b) of Definition 1.1 and thus, in this case,∈ the claim is shown. ∈ 3.5 Condition 3(b) – Second case In the second case of Condition 3(b) of Definition 1.1 described in Proposition 3.1, the situation that λ = 0 or c = 0 for every j 1,...,d is going to be considered. j ∈ { } In this case,

k ℓ , [Xk,Y k] = ckλ →∞ c λ = 0 j 1,...,d , k j j j k −→ j ∀ ∈ { } k while cj λk = 0 for every k and every j 1,...,d . 6 ˜ ∈ { } ˜ Then, ℓ [g,g] = 0 and since ℓ [g,g] obviously also vanishes for all ℓ u⊥, every L(( k)k N) | | ∈ O ∈ O ∈ vanishes on [g, g], which means that the associated representation is a character. Therefore, every limit orbit in the set L(( k)k N) has the dimension 0. O O ∈

3.5.1 Convergence of (πk)k N in G ∈ In order to better understand the sequence of representations (πk)k N and to get an idea of b ∈ how to construct the required linear mappings (νk)k N, in this section, the convergence in G ∈ of (πk)k N is going to be analyzed. This shall serve as a motivation for the construction of ∈ the mappings (νk)k N which will be carried out subsequently. b ∈

23 As in Calculation (6) in Section 3.3.4, identify G again with R2d+2+p. From now on, this identification will be used most of the time. Only in some cases where one applies ℓk or ℓ and thus it is important to know whether one is using the basis depending on k or the limit basis, the calculation will be done in the above defined bases ( )k or ( ) . · · ∞ Now, adapt the methods developed in [26] to this given situation. Let s =(s ,...,s ), α =(α , ,α ), β =(β , ,β ) Rd and define 1 d 1 ··· d 1 ··· d ∈ d 1 1 β 2πiαs k 4 k j ηk,α,β(s)= ηk,α,β(s1, ,sd) := e λkc ηj λkc 2 sj + ··· j | j | λ ck j=1 k j Y for the Schwartz functions η for j 1,...,d chosen at the beginning of Section 3.3.3. This j ∈ { } definition is reasonable because λ ck = 0 for every k and every j 1,...,d . k j 6 ∈ { } k Next, the convergence of (πk)k N to χℓ+ℓα,β in G will be shown. For this, let cα,β be the coefficient function defined by∈ b ck (g) := π (g)η ,η g G = R2d+2+p α,β h k k,α,β k,α,βi ∀ ∈ ∼ and ℓα,β the linear functional

ℓ (g)= ℓ (x,y,t,z,a) := αx + βy g =(x,y,t,z,a) G = R2d+2+p. α,β α,β ∀ ∈ ∼ Since the Hilbert space of the character χ is one-dimensional, one can choose 1 as a ℓ+ℓα,β { } basis and gets the following computation: Using Relation (5), for ρ := ℓ,T and g =(x,y,t,z, a,˙ a¨) R2d+2+p, h i ∈ k cα,β(g)

= πk(g)ηk,α,β(s)ηk,α,β(s)ds RZd d k 1 2πi tρk zλk+ P λkcj (sj 2 xj )yj − − j=1 − = e ηk,α,β(s1 x1,...,sd xd) − − RZd

ηk,α,β(s1,...,sd)d(s1,...,sd)

d k 1 2πi tρk zλk+ P λkcj (sj 2 xj )yj − − j − 2πiαx = e =1 e− RZd d 1 1 β k 4 k 2 j λkc ηj λkc sj xj + j j − λ ck j=1 k j Y d 1 1 β k 4 k 2 j λkc ηj λkc sj + d(s1,...,sd) j j λ ck j=1 k j Y

24 d 1 k 1 2πi(tρk+zλk) 2πiαx 2πi 2 xj yj cj λk k 2 = e− e− e− λkcj j=1 Y

k 1 β 1 β 2πiyj sj cj λk k 2 j k 2 j e ηj λkcj sj xj + k ηj λkcj sj + k dsj − λkc λkc ZR j j !

d 1 k 1 2πi(tρk+zλk) 2πiαx 2πi 2 xj yj cj λk k 2 = e− e− e− λkcj j=1 Y βj k 2πiyj sj c λk 1 1 λ ck j e − k j η λ ck 2 (s x ) η λ ck 2 (s ) ds j k j j − j j k j j j ZR

d 1 k 2πi(tρ +zλ ) 2πiαx 2πi xj yj c λk = e− k k e− e− 2 j jY=1 k k 1 1 2πiyj sj sgn(λ c ) λ c 2 βj k e k j | k j | − η s λ c 2 x η (s )ds j j − k j j j j j ZR d k 2πitρ 2πizλ 2πiαx 2πiyj βj →∞ e− e− e− e− η (s )η (s )ds −→ j j j j j jY=1 ZR ηj 2=1 k k 2πitρ 2πizλ 2πiαx 2πiyβ = e− e− e− e− = χℓ+ℓα,β (g) = χℓ+ℓα,β (g)1, 1 , where this convergence holds uniformly on compacta.

3.5.2 Deﬁnition of νk

Finally, the linear mappings (νk)k N which are needed for the sequence (πk)k N to fulﬁll conditions similar to those required∈ in 3(b) are going to be deﬁned and analyzed.∈

For 0 =η ˜ L2(G/P ,χ ) = L2(Rd), let 6 ∈ k ℓk ∼ P : L2(Rd) Cη,˜ ξ η˜ ξ, η˜ . η˜ → 7→ h i

Pη˜ is the orthogonal projection onto the space Cη˜. Let h C (G/U, χ ). Again, identify G/U with Rd Rd = R2d and as already introduced ∈ ∗ ℓ × ∼ in Section 3.3.4 in order to show the dependence on k, here, the utilization of the limit basis will be expressed by an index if necessary: ∞ h (x,y) := h((x,y) ). ∞ ∞ 2d Now, hˆ can be regarded as a function in C (ℓ+u⊥) = C (R ) and, using this identiﬁcation, ∞ ∞ ∼ ∞ deﬁne the linear operator d(˜x, y˜) νk(h) := hˆ (˜x, y˜)Pη . ∞ k,x,˜ y˜ d k RZ2d λkcj j=1 Q

25 Then, the following proposition holds: Proposition 3.3.

1. For all k N and h (G/U, χ ), the integral deﬁning ν (h) converges in the operator ∈ ∈S ℓ k norm.

2. The operator ν (h) is compact and ν (h) h ∗ . k k k kop ≤ k kC (G/U,χℓ) 3. ν is involutive, i.e. ν (h) = ν (h ) for every h C (G/U, χ ). k k ∗ k ∗ ∈ ∗ ℓ

Proof: 1) Let h (G/U, χ ) = (R2d). Since ∈S ℓ ∼ S P = η 2 k ηk,α,β kop k k,α,βk2 d 2 1 1 β k 2 2πiαj sj k 2 j = λkcj e ηj λkcj sj + k dsj λkc j=1 ZR j Y d 1 1 k 2 k = λ c 2 η (s ) λ c − 2 ds k j | j j | k j j j=1 ZR Y d = η 2 = 1, k jk2 jY=1

one can estimate the operator norm of νk(h) as follows:

d(˜x, y˜) νk(h) op = hˆ (˜x, y˜) Pη ∞ k,x,˜ y˜ d k k k RZ2d λkc op j j=1 Q ˆ d(˜x, y˜) h L1(R2d) hˆ (˜x, y˜) = . ∞ d d ≤ k k Z2d R λkcj λkcj j=1 j=1 Q Q

Therefore, the convergence of the integral νk(h) in the operator norm is shown for h (R2d) = (G/U, χ ). ∈S ∼ S ℓ 2) First, let h (G/U, χ ) = (R2d). ∈S ℓ ∼ S Deﬁne for s =(s ,...,s ) Rd, 1 d ∈ d 1 1 β k 4 k 2 j ηk,β(s) := λkc ηj λkc sj + . j j λ ck j=1 k j Y

Then, 2πiαs ηk,α,β(s)= e ηk,β(s) and thus, one has for ξ (Rd) and s Rd, ∈S ∈

26 d(˜x, y˜) νk(h)ξ(s) = hˆ (˜x, y˜) ξ,ηk,x,˜ y˜ ηk,x,˜ y˜(s) ∞ d k Z2d R λkcj j=1 Q ˆ d(˜x, y˜) = h (˜x, y˜) ξ(r)ηk,x,˜ y˜(r)dr ηk,x,˜ y˜(s) ∞ d 2d d k RZ RZ λkcj j=1 Q ˆ 2πixr˜ 2πixs˜ d(˜x, y˜) = h (˜x, y˜) ξ(r)e− ηk,y˜(r)dr e ηk,y˜(s) ∞ d 2d d k RZ RZ λkcj j=1 Q ˆ 2πix˜(s r) dxd˜ y˜ = h (˜x, y˜)e − ξ(r)ηk,y˜(r)dr ηk,y˜(s) ∞ d k RZd RZd RZd λkcj j=1 dy˜ Q = hˆ2 (s r, y˜)ξ(r)η (r)η (s) dr, (7) k,y˜ k,y˜ d ∞ − k RZd RZd λkcj j=1

Q where, similar as above, hˆ2 denotes the Fourier transform of hˆ in the second variable. ∞ Hence, as the kernel function∞ dy˜ h (s,r) := hˆ2 (s r, y˜)η (r)η (s) K k,y˜ k,y˜ d ∞ − k RZd λkcj j=1 Q of ν (h) is contained in (R2d), the operator ν (h) is compact. k S k Now, it will be shown that νk(h) op hˆ . k k ≤ ∞ Again, for ξ (Rd) one has ∈S ν (h)ξ 2 k k k2 2 dy˜ = hˆ2 (s r, y˜)ξ(r)η (r)η (s) dr ds k,y˜ k,y˜ d ∞ − k RZd RZd RZd λkc j j=1 Q 2 dy˜ = hˆ2 ( , y˜) (ξη )(s)η (s) ds k,y˜ k,y˜ d ∞ · ∗ k RZd RZd λkc j j=1 Cauchy Q − d Schwarz, 1 2 dy˜ λ ck 2 hˆ2 ( , y˜) (ξη )(s) ds k j k,y˜ d ηj ≤∞ 1 ∞ · ∗ k 2 k k ≤ j=1 Zd Zd Y R R λkcj j=1

27 d Plancherel ˆ 2 1 2 h 3 ξηk,y˜ 2dy˜ ≤ ∞ k 2 k k j=1 λkcj Zd Y R

d ˆ 2 1 2 2 = h 3 ξ(s) ηk,y˜(s) dsdy˜ ∞ k 2 | | | | j=1 λkcj Zd Zd Y R R

d 2 2 1 1 y˜ ˆ k 2 j = h k ηj λkcj sj + k dy˜1 dy˜d ∞ λkc ··· ··· λkc ··· j=1 j ZR ZR ZR ZR j Y

νk(h) op = sup νk(h)(ξ) 2 hˆ k k ξ L2(Rd), k k ≤ ∞ ∈ ξ 2=1 k k for h (R2d) = (G/U, χ ). Therefore, with the density of (G/U, χ ) in C (G/U, χ ), ∈ S ∼ S ℓ S ℓ ∗ ℓ one gets the compactness of the operator ν (h) for h C (G/U, χ ), as well as the desired k ∈ ∗ ℓ inequality

ˆ ∗ νk(h) op h = h C (G/U,χℓ). k k ≤ ∞ k k 3) For h C (G/U, χ ) = C (R2d), one has ∈ ∗ ℓ ∼ ∗ d(˜x, y˜) ∗ νk(h)∗ = hˆ (˜x, y˜) Pη ∞ k,x,˜ y˜ d k ! RZ2d λkcj j=1 Q P =P ∗ d(˜x, y˜ ) = hˆ (˜x, y˜) Pη ∞ k,x,˜ y˜ d k RZ2d λkcj j=1 Q d(˜ x, y˜) = h (˜x, y˜) P = ν (h∗). ∗ ηk,x,˜ y˜ d k ∞ k RZ2d λkcj c j=1 Q

This proposition ﬁrstly shows that the image of the operator νk is located in (L2(Rd)) = ( ) as required in Condition 3(b) of Deﬁnition 1.1. Secondly, the B B Hi proposition gives the boundedness and the involutivity of the linear mappings νk for every k N. For the analysis of the sequence (πk)k N, it remains to show a convergence, from ∈ ∈ V which the convergence for the original sequence (πk )k N demanded in Condition 3(b) can be deduced. ∈

28 3.5.3 Theorem – Second Case Theorem 3.4. Deﬁne as in Section 2.1.3

p : L1(G) L1(G/U, χ ), p (f)(˜g) := f(˜gu)χ (u)du g˜ G f L1(G) G/U → ℓ G/U ℓ ∀ ∈ ∀ ∈ UZ

and canonically extend pG/U to a mapping going from C∗(G) to C∗(G/U, χℓ). Furthermore, let a C (G). Then, ∈ ∗ lim πk(a) νk pG/U (a) = 0. k − op →∞ Proof: First, it is well-known that pG/U is a surjective homomorphism from C∗(G) to C∗(G/U, χℓ) and from L1(G) to L1(G/U, χ ) as well as from (G) to (G/U, χ ). ℓ S S ℓ Furthermore, for u =(t,z, a,˙ a¨) U = span T,Z,A1, ..., Ap˜,Ap˜+1, ..., Ap , ∞ ∈ { } 2πi ℓ,(t,z,a,˙ a¨)∞ 2πi(tρ+zλ) χℓ(u)= e− h i = e− and therefore, identifying U again with R R Rp˜ Rp p˜ and L1(G/U, χ ) with L1(R2d), for × × × − ℓ f L1(G) = L1(R2d+2+p) andg ˜ = (˜x, y,˜ 0, 0, 0, 0) R2d, one has ∈ ∼ ∈ ˜ ˜ 2πi(tρ˜ +˜zλ) ˜ ˜ pG/U (f) (˜g) = f (˜x, y,˜ t,˜ z,˜ a,˙ a¨)e− d(0, 0, t,˜ z,˜ a,˙ a¨) ∞ ∞ R R RZp˜ Rp−p˜ × × × = fˆ3,4,5,6(˜x, y,˜ ρ, λ, 0, 0), (8) ∞ where f (˜x, y,˜ ρ, λ, 0, 0) = f((˜x, y,˜ ρ, λ, 0, 0) ). ∞ ∞ R2d+2+p Now, let f (G) ∼= ( ) such that its Fourier transform in [g, g] has a com- ∈ S R2dS+2+p pact support in G ∼= . If one writes the elements g of G as g = (x,y,t,z, a,˙ a¨) like above, where x span X ... span X , y span Y ... span Y , t span T , ∈ { 1}× × { d} ∈ { 1}× × { d} ∈ { } z span Z , a˙ span A ... span A , a¨ span A ... span A ∈ { } ∈ { 1}× × { p˜} ∈ { p˜+1}× × { p} or x span Xk ... span Xk , y span Y k ... span Y k , t span T , ∈ { 1 }× × { d } ∈ { 1 }× × { d } ∈ { k} z span Z , a˙ span Ak ... span Ak , a¨ span Ak ... span Ak , ∈ { k} ∈ { 1}× × { p˜} ∈ { p˜+1}× × { p} respectively, this means that the partial Fourier transform fˆ4,5 has a compact support in G, since [g, g] = span Z ,Ak, ..., Ak = span Z,A , ..., A . { k 1 p˜} { 1 p˜} Moreover, let ξ L2(Rd) and s Rd and deﬁne ∈ ∈ d 1 1 k 4 k 2 ηk,0(s) := λkcj ηj λkcj (sj) . j=1 Y

(Compare the deﬁnition of ηk,β in the last proof.)

29 To prove the theorem, the expression π (f) ν p (f) ξ is going to be regarded, using k − k G/U Results (6) from Section 3.3.4 and (7) from the proof of the proposition above. The obtained integrals are going to be divided into five parts and the norm of each part is going to be estimated separately. For this, G will again be identified with R2d+2+p and therefore, f will be replaced by the functions fk and f , respectively, which are defined as above. ∞

π (f) ν p (f) ξ(s) k − k G/U k (6),(7) λc = fˆ2,3,4,5,6 s r, k (s + r), ρ ,λ , 0, 0 ξ(r)dr k − − 2 k k RZd \ 2 dy˜ p (f) (s r, y˜)ξ(r)η (r)η (s) dr G/U k,y˜ k,y˜ d − ∞ − k RZd RZd λkcj j=1 k Q ηk,0 2=1 λ c k k ˆ2,3,4,5,6 k = fk s r, (s + r), ρk,λk, 0, 0 ξ(r)ηk,0(˜y)ηk,0(˜y)dydr˜ (8) − − 2 RZd RZd dy˜ fˆ2,3,4,5,6(s r, y,˜ ρ, λ, 0, 0)ξ(r)η (r)η (s) dr k,y˜ k,y˜ d − ∞ − k RZd RZd λkcj j=1 k Q λ c = fˆ2,3,4,5,6 s r, k (s + r), ρ ,λ , 0, 0 ξ(r)η (˜y)η (˜y)dydr˜ k − − 2 k k k,0 k,0 RZd RZd ˆ2,3,4,5,6 k f s r, λkc (˜y s), ρ, λ, 0, 0 ξ(r)ηk,0(˜y + r s)ηk,0(˜y)dydr.˜ − ∞ − − − Zd Zd R R

As described above, the integrals just obtained are now going to be divided into ﬁve parts. To do so, the functions qk, uk, vk, nk and wk will be deﬁned. λ ck q (s, y˜) := ξ(r)η (˜y + r s) fˆ2,3,4,5,6 s r, k (s + r), ρ ,λ , 0, 0 k k,0 − k − − 2 k k RZd λ ck fˆ2,3,4,5,6 s r, k (s + r), ρ, λ , 0, 0 dr, − k − − 2 k λ ck u (s, y˜) := ξ(r)η (˜y + r s) fˆ2,3,4,5,6 s r, k (s + r), ρ, λ , 0, 0 k k,0 − k − − 2 k RZd λ ck fˆ2,3,4,5,6 s r, k (s + r), ρ, λ, 0, 0 dr, − k − − 2 λ ck v (s, y˜) := ξ(r)η (˜y + r s) fˆ2,3,4,5,6 s r, k (s + r), ρ, λ, 0, 0 k k,0 − k − − 2 RZd fˆ2,3,4,5,6 s r, λ ck(˜y s), ρ, λ, 0, 0 dr, − k − k −

30 n (s, y˜) := ξ(r)η (˜y + r s) fˆ2,3,4,5,6 s r, λ ck(˜y s), ρ, λ, 0, 0 k k,0 − k − k − Zd R 2,3,4,5,6 k fˆ s r, λkc (˜y s), ρ, λ, 0, 0 dr − ∞ − − and

w (s) := ξ(r)η (˜y) η (˜y) η (˜y + r s) k k,0 k,0 − k,0 − Zd Zd R R λ ck fˆ2,3,4,5,6 s r, k (s + r), ρ ,λ , 0, 0 drdy.˜ k − − 2 k k Then,

π (f) ν p (f) ξ(s) = q (s, y˜)η (˜y)dy˜ + u (s, y˜)η (˜y)dy˜ k − k G/U k k,0 k k,0 Zd Zd R R

+ vk(s, y˜)ηk,0(˜y)dy˜ + nk(s, y˜)ηk,0(˜y)dy˜ + wk(s). RZd RZd

In order to show that k π (f) ν p (f) →∞ 0, k − k G/U op −→ it suﬃces to prove that there are κ , γ , δ , ω and ǫ which are tending to 0 for k , k k k k k → ∞ such that

q κ ξ , u γ ξ , v δ ξ , n ω ξ and w ǫ ξ . k kk2 ≤ kk k2 k kk2 ≤ kk k2 k kk2 ≤ kk k2 k kk2 ≤ kk k2 k kk2 ≤ kk k2

First, regard the last factor of the function qk:

λ ck λ ck fˆ2,3,4,5,6 s r, k (s + r), ρ ,λ , 0, 0 fˆ2,3,4,5,6 s r, k (s + r), ρ, λ , 0, 0 k − − 2 k k − k − − 2 k 1 λ ck = (ρ ρ) ∂ fˆ2,3,4,5,6 s r, k (s + r), ρ + t(ρ ρ),λ , 0, 0 dt. k − 3 k − − 2 k − k Z0

Since f is a Schwartz function, one can ﬁnd a constant C1 > 0 (depending on f) such that

λ ck λ ck fˆ2,3,4,5,6 s r, k (s + r), ρ ,λ , 0, 0 fˆ2,3,4,5,6 s r, k (s + r), ρ, λ , 0, 0 k − − 2 k k − k − − 2 k

C ρ ρ 1 . ≤ | k − | (1 + s r )2d k − k

31 Hence, one gets the following estimate for qk:

q 2 = q (s, y˜) 2d(s, y˜) k kk2 | k | RZ2d C 2 ξ(r)η (˜y + r s) ρ ρ 1 dr d(s, y˜) ≤ k,0 − | k − | (1 + s r )2d Z2d Zd R R k − k

Cauchy ξ(r)η (˜y + r s) 2 − 2 2 k,0 − C1 ρk ρ d dr Schwarz≤ | − | (1 + s r ) ! Z2d Zd R R k − k

1 2 d dr d(s, y˜) (1 + s r ) ! RZd k − k

2 2 ξ(r) 2 = C′ ρ ρ | | η (˜y + r s) d(r,s, y˜) 1| k − | (1 + s r )2d k,0 − Z3d R k − k

ηk,0 2=1 k k 2 2 C′′ ρ ρ ξ ≤ 1 | k − | k k2 for matching constants C1′ > 0 and C1′′ > 0 depending on f. Thus, for κk := C1′′ ρk ρ , k k | − | κ →∞ 0 since ρ →∞ ρ, and k −→ k −→ p q κ ξ . k kk2 ≤ kk k2 k As λ →∞ λ, the estimation for the function u can be done analogously. k −→ k

Now, regard vk. Like for qk and uk, one has

λ ck fˆ2,3,4,5,6 s r, k (s + r), ρ, λ, 0, 0 fˆ2,3,4,5,6 s r, λ ck(˜y s), ρ, λ, 0, 0 k − − 2 − k − k − 1 = λ ck (r s) (r s +y ˜) k 2 − − − 1 1 ∂ fˆ2,3,4,5,6 s r, λ ck(˜y s)+ tλ ck (r s) (r s +y ˜) , ρ, λ, 0, 0 dt, · 2 k − k − k 2 − − − Z0 where is the scalar product, and hence there exists again a constant C depending on f such · 3 that λ ck fˆ2,3,4,5,6 s r, k (s + r), ρ, λ, 0, 0 fˆ2,3,4,5,6 s r, λ ck(˜y s), ρ, λ, 0, 0 k − − 2 − k − k −

C λ ck(r s) + ck(r s +y ˜) 3 . ≤ | k| − − (1 + s r )2d+1 k − k

32 Therefore, deﬁningη ˜ (t) := t η (t), one gets a similar estimation for v : j k k j k v 2 k kk2 = v (s, y˜) 2d(s, y˜) | k | RZ2d

ξ(r)η (˜y + r s) λ ck(r s) + ck(r s +y ˜) ≤ k,0 − | k| − − Z2d Zd R R

2 2 ξ(r) k 2 2 2C3′ λk | | 2d+2 d(r, s) c y˜ ηk,0(˜y) dy˜ ≤ | | (1 + s r ) ! Z2d Zd R k − k R 2 k 2 ξ(r) 2 + 2C′ λ c | | η (˜y + r s) d(r,s, y˜) 3 k (1 + s r )2d k,0 − Z3d R k − k

2 k ξ(r) 2C3′ λkc | | 2d+2 d(r, s) ≤ (1 + s r ) ! Z2d R k − k d 1 1 2 1 2 k 2 k 2 k 2 λkcj λkcj y˜j ηj λkcj (˜yj) dy˜j j=1 ZR Y 2 k 2 ξ(r) 2 + 2C′ λ c | | η (˜y + r s) d(r,s, y˜) 3 k (1 + s r )2d k,0 − Z3d R k − k d ηk,0 2=1 k k k 2 2 k 2 2 2C′ λ c ξ η˜ (˜y ) dy˜ + 2C′ λ c ξ ≤ 3 k k k2 | j j | j 3 k k k2 j=1 ZR Y d 2 k k 2 = C′′ η˜ + λ c λ c ξ 3 k jk2 k k k k2 j=1 Y

with constants C3′ > 0 and C3′′ > 0, again depending on f.

33 1 d k 2 k Now, since λ ck →∞ 0, δ := C η˜ 2 + λ ck λ ck fulﬁlls δ →∞ 0 and k −→ k 3′′ k jk2 k k k −→ j=1 Q v δ ξ . k kk2 ≤ kk k2

For the estimation of nk, the fact that the Fourier transform in [g, g], i.e. in the 4th and 5th variable, fˆ4,5 =: f˜ has a compact support will be needed. Therefore, let the support of f be located in the compact set K K K K K K Rd Rd R R Rp˜ Rp p˜ 1 × 2 × 3 × 4 × 5 × 6 ⊂ × × × × × − and let K := K K K Rd R Rp p˜. 2 × 3 × 6 ⊂ × × − Furthermore, since a Fourier transform is independent of the choice of the basis, the value of f˜ on [g, g] = span Z ,Ak, ..., Ak = span Z,A , ..., A expressed in the k-basis and its value { k 1 p˜} { 1 p˜} expressed in the limit basis are the same:

f( , , , (z, a˙)k, )= f( , , , (z, a˙) , ). · · · · · · · ∞ · So, in the course of this proof, the limit basis will be chosen for the representation of the 4th and 5th position of an element g. Then, ˆ2,3,4,5,6 k ˆ2,3,4,5,6 k fk s r, λkc (˜y s), ρ, λ, 0, 0 f s r, λkc (˜y s), ρ, λ, 0, 0 − − − ∞ − − k 2πi(λkc (˜y s)y+ρt) = f˜k(s r,y,t,λ, 0, a¨) f˜ (s r,y,t,λ, 0, a¨) e− − d(y,t, a¨) − − ∞ − Rd RZ Rp−p˜ × ×

= f˜((s r,y,t)k, (λ, 0) , (¨a)k) f˜((s r,y,t,λ, 0, a¨) ) − ∞ − − ∞ Rd RZ Rp−p˜ × × 2πi(λ ck(˜y s)y+ρt) e− k − d(y,t, a¨)

k 2πi(λkc (˜y s)y+ρt) = f˜((s r,y,t)k, (λ, 0) , (¨a)k) f˜((s r,y,t,λ, 0, a¨) ) e− − d(y,t, a¨). − ∞ − − ∞ KZ Furthermore,

f˜((s r,y,t)k, (λ, 0) , (¨a)k) f˜((s r,y,t,λ, 0, a¨) ) − ∞ − − ∞ d d p = f˜ (s r )Xk + y Y k + tT + λZ + a Ak i − i i i i k i i Xi=1 Xi=1 i=˜Xp+1 d d p f˜ (s r )X + y Y + tT + λZ + a A − i − i i i i i i Xi=1 Xi=1 i=˜Xp+1 d d p = (s r )(Xk X )+ y (Y k Y )+ t(T T )+ a (Ak A ) i − i i − i i i − i k − i i − i Xi=1 Xi=1 i=˜Xp+1 1 d d p ∂f˜ (s r )X + y Y + tT + λZ + a A · i − i i i i i i Z0 Xi=1 Xi=1 i=˜Xp+1 d d p + t˜ (s r )(Xk X )+ y (Y k Y )+ t(T T )+ a (Ak A ) dt.˜ i − i i − i i i − i k − i i − i Xi=1 Xi=1 i=˜Xp+1 34 k k k k k k k Since (X1 ,...,Xd ,Y1 ,...,Yd ,Tk,Zk,A1, ..., Ap) →∞ (X1,...,Xd,Y1,...,Yd,T,Z,A1, ..., Ap), k −→ there exist ωi →∞ 0 for i 1,..., 4 and a constant C > 0 depending on f such that k −→ ∈ { } 4

f˜((s r,y,t)k, (λ, 0) , (¨a)k) f˜((s r,y,t,λ, 0, a¨) ) − ∞ − − ∞

C s r ω1 + y ω2 + t ω3 + a¨ ω4 4 . ≤ k − k k k k k | | k k k k (1 + s r )2d+1 k − k Now, with the help of the two computations above, n 2 can be estimated: k kk2 n 2 k kk2 = n (s, y˜) 2d(s, y˜) | k | RZ2d

= ξ(r)η (˜y + r s) fˆ2,3,4,5,6 s r, λ ck(˜y s), ρ, λ, 0, 0 k,0 − k − k − Z2d Zd R R 2 2,3,4,5,6 k fˆ s r, λkc (˜y s), ρ, λ, 0, 0 dr d(s, y˜) − ∞ − −

ξ(r)η (˜y + r s) s r ω1 + y ω 2 + t ω3 + a¨ ω4 ≤ k,0 − k − k k k k k | | k k k k Z2d Zd Z R R K 2 C4 2πi(λ ck(˜y s)y+ρt) e− k − d(y,t, a¨)dr d(s, y˜) (1 + s r )2d+1 k − k 1 2 = C2 ξ(r) η (˜y + r s) s r ω5 + ω6 dr d(s, y˜) 4 (1 + s r )2d+1 | | k,0 − k − k k k Z2d Zd R R k − k

Cauchy 2 − 2 1 2 2 5 6 C4 2d+2 ξ(r) ηk,0(˜y + r s) s r ωk + ωk dr Schwarz≤ (1 + s r ) | | − k − k Z2d Zd R R k − k 1 dr d(s, y˜) (1 + s r )2d RZd k − k

7 1 2 2 C′ ω ξ(r) η (˜y + r s) drd(s, y˜) ≤ 4 k (1 + s r )2d | | k,0 − Z2d Zd R R k − k

ηk,0 2=1 7 1 2 k =k C′ ω ξ(r) drds 4 k (1 + s r )2d | | RZd RZd k − k 7 2 = C′′ω ξ 4 kk k2 i k with constants C4′ > 0 and C4′′ > 0 depending on f and ωk →∞ 0 for i 5,..., 7 . Thus, k −→ ∈ { } ω := C ω7 fulﬁlls ω →∞ 0 and k 4′′ k k −→ q n ω ξ . k kk2 ≤ kk k2

35 Last, it still remains to examine wk:

η (˜y) η (˜y + r s) k,0 − k,0 − d 1 = (r s ) ∂ η (˜y + t(r s))dt j − j j k,0 − Xj=1 Z0 d 1 d 1 1 = (r s ) λ ck 4 η λ ck 2 (˜y + t(r s )) j − j k i i k i i i − i j=1 Z i=1 X 0 Yi=j 6 3 1 k 4 k 2 λkcj ∂ηj λkcj (˜yj + t(rj sj)) dt. − !

Since f and the functions (ηj)j 1,...,d are Schwartz functions, one can ﬁnd a constant C5 ∈{ } depending on (ηj)j 1,...,d such that ∈{ } λ ck η (˜y) η (˜y + r s) fˆ2,3,4,5,6 s r, k (s + r), ρ ,λ , 0, 0 k,0 − k,0 − k − − 2 k k

d d 1 1 C5 r s λ ck 4 λ ck 2 . ≤ k − k k i k j (1 + r s )2d+1 j=1 i=1 X Y k − k

Now, one has the following estimation for w , which is again similar to the above ones: k kk2 w 2 = w (s) 2ds k kk2 | k | RZd d d 1 1 ξ(r) η (˜y) r s λ ck 4 λ ck 2 ≤ | | k,0 k − k k i k j Zd Zd Zd j=1 i=1 R R R X Y

C 2 5 drdy˜ ds (1 + r s )2d+1 k − k d d Cauchy 1 ξ(r) 2 − k 2 k 2 2 C5′ λkci λkcj | | 2d+2 ηk,0(˜y) r s d(r, y,s˜ ) Schwarz≤ (1 + r s ) k − k j=1 i=1 Z3d X Y R k − k d d ηk,0 2=1 1 k k k k 2 C′′ λ c 2 λ c ξ , ≤ 5 k i k j k k2 j=1 i=1 X Y

where the constants C5′ > 0 and C5′′ > 0 depend on the functions (ηj)j 1,...,d . Therefore, for 1 ∈{ } d d 1 2 k ǫ := C λ ck 2 λ ck , the desired properties ǫ →∞ 0 and k 5′′ k i k j k −→ j=1 i=1 P Q w ǫ ξ k kk2 ≤ kk k2 are fulﬁlled.

36 Thus, for those f (R2d+2+p) = (G) whose Fourier transform in [g, g] has a com- ∈ S ∼ S pact support,

k →∞ πk(f) νk pG/U (f) op = sup πk(f) νk pG/U (f) (ξ) 2 0. − ξ L2(Rd) − −→ ∈ ξ 2=1 k k Because of the density in L1(G) and thus in C (G) of the set of Schwartz functions f (G) ∗ ∈S whose partial Fourier transform has a compact support, the claim is true for general elements a C (G) by Corollary 7.6 in the appendix. ∈ ∗

V 3.5.4 Transition to πk k N ∈ Finally, from the results shown above for the sequence (πk)k N, the assertion for the original V ∈ sequence πk k N shall be deduced. ∈ Since for every k N the two representations π and πV are equivalent, there exist unitary ∈ k k intertwining operators

2 Rd 2 Rd V Fk : πV = L ( ) πk = L ( ) such that Fk πk (a)= πk(a) Fk a C∗(G). H k ∼ →H ∼ ◦ ◦ ∀ ∈

V V N Furthermore, since the limit set L πk k N of the sequence πk )k is contained in Si 1, ∈ ∈ − as discussed in Section 3.3.1, by identifying G with the set of coadjoint orbits g∗/G, one can restrict an operator field ϕ CB(Si 1) to L(( k)k N) = ℓ + u⊥ and obtains an element in ∈ − O ∈ CB(ℓ + u⊥). Thus, as (a) L(( ) N) a bC∗(G) = C (L(( k)k N)) = C (ℓ + u⊥), one F | Ok k∈ | ∈ ∞ O ∈ ∞ can define the -isomorphism ∗ R2d R2d τ : C ( ) = C (ℓ + u⊥) C∗(G/U, χℓ) = C∗( ), (a) L(( ) N) pG/U (a). ∞ ∼ ∞ → ∼ F | Ok k∈ 7→ Now, for k N, defineν ˜ as ∈ k

ν˜k(ϕ) := Fk∗ (νk τ) ϕ L(( ) ) Fk ϕ CB(Si 1). ◦ ◦ | Ok˜ k˜∈N ◦ ∀ ∈ − Since the image of ν is in (L2(Rd)) and F is an intertwining operator and thus bounded, k B k the image ofν ˜ is contained in (L2(Rd)) as well. k B Moreover, the operatorν ˜k is bounded: From the boundedness of νk (see Proposition 3.3) and using that τ is an isomorphism, one gets for every ϕ CB(Si 1), ∈ −

ν˜k(ϕ) op = Fk∗ (νk τ) ϕ L(( ) ) Fk k k ◦ ◦ | Ok˜ k˜∈N ◦ op (νk τ) ϕ L(( ) ) ≤ ◦ | Ok˜ k˜∈N op τ ϕ L(( ) ) ≤ | Ok˜ k˜∈N C∗(R2 d)

ϕ L(( ) ) ϕ Si−1 . ≤ | Ok˜ k˜∈N ≤ k k ∞

37 Next, the involutivity ofν ˜k will be shown. With the involutivity of νk (see Proposition 3.3) and τ, for every ϕ CB(Si 1), ∈ − ∗ ν˜k(ϕ)∗ = Fk∗ (νk τ) ϕ L(( ) ) Fk ◦ ◦ | Ok˜ k˜∈N ◦ = Fk∗ (νk τ)∗ ϕ L(( ) ) Fk ◦ ◦ | Ok˜ k˜∈N ◦

= Fk∗ νk τ ∗ ϕL(( ) ) Fk ◦ | Ok˜ k˜∈N ◦ = Fk∗ (νk τ) ϕ∗L(( ) ) Fk =ν ˜k(ϕ∗). ◦ ◦ | Ok˜ k˜∈N ◦ Finally, the demanded convergence of Condition 3(b) of Deﬁnition 1.1 can also be shown: V With the equivalence of the representations πk and πk stated above, one gets

V πk (a) ν˜k (a) Si = Fk∗ πk(a) Fk Fk∗ (νk τ) (a) L(( ) ) Fk − F | −1 op ◦ ◦ − ◦ ◦ F | Ok˜ k˜∈N ◦ op

= F ∗ πk(a) Fk F ∗ νk p (a) Fk k ◦ ◦ − k ◦ G/U ◦ op = F ∗ πk(a) νk pG/U (a) Fk k ◦ − ◦ op k ν p (a) π(a) →∞ 0. ≤ k G/U − k op −→ V Therefore, the representations πk k N fulﬁll Property 3(b) and thus, the conditions of Deﬁ- nition 1.1 are proved. ∈ 3.6 Condition 3(b) – Third case Now, the third case mentioned in Proposition 3.1 is going to be considered. The approach will be similar to the one in the second case.

In the third and last case of Condition 3(b) of Deﬁnition 1.1, λ = 0 and there exists 1 m

k ℓ , [Xk,Y k] = ckλ →∞ c λ = 0 j m + 1,...,d . k j j j k −→ j ⇐⇒ ∈ { }

In this case, p := span Xm+1,...,Xd,Y1,...,Yd,T,Z,A1, ..., Ap is a polarization of ℓ. { } k Moreover, for p˜ := span Xk ,...,Xk,Y k,...,Y k,T ,Z ,Ak, ..., Ak , one has p˜ →∞ p. k { m+1 d 1 d k k 1 p} k −→ Let P := exp(p) and P˜k := exp(p˜k).

3.6.1 Convergence of (πk)k N in G ∈ In this section, the convergence in G of the sequence (πk)k N is going to be examined in b ∈ order to motivate the choice of the linear mappings (νk)k N. b ∈ Let (x) =(˙x, x¨) with (˙x) := (x1,...,xm) and (¨x) := (xm+1,...,xd) ∞ ∞ ∞ ∞ ∞ ∞ and analogously

(y) =(˙y, y¨) with (˙y) := (y1,...,ym) and (¨y) := (ym+1,...,yd) . ∞ ∞ ∞ ∞ ∞ ∞

38 Moreover, as in Section 3.3.3 above, let

(a) =(˙a, a¨) with (˙a) =(a1,...,ap˜) and (¨a) =(ap˜+1,...,ap) ∞ ∞ ∞ ∞ ∞ ∞ and let (g) =(˙x, x,¨ y,˙ y,t,z,¨ a,˙ a¨) =(x,y,t,z,a) =(x,h) . ∞ ∞ ∞ ∞

d m d m Now, letα ¨ := (αm+1,...,αd) R − and β¨ := (βm+1,...,βd) R − , considerα ¨ and β¨ as d ∈ ∈ elements of R identifying them with (0,..., 0,αm+1,...,αd) and (0,..., 0,βm+1,...,βd), respectively, and let π˜ :=π ˜ := indGχ . α,¨ β¨ P ℓ+ℓα,¨ β¨ 2 Then, for a function ξ˙ in the corresponding Hilbert space π˜ = L G/P, χℓ+ℓ ofπ ˜ and for H α,¨ β¨ s1,...,sm R, (˙s) =(s1,...,sm) span X1 ... span Xm andc ˙ =(c1,...,cm), letting ∈ ∞ ∞ ∈ { }× × { } ρ := ℓ,T , one has similarly as in (5): h i π˜ (g) ξ˙ (˙s) ∞ ∞ ˙ 1 = ξ (g)− (˙s) ∞ · ∞ 1 = ξ˙ (˙s) (x) (h) + [ (x) (h) , (˙s) ] ∞ − ∞ − ∞ 2 − ∞ − ∞ ∞ = ξ˙ (˙s) (˙x) (h) (¨x) +[(˙s) , (h) + (¨x) ] ∞ − ∞ · − ∞ − ∞ ∞ ∞ ∞ 1 [(˙x) , (h) +(˙s) + (¨x) ] − 2 ∞ ∞ ∞ ∞ 1 2πi ℓ+ℓ , (h)∞ (¨x)∞+[(˙s)∞,(h)∞+(¨x)∞] [(˙x)∞,(h)∞+(˙s)∞+(¨x)∞] = e h α,¨ β¨ − − − 2 iξ˙ (˙s x˙) − ∞ 2πi ℓ+ℓ , (˙y)∞ (ÿ)∞ (t)∞ (z)∞ (a)∞ (¨x)∞+[(˙s)∞,(˙y)∞+(ÿ)∞+(t)∞+(z)∞+(a)∞+(¨x)∞] = e h α,¨ β¨ − − − − − − i 1 2πi ℓ+ℓ , [(˙x)∞,(˙y)∞+(ÿ)∞+(t)∞+(z)∞+(a)∞+(˙s)∞+(¨x)∞] e h α,¨ β¨ − 2 iξ˙((˙s x˙) ) (9) − ∞ 1 2πi ℓ, (t)∞ (z)∞+[(˙s)∞,(˙y)∞] [(˙x)∞,(˙y)∞] 2πi ℓ ¨, (¨x)∞ (ÿ)∞ = e h − − − 2 ie h α,¨ β − − iξ˙ (˙s x˙) ∞ m − 1 2πi tρ zλ+ P λcj (sj 2 xj )yj − − j − 2πi(¨α(¨x)∞+β¨(ÿ)∞) = e =1 e− ξ˙ (˙s x˙) − ∞ 1 2πi( tρ zλ+λc˙(((˙s)∞ (˙x)∞)(˙y)∞)) 2πi(¨α(¨x)∞+β¨(ÿ)∞) = e − − − 2 e− ξ˙ (˙s x˙) , − ∞ since ℓ(Y )= ℓ(X ) = 0 for all j 1,...,d . j j ∈ { } 2d+2+p From now on again, most of the time, G will be identified with R . C∗(G) does not 2d+2+p coincide with C∗(R ) though, since this last algebra is commutative while C∗(G) is non-commutative. To avoid this inconvenience, most of the calculations will be accomplished in L1(G) or (G) which in turn are isomorphic to L1(R2d+2+p) and (R2d+2+p), respectively, S S as Fréchet spaces. Because of the density of L1(G) and (G) in C (G), the results will then S ∗ follow. So, in the sequel, the isomorphisms will be considered as Fréchet space-isomorphisms and not as algebra-isomorphisms.

39 Deﬁne fors ¨ =(s ,...,s ) span X ... span X = Rd m, m+1 d ∈ { m+1}× × { d} ∼ − d 1 1 β 2πiα¨s¨ k 4 k 2 j η¨ ¨(¨s) := e λkc ηj λkc sj + k,α,¨ β j j λ ck j=m+1 k j Y 2 2 m and furthermore, for a function ξ˙ π˜ = L G/P, χℓ+ℓ = L (R ) and an element ∈ H α,¨ β¨ ∼ s =(˙s, s¨) span X ... span X span X ... span X = Rm Rd m, ∈ { 1}× × { m} × { m+1}× × { d} ∼ × − ˙ ξk(s) := ξ(˙s)¨ηk,α,¨ β¨(¨s). As above in the second case, using Relation (5), one therefore gets for an element g =(x,y,t,z,a)=(˙x, x,¨ y,˙ y,t,z,¨ a,˙ a¨) Rm Rd m Rm Rd m R R Rp˜ Rp p˜, ∈ × − × × − × × × × − π (g)ξ ,ξ h k k ki

= πk(g)ξk(s)ξk(s)ds RZd d 2πi tρ zλ + P λ ck(s 1 x )y (5) k k k j j 2 j j − − j=1 − = e ξ˙(s1 x1,...,sm xm) − − RZd η¨ (s x ,...,s x )ξ˙(s ,...,s )¨η (s ,...,s )d(s ,...,s ) k,α,¨ β¨ m+1 − m+1 d − d 1 m k,α,¨ β¨ m+1 d 1 d m k 1 2πi P λkcj (sj 2 xj )yj as in j=1 − = ξ˙(s1 x1,...,sm xm)ξ˙(s1,...,sm) e d(s1,...,sm) 2nd case − − RZm d 1 k k k 1 2πi(tρ +zλ ) 2πiα¨x¨ 2πi xj yj c λk 2πiyj (sj sgn(λkc ) λkc 2 βj ) e− k k e− e− 2 j e j | j | −

j=Ym+1 ZR 1 η s λ ck 2 x η (s )ds j j − k j j j j j ! m 1 2 πi P λc j (sj xj )yj k − 2 →∞ ξ˙(s x ,...,s x )ξ˙(s ,...,s ) e j=1 d(s ,...,s ) −→ 1 − 1 m − m 1 m 1 m RZm d 2πitρ 2πizλ 2πiα¨x¨ 2πiyj βj e− e− e− e− ηj(sj)ηj(sj)dsj j=Ym+1 ZR ηj 2=1 2πitρ 2πizλ 2πiα¨x¨ 2πiβÿ¨ 2πiλc˙(˙s 1 x˙)˙y k =k e− e− e− e− e − 2 ξ˙(˙s x˙)ξ˙(˙s)ds˙ − RZm = π˜(g)ξ,˙ ξ˙ k and the coefficient functions c ¨ defined by α,¨ β k N ∈ ck (g) := π (g)ξ ,ξ g G = R2d+2+p α,¨ β¨ h k k ki ∀ ∈ ∼ converge uniformly on compacta to cα,¨ β¨ which in turn is defined by c (g) := π˜ (g)ξ,˙ ξ˙ g G = R2d+2+p. α,¨ β¨ α,¨ β¨ ∀ ∈ ∼

40 3.6.2 Deﬁnition of νk

Again, the linear mappings (νk)k N are going to be deﬁned and analyzed. ∈ For 0 = η L2(P˜ /P ,χ ) = L2(Rd m), let 6 ′ ∈ k k ℓk ∼ − 2 d m P ′ : L (R − ) Cη′, ξ η′ ξ,η′ . η → 7→ h i

Then, Pη′ is the orthogonal projection onto the space Cη′. Define now for k N and h C (G/U, χ ) the linear operator ∈ ∈ ∗ ℓ d(˜x, y˜) ν (h) := π (h) P , k ℓ+(˜x,y˜) η¨k,x,˜ y˜ d ⊗ k R2(dZ−m) λkcj j=m+1 Q G where πℓ+(˜x,y˜) is defined as indP χℓ+(˜x,y˜) for an element ℓ + (˜x, y˜) which is located in ℓ + span X ... span X span Y ... span Y ∗ = ℓ + R2(d m). { m+1}× × { d} × { m+1}× × { d} ∼ − 2 d ∞ 2 m 2 d m Thus, for L (R ) ξ = ξ˙i ξï with ξ˙i L (R ) and ξï L (R − ) for all i N, one has ∋ i=1 ⊗ ∈ ∈ ∈ P ∞ d(˜x, y˜) ν (h)(ξ) := π (h)(ξ˙ ) P (ξ¨ ) k ℓ+(˜x,y˜) i η¨k,x,˜ y˜ i d ⊗ k i=1 2(dZ−m) X R λkcj j=m+1

Q and for s =(s ,...,s ) Rd, 1 d ∈ ∞ d(˜x, y˜) ν (h)ξ(s) := π (h)(ξ˙ )(s ,...,s ) P (ξ¨ )(s ,...,s ) . k ℓ+(˜x,y˜) i 1 m η¨k,x,˜ y˜ i m+1 d d · k i=1 2(dZ−m) X R λkcj j=m+1 Q Since the operators

π (h): L2(G/P, χ ) = L2(Rm) L2(G/P, χ ) = L2(Rm) and ℓ+(˜x,y˜) ℓ ∼ → ℓ ∼ 2 2 d m 2 2 d m P : L (P˜ /P ,χ ) = L (R − ) L (P˜ /P ,χ ) = L (R − ) η¨k,x,˜ y˜ k k ℓk ∼ → k k ℓk ∼ are bounded and the tensor product of two bounded operators on Hilbert spaces is bounded by the product of the two operator norms, the operator

π (h) P : L2(Rd) L2(Rd) ℓ+(˜x,y˜) ⊗ η¨k,x,˜ y˜ → is bounded on L2(Rd) by π (h) P . k ℓ+(˜x,y˜) kopk η¨k,x,˜ y˜ kop

41 Proposition 3.5.

1. For all k N and h (G/U, χ ), the integral deﬁning ν (h) converges in the operator ∈ ∈S ℓ k norm.

2. The operator ν (h) is compact and ν (h) h ∗ . k k k kop ≤ k kC (G/U,χℓ) 3. νk is involutive.

Proof: 2 m 2 m Let = (L (R )) be the C∗-algebra of the compact operators on the Hilbert space L (R ) K K2(d m) 2(d m) and C (R − , ) the C∗-algebra of all continuous mappings from R − into vanishing ∞ K K at infinity. 2(d m) Define for ϕ C (R − , ) and k N the linear operator ∈ ∞ K ∈ d(˜x, y˜) µ (ϕ) := ϕ(˜x, y˜) P k η¨k,x,˜ y˜ d ⊗ k R2(dZ−m) λkcj j=m+1 2 d 2(d m) Q on L (R ). Then, as (h) C (R − , ) for every h C ∗(G/U, χℓ), F ∈ ∞ K ∈ ν (h)= µ ( (h)). k k F 1) Since (h) (R2(d m), ) for h (G/U, χ ) and since F ∈S − K ∈S ℓ d(˜x, y˜) µ (ϕ) ϕ(˜x, y˜) k op op d k k ≤ k k k R2(dZ−m) λkcj j=m+1 Q for every ϕ (R2(d m), ) and k N, the first assertion follows immediately. ∈S − K ∈ 2) As p is surjective from (G) to (G/U, χ ), for every h (G/U, χ ) = (R2d), there G/U S S ℓ ∈S ℓ ∼ S exists a function f (G) = (R2d+2+p) such that h = p (f) and, as shown in the second ∈S ∼ S G/U case, forg ˜ =(˙x, x,¨ y,˙ y,¨ 0, 0, 0, 0) G/U = R2d, one has ∈ ∼ h (˜g)= fˆ5,6,7,8(˙x, x,¨ y,˙ y,¨ ρ, λ, 0, 0), (10) ∞ ∞ where again h = h(( ) ) and f = f(( ) ). ∞ · ∞ ∞ · ∞ m Now, let s1,...,sm R ands ˙ =(s1,...,sm) = sjXj and moreover, withx, ˙ x,¨ y,˙ y,¨ a,˙ a¨ and ∈ ∞ j=1 c˙ as above, let (g) =(˙x, x,¨ y,˙ y,t,z,¨ a,˙ a¨) =(x,y,t,z,aP ) =(x,h) . By Calculation (9) for ∞ ∞ ∞ ∞ ℓ + (˜x, y˜) instead of ℓ + ℓ , one gets for ξ˙ L2(Rm), α,¨ β¨ i ∈ ˙ πℓ+(˜x,y˜) (g) ξi (˙s) ∞ ∞

2πi ℓ+(˜x,y˜), (˙y)∞ (¨y)∞ (t)∞ (z)∞ (a)∞ (¨x)∞+[(˙s)∞,(˙y)∞+(¨y)∞+(t)∞+(z)∞+(a)∞+(¨x)∞] = e h − − − − − − i

1 2πi ℓ+(˜x,y˜), [(˙x)∞,(˙y)∞+(ÿ)∞+(t)∞+(z)∞+(a)∞+(˙s)∞+(¨x)∞] e h − 2 iξ˙i (˙s x˙) − ∞ 1 2πi ℓ, (t)∞ (z)∞+[(˙s)∞,(˙y)∞] [(˙x)∞,(˙y)∞] + (˜x,y˜), (ÿ)∞ (¨x)∞ = e h − − − 2 i h − − iξ˙i (˙s x˙) − ∞ 1 2πi( tρ zλ+λc˙(((˙s)∞ (˙x)∞)(˙y)∞) x,˜ (¨x)∞ y,˜ (ÿ)∞ ) = e − − − 2 −h i−h i ξ˙i (˙s x˙) . − ∞ 42 Identifying with R2d+2+p again, one gets with (10) for h = p (f) (G/U, χ ) = (R2d) G/U ∈ S ℓ ∼ S for a function f (G) = (R2d+2+p), ∈S ∼ S ˙ πℓ+(˜x,y˜)(h)ξi(˙s)

˙ = pG/U (f) (˜g)πℓ+(˜x,y˜)(˜g)ξi(˙s)dg˜ ∞ Z2d R ˙ = f (˜gu)χℓ(u)du πℓ+(˜x,y˜)(˜g)ξi(˙s)dg˜ ∞ RZ2d R2+˜p+(Z p−p˜)

2πi(λc˙((˙s 1 x˙)˙y) x,˜ x¨ y,˜ y¨ ) = f (˙x, x,¨ y,˙ y,t,z,¨ a,˙ a¨)e − 2 −h i−h i ∞ Rd Rd R ZR Rp˜ Rp−p˜ × × × × × 2πi(tρ+zλ) e− ξ˙ (˙s x˙)d(˙x, x,¨ y,˙ y,t,z,¨ a,˙ a¨) i − 2,4,5,6,7,8 2πiλc˙((˙s 1 x˙)˙y) = fˆ (˙x, x,˜ y,˙ y,˜ ρ, λ, 0, 0)e − 2 ξ˙i(˙s x˙)d(˙x, y˙) ∞ − RZ2m

(10) 2,4 2πiλc˙((˙s 1 x˙)˙y) = hˆ (˙x, x,˜ y,˙ y˜)e − 2 ξ˙i(˙s x˙)d(˙x, y˙) ∞ − RZ2m

2,3,4 1 = hˆ x,˙ x,λ˜ c˙ x˙ s˙ , y˜ ξ˙i(˙s x˙)dx.˙ (11) ∞ 2 − − RZm

Regard now the second factor Pη¨k,x,˜ y˜ of the tensor product. As in the second case above, deﬁne

d 1 1 β k 4 k 2 j η¨ ¨(¨s) := λkc ηj λkc sj + . k,β j j λ ck j=m+1 k j Y

Then, 2πiα¨s¨ η¨k,α,¨ β¨(¨s)= e η¨k,β¨(¨s) and therefore, with ξ¨ L2(Rd m), i ∈ − ¨ ¨ Pη¨k,x,˜ y˜ (ξi)(¨s) = ξi, η¨k,x,˜ y˜ η¨k,x,˜ y˜(¨s)

¨ = ξi(¨r)η¨k,x,˜ y˜(¨r)dr¨ η¨k,x,˜ y˜(¨s) RdZ−m

¨ 2πix˜r¨ 2πix˜s¨ = ξi(¨r)e− η¨k,y˜(¨r)dr¨ e η¨k,y˜(¨s) RdZ−m

¨ 2πix˜(¨s r¨) = ξi(¨r)e − η¨k,y˜(¨r)dr¨ η¨k,y˜(¨s). RdZ−m

43 Joining together the calculation above and the one for the first factor of the tensor product d ∞ m d m (11), one gets for (R ) ξ = ξ˙i ξï with ξ˙i (R ) and ξï (R − ) for all i N, S ∋ i=1 ⊗ ∈ S ∈ S ∈ h (R2d) and s =(˙s, s¨) Rd, P ∈S ∈

νk(h)(ξ)(s)

∞ d(˜x, y˜) = π (h)(ξ˙ )(˙s) P (ξ¨ )(¨s) ℓ+(˜x,y˜) i η¨k,x,˜ y˜ i d · k i=1 2(dZ−m) X R λkcj j=m+1 Q ∞ 2,3,4 1 = hˆ x,˙ x,λ˜ c˙ x˙ s˙ , y˜ ξ˙i(˙s x˙)dx˙ ∞ 2 − − i=1 X R2(dZ−m) RZm

2πix˜(¨s r¨) d(˜x, y˜) ξ¨ (¨r)e − η¨ (¨r)dr¨ η¨ (¨s) · i k,y˜ k,y˜ d d−m k R Z λkcj j=m+1 Q ∞ 2,3,4 1 = hˆ x,˙ x,λ˜ c˙ x˙ s˙ , y˜ ξ˙i(˙s x˙) ∞ 2 − − i=1 X R2(dZ−m) RdZ−m RZm 2πix˜(¨s r¨) d(˜x, y˜) ξ¨ (¨r)e − η¨ (¨r)dxd˙ r¨ η¨ (¨s) i k,y˜ k,y˜ d k λkcj j=m+1 Q ∞ 3,4 1 = hˆ x,˙ s¨ r,λ¨ c˙ x˙ s˙ , y˜ ξ˙i(˙s x˙) ∞ − 2 − − Xi=1 RdZ−m RdZ−m RZm dy˜ ξ¨ (¨r)η¨ (¨r)dxd˙ r¨ η¨ (¨s) i k,y˜ k,y˜ d k λkcj j=m+1 Q ∞ 3,4 λ = hˆ s˙ x,˙ s¨ r,¨ c˙( x˙ s˙), y˜ ξ˙i(˙x) ∞ − − 2 − − Xi=1 RdZ−m RdZ−m RZm dy˜ ξ¨ (¨r)η¨ (¨r)dxd˙ r¨ η¨ (¨s) i k,y˜ k,y˜ d k λkcj j=m+1 λ Q dy˜ = hˆ3,4 s˙ x,˙ s¨ r,¨ c˙( x˙ s˙), y˜ η¨ (¨r)¨η (¨s) k,y˜ k,y˜ d ∞ − − 2 − − k RZd RdZ−m λkcj j=m+1 Q ξ(˙x, r¨)d(˙x, r¨). (12)

Therefore, the kernel function λ dy˜ h (˙s, s¨), (˙x, r¨) := hˆ3,4 s˙ x,˙ s¨ r,¨ c˙( x˙ s˙), y˜ η¨ (¨r)¨η (¨s) K k,y˜ k,y˜ d ∞ − − 2 − − k dZ−m R λkcj j=m+1 Q

44 of ν (h) is in (R2d) and thus, ν (h) is a compact operator for h (R2d) = (G/U, χ ). k S k ∈ S ∼ S ℓ With the density of (G/U, χ ) in C (G/U, χ ), it is compact for every h C (G/U, χ ). S ℓ ∗ ℓ ∈ ∗ ℓ 2(d m) Now, it will be shown that for every ϕ C (R − , ), ∈ ∞ K µk(ϕ) op ϕ := sup ϕ(˜x, y˜) op. k k ≤ k k∞ (˜x,y˜) R2(d−m) k k ∈ For this, for any ψ L2(Rd), deﬁne ∈ 2(d m) m f (˜x, y˜)(˙s) := ψ(˙s, s¨)η¨ (¨s)ds¨ (˜x, y˜) R − s˙ R . ψ,k k,x,˜ y˜ ∀ ∈ ∀ ∈ RdZ−m Then, as d(˜x, y˜) I 2 Rd−m = P , L ( ) η¨k,x,˜ y˜ d k R2(dZ−m) λkcj j=m+1 Q one gets the identity

2 2 d(˜x, y˜) ψ d = f (˜x, y˜) 2 m . (13) L2(R ) ψ,k L (R ) d k k k k k R2(dZ−m) λkcj j=m+1 Q Now, for ξ,ψ L2(Rd), ∈ µ (ϕ)ξ,ψ 2 Rd h k iL ( )

d(˜x, y˜) = (ϕ(˜x, y˜) P )ξ,ψ η¨k,x,˜ y˜ L2(Rd) d ⊗ k R2(dZ−m) λkc j j=m+1 Q

I I m = (ϕ(˜x, y˜) L2(Rd−m)) ( L2(R ) Pη¨k,x,˜ y˜ )ξ, ⊗ ◦ ⊗ R2(dZ−m)

d(˜x, y˜) (I 2 Rm P )ψ L ( ) η¨k,x,˜ y˜ L2(Rd) d ⊗ k λkc j j=m+1 Q d(˜x, y˜) = ϕ(˜x, y˜)f (˜x, y˜),f (˜x, y˜) 2 Rm ξ,k ψ,k L ( ) d h i k R2(dZ−m) λkc j j=m+1 Q 1 Cauchy 2 − 2 d(˜x, y˜) ϕ(˜x, y˜)f (˜x, y˜) 2 m ξ,k L (R ) d Schwarz≤ k k k ! R2(dZ−m) λkcj j=m+1 Q 1 2 2 d(˜x, y˜) f (˜x, y˜) 2 m ψ,k L (R ) d k k k ! R2(dZ−m) λkcj j=m+1 Q

45 (13) sup ϕ(˜x, y˜) op ≤ (˜x,y˜) R2(d−m) k k ∈ 1 2 2 d(˜x, y˜) f (˜x, y˜) 2 m ψ 2 Rd ξ,k L (R ) d L ( ) k k k ! k k R2(dZ−m) λkcj j=m+1

(13) Q ϕ ξ L2(Rd) ψ L2(Rd). ≤ k k∞k k k k Hence, for every h C (G/U, χ ), ∈ ∗ ℓ νk(h) op = µk( (h)) op (h) = h C∗(G/U,χ ). k k k F k ≤ kF k∞ k k ℓ 3) To show that νk is involutive is straightforward as in the second case.

Like in the second case mentioned in Proposition 3.1, this proves that the image of νk is contained in ( ), as well as the required boundedness and involutiveness of the B Hi mappings ν for every k N. Hence, only a modiﬁcation of the demanded convergence of k ∈ Condition 3(b) of Deﬁnition 1.1 remains to be shown.

3.6.3 Theorem – Third Case Theorem 3.6. For a C (G) ∈ ∗ lim πk(a) νk pG/U (a) = 0. k − op →∞ Proof: R2d+2+p Let f (G) ∼= ( ) such that its Fourier transform in [g, g] has a compact support ∈R S2d+2+p S ˆ6,7 in G ∼= . In the setting of this third case, this means that f has a compact support in G (compare the proof of Theorem 3.4).

Identify G with R2d+2+p again, let ξ L2(Rd) and s =(s ,...,s )=(˙s, s¨) Rm Rd m = Rd ∈ 1 d ∈ × − ∼ and deﬁne d 1 1 k 4 k 2 η¨k,0(¨s) := λkcj ηj λkcj (sj) . j=m+1 Y k k k k k k Letc ˙ =(c1,...,cm),c ¨ =(cm+1,...,cd) = (0,..., 0),c ˙ =(c1,...,cm) andc ¨ =(cm+1,...,cd). As in the second case, the expression π (f) ν p (f) will now be regarded, divided k − k G/U into several parts and then estimated. For this, Equation (6) from Section 3.3.4 will be used again but its notation needs to be adapted. k (6) λ c π (f)(s) = fˆ2,3,4,5,6 s r, k (s + r), ρ ,λ , 0, 0 ξ(r)dr k k − − 2 k k RZd λ c˙k λ c¨k = fˆ3,4,5,6,7,8 s˙ r,˙ s¨ r,¨ k (˙s +r ˙), k (¨s +r ¨), ρ ,λ , 0, 0 k − − − 2 − 2 k k RZd ξ(˙r, r¨)d(˙r, r¨).

46 Using the above equation, Result (12) and the fact that p (f) = fˆ5,6,7,8( , , , , ρ, λ, 0, 0), G/U · · · · one gets

π (f) ν p (f) ξ(s) k − k G/U k k (12) λ c˙ λ c¨ = fˆ3,4,5,6,7,8 s˙ r,˙ s¨ r,¨ k (˙s +r ˙), k (¨s +r ¨), ρ ,λ , 0, 0 ξ(˙r, r¨)d(˙r, r¨) k − − − 2 − 2 k k RZd \ 3,4 λ dy˜ p (f) s˙ r,˙ s¨ r,¨ c˙( r˙ s˙), y˜ η¨ (¨r)¨η (¨s) G/U k,y˜ k,y˜ d − ∞ − − 2 − − k RZd RdZ−m λkcj j=m+1

ξ(˙r, r¨)d(˙r, r¨) Q k k η˜k,0 2=1 λ c˙ λ c¨ k =k fˆ3,4,5,6,7,8 s˙ r,˙ s¨ r,¨ k (˙s +r ˙), k (¨s +r ¨), ρ ,λ , 0, 0 k − − − 2 − 2 k k RZd RdZ−m

η¨k,0(˜y)¨ηk,0(˜y)dy˜ ξ(˙r, r¨)d(˙r, r¨)

ˆ3,4,5,6,7,8 λ f s˙ r,˙ s¨ r,¨ c˙( r˙ s˙), y,˜ ρ, λ, 0, 0 η¨k,y˜(¨r) − ∞ − − 2 − − RZd RdZ−m dy˜ η¨ (¨s) ξ(˙r, r¨)d(˙r, r¨) k,y˜ d k λkcj j=m+1 Q λ c˙k λ c¨k = fˆ3,4,5,6,7,8 s˙ r,˙ s¨ r,¨ k (˙s +r ˙), k (¨s +r ¨), ρ ,λ , 0, 0 k − − − 2 − 2 k k RZd RdZ−m

η¨k,0(˜y)¨ηk,0(˜y)dy˜ ξ(˙r, r¨)d(˙r, r¨)

3,4,5,6,7,8 λ k fˆ s˙ r,˙ s¨ r,¨ c˙(˙s +r ˙),λkc¨ (˜y s¨), ρ, λ, 0, 0 − ∞ − − − 2 − RZd RdZ−m η¨ (˜y +r ¨ s¨)¨η (˜y)dy˜ ξ(˙r, r¨)d(˙r, r¨). k,0 − k,0

Similarly as in the second case, functions qk, uk, vk, ok, nk and wk are going to be deﬁned in order to divide the above integrals into six parts.

q (s, y˜) := ξ(˙r, r¨)η¨ (˜y +r ¨ s¨) k k,0 − RZd λ c˙k λ c¨k fˆ3,4,5,6,7,8 s˙ r,˙ s¨ r,¨ k (˙s +r ˙), k (¨s +r ¨), ρ ,λ , 0, 0 k − − − 2 − 2 k k λ c˙k λ c¨k fˆ3,4,5,6,7,8 s˙ r,˙ s¨ r,¨ k (˙s +r ˙), k (¨s +r ¨), ρ, λ , 0, 0 d(˙r, r¨), − k − − − 2 − 2 k

47 u (s, y˜) := ξ(˙r, r¨)η¨ (˜y +r ¨ s¨) k k,0 − RZd λ c˙k λ c¨k fˆ3,4,5,6,7,8 s˙ r,˙ s¨ r,¨ k (˙s +r ˙), k (¨s +r ¨), ρ, λ , 0, 0 k − − − 2 − 2 k λ c˙k λ c¨k fˆ3,4,5,6,7,8 s˙ r,˙ s¨ r,¨ k (˙s +r ˙), k (¨s +r ¨), ρ, λ, 0, 0 d(˙r, r¨), − k − − − 2 − 2 v (s, y˜) := ξ(˙r, r¨)η¨ (˜y +r ¨ s¨) k k,0 − RZd λ c˙k λ c¨k fˆ3,4,5,6,7,8 s˙ r,˙ s¨ r,¨ k (˙s +r ˙), k (¨s +r ¨), ρ, λ, 0, 0 k − − − 2 − 2 λ c˙k fˆ3,4,5,6,7,8 s˙ r,˙ s¨ r,¨ k (˙s +r ˙),λ c¨k(˜y s¨), ρ, λ, 0, 0 d(˙r, r¨), − k − − − 2 k − o (s, y˜) := ξ(˙r, r¨)η¨ (˜y +r ¨ s¨) k k,0 − RZd λ c˙k fˆ3,4,5,6,7,8 s˙ r,˙ s¨ r,¨ k (˙s +r ˙),λ c¨k(˜y s¨), ρ, λ, 0, 0 k − − − 2 k − λ fˆ3,4,5,6,7,8 s˙ r,˙ s¨ r,¨ c˙(˙s +r ˙),λ c¨k(˜y s¨), ρ, λ, 0, 0 d(˙r, r¨), − k − − − 2 k − n (s, y˜) := ξ(˙r, r¨)η¨ (˜y +r ¨ s¨) k k,0 − RZd λ fˆ3,4,5,6,7,8 s˙ r,˙ s¨ r,¨ c˙(˙s +r ˙),λ c¨k(˜y s¨), ρ, λ, 0, 0 k − − − 2 k − 3,4,5,6,7,8 λ k fˆ s˙ r,˙ s¨ r,¨ c˙(˙s +r ˙),λkc¨ (˜y s¨), ρ, λ, 0, 0 d(˙r, r¨) − ∞ − − − 2 − and

w (s) := ξ(˙r, r¨)¨η (˜y) η¨ (˜y) η¨ (˜y +r ¨ s¨) k k,0 k,0 − k,0 − RdZ−m RZd λ c˙k λ c¨k fˆ3,4,5,6,7,8 s˙ r,˙ s¨ r,¨ k (˙s +r ˙), k (¨s +r ¨), ρ ,λ , 0, 0 d(˙r, r¨)dy.˜ k − − − 2 − 2 k k Then,

π (f) ν p (f) ξ(s) = q (s, y˜)¨η (˜y)dy˜ + u (s, y˜)¨η (˜y)dy˜ k − k G/U k k,0 k k,0 dZ−m dZ−m R R

+ vk(s, y˜)¨ηk,0(˜y)dy˜ + ok(s, y˜)¨ηk,0(˜y)dy˜

RdZ−m RdZ−m

+ nk(s, y˜)¨ηk,0(˜y)dy˜ + wk(s).

RdZ−m

48 As in the second case, to show that

k π (f) ν p (f) →∞ 0, k − k G/U op −→ one has to prove that there are κ , γ , δ , τ , ω and ǫ which are tending to 0 for k k k k k k k → ∞ such that

q κ ξ , u γ ξ , v δ ξ , o τ ξ , k kk2 ≤ kk k2 k kk2 ≤ kk k2 k kk2 ≤ kk k2 k kk2 ≤ kk k2 n ω ξ and w ǫ ξ . k kk2 ≤ kk k2 k kk2 ≤ kk k2 The estimation of the functions qk, uk, vk, nk and wk is similar to their estimation in the second case and will thus be skipped. So, only the estimation of ok remains.

For this, ﬁrst regard the last factor of the function ok:

λ c˙k fˆ3,4,5,6,7,8 s˙ r,˙ s¨ r,¨ k (˙s +r ˙),λ c¨k(˜y s¨), ρ, λ, 0, 0 k − − − 2 k − λ fˆ3,4,5,6,7,8 s˙ r,˙ s¨ r,¨ c˙(˙s +r ˙),λ c¨k(˜y s¨), ρ, λ, 0, 0 − k − − − 2 k − 1 1 = (λc˙ λ c˙k)(˙s +r ˙) ∂ fˆ3,4,5,6,7,8 s˙ r,˙ s¨ r,¨ 2 − k · 3 k − − Z0 λ 1 c˙(˙s +r ˙)+ t (λc˙ λ c˙k)(˙s +r ˙) ,λ c¨k(˜y s¨), ρ, λ, 0, 0 dt. − 2 2 − k k − Thus, there exists a constant C1 > 0 depending on f such that

λ c˙k fˆ3,4,5,6,7,8 s˙ r,˙ s¨ r,¨ k (˙s +r ˙),λ c¨k(˜y s¨), ρ, λ, 0, 0 k − − − 2 k −

λ fˆ3,4,5,6,7,8 s˙ r,˙ s¨ r,¨ c˙(˙s +r ˙),λ c¨k(˜y s¨), ρ, λ, 0, 0 − k − − − 2 k −

C λc˙ λ c˙k s˙ +r ˙ 1 . ≤ − k k k (1 + s˙ +r ˙ )2d+1 (1 + s¨ r¨ )2d k k k − k Therefore, one gets

o 2 k kk2 = o (s, y˜) 2d(s, y˜) | k | Rd+(Zd−m)

ξ(˙r, r¨) η¨ (˜y +r ¨ s¨) λc˙ λ c˙k s˙ +r ˙ ≤ | | k,0 − − k k k d+(Zd−m) Zd R R

C 2 1 d(˙r, r¨) d(˙s, s,¨ y˜) (1 + s˙ +r ˙ )2d+1 (1 + s¨ r¨ )2d k k k − k

49 Cauchy − 2 k 2 1 C1 λc˙ λkc˙ 2d 2d d(˙r, r¨) Schwarz≤ − (1 + s˙ +r ˙ ) (1 + s¨ r¨ ) d+(Zd−m) Zd R R k k k − k 2 2 s˙ +r ˙ ξ(˙r, r¨) 2 η¨ (˜y +r ¨ s¨) k k d(˙r, r¨) d(˙s, s,¨ y˜) | | k,0 − (1 + s˙ +r ˙ )2d+2(1 + s¨ r¨ )2d Zd R k k k − k

k 2 2 2 C′ λc˙ λ c˙ ξ(˙r, r¨) η¨ (˜y +r ¨ s¨) ≤ 1 − k | | k,0 − d+(Zd−m) Zd R R 1 d(˙r, r¨)d(˙s, s,¨ y˜) (1 + s˙ +r ˙ )2d (1 + s¨ r¨ )2d k k k − k ηk,0 2=1 k 2 2 k =k C′′ λc˙ λ c˙ ξ , 1 − k k k2 k with matching constants C 1′ > 0 and C1′′ > 0 depending on f. Hence, τk := C1′′ λc˙ λkc˙ k − fulﬁlls τ →∞ 0 and k −→ p o τ ξ . k kk2 ≤ kk k2

Thus, for those f (R2d+2+p) = (G) whose Fourier transform in [g, g] has a compact ∈ S ∼ S support, k →∞ πk(f) νk pG/U (f)) op = sup πk(f) νk pG/U (f) (ξ) 2 0. − ξ L2(Rd) − −→ ∈ ξ 2=1 k k As in the second case, because of the density in C∗(G) of the set of Schwartz functions whose partial Fourier transform has a compact support, the claim follows for all a C (G) by ∈ ∗ Corollary 7.6.

V 3.6.4 Transition to πk k N ∈ V Now, the assertions for the sequence πk k N can be deduced. ∈ Again, because of the equivalence of the representations π and πV for every k N, there k k ∈ exist unitary intertwining operators 2 Rd 2 Rd V Fk : πV = L ( ) πk = L ( ) such that Fk πk (a)= πk(a) Fk a C∗(G). H k ∼ →H ∼ ◦ ◦ ∀ ∈ With the injective -homomorphism ∗ R2(d m) τ : C − , C∗(G/U, χℓ), (a) L(( ) ) pG/U (a), ∞ K → F | Ok˜ k˜∈N 7→ define ν˜k(ϕ) := Fk∗ (νk τ) ϕ L(( ) ) Fk ϕ CB(Si 1). ◦ ◦ | Ok˜ k˜∈N ◦ ∀ ∈ − Then, like in the second case,ν ˜k complies with the demanded requirements and thus, the V original representations πk k N fulfill Property 3(b). ∈ This completes the proof of the three conditions of Definition 1.1 which are needed in order to be able to use the above-mentioned theorem of H.Regeiba and J.Ludwig (see [31], Theorem 3.5) for the determination of C∗(G).

50 3.7 Result for the connected real two-step nilpotent Lie groups

Finally, with the families of subsets (Si)i 0,...,r and (Γi)i 0,...,r of G that were constructed in ∈{ } ∈{ } Section 3.2 and the Hilbert spaces ( i)i 0,...,r which were defined in the same section fulfilling H ∈{ } the conditions listed in Definition 1.1, one obtains the following resub lt for the connected real two-step nilpotent Lie groups which was already stated in Section 1: Theorem 3.7. The C∗-algebra C∗(G) of a connected real two-step nilpotent Lie group G is isomorphic (under the Fourier transform) to the set of all operator fields ϕ defined over the spectrum G of the respective group such that: b 1. ϕ(γ) ( ) for every i 1,...,r and every γ Γ . ∈K Hi ∈ { } ∈ i 2. ϕ l (G). ∈ ∞ 3. The mappings γ ϕ(γ) are norm continuous on the different sets Γ . b 7→ i 4. For any sequence (γk)k N G going to infinity, lim ϕ(γk) op = 0. ∈ ⊂ k k k →∞ 5. For i 1,...,r and anyb properly converging sequence γ = (γk)k N Γi whose ∈ { } ∈ ⊂ limit set is contained in Si 1 (taking a subsequence if necessary) and for the mappings − ν˜k =ν ˜γ,k : CB(Si 1) ( i) constructed in the preceding sections, one has − →B H

lim ϕ(γk) ν˜k ϕ Si = 0. k − | −1 op →∞ 3.8 Example: The free two-step nilpotent Lie groups of 3 and 4 generators Now, as an example of two-step nilpotent Lie groups, the free two-step nilpotent Lie groups of n generators shall be considered in order to closely regard the cases n = 3 and n = 4. More detailed calculations can be found in the doctoral thesis of R.Lahiani (see [21]). Let f(2,n) be the free two-step nilpotent Lie algebra with n generators H1,...,Hn and F (2,n) the associated connected simply connected Lie group. Furthermore, let 0 = U := [H ,H ] for 6 ij i j all 1 i

51 3.8.1 The case n = 3

Now, let n = 3 and H4 := U12, H5 := U13 and H6 := U23. Then,

g := f(2, 3) = span H ,H ,H ,U ,U ,U = span H ,...H . { 1 2 3 12 13 23} { 1 6} Since [H ,H ] span H 4 i 6 for all 1 j,k 3 and [H , g] = 0 for all 4 i 6, the j k ⊂ { i| ≤ ≤ } ≤ ≤ i ≤ ≤ basis = H ,...,H is a Jordan-H¨older basis of g. B { 1 6} 6 R Let ℓ = αiHi∗ for αi , where Hi∗ is the dual basis element of Hi for all 1 i 6. i=1 ∈ ≤ ≤ Then, oneP gets the following results for the stabilizer, the Pukanszky index set, the polari- V zation pℓ of ℓ in g, the possible coadjoint orbits and the associated ℓ g∗ (compare O O ∈ Section 3.1 for the notation and approach):

1. If α4 = α5 = α6 = 0, the stabilizer g(ℓ) equals the whole Lie algebra: g(ℓ) = g. In this case, IPuk = and pV = g(ℓ)= g. ℓ ∅ ℓ In addition, 3 1 := Ad ∗(G)ℓ = ℓ + g(ℓ)⊥ = ℓ = α H∗. O i i Xi=1 So, 1 consists of one single element and obviously, ℓ vanishes on H i IPuk , i.e. ℓ O { i| ∈ ℓ } is the element ℓ 1 that was chosen in Section 3.2. O 2. If α = 0 or α = 0 or α = 0, 4 6 5 6 6 6 g(ℓ) = span α H + α H + α H ,H ,H ,H . { 6 1 5 2 4 3 4 5 6} V 1 6+4 Then, the dimension of the polarization is dim(pℓ )= 2 dim(g)+dim(g(ℓ)) = 2 = 5 V and thus, for the construction of the polarization pℓ , one has to ﬁnd one further element ℓ,V Y1 which is not in the stabilizer. In this case, one has to regard three subcases:

(a) If α = 0, IPuk = 2, 3 . Now, the largest index in 1,..., 6 with the property 6 6 ℓ { } { } that H g(ℓ) is the index j (ℓ) = 3. Furthermore, the largest index such that j1(ℓ) 6∈ 1 ℓ, [H ,H ] =0 is k (ℓ) = 2. Hence, k1(ℓ) j1(ℓ) 6 1

Y V,ℓ = H , XV,ℓ = H , pV = span H g(ℓ) and g = span H pV . 1 3 1 2 ℓ { 3}⊕ { 2}⊕ ℓ Moreover, in this case, g(ℓ) = span α H + α H , α H + α H and thus, ⊥ {− 5 1∗ 6 2∗ − 4 1∗ 6 3∗} 2 := Ad ∗(G)ℓ = ℓ + g(ℓ)⊥ = ℓ + span α H∗ + α H∗, α H∗ + α H∗ O {− 5 1 6 2 − 4 1 6 3 } = (α α a α b)H∗ +(α + α a)H∗ +(α + α b)H∗ 1 − 5 − 4 1 2 6 2 3 6 3 6 + α H∗ a,b R . i i ∈ i=4 X

52 If one chooses a := α2 and b := α3 , the corresponding element vanishes on − α6 − α6 H ,H = H i IPuk and therefore, one needs to define { 2 3} { i| ∈ ℓ } 6 α2α5 α3α4 ℓ 2 := α1 + + H1∗ + αiHi∗. O α6 α6 Xi=4 (b) If α = 0 and α = 0, IPuk = 1, 3 , j (ℓ) = 3 and k (ℓ) = 1. In this case, 6 5 6 ℓ { } 1 1 Y V,ℓ = H , XV,ℓ = H , pV = span H g(ℓ) and g = span H pV . 1 3 1 1 ℓ { 3}⊕ { 1}⊕ ℓ Furthermore, g(ℓ) = span H , α H + α H and hence, ⊥ { 1∗ − 4 2∗ 5 3∗} 3 := Ad ∗(G)ℓ = ℓ + g(ℓ)⊥ = ℓ + span H∗, α H∗ + α H∗ O { 1 − 4 2 5 3 } = aH∗ +(α α b)H∗ +(α + α b)H∗ + α H∗ + α H∗ a,b R . 1 2 − 4 2 3 5 3 4 4 5 5 | ∈ Here, if one chooses a = 0 and b = α3 , the corresponding element vanishes on − α5 H ,H = H i IPuk and thus, one defines { 1 3} { i| ∈ ℓ } α3α4 ℓ 3 := α2 + H2∗ + α4H4∗ + α5H5∗. O α5 (c) If α = α = 0 and α = 0, IPuk = 1, 2 , j (ℓ) = 2 and k (ℓ) = 1. Here, 6 5 4 6 ℓ { } 1 1 Y V,ℓ = H , XV,ℓ = H , pV = span H g(ℓ) and g = span H pV . 1 2 1 1 ℓ { 2}⊕ { 1}⊕ ℓ In addition, g(ℓ) = span H ,H and therefore, ⊥ { 1∗ 2∗} 4 := Ad ∗(G)ℓ = ℓ + g(ℓ)⊥ = ℓ + span H∗,H∗ O { 1 2 } = aH∗ + bH∗ + α H∗ + α H∗ a,b R . 1 2 3 3 4 4 | ∈ In this last case, for a =b = 0 the corresponding element in 4 vanishes on the set O H ,H = H i IPuk and one has to define { 1 2} { i| ∈ ℓ }

ℓ 4 := α3H3∗ + α4H4∗. O

In order to calculate the sets Γ for i 0,...,r , one has to compute the dimensions d of the i ∈ { } Pukanszky index sets for the different ℓ g . Moreover, one has to find the occuring pairs of ∈ ∗ sets (J, K), where J, K 1,..., 6 , J = K = d and J K = and such that there exists ⊂ { } | | | | ∩ ∅ an ℓ g that fulfills J(ℓ)= J and K(ℓ)= K. ∈ ∗ From the cases distinguished above, one can deduce that the only dimensions that appear are d = 0 and d = 1, that the only pair of dimension 0 is ( , ) and the ones of dimension 1 are ∅ ∅ ( 3 , 2 ), ( 3 , 1 ) and ( 2 , 1 ). { } { } { } { } { } { } Since ( , ) < ( 2 , 1 ) < ( 3 , 1 ) < ( 3 , 2 ), the pair ( , ) is assigned the number 0, ∅ ∅ { } { } { } { } { } { } ∅ ∅ ( 2 , 1 ) the number 1, ( 3 , 1 ) the 2 and the pair ( 3 , 2 ) is assigned the number 3. { } { } { } { } { } { }

53 Thus, r = 3 and

3 V Γ = π α ,α ,α R : ℓ = α H∗ , 0 ℓ ∃ 1 2 3 ∈ i i i=1 X V Γ = π α R,α R∗ : ℓ = α H∗ + α H∗ , 1 ℓ ∃ 3 ∈ 4 ∈ 3 3 4 4 V R R α3α4 Γ2 = πℓ α2,α3,α4 ,α5 ∗ : ℓ = α2 + H2∗ + α4H4∗ + α5H5∗ and ∃ ∈ ∈ α5 6 V R R α2α5 α3α4 Γ3 = πℓ α1,...,α5 ,α6 ∗ : ℓ = α1 + + H1∗ + αiHi∗ . ∃ ∈ ∈ α6 α6 i=4 X V In this example, one can also see that the situation in which the polarizations pℓ are discontinuous in ℓ on the set ℓ ′ ′ (g∗/G)2d occurs: { O | O ∈ } One can choose for every k N the orbit ∈ 1 1 1 = bH∗ + aH∗ + bH∗ H∗ + H∗ a,b R Ok 1 k 2 k 3 − 4 k 6 ∈

and the orbit = aH∗ + bH∗ H∗ a,b R , O { 1 2 − 4 | ∈ } which both have the dimension 2 and are thus located in the set ℓ ′ ′ (g∗/G)2 . More- V V { O | O ∈ } over, πℓ Γ3 and πℓ Γ1. Now, let Ok ∈ O ∈ ℓ˜= a H∗ + b H∗ H∗ O∋ ℓ˜ 1 ℓ˜ 2 − 4 k k N ˜ →∞ ˜ →∞ and choose sequences (˜ak)k and bk k N such thata ˜k aℓ˜ and bk bℓ˜. If one lets ∈ ∈ −→ −→ 1 1 1 ℓ :=a ˜ H∗ + k˜b H∗ + a˜ H∗ H∗ + H∗ , k k 1 k k 2 k k 3 − 4 k 6 ∈Ok

k ℓk →∞ ℓ˜. Hence, L(( k)k N). But on the other hand, for every k, −→ O∈ O ∈ V pℓ = span H3 g(ℓ) = span H1,H3,...,H6 and Ok { }⊕ { } pV = span H g(ℓ) = span H ,...,H . ℓO { 2}⊕ { 2 6} Thus, the sequence of polarizations pV does not converge to the polarization pV and ℓO k N ℓO k ∈ the discontinuity is proved. This shows the necessity of considering the sets ℓ ′ ′ (g∗/G)(J,K) instead of the sets { O | O ∈ } ℓ ′ ′ (g∗/G)2d . { O | O ∈ } Now, regard the three cases of Proposition 3.1 that were distinguished in the proof of the three conditions of Deﬁnition 1.1.

1. If one chooses the sequence ( k)k N as O ∈ 1 := aH∗ + bH∗ + H∗ + 1+ H∗ a,b R , Ok 1 2 3 k 4 ∈

54 V then πℓ Γ1 for every k. Furthermore, every ℓk k is of the form Ok ∈ ∈O 1 ℓ = a H∗ + b H∗ + H∗ + 1+ H∗ k ℓk 1 ℓk 2 3 k 4 k k for aℓ ,bℓ R. If aℓ →∞ a˜ and bℓ →∞ b˜, the sequence (ℓk)k N converges to k k ∈ k −→ ℓ k −→ ℓ ∈

ℓ˜= a H∗ + b H∗ + H∗ + H∗ aH∗ + bH∗ + H∗ + H∗ a,b R =: ℓ˜ 1 ℓ˜ 2 3 4 ∈ { 1 2 3 4 | ∈ } O V and therefore, π Γ1. ℓO ∈ Here, the orbit does not depend on ℓ˜ and is thus the only limit point of the sequence O ( k)k N. Hence, O ∈

L(( k)k N)= , where 2=dim( k) = dim( ) k N. O ∈ {O} O O ∀ ∈ This is an example for the ﬁrst case regarded in the proof of the conditions of Deﬁnition 1.1.

2. Now, choose the sequence ( k)k N as follows: O ∈ 1 := aH∗ + bH∗ + H∗ + H∗ a,b R . Ok 1 2 3 k 4 ∈

V Then, again πℓ Γ1 for every k and every ℓk k is of the form Ok ∈ ∈O 1 ℓ = a H∗ + b H∗ + H∗ + H∗ k ℓk 1 ℓk 2 3 k 4

k k for aℓ ,bℓ R. If aℓ →∞ a˜ and bℓ →∞ b˜, the sequence (ℓk)k N converges to k k ∈ k −→ ℓ k −→ ℓ ∈

ℓ = a H∗ + b H∗ + H∗ =: ℓ˜ 1 ℓ˜ 2 3 Oℓ˜ and πV Γ . Furthermore, ℓO 0 ℓ˜ ∈ h i L(( k)k N)= ˜ a˜,b˜ R , where dim( k) = 2, dim( ˜) = 0 O ∈ {Oℓ| ℓ ℓ ∈ } O Oℓ k N ˜ L(( k)k N). ∀ ∈ ∀Oℓ ∈ O ∈ This is therefore an example for the second case that was regarded in Proposition 3.1.

3. For an example of the third case mentioned in Proposition 3.1, one has to ﬁnd a sequence of orbits ( k)k N whose limit set L(( k)k N) consists of orbits of dimension strictly O ∈ O ∈ larger than 0 but strictly smaller than dim( ). But since in this regarded example of Ok G = F (2, 3) only orbits of the dimensions 0 and 2 appear, such a sequence ( k)k N does O ∈ not exist. So, in this example, there are just two cases to be distinguished: the ﬁrst one and the second one.

55 3.8.2 The case n = 4

Let n = 4 and H5 := U12, H6 := U13, H7 := U14, H8 := U23, H9 := U24 and H10 := U34. Then, g := f(2, 4) = span H ,H ,H ,H ,U ,U ,U ,U ,U ,U = span H ,...H . { 1 2 3 4 12 13 14 23 24 34} { 1 10} Since, as in the case n = 3, [H ,H ] span H 5 i 10 for all 1 j,k 4 and j k ⊂ { i| ≤ ≤ } ≤ ≤ [H , g] = 0 for all 5 i 10, the basis = H ,...,H is a Jordan-Hölder basis of g. i ≤ ≤ B { 1 10} 10 Now, let ℓ = αiHi∗. One gets the following results: i=1 P 1. If α5 = ... = α10 = 0, g(ℓ)= g and like for n = 3, IPuk = and pV = g(ℓ)= g. ℓ ∅ ℓ Moreover, 4 1 := Ad ∗(G)ℓ = ℓ + g(ℓ)⊥ = ℓ = α H∗ O i i Xi=1 1 and, as consists of one single element, ℓ is the element ℓ 1 that was chosen in O O Section 3.2. 2. If at least one of the α for i 5,..., 10 is not zero, one has to regard two cases: i ∈ { } (a) If α α = α α + α α , 6 9 6 5 10 7 8 g(ℓ) = span H ,...,H =[g, g] { 5 10} Puk and Iℓ = 1, 2, 3, 4 . { } V 10+6 Hence, the dimension of the polarization is dim(pℓ ) = 2 = 8 and thus, for its ℓ,V ℓ,V construction, one has to find two further elements Y1 and Y2 which are not in the stabilizer. Here, the largest index in 1,..., 10 with the property that H g(ℓ) is the { } j1(ℓ) 6∈ index j (ℓ) = 4. To find the largest index k (ℓ) such that ℓ, [H ,H ] = 0, 1 1 k1(ℓ) j1(ℓ) 6 one has to consider several cases: V,ℓ V,ℓ (i) If α10 = 0, one has k1(ℓ) = 3 and therefore, Y1 = H4 and X1 = H3.A 6 1 V,ℓ V,ℓ basis for the ideal g which is needed for the construction of Y2 and X2 is given by the set α α H1 = H 7 H ,H1 = H 9 H ,H1 = H ,...,H1 = H . 1 1 − α 3 2 2 − α 3 4 4 10 10 10 10 1 The stabilizer g ℓ g1 can then be calculated to be | 1 1 1 g ℓ g1 = H4 ,...,H10 . | 1 1 Thus, the largest index j2(ℓ) 1, 2, 4,..., 10 such that Hj (ℓ) g ℓ g1 , is ∈ { } 2 6∈ | j2(ℓ) = 2. Last, the largest index k2(ℓ) 1, 2, 4,..., 10 in such a way that 1 1 ∈ { } ℓ, [H ,H ] = 0, is k2(ℓ) = 1. So, k2(ℓ) j2(ℓ) 6 V,ℓ V,ℓ V,ℓ V,ℓ Y1 = H4, Y2 = H2, X1 = H3, X2 = H1, pV = span H ,H g(ℓ) and g = span H ,H pV . ℓ { 4 2}⊕ { 3 1}⊕ ℓ

56 (ii) If α = 0 and α = 0, one has k (ℓ) = 2, Y V,ℓ = H and XV,ℓ = H . In this 10 9 6 1 1 4 1 2 case, a basis for g1 is given by

α H1 = H 7 H ,H1 = H ,...,H1 = H 1 1 − α 2 3 3 10 10 9 1 and again the stabilizer g ℓ g1 can be computed to be | 1 1 1 g ℓ g1 = H4 ,...,H10 . | 1 1 Moreover, the largest index j2(ℓ) 1, 3,..., 10 such that Hj (ℓ) g ℓ g1 , ∈ { } 2 6∈ | is j2(ℓ) = 3 and the largest index k2(ℓ) 1, 3,..., 10 in such a way that 1 1 ∈ { } ℓ, [H ,H ] = 0, is k2(ℓ) = 1. Therefore, k2(ℓ) j2(ℓ) 6 V,ℓ V,ℓ V,ℓ V,ℓ Y1 = H4, Y2 = H3, X1 = H2, X2 = H1,

pV = span H ,H g(ℓ) and g = span H ,H pV . ℓ { 4 3}⊕ { 2 1}⊕ ℓ (iii) If α = α = 0, from the assumption that α α = α α + α α follows that 10 9 6 9 6 5 10 7 8 α = 0. Hence, k (ℓ) = 1, Y V,ℓ = H and XV,ℓ = H . Here, a basis for g1 is 8 6 1 1 4 1 1 given by 1 1 H2 = H2,...,H10 = H10 1 and again the stabilizer g ℓ g1 can be calculated to be | 1 1 1 g ℓ g1 = H4 ,...,H10 . | Furthermore, j2(ℓ) = 3 and k2(ℓ) = 2 and thus,

V,ℓ V,ℓ V,ℓ V,ℓ Y1 = H4, Y2 = H3, X1 = H1, X2 = H2,

pV = span H ,H g(ℓ) and g = span H ,H pV . ℓ { 4 3}⊕ { 1 2}⊕ ℓ Now, in Case (a), g(ℓ) = span H ,H ,H ,H and therefore, ⊥ { 1∗ 2∗ 3∗ 4∗} 2 := Ad ∗(G)ℓ = ℓ + g(ℓ)⊥ = ℓ + span H∗,H∗,H∗,H∗ O { 1 2 3 4 } 10 = aH∗ + bH∗ + cH∗ + dH∗ + α H∗ a,b,c,d R . 1 2 3 4 i i ∈ i=5 X If one chooses a = b = c = d = 0, the corresponding element vanishes on H ,H ,H ,H = H i IPuk and thus, one deﬁnes { 1 2 3 4} { i| ∈ ℓ } 10

ℓ 2 := αiHi∗. O Xi=5

(b) If α6α9 = α5α10 +α7α8, again, there are several cases. In all of them, the dimension V of the stabilizer is 8 and hence, dim(pℓ ) = 9. Thus, for the construction of the V ℓ,V polarization pℓ , one has to ﬁnd one further element Y1 which is not in the stabilizer.

57 (i) If α = 0, 10 6 g(ℓ) = span α H α H + α H ,α H α H + α H ,H ,...,H { 10 1 − 7 3 6 4 10 2 − 9 3 8 4 5 10} and IPuk = 3, 4 . Now, the largest index in 1,..., 10 with the property ℓ { } { } that H g(ℓ), is the index j (ℓ) = 4. Furthermore, the largest index such j1(ℓ) 6∈ 1 that ℓ, [H ,H ] = 0, is k (ℓ) = 3. Hence, k1(ℓ) j1(ℓ) 6 1 Y V,ℓ = H , XV,ℓ = H , pV = span H g(ℓ) and g = span H pV . 1 4 1 3 ℓ { 4}⊕ { 3}⊕ ℓ In addition, g(ℓ) = span α H α H + α H ,α H + α H + α H ⊥ {− 7 1∗ − 9 2∗ 10 3∗ 6 1∗ 8 2∗ 10 4∗} and therefore,

3 := Ad ∗(G)ℓ = ℓ + g(ℓ)⊥ O = ℓ + span α H∗ α H∗ + α H∗,α H∗ + α H∗ + α H∗ {− 7 1 − 9 2 10 3 6 1 8 2 10 4 } = (α α a + α b)H∗ +(α α a + α b)H∗ +(α + α a)H∗ 1 − 7 6 1 2 − 9 8 2 3 10 3 10 +(α + α b)H∗ + α H∗ a,b R . 4 10 4 i i ∈ i=5 X If one chooses a := α3 and b := α4 , the corresponding element vanishes − α10 − α10 on H ,H = H i IPuk and hence, one needs to deﬁne { 3 4} { i| ∈ ℓ } 10 α3α7 α4α6 α3α9 α4α8 ℓ 3 := α1 + H1∗ + α2 + H2∗ + αiHi∗. O α10 − α10 α10 − α10 Xi=5 (ii) If α = 0 and α = 0, 10 9 6 g(ℓ) = span α H α H + α H ,α H α H ,H ,...,H , { 9 1 − 7 2 5 4 9 3 − 8 4 5 10} IPuk = 2, 4 , j (ℓ) = 4 and k (ℓ) = 2. Thus, ℓ { } 1 1 Y V,ℓ = H , XV,ℓ = H , pV = span H g(ℓ) and g = span H pV . 1 4 1 2 ℓ { 4}⊕ { 2}⊕ ℓ Furthermore, g(ℓ) = span α H +α H ,α H +α H +α H and hence, ⊥ {− 7 1∗ 9 2∗ 5 1∗ 8 3∗ 9 4∗} 4 := Ad ∗(G)ℓ = ℓ + g(ℓ)⊥ O = ℓ + span α H∗ + α H∗,α H∗ + α H∗ + α H∗ {− 7 1 9 2 5 1 8 3 9 4 } = (α α a + α b)H∗ +(α + α a)H∗ +(α + α b)H∗ 1 − 7 5 1 2 9 2 3 8 3 9 +(α + α b)H∗ + α H∗ a,b R . 4 9 4 i i ∈ i=5 X In this case, if one chooses a = α2 and b = α4 , the corresponding element − α9 − α9 vanishes on H ,H = H i IPuk and hence, here one deﬁnes { 2 4} { i| ∈ ℓ } 9 α2α7 α4α5 α4α8 ℓ 4 := α1 + H1∗ + α3 H3∗ + αiHi∗. O α9 − α9 − α9 Xi=5

58 (iii) If α = α = 0 and α = 0, 10 9 8 6 g(ℓ) = span α H α H + α H ,H ,...,H , { 8 1 − 6 2 5 3 4 10} IPuk = 2, 3 , j (ℓ) = 3 and k (ℓ) = 2. In this case, ℓ { } 1 1 Y V,ℓ = H , XV,ℓ = H , pV = span H g(ℓ) and g = span H pV . 1 3 1 2 ℓ { 3}⊕ { 2}⊕ ℓ Here, g(ℓ) = span α H + α H ,α H + α H and ⊥ {− 6 1∗ 8 2∗ 5 1∗ 8 3∗} 5 := Ad ∗(G)ℓ = ℓ + g(ℓ)⊥ O = ℓ + span α H∗ + α H∗,α H∗ + α H∗ {− 6 1 8 2 5 1 8 3 } = (α α a + α b)H∗ +(α + α a)H∗ +(α + α b)H∗ 1 − 6 5 1 2 8 2 3 8 3 8 + α H∗ a,b R . i i ∈ i=4 X For a = α2 and b = α3 , the corresponding element in 5 vanishes on the − α8 − α8 O set H ,H = H i IPuk and one has to deﬁne { 2 3} { i| ∈ ℓ } 8 α2α6 α3α5 ℓ 5 := α1 + H1∗ + αiHi∗. O α8 − α8 Xi=4 (iv) If α = α = α = 0 and α = 0, 10 9 8 7 6 g(ℓ) = span α H α H ,α H α H ,H ,...,H , { 7 2 − 5 4 7 3 − 6 4 5 10} IPuk = 1, 4 , j (ℓ) = 4 and k (ℓ) = 1. In this case, ℓ { } 1 1 Y V,ℓ = H , XV,ℓ = H , pV = span H g(ℓ) and g = span H pV . 1 4 1 1 ℓ { 4}⊕ { 1}⊕ ℓ Moreover, g(ℓ) = span H , α H α H + α H and hence, ⊥ { 1∗ − 5 2∗ − 6 3∗ 7 4∗} 6 := Ad ∗(G)ℓ = ℓ + g(ℓ)⊥ O = ℓ + span H∗, α H∗ α H∗ + α H∗ { 1 − 5 2 − 6 3 7 4 } = aH∗ +(α α b)H∗ +(α α b)H∗ +(α + α b)H∗ 1 2 − 5 2 3 − 6 3 4 7 4 7 + α H∗ a,b R . i i ∈ i=5 X Here, for a = 0 and b = α4 , the corresponding element in 6 vanishes on − α7 O the set H ,H = H i IPuk and one has to deﬁne { 1 4} { i| ∈ ℓ } 7 α4α5 α4α6 ℓ 6 := α2 + H2∗ + α3 + H3∗ + αiHi∗. O α7 α7 Xi=5

59 (v) If α = ... = α = 0 and α = 0, 10 7 6 6 g(ℓ) = span α H α H ,H ,...,H , { 6 2 − 5 3 4 10} IPuk = 1, 3 , j (ℓ) = 3 and k (ℓ) = 1. Hence, ℓ { } 1 1 Y V,ℓ = H , XV,ℓ = H , pV = span H g(ℓ) and g = span H pV . 1 3 1 1 ℓ { 3}⊕ { 1}⊕ ℓ In this case, g(ℓ) = span H , α H + α H and thus, ⊥ { 1∗ − 5 2∗ 6 3∗} 7 := Ad ∗(G)ℓ = ℓ + g(ℓ)⊥ = ℓ + span H∗, α H∗ + α H∗ O { 1 − 5 2 6 3 } 6 = aH∗ +(α α b)H∗ +(α + α b)H∗ + α H∗ a,b R . 1 2 − 5 2 3 6 3 i i ∈ i=4 X Now, for a = 0 and b = α3 , the corresponding element in 7 vanishes on − α6 O the set H ,H = H i IPuk and one has to deﬁne { 1 3} { i| ∈ ℓ } 6 α3α5 ℓ 7 := α2 + H2∗ + αiHi∗. O α6 Xi=4 (vi) If α = ... = α = 0 and α = 0, 10 6 5 6 g(ℓ) = span H ,...,H , { 3 10} IPuk = 1, 2 , j (ℓ) = 2 and k (ℓ) = 1. Thus, ℓ { } 1 1 Y V,ℓ = H , XV,ℓ = H , pV = span H g(ℓ) and g = span H pV . 1 2 1 1 ℓ { 2}⊕ { 1}⊕ ℓ Here, g(ℓ) = span H ,H and ⊥ { 1∗ 2∗} 8 := Ad ∗(G)ℓ = ℓ + g(ℓ)⊥ = ℓ + span H∗,H∗ O { 1 2 } 5 = aH∗ + bH∗ + α H∗ a,b R . 1 2 i i ∈ i=3 X In this last case, for a = b = 0, the corresponding element in 8 vanishes on O the set H ,H = H i IPuk and one has to deﬁne { 1 2} { i| ∈ ℓ } 5

ℓ 8 := αiHi∗. O Xi=3 For n = 4, the dimensions d = 0, d = 1 and d = 2 appear. The only pair of dimension 0 is ( , ), the ones of dimension 1 are ( 4 , 3 ), ( 4 , 2 ), ( 3 , 2 ), ( 4 , 1 ), ( 3 , 1 ) and ∅ ∅ { } { } { } { } { } { } { } { } { } { } ( 2 , 1 ) and the ones of dimension 2 are ( 4, 2 , 3, 1 ) ( 4, 3 , 2, 1 ) and ( 4, 3 , 1, 2 ). { } { } { } { } { } { } { } { } Then,

( , ) < ( 2 , 1 ) < ( 3 , 1 ) < ( 3 , 2 ) < ( 4 , 1 ) < ( 4 , 2 ) < ( 4 , 3 ) ∅ ∅ { } { } { } { } { } { } { } { } { } { } { } { } < ( 4, 3 , 1, 2 ) < ( 4, 3 , 2, 1 ) < ( 4, 2 , 3, 1 ). { } { } { } { } { } { }

60 Therefore, ( , ) is assigned the number 0, ( 2 , 1 ) the number 1, ( 3 , 1 ) the number ∅ ∅ { } { } { } { } 2, ( 3 , 2 ) the 3, ( 4 , 1 ) the 4, ( 4 , 2 ) the 5, ( 4 , 3 ) the 6, ( 4, 3 , 1, 2 ) the 7, { } { } { } { } { } { } { } { } { } { } ( 4, 3 , 2, 1 ) the 8 and ( 4, 2 , 3, 1 ) is assigned the number 9. { } { } { } { } Thus, r = 9 and

4 V S = Γ = π α ,...,α R : ℓ = α H∗ , 0 0 ℓ ∃ 1 4 ∈ i i Xi=1 5 V Γ = π α ,α R,α R∗ : ℓ = α H∗ , 1 ℓ ∃ 3 4 ∈ 5 ∈ i i i=3 X 6 V R R α3α5 Γ2 = πℓ α2,...,α5 ,α6 ∗ : ℓ = α2 + H2∗ + αiHi∗ , ∃ ∈ ∈ α6 i=4 X 8 V R R α2α6 α3α5 Γ3 = πℓ α1,...,α7 ,α8 ∗ : ℓ = α1 + H1∗ + αiHi∗ , ∃ ∈ ∈ α8 − α8 i=4 X V R R α4α5 α4α6 Γ4 = πℓ α2,...,α6 ,α7 ∗ : ℓ = α2 + H2∗ + α3 + H3∗ ∃ ∈ ∈ α7 α7 7

+ αiHi∗ , Xi=5 V R R α2α7 α4α5 Γ5 = πℓ α1,...,α8 ,α9 ∗ : ℓ = α1 + H1∗ ∃ ∈ ∈ α9 − α9 9 α4α8 + α3 H3∗ + αiHi∗ , − α9 Xi=5 V R R α3α7 α4α6 Γ6 = πℓ α1,...,α9 ,α10 ∗ : ℓ = α1 + H1∗ ∃ ∈ ∈ α10 − α10 10 α3α9 α4α8 + α2 + H2∗ + αiHi∗ , α10 − α10 Xi=5 8 V Γ = π α ,α ,α R,α R∗ : ℓ = α H∗ , 7 ℓ ∃ 5 6 7 ∈ 8 ∈ i i Xi=5 9 V Γ = π α ,...,α R,α R∗ : ℓ = α H∗ and 8 ℓ ∃ 5 8 ∈ 9 ∈ i i Xi=5 10 V Γ = π α ,...,α R,α R∗ : ℓ = α H∗ . 9 ℓ ∃ 5 9 ∈ 10 ∈ i i i=5 X

As for n = 3, here, one can also see by a similar example that the situation in which the V polarizations pℓ are discontinuous in ℓ on the set ℓ ′ ′ (g∗/G)2d occurs. { O | O ∈ } Now again, regard the three cases of Proposition 3.1 that were distinguished in the proof of the conditions of Deﬁnition 1.1.

61 1. If one chooses the sequence ( k)k N as O ∈ 1 1 1 := aH∗ + bH∗ + + b H∗ H∗ + H∗ a,b R , Ok 1 k 2 k 4 − k 5 7 ∈ V then πℓ Γ4 for every k. Furthermore, every ℓk k is of the form Ok ∈ ∈O 1 1 1 ℓ = a H∗ + b H∗ + + b H∗ H∗ + H∗ k ℓk 1 k ℓk 2 k ℓk 4 − k 5 7 k k for aℓ ,bℓ R. If aℓ →∞ a˜ and bℓ →∞ b˜, the sequence (ℓk)k N converges to k k ∈ k −→ ℓ k −→ ℓ ∈

ℓ˜= a H∗ + b H∗ + H∗ aH∗ + bH∗ + H∗ a,b R =: ℓ˜ 1 ℓ˜ 4 7 ∈ { 1 4 7 | ∈ } O V and therefore, π Γ4. ℓO ∈ As the orbit does not depend on ℓ˜, it is the only limit point of the sequence ( k)k N. O O ∈ Hence, L(( k)k N)= , where 2=dim( k) = dim( ) k N, O ∈ {O} O O ∀ ∈ and this is an example for the ﬁrst case of Proposition 3.1 regarded in the proof of the conditions of Deﬁnition 1.1.

2. Now, choose the sequence ( k)k N as follows: O ∈ 1 1 := aH∗ + H∗ + bH∗ + H∗ + H∗ a,b R . Ok 1 2 k 3 4 k 6 ∈ V Then, πℓ Γ2 for every k, and every ℓk k is of the form Ok ∈ ∈O 1 1 ℓ = a H∗ + H∗ + b H∗ + H∗ + H∗ k ℓk 1 2 k ℓk 3 4 k 6 k k for aℓ ,bℓ R. If aℓ →∞ a˜ and bℓ →∞ b˜, the sequence (ℓk)k N converges to k k ∈ k −→ ℓ k −→ ℓ ∈

ℓ˜= a H∗ + H∗ + H∗ =: ℓ˜ 1 2 4 Oℓ˜ and πV Γ . Moreover, ℓO 0 ℓ˜ ∈ h i L(( k)k N)= ˜ a˜,b˜ R , where dim( k) = 2, dim( ˜) = 0 O ∈ {Oℓ| ℓ ℓ ∈ } O Oℓ k N ˜ L(( k)k N). ∀ ∈ ∀Oℓ ∈ O ∈ Hence, this is an example for the second case that was regarded in Proposition 3.1. 3. For an example of the third case, choose

8 1 1 := aH∗ + bH∗ + cH∗ + dH∗ + 1+ H∗ + H∗ a,b,c,d R . Ok 1 2 3 4 k 5 k i ∈ i=6 X V Then, πℓ Γ7 for every k, and every ℓk k is of the form Ok ∈ ∈O 8 1 1 ℓ = a H∗ + b H∗ + c H∗ + d H∗ + 1+ H∗ + H∗ k ℓk 1 ℓk 2 ℓk 3 ℓk 4 k 5 k i Xi=6 62 k k k k for a ,b ,c ,d R. If a →∞ a , b →∞ b , c →∞ c and d →∞ d , the ℓk ℓk ℓk ℓk ∈ ℓk −→ ℓ˜ ℓk −→ ℓ˜ ℓk −→ ℓ˜ ℓk −→ ℓ˜ sequence (ℓk)k N converges to ∈

ℓ˜= a H∗ +b H∗ +c H∗ +d H∗ +H∗ aH∗ +bH∗ +c H∗ +d H∗ +H∗ a,b R =: ℓ˜ 1 ℓ˜ 2 ℓ˜ 3 ℓ˜ 4 5 ∈ { 1 2 ℓ˜ 3 ℓ˜ 4 5 | ∈ } Oℓ˜ and thus, πV Γ . Here, ℓO 1 ℓ˜ ∈ h i L(( k)k N)= ˜ c˜,d˜ R O ∈ {Oℓ| ℓ ℓ ∈ } and hence,

0 < 2 = dim( ˜) < 4 = dim( k) k N ˜ L(( k)k N). Oℓ O ∀ ∈ ∀Oℓ ∈ O ∈ Therefore, this represents an example for the third case appearing in the proof of the conditions of Deﬁnition 1.1.

63 4 The C∗-algebra of SL(2, R)

In this section, the C∗-algebra of SL(2, R) will be examined. At the beginning, the Lie group SL(2, R) and some of its subgroups will be introduced and some important definitions and results which are needed in order to prove the conditions listed in Definition 1.1 and hence to determine the C∗-algebra of SL(2, R) will be recalled. Section 4.2 is about the spectrum of SL(2, R) and its topology and in the following two subsections, the above specified conditions will be computed for the group G = SL(2, R). Finally, in Section 4.5, a result about the C∗-algebra of SL(2, R) will be presented and the concrete structure of C∗(G) will be given.

For the group SL(2, R), Condition 1 of Definition 1.1 is obvious by the definition of the sets Γ and S for i 0,...,r which will be given in Section 4.2. Condition 2 follows i i ∈ { } directly as well since SL(2, R) is a connected linear semisimple Lie group. The proof of Condition 3(a) is rather short and straightforward, while the main work of the proof of the conditions of Definition 1.1 consists again in the verification of Condition 3(b). Nevertheless, its proof is less long and less technical than in the case of the two-step nilpotent Lie groups.

4.1 Preliminaries In this section, let G := SL(2, R)= A M(2, R) det A = 1 { ∈ | } and let cos ϕ sin ϕ K := SO(2) = kϕ := − ϕ [0, 2π) sin ϕ cos ϕ ∈ be its maximal compact subgroup. Furthermore, deﬁne the one-dimensional nilpotent subgroup N of G and the one-dimensional abelian subgroup A of G by t 1 x R e 0 R N := x and A := t t . 0 1 ∈ 0 e− ∈

Let

g = sl(2, R)= A M(2, R) tr A = 0 { ∈ | } be the Lie algebra of G.

From the Iwasawa decomposition, G = KAN and thus, for every g G there exist t 0 ∈ κ(g) K, µ N and H(g) a, where a = t R is the Lie algebra of A, such ∈ ∈ ∈ 0 t ∈ − that g = κ(g)eH(g)µ.

Moreover, deﬁne on a the mappings ρ and ν for u C as u ∈ t 0 t 0 ρ := t and ν := ut t R. 0 t u 0 t ∀ ∈ − − Furthermore, let L2(K) := f L2(K) f(k)= f( k) k K and + ∈ | − ∀ ∈ L2(K) := f L2(K) f(k)= f( k) k K − ∈ | − − ∀ ∈ 64 +,u 2 ,u and deﬁne for every u C the representations on +,u := L (K)+ and − on 2 ∈ P HP P −,u := L (K) as HP − −1 ,u (νu+ρ)H(g k) 1 2 ± (g)f(k) := e− f κ g− k g G f L (K) k K. P ∀ ∈ ∀ ∈ ± ∀ ∈ Remark 4.1. +,u The representation , +,u is irreducible if and only if u 2Z+1 and the representation ,u P HP 6∈ − , −,u is irreducible if and only if u 2Z. P HP +,u ,u 6∈ Furthermore, , +,u and − , −,u are unitary for u iR. P HP P HP ∈ For the proof, see [20], Chapter II.5, II.6, VII.1 and VII.2.

For simplicity, within this section, the representations will be identiﬁed with their equivalence classes.

Lemma 4.2. For every function f L2(K) and every g G, ∈ ± ∈ 2 2ρH(g−1k) 1 2 e− f κ g− k dk = f 2 . k kL (K) KZ

The proof can be found in [20], Chapter VII.2 and is part of the proof of the unitarity of the representations ,u for u iR. P± ∈ Lemma 4.3. 1 For every g G and every u R, the operator +, u(g) − is the adjoint operator of ∈ ∈ P − +,u(g) with respect to the usual L2(K)-scalar product. P Proof: −1 (νu+ρ)H(g k) As u and thus also e− are real, with Result (7.4) in [20], Chapter VII.2, which is similar to the equation given in Lemma 4.2, one gets for f ,f L2(K) , 1 2 ∈ + +, u +,u − (g)f , (g)f P 1 P 2 L2(K)

+, u +,u = − (g)f (k) (g)f (k)dk P 1 P 2 KZ

−1 −1 (ν−u+ρ)H(g k) 1 (νu+ρ)H(g k) 1 = e− f1 κ g− k e− f2 κ g− k dk Z K 2ρH(g−1k) 1 = e− f1f2 κ g− k dk Z K

= f1f2 (k)dk Z K = f ,f 2 . h 1 2iL (K)

65 Deﬁnition 4.4 (n-th isotypic component). For a representation (˜π, ) of K deﬁne for every n Z the n-th isotypic component or Hπ˜ ∈ K-type of π˜ as (n) := v π˜(k )v = einϕv ϕ [0, 2π) . Hπ˜ ∈Hπ˜| ϕ ∀ ∈ A representation (˜π, ) of G is called even (respectively odd), if (n)= 0 for all odd n Hπ˜ Hπ˜|K { } (respectively for all even n).

Every irreducible unitary representation of G is even or odd. Furthermore, the algebraic direct sum

(n) Hπ˜ n Z M∈ is dense in . Hπ˜ Remark 4.5. 2 ,u By the deﬁnition of the Hilbert spaces L (K) of ± for u C, it is easy to verify that ± P ∈ +,u is even for every u C and that ,u is odd for every u C. P ∈ P− ∈ By the deﬁnition of the n-th isotypic component, one can remark that

in in +,u (n)= C e− · for all even n Z and −,u (n)= C e− · for all odd n Z. HP · ∈ HP · ∈

In order to prove this, let n Z even, f +,u (n) and deﬁne cf := f(1G) C. Since ∈ ∈HP ∈ 2 +,u inϕ +,u (n) = f˜ L (K)+ (kϕ)f˜ = e f˜ ϕ [0, 2π) HP ∈ | P ∀ ∈ 2 1 inϕ = f˜ L (K) f˜ k− = e f˜ ϕ [0, 2π) , ∈ +| ϕ · ∀ ∈ one gets 1 1 inϕ inϕ f k− = f k− 1 = e f(1 )= e c . ϕ ϕ · G G f inϕ in Therefore, f(kϕ)= e− cf and since one can choose cf C arbitrarily, +,u (n)= C e− ·. ∈ HP · The second statement follows analogously.

Definition 4.6 (Casimir operator). Let , be the non-degenerate symmetric and Ad -invariant bilinear form on g defined by h· ·i X,Y := 2tr(XY ) X,Y g. h i ∀ ∈ Furthermore, choose the basis X ,X ,X of g, where { 1 2 3} 0 1 0 0 1 1 0 X := , X := and X := , 1 0 0 2 1 0 3 2 0 1 − as well as its dual basis X˜ , X˜ , X˜ with respect to , with X˜ := 1 X , X˜ := 1 X and 1 2 3 h· ·i 1 2 2 2 2 1 X˜ := X . 3 3 Then, the Casimir operator C of g with respect to , is defined as h· ·i 3 1 C := X X˜ = (X X + X X )+ X2. i i 2 1 2 2 1 3 Xi=1 66 Here, products are regarded in the universal enveloping algebra (g). U If now π : g gl(V ) is a representation of g, one can define π(C ) as → 3 1 π(C ) := π X π X˜ = π(X )π(X )+ π(X )π(X ) + π(X )2 gl(V ). i i 2 1 2 2 1 3 ∈ i=1 X For every representation π : G GL(V ) of the Lie group G, π(C ) satisfies → π(g) π(C )= π(C ) π(g) g G ◦ ◦ ∀ ∈ on the space V := v V G g π(g)v V is smooth of smooth vectors of V . ∞ ∈ | ∋ 7→ ∈ In general, the definition of the Casimir operator can be made with an arbitrary basis of g and an arbitrary associated dual basis. If one takes an arbitrary non-degenerate symmetric and Ad -invariant bilinear form on g, its definition changes by a scalar multiple.

As 1 (X X + X X )+ X2 = X2 X + X X , one can also write the Casimir operator as 2 1 2 2 1 3 3 − 3 1 2 C = X2 X + X X , 3 − 3 1 2 and for a representation π of g, one then gets

π(C )= π(X )2 π(X )+ π(X )π(X ). 3 − 3 1 2

Definition 4.7 (pn). 2 Denote for n Z by bn(f) the n-th Fourier coefficient of f L (K) which is defined as ∈ ∈ ±

1 inϕ b (f) := f(k )e− dk , n K ϕ ϕ | | KZ and let in pn(f) := b n(f)e− ·. −

One can easily show that for every u C and for every n Z the operator pn is the projection 2 ∈ ∈ ,u from ±,u = L (K) to the n-th isotypic component of the representation ± . HP ± P 4.2 The spectrum of SL(2, R)

4.2.1 Introduction of the operator Ku

Now, an operator Ku which is needed in order to describe the spectrum of G will be introduced using the Knapp-Stein operator.

Deﬁne

C∞(K) := f C∞(K) f(k)= f( k) k K and + ∈ | − ∀ ∈ C∞(K) := f C∞(K) f(k)= f( k) k K − ∈ | − − ∀ ∈ and let J : C∞(K) C∞(K) for u C with Re u> 0 u + → + ∈

67 be the Knapp-Stein intertwining operator, which intertwines the representation +,u with P +, u 0 1 2 the representation − . Furthermore, let w := k π = − and extend f L (K) to G P 2 1 0 ∈ by using the Iwasawa decomposition G g = κ(g)eH(g)µ for κ(g) K, H(g) a and µ N. H(g) (νu+∋ρ)H(g) ∈ ∈ ∈ Then, deﬁning f˜u κ(g)e µ := e− f κ(g) , this operator can be written as J f(k)= f˜ (kµw)dµ f C∞(K) k K. (14) u u ∀ ∈ + ∀ ∈ NZ This integral converges for Re u> 0 (see [20], Chapter VII or [33], Chapter 10.1). The mapping f J f is continuous and the family of operators J u C is holomorphic 7→ u { u| ∈ } in u for Re u > 0 with respect to appropriate topologies (see [20], Chapter VII.7 or [33], Chapter 10.1). For u R , the operator J is self-adjoint with respect to the usual L2(K)-scalar product. ∈ >0 u Moreover, one can extend the function u J meromorphically to C (see [33], Chapter 10.1). 7→ u Then, for every u C for which the operator J is regular, it is an intertwining operator from ∈ u +,u to +, u. P P − Remark 4.8. The operator J commutes with the projections p for all n Z and for every u C for which u n ∈ ∈ Ju is regular.

This can be explained in the following way: Since the operator Ju is a G-intertwining operator, it is a K-intertwining operator as well. Thus, one can show as follows that it leaves +,u (n), HP the above deﬁned n-th isotypic component of the representation +,u, invariant: +,u inϕ P Let f +,u (n), i.e. kϕ f = e f for every ϕ [0, 2π). Because of the intertwining ∈ HP P ∈ property, one gets inϕ +,u +, u e J f = J k f = − k J f. u u ◦P ϕ P ϕ ◦ u C Therefore, Juf +,−u (n) and since +,u(n) = +,u˜ (n) for everyu ˜ , one has ∈ HP HP HP ∈ Juf +,u (n). ∈HP From this, one can easily conclude that p J = J p and the assertion follows. n ◦ u u ◦ n

One can now deduce that the operators Ju have the property Z Ju +,u (n) = cn(u) id +,u (n) for all even n , (15) |HP · |HP ∈ as an equality of meromorphic functions, where c : C C is a meromorphic function for n → every even n Z. This follows from the above remark together with the fact that +,u (n) ∈ HP is one-dimensional.

Using standard integral formulas (see [20], Chapter V.6), one gets by (14) for u = 1

J (f)= c f(k)dk f C∞(K) 1 ∀ ∈ + KZ in for a constant c > 0. Hence, and since +,u (n) = C e− · for every even n Z which was HP · ∈ stated above, one gets = 0 for n = 0 cn(1) 6 (16) (= 0 for n Z 0 even. ∈ \{ } 68 For convenience, an explicit formula for the quotients of the functions cn will now be given. However, this formula will not be used in this work. Instead, further necessary properties of the intertwining operator Ju and the functions cn will be concluded from the irreducibility of the representations ,u. P± Remark 4.9. The quotients of the functions cn can be given by

cn(u) (u 1)(u 3) u ( n 1) n = − − ··· − | |− ( 1) 2 for u C and all even n Z. c0(u) (u + 1)(u + 3) u +( n 1) · − ∈ ∈ ··· | |− This formula can be deduced from a formula in [7], Chapter 23, Appendix 1, which expresses cn(u) in terms of gamma functions, together with the gamma function recurrence formula. Here, one has to remark that the definition of cn(u) in [7] differs by a sign from its definition in this work.

Next, it can be shown that c (u) = 0 for u (0, 1). (17) 0 6 ∈ For this, assume that c (u) = 0 for an element u (0, 1). Then, by (15), the operator J 0 ∈ u has a non-zero kernel. Furthermore, the representation +,u is irreducible on C (K) (see P ∞ + Remark 4.1). As ker(Ju) is a closed invariant subspace with respect to the representation +,u, the kernel of J has to be the whole space C (K) and hence, the operator J is P u ∞ + u identically zero. But there is no u C with Re u> 0 such that the operator J is identically ∈ u zero (see [33], Chapter 10.1). Therefore, c (u) = 0 for every u (0, 1). 0 6 ∈ Thus, one can define 1 J˜u := Ju c0(u) for u C, as a meromorphic function. ∈ Lemma 4.10. J˜u is regular at u = 0 and J˜0 = id. Proof: First, one can observe that on +,u (0), the operator J˜u is always equal to the identity and ˜ HP that in particular J0 +,0 (0) also equals id +,0 (0). |HP |HP Now, it has to be shown that J˜u is regular at u = 0. Since the mapping u J˜ is an operator-valued meromorphic function for u C, one 7→ u ∈ can represent it locally as a Laurent series in 0 with finite principal part and operators as coefficients in the following way:

∞ k J˜u = Lku on C∞(K)+ kX=k0 for operators L going from C (K) to C (K) for k k and where k Z is the smallest k ∞ + ∞ + ≥ 0 0 ∈ number such that L = 0. k0 6 Moreover, this gives k0 Lk = lim u− J˜u. 0 u 0 → 69 Now, for the regularity of J˜ at u = 0 desired above, it has to be shown that k 0. u 0 ≥ So, assume that k0 < 0. ˜ +,u +, u As every Ju is an intertwining operator from to − , Lk0 is an intertwining operator +,0 +,P0 P , from to itself, i.e. it commutes with . Furthermore, Lk0 vanishes on + 0 (0), the P P HP space of all constant functions on K, because J˜u equals the identity on +,u (0) and thus HP does not have a pole there. +,0 +,0 , Since Lk0 commutes with , it vanishes on (g) + 0 (0) for every g G. In addition, P P HP 2 ∈ as Remark 4.1 stays valid if one replaces the space +,0 = L (K)+ by C∞(K)+, the repre- +,0 HP sentation is irreducible on the space C∞(K)+. Furthermore, for every 0 = ξ +,0 (0), P 6 ∈HP the subspace span +,0(g)ξ g G is G-invariant and hence, by the irreducibility, P | ∈ C (K) span +,0(g)ξ g G . Thus, L vanishes on the whole space C (K) , which ∞ + ⊂ P | ∈ k0 ∞ + is a contradiction to the choice of k . 0 Therefore, one gets k 0, which means that the mapping u J˜ does not have any poles 0 ≥ 7→ u in u = 0. Hence, J˜u is regular at u = 0 on C∞(K)+. As above, as a limit of intertwining operators, J˜0 = lim J˜u is an intertwining operator, u 0 which intertwines the irreducible unitary representation →+,0 with itself (see Remark 4.1 for +,0 P 2 the irreducibility and the unitarity of , again replacing the space +,0 = L (K)+ by P HP C∞(K)+). Thus, by Schur’s lemma (see Theorem 2.6), J˜0 is a scalar multiple of the identity. Since it equals the identity on +,0 (0), one gets J˜0 = id. HP

From (17) and Lemma 4.10, one can conclude that

c (u) n > 0 for u (0, 1) and for all even n Z : (18) c0(u) ∈ ∈

Using the same argumentation for the operator J˜u as in the proof of (17), one also gets that cn(u) = 0 for every u (0, 1). c0(u) 6 ∈ Furthermore, from Lemma 4.10, one can deduce that cn(0) = 1, and with the continuity of c0(0) cn on (0, 1), one gets cn(u) > 0 for all u (0, 1) and all even n Z and (18) is shown. c0 c0(u) ∈ ∈

Now, deﬁne a scalar product on C∞(K)+ as follows:

f ,f := J˜ f ,f . h 1 2iu u 1 2 L2(K) Lemma 4.11. , is an invariant positive deﬁnite scalar product for u (0, 1). h· ·iu ∈ Proof: , u is hermitian: h· ·i 2 By the deﬁnition of J˜u and as Ju is self-adjoint with respect to the usual L (K)-scalar product for u (0, 1), this is straightforward. ∈ , u is invariant: h· ·i 1 By Lemma 4.3, for every g G, the operator +,u(g) − is the adjoint operator of +, u(g) ∈ P P − with respect to the usual L2(K)-scalar product. Now, let f ,f C (K) . Then, as J˜ 1 2 ∈ ∞ + u intertwines +,u and +, u, one gets for every g G, P P − ∈

70 +,u(g)f , +,u(g)f = J˜ +,u(g)f , +,u(g)f P 1 P 2 u u ◦P 1 P 2 L2(K) +, u +,u = − (g) J˜ f , (g)f P ◦ u 1 P 2 L2(K) = J˜ f ,f = f ,f . u 1 2 L2(K) h 1 2iu

, is positive deﬁnite: h· ·iu cn(u) Z As c (u) > 0 by (18), one gets for every n and f +,u (n) by (15) above, 0 ∈ ∈HP

cn(u) cn(u) ˜ 2 f,f u = Juf,f L2(K) = id +,u (n)f,f = f,f L (K) 0 and h i c0(u) |HP L2(K) c0(u) h i ≥ D E

f,f = 0 f,f 2 = 0 f = 0. h iu ⇐⇒ h iL (K) ⇐⇒ Now, it will be shown that the algebraic direct sum

+,u (n) is orthogonal with respect to , u. (19) HP h· ·i n Z M∈ As +,u is an even representation, only K-types for even n Z appear. So, let n ,n Z P ∈ 1 2 ∈ even with n1 = n2 and let f1 +,u (n1) and f2 +,u (n2), i.e. 6 ∈HP ∈HP +,u(k )f = ein1ϕf and +,u(k )f = ein2ϕf ϕ [0, 2π). P ϕ 1 1 P ϕ 2 2 ∀ ∈ Then, because of the invariance of , shown above, one gets for all ϕ [0, 2π), h· ·iu ∈ f ,f = +,u(k )f , +,u(k )f = ein1ϕf ,ein2ϕf h 1 2iu P ϕ 1 P ϕ 2 u 1 2 u i(n1 n2)ϕ = e − f1,f2 u. h i As this is true for all ϕ [0, 2π) and as n = n , one gets f ,f = 0 and (19) is shown. ∈ 1 6 2 h 1 2iu Now, since the direct sum +,u (n) is orthogonal with respect to , u and dense in n Z HP h· ·i 2 ∈ +,u = L (K)+, the aboveL observations hold for all f C∞(K)+. HP ∈

The completion of C (K) with respect to the scalar product , yields a Hilbert ∞ + h· ·iu space . Considering the restriction of the representation +,u to C (K) and then Hu P ∞ + continuously extending it to the space , one gets a unitary representation which will be Hu denoted by +,u as well. G acts on by this unitary representation +,u. P Hu P

cn(u) Furthermore, let dn(u) := > 0 for u (0, 1). Next, a unitary bijection c0(u) ∈ q K : L2(K) u (0, 1) u Hu → + ∀ ∈ shall be deﬁned. On the n-th isotypic component in , deﬁne K by Hu u Z Ku +,u (n) := dn(u) id +,u (n) for all even n . |HP · |HP ∈

71 Then, one can extend this definition to finite sums of K-types. This operator also is self-adjoint 2 with respect to the usual L (K)-scalar product and for finite sums of K-types f1 and f2

K f ,K f = K2f ,f = J˜ f ,f = f ,f . u 1 u 2 L2(K) u 1 2 L2(K) u 1 2 L2(K) h 1 2iu

From this follows directly that it is unitary (if one regards the space u equipped with , u 2 H h· ·i and L (K)+ with , L2(K)) and hence, because of the density of +,u (n) in u, one h· ·i n Z HP H ∈ can extend Ku continuously on the whole space u. L H Moreover, Ku is continuous in u and one also has the property lim Ku = id. u 0 By its deﬁnition, the operator K commutes with the projections→p for all n Z as well. u n ∈ Now, by [20], Chapter XIV.4, one gets the identity

J u Ju = γ(u) id, (20) − ◦ · where u γ(u) is a meromorphic function. One can also obtain this equation by observing 7→ +,u that J u Ju intertwines the representation , which is irreducible for almost all u C − ◦ P ∈ (see Remark 4.1), with itself, and by using a version of Schur’s lemma for (g,K)-modules. By restricting the above equation to the n-th isotypic component, one thus gets the relation

c (u)c ( u)= γ(u) for all even n Z (21) n n − ∈ as meromorphic functions.

Next, another scalar product on the space C (K) := f C (K) p (f) = 0 n 0 ∞ ++ ∈ ∞ +| n ∀ ≤ is needed. For this, deﬁne ˜ 1 J(u) := Ju c2(u) C ˜ for u as a meromorphic family of operators. Then, on +,u (2) the operator J(u) is equal ∈ C ˜ HP to the identity for every u and in particular J(1) +,1 (2) equals id +,1 (2). ∈ |HP |HP Moreover, deﬁne the space C∞(K)+ := f C∞(K)+ pn(f) = 0 n 0 . +,1 − ∈ | ∀ ≥ The representation is irreducible on the spaces C∞(K)++ and C∞(K)+ (see [32], P − Chapter 5.6).

Lemma 4.12.

˜ ˜ ˜ ˜ (a) J( u) J(u) = J(u) J( u) = id as a meromorphic family. − ◦ ◦ − (b) J˜ is regular at u = 1 and (u) − ˜ ˜ ker J( 1) C∞(K)++ = ker J( 1) C∞(K)+ = 0 . − ∩ − ∩ − { } ˜ +, 1 +,1 Furthermore, J( 1) is an intertwining operator of − with . − P P

˜ ∞ ∞ (c) J(u) C (K)++ C (K)+− is regular at u = 1. | ⊕

72 Proof: For all n Z and all u C, ∈ ∈ (20) γ(u) (21) J˜ J˜ = id = id ( u) (u) (n) P+,u (n) P+,u (n) − ◦ P+,u c (u)c ( u) · |H |H |H 2 2 − and thus, ˜ ˜ ˜ ˜ J( u) J(u) = J(u) J( u) = id − ◦ ◦ − for u C as a meromorphic family and (a) follows. ∈ Because the mapping u J˜ is an operator-valued meromorphic function for u C, one 7→ (u) ∈ can represent it locally as a Laurent series in 1 with ﬁnite principal part and operators as − coeﬃcients in the following way:

˜ ∞ k J(u) = Lk(u + 1) on C∞(K)+ kX=k0 for operators L going from C (K) to C (K) for k k and where k Z is the smallest k ∞ + ∞ + ≥ 0 0 ∈ number such that L = 0. k0 6 Then, one gets J k0 ˜ k0 u Lk0 = lim( u + 1)− J( u) = lim( u + 1)− − (22) u 1 − − u 1 − c ( u) → → 2 − and thus, as a limit of intertwining operators, L is an intertwining operator of +, 1 with k0 P − +,1. Now, it shall ﬁrst be shown that P

ker(Lk ) C∞(K)++ = ker(Lk ) C∞(K)+ = 0 . (23) 0 ∩ 0 ∩ − { }

Since +,1 (n)= +,−1 (n), HP HP c ( u) L = a id for a := lim n ( u + 1) k0 . k0 +,1 (n) n +,1 (n) n − − |HP · |HP u 1 c ( u) − → 2 −

As C∞(K)++ = +,1 (n) and C∞(K)+ = +,1 (n), for (23) it has to be shown − n Z>0 HP n Z<0 HP n∈L even n∈L even that a = 0 for all even n Z 0 . n 6 ∈ \{ } By (21), one has c (u)c ( u) γ(u) n n − = , c ( u) c ( u) 2 − 2 − i.e. the left hand side does not depend on n. So, the order of the pole for the limit for u 1 → has to be the same for every n Z. But as c (1) = 0 for n = 0 and c (1) = 0 by (16), c ( 1) ∈ n 6 0 6 0 − has to be 0 in order to get the same order of the pole for n = 0 and n = 0. It follows that 6 a = 0. As k was chosen such that L = 0, there has to exist 0 = n Z with a = 0. 0 0 k0 6 6 ∈ n 6 Furthermore, from (14) one can conclude that J f = J f for all u C with Re u > 0. u u ∈ By extending meromorphically, this holds for all u C and, by regarding the n-th isotypic ∈ component, one can deduce that cn(u)= c n(u). Hence, one gets for every n Z and every − ∈

73 u R that cn(u) = c n(u). This means that there also exists an integer n1 > 0 such that ∈ − an1 = 0 = a n1 . Therefore, 6 6 − L = a id = 0 = a id = L k0 +,1 (n1) n1 +,1 (n1) n1 +,1 ( n1) k0 +,1 ( n1) |HP · |HP 6 6 − · |HP − |HP − and thus,

Im L +,1 (n ) C (K) and k0 +,1 (n1) 1 ∞ ++ |HP ⊂HP ⊂ Im L +,1 ( n ) C (K) . k0 +,1 ( n1) 1 ∞ + |HP − ⊂HP − ⊂ − +,1 So, in particular Im (Lk ) C∞(K)++ = 0 = Im(Lk ) C∞(K)+ . Since is irreducible 0 ∩ 6 { } 6 0 ∩ − P on C∞(K)++ and on C∞(K)+ , one gets −

Im(Lk0 ) C∞(K)++ = C∞(K)++ and Im(Lk0 ) C∞(K)+ = C∞(K)+ , ∩ ∩ − − where the completions are regarded in the spaces C∞(K)++ and C∞(K)+ , respectively. − Hence, +,1 (n) Im(L ) for all even n = 0. Since L = a id , one k0 k0 +,1 (n) n +,1 (n) HP ⊂ 6 |HP · |HP therefore gets a = 0 for every even n = 0 and thus, (23) follows. n 6 6 One can now conclude that k0 = 0: Because k0 Lk0 (2) = lim( u + 1)− id +,1 (2) P+,1 u 1 − · |HP |H → does not have any poles, k > 0, and as it does not have any zeros, k < 0. 0 6 0 6 Hence, ˜ ˜ Lk0 = lim J( u) = J( 1) u 1 − − → +, 1 +,1 and thus, (b) follows by (23) and as Lk0 intertwines − with . ˜ P ˜ P ˜ ˜ ˜ Now, by (b), J(u) is regular at u = 1. As by (a), one has J( 1) J(1) = J(1) J( 1) = id ˜ − − ◦ ◦ − and since by (b) the operator J( 1) is nowhere equal to 0 on C∞(K)++ C∞(K)+ , the ˜ − ⊕ − operator J(u) has to be regular on C∞(K)++ C∞(K)+ at u = 1 as well and (c) is shown. ⊕ −

˜ J(1) is not identically 0 on the space C∞(K)++, because it equals the identity on ˜ +,1 (2). Thus, regard the operator J(1) on the space C∞(K)++. This operator is injec- HP ˜ tive, since for every f C∞(K)++ with J(1)(f) = 0, one gets by Lemma 4.12(a) that ˜ ˜ ∈ 0= J( 1) J(1)(f)= id(f)= f. − ◦ Now, deﬁne for all functions f ,f C (K) , 1 2 ∈ ∞ ++ f ,f := J˜ f ,f . h 1 2i(1) (1) 1 2 L2(K) ˜ ˜ ˜ Furthermore, choose f1 in such a way that J( 1) f1 = f1. Then, by Lemma 4.12, − f ,f = f˜ ,f (24) h 1 2i(1) 1 2 L2(K) since the scalar product does not depend on the choice of f˜1: ˜ ˜ ˜ It is possible to add an element fker ker J( 1) to the function f1. But by the above ˜ ∈ − ˜ Lemma 4.12(b), one has ker J( 1) C∞(K)++ = ker J( 1) C∞(K)+ = 0 and − ∩ − ∩ − { } therefore, f˜ f C (K) p (f) = 0 n = 0 . Thus, the function f˜ is orthogonal to ker ∈ ∈ ∞ + | n ∀ 6 ker f C (K) = f C (K) p (f) = 0 n 0 and the scalar product stays the same. 2 ∈ ∞ ++ ∈ ∞ +| n ∀ ≤ 74 Lemma 4.13. , is an invariant positive deﬁnite scalar product. h· ·i(1) Proof: , is hermitian: h· ·i(1) Like above for , , this is straightforward. h· ·iu , (1) is invariant: h· ·i ˜ ˜ ˜ Let f1,f2 C∞(K)++ and choose f1 such that J( 1) f1 = f1. ˜∈ +, 1 +,1 − Then, as J( 1) intertwines − and by Lemma 4.12(b), for every g G, − P P ∈ ˜ +, 1 ˜ +,1 ˜ ˜ +,1 J( 1) − (g)f1 = (g) J( 1) f1 = (g)f1. − P P ◦ − P Hence, one can choose +^,1 +, 1 (g)f := − (g)f˜ . P 1 P 1 1 Since by Lemma 4.3 for all g G the operator +,1(g) − is the adjoint operator of +, 1(g) ∈ P P − with respect to the usual L2(K)-scalar product, one gets for every g G, ∈

+,1 +,1 +^,1 +,1 (g)f1, (g)f2 = (g)f1, (g)f2 P P (1) P P L2(K) D +, 1 +,1 E = − (g)f˜1, (g)f2 P P L2(K) D˜ E = f1,f2 L2(K) = f1,f2 (1). h i , is positive deﬁnite: h· ·i(1) cn(u) > 0 for every u (0, 1), since c2(u) ∈ c (u) c (u) c (u) n = n 0 > 0 c2(u) c0(u) · c2(u)

by (18) of the beginning of this subsection. Hence, its limit lim cn(u) is larger or equal to 0 u 1 c2(u) as well. Therefore, similarly as above for , , for every n →N := N 0 and for every h· ·iu ∈ ∗ \{ } f1 +,1 (n), one gets ∈HP ˜ cn(u) f1,f1 (1) = J(1)f1,f1 2 = lim f1,f1 L2(K) 0 h i L (K) u 1 c (u) h i ≥ → 2

from the argument above and since +,1 (n) = +,u (n) for every u (0, 1). Moreover, ˜ HP HP ∈ because of the injectivity of J(1)

˜ cn(u) f1 = 0 J(1)f1 = 0 lim f1 = 0, ⇐⇒ ⇐⇒ u 1 c (u) → 2 which means that lim cn(u) > 0. Hence, u 1 c2(u) →

f ,f = 0 f ,f 2 = 0 f = 0. h 1 1i(1) ⇐⇒ h 1 1iL (K) ⇐⇒ 1

75 Like in (19), one can prove that the algebraic direct sum +,1 (n) is orthogonal with n N∗ HP respect to , . Due to this and as the direct sum is denseL∈ in C (K) , the positive h· ·i(1) ∞ ++ deﬁniteness everywhere is shown.

The completion of the space C∞(K)++ with respect to this scalar product gives a Hilbert space which will be called . H(1)

The same procedure can be accomplished for C∞(K)+ = f C∞(K)+ pn(f) = 0 n 0 : − ∈ | ∀ ≥ Here, define the operator J˜ as [u] ˜ 1 J[u] := Ju c 2(u) − for u (0, 1) as a meromorphic family of operators. Like above, it can be shown that J˜ is ∈ [u] regular at u = 1 on C∞(K)+ and is not identically 0 there. − Thus, define again for all f1,f2 C∞(K)+ , ∈ − f ,f := J˜ f ,f . h 1 2i[1] [1] 1 2 L2(K) This is another invariant positive definite scalar product. The completion of the space C∞(K)+ with respect to this scalar product gives a Hilbert space called [1]. − H 4.2.2 Description of the irreducible unitary representations Now, some convenient realizations for the spectrum of SL(2, R) shall be provided.

The spectrum G of G = SL(2, R) consists of the following representations:

1. The principalb series representations: (a) +,iv for v [0, ). P ∈ ∞ (b) ,iv for v (0, ). P− ∈ ∞ See Section 4.1 above for the deﬁnitions.

2. The complementary series representations u for u (0, 1): C ∈ The Hilbert space u is deﬁned by HC 2 u := L (K)+ HC and the action is given by

u +,u 1 (g) := K (g) K− C u ◦P ◦ u for all g G, where here again +,u(g) is meant in the following way: One considers ∈ P the restriction of +,u(g) to C (K) and then continuously extends it to the space P ∞ + Hu (see the deﬁnition of in Section 4.2.1). Hu

76 3. The discrete series representations: (a) + for odd m N : Dm ∈ ∗ (i) +: D1 The Hilbert space + is given by HD1 + := (1) HD1 H defined in Section 4.2.1. The action is given by + := +,1. D1 P Here again, as well as in all the definitions in this subsection, the representation +,u for the different values u C is meant as described above: One P ∈ restricts it to the respective subspace of L2(K) and then continuously extends it to the respective Hilbert space. + N (ii) m for odd m 3: D ∈ ≥ + N As a Hilbert space + for m for odd m 3, one can take the completion H m D ∈ ≥ of the space D f C∞(K) p (f) = 0 n m 1 ∈ +| n ∀ ≤ − with respect to an appropriate scalar product, and as the action, one can take + := +,m. Dm P With this realization, the Hilbert spaces + for odd m N 3 depend on H m ∈ ≥ m. But as all of them are infinite-dimensionalD and separable, one can identify them if one conjugates the respective G-action. So, fix an infinite-dimensional separable Hilbert space . The G-action is not needed for the determination HD of C∗(G). (b) for odd m N : Dm− ∈ ∗ (i) −: D1 The Hilbert space − is given by HD1 − := [1] HD1 H defined in Section 4.2.1 above and the action is given by

+,1 − := . D1 P N (ii) m− for odd m 3: D ∈ +≥ N Similarly as for m, as a Hilbert space − for m− for odd m 3, one D H m D ∈ ≥ can take the completion of the space D

f C∞(K) p (f) = 0 n m + 1 ∈ +| n ∀ ≥− with respect to an appropriate scalar product, and as the action, one can take

+,m − := . Dm P Again, the Hilbert spaces depend on m. One identifies them and takes the common infinite-dimensional separable Hilbert space fixed in (a)(ii). HD Again, the G-action is not needed for the determination of C∗(G).

77 + N (c) m for even m ∗: D ∈ + As a Hilbert space + for for even m N∗, one can take the completion of H m Dm ∈ the space D f C∞(K) pn(f) = 0 n m 1 ∈ −| ∀ ≤ − with respect to an appropriate scalar product, and as the action, one can take

+ ,m := − . Dm P Again, the Hilbert spaces are identiﬁed and one takes the common inﬁnite- dimensional separable Hilbert space , as in (a)(ii). HD (d) for even m N : Dm− ∈ ∗ As a Hilbert space − for − for even m N∗, one can take the completion of H m Dm ∈ the space D f C∞(K) pn(f) = 0 n m + 1 ∈ −| ∀ ≥− with respect to an appropriate scalar product, and as the action, one can take

,m − := − . Dm P Here again, the Hilbert spaces are identiﬁed and one takes the common Hilbert space , as in (a)(ii). HD 4. The limits of the discrete series:

(a) : D+ The Hilbert space + is defined by HD 2 + := f L (K) pn(f) = 0 n 0 HD ∈ −| ∀ ≤ and the action is given by ,0 := − . D+ P (b) : D− The Hilbert space − is defined by HD 2 − := f L (K) pn(f) = 0 n 0 HD ∈ −| ∀ ≥ and the action is given by ,0 := − . D− P 5. The trivial representation 1: C F Its Hilbert space 1 = will be identified with the space of constant functions HF f L2(K) p (f) = 0 n = 0 . ∈ +| n ∀ 6 Here, the action is given by (g) := id F1 for all g G. One also has ∈ +, 1 = − , F1 P

as ν 1 + ρ = 0 and every f 1 is a constant function. − ∈HF

78 A discussion of all irreducible representations of SL(2, R) without scalar products can be found in [32], Chapter 5.6. But as the scalar products in this thesis are shown to be unitary and invariant and as such scalar products are uniquely determined, they are the correct ones. In [20], Chapter II.5, a diﬀerent realization of the discrete series representations that is not used in this work can be found. An alternative description of all irreducible unitary representations can be found in [22], Chapter 6.6.

Remark 4.14.

−,0 = + − . HP ∼ HD ⊕HD Remark 4.15. By the definition of the operator Ku by means of its value on the space +,u , one can easily HP verify that pn also projects u to its n-th isotypic component u (n) for every n Z. HC HC ∈ Furthermore, for every irreducible unitary representation π of G, the operator pn leaves the above defined Hilbert space invariant, since all of the Hilbert spaces are completions of the Hπ space C (K) fulfilling p -cancellation properties for certain n Z. ∞ n ∈ Hence, for every n Z and for every irreducible unitary representation π of G, the operator ∈ p is the projection of to the n-th isotypic component (n). n Hπ Hπ 4.2.3 The SL(2, R)-representations applied to the Casimir operator In order to be able to describe the topology on G, the above listed representations applied to the Casimir operator will now be computed. b Lemma 4.16. Applied to the Casimir operator normalized as in Section 4.1, the representations ,u for P± u C give the following: ∈ ,u 1 2 ± (C )= u 1 id. P 4 − · Proof: Following the discussion about the Casimir operator in Section 4.1, one can write ,u(C ) as P± ,u ,u 2 ,u ,u ,u ± (C )= ± (X ) ± (X )+ ± (X ) ± (X ). P P 3 −P 3 P 1 P 2 ,u First regard ± (X3). For this, let f C∞(K) and let k = id for the moment. Then, P ∈ ±

,u d ,u ± (X3)(f)(id) = ± exp(tX3) (f)(id), P dt t=0P and, as tX a, this gives 3 ∈

d (νu+ρ)( tX3) (νu+ρ)(tX3) e− − f(id) = (νu + ρ)(X3)e f(id) dt t=0 t=0 = (νu + ρ)(X3)f(id) 1 = (u + 1)f(id). 2

,u 2 1 2 Thus, one can deduce that ± (X3) (f)(id)= 4 (u + 1) f(id). ,u P Next, regard ± (X1). Again, let f C∞(K) and k = id. Then, since exp(tX1) N for P ∈ ± ∈ every t R, one gets ∈

79 ,u d ,u ± (X1)(f)(id) = ± exp(tX1) (f)(id) P dt t=0P d = f(id) = 0. dt t=0

,u ,u Hence, ± (X1) ± (X2)(f)(id) = 0 as well. P P Therefore,

,u 1 2 1 1 2 ± (C )(f)(id) = (u + 1) f(id) (u + 1)f(id) = u 1 f(id). P 4 − 2 4 − Now, to show this equality for general k K, one uses the above stated fact that ∈ ,u ,u ,u ,u ± (g) ± (C )= ± (C ) ± (g) g G. P ◦P P ◦P ∀ ∈

So, let k K. Then, for every f C∞(K) , on the one hand, ∈ ∈ ± ,u ,u 1 ,u 1 ,u ± (C ) ± (k− )(f)(id) = ± k− ± (C )(f)(id) P ◦P P ,u ◦P = ± (C )(f)(k). P On the other hand,

,u ,u 1 1 2 ,u 1 ± (C ) ± k− (f)(id) = u 1 ± k− (f)(id) P ◦P 4 − P 1 = u2 1f(k). 4 − Thus, ,u 1 2 ± (C )(f)(k)= u 1 f(k), P 4 − as desired.

From Lemma 4.16, one can deduce for the irreducible unitary representations of SL(2, R) listed above:

1. ,iv(C )= 1 v2 1 id v [0, ), P± 4 − − · ∀ ∈ ∞ 2. u(C )= K 1 u2 1 id K 1 = 1 u2 1 id u (0, 1), C u ◦ 4 − · ◦ u− 4 − · ∀ ∈ 3. (C )= 1 m2 1 id m N , Dm± 4 − · ∀ ∈ ∗ 4. (C )= 1 (0 1) id = 1 id and D± 4 − · − 4 ·

5. (C )= 1 ( 1)2 1 id = 0. F1 4 − − ·

80 \ 4.2.4 The topology on SL(2, R) With the help of the computations above, it is now possible to describe the topology on G.

Proposition 4.17. b The topology on G can be characterized in the following way:

1. For all sequences (vj)j N and all v in [0, ), b ∈ ∞ j j +,ivj →∞ +,iv v →∞ v. P −→ P ⇐⇒ j −→

2. For all sequences (vj)j N and all v in (0, ), ∈ ∞

,ivj j ,iv j − →∞ − v →∞ v. P −→ P ⇐⇒ j −→

For all sequences (vj)j N in [0, ), ∈ ∞

,ivj j j − →∞ +, vj →∞ 0. P −→ D D− ⇐⇒ −→ 3. For all sequences (uj)j N and all u in (0, 1), ∈ j j uj →∞ u u →∞ u. C −→ C ⇐⇒ j −→

For all sequences (uj)j N in (0, 1), ∈ j j uj →∞ +,0 u →∞ 0. C −→ P ⇐⇒ j −→

For all sequences (uj)j N in (0, 1), ∈

uj j + j →∞ , −, u →∞ 1. C −→ D1 D1 F1 ⇐⇒ j −→ All other sequences (πj)j N can only converge if they fulﬁll one of the following conditions: ∈ (a) They are constant for large j N. In that case, they have one single limit point, namely ∈ the value they take for large j.

(b) For large j N, they are one of the sequences listed above. Then, they converge in the ∈ way described above.

(c) For large j N, the sequence consists only of members +,ivj and uj . Then, it converges ∈ j j P C to +,0 if and only if v →∞ 0 as well as u →∞ 0 (compare 1. for v = 0 and the second P j −→ j −→ part of 3. above).

N uj + (d) For large j , the sequence consists only of members and 1 . Then, it converges ∈ j C D to + if and only if u →∞ 1 (compare the third part of 3. above). D1 j −→ N uj (e) For large j , the sequence consists only of members and 1−. Then, it converges ∈ j C D to − if and only if u →∞ 1 (compare the third part of 3. above). D1 j −→

81 uj (f) For large j N, the sequence consists only of members and 1. Then, it converges to ∈ j C F if and only if u →∞ 1 (compare the third part of 3. above). F1 j −→ ,ivj (g) For large j N, the sequence consists only of members − and +. Then, it converges ∈ j P D to if and only if v →∞ 0 (compare the second part of 2. above). D+ j −→ ,ivj (h) For large j N, the sequence consists only of members − and . Then, it converges ∈ j P D− to if and only if vj →∞ 0 (compare the second part of 2. above). D− −→ Proof: If a sequence of representations (πj)j N converges to a representation π, the sequence C C ∈ πj( ) j N has to converge to π( ) as well. By the observations of Section 4.2.3, the left hand side∈ thus implies the right hand side in all cases. Now, for the other implication, it will ﬁrst be shown that for a sequence (uj)j N in C, u C ∈ ∈ and for every compact set K˜ G, ⊂ j j →∞ ,uj ,u →∞ uj u = sup ± (g) ± (g) op 0. (25) −→ ⇒ g K˜ P −P −→ ∈ One has

,uj ,u 2 sup ± (g) ± (g) op g K˜ P −P ∈ ,uj ,u 2 = sup sup ± (g)(f)(k) ± (g)(f)(k) dk 2 g K˜ f L (K)± P −P ∈ ∈ KZ f 2=1 k k −1 −1 2 −1 2 νu H(g k) νuH(g k) 2ρH(g k) 1 = sup sup e− j e− e− f κ g− k dk. 2 g K˜ f L (K)± − ∈ ∈ KZ f 2=1 k k For g K˜ and k K, g 1k is also contained in a compact set and therefore, H g 1k is ∈ ∈ − − contained in a compact set as well. As 1 1 h(g− k) 0 1 R H g− k = 1 for h(g− k) , 0 h(g− k) ∈ − there is a compact set I R such that h(g 1k) I for all g K˜ and all k K. Hence, ⊂ − ∈ ∈ ∈ −1 −1 2 −1 2 νu H(g k) νuH(g k) 2ρH(g k) 1 sup sup e− j e− e− f κ g− k dk 2 g K˜ f L (K)± − ∈ ∈ KZ f 2=1 k k 2 −1 2 uj x ux 2ρH(g k) 1 sup e− e− sup sup e− f κ g− k dk 2 ≤ x I − g K˜ f L (K)± ∈ ∈ ∈ KZ f 2=1 k k 2 Lemma 4.2 uj x ux 2 = sup e− e− sup f L2(K) 2 x I − f L (K)± k k ∈ ∈ f 2=1 k k 2 uj x ux j = sup e− e− →∞ 0. x I − −→ ∈

Therefore, (25) follows.

82 From this, one can easily deduce that for a sequence (uj)j N in C and u C, ∈ ∈

j ,uj j ,u u →∞ u = ± →∞ ± (26) j −→ ⇒ P −→ P in the sense of convergence of matrix coeﬃcients described in Theorem 2.8. Hence, one has to N show that for some f ±,u , there exists for every j a function fj ±,uj such that ∈HP ∈ ∈HP

,uj j ,u ± ( )fj,fj →∞ ± ( )f,f ±,u ±,u P · HP j −→ P · HP uniformly on compacta. 2 2 N Let f ±,u = L (K) and choose fj := f L (K) = ±,uj for all j . Moreover, ∈ HP ± ∈ ± H ∈ let K˜ G be compact. Then, P ⊂

,uj ,u sup ± (g)fj,fj L2(K) ± (g)f,f L2(K) g K˜ P − P ∈ ,uj ,u = sup ± (g) ± (g) f,f 2 g K˜ P −P L (K) ∈ D E j ,uj ,u 2 →∞ sup ± (g) ± (g) op f L2(K) 0, ≤ g K˜ P −P k k −→ ∈ by (25). Therefore, Claim (26) is shown, which directly yields the inverse implications of 1. and 2. (using that + , − −,iv for all v (0, )). HD HD ⊂HP ∈ ∞ Next, it will be shown that

+,u 1 +,u K (g) K− f,f = (g)f,f (27) u ◦P ◦ u L2(K) P L2(K) forn ˜ Z, u (0, 1), g G and f +,u (˜n). ∈ ∈ ∈ ∈HP So, letn ˜ Z, u (0, 1), g G and f +,u (˜n). Then, ∈ ∈ ∈ ∈HP 1 1 Kuf = dn˜(u)f and Ku− f = f. dn˜(u)

2 Since Ku is self-adjoint with respect to the usual L (K)-scalar product, one gets

+,u 1 +,u 1 K (g) K− f,f = (g) K− f,K f u ◦P ◦ u L2(K) P ◦ u u ◦ L2(K) dn˜(u) +,u +,u = (g)f,f L2(K) = (g)f,f L2(K) dn˜(u) P P

and (27) follows.

For the inverse implications of 3., one uses Equality (27) to be able to express the +,u matrix coeﬃcients π(g)f,f for a representation π G as (g)f,f 2 . Then, by π L (K) H ∈ P (26) for fj = f, one gets the convergences needed: b

So ﬁrst, it has to be shown that for a sequence (uj)j N in (0, 1) and u (0, 1), ∈ ∈ j j u →∞ u = uj →∞ u in G. (28) j −→ ⇒ C −→ C

83 b N For this, let f +,u (0) = +,uj (0). Furthermore, choose fj := f for all j . Then, ∈ HP H ∈ letting K˜ G compact, by (27)P one gets ⊂

uj u sup (g)fj,fj L2(K) (g)f,f L2(K) g K˜ C − C ∈

= sup K +,uj (g) K 1f,f K +,u(g) K 1f,f uj u−j L2(K) u u− L2(K) g K˜ ◦P ◦ − ◦P ◦ ∈ j +,uj +,u →∞ = sup (g)f,f L2(K) (g)f,f L2(K) 0 g K˜ P − P −→ ∈

by (26). Therefore, (28) follows.

The same reasoning is valid if one regards u = 0 and +,0 instead of u. Thus, P C j j u →∞ 0 = uj →∞ +,0 in G. j −→ ⇒ C −→ P

Now, it still needs to be proved that for a sequence (uj)j N bin (0, 1), ∈

j uj j + u →∞ 1 = →∞ , −, in G. (29) j −→ ⇒ C −→ D1 D1 F1 + First, the convergence to 1 will be regarded: b D ˜ For this, let f +,1 (2). Then, f + and J(1)f = f. By Lemma 4.12(a) and with the ∈ HP ∈ H 1 ˜ ˜ ˜ ˜ D ˜ notation used in (24), f = J(1) J( 1)f = J(1)f = f. Thus, by (24), ◦ − +(g)f,f = +,1(g)f,f = f, +,1(g)f = f,˜ +,1(g)f D1 (1) P (1) P (1) P L2(K) = f, +,1(g)f = +,1(g)f,f . P L2(K) P L2(K)

Now, choose fj := f for all j N. Hence, fj = f uj (2). ∈ ∈HC Using (27) and (26) again, one gets for compact K˜ G, ⊂

uj + sup (g)fj,fj L2(K) 1 (g)f,f (1) g K˜ C − D ∈ j +,uj +,1 →∞ = sup (g)f,f L2(K) (g)f,f L2(K) 0. g K˜ P − P −→ ∈

j uj →∞ The proof of the convergence of j N to 1− for uj 1 is similar. One only has to C ∈ D −→ choose f +,1 ( 2). ∈HP − For 1, let f +,1 (0). Then, f is a constant function and therefore, with Lemma 4.2, F ∈HP 2 +,1 2ρH(g−1k) 1 2 (g)f,f = e− f κ g− k dk = f 2 P L2(K) k kL (K) KZ 2 +, 1 = f(k) dk = − (g)f,f = (g)f,f . | | P L2(K) F1 L2(K) Z K

84 Again, choose f := f for all j N. As above, letting K˜ G compact, with (27) and (26) j ∈ ⊂ one gets

uj sup (g)fj,fj L2(K) 1(g)f,f L2(K) g K˜ C − F ∈ j +,uj +,1 →∞ = sup (g)f,f L2(K) (g)f,f L2(K) 0, g K˜ P − P −→ ∈

and (29) is shown. This yields the inverse implications of 3. and thus, the inverse implication is shown for every case.

At the end, it still has to be shown that there are no other possibilities of convergence than the ones listed in Proposition 4.17. For this, one needs some preliminaries. First, one can observe that if a sequence of representations (πj)j N converges to a representa- ∈ tion π, then for all n with (n) = 0 , there exists an integer J N such that (n) = 0 Hπ 6 { } ∈ Hπj 6 { } for all j J. ≥ To show this, choose f π in such a way that its projection to the n-th isotypic component n ∈H N pn(f) =: f is non-zero. Furthermore, let (fj)j N be a sequence with fj πj for every j ∈ ∈H ∈ such that the sequence of matrix coeﬃcients πj( )fj,fj converges uniformly on · πj j N H ∈ compacta to π( )f,f . Then, there is a g G such that π(g)f n,f n is non-zero and, · π ∈ π deﬁning f n the projectionH to the n-th isotypic component of f , it canH be shown that the j j n n n n n sequence πj(g)f ,f converges to π(g)f ,f . Therefore, f = 0 for large j j π N π j H j j H 6 j N and the claim is shown. ∈ ∈ So, in particular, if (πj)j N converges to π = 0, then for large j N, there has to be a ∈ 6 ∈ common K-type for (π ) and π, i.e. an n Z such that (n) = 0 and (n) = 0 j j large ∈ Hπ 6 { } Hπj 6 { } for large j. (⋆)

N u Now, by Remark 4.5, the representations 1, m± for odd m ∗, for u (0, 1) +,iv F D ∈ C ∈ N and for v [0, ) are even, while the representations +, , m± for even m ∗ P ,iv ∈ ∞ D D− D ∈ and − for v (0, ) are odd. With the result shown above, this leads to the fact that the P ∈ ∞ u +,iv N set of representations 1, m± , , odd m ∗, u (0, 1), v [0, ) is separated F D ,ivC P | N ∈ ∈ ∈ ∞ from the set +, , m± , − even m ∗, v (0, ) . D D− D P | ∈ ∈ ∞ First, regard the ﬁrst set of representations. +,ivj +,ivj C 1 2 If a sequence N for vj [0, ) converges, ( ) N = vj 1 id P j ∈ ∞ P j 4 − − · j N ∈ ∈ ∈ converges as well and thus, the sequence (vj)j N has to converge. This sequence can only ∈ converge to v [0, ), which implies, as seen above, that lim +,ivj = +,iv. So, the ∈ ∞ j P P →∞ +,ivj convergence indicated in 1. is the only possible one for j N. P ∈ With the same reasoning, if a sequence uj converges, there are three possibilities j N C ∈ for the sequence (uj)j N: It can converge to u (0, 1), to 0 or to 1, which are the three ∈ ∈ possibilities listed in 3. Moreover, the three sets of representations , + m N and m N can be {F1} Dm| ∈ ∗ Dm− | ∈ ∗ separated from each other, since they do not have any common non-zero K-type. Further- more, as the values, that the representations + and for m N take on the Casimir Dm Dm− ∈ ∗ operator, form discrete sets, the representations + and + and the representations Dm1 Dm2 Dm−1

85 and , respectively, can be separated from each other for every m = m N . Again, Dm−2 1 6 2 ∈ ∗ due to the values on the Casimir operator, + m N and m N can be Dm| ∈ >1 Dm− | ∈ >1 isolated from all the other representations. + N N For these reasons, sequences (πj)j N with elements in m m ∗ or m− m ∗ can ∈ D | ∈ D | ∈ only converge if they are constant for large k and in this case, they have one single limit. Now, considering other sequences (πj)j N of representations in the set of even representations ∈ given above, one has to regard again the value they take on the Casimir operator. Further-

more, taking into account (⋆) and the fact that + − = + 1 = − 1 = 0 , H 1 ∩H 1 H 1 ∩HF H 1 ∩HF { } it is obvious that a sequence which is neither constantD D for largeD j nor one ofD the sequences listed in 1. to 3. for large j, has to fulﬁll Property (c), (d), (e) or (f) in order to converge. That the convergence is achieved in these cases, can be deduced from the proof given above for 1. and 3. Cases (a) and (b) are clear. Moreover, regarding the second set of representations, with the same argumentation as +,iv ,ivj N above for , if a sequence − j N converges, then (vj)j converges as well, either to P P ∈ ∈ v (0, ) or to 0. This means that ,ivj can only converge to a representation ,iv − j N − ∈ ∞ P ∈ P or to +, . D D− Again, a converging sequence in + m N even or m N even has to be Dm| ∈ ∗ Dm− | ∈ ∗ constant and it has one single limit point. As for the set of even representations, since + − = 0 and by (⋆), the only possibility HD ∩HD { } of convergence for a sequence of representations that is neither constant for large j nor one of the sequences listed in 1. to 3. for large j, is to fulﬁll Property (g) or (h). Also, it can be deduced from the above proof for 2. that in this case, convergence is indeed achieved. Again, Cases (a) and (b) are clear. Hence, the given ones are all possibilities for sequences of irreducible unitary representations of G to converge.

4.2.5 Definition of subsets Γi of the spectrum Now, the spectrum will be divided into different subsets which are thereafter proved to meet the requirements of the norm controlled dual limit conditions of Definition 1.1.

Deﬁne

Γ := , 0 F1 Γ := + , 1 D1 Γ := − , 2 D1 Γ := , 3 D+ Γ4 := , D− Γ := ± m N , 5 Dm | ∈ >1 Γ := +,iv v [0, ) , 6 P | ∈ ∞ ,iv Γ := − v (0, ) and 7 P | ∈ ∞ Γ := u u (0, 1) . 8 C | ∈

86 Obviously, all the sets Γ for i 0,..., 8 are Hausdorff. The sets i ∈ { } Si := Γj j 0,...,i ∈{[ } are closed and the set S0 = Γ0 consists of all the characters of G = SL(2, R). In addition, as defined in Section 4.2.2, for every i 0,..., 8 , there exists one common Hilbert space ∈ { } Hi that all the representations in Γi act on. Therefore, Condition 1 of Definition 1.1 is fulfilled.

As every connected real linear semisimple Lie group meets the CCR-condition (see [9], Theorem 15.5.6), Condition 2 of Deﬁnition 1.1 is fulﬁlled as well. Thus, Condition 3 remains to be shown.

4.3 Condition 3(a) For the proof of Condition 3(a) of Deﬁnition 1.1, as well as for the proof of Condition 3(b), some preliminaries are needed.

Deﬁne the K K-representation πK K on the space VπK×K := C0∞(G) of compactly supported × × C∞(G)-functions as

πK K : K K C0∞(G) , × × →B 1 πK K (k1,k2)h(g) := h(k1− gk2) (k1,k2) K K h C0∞(G) g G. × ∀ ∈ × ∀ ∈ ∀ ∈ For all l,n Z, ∈ ilϕ1+inϕ2 VπK×K (l,n) = h C0∞(G) πK K kϕ1 ,kϕ2 h = e h ϕ1,ϕ2 [0, 2π) ∈ × ∀ ∈ n 1 ilϕ1+inϕ2 o = h C∞(G) h k− gk = e h(g) ϕ ,ϕ [0, 2π) . ∈ 0 ϕ1 ϕ2 ∀ 1 2 ∈ Then, the algebraic directn sum V (l,n) is dense in V = C (G) witho respect πK×K πK×K 0∞ l,n Z 1 ∈ 1 to the L (G)-norm and as ∗L 1 on L (G), it is dense with respect to the k · kC (G) ≤ k · kL (G) C∗(G)-norm as well. C0∞(G) in turn is dense in C∗(G). Hence, the algebraic direct sum

VπK×K (l,n) is also dense in C∗(G). l,n Z ∈ LetL pl,n be the projection going from VπK×K to VπK×K (l,n) deﬁned in the following way: For h V = C (G) and g G, ∈ πK×K 0∞ ∈

1 1 ilϕ1 inϕ2 p (h)(g) := h k gk− e e d k ,k . l,n K 2 ϕ1 ϕ2 ϕ1 ϕ2 | | KZK × Due to the density discussed above and to the Corollaries 7.4 and 7.5 in the appendix, instead of dealing with a general element a C∗(G), the calculations in Sections 4.3 and 4.4 can be ∈ Z accomplished with a function h C0∞(G) fulﬁlling h = pl, n(h) for some integers l,n . ∈ − ∈ Lemma 4.18. 2 Z For f L (K)+ and h C0∞(G) with h = pl, n(h) for l,n , one gets ∈ ∈ − ∈ +,u(h)(f)= +,u(h) p (f) = p +,u(h) p (f) u C. P P n l P n ∀ ∈ Furthermore, if h = 0, the integersl and n must be even. 6

87 Proof: First, one gets for k K, ψ ∈

inϕ +,u inϕ 1 e− k f k dk = e− f k− k dk P ϕ ψ ϕ ϕ ψ ϕ Z Z K K in(ϕ ψ) = e − f kϕ dkϕ Z K inψ inϕ = e− e f k dk = K p (f) k . ϕ ϕ | | n ψ Z K Therefore,

+,u(h)(f) = h(g) +,u(g)fdg P P GZ +,u = pl, n(h)(g) (g)fdg − P GZ

1 1 ilϕ1 inϕ2 +,u = h k gk− e e− d k ,k (g)fdg K 2 ϕ1 ϕ2 ϕ1 ϕ2 P | | GZ KZK ×

1 ilϕ1 inϕ2 1 +,u = e e− h k gk− (g)fdgd k ,k K 2 ϕ1 ϕ2 P ϕ1 ϕ2 | | KZK GZ ×

1 ilϕ1 inϕ2 +,u 1 = e e− h(g) k− gk fdgd k ,k K 2 P ϕ1 ϕ2 ϕ1 ϕ2 | | KZK GZ ×

1 ilϕ1 +,u 1 +,u inϕ2 +,u = e k− h(g) (g) e− k fdk dgdk K 2 P ϕ1 P P ϕ2 ϕ2 ϕ1 | | Z Z Z K G K

1 ilϕ1 +,u 1 +,u = e k− h(g) (g) p (f) dgdk K P ϕ1 P n ϕ1 | | Z Z K G

1 ilϕ1 +,u 1 +,u = e k− (h) p (f) dk K P ϕ1 P n ϕ1 | | Z K = p +,u(h) p (f) . l P n From this equality, one can also immediately deduce that +,u(h)(f) = +,u(h) p (f) by P P n replacing f by p (f). Hence, the equalities of Lemma 4.18 are shown. n By these equalities and as f and +,u(h) p (f) are in L2(K) , for h = 0, the integers l and P n + 6 n must be even.

Now, the proof of Condition 3(a) of Deﬁnition 1.1 can be executed. Condition 3(a) is obvious for the sets Γ for i 0,..., 5 , as these are discrete sets. i ∈ { }

88 j +,ivj +,iv →∞ So, let vj,v [0, ) and j N a sequence in Γ6 converging to . Then, vj v. ∈ ∞ P ∈ P −→ Let h C (G) be supported in the compact set K˜ G. Hence, similarly as in the proof of ∈ 0∞ ∈ (25), there is a compact set I R such that ⊂ 2 +,ivj (h) +,iv(h) P −P op 2 = sup h(g) +,ivj (g)(f)(k) +,iv(g)(f)(k) dg dk 2 f L (K)+ P −P ∈ KZ Z˜ f 2=1 K k k 2 2 −1 ivj x ivx ρH(g k) 1 sup e− e− sup h(g) e− f κ g− k dg dk 2 ≤ x I − f L (K)+ | | ∈ ∈ KZ Z˜ f 2=1 K k k

H¨older 2 −1 2 ivj x ivx 2 2ρH(g k) 1 sup e− e− h L2(G) sup e− f κ g− k dgdk 2 ≤ x I − k k f L (K)+ ∈ ∈ KZ Z˜ f 2=1 K k k Lemma 4.2 2 j ˜ ivj x ivx 2 →∞ = K sup e− e− h L2(G) 0, (30) x I − k k −→ ∈ j as v →∞ v. j −→ Because of the density of C0∞(G) in C∗(G) and by Corollary 7.4 which can be found in the appendix, one gets the desired convergence for a C (G). ∈ ∗ The reasoning is the same for Γ7.

For Γ , let u ,u (0, 1) and uj a sequence in Γ converging to u Γ . Then, 8 j ∈ C j N 8 C ∈ 8 j ∈ uj →∞ u. Moreover, let h C∗(G) and let l,n Z such that h = pl, n(h), as discussed at −→ ∈ ∈ − the beginning of this subsection. 2 Let f L (K)+ with f L2(K) = 1. ∈ 1 k k Since K− commutes with p , by Lemma 4.18, one has for everyu ˜ (0, 1), u˜ n ∈ u˜ +,u˜ 1 +,u˜ 1 (h)(f) = K (h) K− (f) = K p (h) p K− (f) C u˜ ◦P ◦ u˜ u˜ ◦ l P n u˜ +,u˜ 1 dl u˜ +,u˜ = dl u˜ pl (h) K− pn(f) = pl (h) pn(f) P u˜ d u˜ P n cl u˜) = +,u˜(h)(f). (31) scnu˜ P Hence, 2 2 cl(uj) cl(u) j uj u +,uj +,u →∞ (h) (h) op = (h) (h) 0, (32) C −C scn(uj) P − scn(u) P −→ op

j with the same reasoning as in (30) and since u →∞ u. j −→ Because of the density of C0∞(G) in C∗(G) and with Corollary 7.4, one gets the desired convergence for a C (G) and thus, the claim is also shown for Γ . ∈ ∗ 8

89 4.4 Condition 3(b) Now, only Condition 3(b) remains to be shown. This is the most complicated part of the proof of the conditions listed in Deﬁnition 1.1. Remark 4.19. The setting of a sequence (γj)j N in Γi converging to a limit set contained in Si 1 = Γl ∈ − l

N ,ivj (i) (γj)j = − j N is a sequence in Γ7 whose limit set is Γ3 Γ4 = +, . ∈ P ∈ ∪ D D− N uj +,0 (ii) (γj)j = j N is a sequence in Γ8 whose limit set is Γ7. ∈ C ∈ P ⊂ + N uj (iii) (γj)j = j N is a sequence in Γ8 whose limit set is Γ0 Γ1 Γ2 = 1, 1 , 1− . ∈ C ∈ ∪ ∪ F D D For i 0,..., 6, the sets Γ are closed and thus, the regarded situation cannot appear for ∈ { } i sequences (γj)j N in Γi for i 0,..., 6 . ∈ ∈ { } Since all the sequences regarded in the Cases (i), (ii) and (iii) of Remark 4.19 are properly converging, the transition to subsequences will be omitted.

First, regard Case (i) mentioned in Remark 4.19. j ,ivj ,ivj Let − N be a sequence in Γ7 whose limit set is +, . As − →∞ +, , P j D D− P −→ D D− ∈ j it follows that v →∞ 0. j −→ Now, a bounded, linear and involutive mapping

2 ν˜j : CB(S6) L (K) →B − fulﬁlling ,ivj lim − (a) ν˜j (a) S6 = 0 a C∗(G) j P − F | op ∀ ∈ →∞ has to be deﬁned.

Since in this and in the following cases, this mapping will not depend on j, it will from now on be denoted byν ˜ instead ofν ˜j. 2 2 Let p+ be the projection from L (K) to the space + and p the projection from L (K) − 2 HD − − 2 to − . Then, by Remark 4.14, one has L (K) = + − , i.e. id L (K)− = p+ + p . HD − HD ⊕HD | − Now, let

ν˜(ψ) :=ν ˜ +, − (ψ) := ψ( +) p+ + ψ( ) p ψ CB(S6). {D D } D ◦ D− ◦ − ∀ ∈ 2 This is well-deﬁned, as +, S6 and furthermore,ν ˜(ψ) L (K) . D D− ∈ ∈B − The linearity of the mappingν ˜ is clear. For the involutivity, let ψ CB(S ). Then, since p ∈ 6 + and p equal the identity on the image of ψ( +) and ψ( ), respectively, − D D− ∗ ∗ ν˜(ψ) ∗ = ψ( +) p+ + ψ( ) p D ◦ D− ◦ − ∗ ∗ = p+ ψ( +) p+ + p ψ( ) p ◦ D ◦ − ◦ D− ◦ − = p+∗ ψ∗( +) p+∗ + p∗ ψ∗( ) p∗ ◦ D ◦ − ◦ D− ◦ − = p+ ψ∗( +) p+ + p ψ∗( ) p ◦ D ◦ − ◦ D− ◦ − = ψ∗( +) p+ + ψ∗( ) p =ν ˜(ψ∗). D ◦ D− ◦ −

90 To show thatν ˜ is bounded, again let ψ CB(S ): ∈ 6 ν˜(ψ) = ψ( +) p+ + ψ( ) p = max ψ( +) , ψ( ) op D ◦ D− ◦ − op D op D− op n o sup ψ(γ) op = ψ S6 . ≤ γ S6 k k k k ∈ Now, only the demanded convergence remains to be shown: For h C (G), one has ∈ 0∞ 2 2 ,ivj ,ivj − (h) ν˜ (h) S6 = − (h) (h)( +) p+ + (h)( ) p P − F | op P − F D ◦ F D− ◦ − op 2 ,ivj = − (h) +(h) p+ + (h) p P − D ◦ D− ◦ − op 2 ,ivj ,0 ,0 = − (h) − (h) p+ + − (h) p P − P ◦ P ◦ − op ,ivj ,0 2 j = − (h) − (h) →∞ 0 P −P op −→ j as in (30) and since vj →∞ 0. −→ Again, because of the density of C0∞(G) in C∗(G) and with Corollary 7.5, one gets the desired convergence for a C (G). ∈ ∗ Now, regard Case (ii) mentioned in Remark 4.19. j Let uj be a sequence in Γ whose limit set is +,0 . Thus, u →∞ 0. C j N 8 P j −→ Here, a bounded,∈ linear and involutive mapping ν˜ : CB(S ) L2(K) 7 →B + fulfilling uj lim (a) ν˜ (a) S7 = 0 a C∗(G) j C − F | op ∀ ∈ →∞ is needed. Define +,0 ν˜(ψ) :=ν ˜ +,0 (ψ) := ψ ψ CB(S7). P P ∀ ∈ ν˜(ψ) L2(K) for every ψ CB(S ) andν ˜ is well-defined, as +,0 S . ∈B + ∈ 7 P ∈ 7 The linearity and the involutivity ofν ˜ are clear. For the boundedness ofν ˜, let ψ CB(S ). Then, ∈ 7 +,0 ν˜(ψ) op = ψ op sup ψ(γ) op = ψ S7 . P ≤ γ S7 k k k k ∈ Again, it remains to show the demanded convergence: Z Let h C0∞(G). Then, one can assume again that there exist l,n such that h = pl, n(h). − ∈ cn˜ (u) ∈ Since lim = 1 for alln,n ˜ ′ Z by (15) and Lemma 4.10, one gets with (31), u 0 cn′ (u) ∈ → 2 2 uj cl(uj) +,uj +,0 (h) ν˜ (h) S7 = (h) (h) C − F | op scn(uj) P − F P op 2 cl(uj) j = +,uj (h) +,0(h) →∞ 0, scn(uj) P − P −→ op

91 j since u →∞ 0 and with the same arguments as in the subsection above. j −→ The desired convergence for a C (G) follows. ∈ ∗ Last, regard Case (iii) of Remark 4.19. u + j Let j be a sequence in Γ whose limit set is , , − . This means that u →∞ 1. C j N 8 F1 D1 D1 j −→ Again, a bounded,∈ linear and involutive mapping ν˜ : CB(S ) L2(K) 7 →B + fulﬁlling uj lim (a) ν˜ (a) S7 = 0 a C∗(G) j C − F | op ∀ ∈ →∞ is needed. 2 2 For this, let p+ be the projection from L (K)+ to the space f L (K)+ pn(f) = 0 n 0 2 2 ∈ | ∀ ≤ and p the projection from L (K)+ to f L (K)+ pn(f) = 0 n 0 . Then, since − ∈ | ∀ ≥ L2(K) = f L2(K) p (f) = 0 n 0 + f L2(K) p (f) = 0 n 0 + ∈ +| n ∀ ≤ ∈ +| n ∀ ≥ + f L2(K) p (f) = 0 n = 0 , ∈ +| n ∀ 6 2 every f L (K)+ can be written as f = p+(f)+ p (f)+ p0(f). ∈ − Furthermore, let

cn(u) dn,2(1) := lim for all even n> 0 and u 1 c (u) → s 2

cn(u) dn, 2(1) := lim for all even n< 0. − u 1 sc 2(u) → − The existence of these limits follows with Lemma 4.12(c) in Section 4.2.1 and the analogous ˜ statement for J[1]. Now, deﬁne the operators

2 K(1) : (1) L (K)+ by K(1) := dn,2(1) id (n) for all even n> 0 +,1 (n) P+,1 H → |HP · |H and

2 K[1] : [1] L (K)+ by K[1] := dn, 2(1) id (n) for all even n< 0. +,1 (n) P+,1 H → |HP − · |H By deﬁnition, these operators are linear and they are unitary by the construction of the scalar ˜ ˜ products deﬁned on (1) and [1]. Moreover, like J(1) and J( 1), they are injective and, as H H − proved in Remark 4.8 for the operator Ju, K(1) and K[1] commute with the projections pn for all n Z. ∈ One can easily see that

K = f L2(K) p (f) = 0 n 0 = p L2(K) and (1) H(1) ∈ +| n ∀ ≤ + + 2 2 K[1] [1] = f L (K)+ pn(f) = 0 n 0 = p L (K)+. H ∈ | ∀ ≥ − 

92 Hence, one can build the inverse of the operators K(1) and K[1] on the image of p+ and p , − respectively. Therefore, one can deﬁne

+ 1 1 ν˜(ψ) :=ν ˜ + − (ψ) := K ψ( ) K− p + K ψ( −) K− p 1, , (1) 1 (1) + [1] 1 [1] {F D1 D1 } ◦ D ◦ ◦ ◦ D ◦ ◦ − + ψ( ) p ψ CB(S ). F1 ◦ 0 ∀ ∈ 7 + 2 The mappingν ˜ is well-deﬁned, since , −, are contained in S , andν ˜(ψ) L (K) D1 D1 F1 7 ∈B + for every ψ CB(S ). In addition, its linearity is clear again. ∈ 7 For the involutivity and the boundedness, let ψ CB(S ). Then, as K and K are unitary ∈ 7 (1) [1] and as p+, p and p0 equal the identity on the image of K(1), K[1] and ψ( 1), respectively, − F ν˜(ψ) ∗

+ 1 1 ∗ = K(1) ψ( 1 ) K− p+ + K[1] ψ( 1−) K− p + ψ( 1) p0 ◦ D ◦ (1) ◦ ◦ D ◦ [1] ◦ − F ◦ + 1 1 ∗ = p+ K(1) ψ( 1 ) K− p+ + p K[1] ψ( 1−) K− p + p0 ψ( 1) p0 ◦ ◦ D ◦ (1) ◦ − ◦ ◦ D ◦ [1] ◦ − ◦ F ◦ + 1 1 = p+ K(1) ψ∗( 1 ) K− p+ + p K[1] ψ∗( 1−) K− p + p0 ψ∗( 1) p0 ◦ ◦ D ◦ (1) ◦ − ◦ ◦ D ◦ [1] ◦ − ◦ F ◦ =ν ˜ ψ∗ .

1 1 Furthermore, since K(1) op K(1)− op = K[1] op K[1]− op = 1, one gets

+ 1 1 ν˜(ψ) = K(1) ψ( 1 ) K− p+ + K[1] ψ( 1−) K− p + ψ( 1) p0 op ◦ D ◦ (1) ◦ ◦ D ◦ [1] ◦ − F ◦ op

+ 1 1 = max K(1) ψ( 1 ) K− , K[1] ψ( 1−) K− , ψ( 1) ◦ D ◦ (1) op ◦ D ◦ [1] op F op + = max ψ( ) , ψ( −) , ψ( ) D1 op D1 op F1 op n o sup ψ(γ) op = ψ S7 . ≤ γ S7 k k k k ∈ For the demanded convergence, let h C (G). Like in the proof of (ii) and Condition 3(a), ∈ 0∞ one can assume that there exist l,n Z such that h = pl, n(h). 2 ∈ − Let f L (K)+ with f L2(K) = 1. ∈ 1 1 k k Since K(1)− and K[1]− commute with pn, similarly as in the proof of (31), by Lemma 4.18, one gets

ν˜ (h) S7 (f) F | + 1 1 = K(1) (h)( 1 ) K− p+(f) + K[1] (h)( 1−) K− p (f) ◦ F D ◦ (1) ◦ ◦ F D ◦ [1] ◦ − + (h)( 1) p0(f) F +F ◦ 1 1 = K(1) 1 (h) K− p+(f) + K[1] 1−(h) K− p (f) + 1(h) p0(f) ◦D ◦ (1) ◦ ◦D ◦ [1] ◦ − F ◦ +,1 1 +,1 1 +, 1 = K(1) (h) K− p+(f) + K[1] (h) K− p (f) + − (h) p0(f) ◦P ◦ (1) ◦ ◦P ◦ [1] ◦ − P ◦

+,1 1 = K(1) pl (h) pn K(1)− p+(f) ◦ P ◦ ! +,1 1 +, 1 + K[1] pl (h) pn K− p (f) + pl − (h) pn p0(f) ◦ P [1] ◦ − P ! 93 +,1 1 = dl,2(1) pl (h) K(1)− pn p+(f) P ◦ ! +,1 1 +, 1 + dl, 2(1) pl (h) K[1]− pn p (f) + pl − (h) pn p0(f) . − P ◦ − ! P ◦ There are three cases to consider: In the ﬁrst case, n > 0, in the second case, n < 0 and in the third case, n = 0.

So, ﬁrst let n> 0. Then,

+,1 1 ν˜ (h) S7 (f) = dl,2(1) pl (h) K− pn p+(f) F | P (1) ◦ ! dl,2(1) +,1 cl(u) +,1 = pl (h) pn(f) = lim (h)(f). d (1) P u 1 c (u) P n,2 → s n Therefore, joining this result with (31), one gets 2 2 cl(uj) cl(u) j uj +,uj +,1 →∞ (h) ν˜ (h) S7 = (h) lim (h) 0, C − F | op scn(uj) P − u 1 scn(u) P −→ → op j with the same reasoning as in (30) and since u →∞ 1. j −→ The proof for the case n < 0 is the same as the one for the ﬁrst case. Hence, only the case n = 0 remains: Since by (16) c (1) 1 if l = 0 l = c0(1) (0 if l = 0, 6 one gets for l = 0 with (31), 6 2 2 cl(uj) j uj +,uj →∞ (h) ν˜ (h) S7 = (h) 0 0, C − F | op sc0(uj) P − −→ op j since u →∞ 1. j −→ So, let l = 0 and deﬁne Ch := h(g)dg. G Then, R

(h) p (f)= h(g) (g)dg p (f) = id p (f) C = C p (f). F1 ◦ 0 F1 0 0 · h h 0 Z G Furthermore, with Lemma 4.2, one gets for g G, ∈ +,1 1 +,1 p0(f) 2ρH(g−1k) p (g) p (f) = (g)p (f)(k)dk = e− dk 0 ◦P ◦ 0 K P 0 K | | KZ | | KZ 2 p0(f) 2ρH(g−1k) 1 p0(f) = e− 1 κ g− k dk = 1 2 = p (f). K K k kL (K) 0 | | KZ | |

94 Thus, for h one has

p +,1(h) p (f)= h(g)p (f)dg = C p (f)= (h) p (f). 0 ◦P ◦ 0 0 h 0 F1 ◦ 0 GZ

Therefore, for l = n = 0, by (31),

2 2 j uj +,uj +,1 →∞ (h) ν˜ (h) S7 = p0 (h) p0 p0 (h) p0 0, C − F | op ◦P ◦ − ◦P ◦ op −→ j since u →∞ 1. j −→ Hence, the claim is also shown in the case n = 0 and thus,

2 j uj →∞ (h) ν˜ (h) S7 0, C − F | op −→

as demanded. Again, the desired convergence for a C (G) follows. ∈ ∗

4.5 Result for the Lie group SL(2, R) Now, after having verified all the conditions listed in Definition 1.1, Theorem 1.2 is proved for the Lie group SL(2, R). Therefore, with the sets Γ and S and the Hilbert spaces for i i Hi i 0,..., 8 defined in Section 4.2.5 and Section 4.2.2 and the mappingsν ˜ constructed in ∈ { } Section 4.4, the C∗-algebra of G = SL(2, R) can be characterized by the following theorem which was already stated in Section 1:

Theorem 4.20. The C∗-algebra C∗(G) of G = SL(2, R) is isomorphic (under the Fourier transform) to the set of all operator ﬁelds ϕ deﬁned over G such that:

1. ϕ(γ) ( ) for every i 1,..., 8 and every γ Γ . ∈K Hi ∈ { b} ∈ i 2. ϕ l (G). ∈ ∞ 3. The mappings γ ϕ(γ) are norm continuous on the diﬀerent sets Γ . b 7→ i 4. For any sequence (γj)j N G going to inﬁnity, lim ϕ(γj) op = 0. ∈ ⊂ j k k →∞

5. For every i 1,..., 8 and anyb properly converging sequence γ = (γj)j N Γi whose ∈ { } ∈ ⊂ limit set is contained in Si 1 (taking a subsequence if necessary) and for the mappings − ν˜ =ν ˜γ,j : CB(Si 1) ( i) constructed in the preceding subsections, one has − →B H

lim ϕ(γj) ν˜ ϕ Si = 0. j − | −1 op →∞

Let for a topological Hausdorff space V and a C∗-algebra B, C (V,B) be the C∗-algebra of ∞ all continuous functions defined on V with values in B that are vanishing at infinity. Then, from this theorem one can deduce more concretely the following result for G = SL(2, R):

95 Theorem 4.21. 2 2 Let the operator p+ be the projection from L (K) to f L (K) pn(f) = 0 n 0 , 2 ± 2 ∈ ±| ∀ ≤ p the projection from L (K) to the space f L (K) pn(f) = 0 n 0 and p0 the − ± ∈ ±| ∀ ≥ projection from L2(K) to f L2(K) p (f) = 0 n = 0 = C. + ∈ +| n ∀ 6 Then, the C -algebra C (G) of G = SL(2, R) is isomorphic to the direct sum of C -algebras ∗ ∗ ∗ 2 F C i[0, ) [0, 1], L (K)+ F (1) commutes with p+, p and p0 ∈ ∞ ∞ ∪ K − 2 F C i[0, ), L (K) F (0) commutes with p+ and p ⊕ ∈ ∞ ∞ K − − C Z 1, 0, 1 , ⊕ ∞ \ {− } K HD for the inﬁnite-dimensional and separable Hilbert space ﬁxed in Section 4.2.2. HD Proof: The spectrum of C∗(G) or equivalently of G is given by the disjoint union . . G G G , even ∪ odd ∪ discrete +,iv u where the set Geven consists of the even representations for v [0, ), for u (0, 1), + b b b P ∈ ,iv∞ C ∈ 1 , 1− and 1, the set Godd consists of the odd representations − for v (0, ), + D D F P ∈ ∞ + D and and theb set Gdiscrete consists of the even or respectively odd representations m for D− D m N and for m bN . ∈ >1 Dm− ∈ >1 These three listed setsb of representations are topologically separated from each other (see Section 4.2.4). +,iv u + Mapping for v [0, ) to iv, for u (0, 1) to u and , − and to 1, one gets P ∈ ∞ C ∈ D1 D1 F1 a surjection from Geven to the set i[0, ) [0, 1] =: I1. ,iv ∞ ∪ Furthermore, mapping − for v (0, ) to iv and + and to 0, one gets a surjection P ∈ ∞ D D− from G to i[0, b) =: I . odd ∞ 2 Last, mapping + for m N to m and for m N to m, one gets a surjection from Dm ∈ >1 Dm− ∈ >1 − G b to Z 1, 0, 1 =: I . discrete \ {− } 3 Hence, one regards the three sets b I = i[0, ) [0, 1], I = i[0, ) and I = Z 1, 0, 1 . 1 ∞ ∪ 2 ∞ 3 \ {− }

In order to prove this theorem, it has to be shown that for every operator field ϕ = (a) F for a C (G) that fulfills the properties listed in Theorem 4.20, there exists a mapping ∈ ∗ 1 2 Fa F C i[0, ) [0, 1], L (K)+ F (1) commutes with p+, p and p0 =: P1, ∈ ∈ ∞ ∞ ∪ K − 2 2 a mapping Fa F C i[0, ), L (K ) F (0) commutes with p+ and p =: P2 ∈ ∈ ∞ ∞ K − − 3 Z and a mapping Fa C 1, 0, 1 , =: P3. ∈ ∞ \ {− } K HD On the other hand, for every F1 P1, every F2 P2 and every F3 P3, one has to construct ∈ ∈ ∈ an operator field ϕF1,F2,F3 over G that fulfills the properties of Theorem 4.20. Since the above-mentioned sets of representations Geven, Godd and Gdiscrete are topologically separated b from each other, it suffices to define three different operator fields ϕF1 over Geven, ϕF2 over

Godd and ϕF3 over Gdiscrete. b b b b b b 96 For every a C (G), define a function F 1 : I L2(K) by ∈ ∗ a 1 →B + 1 +,x Fa (x) := (a) x i[0, ), 1 F Px ∀ ∈ ∞ Fa (x) := (a) x (0, 1) and 1 F C ∀ +∈ 1 1 Fa (1) := K(1) (a) 1 K− p+ + K[1] (a) 1− K− p + (a) 1 p0. ◦ F D ◦ (1) ◦ ◦ F D ◦ [1] ◦ − F F ◦ 1 2 By Property 1 of Theorem 4.20, Fa (x) L (K)+ for all x I1 1 . Furthermore, + ∈ K ∈ \{ } since (a) 1 , (a) 1− and (a) 1 are also compact, their composition with the F D F D 1 F F1 bounded operators K(1), K− , K[1], K− , p+, p and p0 is compact as well. Therefore, (1) [1] − F 1(1) L2(K) , too. a ∈K + By Property 4 of Theorem 4.20, F 1 vanishes at infinity. Moreover, for all x I 0, 1 , F 1 a ∈ 1 \{ } a is obviously continuous in x. For the continuity in 0, let u =(uj)j N be a sequence in (0, 1) converging to 0. Then, ∈ 1 1 uj +,0 lim Fa (uj) Fa (0) = lim (a) (a) j − op j F C − F P op →∞ →∞ = lim uj (a) +,0(a) j C −P op →∞ +,uj 1 +,0 1 = lim Kuj (a) Ku− id (a) id− = 0, j ◦P ◦ j − ◦P ◦ op →∞ j as K →∞ id and with the same arguments as in the proof of (32) in Section 4.3. uj −→ For the continuity in 1, let u =(uj)j N be a sequence in (0, 1) converging to 1. By Property 5 ∈ of Theorem 4.20 and since F 1(1) =ν ˜ (a) =ν ˜ (a) by the definition ofν ˜ in Case (iii) a u,j F F of Remark 4.19 in Section 4.4, 1 1 uj lim Fa (uj) Fa (1) = lim (a) ν˜ (a) = 0. j − op j F C − F op →∞ →∞ 1 1 2 Hence, Fa is also continuous in 0 and in 1 and thus, Fa C i[0, ) [0, 1], L (K)+ . ∈ ∞ ∞ ∪ K Since p+, p and p0 equal the identity on the image of K(1), K[1]and (a) 1 , respectively, as − F F discovered in Section 4.4, and since p+ p = p p+ = p+ p0 = p0 p+ = p p0 = p0 p = 0, 1 ◦ − −◦ ◦ ◦ −◦ ◦ − Fa (1) commutes with p+, p and p0. − 2 On the other hand, taking a function F1 C i[0, ) [0, 1], L (K)+ that ∈ ∞ ∞ ∪ K commutes with p+, p and p0, it has to be shown that there is an operator field ϕF1 over − Geven that meets the Properties 1 to 5 of Theorem 4.20. Define b +,x 2 ϕF1 := F1(x) L (K)+ x i[0, ), P x ∈B 2 ∀ ∈ ∞ ϕF1 := F1(x) L (K)+ x (0, 1), C+ 1 ∈B ∀ ∈ ϕ := K− p F (1) K , F1 D1 (1) ◦ + ◦ 1 ◦ (1) ∈B H(1) 1 ϕF1 1− := K− p F1(1) K[1] [1] and D [1] ◦ − ◦ ◦ ∈B H ϕF 1 := p0 F1(1) p0 C. 1 F ◦ ◦ ∈ 2 By the definition of C i[0, ) [0, 1], L (K)+ and as the composition of the compact ∞ ∞ ∪ K operator F1(1) with bounded operators is compact again, the Properties 1 to 4 are obviously fulfilled.

97 For Property 5, there are two cases to consider: a sequence in (0, 1) converging to 0 and a sequence in (0, 1) converging to 1. First, let (uj)j N be a sequence in (0, 1) converging to 0. Then, by the deﬁnition ofν ˜ in ∈ Case (ii) of Remark 4.19 in Section 4.4,

j ϕ uj ν˜ ϕ = F (u ) ϕ +,0 = F (u ) F (0) →∞ 0, F1 C − F1 op 1 j − F1 P op 1 j − 1 op −→ since F1 is continuous in 0. Now, let (uj)j N be a sequence in (0, 1) converging to 1. As F1(1) commutes with p+, p and ∈ − 2 2 p0, by the deﬁnition ofν ˜ in Case (iii) of Remark 4.19 and since p++p +p0 = idL (K)+ L (K)+ , − → ϕ uj ν˜ ϕ F1 C − F1 op + 1 1 = F1(uj) K(1) ϕF1 1 K− p+ K[1] ϕF1 1− K− p ϕF1 1 p0 − ◦ D ◦ (1) ◦ − ◦ D ◦ [1] ◦ − − F ◦ op 1 1 = F (u ) K K− p F (1) K K− p 1 j − (1) ◦ (1) ◦ + ◦ 1 ◦ (1) ◦ (1) ◦ + 1 1 K[1] K− p F1(1) K[1] K− p p0 F1(1) p0 p0 − ◦ [1] ◦ − ◦ ◦ ◦ [1] ◦ − − ◦ ◦ ◦ op

= F1(uj) p+ F1(1) p+ p F1(1) p p0 F1(1) p0 − ◦ ◦ − − ◦ ◦ − − ◦ ◦ op = F1(uj) F1(1) p+ + p + p0 − ◦ − op j = F (u ) F (1) →∞ 0 k 1 j − 1 kop −→ because of the continuity of F1 in 1. Therefore, Property 5 is fulﬁlled as well.

2 2 One defines for all a C∗(G) the function Fa : I2 L (K) by ∈ →B − 2 ,x Fa (x) := (a) − x i(0, ) and 2 F P ∀ ∈ ∞ Fa (0) := (a) + p+ + (a) p . F D ◦ F D− ◦ − As for I , the compactness of F 2(x) for all x I follows from Property 1 of Theorem 4.20 1 a ∈ 2 and with the fact that the compositions of the compact operators (a) + and (a) F D F D−2 with the bounded operators p+ and p , respectively, are compact. The continuity of Fa − 2 outside of 0 follows from Property 3 and Fa vanishes at infinity by Property 4. Using again Property 5 of Theorem 4.20 and the fact that F 2(0) =ν ˜ (a) by the definition ofν ˜ in a F Case (i) of Remark 4.19 in Section 4.4, one also obtains the continuity of F 2 in 0. Hence, a 2 2 Fa C i[0, ), L (K) . ∈ ∞ ∞ K − As 2 (a) + = +(a) + = f L (K) pn(f) = 0 n 0 and D − F D D ∈H ∈ 2 | ∀ ≤ (a) = (a) − = f L (K) pn(f) = 0 n 0 , F D− D− ∈HD ∈ −| ∀ ≥ one gets p+ (a) + = (a) + , p (a) = (a) and ◦ F D F D − ◦ F D− F D− p (a) + =0= p+ (a) . − ◦ F D ◦ F D− Therefore, F (0) commutes with p+ and p . −

98 2 Next, take a function F2 C i[0, ), L (K) that commutes with p+ and p . An ∈ ∞ ∞ K − − operator ﬁeld ϕF2 over Godd meeting the Properties 1 to 5 of Theorem 4.20 has to be constructed. Deﬁne b

,x 2 ϕF − := F2(x) L (K) x i(0, ), 2 P ∈B − ∀ ∈ ∞ ϕF2 + := p+ F2(0) + and D ◦ ∈B HD ϕF2 := p F2(0) − . D− − ◦ ∈BHD Here again, the Properties 1 to 4 are obviously fulﬁlled. For Property 5, let (uj)j N be a se- ∈ quence in i(0, ) converging to 0. Then, as F2(0) commutes with p+ and p , by the deﬁnition ∞ − 2 2 ofν ˜ in Case (i) of Remark 4.19 in Section 4.4 and since p+ + p = idL (K)− L (K)− , − →

,uj ϕF − ν˜ ϕF = F2(uj) ϕF + p+ ϕF p 2 P − 2 op − 2 D ◦ − 2 D− ◦ − op = F2(uj) p+ F2(0) p+ p F2(0) p − ◦ ◦ − − ◦ ◦ − op = F2(uj) F2(0) p+ + p − ◦ − op j = F (u ) F (0) →∞ 0 k 2 j − 2 kop −→

because of the continuity of F2 in 0.

Now, take the infinite-dimensional and separable Hilbert space for the representa- + HD tions m and m− for m > 1, fixed in Section 4.2.2. Then, define for every a C∗(G) the D 3 D ∈ function Fa : I3 by →B HD F 3(x) := (a) + x Z and a F Dx ∀ ∈ >1 3 Z Fa (x) := (a) −x x < 1. F D− ∀ ∈ − Here, Property 5 of Theorem 4.20 does not emerge and the Properties 1 to 4 are obvious.

Taking a function F3 C Z 1, 0, 1 , , one has to choose ∈ ∞ \ {− } K HD + Z ϕF3 x := F3(x) x >1 and D ∈B HD ∀ ∈ Z ϕF3 x− := F3( x) x >1 D − ∈B HD ∀ ∈ and it is again easy to verify that ϕF complies with the properties of Theorem 4.20.

99 5 On the dual topology of the groups U(n) ⋉Hn

In this section, the semidirect product G = U(n) ⋉H for n N will be analyzed and the n n ∈ ∗ topology of its spectrum shall be described. The first part contains some preliminary definitions and results about the spectrum and the space of all admissible coadjoint orbits of Gn. In Section 5.2, the topology of the orbit space will be examined and thereafter, in Section 5.3, the topology of the spectrum will be discussed. Finally, in Section 5.4, the results of the two preceding subsections will be combined and in that way, the topology of the spectrum of Gn shall be linked to the one of the space of its admissible coadjoint orbits. Until now, this succeeds only partly, as the characterization of the spectrum of Gn is not yet entirely finished. As already mentioned in the introduction, this section is based on a preprint by M.Elloumi and J.Ludwig which can be found in the doctoral thesis of M.Elloumi (see [11], Chapter 3). In the present thesis, it has been elaborated, completed and several important changes have been made. Besides numerous minor changes, the main modifications of this work compared to the preprint by M.Elloumi and J.Ludwig can be found in the proofs of Theorem 5.4(5) and Theorem 5.10(1) and (2). Furthermore, there were some details added in the proof of Theo- rem 5.16. Theorem 5.11 and Proposition 5.12 with their proofs and Conjecture 5.13 were appended in this thesis. In the second implication of Theorem 5.4(5)(i) and (ii) (see Section 5.2), there were mistakes in the preprint which caused the existing proof only to be valid in a particular case, namely k for λk →∞ (compare the second implication of the proof of Theorem 5.4(5)(i)) and for n −→ −∞ k k λ1 →∞ , respectively (compare the second implication of the proof of Theorem 5.4(5)(ii)). −→ ∞ k k Hence, the proofs for the case λk →∞ and λk →∞ , respectively, had to be elaborated n 6−→ −∞ 1 6−→ ∞ and for this reason, Lemma 5.3(2) and its proof were also added in this thesis. Concerning Theorem 5.10, there were further mistakes in both parts, (i) and (ii), of its proof which lead to the necessity to completely revise it and to construct another proof.

5.1 Preliminaries Let Cn be the n-dimensional complex vector space equipped with the standard scalar product .,. Cn given by h i n n x,y Cn = x y x =(x ,...,x ),y =(y ,...,y ) C . h i j j ∀ 1 n 1 n ∈ Xj=1

Moreover, let (.,.)Cn and ω(.,.)Cn denote the real and the imaginary part of .,. Cn , respec- h i tively, i.e. .,. Cn =(.,.)Cn + iω(.,.)Cn . h i The bilinear forms (.,.)Cn and ω(.,.)Cn deﬁne a positive deﬁnite inner product and a sym- plectic structure on the underlying real vector space R2n of Cn, respectively. The associated Heisenberg group H = Cn R of dimension 2n+1 over R is given by the group multiplication n × 1 (z,t)(z′,t′) := z + z′,t + t′ ω(z, z′)Cn (z,t), (z′,t′) H . − 2 ∀ ∈ n

100 Furthermore, consider the unitary group U(n) of automorphisms of Hn preserving .,. Cn on n h i C which embeds into Aut(Hn) via

A.(z,t):=(Az, t) A U(n) (z,t) H . ∀ ∈ ∀ ∈ n

Then, U(n) yields a maximal compact connected subgroup of Aut(Hn) (see [14], Theorem 1.22 and [20], Chapter I.1). Moreover, Gn = U(n) ⋉Hn denotes the semidirect product of U(n) with the Heisenberg group Hn equipped with the group law 1 (A, z, t)(B, z′,t′) := AB, z + Az′,t + t′ ω(z, Az′)Cn (A, z, t), (B, z′,t′) G . − 2 ∀ ∈ n The Lie algebra hn of Hn will be identiﬁed with Hn itself via the exponential map. The Lie bracket of hn is deﬁned as

(z,t), (w,s) := 0, ω(z,w)Cn (z,t), (w,s) h − ∀ ∈ n and the derived action of the Lie algebra u(n) of U(n) on hn is

A.(z,t):=(Az, 0) A u(n) (z,t) h . ∀ ∈ ∀ ∈ n Denoting by g = u(n) ⋉ h the Lie algebra of G , for all (A, z, t) G and all (B,w,s) g , n n n ∈ n ∈ n one gets d Ad (A, z, t)(B,w,s) = Ad (A, z, t) eyB,yw,ys (33) dy y=0

1 = ABA ∗, ABA∗z + Aw, s ω(z, Aw)Cn + ω(A∗z, BA∗z)Cn , − − 2 where A∗ is the adjoint matrix of A. In particular

Ad (A)(B,w,s)=(ABA∗, Aw, s). (34)

From Identity (33), one can deduce the Lie bracket

d (A, z, t), (B,w,s) = Ad eyA,yz,yt (B,w,s) dy y=0

= AB BA, Aw Bz, ω(z,w)Cn − − − for all (A, z, t), (B,w,s) g . ∈ n

5.1.1 The coadjoint orbits of Gn

In this subsection, the coadjoint orbit space of Gn will be described according to [3], Section 2.5.

In the following, u(n) will be identiﬁed with its vector dual space u∗(n) with the help of the U(n)-invariant inner product

A, B := tr(AB) A, B u(n). h iu(n) ∀ ∈

101 For z Cn deﬁne the linear form z in (Cn) by ∈ ∗ ∗ n z∗(w) := ω(z,w)Cn w C . ∀ ∈ Furthermore, one deﬁnes a map : Cn Cn u (n), (z,w) z w by × × −→ ∗ 7→ ×

z w(B) := w∗(Bz)= ω(w, Bz)Cn B u(n). × ∀ ∈ One can verify that for A U(n), B u(n) and z,w Cn, ∈ ∈ ∈ 1 Az∗ := z∗ A− = (Az)∗, (35) ◦ z∗ B = (Bz)∗, ◦ − z w = w z and × × A(z w)A∗ = (Az) (Aw). × × Hence, the dual g = u(n) ⋉ h ∗ will be identiﬁed with u(n) h , i.e. each element ℓ g n∗ n ⊕ n ∈ n∗ can be identiﬁed with an element (U, u, x) u(n) Cn R such that ∈ × ×

(U, u, x), (B,w,s) = U, B u(n) + u∗(w)+ xs (B,w,s) gn. gn h i ∀ ∈ From (34) and (35), one obtains for all A U(n), ∈ n Ad ∗(A)(U,u,x)=(AUA∗, Au, x) (U, u, x) u(n) C R (36) ∀ ∈ × × and for all (A, z, t) G and all (U, u, x) u(n) Cn R, ∈ n ∈ × × x Ad ∗(A, z, t)(U,u,x)= AUA∗ + z (Au)+ z z, Au + xz,x , (37) × 2 × where z w(B)= w (Bz)= ω(w, Bz)Cn . × ∗ Letting A and z vary over U(n) and Cn, respectively, the coadjoint orbit of the linear O(U,u,x) form (U,u,x) can then be written as

x n = AUA∗ + z (Au)+ z z, Au + xz,x A U(n), z C O(U,u,x) × 2 × ∈ ∈ 

or equivalently, by replacing z by Az and using Identity (36),

x n = Ad ∗(A) U + z u + z z,u + xz,x A U(n), z C . O(U,u,x) × 2 × ∈ ∈ T t Here, z is regarded as a column vector z =(z1, . . . , zn) and z ∗ := z . One can show as follows that z u u∗(n) = u(n) is the n n skew-Hermitian matrix i × ∈ ∼ × 2 (uz∗ + zu∗): For all B u(n) ∈

uz∗ + zu∗,B = tr (uz∗ + zu∗)B = B z u u B z = 2iz u(B). u(n) ji i j − i ij j − × 1 i,j n 1 i,j n ≤X≤ ≤X≤ In particular, z z is the skew-Hermitian matrix izz whose entries are determined by × ∗ (izz∗)lj = izlzj.

102 5.1.2 The spectrum of U(n) As in the preceding sections, within Section 5, the representations are identiﬁed with their equivalence classes again.

First of all, as U(n) is a compact group, one knows that its spectrum is discrete and that every representation of U(n) is ﬁnite-dimensional.

Now, let eiθ1 ... 0 T . .. . R n = T =  . . .  θj j 1,...,n  ∈ ∀ ∈ { }  0 ... eiθn      be a maximal torus of the unitary group U(n) and let tn be its Lie algebra. By complexiﬁcation  C  of u(n) and tn, one gets the complex Lie algebras u (n)= gl(n, C)= M(n, C) and

h1 ... 0 C . .. . C tn = H =  . . .  hj j 1,...,n  , ∈ ∀ ∈ { }  0 ... h   n    respectively, which is a Cartan subalgebra of uC( n). For j 1,...,n , define a linear functional  ∈ { }  ej by h1 ... 0 . .. . ej  . . .  := hj. 0 ... h  n    Let Pn be the set of all dominant integral forms λ for U(n) which may be written in the n form iλjej, or simply as λ = (λ1, ,λn), where λj are integers for every j 1,...,n j=1 ··· ∈ { } such thatP λ λ . P is a lattice in the vector dual space t of t , fulfilling P = Zn. 1 ≥···≥ n n n∗ n n ∼ Since each irreducible unitary representation (τλ, λ) of U(n) is determined by its highest [ H weight λ P , the spectrum U(n) of U(n) is in bijection with the set P . ∈ n n λ For each λ in Pn, the highest vector φ in the corresponding Hilbert space λ of τλ, λ λ H verifies τλ(T )φ = χλ(T )φ , where χλ is the character of Tn associated to the linear functional λ and defined by eiθ1 ... 0 . . . iλ1θ1 iλnθn χλ T = . .. . := e− e− .   . .  ··· 0 ... eiθn       Moreover, for two irreducible unitary representations (τ , ) and (τ ′ , ′ ), the Schur or- λ Hλ λ Hλ thogonality relation states that for all ξ,η and all ξ ,η ′ , ∈Hλ ′ ′ ∈Hλ 0 if λ = λ′, τλ(g)ξ,η τλ′ (g)ξ′,η′ dg = ξ,ξ′ η′,η 6 (38) λ λ′ Hλ Hλ H H h i h i if λ = λ′, Z ( dλ U(n) where dλ denotes the dimension of the representation τλ.

103 According to [18], Chapter 1, if ρµ is an irreducible representation of U(n 1) with highest U(n) − weight µ =(µ1,...,µn 1), the induced representation πµ := ind ρµ of U(n) decomposes − U(n 1) with multiplicity one, and the representations of U(n) that appear− in this decomposition are exactly those with the highest weight λ =(λ1,...,λn) such that

λ1 µ1 λ2 µ2 ... λn 1 µn 1 λn. (39) ≥ ≥ ≥ ≥ ≥ − ≥ − ≥

5.1.3 The spectrum and the admissible coadjoint orbits of Gn

The description of the spectrum of Gn is based on a method by Mackey (see [28], Chapter 10), which states that one has to determine the irreducible unitary representations of the subgroup Hn in order to construct representations of Gn.

First, regard the inﬁnite-dimensional irreducible representations of the Heisenberg group Hn, which are parametrized by R∗: For each α R , the coadjoint orbit of the irreducible representation σ is the hyperplane ∈ ∗ Oα α = (z,α) z Cn . Since for every α, this orbit is invariant under the action of U(n), the Oα | ∈ unitary group U(n) preserves the equivalence class of σ . α The representation σα can be realized in the Fock space

n 2 |α| w 2 α(n)= f : C C entire f(w) e− 2 | | dw < F ( −→ | | ∞) CZn

α α iαt z 2 w,z Cn σα(z,t)f(w) := e − 4 | | − 2 h i f(w + z) for α> 0 and α α iαt+ z 2+ w,z Cn σα(z,t)f(w) := e 4 | | 2 h i f(w + z) for α< 0.

See [14], Chapter 1.6 or [19], Section 1.7 for a discussion of the Fock space.

For each A U(n), the operator W (A) deﬁned by ∈ α 1 n W (A): (n) (n), W (A)f(z) := f A− z f (n) z C α Fα → Fα α ∀ ∈ Fα ∀ ∈ intertwines σα and (σα)A given by (σα)A(z,t) := σα(Az, t). Wα is called the projective intertwining representation of U(n) on the Fock space. Then, by [28], Chapter 10, for each α R∗ [ ∈ and each element τλ in U(n),

π (A, z, t) := τ (A) σ (z,t) W (A) (A, z, t) G (λ,α) λ ⊗ α ◦ α ∀ ∈ n is an irreducible unitary representation of G realized in := (n), where is n H(λ,α) Hλ ⊗ Fα Hλ the Hilbert space of τλ. Associate to π the linear functional ℓ := (J , 0,α) g given by (λ,α) λ,α λ ∈ n∗

iλ1 ... 0 . .. . Jλ :=  . . .  . 0 ... iλ  n   

104 Denote by Gn[ℓλ,α], U(n)[ℓλ,α] and Hn[ℓλ,α] the stabilizers of ℓλ,α in Gn, U(n) and Hn, respectively. By Formula (37),

i G [ℓ ] = (A, z, t) G AJ A∗ + αzz∗,αz,α =(J , 0,α) n λ,α ∈ n λ 2 λ = (A, 0,t) G AJ A∗ = J , ∈ n| λ λ U(n)[ℓ ] = A U(n) (AJ A∗, 0,α)=(J , 0,α) λ,α ∈ | λ λ = A U(n) AJ A∗ = J and ∈ | λ λ i H [ℓ ] = (z,t) H J + αzz ∗,αz,α =(J , 0,α) = 0 R. n λ,α ∈ n λ 2 λ { }×

It follows that Gn[ℓλ,α]= U(n)[ℓλ,α] ⋉Hn[ℓλ,α]. Hence, ℓλ,α is aligned in the sense of Lipsman (see [25], Lemma 4.2).

The ﬁnite-dimensional irreducible representations of H are the characters χ for v Cn, n v ∈ deﬁned by

i(v,z)Cn χ (z,t) := e− (z,t) H . v ∀ ∈ n

Denote by U(n)v the stabilizer of the character χv, or equivalently of the vector v, under the action of U(n). Then, for every irreducible unitary representation ρ of U(n)v, the tensor product ρ χ is an irreducible representation of U(n) ⋉H whose restriction to H is a ⊗ v v n n multiple of χv, and the induced representation

U(n)⋉Hn π(ρ,v) := ind ⋉H ρ χv U(n)v n ⊗ is an irreducible representation of Gn. Furthermore, the restriction of π(ρ,v) to U(n) is equivalent to indU(n) ρ. U(n)v For any v = Av for A U(n) (i.e. v and v belong to the same sphere centered at 0 and ′ ∈ ′ of radius r = v Cn ), one has U(n)v′ = AU(n)vA∗ and thus, the representation π(ρ,v) is k k \ equivalent with π ′ ′ for any ρ U(n) ′ such that ρ (B)= ρ(A BA) for each B U(n) ′ . (ρ ,v ) ′ ∈ v ′ ∗ ∈ v Hence, one can regard the character χr associated to the linear form vr which is identiﬁed T n with the vector (0,..., 0,r) in C . Throughout this text, denote by ρµ the representation of the subgroup U(n 1) = U(n) with highest weight µ and by π the representation − vr (µ,r) π = indGn ρ χ . Its Hilbert space is given by (ρµ,vr) U(n 1)⋉Hn µ r (µ,r) − ⊗ H = L2 G / U(n 1) ⋉H , ρ χ . H(µ,r) n − n µ ⊗ r Again, π is linked to the linear functional ℓ := (J ,v , 0) g for (µ,r) µ,r µ r ∈ n∗

iµ1 ... 0 0 ...... Jµ :=   . 0 ... iµn 1 0  −   0 ... 0 0     

105 By (37), one can check that

G [ℓ ] = (A, z, t) G AJ A∗ + z (Av ), Av , 0 = J ,v , 0 n µ,r ∈ n µ × r r µ r n i o = (A, z, t) G A U(n 1), AJ A∗ + v z∗ + zv∗ = J ∈ n ∈ − µ 2 r r µ = (A, z, t) G z iRv ,A U(n 1), AJ A∗ = J , ∈ n| ∈ r ∈ − µ µ since AJ A u(n 1) and µ ∗ ∈ − 0 ... 0 rz1 . .. . .  . . . .  vrz∗ + zvr∗ = . 0 ... 0 rzn 1  −   rz1 ... rzn 1 2r Re(zn)   −    In addition,

U(n)[ℓ ] = A U(n 1) AJ A∗ = J and µ,r ∈ − | µ µ H [ℓ ] = iRv R. n µ,r r × Thus, similarly to ℓλ,α, the linear functional ℓµ,r is aligned.

One obtains in this way all the ﬁnite-dimensional irreducible unitary representations of Gn which are not trivial on Hn. [ On the other hand, the trivial extension of each element τλ of U(n) to the entire group Gn is an irreducible representation which will also be denoted by τλ. The corresponding linear functional is ℓλ := (Jλ, 0, 0).

Therefore, by Mackey’s theory, the spectrum Gn consists of the following families of representations: b (i) π for λ P and α R = R 0 , (λ,α) ∈ n ∈ ∗ \{ } R (ii) π(µ,r) for µ Pn 1 and r >0 and ∈ − ∈ (iii) τ for λ P . λ ∈ n Hence, Gn is in bijection with the set

(Pn R∗) Pn 1 R>0 Pn. c × ∪ − × ∪ A linear functional ℓ in gn∗ is deﬁned to be admissible if there exists a unitary character χ of the connected component of Gn[ℓ] such that dχ = iℓ gn[ℓ]. A calculation shows that all the | linear functionals ℓλ,α, ℓµ,r and ℓλ are admissible. Then, according to [25], the representations π(λ,α), π(µ,r) and τλ described above are equivalent to the representations of Gn obtained by holomorphic induction from their respective linear functionals ℓλ,α, ℓµ,r and ℓλ. Denote by , and the coadjoint orbits associated to the linear forms ℓ , ℓ O(λ,α) O(µ,r) Oλ λ,α µ,r and ℓ , respectively. Let gn‡ g be the union of all the elements in , and λ ⊂ n∗ O(λ,α) O(µ,r) Oλ and denote by gn‡ /Gn the corresponding set in the orbit space. Now, from [25] follows that gn‡ is the set of all admissible linear functionals of gn.

106 5.2 Convergence in the quotient space gn‡ /Gn

According to the last subsection, the spectrum of Gn is parametrized by the dominant integral forms λ for U(n) and µ for U(n 1), the non-zero α R attached to the generic − ∈ orbits in h and the positive real r derived from the natural action of the unitary group Oα n∗ U(n) on the characters of the Heisenberg group Hn. Moreover, it has been elaborated that the quotient space gn‡ /Gn of admissible coadjoint orbits is in bijection with Gn. Now, the convergence of the admissible coadjoint orbits will be linked to the convergence in \ [ the parameter space α cR ,r> 0, ρ U(n 1),τ U(n) . ∈ ∗ µ ∈ − λ ∈ n o i Letting be the subspace of u(n) generated by the matrices z vr = 2 (vrz∗ + zvr∗) Wn × for z C , the space gn‡ /G is the set of all orbits ∈ n

iα n = AJ A∗ + zz∗,αz,α z C ,A U(n) , O(λ,α) λ 2 ∈ ∈ = A(J + )A∗, Av , 0 A U(n) and O(µ,r) µ W r ∈ = n(AJ A∗, 0, 0) A U(n) o Oλ λ ∈ R R for α ∗, r >0, µ Pn 1 and λ Pn . ∈ ∈ ∈ − ∈ Before beginning the discussion on the convergence of the admissible coadjoint orbits, the following preliminary lemmas are needed:

Lemma 5.1. For n N∗ and for any scalars X1,...,Xn and Y1,...,Yn 1 fulﬁlling Yi = Yj for i = j, one has ∈ − 6 6 n (Xi Yj) n 1 i=1 − n n 1 − i=k − Q6 = X Y n 1 j j − − j=1 (Yi Yj) j=1 j=1 X Xj=k X i=1 − 6 i=j Q6 for each k 1,...,n . ∈ { } Proof: For n = 1, the formula is trivial. So, let n> 1 and assume that the assertion is true for this n. For k = n + 1, a simple calculation gives the result. If k = n + 1, one gets 6

107 n+1 (Xi Yj) n i=1 − i=k Q6 n j=1 (Yi Yj) X i=1 − i=j Q6 n+1 n+1 (Xi Yn) (Xi Yj) i=1 − n 1 i=1 − i=k − i=k = Q6 + Q6 n 1 n − j=1 (Y Y ) (Yi Yn) X i j i=1 − i=1 − i=j Q Q6 n n (Xi Yn) (Xi Yj) i=1 − n 1 i=1 − i=k − i=k (Xn+1 Yj) = (X Y ) Q6 + Q6 − n+1 n n 1 n 1 − − − · Yn Yj (Yi Yn) Xj=1 (Yi Yj) − i=1 − i=1 − i=j Q Q6 n n n (Xi Yn) (Xi Yj) (Xi Yj) i=1 − n 1 i=1 − n 1 i=1 − i=k − i=k (Xn+1 Yn) − i=k = (X Y ) Q6 + Q6 + Q6 n+1 n n 1 n 1 − n 1 − − − · Yn Yj − (Yi Yn) Xj=1 (Yi Yj) − Xj=1 (Yi Yj) i=1 − i=1 − i=1 − i=j i=j Q Q6 Q6 n n−1 = P Xj P Yj j=1 −j=1 | j=k {z } 6 n (Xi Yj) n i=1 − n n 1 n+1 n i=k − = (X Y ) Q6 + X Y = X Y n+1 − n n j − j j − j j=1 (Yi Yj) j=1 j=1 j=1 j=1 X i=1 − Xj=k X Xj=k X i=j 6 6 Q6 =1 by [12], Lemma 5.3 and the claim is shown.| {z }

Lemma 5.2. Let µ Pn 1 and λ Pn. Then, λ1 µ1 λ2 ... µn 1 λn if and only if there is a ∈ − ∈ ≥ ≥ ≥ ≥ − ≥ skew-Hermitian matrix 0 0 ... 0 z − 1 0 0 ... 0 z2  . . . . −.  B = ......    0 0 ... 0 zn 1   − −   z1 z2 ... zn 1 ix   −    in such that A(J + B)A = J for an element A U(n). W µ ∗ λ ∈

108 Proof: For y R, a computation shows that det(J + B iyI)=( i)nP (y), where ∈ µ − − n 1 n 1 n 1 − − − P (y):=(y x) (y µ ) z 2 (y µ ) . − − i − | j| − i i=1 j=1 i=1 Y X Yi=j 6 y Furthermore, one can observe that P (y) →∞ and that P (µ ) 0 if j is odd and P (µ ) 0 −→ ∞ j ≤ j ≥ if j is even. Now, if A(J +B)A = J for an element A U(n), by the spectral theorem, iλ ,iλ , ,iλ µ ∗ λ ∈ 1 2 ··· n are all the elements of the spectrum of Jµ + B fulﬁlling λ1 µ1 λ2 ... µn 1 λn. ≥ ≥ ≥ ≥ − ≥ Conversely, suppose ﬁrst that all the µ for j 1,...,n 1 are pairwise distinct. In this j ∈ { − } case, let B be the skew-Hermitian matrix with the entries z1, , zn 1,x satisfying ··· − n (λ µ ) i − j z 2 = i=1 for every j 1,...,n 1 and j n 1 | | − Q− ∈ { − } (µi µj) i=1 − i=j Q6 n n 1 − x = λ µ . j − j Xj=1 Xj=1 From Lemma 5.1, n (λi µj) n 1 n n 1 n 1 i=1 − n 1 − − −  i=k −  P (λ ) = µ λ (λ µ ) + Q6 (λ µ ) k j j k i n 1 k i − ! −  − −  j=1 j=1 i=1 j=1  (µi µj) i=1  X Xj=k Y X  − Y  6  i=1   iQ=j   6  n (λi µj) n 1 n 1 n n 1 i=1 − −  − − i=k  = (λ µ ) µ λ + Q6 = 0. k i j j n 1 −  − −  i=1 j=1 j=1 j=1 (µi µj) Y X Xj=k X −   6 i=1   iQ=j   6  Hence, the spectrum of the matrix J + B is the set iλ ,iλ , ,iλ and thus, the spectral µ { 1 2 ··· n} theorem implies that A(J + B)A = J for an element A U(n). µ ∗ λ ∈ Now, if the µ for j 1,...,n 1 are not pairwise distinct, there exist two families of integers j ∈ { − } p 1 l s and q 1 l s such that 1 p

p1 p2 n 1 − and Q (y) := (y µ ). j ··· − i i=1 i=q1+1 i=qs+1 Yi=j Yi=j Yi=j 6 6 6

109 Then, for

s ql p1 1 p2 1 n 1 − − − P (y):=(y x)Q(y) z 2 Q˜ (y) ... z 2Q (y) , − − | j| l − | j| j Xl=1 jX=pl Xj=1 j=Xq1+1 j=Xqs+1 s I n ql pl one gets det(Jµ + B iy )=( i) (y µpl ) − P (y). − − l=1 − Now, the entries zj of the skew-HermitianQ matrix B can be chosen as follows:

n p1 p2 n (λi µj) (λi µj) 2 i=1 − i=1 i=q1+1 ··· i=qs+1 − zj := = n 1 Qp1 Qp2 nQ1 | | − Q− − − (µi µj) (µi µj) i=1 − i=1 i=q1+1 ··· i=qs+1 − i=j i=j i=j i=j Q6 Q6 Q6 Q6 for each j 1,...,p 1,q + 1,...,p 1,q + 1,...,n 1 and ∈ { 1 − 1 s − s − } p1 p2 n

(λi µpl ) 2 2 2 i=1 i=q1+1 ··· i=qs+1 − zp + ... + zq 1 + zq := l l l p1 Qp2 Q ps Q n 1 | | | − | | | − − (µi µpl ) i=1 i=q1+1 ··· i=qs−1+1 i=qs+1 − i=pl i=pl i=p 6Q 6Q 6Ql Q for each l 1,...,s . The entry x can be deﬁned as ∈ { }

n n 1 p1 p2 n p1 p2 n 1 − − x := λ µ = ... λ ... µ . j − j j − j Xj=1 Xj=1 Xj=1 j=Xq1+1 j=Xqs+1 Xj=1 j=Xq1+1 j=Xqs+1

Then, if λk = µpl , one obviously has P (λk)= Q(λk) = 0. Otherwise, one gets

p1 p2 n p1 p2 n 1 − P (λ ) = λ λ + µ Q(λ ) k k − ··· j ··· j k Xj=1 j=Xq1+1 j=Xqs+1 Xj=1 j=Xq1+1 j=Xqs+1 p1 p2 n

p1 1 p2 1 n 1 (λi µj) − − − i=1 i=q1+1 ··· i=qs+1 − + Qj(λk) Qp1 Qp2 nQ1 ··· − ! Xj=1 j=Xq1+1 j=Xqs+1 (µi µj) i=1 i=q1+1 ··· i=qs+1 − i=j i=j i=j Q6 Q6 Q6 p1 p2 n s (λi µpl ) i=1 i=q1+1 ··· i=qs+1 − + Q˜l(λk) p1 Qp2 Q ps Q n 1 l=1 − ! X (µi µpl ) i=1 i=q1+1 ··· i=qs−1+1 i=qs+1 − i=pl i=pl i=p 6Q 6Q 6Ql Q

110 p1 p2 n 1 p1 p2 n − = Q(λ ) µ λ k ··· j − ··· j j=1 j=q1+1 j=qs+1 j=1 j=q1+1 j=qs+1 X X X Xj=k Xj=k Xj=k 6 6 6 p1 p2 n (λi µj) p1 1 p2 1 n 1 i=1 i=q1+1 ··· i=qs+1 − − − − i=k i=k i=k + Q6 Q6 Q6 p1 p2 n 1 ··· − Xj=1 j=Xq1+1 j=Xqs+1 (µi µj) i=1 i=q1+1 ··· i=qs+1 − i=j i=j i=j Q6 Q6 Q6 p1 p2 n

(λi µpl ) s i=1 i=q1+1 ··· i=qs+1 − i=k i=k i=k + Q6 Q6 Q6 p1 p2 ps n 1 l=1 − ! X (µi µpl ) i=1 i=q1+1 ··· i=qs−1+1 i=qs+1 − i=pl i=pl i=p 6Q 6Q 6Ql Q

p1 p2 n 1 p1 p2 n − = Q(λ ) µ λ k ··· j − ··· j j=1 j=q1+1 j=qs+1 j=1 j=q1+1 j=qs+1 X X X Xj=k Xj=k Xj=k 6 6 6 p1 p2 n (λi µj) p1 p2 n 1 i=1 i=q1+1 ··· i=qs+1 − − i=k i=k i=k + Q6 Q6 Q6 = 0. p1 p2 n 1 ··· − ! Xj=1 j=Xq1+1 j=Xqs+1 (µi µj) i=1 i=q1+1 ··· i=qs+1 − i=j i=j i=j Q6 Q6 Q6 Hence, the spectrum of the matrix J + B equals the set iλ ,iλ , ,iλ . As above, this µ { 1 2 ··· n} completes the proof.

Lemma 5.3.

1. Let λ P , α R and z Cn. Then, the matrix J + i zz admits n eigenvalues ∈ n ∈ ∗ ∈ λ α ∗ iβ ,iβ ,...,iβ in such a way that β λ β λ β λ if α > 0 and 1 2 n 1 ≥ 1 ≥ 2 ≥ 2 ≥···≥ n ≥ n λ β λ β λ β if α< 0. 1 ≥ 1 ≥ 2 ≥ 2 ≥···≥ n ≥ n 2. Let λ,β P . If β λ β λ ... β λ , there exists a number ∈ n 1 ≥ 1 ≥ 2 ≥ 2 ≥ ≥ n ≥ n z Cn in such a way that the matrix J + izz admits the n eigenvalues iβ ,...,iβ . If ∈ λ ∗ 1 n λ β λ β ... λ β , there exists z Cn such that the matrix J izz 1 ≥ 1 ≥ 2 ≥ 2 ≥ ≥ n ≥ n ∈ λ − ∗ admits the n eigenvalues iβ1,...,iβn. Proof: 1 zz∗ 1) One can prove by induction that the characteristic polynomial of the matrix i Jλ + α is λ,z,α equal to Qn deﬁned by n n n z 2 Qλ,z,α(x) := (x λ ) | j| (x λ ). n − i − α − i i=1 j=1 i=1 Y X Yi=j 6

111 λ,z,α x λ,z,α Assume that α is negative. Then, Qn (x) →∞ and Qn (λj) 0 if j is odd and λ,z,α λ,z,α −→x ∞ ≥ λ,z,α x Qn (λj) 0 if j is even. Furthermore, Qn (x) →∞ if n is odd and Qn (x) →−∞ ≤ 1 −→zz −∞∗ −→ ∞ if n is even and therefore, one can deduce that i Jλ + α admits n eigenvalues β1,β2,...,βn verifying λ β λ β λ β . Hence, J + i zz admits the n eigenvalues 1 ≥ 1 ≥ 2 ≥ 2 ≥···≥ n ≥ n λ α ∗ iβ ,iβ ,...,iβ fulﬁlling λ β λ β λ β . 1 2 n 1 ≥ 1 ≥ 2 ≥ 2 ≥···≥ n ≥ n The same reasoning applies when α is positive.

2) Let β1 λ1 β2 λ2 ... βn λn. ≥ n ≥ ≥ ≥ ≥ ≥ 1 λ,z,1 λ,z,1 λ,z For any z C , the characteristic polynomial of J +zz is equal to Qn with Qn =: Qn ∈ i λ ∗ like above. First, assume that β1 >λ1 >...>βn >λn. Let n (λ β ) j − i z 2 := i=1 . | j| − Qn (λj λi) i=1 − i=j Q6 Then, as λj < βi for all i 1,...,j , as λj > βi for all i j + 1,...,n , as λj < λi for all ∈ { } ∈ { 2} ( 1)j i 1,...,j 1 and as λ >λ for all i j + 1,...,n , one gets sgn z =( 1) − = 1 ∈ { − } j i ∈ { } | j| − ( 1)j−1 and thus, this deﬁnition is reasonable. − λ,z One now has to show that Qn (β ) = 0 for all ℓ 1,...,n . ℓ ∈ { } n n n (λ β ) n j − i Qλ,z(β ) = (β λ ) + i=1 (β λ ) n ℓ ℓ − i Qn ℓ − i i=1 j=1 (λj λi) i=1 Y X Yi=j i=1 − 6 i=j Q6 n n n (λj βi)  i=1 −  = (βℓ λi) 1+ n −  Q  i=1  j=1 (λj λi)(βℓ λj) Y  X i=1 − −   i=j   Q6   n  (λj βi) n n i=1 −  i=ℓ  Q6 = (βℓ λi) 1 n = 0, −  −  i=1  j=1 (λj λi) Y  X i=1 −   i=j   Q6    n Q (λj βi) − n i=1 i=ℓ as by [12], Lemma 5.3, one obtains 6 n = 1. j=1 Q (λj λi) i=1 − P i=j 6 Now, regard arbitrary β λ β λ ... β λ . 1 ≥ 1 ≥ 2 ≥ 2 ≥ ≥ n ≥ n For n = 1, one can choose z 2 := (β λ ) 0 and the claim is shown. | 1| 1 − 1 ≥ Let n> 1 and assume that the assertion is true for n 1. −

112 If λℓ 1 = βℓ = λℓ for all ℓ 1,...,n , the claim is already shown above. So, without − 6 6 ∈ { } restriction let ℓ 1,...,n with βℓ = λℓ. The case λℓ 1 = βℓ is very similar. ℓ ∈ { } ℓ − Hence, for λ := (λ1,...,λℓ 1,λℓ+1,...,λn) and β := (β1,...,βℓ 1,βℓ+1,...,βn), − −

β1 λ1 ... βℓ 1 λℓ 1 βℓ+1 λℓ+1 ... βn λn ≥ ≥ ≥ − ≥ − ≥ ≥ ≥ ≥ ≥ n 1 ℓ and thus, by the induction hypothesis, there exists C − z := (z1, ..., zℓ 1, zℓ+1, ..., zn) such ℓ ℓ ∋ − that Qλ ,z (β ) = 0 for all i 1,...,ℓ 1,ℓ + 1,...,n , where n,ℓ i ∈ { − } n n n ℓ ℓ Qλ ,z (x) := (x λ ) z 2 (x λ ). n,ℓ − i − | j| − i i=1 j=1 i=1 Yi=ℓ Xj=ℓ i=Yj,i=ℓ 6 6 6 6

Now, let zℓ := 0, i.e. z := (z1, ..., zℓ 1, 0, zℓ+1, ..., zn). Then, − n n n n Qλ,z(x) = (x λ ) (x λ ) z 2(x λ ) (x λ ) z 2 (x λ ) n − ℓ − i − | j| − ℓ − i − | ℓ| − i i=1 j=1 i=1 i=1 Yi=ℓ Xj=ℓ i=Yj,i=ℓ Yi=ℓ 6 6 6 6 6 n ℓ ℓ = (x λ )Qλ ,z (x) z 2 (x λ ) − ℓ n,ℓ − | ℓ| − i i=1 Yi=ℓ 6 ℓ ℓ = (x λ )Qλ ,z (x). − ℓ n,ℓ λℓ,zℓ λ,z If i 1,...,ℓ 1,ℓ + 1,...,n , then Qn,ℓ (βi) = 0 and thus, Qn (βi) = 0. Furthermore, λ,z ∈ { − } Qn (βℓ) = 0, as βℓ = λℓ. λ,z Therefore, Qn (β ) = 0 for all i 1,...,n and the claim is shown. i ∈ { }

Next, let λ1 β1 λ2 β2 ... λn βn. ≥ ≥ n ≥ ≥ ≥ ≥ 1 λ,z, 1 Then, for any z C , the characteristic polynomial of J zz is equal to Qn − . ∈ i λ − ∗ If λ1 >β1 >...>λn >βn, let n (λ β ) j − i z 2 := i=1 . | j| Qn (λj λi) i=1 − i=j Q6 2 ( 1)j−1 Here, sgn zj = (−1)j−1 = 1 and hence, this deﬁnition is reasonable. | | − The rest of the proof is the same as in the ﬁrst part of (2).

With these lemmas, one can now prove the following theorem which describes the topology of the space of admissible coadjoint orbits of Gn. Theorem 5.4. Let α R∗, r> 0, µ Pn 1 and λ Pn. Then, the following holds: ∈ ∈ − ∈ 1. A sequence of coadjoint orbits k converges to in g‡ /G if and only if (µ ,rk) k N (µ,r) n n k O ∈ O lim rk = r and µ = µ for large k. k →∞ 113 2. A sequence of coadjoint orbits k converges to in g‡ /G if and only if (µ ,rk) k N λ n n k O k ∈ kO (rk)k N tends to 0 and λ1 µ1 λ2 µ2 ... λn 1 µn 1 λn for k large enough. ∈ ≥ ≥ ≥ ≥ ≥ − ≥ − ≥

3. A sequence of coadjoint orbits k converges to the orbit in g‡ /G if (λ ,αk) k N (λ,α) n n k O ∈ O and only if lim αk = α and λ = λ for large k. k →∞

4. A sequence of coadjoint orbits k converges to the orbit in g‡ /G if (λ ,αk) k N (µ,r) n n O ∈ O and only if lim αk = 0 and the sequence (λk,α ) N satisﬁes one of the following k O k k →∞ ∈ conditions: k k r2 (i) For k large enough, αk > 0, λj = µj for all j 1,...,n 1 and lim αkλn = . ∈ { − } k − 2 →∞ k k r2 (ii) For k large enough, αk < 0, λj = µj 1 for all j 2,...,n and lim αkλ1 = . − ∈ { } k − 2 →∞

5. A sequence of coadjoint orbits k converges to the orbit in g‡ /G if (λ ,αk) k N λ n n O ∈ O and only if lim αk = 0 and the sequence (λk,α ) N satisﬁes one of the following k O k k →∞ ∈ conditions: k k k (i) For k large enough, αk > 0, λ1 λ1 ... λn 1 λn 1 λn λn and k ≥ ≥ ≥ − ≥ − ≥ ≥ lim αkλn = 0. k →∞ k k k (ii) For k large enough, αk < 0, λ1 λ1 λ2 λ2 ... λn 1 λn λn and k ≥ ≥ ≥ ≥ ≥ − ≥ ≥ lim αkλ1 = 0. k →∞

k ‡ 6. A sequence of coadjoint orbits λ k N converges to the orbit λ in gn/Gn if and only k O ∈ O if λ = λ for large k. Proof: Examining the shape of the coadjoint orbits listed at the beginning of this subsection, 3) and 6) are clear and Assertion 2) follows immediately from Lemma 5.2. Furthermore, the proof of 1) is similar to that of [12], Theorem 4.2.

4) Assume that k converges to the orbit . Then, there exist a sequence (λ ,αk) k N (µ,r) O ∈ O N Cn (Ak)k in U(n) and a sequence of vectors z(k) k N in such that ∈ ∈ i lim Ak Jλk + z(k)z(k)∗ Ak∗, √2Akz(k),αk =(Jµ,vr, 0). k αk →∞ Let A =(amj)1 m,j n be the limit of a subsequence (As)s I for I N. Then, ≤ ≤ ∈ ⊂ i r lim Jλs + z(s)z(s)∗ = A∗JµA, lim zj(s)= anj for j 1,...,n and lim αs = 0. s s √ s →∞ αs →∞ 2 ∈ { } →∞ n 1 − On the other hand, one has (A∗JµA)mj = i µlalmalj and l=1 2 P s z1(s) z1(s)z2(s) z1(s)zn(s) iλ + i | | i ... i 1 αs αs αs 2 z2(s)z1(s) s z2(s) z2(s)zn(s) i iλ + i | | ... i i  αs 2 αs αs  J s + z(s)z(s)∗ = . λ α . . .. . s  . . . .  2  zn(s)z1(s) zn(s)z2(s) s zn(s)   i i ... iλ + i | |   αs αs n αs    114 n 1 zm(s)zj (s) − r Hence, for m = j, lim = µlalmalj < , and since lim z(s) Cn = = 0, s αs s √2 6 →∞ l=1 ∞ →∞k k 6 r iθ R there is a unique i0 1,...,n such that P lim zi0 (s )= e for a certain number θ and ∈ { } s √2 ∈ →∞ iθ lim zj(s) = 0 for j = i0. One obtains ani = e− and anj = 0 for j = i0, i.e. the matrices A s 0 →∞ 6 6 and A∗JµA can be written in the following way: 0 ∗. · · · ∗. ∗. · · · ∗.  . . 0 . .  . . . . .  . . . . .  A =   and  . . . .   . . 0 . .     0   ∗ · · · ∗ ∗ · · · ∗   0 0 e iθ 0 0   ··· − ···    i0−th position |{z} 0 ∗ · · · ∗ ∗ · · · ∗ . . . . .  . . . . .  0  ∗ · · · ∗ ∗ · · · ∗  A∗J A =  0 0 0 0 0  i th position µ  ··· ··· } 0 −  0   ∗ · · · ∗ ∗ · · · ∗   . . . . .   . . . . .     0   ∗ · · · ∗ ∗ · · · ∗    i0−th position |{z}n 1 since (A J A) = (A J A) = i − µ a a = 0 for j 1,...,n . It follows that ∗ µ i0j ∗ µ ji0 l lj li0 − − l=1 ∈ { } 2 s zi0 (s) P s lim λ + | | = 0, which in turn implies that lim λ = and that for each j = i0 s i0 αs s i0 →∞ →∞ ∞ 6 n 1 − 2 s 2 zj(s)zi0 (s) zj(s) zj(s) lim λ = µl alj , lim = 0, lim = 0 and lim | | = 0. s j s s s →∞ | | →∞ αs →∞ αs →∞ αs Xl=1 This proves that i0 can only take the value 1 if αs < 0 and n if αs > 0. Otherwise, since s s s s s λ λ λ , one gets lim λ = if αs < 0 and lim λ = if αs > 0 which i0 1 i0 i0+1 s i0 1 s i0+1 − ≥ ≥ →∞s − ∞ →∞ −∞ contradicts the fact that lim λ is ﬁnite for all j = i0. s j →∞ 6

Case i0 = n: n 1 s r2 s − 2 In this case, one has lim αsλ = and lim λ = µl alj for all j 1,...,n 1 . s n 2 s j →∞ − →∞ l=1 | | ∈ { − } Furthermore, the matrices A and A∗JµA have the form P 0 ... 0 . ∗. ∗. . A˜ . . . . A =   and A∗JµA =   , 0 ... 0    ∗ ∗   0 0 e iθ   0 ... 0 0   ··· −        115 i where A˜ U(n 1). However, the limit matrix of the subsequence J s + z(s)z(s) λ αs ∗ s I ∈ − ∈ has to be diagonal because lim zm(s)zj (s) = 0 for all m = j. This implies that s αs →∞ 6

iµ1 ... 0 0 ...... A∗JµA =   0 ... iµn 1 0  −   0 ... 0 0      and consequently, λs = µ for large s and j 1,...,n 1 . j j ∈ { − }

Case i0 = 1: n 1 s r2 s − 2 In this case, lim αsλ = and lim λ = µl alj for every j 2,...,n . Moreover, s 1 2 s j →∞ − →∞ l=1 | | ∈ { } there is an element A˜ U(n 1) such that theP matrix A is given by ∈ − 0 0 0 0 ··· . 0 . A˜ A =   and hence, A∗JµA =  . ∗. · · · ∗.  . 0 . . .      e iθ 0 0   0   − ···   ∗ · · · ∗      s Using the same arguments as above, one has λj+1 = µj for s large enough and for every j 1,...,n 1 . ∈ { − }

Conversely, suppose that lim αk = 0. If the regarded sequence of orbits satisﬁes the k →∞ 0 . . ﬁrst condition, one can take z(k) :=   and Ak := I for k N and N N large 0 ≥ ∈    α λk   − k n  enough. In the other case, one lets   p k 0 1 0 0 αkλ − 1 ···.  q 0   0 0 1 .. 0  ...... z(k) :=  .  and Ak :=  ......  for k N.     ≥  .   ..   .   0 0 0 . 1       0   1 0 0 0     ···      i Thus, lim Ak Jλk + z(k)z(k)∗ Ak∗, √2Akz(k),αk =(Jµ,vr, 0). k αk →∞ 5) Suppose that k converges to the orbit . Then, there exist a sequence (λ ,αk) k N λ O ∈ O N Cn (Ak)k in U(n) and a sequence z(k) k N in such that ∈ ∈ i lim Ak Jλk + z(k)z(k)∗ Ak∗, √2Akz(k),αk =(Jλ, 0, 0). k αk →∞

116 Cn It follows that lim αk = 0 and that z(k) N tends to 0 in . Denote by A =(amj)1 m,j n k k ≤ ≤ →∞ ∈ N the limit matrix of a subsequence (As)s Ifor an index set I . Then, ∈ ⊂ n i lim Jλs + z(s)z(s)∗ = A∗JλA with (A∗JλA)mj = i λlalmalj. s →∞ αs Xl=1 Since lim αk = 0, one can assume that αs is either strictly positive for all s I or strictly k ∈ negative→∞ for all s I. ∈ zm(s)zj (s) Let αs be the square root of αs . The fact that lim is ﬁnite for all m, j 1,...,n s αs | | | | →∞ ∈ { } zi0 (s) impliesp that there exists at most one integer 1 i0 n such that lim = . Therefore, s √ αs ≤ ≤ →∞ | | ∞ z (s) lim j s α →∞ | s| exists for all j distinct from i0. Hence, for thep same reasons as in the proof of 4), necessarily i 1,n . 0 ∈ { } If there is no such i , then there exists for all j 1,...,n an integer λ Z such that λ = λs 0 ∈ { } j′ ∈ j′ j zj (s) for all s I (by passing to a subsequence if necessary) andz ˜j := lim is ﬁnite for all s √ αs ∈ →∞ | | j 1,...,n . Thus, one gets ∈ { }

A J A = J ′ + iz˜(˜z) , if α > 0 s I ∗ λ λ ∗ s ∀ ∈ (A∗JλA = Jλ′ iz˜(˜z)∗, if αs < 0 s I. − ∀ ∈ It follows by Lemma 5.3 applied toz ˜ and α = 1 or α = 1, respectively, that − s s s λ1 λ1′ = λ1 λ2 λ2′ = λ2 ... λn λn′ = λn, if αs > 0 s I ≥ s ≥ ≥ s ≥ ≥ ≥ s ∀ ∈ (λ1′ = λ1 λ1 λ2′ = λ2 λ2 ... λn′ = λn λn, if αs < 0 s I. ≥ ≥ ≥ ≥ ≥ ≥ ∀ ∈

Case i0 = n: 2 s s zn(s) s In this case, lim αsλ = 0, as lim λ + | | < . Furthermore, lim λ = and s n s n αs ∞ s n −∞ n →∞ →∞ →∞ s 2 lim λ = λl alj for every j 1,...,n 1 and αs has to be positive for large s. Since s j →∞ l=1 | | ∈ { − } zj (s) zn(s) zj (s) lim existsP and lim | | = , it follows that lim = 0 for all j 1,...,n 1 . s αs s αs ∞ s αs ∈ { − } Now,→∞ choose →∞ →∞

2 s zn(s) s zj(s)zn(s) x := lim λ + | | , λ′ := lim λ and wj := i lim j 1,...,n 1 . s n j s j s →∞ αs →∞ − →∞ αs ∀ ∈ { − } i Then, the limit matrix A J A of the sequence J s + z(s)z(s) has the form ∗ λ λ αs ∗ s I ∈ iλ 0 ... 0 w 1′ − 1 0 iλ2′ ... 0 w2  . . . . −.  ......    0 0 ... iλn′ 1 wn 1   − − −   w1 w2 ... wn 1 ix   −    117 By Lemma 5.2, one obtains λ1 λ1′ λ2 λ2′ ... λn 1 λn′ 1 λn, and thus, s s ≥ s ≥ ≥ s ≥ ≥ − ≥ − ≥ λ1 λ1 λ2 λ2 ... λn 1 λn 1 λn λn for large s. ≥ ≥ ≥ ≥ ≥ − ≥ − ≥ ≥

Case i0 = 1: 2 s s z1(s) Here, lim αsλ = 0, since lim λ + | | < . Moreover, s 1 s 1 αs →∞ →∞ ∞ n s s 2 zj(s) lim λ = , lim λ = λl alj and lim = 0 j 2,...,n . s 1 s j s →∞ ∞ →∞ | | →∞ αs ∀ ∈ { } Xl=1 Hence, αs < 0 for s large enough. If one sets 2 s z1(s) s z1(s)zj+1(s) x := lim λ + | | , λ′ := lim λ and wj := i lim j 1,...,n 1 , s 1 j s j+1 s →∞ αs →∞ − →∞ αs ∀ ∈ { − } i the limit matrix A J A of J s + z(s)z(s) can be written as follows: ∗ λ λ αs s I ∈ ix w1 w2 ... wn 1 iλ1′ 0 ... 0 w1 − − w iλ 0 ... 0 0 iλ ... 0 w  − 1 1′   2′ − 2  w2 0 iλ′ ... 0 ˜ ...... ˜ − 2 = A∗ . . . . . A, (40)  ......     . . . . .   0 0 ... iλn′ 1 wn 1     − − −   wn 1 0 0 ... iλn′ 1   w1 w2 ... wn 1 ix   − − −   −  where    0 1 0 0 ···.  0 0 1 .. 0  ˜ . . . . . A =  ......  .    ..   0 0 0 . 1     1 0 0 0   ···  s s  s  s This proves that λ1 λ1 λ2 λ2 ... λn 1 λn 1 λn λn for large s. ≥ ≥ ≥ ≥ ≥ − ≥ − ≥ ≥

Conversely, suppose that the sequence k satisfies the first condition. (λ ,αk) k N O ∈ k k First, consider the case λn →∞ . −→ −∞s N s Then, there is a subsequence (λ )s I for an index set I fulfilling λj = λj′ for every ∈ ⊂ j 1,...,n 1 and all s I. By Lemma 5.2, there exist w1,w2,...,wn 1 C, x R and ∈ { − } ∈ − ∈ ∈ A U(n) such that ∈ iλ 0 ... 0 w 1′ − 1 0 iλ2′ ... 0 w2  . . . . −.  A∗JλA = ......    0 0 ... iλn′ 1 wn 1   − − −   w1 w2 ... wn 1 ix   −   n  n 1 k k k − In this case, λ = λ for large k, as λn →∞ . Choose x := λj λj′ . (Compare the 6 −→ −∞ j=1 − j=1 proof of Lemma 5.2.) It follows that P P n α (x λs )= α (λ λs) > 0. s − n s j − j Xj=1 118 Cn Furthermore, define the sequence z(s) s I in by ∈ s αswj zn(s) := αs(x λ ) and zj(s) := i j 1, 2,...,n 1 . n s − αs(x λn) ∀ ∈ { − } p − Then, one gets p

lim z(s) = 0, s →∞ 2 s zn(s) λn + | | = x, αs z (s) 2 w 2 lim | j | = lim | j| = 0 j 1,...,n 1 , s α s x λs ∀ ∈ { − } →∞ s →∞ − n z (s)z (s) w w lim m j = lim m j = 0 m = j 1,...,n 1 and s α s x λs ∀ 6 ∈ { − } →∞ s →∞ − n zj(s)zn(s) lim = iwj j 1,...,n 1 . s →∞ αs ∀ ∈ { − }

i Hence, A J s + z(s)z(s) A converges to J and z(s) to 0. λ αs ∗ ∗ λ s I s I ∈ ∈ k s N s If lim λn = , there is a subsequence (λ )s I for an index set I fulﬁlling λj = λj′ for k 6 −∞ ∈ ⊂ all j→∞ 1,...,n and all s I. Therefore, ∈ { } ∈

λ λ′ λ λ′ ... λ λ′ 1 ≥ 1 ≥ 2 ≥ 2 ≥ ≥ n ≥ n and thus, by Lemma 5.3(2), there existsz ˜ Cn such that iλ ,...,iλ are the eigenvalues of ∈ 1 n Jλ′ + iz˜(˜z)∗. Let z(s) :=z ˜√αs, which is reasonable since αs > 0 in this case. As the matrices Jλ′ + iz˜(˜z)∗ and Jλ are both skew-Hermitian and have the same eigenvalues, they are unitarily conjugated. Therefore, there exists an element A U(n) in such a way that ∈ Jλ′ + iz˜(˜z)∗ = A∗JλA. Hence,

z(s)z(s)∗ A∗JλA = Jλ′ + iz˜(˜z)∗ = lim Jλ′ + iz˜(˜z)∗ = lim Jλ′ + i , s s →∞ →∞ αs

z(s)z(s)∗ i.e. A Jλ′ + i A∗ converges to Jλ. Furthermore, αs s I ∈ s z(s) =z ˜√α →∞ 0, s −→ k as α →∞ 0. k −→ Thus, the claim is shown in this case.

k k k Suppose now that for k large enough αk < 0, λ1 λ1 ... λn 1 λn 1 λn λn and ≥ ≥ ≥ − ≥ − ≥ ≥ k k k lim αkλ1 = 0. First consider the case α1 →∞ . k −→ ∞ →∞ s N s In this case, there is a subsequence (λ )s I for an index set I such that λj = λj′ 1 for all ∈ ⊂ −

119 j 2,...,n and all s I. By Identity (40) and Lemma 5.2 , there exist w1,w2,...,wn 1 C, ∈ { } ∈ − ∈ x R and A U(n) such that ∈ ∈ ix w1 w2 ... wn 1 − w iλ 0 ... 0  − 1 1′  w 0 iλ ... 0 A∗JλA = − 2 2′ .  . . . . .   ......     wn 1 0 0 ... iλn′ 1   − −   −  n n 1 − Similarly to the last case, one takes x := λj λj′ and thus gets j=1 − j=1 P P n α (x λs)= α (λ λs) > 0. s − 1 s j − j Xj=1 Cn Hence, one can deﬁne the sequence z(s) s I in by ∈ αswj 1 z (s) := α (x λs) and z (s) := i − j 2,...,n . 1 s 1 j s − − αs(x λ ) ∀ ∈ { } q − 1 Here again, one gets p lim z(s) = 0, s →∞ 2 s z1(s) λ1 + | | = x, αs 2 2 zj(s) wj 1 lim | | = lim | − | = 0 j 2,...,n , s α s x λs ∀ ∈ { } →∞ s →∞ − 1 zm(s)zj(s) wm 1wj 1 lim = lim − − = 0 m = j 2,...,n and s α s x λs ∀ 6 ∈ { } →∞ s →∞ − 1 zj(s)z1(s) lim = iwj 1 j 2,...,n . s − →∞ αs ∀ ∈ { }

i Again, one can conclude that A J s + z(s)z(s) A , √2Az(s),α converges to λ αs ∗ ∗ s s I ∈ (Jλ, 0, 0).

k s N s If lim λ1 = , there is a subsequence (λ )s I for an index set I fulﬁlling λj = λj′ for all k 6 ∞ ∈ ⊂ j →∞1,...,n and all s I. Hence, ∈ { } ∈ λ′ λ λ′ λ ... λ′ λ 1 ≥ 1 ≥ 2 ≥ 2 ≥ ≥ n ≥ n and therefore, by Lemma 5.3(2), there existsz ˜ Cn such that iλ ,...,iλ are the eigenvalues ∈ 1 n of J ′ iz˜(˜z) . λ − ∗ Let now z(s) :=z ˜√ α , which is reasonable since this time α < 0. − s s As above, there exists an element A U(n) such that J ′ iz˜(˜z) = A J A and thus, ∈ λ − ∗ ∗ λ z(s)z(s)∗ z(s)z(s)∗ A∗JλA = lim Jλ′ i = lim Jλ′ + i , s − α s α →∞ − s →∞ s 120 z(s)z(s)∗ i.e. A Jλ′ + i A∗ converges to Jλ. Furthermore, αs s I ∈ s z(s) =z ˜√ α →∞ 0, − s −→ k as α →∞ 0. k −→ Therefore, the assertion is also shown in this case.

5.3 The topology of the spectrum of Gn

In this subsection, the topology of Gn = U(n) ⋉Hn will be analyzed. The aim is to show that it is determined by the topology of its admissible quotient space. For this, some results on the topology of the spectrum of the semidirect product U(n) ⋉Hn in terms of the Mackey data will be given.

5.3.1 The representation π(µ,r)

Gn First, examine the representation π(µ,r) = ind ⋉H ρµ χr. Its Hilbert space (µ,r) is U(n 1) n ⊗ H given by the space −

L2 G / U(n 1) ⋉H , ρ χ = L2 U(n)/U(n 1), ρ . n − n µ ⊗ r ∼ − µ Let ξ be a unit vector in . For all (z,t) H , and all A, B U(n), H(µ,r) ∈ n ∈

i(Bvr,z)Cn 1 π(µ,r)(A, z, t)ξ(B)= e− ξ A− B .

Therefore,

π C (µ,r) (A, z, t) = π (A, z, t)ξ,ξ ξ (µ,r) 2 L U(n)/U(n 1),ρµ D E − i(Bvr,z)Cn 1 = e− ξ A− B ,ξ(B) dB. ρ Z H µ U(n) D E By (39) in Section 5.1.2, one has

U(n) πµ := π(µ,r) U(n) = ind ρµ = τλ. | ∼ U(n 1) − [ τ U(n) λX∈ λ1 µ1 λ2 µ2 ... λn−1 µn−1 λn ≥ ≥ ≥ ≥ ≥ ≥ ≥

Every irreducible representation τλ of U(n) can be realized as a subrepresentation of the left regular representation on L2 U(n) via the intertwining operator

2 Uλ : λ L U(n) , Uλ(ξ)(A) := ξ,τλ(A)ξλ A U(n) ξ λ H → Hλ ∀ ∈ ∀ ∈H for a ﬁxed unit vector ξ . λ ∈Hλ [ For τ U(n), consider the orthonormal basis λ = φλ j 1,...,d of consisting of λ ∈ B j | ∈ { λ} Hλ eigenvectors for T of . n Hλ

121 Moreover, as a basis of the Lie algebra hn of the Heisenberg group, one can take the left invariant vector ﬁelds Z1,Z2,...,Zn, Z1, Z2,..., Zn,T , where ∂ zj ∂ ∂ zj ∂ ∂ Zj := 2 + i , Zj = 2 i and T := ∂zj 2 ∂t ∂zj − 2 ∂t ∂t and gets the Lie brackets [Z , Z ]= 2iT for j 1,...,n . j j − ∈ { } Now, regard the Heisenberg sub-Laplacian diﬀerential operator which is given by n 1 = Z Z + Z Z . L 2 j j j j j=1 X This operator is U(n)-invariant. Lemma 5.5. \ For every representation π for r> 0 and ρ U(n 1), (µ,r) µ ∈ − dπ ( )= r2I. (µ,r) L − Proof: Since the representation π(µ,r) is trivial on the center of hn, one has n 2 2 ∂ ∂ i(Bvr,z)Cn dπ(µ,r)( )ξ(B) = 2 + e− ξ(B). L ∂zj∂zj ∂zj∂zj Xj=1 Let = e ,...,e be an orthonormal basis for Cn. By writing D { 1 n} 1 (Bv , z)Cn = Bv , z Cn + Bv , z Cn , r 2 h r i h r i one gets n 2 2 dπ ( )ξ(B)= Bv ,e Cn ξ(B)= r ξ(B). (µ,r) L − h r ji − j=1 X

In addition, the following theorem describes the convergence of sequences of representations π k : (µ ,rk) k N ∈ Theorem 5.6. \ [ Let r> 0, ρ U(n 1) and τ U(n). µ ∈ − λ ∈ 1. A sequence π k of irreducible unitary representations of G converges to π (µ ,rk) k N n (µ,r) ∈ k in Gn if and only if lim rk = r and µ = µ for k large enough. k →∞ 2. A sequence π k of irreducible unitary representations of G converges to τ in b (µ ,rk) k N n λ ∈ Gn if and only if lim rk = 0 and τλ occurs in πµk for k large enough. k →∞ These are all possibilities for a sequence π k of irreducible unitary representations b (µ ,rk) k N of G to converge. ∈ n The proof of 1) and 2) of this theorem can be found in [1], Theorem 6.2.A. Furthermore, since the representations π and τ are trivial on (I, 0,t) t R , the center (µ,r) λ | ∈ of G , while the representations π are non-trivial there, the possibilities of convergence n (λ,α) of a sequence π k listed above are the only ones that are possible. (µ ,rk) k N ∈ 122 5.3.2 The representation π(λ,α)

Next, regard the representations π(λ,α). dλ λ Consider the unit vector ξ := φj fj in the Hilbert space (λ,α) = λ α(n) of π(λ,α), j=1 ⊗ H H ⊗ F where f ,...,f belong to theP Fock space (n). Then, for all A U(n) and (z,t) H , 1 dλ Fα ∈ ∈ n dλ α α λ iαt z 2 w,z Cn 1 1 π (A, z, t)ξ(w) = τ (A)φ e − 4 | | − 2 h i f A− w + A− z if α> 0 and (λ,α) λ j ⊗ j j=1 X dλ α α λ iαt+ z 2+ w,z Cn 1 1 π (A, z, t)ξ(w) = τ (A)φ e 4 | | 2 h i f A w + A z if α< 0. (λ,α) λ j ⊗ j − − Xj=1 It follows that

π(λ,α) Cξ (A, z, t) = π(λ,α)(A, z, t)ξ,ξ = H(λ,α) d λ α 2 α α 2 λ λ iαt z w,z Cn 1 1 w τλ(A)φ ,φ ′ e − 4 | | − 2 h i fj A− w + A− z fj′ (w)e− 2 | | dw if α> 0, j j λ j,j′=1 H Cn Pdλ R α α α  λ λ iαt+ z 2+ w,z Cn 1 1 w 2  τλ(A)φ ,φ ′ e 4 | | 2 h i fj A w + A z fj′ (w)e 2 | | dw if α< 0.  j j λ − − j,j′=1 H Cn  P R 

Lemma 5.7. [ For each representation π for α R and τ U(n), one has (λ,α) ∈ ∗ λ ∈ I dπ(λ,α)(T )= iα . Proof: dλ λ Let ξ = φj fj be a unit vector in (λ,α). Then, j=1 ⊗ H P dλ d I d iαt 2 dπ(λ,α)(T )ξ,ξ = π(λ,α)( , 0,t)ξ,ξ = e fj α(n) = iα. H(λ,α) dt t=0 H(λ,α) dt t=0 k kF j=1 X

If α is positive, the polynomials C[Cn] are dense in (n) and its multiplicity free Fα decomposition is ∞ C[Cn]= , Pm m=0 X where is the space of homogeneous polynomials of degree m. Thus, p (z) = zm is the Pm m 1 highest weight vector in with weight (m, 0,..., 0) =: [m]. Applying the classical Pieri’s rule Pm (see [15], Proposition 15.25), one obtains

∞ ′ τλ Wα U(n) = τλ τ[m] = τλ . (41) ⊗ | ⊗ ′ m=0 λ Pn ′ ∈ ′ X λ λ1 X.... λ λn 1≥ ≥ ≥ n≥

123 If α is negative, one gets

∞ ′ τλ Wα U(n) = τλ τ[m] = τλ . ⊗ | ⊗ ′ m=0 λ Pn ′ ∈ ′ X λ1 λ X.... λn λ ≥ 1≥ ≥ ≥ n

Both of the sums again are multiplicity free. This follows from [20], Chapter IV.11, since Wα is multiplicity free. Furthermore, let := h m = (m , . . . , m ) Nn be the orthonormal basis of the Rα m,α| 1 n ∈ Fock space (n) deﬁned by the Hermite functions Fα n 2 m α α | | m hm,α(z)= | | | | z 2π s2 m m! | | with m = m + ... + m , m!= m ! m ! and zm = zm1 zmn (see [14], Chapter 1.7). | | 1 n 1 ··· n 1 ··· n Now, one obtains the following theorem about the convergence of sequences of representations π k : (λ ,αk) k N ∈ Theorem 5.8. [ Let α R and τ U(n). Then, a sequence π k of elements in G converges to ∗ λ (λ ,αk) k N n ∈ ∈ k ∈ π(λ,α) if and only if lim αk = α and λ = λ for large k. k →∞ c Proof: k First, consider the case where α is positive. Assume that α →∞ α and that λk = λ for k k −→ large enough. Moreover, let f C (G ) and let ξ be a unit vector in . Then, ∈ 0∞ n Hλ π k C (λ ,αk) ,f ξ h0,α ∞ 1 ⊗ k L (Gn),L (Gn) D E αk 2 1 n 1 1 2 iαkt z √αk w,z Cn w = f(A, z, t) τλk (A)ξ,ξ e − 4 | | e− 2 h i − 2 | | dwd(z,t)dA Hλk 2π Z HZ CZn U(n) n

π(λ,α) tends to C ,f . Hence, π k converges to π . ξ h0,α ∞ 1 (λ ,αk) k N (λ,α) ⊗ L (Gn),L (Gn) ∈ The sameD reasoningE applies when α is negative.

Conversely, the fact that the sequence π k converges to the representation π (λ ,αk) k N (λ,α) implies by Corollary 2.10 that for ξ of length∈ 1, there is for every k N a unit vector ∈H(∞λ,α) ∈ ξ such that dπ k (T )ξ ,ξ converges to dπ (T )ξ,ξ . k (∞λk,α ) (λ ,αk) k k (λ,α) ∈ H k (λk,α ) k N (λ,α) H k ∈ H Thus, by Lemma 5.7, lim αk = α. k Hence, it remains to show→∞ that λk = λ for k large enough. Let ξ be a unit vector in λ. Then, by Theorem 2.8, for every k N, there exists a vector π k k H (λ ,αk) ∈ k ξk = ζm hm,αk (λ ,αk) of length 1 such that Cξ converges uniformly on Nn ⊗ ∈H k k N m ∈ ∈ π(λ,α) compactaP to C . ξ h0,α ⊗

124 Now, take δ R such that 0 I =(α δ, α + δ), as well as a Schwartz function ϕ on R ∈ >0 6∈ α,δ − fulﬁlling ϕ I 1 and ϕ 0 in a neighbourhood of 0. Then, there is a Schwartz function ψ | α,δ ≡ ≡ on Hn with the property σ (ψ)= ϕ(β)P β R∗, β β ∀ ∈ where P : (n) C is the orthogonal projection onto the one-dimensional subspace Ch β Fβ → 0,β of all constant functions in (n). On the other hand, there exists k N such that α I Fβ δ ∈ k ∈ α,δ for all k k . One obtains σ (ψ)h = h and σ (ψ)h = h for all k k and thus, ≥ δ α 0,α 0,α αk 0,αk 0,αk ≥ δ it follows that

k 2 k k ′ lim ζ0 = lim ζm,ζm′ σαk (ψ)hm,αk ,hm ,αk k λk k λk αk (n) →∞ H →∞m,m′ Nn H F X∈

π k (λ ,αk) I = lim C k ( ,.,.), ψ P ζ hm,α k m k ∞ H 1 H →∞ m∈Nn ⊗ L ( n),L ( n) = σ (ψ)h ,h = 1. α 0,α 0,α α(n) F Hence, one gets k lim ξk ζ0 h0,αk = 0 k − ⊗ (λk,α ) →∞ H k

and one can deduce that

k k lim τλk (A)ζ0 ,ζ0 = τλ(A)ξ,ξ k λk λ →∞ H H k N ζ0 uniformly in A U(n). Therefore, for all k , one can take the unit vector φk = k ζ H ∈ ∈ k 0 k λk τλk τλ in λk to ﬁnally obtain the uniform convergence on compacta of Cφ to Cξ . Thus, H k k N λk = λ for k large enough. ∈

Lemma 5.9. [ For each representation π for α R and τ U(n), (λ,α) ∈ ∗ λ ∈ dπ ( )h ,h = α n + 2 m m Nn. (λ,α) m,α m,α α(n) L F −| | | | ∀ ∈ The proof follows from [4], Proposition 3.20 together with [5], Lemma 3.4.

Theorem 5.10. \ [ Let r> 0, ρ U(n 1) and τ U(n). µ ∈ − λ ∈

1. If a sequence π k of elements of G converges to the representation π (λ ,αk) k N n (µ,r) ∈ in Gn, then lim αk = 0 and the sequence π(λk,α ) N satisﬁes one of the following k k k →∞ b ∈ conditions: b k k r2 (i) For k large enough, αk > 0, λj = µj for all j 1,...,n 1 and lim αkλn = . ∈ { − } k − 2 →∞ k k r2 (ii) For k large enough, αk < 0, λj = µj 1 for all j 2,...,n and lim αkλ1 = . − ∈ { } k − 2 →∞

125 2. If a sequence π k of elements of G converges to the representation τ in G , (λ ,αk) k N n λ n ∈ then lim αk = 0 and the sequence π(λk,α ) N satisﬁes one of the following conditions: k k k →∞ b ∈ b k k k (i) For k large enough, αk > 0, λ1 λ1 ... λn 1 λn 1 λn λn and k ≥ ≥ ≥ − ≥ − ≥ ≥ lim αkλn = 0. k →∞ k k k (ii) For k large enough, αk < 0, λ1 λ1 λ2 λ2 ... λn 1 λn λn and k ≥ ≥ ≥ ≥ ≥ − ≥ ≥ lim αkλ1 = 0. k →∞

Proof: s 1) Letµ ˜ =(µ1,...,µs,µs,µs+1,...,µn 1) for s 1,...,n 1 . By hypothesis, the sequence − ∈ { − } π k converges to the representation π in G . Thus, by Corollary 2.10, for the (λ ,αk) k N (µ,r) n ∈ µ˜s s s s unit vector ξ = dµ˜ C µ˜s µ˜s ∞ , there is a sequence of unit vectors (ξ )k N ∞k (µ,r) k (λ ,αk) φ1 ,φ1 ∈H b ∈ ⊂H such that p s s k s s dπ k (T )ξ ,ξ →∞ dπ(µ,r)(T )(ξ ),ξ = 0 T tn and (λ ,αk) k k k (µ,r) H(λ ,αk) −→ H ∀ ∈

s s k s s 2 dπ k ( )ξ ,ξ →∞ dπ(µ,r)( )(ξ ),ξ = r . (λ ,αk) k k k (µ,r) L H(λ ,αk) −→ L H −

s s s s Since by Lemma 5.7 one gets dπ k (T )ξ ,ξ = iαkξ ,ξ , it follows (λ ,αk) k k k k k k H(λ ,αk) H(λ ,αk) that lim αk = 0. Therefore, one can assume without restriction that αk > 0 for large k (by k →∞ passing to a subsequence if necessary). The case αk < 0 is very similar.

s s On the other hand, the sequence τ k Wα (A)ξ ,ξ converges to the matrix λ k k k k N ⊗ (λ ,αk) k s H ∈ π(µ,r) µ˜ coeﬃcient Cξs (A, 0, 0) = C µ˜s µ˜s (A) uniformly in each A U(n). Hence, from this φ1 ,φ1 ∈ µ˜s 1 convergence, Orthogonality Relation (38) and the fact that C µ˜s µ˜s = 2 s φ1 ,φ1 L U(n) √dµ˜ follows s s µ˜s µ˜s 1 s lim τλk Wαk (A)ξk,ξk τµ˜ (A)φ1 ,φ1 dA = = 0. (42) k ⊗ (λk,α ) (µ,r) d s 6 →∞ Z H k H µ˜ U(n)

By (41), one can write the expression τλk Wαk U(n) as ⊗ | τ k W = τ k λ ⊗ αk U(n) λ˜ | ˜k λXPn λ˜k λk ...∈ λ˜k λk 1 ≥ 1 ≥ ≥ n≥ n and, since for k large enough the above integral is not 0, again by the orthogonality relation, ˜k ˜k k ˜k k ˜k s there has to be one λ Pn with λ1 λ1 ... λn λn such that λ =µ ˜ . But as ˜k s s ˜k ∈ ≥ k ≥ ˜k ≥ s ≥ λs =µ ˜s =µ ˜s+1 = λs+1, one obtains that λs = λs =µ ˜s = µs for k large enough. As this is true for all s 1,...,n 1 , one gets λk = µ for all j 1,...,n 1 . ∈ { − } j j ∈ { − }

126 k r2 So, it remains to show that lim αkλn = . k − 2 →∞ Again, by the decomposition of τ k W in (41), one can decompose k as λ ⊗ αk U(n) H(λ ,αk) follows | k = k H(λ ,αk) Hλ˜ ˜k λXPn λ˜k λk ...∈ λ˜k λk 1 ≥ 1 ≥ ≥ n≥ n and thus, for every k N, the vector ξs can be written as ∈ k s s s ξ = ξ for ξ k k N. k λ˜k λ˜k ∈Hλ˜ ∀ ∈ ˜k λXPn λ˜k λk ...∈ λ˜k λk 1 ≥ 1 ≥ ≥ n≥ n Then, with Orthogonality Relation (38),

s s µ˜s µ˜s s τλk Wαk (A)ξk,ξk τµ˜ (A)φ1 ,φ1 dA ⊗ (λk,α ) (µ,r) Z H k H U(n) s s = τλk Wαk (A)ξ˜k ,ξγ˜k ⊗ λ λ˜k k k H λ˜ P γ˜ Pn Z Xn X∈ U(n) λ˜k λk ...∈ λ˜k λk γ˜k λk ... γ˜k λk 1 ≥ 1 ≥ ≥ n≥ n 1 ≥ 1 ≥ ≥ n≥ n µ˜s µ˜s τµ˜s (A)φ1 ,φ1 dA H(µ,r)

s s k = τν˜ (A)ξλ˜k ,ξγ˜k ˜k k k ˜k λ Pn γ˜ Pn UZ(n) ν˜ Pn Hλ k k X∈ k k k k X∈ k k k k X∈ k k λ˜ λ ... λ˜ λ γ˜ λ ... γ˜ λ ν˜ λ ... ν˜n λn 1 ≥ 1 ≥ ≥ n≥ n 1 ≥ 1 ≥ ≥ n≥ n 1 ≥ 1 ≥ ≥ ≥ µ˜s µ˜s τµ˜s (A)φ1 ,φ1 dA H(µ,r)

s s µ˜s µ˜s s = τλ˜k (A)ξ˜k ,ξγ˜k τµ˜ (A)φ1 ,φ1 dA λ λ˜k (µ,r) k k H H λ˜ P γ˜ Pn Z Xn X∈ U(n) λ˜k λk ...∈ λ˜k λk γ˜k λk ... γ˜k λk 1 ≥ 1 ≥ ≥ n≥ n 1 ≥ 1 ≥ ≥ n≥ n s µ˜s µ˜s s ξ s ,φ φ ,ξ µ˜ 1 s 1 γ˜k s = Hµ˜ Hµ˜ d s k µ˜ γ˜XPn γ˜k λk ...∈ γ˜k λk 1 ≥ 1 ≥ ≥ n≥ n s µ˜s µ˜s s s µ˜s 2 s s s ξµ˜ ,φ1 s φ1 ,ξµ˜ s ξµ˜ ,φ1 s = Hµ˜ Hµ˜ = Hµ˜ . dµ˜ s dµ˜s

From (42) follows that s µ˜s 2 k s →∞ ξµ˜ ,φ1 s 1. Hµ˜ −→

As 1= ξs 2 = ξs 2 , k k λ˜k k kH(λ ,αk) λ˜k ˜k H λXPn λ˜k λk ...∈ λ˜k λk 1 ≥ 1 ≥ ≥ n≥ n

127 s one can assume that ξs = φµ˜ for large k N. Since λk = µ for all k N and for all k 1 ∈ j j ∈ j 1,...,n 1 , one gets for s = n 1 ∈ { − } − n 1 k k µ˜ − = λ + mk for mk = 0,..., 0,µn 1 λn . − − n−1 n 1 µ˜ n 1 From now on, consider only k large enough such that ξk − = φ1 . Then, ξk − is the highest n 1 weight vector with weightµ ˜ − . Moreover,

∞ k = k = k (n)= k . Hλ˜ H(λ ,αk) Hλ ⊗ Fαk Hλ ⊗Pm ˜k m=0 λXPn X λ˜k λk ...∈ λ˜k λk 1 ≥ 1 ≥ ≥ n≥ n Every weight in the decomposition on the left hand side has multiplicity one, as mentioned in (41), and therefore, this is the case for every weight appearing in the sum on the right hand n 1 side as well. From this, one can deduce that there exists one unique Mk such thatµ ˜ − , the n 1 weight of ξ − k , appears in k . k ∈H(λ ,αk) Hλ ⊗PMk By [20], Chapter IV.11, every highest weight appearing in λk Mk is the sum of the highest n 1 H ⊗Pk weight of k and a weight of . Hence,µ ˜ is the sum of λ and a weight of . From Hλ PMk − PMk this follows that the mentioned weight of has the same length asµ ñ 1 λk = m . PMk − − k Therefore, Mk = mk , i.e. Mk = mk . | | P P| | T Taking an orthonormal basis of λk consisting of eigenvectors for n as at the beginning of H n 1 this subsection, then, due to the above considerations, one can write ξk − as n 1 γk ξk − = φ hm˜ k,αk , n−1 ⊗ k µ˜ (γ ,m˜ k) Ω X∈ λk γk T k µñ−1 where φ is a uniquely determined eigenvector for n of λk with weight γ and Ωλk is k n H k the set of all pairs (γ , m˜ k) such thatm ˜ k N with m˜ k = mk and γ is a weight that ∈k |n 1| | | appears in the representation τλk fulfilling γ +m ˜ k =µ ˜ − . Furthermore, k φγ 2 = 1. n−1 k µ˜ (γ ,m˜ kX) Ω k ∈ λ Then, from Lemma 5.9 and the U(n)-invariance of , L 2 k n 1 n 1 r →∞ dπ k ( )ξ − ,ξ − (λ ,αk) k k k − ←− L H(λ ,αk)

γk γ˜k = dπ k ( )φ h ,φ h (λ ,αk) m˜ k,αk m˜ k,αk n−1 n−1 L ⊗ ⊗ (λk,α ) k µ˜ k µ˜ H k (γ ,m˜ k) Ω (˜γ ,m˜ k) Ω D E X∈ λk X∈ λk

γk 2 k = φ dπ(λ ,αk)( )hm˜ k,αk ,hm˜ k,αk αk (n) n−1 L F k µ˜ (γ ,m˜ k) Ω X λk ∈ γk 2 = φ αk n + 2 m˜k n−1 − | | k µ˜ (γ ,m˜ k) Ω X λk ∈ γk 2 = αk n + 2 mk φ − | | n−1 k µ˜ (γ ,m˜ k) Ω X λk ∈ k = αk n + 2 mk = αk n + 2µn 1 2λn . − | | − − − 128 k k k r2 As αk →∞ 0, also αk(n + 2µn 1) →∞ 0 and thus, lim αkλn = . −→ − −→ k − 2 →∞

2) The fact that the sequence π k converges to τ in G implies that for the unit (λ ,αk) k N λ n λ ∈ vector φ ∞, there is a sequence of unit vectors (ξk)k N ∞k such that 1 λ (λ ,αk) ∈H ∈ ⊂Hb k λ λ dπ k (T )ξk,ξk →∞ dτλ(T )φ ,φ T tn and (λ ,αk) k 1 1 λ H(λ ,αk) −→ H ∀ ∈

k λ λ dπ k ( )ξk,ξk →∞ dτλ( )φ ,φ = 0. (λ ,αk) k 1 1 λ L H(λ ,αk) −→ L H

As above in the first part, by Lemma 5.7, from the first convergence it follows that lim αk = 0 k →∞ and one can assume without restriction that αk > 0 for large k. λ k On the other hand, τλ Wαk (A)ξk,ξk converges to Cφλ,φλ (A) uniformly in ⊗ (λk,α ) k N 1 1 H k ∈ each A U(n). Hence, as above one gets ∈ λ λ 1 lim τλk Wαk (A)ξk,ξk τλ(A)φ1 ,φ1 dA = = 0. k ⊗ (λk,α ) λ d 6 →∞ Z H k H λ U(n) Again, as in the first part above, by (41) and the orthogonality relation, one can deduce that λ λk .... λ λk for large k. 1 ≥ 1 ≥ ≥ n ≥ n k So again, it remains to show that lim αkλn = 0. k →∞ n 1 In the same manner as above, by replacingµ ˜ − by λ, one can now show that for large k N, it is possible to assume ξ = φλ. So consider k large enough in order for this equality ∈ k 1 to be true. Then ξk is the highest weight vector with weight λ. Now, λ = λk + m for m = λ λk,...,λ λk , k k 1 − 1 n − n k k where the sequences λ1 λ1 N,..., λn 1 λn 1 N are bounded, because k − k k k − ∈ − − ∈ λ1 λ1 .... λn λn for large k. ≥ ≥ ≥ ≥ n 1 Again, by replacingµ ˜ − by λ in the proof of the first part above, one can also write ξk as k ξ = φγ h , k ⊗ m˜ k,αk k λ (γ ,m˜ k) Ω X∈ λk γk T k λ where φ is a uniquely determined eigenvector for n of λk with weight γ and Ωλk is the k n H k set of all pairs (γ , m˜ k) such thatm ˜ k N with m˜ k = mk and γ is a weight that appears k ∈ | | | | in the representation τλk fulfilling γ +m ˜ k = λ. Furthermore, again k φγ 2 = 1. k λ (γ ,m˜ k) Ω X∈ λk

Now, like in the first part above, by Lemma 5.9 and the U(n)-invariance of , L k 0 →∞ dπ k ( )ξk,ξk (λ ,αk) k ←− L H(λ ,αk) = α n + 2 m − k | k| k k k = αk n + 2 λ1 λ1 + ... + 2 λn 1 λn 1 + 2λn 2λn . − − − − − − 129 k k k k As αk →∞ 0, also αk n + 2 λ1 λ1 + ... + 2 λn 1 λn 1 + 2λn →∞ 0 because of the −→ − − − − −→ k k k boundedness of the sequences λ1 λ1 k N,..., λn 1 λn 1 k N. Therefore, lim αkλn = 0. − − − − k ∈ ∈ →∞ Theorem 5.11. \ [ Let r> 0, ρ U(n 1) and τ U(n). µ ∈ − λ ∈ If lim αk = 0 and the sequence π(λk,α ) N of elements of Gn satisfies one of the following k k k →∞ ∈ conditions: b k k r2 (i) for k large enough, αk > 0, λj = µj for all j 1,...,n 1 and lim αkλn = , ∈ { − } k − 2 →∞ k k r2 (ii) for k large enough, αk < 0, λj = µj 1 for all j 2,...,n and lim αkλ1 = , − ∈ { } k − 2 →∞ then the sequence π k converges to the representation π in G . (λ ,αk) k N (µ,r) n ∈ In order to prove this theorem, one needs the following proposition: b Proposition 5.12. \ [ Let r> 0, ρµ U(n 1) and τλ U(n). ∈ − ∈ k Furthermore, let lim αk = 0, αk > 0 for large k and consider the sequence λ )k N in Pn k ∈ k →∞ k r2 fulfilling λj = µj for all j 1,...,n 1 and lim αkλn = . ∈ { − } k − 2 n 1 →∞ k Denote µ˜ :=µ ˜ − = (µ1,...,µn 1,µn 1), Nk := µn 1 λn and let Nk be the space of − − − − P conjugated homogeneous polynomials of degree Nk. Define the representation π(˜µ,αk) of G on the subspace of the Hilbert space n Hµ˜ ⊗ PNk ⊗PNk (n) (n) by Hµ˜ ⊗ Fαk ⊗ Fαk π(˜µ,αk)(A, z, t) := τ (A) W (A) σ (z,t) W (A) (A, z, t) G . µ˜ ⊗ αk ⊗ αk ◦ αk ∀ ∈ n Then, there exists a vector ξ and for each k N vectors ξ ∈Hµ˜ ⊗H(1,r) ∈ k ∈Hµ˜ ⊗ PNk ⊗PNk such that for all (A, z, t) G , ∈ n

(˜µ,αk) k π (A, z, t)ξk,ξk →∞ τµ˜ π(1,r) (A, z, t)ξ,ξ . µ˜ α (n) α (n) −→ ⊗ µ˜ H ⊗F k ⊗F k H ⊗H(1,r) D E D E Proof: k k r2 Let mk := (0,..., 0,Nk). Then, λ =µ ˜ + mk. Moreover, since lim αkλn = , lim αk = 0 k − 2 k k →∞ →∞ and α > 0, one gets N →∞ . k k −→ ∞ Let φµ˜ be the highest weight vector with weightµ ˜ in . Hµ˜ Now, deﬁne

1 1 ξ := φµ˜ h h = φµ˜ h h . k ⊗ 1 q,αk ⊗ q,αk 1 ⊗ q,αk ⊗ q,αk ∈Hµ˜ ⊗ PNk ⊗PNk Nk 2 q Nn: Nk 2 q Nn: qX∈=N qX∈=N | | k | | k

130 1 2 1 Since Nk is the norm of hq,αk hq,αk , the vector 1 hq,αk hq,αk has norm 1. q Nn: ⊗ Nk 2 q Nn: ⊗ q∈=N q∈=N | |P k | |P k Let (A, z, t) G . One has ∈ n

(˜µ,αk) π (A, z, t)ξk,ξk µ˜ αk (n) αk (n) D EH ⊗F ⊗F 1 = π(˜µ,αk)(A, z, t) φµ˜ h h , 1 ⊗ q,αk ⊗ q,αk * Nk 2 q Nn: qX∈=N | | k 1 φµ˜ h h 1 ⊗ q,α˜ k ⊗ q,α˜ k Nk 2 q˜ Nn: + µ˜ αk (n) αk (n) q˜X∈=N H ⊗F ⊗F | | k 1 = π(˜µ,αk) (I,z,t)(A, 0, 0) φµ˜ h h , 1 ⊗ q,αk ⊗ q,αk * Nk 2 q Nn: qX∈=N | | k 1 φµ˜ h h 1 ⊗ q,α˜ k ⊗ q,α˜ k Nk 2 q˜ Nn: + µ˜ αk (n) αk (n) q˜X∈=N H ⊗F ⊗F | | k 1 = φµ˜ W (A)h σ (z,t) W (A)h , 1 ⊗ αk q,αk ⊗ αk ◦ αk q,αk *Nk 2 q Nn: qX∈=N | | k

1 1 µ˜ τ (A− )φ h h . 1 µ˜ ⊗ q,α˜ k ⊗ q,α˜ k Nk 2 q˜ Nn: + µ˜ αk (n) αk (n) q˜X∈=N H ⊗F ⊗F | | k Now, one can write

k k 0 if m = m′, wm,q(A)wm′,q(A)= 6 q Nn: (1 if m = m′. qX∈=N | | k

131 Hence,

1 φµ˜ W (A)h σ (z,t) W (A)h , 1 ⊗ αk q,αk ⊗ αk ◦ αk q,αk *Nk 2 q Nn: qX∈=N | | k

1 1 µ˜ τ (A− )φ h h 1 µ˜ ⊗ q,α˜ k ⊗ q,α˜ k Nk 2 q˜ Nn: + µ˜ αk (n) αk (n) q˜X∈=N H ⊗F ⊗F | | k 1 = φµ˜ h σ (z,t)h , 1 ⊗ m,αk ⊗ αk m,αk *Nk 2 m Nn: mX∈=N | | k

1 1 µ˜ τ (A− )φ h h 1 µ˜ ⊗ q,α˜ k ⊗ q,α˜ k Nk 2 q˜ Nn: + µ˜ αk (n) αk (n) q˜X∈=N H ⊗F ⊗F | | k µ˜ µ˜ = τµ˜(A)φ ,φ µ˜ D EH 1 hm,αk σαk (z,t)hm,αk , hq,α˜ k hq,α˜ k Nk ⊗ ⊗ * m Nn: q˜ Nn: + (n) (n) αk αk mX∈=Nk q˜X∈=N F ⊗F | | | | k µ˜ µ˜ 1 = τµ˜(A)φ ,φ hm,αk , hq,α˜ k σαk (z,t)hm,αk ,hq,α˜ k µ˜ Nk α (n) α (n) H m,q˜ Nn: F k F k D E m =X∈q˜ =N D E D E | | | | k µ˜ µ˜ 1 = τµ˜(A)φ ,φ σαk (z,t)hq,αk ,hq,αk µ˜ Nk α (n) H q Nn: F k D E qX∈=N D E | | k α α α 1 k 2 k n k 2 µ˜ µ˜ iαkt 4 z 2 w,z C 2 w = τµ˜(A)φ ,φ e − | | e− h i hq,αk (z + w)hq,αk (w)e− | | dw µ˜ Nk Nn H q : CZn D E qX∈=N | | k Nk n α αk 2 µ˜ µ˜ 1 αk k 1 iαkt z = τµ˜(A)φ ,φ e − 4 | | Nk µ˜ Nk 2π 2 q! H q Nn: D E qX∈=N | | k α α k w,z Cn q q k w 2 e− 2 h i (z + w) w e− 2 | | dw. CZn

q q1 qn n Now, by the binomial theorem, letting := for q = (q1,...,qn) N and l l1 ··· ln ∈ l =(l ,...,l ) Nn, 1 n ∈ q1 qn q q1 q1 l1 l1 qn qn ln ln q q l l (z + w) = z1 − w1 zn − wn = z − w . l1 ··· ln n l l1=0 ln=0 l:=(l1,...,ln) N : X X X ∈ l1 q1,...,ln qn ≤ ≤

132 Thus, one gets for q Nn with q = N , ∈ | | k N n k α α α αk αk 1 iα t k z 2 k w,z Cn q q k w 2 e k − 4 | | e− 2 h i (z + w) w e− 2 | | dw 2π 2Nk q! CZn N n k α α α αk αk 1 iα t k z 2 q q l k w,z Cn l q k w 2 = e k − 4 | | z − e− 2 h i w w e− 2 | | dw. Nk 2π 2 q! n l l:=(l1,...,ln) N : CZn X ∈ l1 q1,...,ln qn ≤ ≤ The integrals in w for m 1,...,n can be written as follows: m ∈ { } jm jm ∞ jm αk 2 wm ( zm) αk wm lm qm − e− 2 | | wm wm dwm. jm! 2 jXm=0 ZC Therefore,

N n k α α α αk αk 1 iα t k z 2 q q l k w,z Cn l q k w 2 e k − 4 | | z − e− 2 h i w w e− 2 | | dw Nk 2π 2 q! n l l:=(l1,...,ln) N : CZn X ∈ l1 q1,...,ln qn ≤ ≤ Nk j n α αk 2 j αk k 1 iαkt z αk | | q q l ( z) = e − 4 | | z − − Nk 2π 2 q! n Nn 2 l j! l:=(l1,...,ln) N : j X ∈ X∈ l1 q1,...,ln qn ≤ ≤ α j+l q k w 2 w w e− 2 | | dw. CZn

Because of the orthogonality of the functions Cn Cn,x xa and Cn Cn,x xb for → 7→ → 7→ a,b Nn with respect to the scalar product of the Fock space, j + l = q, i.e. l = q j. ∈ − Moreover, as h 2 = 1, q,αk α (n) k kF k

q 2 1 = N . αk (n) k · F αk n αk 2π 2Nk q!

Hence,

Nk j n α αk 2 j αk k 1 iαkt z αk | | q q l ( z) e − 4 | | z − − Nk 2π 2 q! n Nn 2 l j! l:=(l1,...,ln) N : j X ∈ X∈ l1 q1,...,ln qn ≤ ≤ α j+l q k w 2 w w e− 2 | | dw CZn Nk j n α αk 2 j αk k 1 iαkt z αk | | q j ( z) q 2 = e − 4 | | z − Nk αk (n) 2π 2 q! n 2 q j j! · F j:=(j1,...,jn) N : ∈ − j1 q1X,...,jn qn ≤ ≤ j j αk 2 αk j q! z ( z) iαkt 4 z | | = e − | | − 2 . n 2 (q j)! j! j:=(j1,...,jn) N : ∈ − j1 q1X,...,jn qn ≤ ≤ 133 Thus,

(˜µ,αk) π (A, z, t)ξk,ξk µ˜ αk (n) αk (n) D EH ⊗F ⊗F j j αk 2 1 αk j q! z ( z) µ˜ µ˜ iαkt 4 z | | = τµ˜(A) φ ,φ e − | | − 2 . µ˜ Nk n n 2 (q j)! j! H q N : j:=(j1,...,jn) N : D E ∈ ∈ − qX=Nk j1 q1X,...,jn qn | | ≤ ≤ Now, regard

j j j 1 αk | | q! z ( z) − 2 Nk n n 2 (q j)! j! q N : j:=(j1,...,jn) N : ∈ ∈ − qX=Nk j1 q1X,...,jn qn | | ≤ ≤ 1 αk j1+...+jn = q1(q1 1) (q1 j1 + 1) Nk n 2 − ··· − q1,...,qn N: j:=(j1,...,jn) N : ∈ ∈ q1+...X+qn=Nk j1 q1X,...,jn qn ≤ ≤ zj( z)j qn(qn 1) (qn jn + 1) − . ··· − ··· − j! 2 Then, ﬁxing large k N, one gets for j =(j ,...,j ) Nn, ∈ 1 n ∈

1 αk j1+...+jn q1(q1 1) (q1 j1 + 1) Nk 2 − ··· − q1 N≥j ,...,qn N≥j : ∈ 1 X ∈ n q1+...+qn=Nk

zj( z)j q (q 1) (q j + 1) − (43) ··· n n − ··· n − n 2 j! 

1 α N j1+...+jn q (q 1) (q j + 1) = k k 1 1 − ··· 1 − 1 j1 Nk 2 Nk q1 N ,...,qn N : ∈ ≥j1 X ∈ ≥jn q1+...+qn=Nk

q (q 1) (q j + 1) zj( z)j n n − ··· n − n − jn 2 ··· Nk j!

j 2 j1+ ...+jn j j 2 1 r z ( z) r j 1 + 1 − 2 = + 1 z z , ≤ Nk N N 4 j! 4 (j!) q1 ≥j ,...,qn ≥j : ∈ 1 X ∈ n q1+...+qn=Nk r2 since lim αkNk = . The above expression does not depend on k and k 2 →∞ r2 j 1 r2 + 1 z zj = exp + 1 zz < . n 4 (j!) 4 ∞ j:=(j1,...,jn) N X ∈ So, by the theorem of Lebesgue, the sum in (43) converges and it suﬃces to regard the limit

134 of each summand by itself. Hence, for j =(j ,...,j ) Nn, one has 1 n ∈ 1 αk j1+...+jn q1(q1 1) (q1 j1 + 1) Nk 2 − ··· − q1 N ,...,qn N : ∈ ≥j1 X ∈ ≥jn q1+...+qn=Nk zj( z)j qn(qn 1) (qn jn + 1) − ··· − ··· − j! 2 2 1 r j1+...+jn q1(q1 1) (q1 j1 + 1) = − ··· − ∼ j1 Nk 4 Nk q1 N ,...,qn N : ∈ ≥j1 X ∈ ≥jn q1+...+qn=Nk q (q 1) (q j + 1) zj( z)j n n − ··· n − n − jn 2 ··· Nk j!

2 1 r j1+...+jn q q 1 q j 1 = 1 1 1 1 − Nk 4 Nk Nk − Nk ··· Nk − Nk q1 N ,...,qn N : ∈ ≥j1 X ∈ ≥jn q1+...+qn=Nk j j qn qn 1 qn jn 1 z ( z) − − 2 ··· Nk Nk − Nk ··· Nk − Nk j! 2 1 r j1+...+jn q q 1 q j 1 = 1 1 1 1 − Nk 4 Nk Nk − Nk ··· Nk − Nk q1 N ,...,qn−1 N : ∈ ≥j1 X ∈ ≥jn−1 q1+...+qn−1 N jn ≤ k− qn 1 qn 1 1 qn 1 jn 1 1 − − − − − ··· Nk Nk − Nk ··· Nk − Nk q1 + ... + qn 1 q1 + ... + qn 1 1 1 − 1 − · − N − N − N k k k j j q1 + ... + qn 1 jn 1 z ( z) − 1 − − 2 . ··· − Nk − Nk j! Now, deﬁne for k N the function F : [0, 1]n 1 R by ∈ k − →

Fk(s1,...,sn 1) := − 2 r j1+...+jn 1 j1 1 1 jn 1 1 s1 s1 s1 − sn 1 sn 1 sn 1 − − 4 − Nk ··· − Nk ··· − − − Nk ··· − − Nk 1 1 (s1 + ... + sn 1) 1 (s1 + ... + sn 1) · − − − − − Nk j j jn 1 z ( z) 1 (s1 + ... + sn 1) − − 2 . ··· − − − Nk j!

135 Then, for ε> 0 and large k N, ∈ 1 1 1 q1 qn 1 Fk ,..., − Fk(s1,...,sn 1)ds1...dsn 1 < ε. Nk Nk Nk − ··· − − q ,...,q N 1 n−1 ≤Nk Z0 Z0 X∈

q1 qn−1 Since Fk ,..., = 0, if q1 Nk jn, Nk Nk − − − − it follows that 1 1 1 q1 qn 1 Fk ,..., − Fk(s1,...,sn 1)ds1...dsn 1 < ε. Nk Nk Nk − ··· − − q1 N ,...,qn−1 N : ∈ ≥j1 X ∈ ≥jn−1 Z0 Z0 q1+...+qn−1 N jn ≤ k−

Furthermore, F converges pointwise to the function F : [0, 1]n 1 R deﬁned by k − →

2 j1+...+jn j j r j1 jn−1 jn z ( z) F (s1,...,sn 1) := s1 sn 1 1 (s1 + ... + sn 1) − − 4 ··· − · − − j! 2 and thus, by the theorem of Lebesgue for integrals,

1 1 1 1

lim Fk(s1,...,sn 1)ds1...dsn 1 = F (s1,...,sn 1)ds1...dsn 1. k ··· − − ··· − − →∞ Z0 Z0 Z0 Z0 From these observations, one can now deduce that

2 1 r j1+...+jn q q 1 q j 1 1 1 1 1 − Nk 4 Nk Nk − Nk ··· Nk − Nk q1 N ,...,qn−1 N : ∈ ≥j1 X ∈ ≥jn−1 q1+...+qn−1 N jn ≤ k− qn 1 qn 1 1 qn 1 jn 1 1 − − − − − ··· Nk Nk − Nk ··· Nk − Nk q1 + ... + qn 1 q1 + ... + qn 1 1 1 − 1 − · − Nk − Nk − Nk j j q1 + ... + qn 1 jn 1 z ( z) − 1 − − 2 ··· − Nk − Nk j! 1 1 1 q1 qn 1 k = Fk ,..., − →∞ F (s1,...,sn 1)ds1...dsn 1 Nk Nk Nk −→ ··· − − q1 N ,...,qn−1 N : ∈ ≥j1 X ∈ ≥jn−1 Z0 Z0 q1+...+qn−1 N jn ≤ k−

1 1 2 j1+...+jn j j r j1 jn−1 jn z ( z) = s1 sn 1 1 (s1 + ... + sn 1) − ds1...dsn 1. ··· 4 ··· − − − 2 − Z Z j! 0 0 

136 Therefore,

j j αk 2 1 αk j q! z ( z) µ˜ µ˜ iαkt 4 z | | τµ˜(A) φ ,φ e − | | − 2 µ˜ Nk n n 2 (q j)! j! H q N : j:=(j1,...,jn) N : D E ∈ ∈ − qX=Nk j1 q1X,...,jn qn | | ≤ ≤ 1 1 2 j1+...+jn k µ˜ µ˜ r j1 jn−1 →∞ τµ˜(A) φ ,φ s1 sn 1 − −→ µ˜ n ··· 4 ··· H j:=(j1,...,jn) N Z Z D E X ∈ 0 0 j j jn z ( z) (1 (s1 + ... + sn 1) − ds1...dsn 1 − − j! 2 −

2 2 j1 1 1 z1 s1r −| |4 = τ (A) φµ˜ ,φµ˜ µ˜ 2 µ˜ ··· N j1! ! H Z Z j1 D E 0 0 X∈ Bessel function

2 2 2 2 jn−|1 {z zn 1 }(s1+...+sn−1) r jn zn−1 sn−1r −| | − −| |4 4 ds1...dsn 1 ··· j ! 2 j ! 2 − jn−1 N n 1 ! jn N n ! X∈ − X∈ Bessel function Bessel function | {z 1 } |1 2π 2π {z } n ia µ˜ µ˜ 1 ir Re e 1 √s1 z1 = τµ˜(A) φ ,φ e− µ˜ 2π ··· ··· H Z Z Z Z D E 0 0 0 0 ian ian ir Re e −1 √sn−1 zn−1 ir Re e √1 (s1+...+sn−1) zn e− e− − da1...dands1...dsn 1 ··· − 1 1 2π 2π n µ˜ µ˜ 1 = τµ˜(A) φ ,φ µ˜ 2π ··· ··· H Z Z Z Z D E 0 0 0 0 ia1 ian−1 ian ir √s1e ,...,√sn−1e ,√1 (s1+...+sn−1)e ,(z1,...,zn) n e− − C da1...dands1...dsn 1 − µ˜ µ˜ i(sr,z)Cn = τµ˜(A) φ ,φ e− dσ(s) (44) µ˜ H Zn D E S

µ˜ µ˜ i(Bvr,z)Cn = τµ˜(A) φ ,φ e− dB µ H˜ Z D E U(n) µ˜ µ˜ = τµ˜ π(1,r) (A, z, t) φ 1 ,φ 1 . ⊗ ⊗ ⊗ µ˜ H ⊗H(1,r) D E Equality (44) can be explained in the following way: By substitution, for a function f on the n-dimensional complex unit ball Bn,

1 2n 1 f(x)dx = ρ − f(ρσ)dµ(σ)dρ, (45) BZn Z0 SZn

137 Sn being the n-dimensional complex sphere and dµ its invariant measure. Now, regard the integral

1 2n 1 ρ − f ψ(s1,...sn 1,t1,...,tn, ρ) d(s1,...sn 1,t1,...,tn, ρ) − − Z0 [0,1]n−1Z [0,2π)n × for any continuous function f on Bn and

ψ(s1,...,sn 1,t1,...,tn, ρ) := −

ρ √s1 cos(t1), √s1 sin(t1),..., √sn 1 cos(tn 1), √sn 1 sin(tn 1), − − − − √1 s cos(t ), √1 s cos(t ) − n − n n 1 − with s = si. By substitution and Lemma 7.7 in the appendix, up to a positive constant, i=1 this integralP equals f(x)dx. Together with (45), this shows that the measure used on Bn [0, 1]n 1 [0, 2π)n coincidesR with the invariant measure dσ on the sphere Sn and hence, − × Equality (44) is proved. Choosing ξ := φµ˜ 1 , the claim is shown. ⊗ ∈Hµ˜ ⊗H(1,r)

Proof of Theorem 5.11: Without restriction, one can assume that the sequence π k fulfills Condition (i). (λ ,αk) k N ∈ The case of a sequence π k fulfilling the second condition is very similar. (λ ,αk) k N ∈ Forn ˜ N and ν P , let φν be the heighest weight vector of τ in the Hilbert ∈ ∈ n˜ ν space ν. Letµ ˜ := (µ1,...,µn 1,µn 1) and define the representation σ(˜µ,r) of Gn by H − − σ := τ π . (˜µ,r) µ˜ ⊗ (1,r) The Hilbert space of the representation σ is the space Hσ(˜µ,r) (˜µ,r) = L2 U(n)/U(n 1), Hσ(˜µ,r) − Hµ˜ and G acts on by n Hσ(˜µ,r)

i(Bvr,z)Cn 1 σ (A, z, t)(ξ)(B)= e− τ (A) ξ(A− B) A, B U(n) (z,t) H ξ . (˜µ,r) µ˜ ∀ ∈ ∀ ∈ n ∀ ∈Hσ(˜µ,r) One decomposes the representation τµ˜ U(n 1) into the direct sum of irreducible representations of the group U(n 1) as follows: | − −

τµ˜ U(n 1) = ρν, | − ν S(˜µ) ∈X \ where S(˜µ) denotes the support of τ in U(n 1). Furthermore, let P be the µ˜ U(n 1) − ν orthogonal projection of onto its U(n| 1)-invariant− component . Hµ˜ − Hν

138 The representation ρµ is one of the representations appearing in this sum, since the highest µ˜ weight vector φ of τµ˜ is also the highest weight vector of the representation ρµ.

Deﬁning forν ˜ P the function cν˜ by ∈ n η,φν˜ ν˜ 1 ν˜ c ν˜ (A) := τν˜(A− )η,φ A U(n) η ν˜, η,φ ν H˜ ∀ ∈ ∀ ∈H [ one can identify for any τ U(n) the Hilbert space with the subspace L2 of L2 U(n) ν˜ ∈ Hν˜ ν˜ given by 2 ν˜ L = c ν η . ν˜ η,φ˜ | ∈Hν˜ Now, it will be shown that

σ(˜µ,r) ∼= π(ν,r). (46) ν S(˜µ) ∈X In particular, one then gets

= L2 U(n)/U(n 1), ρ . Hσ(˜µ,r) ∼ − ν ν S(˜µ) ∈X Let for ν S(˜µ), ∈ ν 1 ν U (ξ)(A)(A′) := dν τµ˜(A− )ξ(A), ρν(A′)φ ξ µ˜ A U(n) A′ U(n 1). µ˜ µ H˜ ∀ ∈H ∀ ∈ ∀ ∈ − p Then,

1 ν 1 ν τµ˜(A− )ξ(A), ρν(A′)φ = τµ˜(A− )ξ(A),Pν ρν(A′)φ µ˜ µ H H˜ D 1 νE = Pν τµ˜(A− )ξ(A) , ρν(A′)φ , µ H˜ D E i.e. one has a scalar product on the space . Now, deﬁne Hν U : L2 U(n)/U(n 1), L2 U(n)/U(n 1), ρ , U (ξ) := U ν(ξ). µ˜ − Hµ˜ → − ν µ˜ µ˜ ν S(˜µ) ν S(˜µ) ∈X ∈X For all ξ L2 U(n)/U(n 1), , all A U(n) and all A ,B U(n 1), ∈ − Hµ˜ ∈ ′ ′ ∈ − 1 1 ν Uµ˜ξ(AB′)(A′) = dν τµ˜ B′− A− ξ(A), ρν(A′)φ µ˜ ν S(˜µ) H ∈X p 1 ν = dν τµ˜(A− )ξ(A), ρν(B′A′)φ µ˜ ν S(˜µ) H ∈X p 1 = ρν(B′)− Uµ˜ξ(A)(A′).

2 Hence, the vector Uµ˜ξ fulﬁlls the covariance condition of L U(n)/U(n 1), ρν and is thus 2 − contained in the space L U(n)/U(n 1), ρν . Furthermore, ν S(˜µ) − ∈P

139 2 ν 2 Uµ˜ξ 2 = Uµ˜ (ξ)(A) µ dA k k k kH˜ ν S(˜µ) Z ∈X U(n) 1 ν = dν τµ˜(A− )ξ(A), ρν(A′)φ µ H˜ ν S(˜µ) UZ(n) U(nZ 1) ∈X − p 1 ν dν τµ˜(A )ξ(A), ρν(A )φ dA′dA − ′ µ H˜ p 1 ν = dν Pν τµ˜(A− )ξ(A) , ρν(A′)φ µ H˜ ν S(˜µ) UZ(n) U(nZ 1) ∈X − 1 ν Pν τµ˜(A )ξ(A) , ρν(A )φ dA′dA − ′ µ H˜ 2 1 ν = dν Pν τµ˜(A− )ξ(A) , ρν(A′)φ dA′dA µ H˜ ν S(˜µ) UZ(n) U(nZ 1) ∈X −

1 2 ν 2 = Pν τµ˜(A− )ξ(A) φ dA µ˜ ν S(˜µ) Z H ∈X U(n)

1 2 = Pν τµ˜(A− )ξ(A) dA µ˜ ν S(˜µ) Z H ∈X U(n) 2 1 = Pν τµ˜(A− )ξ(A) dA Z ν S(˜µ) µ˜ U(n) ∈X H

1 2 2 = τ ( − )ξ( ) = ξ . µ˜ · · 2 k k2 2 Moreover, for all (A, z, t) Gn, all ξ L U(n)/U(n 1), µ˜ , all B U(n) and all ∈ ∈ − H ∈ A U(n 1), one gets ′ ∈ −

π(ν,r)(A, z, t)(Uµ˜ξ)(B)(A′) ν S(˜µ) ∈X i(Bvr,z)Cn ν 1 = e− (Uµ˜ ξ)(A− B)(A′) ν S(˜µ) ∈X i(Bvr,z)Cn 1 1 ν = e− dν τµ˜(B− A)ξ(A− B), ρν(A′)φ µ˜ ν S(˜µ) H ∈X p i(Bvr,z)Cn 1 1 ν = e− dν τµ˜(B− ) τµ˜(A)ξ(A− B) , ρν(A′)φ µ˜ ν S(˜µ) H ∈X p D E i(Bvr,z)Cn ν 1 = e− U τ (A)ξ A− (B)(A′) µ˜ µ˜ · ν S(˜µ) ∈X = U τ π (A, z, t)ξ (B)(A′) = U σ (A, z, t)ξ (B)(A′). µ˜ µ˜ ⊗ (1,r) µ˜ (˜µ,r) Therefore, (46) holds.

140 For the element ξ = φµ˜ 1 L2 U(n)/U(n 1), , one has for all A U(n) and all ⊗ ∈ − Hµ˜ ∈ A U(n 1), ′ ∈ − 1 µ˜ ν ν Uµ˜ξ(A)(A′)= dν τµ˜(A− )φ , ρν(A′)φ =: Φ (A)(A′). µ˜ µ˜ ν S(˜µ) H ν S(˜µ) ∈X p ∈X In particular, since φµ = φµ˜, µ 1 µ˜ µ µ˜ µ˜ Φ (A)(I)= dµ τµ˜(A− )φ ,φ = dµ φ ,τµ˜(A)φ = 0. µ˜ µ µ H˜ H˜ 6 p p From Theorem 5.6 follows that the subset π(ν,r) µ = ν Pn 1 is closed in Gn. Hence, | 6 ∈ − there exists Fµ =(Fµ)∗ in C∗(Gn) whose Fourier transform at π(ν,r) is 0 if µ = ν Pn 1 and 6 ∈ − for which c

π (Fµ) =: P µ (µ,r) Φµ˜ is the orthogonal projection onto the space CΦµ . In particular, µ˜ ⊂H(µ,r) U σ (F )(φµ˜ 1) = π (F ) U (φµ˜ 1) = π (F ) U (φµ˜ 1) µ˜ (˜µ,r) µ ⊗ (ν,r) µ µ˜ ⊗ (µ,r) µ µ˜ ⊗ ν S(˜µ) ∈X µ˜ µ = PΦµ Uµ˜(φ 1) = c Φ (47) µ˜ ⊗ · µ˜ for a constant c = 0. Without restriction, one can assume that c = 1. 6 1 Define the coefficient cµ of L (Gn) by c (F ) := σ (F F F )(φµ˜ 1),φµ˜ 1 µ (˜µ,r) µ µ σ ∗ ∗ ⊗ ⊗ H (˜µ,r) = σ (F F )(φµ˜ 1),σ (F )(φ µ˜ 1) (˜µ,r) µ (˜µ,r) µ σ ∗ ⊗ ⊗ H (˜µ,r) µ˜ µ˜ = Uµ˜ σ(˜µ,r)(F Fµ)(φ 1) ,Uµ˜ σ(˜µ,r)(Fµ)(φ 1) ∗ ⊗ ⊗ (µ,r) D EH (47) = π (F ) π (F ) U (φµ˜ 1) , Φµ (ν,r) ◦ (ν,r) µ µ˜ ⊗ µ˜ ν S(˜µ) (µ,r) ∈X H µ˜ µ = π(µ,r)(F ) π(µ,r)(Fµ) Uµ˜(φ 1) , Φµ˜ ◦ ⊗ H(µ,r) (47) µ µ = π(µ,r)(F )Φµ˜, Φµ˜ (48) H(µ,r) for all F L1(G ). ∈ n Let X(˜µ αk) be the collection of allν ˜ = (ν1,...,νn) Pn such that χν˜ is a character T × ∈ of n appearing in µ˜ and such that τν˜k is contained in the representation τµ˜ Wαk for H k ⊗ ν˜k := (ν1,ν2,...,νn 1,λn). − Then, for π(˜µ,αk) defined as in Proposition 5.12, by [20], Chapter IV.11,

(˜µ,αk) π = π(˜νk,αk). ν˜ X(˜µ α ) ∈ X× k Furthermore, decompose the vector 1 ξ = φµ˜ h h k ⊗ 1 q,αk ⊗ q,αk Nk 2 q Nn: qX∈=N | | k

141 for every k N into the orthogonal sum ∈ ν˜ ξk = ξk ν˜ X(˜µ α ) ∈ X× k for ξν˜ . This gives a decomposition k ∈H(˜νk,αk) (˜µ,α ) (˜µ,αk) π k ν˜ π ( )ξ ,ξ = c = c ν . k k (n) (n) ξk ξ˜ · µ˜ αk αk k H ⊗F ⊗F ν˜ X(˜µ α ) ∈ X× k

Let cξν˜ be the weak∗-limit of a subsequence of cξν˜ k N and let for cξν˜ = 0 the re- k ∈ 6 presentation π G be an element of the support of c ν˜ . From Theorem 5.10 follows ∈ n ξ that π = lim π(˜ν ,α ) = π(ν,r) for ν = (ν1,...,νn 1). Furthermore, one observes that for k k k − →∞ b µ˜ =ν ˜ := (ν1,...,νn 1,νn) X(˜µ αk), one has ν = µ. Hence, π(ν,r)(Fµ) = 0. Thus, 6 − ∈ × 6 ν˜ ν˜ ν˜ ν˜ lim π(˜νk,αk)(Fµ)ξk , π(˜νk,αk)(Fµ)ξk = π(ν,r)(Fµ)ξ , π(ν,r)(Fµ)ξ = 0 k (˜νk,αk) (ν,r) →∞ H H and therefore, ν˜ ν˜ lim π(˜νk,αk)(F ) π(˜νk,αk)(Fµ)ξk , π(˜νk,αk)(Fµ)ξk = 0 F C∗(Gn). (49) k ◦ (˜νk,αk) ∀ ∈ →∞ H k k Now, sinceµ ˜k =(µ1,...,µn 1,λn)= λ , by Proposition 5.12 and its proof, for all F C∗(Gn), − ∈ µ µ π(µ,r)(F )Φµ˜, Φµ˜ H(µ,r) = c µ(Fµ) (48) = τ π (F F F )(φµ˜ 1),φµ˜ 1 µ˜ (1,r) µ µ σ ⊗ ∗ ∗ ⊗ ⊗ H (˜µ,r)

Proposition (˜µ,αk) = lim π (Fµ F Fµ)ξk,ξk (n) (n) 5.12 k ∗ ∗ µ˜ αk αk →∞ H ⊗F ⊗F ν˜ ν˜ = lim π(˜νk,αk)(Fµ F Fµ)ξk ,ξk k ∗ ∗ (˜νk,αk) →∞ ν˜ X(˜µ α ) H ∈ X× k (49) µ˜ µ˜ = lim π(˜µk,αk)(Fµ F Fµ)ξk ,ξk + 0 k ∗ ∗ (˜µk,αk) →∞ H µ˜ µ˜ = lim π(˜µk,αk)(F ) π(˜µk,αk)(Fµ)ξk , π(˜µk,αk)(Fµ)ξk k (˜µ ,α ) →∞ D EH k k µ˜ µ˜ k k k = lim π(λ ,αk)(F ) π(λ ,αk)(Fµ)ξk , π(λ ,αk)(Fµ)ξk . k k →∞ H(λ ,αk) D E µ µ˜ Choosing ξ˜ := Φ and ξ˜ := π k (F )ξ k , one has µ˜ ∈H(µ,r) k (λ ,αk) µ k ∈H(λ ,αk)

k ˜ ˜ ˜ ˜ lim π(λ ,αk)(F )ξk, ξk = π(µ,r)(F )ξ, ξ k (λk,α ) (µ,r) →∞ H k H and hence,

π(µ,r) = lim π(λk,α ). k k →∞

The inverse implication of 2) of Theorem 5.10 is formulated in the following conjecture:

142 Conjecture 5.13. [ Let r> 0 and τ U(n). λ ∈ If lim αk = 0 and the sequence π(λk,α ) N of elements of Gn satisﬁes one of the following k k k →∞ ∈ conditions: b k k k k (i) for k large enough, αk > 0, λ1 λ1 ... λn 1 λn 1 λn λn and lim αkλn = 0, ≥ ≥ ≥ − ≥ − ≥ ≥ k →∞ k k k (ii) for k large enough, αk < 0, λ1 λ1 λ2 λ2 ... λn 1 λn λn and k ≥ ≥ ≥ ≥ ≥ − ≥ ≥ lim αkλ1 = 0, k →∞

then the sequence π k converges to the representation τ in G . (λ ,αk) k N λ n ∈ 5.3.3 The representation τλ b [ As τλ only acts on U(n) and U(n) is discrete, every converging sequence (τλk )k N has to be constant for large k. Hence, ∈

k k τ k →∞ τ λ = λ for large k. λ −→ λ ⇐⇒ 5.4 Results Combining Theorem 5.4 in Section 5.2 that describes the topology of the space of all admissible coadjoint orbits of Gn with the Theorems 5.6, 5.8 and 5.10 in Section 5.3 and the result in Subsection 5.3.3 which characterize the spectrum Gn of Gn, one gets the following result: Theorem 5.14. c The mapping

G g‡ /G , π n −→ n n 7→ Oπ is continuous. b Once one has succeeded to prove Conjecture 5.13, together with the Theorems 5.6, 5.8, 5.10 and 5.11 and the result in Subsection 5.3.3, one gets the result below: Conjecture 5.15. For general n N , the spectrum of the group G = U(n) ⋉H is homeomorphic to its space ∈ ∗ n n of admissible coadjoint orbits gn‡ /Gn. For n = 1, the situation is a lot easier than for general n N and therefore, this conjecture ∈ ∗ could directly be proved in this case: Theorem 5.16. The spectrum of the semidirect product U(1)⋉H1 is homeomorphic to its admissible coadjoint orbit space. Proof: r2 Assume that (αk)k N tends to 0 and that lim λkαk = . If αk is positive (respectively ∈ k − 2 →∞ negative) for k large enough, one can deﬁne the sequence (fk)k N of elements in the Fock ∈ space (1) by f (w) := c w λk (respectively f (w) := c wλk ) with c C such Fαk k αk,λk − k αk,λk αk,λk ∈ that fk α (1) = 1. Then, for f C0∞(G1), one has k kF k ∈

GZ1

λk λ − k j α ∞ 2 −l αk j+l λ j ( λ l) k w 2 c w w− k ( z) z − k− e− 2 | | dw d(θ,z,t) | αk,λk | j! 2 − j=0 l=0 ! X X ZC λk α j iα t k z 2 − ( λk)! αk 2 j = f(θ,z,t)e k − 4 | | − z d(θ,z,t). ( λ j)!(j!)2 2 −| | Z j=0 − k − ! G1 X Now, ﬁxing large k, one gets for every j 0,..., λ , ∈ { − k} ( λ )! α j j − k k z 2 ( λ j)!(j!)2 2 −| | − k − α λ j ( λ )! 1 k k k 2 j = − j− 2 z 2 ( λk) ( λk j)! (j!) −| | − − − 2 r j λ λ 1 λ j + 1 1 j + 1 − k − k − − k − z 2 ≤ 4 λ · λ ··· λ (j!)2 −| | − k − k − k r2 j 1 + 1 z 2 . ≤ 4 | | (j!) This expression does not depend on k and

j ∞ r2 1 r2 + 1 z 2 = exp + 1 z 2 < . 4 | | (j!) 4 | | ∞ Xj=0 So, by the theorem of Lebesgue, this sum converges and it suﬃces to regard the limit of each summand by itself:

r2 z 2 j αkλk j ( λk)! 1 j k − | | − − z 2 →∞ 4 . 2 ( λ )j( λ j)! (j!)2 −| | −→ (j!)2 − k − k − 

144 Therefore, one gets

r2 z 2 j π ∞ − | | lim C (λk,αk) ,f = f(θ,z,t) 4 d(θ,z,t) fk ∞ 1 2 k L (G1),L (G1) (j!) →∞ Z j=0 ! D E G1 X Bessel function 2π |1 {z ir Re(e}iβ z) = f(θ,z,t) e− dβd(θ,z,t) 2π 0 GZ1 Z 2π iβ 1 i(e r,z)Cn = f(θ,z,t) e− dβd(θ,z,t) 2π 0 GZ1 Z

i(Bvr,z)Cn = f(θ,z,t) e− dBd(θ,z,t)

GZ1 UZ(1)

G1 = f(θ,z,t) indH χr (θ,z,t)(1), 1 . 1 L2(G /H ,χ ) Z 1 1 r G1 D E Hence, π converges to the irreducible unitary representation π := indG1 χ . (λk,αk) k N r H1 r ∈ Assume now that lim λkαk = 0. For λ λk (respectively for λ λk) for k large enough, k ≥ ≤ →∞ λ λk λk λ deﬁne the sequence (fk)k N by fk(w) := cαk,λk,λw − (respectively fk(w) := cαk,λk,λw − ) C ∈ for cαk,λk,λ such that fk α (1) = 1. Then, by the same computation as above, one gets ∈ k kF k

π (λk,αk) lim C ,f = f(θ,z,t)χλ(θ)d(θ,z,t)= χλ,f . fk ∞ 1 ∞ 1 k L (G1),L (G1) h i L (G1),L (G1) →∞ D E GZ1

Thus, π(λk,αk) k N converges to the character χλ of U(1). ∈

145 6 Résuméétendu de cette thèse en fran¸cais

6.1 Introduction

Dans la présente thèse de doctorat, la structure des C∗-algèbres des groupes de Lie connexes réels nilpotents de pas deux et la structure de la C∗-algèbre du groupe de Lie SL(2, R) sont analysées. En outre, en vue de la détermination de sa C∗-algèbre, la topologie du spectre du produit semi-direct U(n) ⋉Hn est décrite, où Hn dénote le groupe de Lie de Heisenberg et U(n) le groupe unitaire qui agit sur Hn par automorphismes.

La recherche des C∗-algèbres – une abstraction des algèbres d’opérateurs linéaires bornés sur des espaces de Hilbert – a commencédans les années 1930 en raison d’un besoin dans la mécanique quantique et plus précisément pour servir de modèles mathématiques pour des algèbres d’observables physiques. Des systèmes quantiques sont décris àl’aide d’opérateurs auto-adjoints sur des espaces de Hilbert. Par conséquent, des algèbres d’opérateurs bornés sur ces espaces étaient considérées. Le terme “C∗-algèbre” a premièrement étéintroduit dans les années 1940 par I.Segal pour décrire des sous-algèbres fermées pour la norme de l’algèbre d’opérateurs linéaires bornés sur un espace de Hilbert. Entre-temps, les C∗-algèbres représentent un outil important dans la théorie des représentations unitaires des groupes localement compacts et dans la description mathématique de la mécanique quantique. Les groupes de Lie ont étéintroduits dans les années 1870 par S.Lie dans le cadre de la théorie de Lie pour examiner des symétries continues dans des équations différentielles et leur théorie a évoluépendant le 20ième siècle. Aujourd’hui, les groupes de Lie sont utilisés dans plusieurs domaines mathématiques et dans la physique théorique, par exemple dans la physique des particules. Puisque des groupes de Lie et d’autres groupes de symétries jouent un rôle capital dans la physique, l’examination de leurs C∗-algèbres de groupes – des C∗-algèbres qui sont construites en formant une fermeture de l’espace de toutes les fonctions L1 sur ces groupes respectifs – a étépoursuivie et avancée. Cette tâche représente l’objectif de cette thèse de doctorat. La méthode d’analyse d’une C∗-algèbre d’un groupe utilisée dans ce travail a étéinitiée par G.M.Fell dans les années 1960. On utilise la transformation de Fourier non-abélienne afin d’écrire les C∗-algèbres comme des algèbres de champs d’opérateurs et d’étudier ainsi leurs éléments. Une exigence afin d’utiliser cette méthode, et ainsi de comprendre ces C∗-algèbres, est la connaissance de leur spectre et de sa topologie, lesquels sont inconnus pour beaucoup de groupes. Par conséquent, cela représente un problème majeur de la procédure de Fell. L’avancement de la méthode utilisée dans cette thèse a été élaborée pendant les dernières années (voir [24] et [26]) et cela rend l’étude d’une large classe de C∗-algèbres de groupes possible.

Afin de comprendre les C∗-algèbres, la transformation de Fourier est un outil important. Si on note A le dual unitaire ou le spectre d’une C∗-algèbre A, c.à.d. l’ensemble de toutes les classes d’équivalence des représentations irréductibles unitaires de A, la transformée de Fourier (a) =ba ˆ d’un élément a A est définie de la manière suivante: On choisit dans F ∈ chaque classe d’équivalence γ A une représentation (π , ) et définit ∈ γ Hγ (a)(γ) := π (a) ( ), bF γ ∈B Hγ

146 où ( ) désigne la C -algèbre des opérateurs linéaires bornés sur . Alors, (a) est contenu B Hγ ∗ Hγ F dans l’algèbre de tous les champs d’opérateurs bornés sur A

l∞(A)= φ = φ(πγ) ( γ) φ :=b sup φ(πγ) op < γ A ∞ ∈B H ∈ b k k γ A k k ∞ n ∈ b o b et l’application : A l∞(A), a aˆ F → 7→ est un -homomorphisme isométrique. ∗ b Soit dx la mesure de Haar du groupe localement compact G. On peut définir sur L1(G) := L1(G, C) le produit de convolution comme suit: ∗

1 1 f f ′(g) := f(x)f ′ x− g dx f,f ′ L (G) g G ∗ ∀ ∈ ∀ ∈ Z G et cela donne une algèbre de Banach L1(G), . En outre, on obtient une involution ∗ isométrique en définissant 1 1 f ∗(g):=∆ (g)− f g g G, G − ∀ ∈ où∆G est la fonction modulaire. Pour toute représentation irréductible unitaire (π, ) de G, une représentation (˜π, ) de H H L1(G) peut être définie de la manière suivante:

π˜(f) := f(g)π(g)dg f L1(G). ∀ ∈ GZ Cette représentation est irréductible et unitaire aussi. Maintenant, la C∗-algèbre de G est définie comme la fermeture de l’algèbre de convolution 1 1 L (G) par rapport àla C∗-norme de L (G), c.à.d.

1 C∗(G) C∗(G) := L (G,dx)k·k avec f C∗(G) := sup π(f) op, k k [π] G k k ∈ b où G est le spectre de G qui est défini comme en haut comme l’ensemble de toutes les classes d’équivalence des représentations irréductibles unitaires de G. De plus,b toute représentation irréductible unitaire (˜π, ) de L1(G) peut être écrite de fa¸con H unique comme une intégrale de la fa¸con montrée en haut et alors, on obtient une représentation irréductible unitaire (π, ) de G. C’est pourquoi on a le résultat suivant, bien connu, qui peut H être trouvédans [9], Chapitre 13.9, et qui dit que le spectre de C∗(G) se correspond avec le spectre de G: \ C∗(G)= G.

b La structure des C∗-algèbres des groupes est déjàconnue pour certaines classes de groupes de Lie, comme le groupe de Heisenberg et les groupes de Lie filiformes (voir [26]) et les groupes “ax+b” (voir [24]). En outre, les C∗-algèbres des groupes de Lie nilpotents de dimension 5 ont étédéterminées dans [31], pendant que celles de tous les groupes de Lie nilpotents de

147 dimension 6 ont étécaractérisées par H.Regeiba dans [30]. Dans cette thèse, un résultat démontrépar H.Regeiba et J.Ludwig (voir [31], Théorème 3.5) est utilisépour la caractérisation des C∗-algèbres mentionnées en haut. Pour les C∗-algèbres des groupes de Lie nilpotents de pas deux, l’approche est en partie similaire, mais plus complexe, que celle qui est utilisée pour la caractérisation de la C∗-algèbre du groupe de Lie de Heisenberg (voir [26]), qui est, lui aussi, nilpotent de pas deux et sert ainsi d’exemple. Dans le cas du groupe de Lie de Heisenberg, la situation est beaucoup plus facile que pour des groupes de Lie nilpotents de pas deux généraux, parce que dans ce cas spécial les orbites coadjointes qui apparaissent ne peuvent qu’avoir deux différentes dimensions, tandis que dans le cas général il y a beaucoup de différentes dimensions qui peuvent apparaˆıtre. Par conséquent, des cas beaucoup plus difficiles se produisent. En revanche, les groupes de Lie nilpotents de dimension 5 et 6 ont une structure plus compliquée que les groupes de Lie nilpotents de pas deux. Ici, des orbites coadjointes non-plates apparaissent, alors que dans le cas des groupes de Lie nilpotents de pas deux, on est justement obligéde traiter les orbites coadjointes plates. Pour cette raison, dans ces cas des groupes de Lie nilpotents de dimension 5 et 6, on examine tout groupe cas par cas, tandis que pour les groupes de Lie nilpotents de pas deux, on traite toute cette classe de groupes sans connaˆıtre leurs structures exactes. Comme mentionnéen haut, l’approche pour le groupe SL(2, R) est complètement différente. Puisqu’on traite un seul groupe qui est explicitement donné, dans ce cas, on peut effectuer des calculs plus concrets. Pour des groupes de Lie semi-simples en général, il n’y a pas de description explicite de leurs C∗-algèbres de groupe dans la littérature. Cependant, pour les groupes de Lie semi-simples dont le spectre est classifié, la procédure de détermination du C∗-algèbre de groupe utilisée dans ce travail pourrait être appliquée avec succès d’une fa¸con similaire. Une description de la C∗-algèbre du groupe de Lie SL(2, C) est donnée dans [13], Théorème 5.3 et Théorème 5.4, et une caractérisation des C∗-algèbres réduites des groupes de Lie semi-simples peut être trouvée dans [34]. Dans la présente thèse de doctorat, les C∗-algèbres des groupes de Lie connexes réels nilpotents de pas deux et de SL(2, R) sont décrites très explicitement. Elles sont caractérisées par des conditions qui s’appellent “limites duales sous contrôle normique” et qui seront données ci-dessous. L’objectif principal est de calculer ces conditions d’une fa¸con concrète et de construire les “contrôles normiques” nécessaires. Dans un résultat abstrait d’existence (voir [2], Théorème 4.6), ces conditions sont vérifiées pour tous les groupes de Lie connexes simplement connexes. Elles sont explicitement calculées pour tous les groupes de Lie nilpotents de dimension 5 et 6 (voir [31]), pour les groupes de Lie de Heisenberg et les groupes de Lie filiformes (voir [26]). Les résultats sur les C∗-algèbres des groupes de Lie connexes réels nilpotents de pas deux ont étépubliés dans la “Revista Matemática Complutense”, article en commun avec J.Ludwig (voir [17]). Un autre article sur la C∗-algèbre du groupe SL(2, R) a étésoumis au “Journal of Lie Theory” et peut être trouvésur arXiv (voir [16]).

Puisque pour la détermination d’une C∗-algèbre de groupe le spectre du groupe et sa topologie sont nécessaires, dans cette thèse de doctorat, la topologie du spectre des groupes G = U(n) ⋉H pour n N = N 0 est analysée. Ce travail a étéinitiépar M.Elloumi n n ∈ ∗ \{ } et J.Ludwig (voir [11], Chapitre 3) et leur prépublication est complétée dans cette thèse.

148 Comme pour plusieurs autres classes de groupes de Lie, par exemple les groupes de Lie exponentiels résoluble et les groupes de mouvements euclidiens, le spectre Gn peut être paramétrépar l’espace des orbites coadjointes admissibles de l’espace dual gn∗ de l’algèbre de Lie gn de Gn. c Le but est de démontrer que cette bijection entre l’espace des orbites coadjointes admissibles muni de la topologie quotient et le spectre Gn de Gn est un homéomorphisme. Cela a déjà étédémontrépour les groupes de mouvements classiques SO(n) ⋉Rn pour n 2 dans [12]. ≥ Pour les groupes Gn = U(n) ⋉Hn, cette assertionc a étéprouvée pour n = 1. Dans le cas général, il a pu être démontréque l’application entre le spectre Gn et l’espace des orbites coadjointes admissibles de Gn est continue. Les parties restantes de la preuve de l’assertion sont en progrès. c

Dans cette sous-section, la définition d’une C∗-algèbre à “limites duales sous contrôle normique” sera donnée. Les conditions dont on a besoin pour cette définition caractérisent des C∗-algèbres de groupes. Après cela, dans la Section 6.3, ces conditions seront analysées pour les groupes de Lie nilpotents de pas deux. Dans la Section 6.4, elles seront vérifiées dans le cas du groupe de Lie SL(2, R), dans une approche très différente àcelle utilisée pour les groupes de Lie nilpotents de pas deux. Définition 6.1. Une C∗-algèbre A est appelée C∗-algèbre à“limites duales sous contrôle normique” si elle satisfait les conditions suivantes: - Condition 1: Stratification du spectre: (a) Il existe une famille finie croissante S S ... S = A de sous-ensembles 0 ⊂ 1 ⊂ ⊂ r fermésdu spectre A de A telle que pour i 1, ,r les sous-ensembles Γ = S et ∈ { ··· } 0 0 Γi := Si Si 1 sont Hausdorff dans leurs topologies relatives et telleb que S0 constitue \ − l’ensemble de tousb les caractères, c.à.d. les représentations de dimension 1, de A. (b) Pour tout i 0, ,r il y a un espace de Hilbert et pour tout γ Γ il existe ∈ { ··· } Hi ∈ i une réalisation concrète (π , ) de γ sur l’espace de Hilbert . γ Hi Hi - Condition 2: CCR C∗-algèbre: A est une CCR C∗-algèbre (ou C∗-algèbre liminaire) séparable, c.à.d. une C∗-algèbre séparable telle que l’image de toute représentation irréductible (π, ) de A est contenue H dans l’algèbre d’opérateurs compacts ( ) (ce qui implique que l’image est égale à ( )). K H K H - Condition 3: Changement de couches: Soit a A. ∈ (a) Sur chacun des différents ensembles Γ , l’application γ (a)(γ) est continue par i 7→ F rapport àla norme. (b) Pour tout i 0, ,r et pour toute suite convergente contenue dans Γ dont ∈ { ··· } i l’ensemble limite se trouve au dehors de Γi (alors dans Si 1), il existe une sous-suite − proprement convergente γ = (γk)k N (c.à.d. les sous-suites de γ ont toutes le même ensemble limite – voir la Définition∈ 6.7), ainsi qu’une constante C > 0 et pour tout k N une application linéaire involutive ν˜k =ν ˜γ,k : CB(Si 1) ( i), qui est bornée ∈ − →B H par C , telles que k · kSi−1

lim (a)(γk) ν˜k (a) Si = 0. k F − F | −1 op →∞

149 Ici, CB(Si 1) est la -algèbre de tous les champs d’opérateurs uniformément bornés − ∗ ψ(γ) ( j) , qui sont continus pour la norme d’opérateurs sur les ∈ B H γ Γj ,j=0, ,i 1 sous-ensembles Γ ∈ pour tout··· −j 0, ,i 1 , munie avec la norme infinie j ∈ { ··· − }

ψ Si−1 := sup ψ(γ) op. k k γ Si k k ∈ −1 Théorème 6.2 (Théorème principal). Les C∗-algèbres des groupes de Lie nilpotents de pas deux G et du groupe de Lie G = SL(2, R) sont des C∗-algèbres àlimites duales sous contrôle normique.

Les propriétés en haut caractérisent complètement la structure d’une C∗-algèbre (voir [31], Théorème 3.5).

6.2 Définitions et résultats généraux 6.2.1 Les groupes de Lie nilpotents – Préliminaires Soit g une algèbre de Lie nilpotente. On prend sur g un produit scalaire , et la multiplication de Campbell-Baker-Hausdorff h· ·i 1 1 1 u v = u + v + [u,v]+ u, [u,v] + v, [v,u] + ... u,v g, · 2 12 12 ∀ ∈ laquelle est une somme finie comme g est nilpotente. Cela donne le groupe de Lie connexe simplement connexe G = (g, ) avec l’algèbre de Lie g. · L’application exponentielle exp : g G =(g, ) est en ce cas l’identité. → · La mesure de Haar de ce groupe est une mesure de Lebesgue qui est dénommée dx.

Convention 6.3. Dans cette section, tous les espaces de fonctions sont des espaces de fonctions `avaleurs complexes.

Maintenant, pour une fonctionnelle linéaire ℓ de g, on considère la forme antisymétrique

B (X,Y ) := ℓ, [X,Y ] ℓ h i sur g. De plus, soit

g(ℓ) := X g ℓ, [X, g] = 0 ∈ | h i { } le radical de Bℓ et le stabilisateur de la fonctionnelle linéaire ℓ. Définition 6.4 (Polarisation). Une sous-algèbre p de g qui est subordonnéeà ℓ (c.à.d. qui satisfait ℓ, [p, p] = 0 ) et qui a h i { } la dimension 1 dim(p)= dim(g)+ dim(g(ℓ)) , 2 ce qui veut dire que p est maximal isotrope pour Bℓ, s’appelle une polarisation de ℓ.

150 Si p g est une sous-algèbre quelconque de g qui est subordonnée à ℓ, la fonctionnelle linéaire ⊂ ℓ définit un caractère unitaire χℓ de P := exp(p): 2πi ℓ,log(x) 2πi ℓ,x χ (x) := e− h i = e− h i x P. ℓ ∀ ∈ Définition 6.5 (L’action coadjointe). On définit pour tout g G les applications ∈ 1 α : G G, x gxg− et Ad (g) := d(α ) : g g. g → 7→ g e → Alors, Ad : G GL(g) est une représentation du groupe. L’action coadjointe Ad ∗ peut être → définie comme suit:

1 Ad ∗(g): g∗ g∗ g G, Ad ∗(g)ℓ(X)= ℓ Ad g− X ℓ g∗ X g. → ∀ ∈ ∀ ∈ ∀ ∈ Ad ∗ : G GL(g ) est une autre représentation du groupe. → ∗ L’espace d’orbites coadjointes Ad ∗(G)ℓ ℓ g est dénommé g /G. { | ∈ ∗} ∗ En outre, on définit l’application

ad ∗(X): g∗ g∗ X g, ad ∗(X)ℓ(Y )= ℓ([Y,X]). → ∀ ∈ ad ∗ : g GL(g ) est une représentation de l’algèbre de Lie g. → ∗ Définition 6.6 (Représentation induite). Soit H un sous-groupe ferméde G et soit

L2(G/H, χ ) := ξ : G C ξ mesurable, ξ(gh)= χ (h)ξ(g) g G h H, ℓ → ℓ ∀ ∈ ∀ ∈ 

ξ 2 := ξ(g) 2dg˙ < , k k2 | | ∞ G/HZ où dg˙ est une mesure invariante sur G/H qui existe toujours pour un groupe G nilpotent. 2 Alors, L (G/H, χℓ) est un espace de Hilbert et on peut définir la représentation induite G 1 2 ind χ (g)ξ(u) := ξ g− u g,u G ξ L (G/H, χ ). H ℓ ∀ ∈ ∀ ∈ ℓ Cela donne une représentation unitaire. Si l’algèbre de Lie associée h de H est une polarisation de ℓ, cette représentation est irréductible.

6.2.2 Les groupes de Lie nilpotents – La méthode des orbites D’après la théorie de Kirillov (voir [8], Section 2.2), pour toute classe de représentations γ G, il existe un élément ℓ g et une polarisation p de ℓ dans g tels que γ = indGχ , ∈ ∈ ∗ P ℓ où P := exp(p). En outre, si ℓ,ℓ g se trouvent dans la même orbite coadjointe g /G ′ ∈ ∗ O ∈ ∗ et si bp et p′ sont des polarisations de ℓ et de ℓ′ respectivement, les représentations induites G G indP χℓ et indP ′ χℓ′ sont équivalentes et l’application de Kirillov qui va de l’espace des orbites coadjointes g∗/G dans le spectre G de G G K : g∗/G G, Ad ∗(G)ℓ ind χ b → 7→ P ℓ est un homéomorphisme (voir [6] où[23], Chapitre 3). Par conséquent, b g∗/G ∼= G en tant qu’espaces. b

151 Définition 6.7. Une suite (tk)k N d’un espace topologique séparable T est dite proprement convergente si ∈ (tk)k N a des points limites et si toute sous-suite de (tk)k N a le même ensemble limite que ∈ ∈ (tk)k N. ∈ Il est bien connu que toute suite convergente dans T possède une sous-suite qui est proprement convergente.

6.2.3 Les groupes de Lie nilpotents – La C∗-algèbre C∗(G/U, χℓ) Soit u g un idéal de g contenant [g, g], U := exp(u) et soit ℓ g vérifiant ℓ, [g, u] = 0 ⊂ ∈ ∗ h i { } et u g(ℓ). Alors, le caractère χℓ du groupe U = exp(u) est G-invariant. Par conséquent, on ⊂ 1 peut définir l’algèbre de Banach involutive L (G/U, χℓ) comme suit:

1 1 L (G/U, χ ) := f : G C f mesurable, f(gu)= χ (u− )f(g) g G u U, ℓ → ℓ ∀ ∈ ∀ ∈ 

f := f(g) dg˙ < . k k1 | | ∞ G/UZ 

La convolution

1 f f ′(g) := f(x)f ′(x− g)dx˙ g G ∗ ∀ ∈ G/UZ

et l’involution

1 f ∗(g) := f(g ) g G − ∀ ∈ sont bien d´eﬁnies pour f,f L1(G/U, χ ) et ′ ∈ ℓ

f f ′ f f ′ . k ∗ k1 ≤ k k1 k k1 En outre, l’application lin´eaire

p : L1(G) L1(G/U, χ ), p (F )(g) := F (gu)χ (u)du F L1(G) g G G/U → ℓ G/U ℓ ∀ ∈ ∀ ∈ UZ est un -homomorphisme surjectif entre les alg`ebres L1(G) et L1(G/U, χ ). ∗ ℓ Soit

I Gu,ℓ := [π] G π U = χℓ U π . ∈ | | H n o Alors Gu,ℓ est un sous-ensembleb ferméde Gb qui peut être identifiéavec le spectre de l’algèbre L1(G/U, χ ). En fait, il est facile de voir que toute représentation irréductible unitaire (π, ) ℓ Hπ dont lab classe d’équivalence est contenueb dans Gu,ℓ définit une représentation irréductible (˜π, ) de l’algèbre L1(G/U, χ ) comme suit: Hπ ℓ b π˜ p (F ) := π(F ) F L1(G). G/U ∀ ∈ 152 De fa¸con similaire, si (˜π, ) est une représentation irréductible unitaire de L1(G/U, χ ), alors Hπ˜ ℓ [π] donnépar

π :=π ˜ p ◦ G/U est un ´el´ement de Gu,ℓ. Soit s g un sous-espace de g tel que g = g(ℓ) s. Comme u contient [g, g], il est facile de ⊂ ⊕ voir que b

G = χ π q (u + s)⊥ , u,ℓ q ⊗ ℓ | ∈ G si on note πℓ := indP χℓ, où pb désigne une polarisation de ℓ et P := exp(p). 1 On dénomme C∗(G/U, χℓ) la C∗-algèbre associée à L (G/U, χℓ) dont le spectre peut être identifiéavec Gu,ℓ. En définissant π := indGχ , la transformation de Fourier définie par ℓ+q P ℓ+q F b (a)(q) := π (a) q (u + s)⊥, F ℓ+q ∀ ∈

envoie la C∗-algèbre C∗(G/U, χℓ) sur l’algèbre C (u + s)⊥, ( π ) des applications ∞ K H ℓ continues ϕ : (u + s) ( ) qui tendent vers zéro àl’infini et àvaleurs dans l’algèbre ⊥ → K Hπℓ des opérateurs compacts sur l’espace de Hilbert de la représentation π . Hπℓ ℓ Si on restreint p àl’algèbre de Fréchet (G) L1(G), son image est l’algèbre de Fréchet G/U S ⊂ 1 (G/U, χ ) = f L (G/U, χ ) f lisse et pour tout sous-espace s′ g avec g = s′ u S ℓ ∈ ℓ | ⊂ ⊕ et pour S′ = exp(s′), f S′ (S′) . | ∈S

6.2.4 Résultats généraux Théorème 6.8 (La topologie du spectre).

Soit (πk, πk )k N une famille de représentations irréductibles unitaires d’un groupe de Lie G. H ∈ Alors (πk)k N converge vers π dans G si et seulement si pour un vecteur non nul (respectivement pour tous∈ les vecteurs non nul) ξ dans et pour tout k N, il existe ξ tel que Hπ ∈ k ∈Hπk la suite des coefficients matriciels bπ ( )ξ ,ξ converge uniformément vers π( )ξ,ξ k · k k k N · sur tout espace compact. ∈ La démonstration de ce théorème se trouve dans [9], Théorème 13.5.2.

Lemme 6.9. Soient G un groupe de Lie et g l’algèbre de Lie correspondante. On définit la projection canonique pG de g∗ àvaleurs dans l’espace des orbites coadjointes g∗/G et on munit l’espace g∗/G de la topologie quotient, c.à.d. un sous-ensemble U dans g∗/G est ouvert si et seulement 1 si p− (U) est ouvert dans g∗. Alors, une suite ( k)k N d’éléments dans g∗/G converge vers G O ∈ l’orbite g /G si et seulement si pour ℓ , il existe ℓ pour tout k N tel que O ∈ ∗ ∈ O k ∈ Ok ∈ ℓ = lim ℓk. k →∞ Pour la preuve de ce lemme, voir [23], Chapitre 3.1.

153 6.3 Les C∗-algèbres des groupes de Lie connexes réels nilpotents de pas deux Dans cette sous-section, les groupes de Lie connexes réels nilpotents de pas deux seront examinés. Dans la première partie, quelques préliminaires sur les groupes de Lie nilpotents de pas deux seront donnés lesquels seront nécessaires pour comprendre la situation et en particulier la preuve des conditions énumérées dans la Définition 6.1. Dans la Section 6.3.3, les Conditions 1, 2 et 3(a) de la Définition 6.1 seront discutées et dans les Sections 6.3.4 à6.3.8, la Condition 3(b) sera traitée. Ensuite, un résultat qui décrit les C∗-algèbres des groupes de Lie connexes réels nilpotents de pas deux sera donné.

La démonstration de la Condition 1 de la Définition 6.1 consiste en la construction des différentes couches du spectre de G et repose sur une méthode de construction d’une polarisation qui est développée dans [27]. Elle est plutôttechnique mais pas très longue. La Condition 2 est la conséquence directe d’un résultat dans [8] et la Condition 3(a) dérive de la construction de la polarisation mentionnée plus haut. La majeure partie du travail pour la détermination des C∗-algèbres des groupes de Lie nilpotents de pas deux consiste en la preuve de la Propriété3(b) de la Définition 6.1, et en particulier en la construction des applications (˜νk)k N. ∈

6.3.1 Préliminaires – Les groupes de Lie nilpotents de pas deux Soit g une algèbre de Lie réelle nilpotente de pas deux. Cela signifie que

[g, g] := vect [X,Y ] X,Y g | ∈ est contenu dans le centre de g. On fixe un produit scalaire , sur g comme pour les algèbres de Lie nilpotentes générales. h· ·i La multiplication de Campbell-Baker-Hausdorff est dans ce cas de la forme 1 u v = u + v + [u,v] u,v g · 2 ∀ ∈ et on obtient de nouveau le groupe de Lie connexe simplement connexe G = (g, ) avec · l’algèbre de Lie g.

Comme g est nilpotente de pas deux, [g, g] g(ℓ) et alors g(ℓ) est un id´eal de g. ⊂ De plus, pour ℓ g , tout sous-espace maximal isotrope p de g pour B qui contient [g, g] est ∈ ∗ ℓ une polarisation de ℓ.

6.3.2 Préliminaires – La méthode des orbites Dans le cas des groupes de Lie nilpotents de pas deux, pour tout élément ℓ g et pour tout ∈ ∗ g G =(g, ) l’élément Ad ∗(g)ℓ est donnépar ∈ ·

Ad ∗(g)ℓ = Ig∗ + ad ∗(g) ℓ ℓ + g(ℓ)⊥. ∈ Par cons´equent, comme ad ∗(g)ℓ = g(ℓ)⊥,

:= Ad ∗(G)ℓ = Ad ∗(g)ℓ g G = ℓ + g(ℓ)⊥ ℓ g∗, (50) Oℓ { | ∈ } ∀ ∈

154 ce qui veut dire que uniquement des orbites plates apparaissent. Ainsi,

g∗/G = Ad ∗(G)ℓ ℓ g∗ = ℓ + g(ℓ)⊥ ℓ g∗ . { | ∈ } | ∈ Alors, par la th´eorie de Kirillov, on obtient 

G = g∗/G = ℓ + g(ℓ)⊥ ℓ g∗ ∼ | ∈ en tant qu’espaces topologiques.b

Maintenant, on considère une suite proprement convergente [πk] k N dans G avec ∈ pour ensemble limite L [πk] k N . Soient g∗/G l’orbite coadjointe d’un élément ∈ O ∈ b N [π] L [πk] k N , k l’orbite coadjointe de [πk] pour tout k et soit ℓ . Alors, selon ∈ ∈ O ∈ ∈O le Lemme 6.9, il existe pour tout k N un élément ℓk k tel que lim ℓk = ℓ dans g∗. En ∈ ∈ O k →∞ considérant une sous-suite si nécessaire, on peut supposer que la suite (g(ℓk))k N converge par rapport àla topologie de la grassmannienne vers une sous-algèbre u de g(ℓ) et∈ qu’il existe un nombre d N tel que dim( ) = 2d pour tout k N. D’après (50), il résulte que ∈ Ok ∈

L(( k)k N)= lim ℓk + g(ℓk)⊥ = ℓ + u⊥ = ℓ + u⊥ g∗/G. O ∈ k O ⊂ →∞

Puisque dans le cas des groupes de Lie nilpotents de pas deux g(ℓk) contient [g, g] pour tout k N, le sous-espace u contient également [g, g]. Ainsi, étant donnéque l’application ∈ de Kirillov est un homéomorphisme et vu que u⊥ n’est constituéque de caractères de g, l’ensemble limite L [πk] k N dans G de la suite [πk] k N est le sous-ensemble “affine” ∈ ∈ b G L [πk] k N = χq indP χℓ q u⊥ ∈ ⊗ | ∈ pour une polarisation p deℓ et P := exp(p).

Ces observations conduisent àla proposition suivante: Proposition 6.10. Il y a trois types différents d’ensembles limites possibles de la suite ( k)k N d’orbites coad- O ∈ jointes:

1. dim( ℓ) = 2d: dans ce cas, l’ensemble limite L(( k)k N) est le singleton ℓ = ℓ + g(ℓ)⊥, O O ∈ O c.`a.d. u = g(ℓ).

N 2. dim( ℓ) = 0: ici, L(( k)k )= ℓ+u⊥ et l’ensemble limite L [πk] k N n’est constitué O O ∈ ∈ que de caractères, c.à.d. q [g, g] = 0 pour tout q L(( k)k N). { } O ∈ O ∈ 3. La dimension de l’orbite est strictement supérieure à 0 et strictement inférieure à Oℓ 2d. Dans ce cas 0 < dim( ) < 2d pour tout L(( k)k N) et O O∈ O ∈ G N L(( k)k )= q + ℓ, c.à.d. L [πk] k N = χq indP χℓ O ∈ O ∈ ⊗ q u⊥ q u⊥ ∈[ [∈ pour une polarisation p de ℓ et P := exp(p).

155 6.3.3 Les Conditions 1, 2 et 3(a) A présent, on commence la vérification des conditions de la Définition 6.1.

A l’aide d’une procédure de construction de la polarisation de Vergne qui est élaborée dans [27], Section 1, on peut définir les familles de sous-ensembles (Si)i 0,...,r et (Γi)i 0,...,r du ∈{ } ∈{ } spectre G de G mentionnés dans la Condition 1 de la Définition 6.1 et on peut prouver qu’ils satisfont les propriétés souhaitées dans la Condition 1. Dans cetteb construction, pour chaque i 0,...,r , on obtient un nombre d N tel que ∈ { } i ∈ toute orbite appartenant àune représentation π Γ a la même dimension 2d , comme O ∈ i i vu précédemment. On peut aussi démontrer que l’espace de Hilbert commun qui apparaˆıt Hi dans la Condition 1(b) peut être choisi comme étant = L2(Rdi ). Hi Pour la preuve des Propriétés 2 et 3(a) de la Définition 6.1, on a la proposition suivante: Proposition 6.11. Pour tout a C (G) et pour tout i 0,...,r , l’application ∈ ∗ ∈ { } Γ L2(Rdi ), γ (a)(γ) i → 7→ F est continue par rapport àla norme et l’opérateur (a)(γ) est compact pour tout γ Γ . F ∈ i La compacitéest la conséquence directe d’un théorème général qui peut être trouvédans [8], Chapitre 4.2 ou [29], Partie II, Chapitre II.5 et qui dit que la C∗-algèbre C∗(G) de tout groupe de Lie connexe nilpotent G satisfait la Condition CCR, c.à.d. l’image de toute représentation irréductible de C∗(G) est un opérateur compact. La continuitérésulte de la construction des ensembles Γ pour i 0,...,r et du fait que les i ∈ { } polarisations de Vergne pV de ℓ g qui sont utilisées pour cette construction sont continues ℓ ∈ ∗ en ℓ par rapport àla topologie de la grassmannienne. Comme C∗(G) est séparable, cela démontre les Propriétés 2 et 3(a) de la Définition 6.1. Il reste àprésent àdémontrer la Condition 3(b).

6.3.4 Condition 3(b) – Introduction de la situation Pour simplifier, àpartir de maintenant, les représentations seront identifiées avec leurs classes d’équivalence.

On fixe i 0,...,r . ∈ { } V V V Pour ℓ′ g∗, soient pℓ′ la polarisation de Vergne de ℓ′ mentionnée en haut, Pℓ′ := exp pℓ′ et V ∈ G ′ ′ πℓ′ := indP V χℓ . De plus, pour une orbite ′ soit ℓ ′ un élément fixe dont on a besoin ℓ′ O O ∈ O pour la construction de la polaristation de Vergne.

Soit πV = πV une suite dans Γ dont l’ensemble limite se trouve hors de k k N ℓO k N i ∈ k ∈ Γi. Comme toute suite convergente a une sous-suite proprement convergente, on supposera V que πk k N est proprement convergente et la transition àune sous-suite sera omise. ∈ Alors la suite πV correspondra àla sous-suite (γ ) N de la Condition 3(b) de la k k N k k Définition 6.1. C’est pourquoi∈ pour tout k N, on doit construire∈ une application linéaire ∈ involutiveν ˜k : CB(Si 1) ( i) bornée par C Si qui satisfait − →B H k · k −1 V lim (a) πk ν˜k (a) Si−1 = 0, k F − F | op →∞

156 avec une constante C > 0 indépendante de k. Pour réaliser ceci, on regarde la suite d’orbites coadjointes ( k)k N qui correspond avec la V O ∈ suite πk k N. Comme vu en haut, toute orbite k a la même dimension 2d := 2di. De plus, ∈ O la suite ( k)k N converge proprement vers un ensemble d’orbites L(( k)k N). O ∈ O ∈ Maintenant, l’algèbre de Lie g sera examinée et décomposée en différentes parties. Ainsi, une nouvelle suite de représentations (πk)k N lesquelles seront équivalentes aux V ∈ représentations πk k N sera définie. Puis, dans les deuxième et troisième cas mentionnés ∈ dans la Proposition 6.10, on peut étudier (πk)k N et construire pour cette nouvelle suite des ∈ applications (νk)k N avec certaines propriétés similaires àcelles dont on a besoin dans la Condition 3(b). A∈ la fin, la convergence demandée peut être déduite de la convergence de V (πk)k N et de l’équivalence des représentations πk et πk . ∈

6.3.5 Condition 3(b) – Changement de la base de Jordan-Hölder ˜ ˜ ˜ ˜ ˜ Soit ℓ L(( k)k N). Alors il existe une suite ℓk N dans ( k)k N telle que ℓ = lim ℓk. ∈ O∈ O ∈ k O ∈ k ∈ →∞ ˜ ˜ ⊥ ˜ Comme on s’intéresse aux orbites k = ℓk + g ℓk , on peut remplacer la suite ℓk k N par O ∈ une suite (ℓk)k N avec ℓk(A) = 0 pour tout A g ℓ˜k ⊥ = g(ℓk)⊥. ∈ ∈ Ainsi, on obtient une autre suite convergente (ℓk)k N dans ( k)k N dont la limite ℓ appartient ∈ O ∈ àune orbite L(( k)k N). O∈ O ∈ Dans ce qui suit, cette suite (ℓk)k N sera étudiée et permettra d’effectuer la décomposition de g mentionnée en haut. ∈

On peut supposer que les sous-algèbres (g(ℓk))k N convergent vers une sous-algèbre u ∈ dont le groupe de Lie correspondant exp(u) est dénommé U. Ces sous-algèbres g(ℓk) pour k N peuvent être écrites comme ∈ g(ℓ ) = [g, g] s , k ⊕ k

où sk [g, g]⊥. En outre, soient nk,0 le noyau de ℓk [g,g] et sk,0 le noyau de ℓk s pour tout ⊂ | | k k N. On peut supposer que s = s et on choisit T s orthogonal à s et de longueur ∈ k,0 6 k k ∈ k k,0 1. Etant plus facile, le cas s = s pour k N sera omis. k,0 k ∈ De la même manière, on choisit Z [g, g] orthogonal à n et de longueur 1. De plus, on k ∈ k,0 note r = g(ℓ ) g. k k ⊥ ⊂ En prenant une sous-suite si nécessaire, on peut supposer que les limites lim Zk =: Z, k →∞ lim Tk =: T et lim rk =: r existent. k k →∞ →∞ Maintenant, des nouvelles polarisations pk de ℓk seront construites pour ensuite définir les représentations (πk)k N. ∈ La restriction à rk de la forme antisymétrique Bk définie par

B (V,W ) := ℓ , [V,W ] V,W g k h k i ∀ ∈

est non-dégénérée sur rk et il existe un endomorphisme inversible Sk de l’espace rk tel que

x,S (x′) = B (x,x′) x,x′ r . h k i k ∀ ∈ k

157 Alors S est antisymétrique, c.à.d. St = S , et on peut décomposer r en une somme directe k k − k k orthogonale d j rk = Vk Xj=1 k k de sous-espaces de deux dimensions Sk-invariants. On choisit une base de Hilbert Xj ,Yj j { } de Vk . Alors,

[Xk,Xk] n i,j 1,...,d , i j ∈ k,0 ∀ ∈ { } [Y k,Y k] n i,j 1,...,d et i j ∈ k,0 ∀ ∈ { } [Xk,Y k]= δ ckZ mod n i,j 1,...,d , i j i,j j k k,0 ∀ ∈ { } k R k où0 = cj et sup cj < pour tout j 1,...,d . 6 ∈ k N ∞ ∈ { } ∈ k En prenant encore une sous-suite si nécessaire, la suite (cj )k N converge pour tout j 1,...,d ∈ ∈ { } vers un cj. k k k k Puisque Xj ,Yj rk et ℓk(A) = 0 pour tout A rk, on a ℓk(Xj ) = ℓk(Yj ) = 0 pour tout ∈ ∈ k k j 1,...,d . En outre, on peut supposer que les suites (Xj )k N, (Yj )k N convergent dans g ∈ { } ∈ ∈ vers les vecteurs Xj,Yj qui forment une base modulo u dans g. Il résulte que

ℓ , [Xk,Y k] = ckλ , où h k j j i j k k λ = ℓ ,Z →∞ ℓ,Z =: λ. k h k ki −→ h i Comme Z a étéchoisi orthogonal à n , λ = 0 pour tout k N. k k,0 k 6 ∈ Maintenant, soient p := vect Y k, ,Y k, g(ℓ ) k { 1 ··· d k } et Pk := exp(pk). Alors, pk est une polarisation de ℓk. De plus, on définit la représentation πk par G πk := indPk χℓk . V Puisque πk et πk sont des représentations induites par des polarisations et des caractères χℓk et χℓ , où ℓk et ℓ se trouvent dans la même orbite coadjointe k, les deux représentations Ok Ok O sont équivalentes, comme on a vu dans la Section 6.2.2.

Soit ak := nk,0 + sk,0. Alors ak est un idéal de g sur lequel ℓk est 0. C’est pourquoi le sous-groupe normal exp(ak) est contenu dans le noyau de la représentation πk. En outre, soit a := lim ak. k →∞ De plus, soient p N etp ˜ 1,...,p . On note nk,0 la partie de ak incluse dans [g, g] et on ∈ ∈ { k } k se donne une base de Hilbert A1, ,Ap˜ de nk,0. Puis on note sk,0 la partie de ak qui { ··· } k k n’appartient pas à[g, g] et on se donne une base de Hilbert Ap˜+1, ,Ap de sk,0. Alors k k { ··· } A1, ,Ap est une base de Hilbert de ak et, comme avant, on peut supposer que la limite { ···k } lim Aj = Aj existe pour tout j 1,...,p . k ∈ { } →∞

158 Maintenant, pour tout k N, on peut prendre comme base de Hilbert de g aussi bien ∈ l’ensemble de vecteurs

Xk, ,Xk,Y k, ,Y k,T ,Z ,Ak, ..., Ak { 1 ··· d 1 ··· d k k 1 p} que l’ensemble X , ,X ,Y , ,Y ,T,Z,A , ..., A . { 1 ··· d 1 ··· d 1 p} Cela donne les crochets de Lie suivants:

k k k [Xi ,Yj ] = δi,jcj Zk mod ak, k k [Xi ,Xj ] =0mod ak et k k [Yi ,Yj ] =0mod ak.

Les vecteurs Zk et Tk sont centrals modulo ak.

6.3.6 Condition 3(b) – Premier cas

Premièrement, on considérera le cas où L(( k)k N) consiste en un seul point limite , le O ∈ O premier cas mentionnédans la Proposition 6.10. Dans ce cas, pour tout k N, ∈ 2d = dim( ) = dim( ). Ok O Par conséquent, la situation regardée apparaˆıt si et seulement si λ = 0 et c = 0 pour tout 6 j 6 j 1,...,d . ∈ { }

Le premier cas est le plus facile. Après quelques observations, les opérateurs (˜νk)k N peuvent immédiatement être définis. En outre, les assertions de la Condition 3(b) de∈ la Définition 6.1 peuvent facilement être démontrées. Dans ce premier cas, la transition aux représentations (πk)k N n’est pas nécessaire. ∈ On considère encore la suite (ℓk)k N qui a étéchoisie en haut et qui converge vers ℓ . ∈ ∈ O Comme les dimensions des orbites k et sont les mêmes, il existe une sous-suite de O O V (ℓk)k N (qui sera aussi dénommée (ℓk)k N pour simplifier) telle que p := lim pℓ soit une ∈ ∈ k k polarisation de ℓ, mais pas nécessairement la polarisation de Vergne. De→∞ plus, on définit V P := exp(p)= lim Pℓ et k k →∞ G π := indP χℓ. V G Si on identifie les espaces de Hilbert V et π des représentations π = ind V χℓ et π avec πℓ ℓk P k H k H ℓk L2(Rd), on peut déduire de [31], Théorème 2.3, que

V G G k π (a) π(a) = ind V χ (a) ind χ (a) →∞ 0 ℓk op P ℓk P ℓ − ℓk − op −→

pour tout a C∗( G). ∈ V G Comme π et π = ind V χℓ sont toutes les deux des représentations induites de polarisa- ℓ Pℓ tions et du même caractère χℓ, elles sont équivalentes. Par conséquent, il existe un opérateur d’entrelacement unitaire 2 Rd 2 Rd V F : πV = L ( ) π = L ( ) telque F πℓ (a)= π(a) F a C∗(G). H ℓ ∼ →H ∼ ◦ ◦ ∀ ∈

159 V V G V G De plus, les deux représentations π = π = ind V χ et π = ind V χ sont k ℓO P ℓOk ℓk P ℓk k ℓO ℓk N k équivalentes pour tout k parce que ℓOk et ℓk sont inclus dans la même orbite coadjointe V V ∈ k et pℓ et pℓ sont des polarisations. Alors il existe d’autres opérateurs d’entrelacement O Ok k unitaires

2 d 2 d V V V R V R Fk : π = L ( ) π = L ( ) tels que Fk πk (a)= πℓk (a) Fk a C∗(G). H k ∼ →H ℓk ∼ ◦ ◦ ∀ ∈ On définit les opérateurs souhaitésν ˜ pour tout k N par k ∈ V ν˜k(ϕ) := Fk∗ F ϕ πℓ F ∗ Fk ϕ CB(Si 1), ◦ ◦ ◦ ◦ ∀ ∈ − V V ce qui est raisonnable puisque πℓ est un point limite de la suite πk k N et donc contenu ∈ dans Si 1 d’après la définition de Si 1. − − Comme ϕ πV (L2(Rd)) et F et F sont des opérateurs d’entrelacement, ces opérateurs ℓ ∈ B k sont bornés et l’image deν ˜ est contenue dans (L2(Rd)), comme demandé. k B Il est facile de démontrer queν ˜k est bornéet involutif. La dernière chose àvérifier est la convergence dont on a besoin pour la Condition 3(b): Pour tout a C (G) ∈ ∗ V V V πk (a) ν˜k (a) Si = πk (a) Fk∗ F (a) Si πℓ F ∗ Fk − F | −1 op − ◦ ◦ F | −1 ◦ ◦ op V V = π (a) F ∗ F π (a) F∗ F k − k ◦ ◦ ℓ ◦ ◦ k op V = F ∗ π (a) F F ∗ π(a) F k ◦ ℓk ◦ k − k ◦ ◦ k op k πV (a) π(a) →∞ 0. ≤ ℓk − op −→

V N Ainsi, les représentations πk k N et les (˜νk)k construits satisfont la Condition 3(b) de la Définition 6.1. Alors l’assertion∈ est bien démontrée∈ pour le premier cas. 6.3.7 Condition 3(b) – Deuxième cas

Dans le deuxième cas mentionnédans la Proposition 6.10, la situation λ = 0 ou cj = 0 pour tout j 1,...,d sera considérée. ∈ { } Dans ce cas,

k ℓ , [Xk,Y k] = ckλ →∞ c λ = 0 j 1,...,d , h k j j i j k −→ j ∀ ∈ { } k tandis que cj λk = 0 pour tout k et tout j 1,...,d . 6 ˜ ∈ { } ˜ Alors ℓ [g,g] = 0, et comme ℓ [g,g] s’annule évidemment aussi pour tout ℓ u⊥, chaque | | ∈ L(( k)k N) s’annule sur [g, g]. Ceci veut dire que la représentation associée est un O ∈ O ∈ caractère. C’est pourquoi toute orbite limite dans l’ensemble L(( k)k N) est de dimension 0. O O ∈ Maintenant, on doit adapter les méthodes développées dans [26] à la situation donnée ici. R On choisit pour j 1,...,d les fonctions de Schwartz ηj ( ) telles que ηj L2(R) = 1 ∈ { } ∈ S k dk et η ∞ R 1. Soient s = (s ,...,s ), α = (α , ,α ), β = (β , ,β ) R . Alors on k jkL ( ) ≤ 1 d 1 ··· d 1 ··· d ∈ définit d 1 1 β 2πiαs k 4 k j ηk,α,β(s)= ηk,α,β(s1, ,sd) := e λkc ηj λkc 2 sj + . ··· j | j | λ ck j=1 k j Y

160 Cette d´eﬁnition est raisonnable parce que λ ck = 0 pour chaque k et chaque j 1,...,d . k j 6 ∈ { } Pour 0 =η ˜ L2(G/P ,χ ) = L2(Rd) soit 6 ∈ k ℓk ∼ P : L2(Rd) Cη,˜ ξ η˜ ξ, η˜ . η˜ → 7→ h i

Alors Pη˜ est la projection orthogonale sur l’espace Cη˜. Soit h C (G/U, χ ). On identifie G/U avec Rd Rd = R2d et l’utilisation de la base limite ∈ ∗ ℓ × ∼ introduite dans la Section 6.3.5 est exprimée par un index : ∞ d d h (x,y)= h (x1,...,xd,y1,...,yd) := xjXj + yjYj. ∞ ∞ Xj=1 Xj=1 2d Maintenant, hˆ peut être vu comme une fonction dans C (ℓ + u⊥) = C (R ). En utilisant ∞ ∞ ∼ ∞ cette identification, on peut définir l’opérateur linéaire d(˜x, y˜) νk(h) := hˆ (˜x, y˜) Pη . ∞ k,x,˜ y˜ d k RZ2d λkcj j=1

Q Alors on a la proposition suivante: Proposition 6.12.

1. Pour tout k N et tout h (G/U, χ ), l’intégrale qui définit ν (h) converge par rapport ∈ ∈S ℓ k àla norme d’opérateurs.

2. L’opérateur ν (h) est compact et ν (h) h ∗ . k k k kop ≤ k kC (G/U,χℓ) 3. ν est involutif, c.à.d. ν (h) = ν (h ) pour tout h C (G/U, χ ). k k ∗ k ∗ ∈ ∗ ℓ La preuve de cette proposition est plutôtlongue et technique. Avec cette proposition sur la suite de représentations (πk)k N, on peut maintenant déduire V ∈ l’assertion pour la suite initiale πk )k N. Comme pour tout k N les deux représentations∈ π et πV sont équivalentes, il existe des ∈ k k opérateurs d’entrelacement unitaires 2 Rd 2 Rd V Fk : πV = L ( ) πk = L ( ) tels que Fk πk (a)= πk(a) Fk a C∗(G). H k ∼ →H ∼ ◦ ◦ ∀ ∈

V V N En outre, puisque l’ensemble limite L πk k N de la suite πk )k est contenu dans Si 1, ∈ ∈ − en identifiant G avec l’ensemble des orbites coadjointes g∗/G, on peut restreindre un champ d’opérateurs ϕ CB(Si 1) à L(( k)k N)= ℓ+u⊥ et on obtient un élément dans CB(ℓ+u⊥). ∈ − O ∈ Alors, comme b (a) L(( ) N) a C∗(G) = C (L(( k)k N)) = C (ℓ+u⊥), on peut définir F | Ok k∈ | ∈ ∞ O ∈ ∞ le -isomorphisme ∗ R2d R2d τ : C ( ) = C (ℓ + u⊥) C∗(G/U, χℓ) = C∗( ), (a) L(( ) N) pG/U (a). ∞ ∼ ∞ → ∼ F | Ok k∈ 7→ Maintenant, pour chaque k N on définitν ˜ par ∈ k

ν˜k(ϕ) := Fk∗ (νk τ) ϕ L(( ) ) Fk ϕ CB(Si 1). ◦ ◦ | Ok˜ k˜∈N ◦ ∀ ∈ − 161 Puisque l’image de ν est dans (L2(Rd)) et F est un opérateur d’entrelacement et donc k B k borné, l’image deν ˜ est aussi contenue dans (L2(Rd)). k B De plus, on obtient que l’opérateurν ˜k est bornéet involutif en utilisant le fait que νk possède ces mêmes propriétés (voir Proposition 6.12). La convergence demandée dans la Condition 3(b) de la Définition 6.1 convient grâce àl’équivalence des représentations (πk)k N V ∈ et πk k N. ∈ V Il s’ensuit que les représentations πk k N satisfont la Propriété 3(b) et ainsi, les conditions de la Définition 6.1 sont démontrées.∈ 6.3.8 Condition 3(b) – Troisième cas Dans le troisième et dernier cas mentionnédans la Proposition 6.10, λ = 0 et il existe 6 1 m

6.3.9 R´esultat pour les groupes de Lie connexes r´eels nilpotents de pas deux

Avec les familles de sous-ensembles (Si)i 0,...,r et (Γi)i 0,...,r de G et les espaces de Hilbert ∈{ } ∈{ } ( i)i 0,...,r qui satisfont les conditions citées dans la Définition 6.1, on obtient le résultat H ∈{ } suivant: b Théorème 6.13. La C∗-algèbre C∗(G) d’un groupe de Lie G connexe réelnilpotent de pas deux est isomorphe (sous la transformation de Fourier) àl’ensemble de tous les champs d’opérateurs ϕ définis sur le spectre G du groupe respectif tels que:

1. ϕ(γ) ( i) pour tout i 1,...,r et tout γ Γi. ∈KbH ∈ { } ∈ 2. ϕ l (G). ∈ ∞ 3. Sur chacun des différents ensembles Γ , l’application γ ϕ(γ) est continue par rapport b i 7→ àla norme.

4. Pour toute suite (γk)k N G tendant vers l’infini, lim ϕ(γk) op = 0. ∈ ⊂ k k k →∞ 5. Pour i 1,...,r et touteb suite proprement convergente γ = (γk)k N Γi dont ∈ { } ∈ ⊂ l’ensemble limite est contenu dans Si 1 (en prenant une sous-suite si nécessaire) et pour − les applications ν˜k =ν ˜γ,k : CB(Si 1) ( i) construites dans les sections précédentes, − →B H on a

lim ϕ(γk) ν˜k ϕ Si = 0. k − | −1 op →∞

162 6.4 La C∗-alg`ebre du groupe de Lie SL(2, R)

Dans cette sous-section, la C∗-algèbre de SL(2, R) sera examinée. Au début, le groupe de Lie SL(2, R) et certains de ses sous-groupes seront introduits, et des définitions et des résultats importants, nécessaires pour démontrer les conditions énumérées dans la Définition 6.1 et alors pour la détermination de la C∗-algèbre de SL(2, R), seront évoqués. Les Sections 6.4.2 à 6.4.4 traitent du spectre de SL(2, R) et de sa topologie et dans la Section 6.4.5 les conditions spécifiées en haut seront discutées pour le groupe G = SL(2, R). Finalement, dans la Section 6.4.6, un résultat sur la C∗-algèbre de SL(2, R) sera présenté, et la structure concrète de C∗(G) sera donnée.

Pour le groupe SL(2, R), la Condition 1 de la Définition 6.1 est rendue claire par les définitions des ensembles Γ et S pour i 0,...,r , lesquelles seront données dans la Section 6.4.5. La i i ∈ { } Condition 2 est satisfaite aussi, comme SL(2, R) est un groupe de Lie connexe linéaire semisimple. La démonstration de la Condition 3(a) est assez courte et ne représente pas de difficulté particulière, tandis que l’essentiel du travail de la preuve des conditions de la Définition 6.1 consiste encore une fois en la vérification de la Condition 3(b). Néanmoins, sa preuve est moins longue et moins technique que dans le cas des groupes de Lie nilpotents de pas deux.

6.4.1 Pr´eliminaires Dans cette section, soit

G := SL(2, R)= A M(2, R) det A = 1 { ∈ | } et soit cos ϕ sin ϕ K := SO(2) = kϕ := − ϕ [0, 2π) sin ϕ cos ϕ ∈ son sous-groupe compact maximal. De plus, on définit les sous-groupes unidimensionnels nilpotent N et abélien A de G comme étant

t 1 x R e 0 R N := x et A := t t . 0 1 ∈ 0 e− ∈ 

Soit g = sl(2, R)= A M(2, R) tr A = 0 { ∈ | } l’alg`ebre de Lie de G.

Grâce à la décomposition de Iwasawa, on sait que G = KAN. Par conséquent, pour t 0 tout g G il existe κ(g) K, µ N et H(g) a, où a = t R est l’algèbre ∈ ∈ ∈ ∈ 0 t ∈ − de Lie de A, tels que g = κ(g)eH(g)µ.

En outre, on d´eﬁnit sur a les applications ρ et ν pour u C via u ∈ t 0 t 0 ρ := t et ν := ut t R. 0 t u 0 t ∀ ∈ − − 

163 Soient

L2(K) := f L2(K) f(k)= f( k) k K et + ∈ | − ∀ ∈ L2(K) := f L2(K) f(k)= f( k) k K . − ∈ | − − ∀ ∈ +,u 2 ,u Alors on définit pour tout u C les représentations sur +,u := L (K)+ et − sur 2 ∈ P HP P −,u := L (K) comme suit: HP − −1 ,u (νu+ρ)H(g k) 1 2 ± (g)f(k) := e− f κ g− k g G f L (K) k K. P ∀ ∈ ∀ ∈ ± ∀ ∈ Pour des raisons de simplicité, dans cette section, les représentations seront identifiées avec leurs classes d’équivalence.

Définition 6.14 (n-ième composante isotypique). Pour une représentation (˜π, ) de K on définit pour tout n Z la n-ième composante Hπ˜ ∈ isotypique de π˜ comme suit:

(n) := v π˜(k )v = einϕv ϕ [0, 2π) . Hπ˜ ∈Hπ˜| ϕ ∀ ∈ Une repr´esentation (˜π, ) de G est dite paire (respectivement impaire), si (n) = 0 Hπ˜ Hπ˜|K { } pour tout n impair (respectivement n pair).

Toute représentation irréductible unitaire de G est paire ou impaire. En outre, la somme directe algébrique

(n) Hπ˜ n Z M∈ est dense dans . Hπ˜ Remarque 6.15. 2 ,u D’aprèsla définition des espaces de Hilbert L (K) de ± pour u C, il est facile de ± P ∈ vérifier que +,u est pair pour tout u C et que ,u est impair pour tout u C. P ∈ P− ∈ Définition 6.16 (pn). 2 Pour n Z, on dénote bn(f) le n-ième coefficient de Fourier de f L (K) , défini par ∈ ∈ ±

1 inϕ b (f) := f(k )e− dk , n K ϕ ϕ | | KZ et on note in pn(f) := b n(f)e− ·. −

On peut facilement démontrer que pour tout u C et pour tout n Z l’opérateur pn est 2 ∈ ∈ la projection de ±,u = L (K) sur la n-ième composante isotypique de la représentation HP ± ,u. P±

164 6.4.2 Le spectre de SL(2, R) – Introduction de l’op´erateur Ku

En utilisant l’opérateur de Knapp-Stein, maintenant un opérateur Ku, dont on a besoin pour décrire le spectre de G sera introduit.

On d´eﬁnit

C∞(K) := f C∞(K) f(k)= f( k) k K et + ∈ | − ∀ ∈ C∞(K) := f C∞(K) f(k)= f( k) k K − ∈ | − − ∀ ∈ et soit J : C∞(K) C∞(K) pour u C avec Re u> 0 u + → + ∈ l’opérateur d’entrelacement de Knapp-Stein de la représentation +,u vers la représentation P +, u 0 1 2 − . En outre, soit w := k π = − . On étend f L (K) à G en utilisant la P 2 1 0 ∈ décomposition de Iwasawa G g = κ(g)eH(g)µ pour κ(g) K, H(g) a et µ N. En- H∋(g) (νu+ρ)H(g) ∈ ∈ ∈ suite, en définissant f˜u κ(g)e µ := e− f κ(g) , cet opérateur peut être écrit comme

J f(k)= f˜ (kµw)dµ f C∞(K) k K. (51) u u ∀ ∈ + ∀ ∈ NZ Cette intégrale converge pour Re u> 0 (voir [20], Chapitre VII ou [33], Chapitre 10.1). L’application f J f est continue, et la famille d’opérateurs J u C est holomorphe 7→ u { u| ∈ } en u pour Re u > 0 par rapport àdes topologies appropriées (voir [20], Chapitre VII.7 ou [33], Chapitre 10.1). Pour u R l’opérateur J est autoadjoint par rapport au produit scalaire de L2(K) ∈ >0 u habituel. De plus, on peut étendre la fonction u J méromorphiquement à C (voir [33], 7→ u Chapitre 10.1). Puis, pour tout u C pour lequel l’opérateur J est régulier, J est un ∈ u u opérateur d’entrelacement de +,u vers +, u. P P −

Grâce àla propriétéd’entrelacement de Ju, on obtient la remarque suivante: Remarque 6.17. L’opérateur J commute avec les projections p pour tout n Z et pour tout u C pour u n ∈ ∈ lequel Ju est régulier.

Maintenant, on peut déduire que les opérateurs Ju ont la propriété Z Ju +,u (n) = cn(u) id +,u (n) pour tout n pair, |HP · |HP ∈ comme une équation de fonctions méromorphes, où c : C C est une fonction méromorphe n → pour tout n Z pair. Ceci est une conséquence de la remarque ci-dessus considérée avec le ∈ fait que +,u (n) est unidimensionnel. HP En utilisant des formules d’intégrales standard (voir [20], Chapitre V.6), on obtient avec (51) que pour u = 1

J (f)= c f(k)dk f C∞(K) 1 ∀ ∈ + KZ

165 in pour une constante c> 0. De plus, pour tout n Z pair on a +,u (n)= C e− ·. Ainsi, on ∈ HP · obtient

= 0 pour n = 0 cn(1) 6 (= 0 pour n Z 0 pair. ∈ \{ } On peut d´emontrer que

c (u) = 0 pour u (0, 1) 0 6 ∈ et par conséquent, on peut définir 1 J˜u := Ju c0(u) pour u C comme une fonction méromorphe. De surcroˆıt, on peut prouver que ∈

J˜u est r´egulier pour u =0 et J˜0 = id.

Maintenant, on définit un produit scalaire sur C∞(K)+ de la manière suivante:

f ,f := J˜ f ,f . h 1 2iu u 1 2 L2(K) Ce produit scalaire est invariant et défini positif pour u (0, 1). ∈ Le complétéde C (K) par rapport àce produit scalaire , donne un espace de Hilbert ∞ + h· ·iu . En considérant la restriction de la représentation +,u à C (K) et en la prolongeant Hu P ∞ + de fa¸con continue sur l’espace , on obtient une représentation unitaire que l’on nomme Hu +,u aussi. G agit sur via cette représentation unitaire +,u. P Hu P

cn(u) On pose dn(u) := > 0 pour u (0, 1). On définit une bijection unitaire c0(u) ∈ q K : L2(K) u (0, 1) u Hu → + ∀ ∈ comme suit: sur la n-ième composante isotypique de , on définit K par Hu u Z Ku +,u (n) := dn(u) id +,u (n) pour tout n pair. |HP · |HP ∈ Puis, on peut étendre cette définition àdes sommes finies de composantes isotypiques. Cet opérateur est aussi autoadjoint par rapport au produit scalaire habituel de L2(K) et pour des sommes finies de composantes isotypiques f1 et f2

K f ,K f = K2f ,f = J˜ f ,f = f ,f . u 1 u 2 L2(K) u 1 2 L2(K) u 1 2 L2(K) h 1 2iu

Par conséquent, l’opérateur est unitaire (si on regarde l’espace muni de , et L2(K) Hu h· ·iu + muni de , L2(K)) et ainsi, en raison de la densitéde +,u (n) dans u, on peut h· ·i n Z HP H ∈ prolonger Ku de fa¸con continue sur tout l’espace u. L H En outre, Ku est continu en u et on a aussi la propriétélim Ku = id. u 0 → D’après sa définition, l’opérateur Ku commute également avec les projections pn pour tout n Z. ∈

166 Ensuite, un autre produit scalaire sur C (K) := f C (K) p (f) = 0 n 0 est ∞ ++ ∈ ∞ +| n ∀ ≤ nécessaire. Pour cela, on définit ˜ 1 J(u) := Ju c2(u) pour u C comme une famille méromorphe d’opérateurs. ∈ De surcroˆıt, on définit l’espace C∞(K)+ := f C∞(K)+ pn(f) = 0 n 0 . − ∈ | ∀ ≥ Lemme 6.18.

˜ ˜ ˜ ˜ (a) J( u) J(u) = J(u) J( u) = id en tant qu’opérateurs méromorphes. − ◦ ◦ − (b) J˜ est régulier pour u = 1 et (u) − ˜ ˜ ker J( 1) C∞(K)++ = ker J( 1) C∞(K)+ = 0 . − ∩ − ∩ − { } ˜ +, 1 +,1 De plus, J( 1) est un opérateur d’entrelacement de − vers . − P P

˜ ∞ ∞ (c) J(u) C (K)++ C (K)+− est régulier pour u = 1. | ⊕ ˜ L’opérateur J(1) n’est pas identiquement zéro sur l’espace C∞(K)++, étant donnéque sa restriction à +,1 (2) est l’opérateur identité. Alors, on peut définir pour toutes fonctions HP f ,f C (K) 1 2 ∈ ∞ ++ f ,f := J˜ f ,f . h 1 2i(1) (1) 1 2 L2(K) Ceci est un produit scalaire invariant et défini positif. Maintenant, le complétéde l’espace C∞(K)++ par rapport àce produit scalaire donne un espace de Hilbert que l’on nommera . H(1)

La même procédure peut être accomplie pour C∞(K)+ . − Dans ce cas, on définit pour u (0, 1) l’opérateur J˜ de la manière suivante: ∈ [u] ˜ 1 J[u] := Ju. c 2(u) − ˜ Comme précédemment, on peut démontrer que J[u] est régulier pour u = 1 sur C∞(K)+ et − n’est pas identiquement nul sur cet espace. Par conséquent, on définit de nouveau pour tous f1,f2 C∞(K)+ ∈ − f ,f := J˜ f ,f . h 1 2i[1] [1] 1 2 L2(K)

Ceci est un autre produit scalaire invariant et défini positif. Le complétéde l’espace C∞(K)+ − par rapport àce produit scalaire donne un espace de Hilbert noté . H[1]

167 6.4.3 Le spectre de SL(2, R) – Description des représentations irréductibles unitaires Maintenant, des réalisations appropriées pour le spectre de SL(2, R) seront fournies.

Le spectre G de G = SL(2, R) se compose des repr´esentations suivantes:

1. Les représentationsb de la série principale: (a) +,iv pour v [0, ). P ∈ ∞ (b) ,iv pour v (0, ). P− ∈ ∞ Voir Section 6.4.1 en haut pour les définitions.

2. Les représentations de la série complémentaire u pour u (0, 1): C ∈ L’espace de Hilbert u est défini comme HC 2 u := L (K)+ HC et l’action est donnée par

u +,u 1 (g) := K (g) K− C u ◦P ◦ u pour tout g G, oùici encore on considère la restriction de +,u(g) à C (K) et ∈ P ∞ + puis on le prolonge de fa¸con continue sur l’espace (voir la définition de dans la Hu Hu Section 6.4.2).

3. Les représentations de la série discrète:

(a) + pour m N impair: Dm ∈ ∗ (i) +: D1 L’espace de Hilbert + est donn´epar HD1

+ := (1) HD1 H défini dans la Section 6.4.2. L’action est donnée par

+ := +,1. D1 P Ici encore, comme décrit précédemment et comme dans toutes les définitions dans cette sous-section, on restreint la représentation +,u pour les différentes P valeurs u C au sous-espace correspondant de L2(K) et puis on la prolonge ∈ de fa¸con continue sur l’espace de Hilbert correspondant. + N (ii) m pour m 3 impair: D ∈ ≥ + N Comme espace de Hilbert + pour m et m 3 impair, on peut prendre H m D ∈ ≥ le complétéde l’espace D

f C∞(K) p (f) = 0 n m 1 ∈ +| n ∀ ≤ − par rapport àun produit scalaire approprié, et comme action on peut con- sidérer + := +,m. Dm P

168 Avec cette réalisation, les espaces de Hilbert + pour m N 3 impair H m ∈ ≥ dépendent de m. Mais puisqu’ils sont tous de dimensionD infinie et séparable, on peut les identifier si on conjugue l’action de G correspondante. Alors, on fixe un espace de Hilbert séparable de dimension infinie. En outre, l’action HD de G n’est pas nécessaire pour la détermination de C∗(G). (b) pour m N impair: Dm− ∈ ∗ (i) −: D1 L’espace de Hilbert − est donnépar HD1 − := [1] HD1 H défini précédemment dans la Section 6.4.2 et l’action est donnée par +,1 − := . D1 P N (ii) m− pour m 3 impair: D ∈ ≥ + De la même manière que pour , comme espace de Hilbert − pour m m m− D HD D et m N 3 impair, on peut prendre le complétéde l’espace ∈ ≥ f C∞(K) p (f) = 0 n m + 1 ∈ +| n ∀ ≥− par rapport àun produit scalaire approprié, et comme action on peut con- sidérer +,m − := . Dm P Encore une fois, les espaces de Hilbert dépendent de m. On les identifie et on prend l’espace de Hilbert commun séparable de dimension infinie HD fixédans (a)(ii). Une fois de plus, l’action de G n’est pas nécessaire pour la détermination de C∗(G). + N (c) m pour m ∗ pair: D ∈ + Comme espace de Hilbert + pour et m N∗ pair, on peut prendre le H m Dm ∈ complétéde l’espace D

f C∞(K) pn(f) = 0 n m 1 ∈ −| ∀ ≤ − par rapport àun produit scalaire approprié, et comme action on peut considérer + ,m := − . Dm P Encore une fois, les espaces de Hilbert sont identifiés et on prend l’espace de Hilbert commun séparable de dimension infinie, comme dans (a)(ii). HD (d) pour m N pair: Dm− ∈ ∗ Comme espace de Hilbert − pour − et m N∗ pair, on peut prendre le H m Dm ∈ complétéde l’espace D

f C∞(K) pn(f) = 0 n m + 1 ∈ −| ∀ ≥− par rapport àun produit scalaire approprié, et comme action on peut considérer ,m − := − . Dm P Une fois de plus, les espaces de Hilbert sont identifiés et on prend l’espace de Hilbert commun, comme dans (a)(ii). HD

169 4. Les limites de la série discrète: (a) : D+ L’espace de Hilbert + est défini comme étant HD 2 + := f L (K) pn(f) = 0 n 0 HD ∈ −| ∀ ≤ et l’action est donnée par ,0 := − . D+ P (b) : D− L’espace de Hilbert − est défini comme étant HD 2 − := f L (K) pn(f) = 0 n 0 HD ∈ −| ∀ ≥ et l’action est donnée par ,0 := − . D− P 5. La représentation trivial 1: F C Son espace de Hilbert 1 = sera identifiéavec l’espace de fonctions constantes HF f L2(K) p (f) = 0 n = 0 . ∈ +| n ∀ 6 Ici, l’action est donnée par (g) := id F1 pour tout g G. On a aussi ∈ +, 1 = − , F1 P

puisque ν 1 + ρ = 0 et tout f 1 est une fonction constante. − ∈HF Remarque 6.19. Pour tout n Z et pour toute représentation π irréductible unitaire de G, l’opérateur p est ∈ n la projection de sur la n-ième composante isotypique (n). Hπ Hπ \ 6.4.4 Le spectre de SL(2, R) – La topologie sur SL(2, R) En calculant les valeurs que les représentations ,u prennent appliquées àl’opérateur de P± Casimir, il est maintenant possible de décrire la topologie sur G. Proposition 6.20. b La topologie sur G peut être caractériséede la manière suivante:

1. Pour toute suite (vj)j N et tout v dans [0, ), b ∈ ∞ j j +,ivj →∞ +,iv v →∞ v. P −→ P ⇐⇒ j −→

2. Pour toute suite (vj)j N et tout v dans (0, ), ∈ ∞

,ivj j ,iv j − →∞ − v →∞ v. P −→ P ⇐⇒ j −→ Pour toute suite (vj)j N dans [0, ), ∈ ∞

,ivj j j − →∞ +, vj →∞ 0. P −→ D D− ⇐⇒ −→ 170 3. Pour toute suite (uj)j N et tout u dans (0, 1), ∈ j j uj →∞ u u →∞ u. C −→ C ⇐⇒ j −→

Pour toute suite (uj)j N dans (0, 1), ∈ j j uj →∞ +,0 u →∞ 0. C −→ P ⇐⇒ j −→

Pour toute suite (uj)j N dans (0, 1), ∈

uj j + j →∞ , −, u →∞ 1. C −→ D1 D1 F1 ⇐⇒ j −→ Toutes les autres suites convergentes doivent d’une mani`ere ou d’une autre se composer des suites regard´eesen haut.

6.4.5 Les conditions limites duales sous contrôle normique Dans cette sous-section, le spectre sera diviséen différents sous-ensembles. Après, il doit être vérifiéque ceux-ci satisfont les exigences des conditions “limites duales sous contrôle normique”. On définit

Γ := , 0 F1 Γ := + , 1 D1 Γ := − , 2 D1 Γ := , 3 D+ Γ4 := , D− Γ := ± m N , 5 Dm | ∈ >1 Γ := +,iv v [0, ) , 6 P | ∈ ∞ ,iv Γ := − v (0, ) et 7 P | ∈ ∞ Γ := u u (0, 1) . 8 C | ∈ Clairement, tous les ensembles Γ pouri 0,..., 8 sont Hausdorﬀ. De plus, les ensembles i ∈ { }

Si := Γj j 0,...,i ∈{[ } sont fermés et S0 = Γ0 constitue l’ensemble de tous les caractères de G = SL(2, R). En outre, comme défini dans la Section 6.4.3, pour tout i 0,..., 8 , il existe un espace de Hilbert ∈ { } Hi commun sur lequel toutes les représentations dans Γi agissent. C’est pourquoi la Condition 1 de la Définition 6.1 est satisfaite. Comme tout groupe de Lie réel connexe linéaire semi-simple satisfait la Condition CCR (voir [9], Théorème 15.5.6), la Condition 2 de la Définition 6.1 est remplie aussi. Il reste donc la Condition 3. La Condition 3(a) est claire pour les ensembles Γ pour i 0,..., 5 , comme ceux-ci sont des i ∈ { } ensembles discrets. En revanche, des calculs doivent être effectués pour Γ6,Γ7 et Γ8.

171 La Condition 3(b) est la partie la plus compliquée de la preuve des différentes conditions énumérées dans la Définition 6.1. La situation dans laquelle une suite (γj)j N dans Γi converge ∈ vers un ensemble limite contenu dans Si 1 = Γl qui est regardée dans la Condition 3(b) − l

N ,ivj (i) (γj)j = − j N est une suite dans Γ7 avec l’ensemble limite Γ3 Γ4 = +, . ∈ P ∈ ∪ D D− N uj +,0 (ii) (γj)j = j N est une suite dans Γ8 avec l’ensemble limite Γ7. ∈ C ∈ P ⊂ uj (iii) (γj)j N = N est une suite contenue dans Γ8 qui converge vers l’ensemble limite ∈ j C ∈ + Γ Γ Γ = , , − . 0 ∪ 1 ∪ 2 F1 D1 D1 Pour i 0,..., 6 , les ensembles Γ sont fermés et, par conséquent, la situation regardée ne ∈ { } i peut pas apparaˆıtre pour des suites (γj)j N dans Γi pour i 0,..., 6 . ∈ ∈ { } 6.4.6 Résultat pour le groupe de Lie SL(2, R) Quand on a vérifiétoutes les conditions énumérées dans la Section 6.3.4, le Théorème 6.2 est démontrépour le groupe de Lie SL(2, R). C’est pourquoi, avec les ensembles Γi et Si et les espaces de Hilbert pour i 0,..., 8 définis dans les Sections 6.4.5 et 6.4.3 et les Hi ∈ { } applicationsν ˜j qui sont nécessaires pour la Condition 3(b) de la Définition 6.1, la C∗-algèbre de G = SL(2, R) peut être caractérisée par le théorème suivant:

Théorème 6.21. La C∗-algèbre C∗(G) de G = SL(2, R) est isomorphe (sous la transformation de Fourier) à l’ensemble de tous les champs d’opérateurs ϕ définis sur G tels que:

1. ϕ(γ) ( ) pour tout i 1,..., 8 et tout γ Γ . ∈K Hi ∈ { } ∈ i b 2. ϕ l (G). ∈ ∞ 3. Les applications γ ϕ(γ) sont continues par rapport àla norme sur les différents b 7→ ensembles Γi.

4. Pour toute suite (γj)j N G tendant vers l’inﬁni, lim ϕ(γj) op = 0. ∈ ⊂ j k k →∞

5. Pour tout i 1,..., 8 et touteb suite γ = (γj)j N Γi proprement convergente dont ∈ { } ∈ ⊂ l’ensemble limite se trouve dans Si 1 (en prenant une sous-suite si nécessaire) et pour − les applications ν˜j =ν ˜γ,j : CB(Si 1) ( i) nécessaires pour la Condition 3(b) de la − →B H Définition 6.1, on a

lim ϕ(γj) ν˜ ϕ Si = 0. j − | −1 op →∞

Pour un espace topologique de Hausdorff V et une C∗-algèbre B, on note C (V,B) la ∞ C∗-algèbre de toutes les fonctions continues définies sur V àvaleurs dans B qui s’annulent àl’infini. Alors, de ce théorème on peut déduire plus concrètement le résultat suivant pour G = SL(2, R):

172 Théorème 6.22. 2 2 Soient p+ l’opérateur de projection de L (K) sur f L (K) pn(f) = 0 n 0 , p la 2 2 ± ∈ ±| ∀ ≤ 2 − projection de L (K) sur f L (K) pn(f) = 0 n 0 et p0 la projection de L (K)+ ± ∈ ±| ∀ ≥ sur f L2(K) p (f) = 0 n = 0 = C. ∈ +| n ∀ 6 Alors la C -algèbre C (G) de G = SL(2, R) est isomorphe àla somme directe de C -algèbres ∗ ∗ ∗

2 F C i[0, ) [0, 1], L (K)+ F (1) commute avec p+, p et p0 ∈ ∞ ∞ ∪ K − 2 F C i[0, ), L (K) F (0) commute avec p+ et p ⊕ ∈ ∞ ∞ K − − C Z 1, 0, 1 , ⊕ ∞ \ {− } K HD pour l’espace de Hilbert séparable de dimension infinie fixédans la Section 6.4.3. HD

6.5 Sur la topologie duale des groupes U(n) ⋉Hn Dans cette section, le produit semi-direct G = U(n) ⋉H pour n N sera analys´eet la n n ∈ ∗ topologie de son spectre sera d´ecrit.

Après avoir donné quelques préliminaires, le spectre et l’espace des orbites coadjointes admissibles de Gn seront décrits. Puis, la topologie du spectre de Gn sera comparée avec celle de l’espace des orbites coadjointes admissibles et, pour finir, des résultats seront déduits.

Comme déjàmentionnédans l’introduction, cette section est basée sur une prépublication par M.Elloumi et J.Ludwig qui peut être trouvée dans la thèse de doctorat de M.Elloumi (voir [11], Chapitre 3). Dans la présente thèse, elle a étéélaborée et complétée et plusieurs changements importants ont étééffectués. Pour plus de détails voir Section 5.

6.5.1 Préliminaires n Soit C l’espace vectoriel complexe n-dimensionnel muni du produit scalaire standard .,. Cn h i et soient respectivement (.,.)Cn et ω(.,.)Cn les parties réelle et imaginaire de .,. Cn . Le groupe h i de Heisenberg associé H = Cn R de dimension 2n +1 sur R est donnépar la multiplication n × 1 (z,t)(z′,t′) := z + z′,t + t′ ω(z, z′)Cn (z,t), (z′,t′) H . − 2 ∀ ∈ n En outre, on considère le groupe unitaire U(n) d’automorphismes de Hn qui préserve .,. Cn n h i sur C et qui s’immerge injectivement dans Aut(Hn) via

A.(z,t):=(Az, t) A U(n) (z,t) H . ∀ ∈ ∀ ∈ n

Par cette immersion, U(n) est un sous-groupe compact connexe maximal de Aut(Hn) (voir [14], Th´eor`eme 1.22 et [20], Chapitre I.1). On note Gn = U(n) ⋉Hn le produit semi-direct de U(n) avec le groupe de Heisenberg Hn muni de la multiplication 1 (A, z, t)(B, z′,t′) := AB, z + Az′,t + t′ ω(z, Az′)Cn (A, z, t), (B, z′,t′) G . − 2 ∀ ∈ n

173 L’algèbre de Lie hn de Hn sera identifiée avec Hn lui-même via l’application exponentielle. Le crochet de Lie de hn est défini par

(z,t), (w,s) := 0, ω(z,w)Cn (z,t), (w,s) h − ∀ ∈ n et l’action dérivée de l’algèbre de Lie u(n) de U(n) sur hn est

A.(z,t):=(Az, 0) A u(n) (z,t) h . ∀ ∈ ∀ ∈ n

Notant gn = u(n) ⋉ hn l’alg`ebre de Lie de Gn, on obtient le crochet de Lie

(A, z, t), (B,w,s) = AB BA, Aw Bz, ω(z,w)Cn − − − pour tous (A, z, t), (B,w,s) g . ∈ n

L’algèbre de Lie u(n) sera identifiée avec son espace vectoriel dual u∗(n) àl’aide du produit scalaire U(n)-invariant

A, B := tr(AB) A, B u(n) h iu(n) ∀ ∈ et le dual g = u(n) ⋉ h ∗ sera identifiéavec u(n) h àl’aide du produit scalaire n∗ n ⊕ n (U,u,x), (B,w,s) = U, B u(n) + ω(u,w)Cn + xs (U, u, x), (B,w,s) gn. gn h i ∀ ∈ n n Puis, en définissant : C C u (n) par z w(B) := ω(w, Bz)Cn pour tout B u(n), × × −→ ∗ × ∈ on peut calculer que les orbites coadjointes de Gn ont la forme

x n = AUA∗ + z (Au)+ z z, Au + xz,x A U(n), z C O(U,u,x) × 2 × ∈ ∈ 

pour chaque (U, u, x) g = u(n) Cn R, o`u A est la matrice adjointe de A. ∈ n∗ × × ∗

6.5.2 Le spectre et les orbites coadjointes admissibles de Gn

La description du spectre de Gn est basée sur une méthode de Mackey (voir [28], Chapitre 10) qui dit qu’on doit déterminer les représentations irréductibles unitaires du sous-groupe Hn afin d’en construire des représentations de Gn. Soit Pn l’ensemble des poids entiers maximaux λ pour U(n) qui peuvent être écrits de la n fa¸con iλjej ou simplement comme λ = (λ1, ,λn), où λj sont des entiers pour tout j=1 ··· j 1,...,nP tels que λ λ λ . Alors P = Zn. ∈ { } 1 ≥ 2 ≥···≥ n n ∼

Le spectre Gn se compose des familles de repr´esentations suivantes (i) π pour λ P et α R = R 0 , (λ,α) b ∈ n ∈ ∗ \{ } R (ii) π(µ,r) pour µ Pn 1 et r >0, et ∈ − ∈ (iii) τ pour λ P . λ ∈ n

174 Alors Gn est en bijection avec l’ensemble R R c (Pn ∗) Pn 1 >0 Pn. × ∪ − × ∪ Une fonctionnelle lin´eaire ℓ dans gn∗ est dite admissible s’il existe un caract`ere unitaire χ de la

composante connexe de l’identitéde Gn[ℓ], le stabilisateur de ℓ dans Gn, tel que dχ = iℓ gn[ℓ]. | Maintenant, on peut associer des fonctionnelles linéaires ℓλ,α, ℓµ,r et ℓλ aux représentations π(λ,α), π(µ,r) et τλ et trouver qu’elles sont tous admissibles. Alors d’après [25], les représentations π(λ,α), π(µ,r) et τλ sont équivalentes aux représentations de Gn obtenues par induction holomorphe àpartir des fonctionnelles linéaires associées respectives. On note , et les orbites coadjointes associées respectivement aux formes O(λ,α) O(µ,r) Oλ linéaires ℓ , ℓ et ℓ . En outre, soient gn‡ g l’union de tous les éléments dans , λ,α µ,r λ ⊂ n∗ O(λ,α) et et on note gn‡ /G l’ensemble correspondant dans l’espace des orbites. On peut O(µ,r) Oλ n déduire de [25] que gn‡ est l’ensemble des fonctionnelles linéaires admissibles de gn. Alors l’espace quotient gn‡ /Gn des orbites coadjointes admissibles est en bijection avec le spectre Gn.

T n On notecvr la forme linéaire identifiée avec (0,..., 0,r) dans C et le sous-espace n W de u(n) générépar les matrices z v pour z C . L’espace gn‡ /G peut être écrit comme × r ∈ n l’ensemble des orbites

iα n = AJ A∗ + zz∗,αz,α z C ,A U(n) , O(λ,α) λ 2 ∈ ∈ = A(J + )A∗, Av , 0 A U(n) et O(µ,r) µ W r ∈ = n(AJ A∗, 0, 0) A U(n) o Oλ λ ∈ R pour α ∗, r> 0, µ Pn 1 et λ Pn et où Jλ et Jµ sont des matrices dans U(n) associées ∈ ∈ − ∈ à λ et à µ respectivement.

6.5.3 Convergence dans l’espace quotient gn‡ /Gn et la topologie du spectre de Gn Dans le théorème suivant, les topologies de l’espace des orbites coadjointes admissibles et du spectre de Gn sont caractérisées. Théorème 6.23. Soient α R∗, r> 0, µ Pn 1 et λ Pn. ∈ ∈ − ∈

k ‡ 1. Une suite d’orbites coadjointes λ k N converge vers l’orbite λ dans gn/Gn si et k O ∈ O seulement si λ = λ pour k suffisamment grand. Ceci est le cas si et seulement si la suite (τλk )k N de représentations irréductibles unitaires converge vers la représentation ∈ τλ dans Gn.

2. Une suite d’orbites coadjointes k converge vers dans g‡ /G si et c (µ ,rk) k N (µ,r) n n k O ∈ O seulement si lim rk = r et µ = µ pour k suffisamment grand. Ceci est le cas si et k →∞ seulement si la suite π k de représentations irréductibles unitaires converge (µ ,rk) k N ∈ vers la représentation π(µ,r) dans Gn.

c 175 3. Une suite d’orbites coadjointes k converge vers dans g‡ /G si et seule- (µ ,rk) k N λ n n O k∈ k O k ment si (rk)k N converge vers 0 et λ1 µ1 λ2 µ2 ... λn 1 µn 1 λn pour k ∈ ≥ ≥ ≥ ≥ ≥ − ≥ − ≥ assez grand. Ceci est le cas si et seulement si la suite π k de représentations (µ ,rk) k N ∈ irréductibles unitaires converge vers la représentation τλ dans Gn.

4. Une suite d’orbites coadjointes k converge vers dans g‡ /G si et (λ ,αk) k N c(λ,α) n n k O ∈ O seulement si lim αk = α et λ = λ pour k suffisamment grand. Ceci est le cas si et k →∞ seulement si la suite π k de représentations irréductibles unitaires converge (λ ,αk) k N ∈ vers la représentation π(λ,α) dans Gn.

5. Une suite d’orbites coadjointes k converge vers l’orbite dans g‡ /G c(λ ,αk) k N (µ,r) n n O ∈ O si et seulement si lim αk = 0 et la suite (λk,α ) N satisfait une des conditions k O k k →∞ ∈ suivantes: (i) Pour k suﬃsamment grand, α > 0, λk = µ pour chaque j 1,...,n 1 et k j j ∈ { − } k r2 lim αkλn = . k − 2 →∞ k (ii) Pour k suﬃsamment grand, αk < 0, λj = µj 1 pour chaque j 2,...,n et − ∈ { } k r2 lim αkλ1 = . k − 2 →∞

Ceci est le cas si et seulement si la suite π k de représentations irréductibles (λ ,αk) k N ∈ unitaires converge vers la représentation π(µ,r) dans Gn.

6. Une suite d’orbites coadjointes k converge vers l’orbite dans g‡ /G si et (λ ,αk) k N c λ n n O ∈ O seulement si lim αk = 0 et la suite (λk,α ) N satisfait une des conditions suivantes: k O k k →∞ ∈ k k k (i) Pour k assez grand, αk > 0, λ1 λ1 ... λn 1 λn 1 λn λn et k ≥ ≥ ≥ − ≥ − ≥ ≥ lim αkλn = 0. k →∞ k k k (ii) Pour k assez grand, αk < 0, λ1 λ1 λ2 λ2 ... λn 1 λn λn et k ≥ ≥ ≥ ≥ ≥ − ≥ ≥ lim αkλ1 = 0. k →∞

Ceci est le cas si la suite π k de représentations irréductibles unitaires converge (λ ,αk) k N ∈ vers la représentation τλdans Gn.

Pour la description entière de la topologiec de Gn, il manque encore la seconde implication de 6), c.à.d. que la convergence de la suite de représentations s’ensuit des conditions indiquées dans 6). On suppose que cette implication estc correcte aussi mais la preuve est encore en construction. En revanche, pour n = 1, cette implication a déjàpu être démontrée.

176 6.5.4 Résultats On peut déduire les résultats suivants:

Th´eor`eme 6.24. L’application

G g‡ /G , π n −→ n n 7→ Oπ est continue. b

Si on réussit àdécrire entièrement la topologie de Gn de la fa¸con présentée précédemment, on obtient le résultat ci-dessous: c Conjecture 6.25. Pour n N quelconque, le spectre du groupe G = U(n) ⋉H est homéomorphe àl’espace ∈ ∗ n n gn‡ /Gn des orbites coadjointes admissibles associé. Pour n = 1 la situation est beaucoup plus facile que dans le cas général et donc, comme mentionnéprécédemment, cette conjecture a pu être prouvée pour ce cas-là.

Théorème 6.26. Le spectre du produit semi-direct U(1)⋉H1 est homéomorphe àl’espace des orbites coadjointes admissibles associé.

177 7 Appendix

Lemma 7.1. Using the notation from Section 3.2, let d 0,..., [ n ] and (J, K) with J = K = d. ∈ { 2 } ∈M | | | | Then, the set

S = π (J ′,K′) (J, K): O (g∗/G) ′ ′ iJK ℓO ∃ ≤ ∈ (J ,K ) n o is closed. Proof: k k N Let Ok (g∗/G)(Jk,Kk) for (J ,K ) (J, K) for every k , let O g∗/G and let k ∈ ≤ ∈ ∈ πℓ →∞ πℓ . It needs to be shown that O (g∗/G)(J′,K′) for (J ′,K′) (J, K). Ok −→ O ∈ ≤ Assume that this is not the case, i.e. that O (g /G) ′ ′ for (J ,K ) > (J, K). ∈ ∗ (J ,K ) ′ ′ Since J = K > J = K is obviously not possible because of the convergence of the | ′| | ′| | | | | sequence πℓO to πℓO , one has J ′ = K′ = J = K = d. k k N | | | | | | | | So, let J = j ,...,j ∈ , K = k ,...,k , J = j ,...,j and K = k ,...,k . Then, because { 1 d} { 1 d} ′ { 1′ d′ } ′ { 1′ d′ } (J ′,K′) > (J, K), there are two cases to regard: 1. There is an index m 1,...,d such that j = j and k = k for all i < m and j

1,ℓO g = U g ℓO, U, Hj′ = 0 and ∈ 1 n o 1,ℓOk g = U g ℓOk , U, Hjk = 0 ∈ 1 n o N = U g ℓO , U, Hj′ = 0 k . ∈ k 1 ∀ ∈ n o

178 k 1,ℓO k 1,ℓ Because ℓ →∞ ℓ , one gets g k →∞ g O . Furthermore, by their construction, the basis Ok −→ O −→ elements H1,ℓ depend continuously on ℓ for all i 1,...,n . Hence, i ∈ { }

1,ℓO 1,ℓ k 1,ℓO 1,ℓ k Ok →∞ O ℓOk , Hj′ , g ℓO, Hj′ , g , 2 −→ 2 h i h i which is a contradiction to (52). The second case is similar to the ﬁrst case and will thus be skipped. Therefore, (J ′,K′) > (J, K) is not possible and the closure is shown.

Lemma 7.2. Let V be a ﬁnite-dimensional euclidean vector space and S an invertible skew-symmetric endomorphism. Then, V can be decomposed into an orthogonal direct sum of two-dimensional S-invariant subspaces.

Proof: S extends to a complex endomorphism SC on the complexiﬁcation VC of V , which has purely imaginary eigenvalues. If iλ iR is an eigenvalue, then also iλ is a spectral element. Denote by E the correspon- ∈ − iλ ding eigenspace. These eigenspaces are orthogonal to each other with respect to the Hilbert space structure of VC coming from the euclidean scalar product , on V . h· ·i Let for iλ in the spectrum of SC

λ V := (Eiλ + E iλ) V. − ∩ λ λ λ′ If λ = 0, dim V is even and V is S-invariant and orthogonal to V , whenever λ = λ′ : 6 ′ | | 6 | | Indeed, one then has for x V λ,x V λ , ∈ ′ ∈

x + iy Eiλ and x iy E iλ for some y V, as well as ∈ − ∈ − ∈ x′ + iy′ Eiλ′ and x′ iy′ E iλ′ for some y′ V. ∈ − ∈ − ∈ Therefore, x + iy,x′ + iy′ = 0 and x iy,x′ + iy′ = 0. h i h − i Thus, one has x,x′ + iy′ =0 and hence, x,x′ = 0. h i h i Suppose that dim V λ > 2, choose a vector x V λ of length 1 and let y = S(x). Since 2 2 ∈ SC = λ I, both, on Eiλ and on E iλ, − − S(y)= S2(x)= λ2x. − This shows that W λ := span x,y is an S-invariant subspace of V λ. If V λ denotes the or- 1 { } 1 thogonal complement of W λ in V λ, then V λ is S-invariant, since St = S. 1 1 − In this way, one can ﬁnd a decomposition of V λ into an orthogonal direct sum of two- λ dimensional S-invariant subspaces Wj and by summing up over the eigenvalues, one obtains the required decomposition of V .

179 Lemma 7.3. Let G be a Lie group, M a dense subset of C (G) and a Hilbert space. Furthermore, let ∗ H for every j N the mapping ϑ : C (G) ( ) be linear and bounded by c ∗ for a ∈ j ∗ → B H k · kC (G) constant c > 0 (independent of j) and let (πj)j N be a sequence of representations of G on ∈ the Hilbert space such that H j π (h) ϑ (h) →∞ 0 h M. k j − j kop −→ ∀ ∈ Then, j π (a) ϑ (a) →∞ 0 a C∗(G). k j − j kop −→ ∀ ∈ Proof: Let a C∗(G) and ε> 0. ∈ ε Since M is dense in C (G), there exists h M such that a h ∗ < forc ˜ := max 1,c . ∗ ∈ k − kC (G) 3˜c { } Furthermore, there exists J(ε) > 0 in such a way that for all integers j J(ε), one has ≥ π (h) ϑ (h) < ε . k j − j kop 3 For all j J(ε), ≥ π (a) ϑ (a) π (a) π (h) + π (h) ϑ (h) + ϑ (h) ϑ (a) . k j − j kop ≤ k j − j kop k j − j kop k j − j kop By assumption, π (h) ϑ (h) < ε . In addition, as π is a homomorphism, k j − j kop 3 j ε πj(a) πj(h) op = πj(a h) op sup π˜(a h) op = a h C∗(G) < . k − k k − k ≤ π˜ G − k − k 3˜c ∈ b

Moreover, cε ε ϑ (h) ϑ (a) = ϑ (h a) c h a ∗ < . k j − j kop k j − kop ≤ k − kC (G) 3˜c ≤ 3 Thus, π (a) ϑ (a) <ε k j − j kop and the claim is shown.

Corollary 7.4. Let M be a dense subset of C∗(G) and let π and (πj)j N be irreducible representations of G ∈ such that j π (h) π(h) →∞ 0 h M. k j − kop −→ ∀ ∈ Then, j π (a) π(a) →∞ 0 a C∗(G). k j − kop −→ ∀ ∈ Letting ϑ := π for all j N, Corollary 7.4 follows directly with Lemma 7.3. j ∈ Corollary 7.5. Let the sets Γ and S and the Hilbert spaces for i 0,..., 8 be deﬁned as in Section 4.2.5 i i Hi ∈ { } and Section 4.2.2. Let i 0,..., 8 , let M be a dense subset of C∗(G), let ν˜ : CB(Si 1) ( i) be a linear ∈ { } − → B H

180 map bounded by c Si for a constant c> 0 and let (πj)j N be a sequence of representations k·k −1 ∈ in Γi such that j →∞ πj(h) ν˜ (h) Si 0 h M. − F | −1 op −→ ∀ ∈

Then, j →∞ πj(a) ν˜ (a) Si 0 a C∗(G). − F | −1 op −→ ∀ ∈

Proof: N Let := i and ϑj :=ν ˜ ( ) Si for all j . Then, for all a C∗(G), H H F · | −1 ∈ ∈ ϑj(a) op = ν˜ (a) c (a) = c sup (a)(˜π) Si−1 Si−1 Si−1 op k k F | op ≤ F | π˜ i F ∈S −1 = c sup π˜(a) op c sup γ( a) op = c a C∗( G). π˜ i−1 ≤ γ C∗(G) k k ∈S ∈

Therefore, ϑ : C (G) ( ) is bounded by c ∗ and Corollary 7.5 follows with j ∗ → B H k · kC (G) Lemma 7.3.

Corollary 7.6. Use the notations from Section 3.5 and Section 3.6 and let M be a dense subset of C∗(G) such that j πj(h) νj pG/U (h) →∞ 0 h M. − op −→ ∀ ∈

Then, j πj(a) νj pG/U (a) →∞ 0 a C∗(G). − op −→ ∀ ∈

Proof: Let := L2(Rd) and let ϑ := ν p ( ) for all j N. Then, for all a C (G), by H j j G/U · ∈ ∈ ∗ Proposition 3.3 and Proposition 3.5, respectively,

ϑj(a) op = νj pG/U (a) pG/U (a) ∗ a C∗(G). k k op ≤ C (G/U,χℓ) ≤ k k Therefore, ϑ : C (G) ( ) is bounded by c ∗ and Corollary 7.6 follows with j ∗ → B H k · kC (G) Lemma 7.3.

Lemma 7.7. 2n Let n N , let BR be the 2n-dimensional real unit ball and deﬁne the mapping ∈ ∗ n 1 n 2n ψ : [0, 1] − [0, 2π) (0, 1] BR , × × → ψ(s1,...,sn 1,t1,...,tn, ρ) := −

ρ √s1 cos(t1), √s1 sin(t1),..., √sn 1 cos(tn 1), √sn 1 sin(tn 1), − − − − √1 s cos(t ), √1 s cos(t ) , − n − n n 1 − where s = si. i=1 Then, the absoluteP value of the determinant of the Jacobian of ψ equals 1 ρ2n 1. 2n−1 · −

181 Proof: Denote for i 1,..., 2n by C the i-th column and by R the i-th row of the Jacobian of ψ. ∈ { } i i For i 1,...,n 1 , one has ∈ { − } 0 0 . .  .   .  0 0      ρ cos(ti)   ρ sin(ti)   2√si  i th row  2√si    ← − →    0   0       .   .  C2i 1 =  .  , C2i =  .  −      0   0       ρ√si sin(ti) (n 1+ i) th row ρ√si cos(ti) −     0  ← − − →  0       .   .   .   .       0   0       √s cos(t )   √s sin(t )   i i   i i      and

ρ cos(tn) ρ sin(tn) − 2√1 s − 2√1 s . − . −  .   .  ρ cos(tn) ρ sin(tn)  2√1 s  (n 1) th row  2√1 s   − −  ← − − →  − −   0   0  C2n 1 =   , C2n =   . −  .   .   .   .   .   .       0   0       ρ√1 s sin(tn) ρ√1 s cos(tn) − −   −   √1 s cos(tn)   √1 s sin(tn)   −   −      Now, in several steps, this matrix will be transformed into a new matrix whose determinant can easily be calculated. For simplicity, the columns and rows of the matrices appearing in each step will also be denoted by C and R for i 1,..., 2n . i i ∈ { } First, one takes out the factor ρ in the rows R for i 1,..., 2n 1 , the factor 1 in i ∈ { − } 2 the rows Ri for i 1,...,n 1 , the factor √si in the columns C2i 1 and C2i for every ∈ { − } − i 1,...,n 1 and the factor √1 s in the columns C2n 1 and C2n. Hence, one has the ∈ { 2n− 1 }1 − − prefactor ρ − n−1 s1 sn 1(1 s) and the columns of the remaining matrix have the shape 2 ··· − −

182 0 0 . .  .   .  0 0      cos(ti)   sin(ti)   si  i th row  si    ← − →    0   0       .   .  C2i 1 =  .  , C2i =  .  −      0   0       sin(t ) (n 1+ i) th row cos(t ) − i  ← − − →  i   0   0       .   .   .   .       0   0           cos(ti)  sin(ti)     for all i 1,...,n 1 and ∈ { − }

cos(tn) sin(tn) 1 s 1 s − .− − .−  .   .  cos(tn) (n 1) th row sin(tn)  1 s   1 s   − −  ← − − → − −   0   0  C2n 1 =   , C2n =   . −  .   .   .   .       0   0       sin(tn)  cos(tn)  −     cos(t )   sin(t )   n   n     

Next, for every i 1,...,n , the column C2i 1 shall be replaced by sin(ti)C2i 1 cos(ti)C2i. − − ∈ { } 2n 1 1 s1 sn−1(1 s) − Then, the prefactor changes to ρ n−1 ··· − and for every i 1,...,n , − 2 sin(t1) sin(tn) ··· ∈ { }

0 .  .  0   C2i 1 =  1 . (n 1+ i) th row − −  ← − −  0     .   .     0      The columns C for i 1,...,n stay the same. 2i ∈ { } Now, for all i 1,...,n 1 , the rows Ri and Rn 1+i will be interchanged. Therefore, the ∈ { − } − prefactor is multiplied by ( 1)n 1 and for every i 1,...,n 1 , − − ∈ { − }

183 0 .  .  0 0 .    .  C2i 1 =  1 , i th row C2n 1 = 0 − −  ← − −  0       1  .  −   .   0       0        and 0 .  .  0 0   . cos(ti)  .    i th row  0  ← − 0      .   sin(tn)   .  n th row  1 s  C2i =   , − → C2n = − .−  .  0   .     .   sin(ti)   sin(tn)   si  (n 1+ i) th row  1 s    ← − − − −   0       cos(tn)   .     .   sin(tn)       0        sin(ti)   In the next step, for every i 1,...,n 1 , the matrix will be developed with respect ∈ { − } to the i-th row, which has only one non-zero entry, namely the entry 1 in the (2i 1)-th − − column. Furthermore, one develops with respect to the (2n 1)-th row which also only consists − of one non-zero entry, 1, in the (2n 1)-th column. The prefactor is then multiplied by n −1 n−1 n 2n 1+2n 1 − i+2i 1 − n+i 1 ( 1) ( 1) − − ( 1) − = ( 1) − , i.e. the prefactor now equals − − i=1 − i=1 − Q Q n 1 n 1 n 1 − n+i 1 2n 1 1 s1 sn 1(1 s) − i 2n 1 1 s1 sn 1(1 s) − − ( 1) − ( 1) − ρ − n 1 ··· − = ( 1) ρ − n 1 ··· − . − − 2 − sin(t1) sin(tn) − 2 − sin(t1) sin(tn) Yi=1 ··· Yi=1 ··· One has a n n-matrix left, whose columns have the shape × 0 . .   sin(tn) 0 1 s − −  sin(ti)  .  s  i th row  .  Ci =  i  , ← − Cn =  0  sin(tn)    1 s   .  − −   .   sin(t )     n   0        sin(ti)  

184 for all i 1,...,n 1 . Now, in every column Ci for i 1,...,n , one can take out the factor ∈ { − } n 1 ∈ { } − i 2n 1 1 sin(ti). Then, the prefactor changes to ( 1) ρ − 2n−1 s1 sn 1(1 s) and one has the i=1 − ··· − − following columns: Q

0 .  .  1 0 1 s   − .−  1  i th row .  si   .  Ci =   , ← − Cn = 1  0  1 s   − −   .     .   1       0     1      n 1 1 − for all i 1,...,n 1 . In the last step, the column Cn will be replaced by Cn + 1 s siCi. ∈ { − } − i=1 Since P n 1 1 − 1 s 1 1 1+ s = − + s = , 1 s i 1 s 1 s 1 s − Xi=1 − − − one obtains the columns

0 .  .  0 0   .  1  i th row .  si   .  Ci =   , ← − Cn =  0  0      .   1   .   1 s     −   0     1      n 1 − i 2n 1 1 for i 1,...,n 1 and the prefactor stays the same, i.e. ( 1) ρ − 2n−1 s1 sn 1(1 s). ∈ { − } i=1 − ··· − − Since the remaining matrix is a triangular matrix, one canQ easily calculate its determinant and gets

n 1 n 1 − i 2n 1 1 1 − i 1 2n 1 ( 1) ρ − n 1 s1 sn 1(1 s) = ( 1) n 1 ρ − . − 2 − ··· − − s1 sn 1(1 s) − 2 − · Yi=1 ··· − − Yi=1

185 8 Acknowledgements

First, I want to express my deepest gratitude to my Ph.D. advisor Professor Jean Ludwig for the continuous support during the last years. I appreciated our regular meetings, his patience and his advice that I received despite his retirement two years ago.

Furthermore, my sincere thanks goes to my advisor Professor Martin Olbrich for his guidance during my Ph.D., for many fruitful discussions and for sharing his great expertise with me.

Besides my advisors, I want to thank Professor Ingrid Beltita and Professor Joachim Hilgert for having accepted to refer this thesis and Professor Angela Pasquale and Professor Anton Thalmaier for having joined my Dissertation Committee.

Moreover, I would like to express my thanks to the proofreaders for reading my manuscript, for their suggestions and ideas.

I also want to thank the Ph.D. students, Post-Docs and friends at the University of Luxembourg for the good working atmosphere, the nice and funny lunch and coﬀee breaks and the numerous joint activities that we undertook during the last four years.

Besides, I would like to give thanks to the Ph.D. students and friends at the Univer- sit´ede Lorraine in Metz, for the lunch breaks and the conferences we participated in together.

Last, I want to thank my parents who supported and encouraged me with everything and who have always been there for me. Without them, I would not have succeeded in my Ph.D.

This work is supported by the Fonds National de la Recherche, Luxembourg (Project Code 3964572).

186 9 References

1. L.W.Baggett, A description of the topology on the dual spaces of certain locally compact groups, Transactions of the American Mathematical Society 132, pp.175-215, 1968.

2. D.Beltita, I.Beltita and J.Ludwig, Fourier transforms of C∗-algebras of nilpotent Lie groups, to appear in: International Mathematics Research Notices, doi:10.1093/imrn/rnw040.

3. C.Benson, J.Jenkins, R.Lipsman and G.Ratcliﬀ, A geometric criterion for Gelfand pairs associated with the Heisenberg group, Paciﬁc Journal of Mathematics 178, no.1, pp.1-36, 1997.

4. C.Benson, J.Jenkins and G.Ratcliﬀ, Bounded K-spherical functions on Heisenberg groups, Journal of Functional Analysis 105, pp.409-443, 1992.

5. C.Benson, J.Jenkins, G.Ratcliﬀ and T.Worku, Spectra for Gelfand pairs associated with the Heisenberg group, Colloquium Mathematicae 71, pp.305-328, 1996.

6. I.Brown, Dual topology of a nilpotent Lie group, Annales scientiﬁques de l’E.N.S.´ 4e s´erie, tome 6, no.3, pp.407-411, 1973.

7. L.Cohn, Analytic Theory of the Harish-Chandra C-Function, Springer-Verlag, Berlin- Heidelberg-New York, 1974.

8. L.Corwin and F.P.Greenleaf, Representations of nilpotent Lie groups and their applications. Part I. Basic theory and examples, Cambridge Studies in Advanced Mathematics, vol.18, Cambridge University Press, Cambridge, 1990.

9. J.Dixmier, C∗-algebras. Translated from French by Francis Jellett, North-Holland Mathematical Library, vol.15, North-Holland Publishing Company, Amsterdam-New York-Oxford, 1977.

10. J.Dixmier and P.Malliavin, Factorisations de fonctions et de vecteurs indéfiniment différentiables, Bulletin des Sciences Mathématiques (2) 102, no.4, pp.307-330, 1978.

11. M.Elloumi, Espaces duaux de certains produits semi-directs et noyaux associ´es aux orbites plates, Ph.D. thesis at the Universit´ede Lorraine, 2009.

12. M.Elloumi and J.Ludwig, Dual topology of the motion groups SO(n) ⋉ Rn, Forum Mathematicum 22, pp.397-410, 2010.

13. J.M.G.Fell, The structure of algebras of operator ﬁelds, Acta Mathematica 106, pp.233-280, 1961.

14. G.B.Folland, Harmonic Analysis in Phase Space, Princeton University Press, 1989.

15. W.Fulton and J.Harris, Representation theory, Readings in Mathematics, Springer- Verlag, 1991.

187 16. J.-K.G¨unther, The C∗-algebra of SL(2, R), arXiv:1605.09256, 2016.

17. J.-K.G¨unther and J.Ludwig, The C∗-algebras of connected real two-step nilpotent Lie groups, Revista Matem´atica Complutense 29(1), pp.13-57, 10.1007/s13163-015-0177-7, 2016.

18. W.J.Holman III and L.C.Biedenharn, The Representations and Tensor Operators of the Unitary Groups U(n), in: Group Theory and its Applications, Volume 2, edited by E.M.Loebl, Academic Press, Incorporation, London, 1971.

19. R.Howe, Quantum mechanics and partial diﬀerential equations, Journal of Functional Analysis 38, pp.188-255, 1980.

20. A.Knapp, Representation Theory of Semisimple Groups. An Overview Based on Exam- ples, Princeton University Press, Princeton, New Jersey, 1986.

21. R.Lahiani, Analyse Harmonique sur certains groupes de Lie `acroissance polynomiale, Ph.D. thesis at the University of Luxembourg and the Universit´ePaul Verlaine-Metz, 2010.

22. S.Lang, SL2(R), Graduate Texts in Mathematics 105, Springer-Verlag, New York, 1985. 23. H.Leptin and J.Ludwig, Unitary representation theory of exponential Lie groups, De Gruyter Expositions in Mathematics 18, 1994.

24. Y.-F.Lin and J.Ludwig, The C∗-algebras of ax + b-like groups, Journal of Functional Analysis 259, pp.104-130, 2010.

25. R.L.Lipsman, Orbit theory and harmonic analysis on Lie groups with co-compact nil- radical, Journal de Math´ematiques Pures et Appliqu´ees, tome 59, pp.337-374, 1980.

26. J.Ludwig and L.Turowska, The C∗-algebras of the Heisenberg Group and of thread-like Lie groups, Mathematische Zeitschrift 268, no.3-4, pp.897-930, 2011.

27. J.Ludwig and H.Zahir, On the Nilpotent -Fourier Transform, Letters in Mathematical ∗ Physics 30, pp.23-24, 1994.

28. G.W.Mackey, Unitary group representations in physics, probability and number theory, Benjamin-Cummings, 1978.

29. L.Pukanszky, Le¸cons sur les repr´esentations des groupes, Dunod, Paris, 1967.

30. H.Regeiba, Les C -alg`ebres des groupes de Lie nilpotents de dimension 6, Ph.D. thesis ∗ ≤ at the Universit´ede Lorraine, 2014.

31. H.Regeiba and J.Ludwig, C∗-Algebras with Norm Controlled Dual Limits and Nilpotent Lie Groups, arXiv: 1309.6941, 2013.

32. N.Wallach, Real Reductive Groups I, Academic Press, Pure and Applied Mathematics, San Diego, 1988.

188 33. N.Wallach, Real Reductive Groups II, Academic Press, Pure and Applied Mathematics, San Diego, 1992.

34. A.Wassermann, Une démonstration de la conjecture de Connes-Kasparov pour les groupes de Lie linéaires connexes réductifs, Comptes Rendus de l’Académie des Sciences, Paris Series I Mathematics 304, no.18, pp.559-562, 1987.

189