arXiv:2105.05604v1 [math.RT] 12 May 2021 i group Lie SL n elsltsml i group Lie simple split real any rcsl h otnosda anplnmas n ecnt can we and polynomials, Hahn dual continuous hyperge the certain precisely on operator multiplication coordinate the rsn ae esalsuyti rbe o h metaplecti the for problem group this study shall we paper present h yblo h eei transform. Berezin the of symbol the fhroi siltro h form the of oper oscillator differential partial harmonic some of of spectrum the found tially rmw opt h cino h aii lmn nsm orth some on element Casimir the of of action the compute we trum lsia prahb nigteseta fteCsmrope Casimir the of spectral the finding by approach classical o perin appear not rvdta est o rhgnlt speieyteprod the precisely the is in orthogonality observed for density been that has proved problems b branching connection and The 3.7. polynomials Remark see [11]; transform Berezin the matrices h i algebra Lie the sl ( RNHN FMTPETCRPEETTO OF REPRESENTATION METAPLECTIC OF BRANCHING h rbe ffidn h rnhn ueo ntr represe unitary a of rule branching the finding of problem The h ealci ersnainfor representation metaplectic The ermr hti etase u eut nteSchr¨odinger the in results our transfer we if that remark We o h vnpr ftemtpetcrpeetto hsdec this representation metaplectic the of part even the For ial emninta h bv rbe for problem above the that mention we Finally ofidtedsrt opnnsw ov h qaino highe of equation the solve we components discrete the find To SO 2 eerhspotdprilyb h wds cec Counci Science Swedish the by partially supported Research ( , 2 R , ( G R ) 2 rnil subgroup principle A ) cin nteFc pc.W rv htlws egtrepres weight lowest that prove We space. Fock the in actions ⊂ ) = A BSTRACT e 2.Frgeneral For [2]. see ; Sp G and SL ( ne subgroup a under ( Λ 2 g B ( , G , 2 esuybacigpolmo h ealci representat metaplectic the of problem branching study We . R csa ifrniloeaoswt udai oyoilc polynomial quadratic with operators differential as acts n hsmgtb fidpnetinterests. independent of be might this and , , ) R ) ne t rnia subgroup principal its under ) n efidalhgetwih representations. weight highest all find we and fafie egt epoeta aii lmn suiayequ unitary is element Casimir that prove We weight. fixed a of SL ( 2 , PRINCIPAL R ) G efidtecmlt decomposition. complete the find We . G H nedu ocnuainteei nqepicplsubgro principal unique a is there conjugation to up Indeed . = so udmna neet nrpeetto hoy nth In theory. representation in interests fundamental of is α G Sp ( .I 0. ∂ ∂ A∂, = ( EKIZHANG GENKAI n, Sp NTRODUCTION SL R + ) ( ) 2 ( , ecnuecvratdfeetain oproduce to differentiations covariant use can we 1 2 R β SL , SL ) ( R x x Bx, a eraie nteFc space Fock the on realized be can ) ( ( -SUBGROUP 2 V) . (VR). l 2 , , R R ) ⊂ ) mti rhgnlplnmas more polynomials, orthogonal ometric ) tr fdge orivligsquares involving four degree of ators c fHarish-Chandra of uct ihnnpstv ent symmetric definite non-positive with . rator representation c moiincnas etetdusing treated be also can omposition G L tenhpremti orthogonal hypergeometric etween u n h pcrmof spectrum the find hus 2 = ( C R ntation Sp gnlbsso egtvectors weight of basis ogonal ale ok 1,1] twas it 12], [10, works earlier to oetwih etr for vectors weight lowest or st 2 ofidtecniuu spec- continuous the find To . ) o of ion ( raiain ehv essen- have we -realization, 2 nain of entations Sp , R ( ( Sp ) efiins euethe use We oefficients. Λ 2 a efruae for formulated be can , G , ( R Λ 2 , ) ) R ftesymplectic the of NE ITS UNDER ) fasemisimple a of ne its under c SL fnto and -function F( ( 2 C vln to ivalent , C . R 2 ) ) will and up e 2 GENKAI ZHANG some discrete components, but the results are not complete as for G Sp 2, R and will not be present here; see e.g. [9] for related study. It is also tempting but challenging to study the branching problems for the minimal representations of G under SL 2=, R ,( even) for groups of lower rank such as SL 3, R and G2. The paper is organized as follows. In Section 1 we recall the realization( ) of the metaplectic representation of Sp n,( R )on the Fock space. We find the explicit description of the principal SL 2, R -subgroup in Sp n, R and its action on the metaplectic representation in Section 2. The complete decomposition( ) of the metaplectic representation of Sp 2, R is done in Section 3. Acknowledgement.( ) I would( ) like to thank Wee Teck Gan for posing the question studied in this paper and for some stimulating discussions. ( )

1. PRELIMINARIES We recall some known facts on metaplectic representations of G Sp n, R .

= ( ) 1.1. The group G Sp n, R . The symplectic group G Sp n, R has usually two different realizations, as the group of biholomorphic transformations of the Siegel upper half space or the bounded symmetric= ( domain.) We shall use the latter one,= wh( ich) is sometimes denoted by

G Sp n, R c. See [3, 4, 5, 8]. Let Sp n, C be the group of complex 2n 2n-matrices preserving the standard complex = ( ) symplectic form Ω z,w z1wn 1 znw2n zn 1w1 z2nwn. Let U n, n be the indefinite + + ( ) 2n × unitary group on C preserving the Hermitian form z,w n,n z1w1 znwn z2nw2n of signature n, n (. Let)G = Sp n,+⋯+R U n,− n Sp −⋯−n, C . Elements in(G will) be represented as 2 2-block matrices ⟨ ⟩ = + ⋯ + − ⋯ − ( ) = ( ) ∶= ( ) ∩ ( ) a b (1.1)× g ¯b a¯ =  where a, b are complex n n-matrices. Let K U n be the subgroup of block diagonal matrices g diag a, a¯ , with a U n , U n being in its standard realization. Then K is the maximal subgroup of G, and to save× notation we identify≃ diag( ) a, a¯ with a. = ( ) ∈ ( ) ( ) s C The Cartan decomposition of the Lie algebra g is g k p with k u n , p Mn B t ( ) Mn C ; B B ; more precisely = + = ( ) = ( ) = { ∈ (1.2)( ) = } k diag A, At , A u n ,

= { ( − ) ∈ ( )} 0 B (1.3) p ξ ; B Bt M C , B B¯ 0 n = œ =  = ∈ ( )¡ in the above matrix realization. BRANCHING OF METAPLECTIC REPRESENTATION OF Sp(2, R) UNDER ITS PRINCIPAL SL(2, R)-SUBGROUP3

1.2. Metaplectic representation. Let Cn be the Fock space of entire functions f Cn on Cn such that F =F( ) ∈ 2 2 π z 2 f f z e− S S dz , O( ) Cn α α1 αn where dz is the Lebesgue measure.Y Y The= S monomialsS ( )S z kernel e Y( Y), namely= ( )⋯( )

π ,z¯ π z,w¯ π w 2 n (1.5)F f z f, e (⋅ ) f w e ( )e− S S dw, f , z C , C where z,w z(jw)j =is⟨ the -bilinear⟩ = S form.( ) ∈F ∈ The metaplectic representation Λ, , Gˆ is a unitary representation of the double cover Gˆ Mp n,(R of) =G∑ Sp n, R on the Fock space , and is explicitly given by [5, Theorem 4.37], ( F ) = 1 π 1 1 1 π w 2 Λ( g f) z det= − 2( a ) exp ¯ba z, zF 2 a z, w¯ a bw,¯ w¯ f w e dw, 2 − − − − S S for g(as) in( (1.1).) = In particular( ) S the› group
( U n acts) + as( ) − ( ) ( ) 1 1 Λ a f z det−(2 )a f a− z , a U n . ⊕ ⊕ The space (is) sum( of) = the subspaces( ) ( of even) and∈ odd( functions,) with Λ Λ+ Λ− + − a sum of two irreducible representations. F=F F =

Remark 1.1. The representations Λ+ and Λ− can be realized as reproducing kernel spaces of of holomorphic functions on the Siegel bounded symmetric domain D Sp n, R U n with 1 1 2 1 2 reproducing kernels det 1−zw∗ − and 1−zw∗ − det 1−zw∗ − , respectively. see [6]. These realizations can be used to find the branching rule using geometric methods,= also( for)~ a larger( ) class of holomorphic representations.( ) ( ) ( )

Lemma 1.2. The Lie algebra action of sp n, C on is given as follows 1 π Λ X f − B∂,∂( f,) ΛFY f Bz,z f, 2π 2 ( ) = ( tr)D ( ) = ( ) Λ Z f z − f z − ∂ f z , 2 Dz for ( ) ( ) = ( ) ( )( ) 0 B 0 0 X X ,Y Y , B Bt, B 0 0 B B 0 = =  = =  = D 0 Z Z ,D gl n . D 0 −Dt = =  ∈ ( ) 4 GENKAI ZHANG

d Proof. Let ξB be as in (1.3) and perform the differentiation dt of the action exp tξB G at t 0,

π π z,w¯ π w 2 Λ ξB f z Bz,z − Bw,¯ w¯ e ( )f w e− S S dw.( ) ∈ = 2 The integration of( the) first( term) = is
( ) ( ) ( ) S π Bz,z f z 2 by the reproducing kernel formula (1.5). Differentiating( ) ( ) the formula (1.5) by B∂,∂ we find 2 π z,w¯ π w 2 B∂,∂ f z π Bw,¯ w¯ e ( )f w e− S S dw. ( )

Thus the second integration( above) ( ) is= S ( ) ( ) π 1 1 − B∂,∂ f z − B∂,∂ f z . 2 π2 2π Taking the complex linear and conjugate( linear) ( ) parts= in(B we) get( the) first two formulas. The last formula is a straightforward computation. 

1.3. Unitary representations of SU 1, 1 and their realizations. Let G0 ∶ SU 1, 1 be the group of 2 × 2-matrices preserving the Hermitian form z,w 1,1 z1w¯1 − z2w¯2 with unit determi- nant. We fix the maximal subgroup ( ) = ( ) ⟨ ⟩ = eiθ 0 (1.6) L U 1 uθ iθ G0. 0 e− We fix also the Lie algebra elements= ( ) = œ =  ¡ ⊂ 1 0 0 1 0 0 (1.7) h , e , e , e e + e 0 −1 + 0 0 − 1 0 + − forming the standard= basis of sl 2, C .= Let =  =

1 2 1 2 C (e+e− )+ e−e+ + h 2e+e− − h + h 2 2 be the Casimir element. = = Recall that any unitary irreducible representation σ of SU 1, 1 is one of the following

(1) Spherical principal or complementary series σiλ, with all weights of h being even inte- 1 2 + ( ) 1 gers; the eigenvalue of Casimir C is − 2 − λ for λ R+ or λ 0, 2 .

(2) Non-spherical principal series σiλ, ; for λ R+ with all weights of h are odd integers; the 1 −2 ∈ ∈ ( ) eigenvalue of Casimir C is − 2 − λ for λ R+. ∈ (3) Highest weight representations σ ν with negative integral weights −ν, ν 1; the eigen- 1 2 − ∈ value of Casimir C is 2 ν − ν. ≥ (4) Lowest weight representations σν with positive integral weights ν, ν 1; the eigenvalue 1 2 of Casimir C is 2 ν − ν. ≥ The non-integral highest or lowest weights ν corresponde representations of the universal cov- ering of SU 1, 1 , which will not concern us here.

( ) BRANCHING OF METAPLECTIC REPRESENTATION OF Sp(2, R) UNDER ITS PRINCIPAL SL(2, R)-SUBGROUP5

1.4. Continuous dual Hahn polynomials and spectra of multiplication operators. We recall the orthogonality relation of continuous dual Hahn polynomials. They are special cases of Wilson hypergeometric polynomials[7], and will be used in Section 3 below to find thespectra of Casimir element.

Proposition 1.3. ([7, pp. 196-199]) Let a 0, b 0,c 0, µ be the measure 2 1 1 Γ a + ix Γ b + ix Γ c + ix dµ x ≥ > > dx 2π Γ a + b Γ a + c Γ b + c Γ 2ix ( ) ( ) ( ) R ∞ ( ) = 2 R W 2 W on + 0, and L (+,µ the) ( corresponding) ( ) L -space. Let( ) 2 = [ ) Wm x ( 3F2) −m, a + ix, a − ix; a + b, a + c, 1 , m 0, 1,..., 2 2 R be the continuous dual( Hahn) = polynomials.( Then Wm x form) an orthogonalbasis= of L +,µ ,

m! b + c m Wm, Wl { ( )} δm,l. ( ) a + b m a + c m ( ) ⟨− 2 ⟩ = The multiplication operator by x has the( following) ( 3-term) recursion on the basis, 2 2 2 2 2 2 −x Wm x AmWm 1 x − Am + Cm − a Wm x + CmWm 1 x , + − where ( ) = ( ) ( ) ( ) ( )

Am m + a + b m + a + c ,Cm m m + b + c − 1 . 2 2 R R In particular the spectrum= ( of the multiplication)( )x on =L ( +,µ is the positive) half line + 0, ∞ . ( ) = 2 x 2 [ We) shall need rescaled Wilson polynomials Wm x ∶ Wm 3 . The factor of 3 in our case is closely related to our imbedding of SL 2, R in Sp 2, R via triple symmetric tensor product 3 2 4 È ( ) = È ((x ) ) S R R . We denote the corresponding measure dµ x dµ 3 . ( ) ( ) 2 Corollary= 1.4. Let a 0, b 0,c 0, Am,Cm be as̃( above.) = The( ) polynomials Wm x form an 2 orthogonal basis, L R+, µ , ≥ > > È ( ) 2 m! b + c m (1.8) ( ̃) Wm , a + b m a + c m ( ) and satisfy the following recurrenceYÈ relationY = ( ) ( ) − 1 2 + 2 2 − + − 2 + d 2 + 2 (1.9) 2 x d Wm x AmWm 1 x Am Cm a 2 Wm x CmWm 1 x , 3 + 3 − for any d R(. ) ( ) = ( ) ( ) ( ) ( )

The introduction∈ of d is purely for practical convenience for identifying the spectrum of some abstract operator with the multiplication operator x2 + d in Section 3. 6 GENKAI ZHANG

2. THE GROUP HOMOMORPHISM ι ∶ G0 SU 1, 1 → G AND THE RESTRICTION OF REPRESENTATIONS OF G TO G0 = ( ) We find the explicit realization of the principal G0 SU 1, 1 ( SL 2, R ) subgroup in the symplectic group G and find explicit formulas for pull-back of the metaplectic representation to = ( ) = ( ) G0.

2 2.1. The homomorphism ι ∶ G0 → G. The defining action of SL 2, C on C preserves the complex symplectic form dz1∧dz2. Its symmetric power representation ι ∶ SL 2, C → SL 2n, C 2n 1 2 2n 2n 1 ( ) on S − C C preserves the symplectic form S − dz1 ∧ dz2 . This defines a group homo- morphism ι ∶ SL 2, C → Sp n, C . ( ) ( ) = ( ) 2n 1 It is immediate that ι G0 , G0 SU 1, 1 , preserves also the Hermitian form S − ⋅, ⋅ 1,1 of signature n, n .( Thus) we have( the) group homomorphism ( ) = ( ) ⟨ ⟩ ( ) ι ∶ G0 SU 1, 1 → U n, n ∩ Sp n, C Sp n, R G. We can find explicitly= the image( ) of ι of( SU) 1, 1 (and of) = the( Lie algebra) = sl 2, C . Let m 2n − 1 and realize V SmC2 as the space homogeneous polynomials f x, y degree of m on C2, g GL 2, C ∶ f x, y ↦ f x, y g . The( space) V is equipped with the dual( of) Hermitian= 2n 1 = 2n 1 ( ) form S − ⋅, ⋅ 1,1 and dual of the symplectic form S − dx ∧ dy , namely Ω p, q ∂x1 ∂y2 − ∈m ( ) ( ) (( ) ) ∂y1 ∂x2 p x1,y1 q x2,y2 . We fix an orthonormal and symplectic basis of V as ⟨ ⟩ ( ) ( ) = ( ) ( ( ) ( m )) m 2 k 1 2 k 1 m m 2 k 1 2 k 1 pk x − ( − )y ( − ) x − ( − )y ( − ), ¾ 2 k − 1 ¾ 2 n − k + 1 = ‹  = ‹  m ( m)2 k 1 2 k 1 m( ) m 2 k 1 2 k 1 qk y − ( − )x ( − ) y − ( − )x ( − ), 1 k n; ¾ 2 k − 1 ¾ 2 n − k + 1 namely they= satisfy‹  = ‹  ≤ ≤ ( ) ( ) pk,pj n,n − qk, qj n,n δkj, Ω pk,pj Ω qk, qj 0, Ω pk, qj δkj, 1 k, j n.

Proposition⟨ ⟩ 2.1.= ⟨Let kθ⟩be the= diagonal( matrix) = ( ) = ( ) = ≤ ≤

2n 1 iθ 2n 1 4 iθ 2n 3 iθ kθ diag e( − ) , e( − − ) , ⋯, e−( − ) .

C Then ι uθ kθ and the Lie algebra= ( g -elements )

( ) = H ∶ ι h , E± ∶ ι e± , E ι e E+ + E− are given by = ( ) = ( ) = ( ) = D 0 0 B 0 B + C H , E , E E t, E 0 −D + C 0 − + B + C 0 where =  =  = ( ) =  D diag 2n − 1, 2n − 5, ⋯, − 2n − 3 ,

= ( ( )) BRANCHING OF METAPLECTIC REPRESENTATION OF Sp(2, R) UNDER ITS PRINCIPAL SL(2, R)-SUBGROUP7

B is skew diagonal and symmetric and C lower skew diagonal and symmetric,

0 ... 0 β1 0 ... 0 0 0 ... β2 0 0 ... 0 γ2 B ⎡ ⎤ , C ⎡ ⎤ , ⎢ ⋮ ⋰ ⋮ ⋮ ⎥ ⎢ ⋮ ⋮ ⋰⋮ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢β ... 0 0 ⎥ ⎢0 γ 0 0 ⎥ = ⎢ n ⎥ = ⎢ n ⎥ ⎢ ⎥ ⎢ ⎥ with ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ ⎣ ⎦ βk 2k − 1 2 n − k + 1 , 1 k n, γ » 2 k − 1 n − k + 1 , 2 k n; =k ( )( ( ) ) ≤ ≤ that is B b , C c with » jk jk = ( )( ) ≤ ≤ bjk βkδj,n k 1, cjk γkδj,n k 2, 1 k, j n = ( ) = ( ) − + − + in term of the Kronecker symbol= δpq. = ≤ ≤

Proof. The action of kθ ι uθ is diagonal and is found immediately, so is H ι h . Performing d 1 t =0 on the action ι exp te ι ∶ f x, y ↦ f x, tx + y we find dt t = ( + ) 0 1 = ( )

S ( ( )) = ( )E+p(1 0), ( )

m m 2 k 1 1 2 k 1 1 E+pk π e+ pk 2k − 2 = x − ( − )+ y ( − )− ¾ 2 k − 1 = 2(k −) 2 = ( m ) ‹  2 k 1 ( ) ( − ) qn k 2 ¼m −( − ) ( 2)k 1‰ 1 Ž = ¼ ( − )− 2 k − 1 n − k + 1 qn k 2 ‰ Ž −( − ) for k 2, and » = ( )( ) E+qk 2k − 1 m − 2k + 2 pn k 1, 1 k n. ≥ − + The element E π ι e »E t by our choice of the basis.  − − = (+ )( ) ≤ ≤ C C R We remark that= the( ( homomorphism)) = ( ) of SL 2, into Sp n, realizes also SL 2, as a subgroup in a different real form Sp n, R in Sp n, C , and it is the principle SL 2, R -subgroup of G Sp n, R ; see [2] for the general study of( principle) SL (2, R )-subgroup in real split( groups.) This specific case of SL 2, R in G( Sp)n, R (is also) of interests in topology [1].( ) = ( ) ( ) 2.2. The induced action( of Lie) algebra= ( of sl) 2 on by ι. We shall find polynomials p that are annihilated by the differential operator B∂,∂ , and we shall call them B∂,∂ -harmonic, i.e. ( ) F ( ) ( ) (2.1) B ∂, ∂ p z 0.

When n 2 the matrices kθ,B,C,D are( of the) ( form) = 3iθ iθ 0 3 0 0 (2.2) = kθ diag e , e− , B ,C ,D diag 3, −1 . 3√ 0 0 2 = ( ) = √ =  = ( ) 8 GENKAI ZHANG

This combined with Lemma 1.2 gives the explicit form for the action of Λ restricted to ι sl 2 , − − − − 1 + 2 − 1 2 + (2.3) Λ H 1 3∂1 ∂2 , Λ E+ π− 3∂1∂2 πz2 , Λ E− π− ∂2 π 3z(1z(2. )) √ √ 3.( T)HECOMPLETEDECOMPOSITIONFOR = ( ) ( ) = Λ,Mp 2, R( UNDER) = SU 1, 1

3.1. Orthogonal basis for U 1 -weight vectors. Let( n 2.( We)) shall find orthogonal( ) polyno- mials in C2 of a fixed U 1 ι U 1 -weight. Let be the polynomial space. All weights refer to the metaplectic( representations) under U 1= unless otherwise explicitly stated. DenoteF =F( ) ( ) = ( ( )) P⊂F 2 ( ) π 3 (3.1) I z z1z2 . 3 3 (The coefficient is chosen to simplify the expression( ) = √ for the solutions of the equation (3.7) below.) Then I generates all invariants in the polynomial space of the defining action of ι U 1 . Let

α m α α + 1 ⋯ α + m − 1 be the Pochammer symbol. m1 m2 P ( ( )) The orthogonal basis vectors z1 z2 of are weight vectors of U 1 of weights −1−3m1 + ( ) = ( ) ( ) − − − − + µ m2, and modulo 3 they are of the form µ 3k 1, 3k, 3k 1. We denote U 1 the subspace of all weight vectors of weight µ{. Then if} k F0 we have ( ) ( ) = FS 3k 1 ⊕ C l k (3.2) U− 1− ≥ I z z1 , ( ) l≥0 FS = Q ( ( ) ) 3k ⊕ C l k (3.3) U− 1 I z z1 z2 , ( ) l≥0 and FS = Q ( ( ) ) 3k 1 ⊕ C l k 2 (3.4) U− 1+ I z z1 z2 ; ( ) l≥0 k 3k if k 0 we then replace z1 by z2− FSin the above= Q formulas.( ( ) ) We compute the norm of weight vectors. < m k 2 m k 2 Lemma 3.1. The square norms I z1 and I z2 are given by F F m k 2 1 2 2 1 I zY1 Y m! Y m +Y1 k m m, πk 3 3 1 2 1 YImzk Y2 = (m! )2 (3m + 1) ( ) ( ) . 2 πk k 3 m 3 m Proof. Using (1.4) we see thatY Y = ( ) ( ) ( ) ( ) π2 π4m 1 π4m 1 m k 2 2m m k 3m 2 + ! 3 ! ! + 1 3 ! I z1 z1 + z2 3m 4m k m k m 3m 4m k m m k m . 3 3 3 π + 3 π + NowY Y = ( √ ) Y Y = ( ) ( ) = ( ) ( ) 1 2 2 1 2 1 3m ! 3m 3m − 1 3m − 2 ⋯3 ⋅ 2 ⋅ 1 33mm m − m − ⋯1 ⋅ ⋅ 33mm! , 3 3 3 3 3 m 3 m and( this) = proves ( )( the first)( formula.) = ( )( ) = ( ) ( ) BRANCHING OF METAPLECTIC REPRESENTATION OF Sp(2, R) UNDER ITS PRINCIPAL SL(2, R)-SUBGROUP9

m k 2 Similarly I z2 is F Y Y π4m 1 1 1 ! 3 + ! ! 3 + 1 3 ! 3m 4m k m m k k 3m m m k m 3 π + π 3 with 3m ! being computed as above.( ) = ( ) ( ) 

( ) 3.2. Irreducible decomposition of Λ,Sp n, R . Our main result is the following

Theorem 3.2. The decomposition of( Λ, ,Mp( ))2, R under SU 1, 1 is given by

( F ∞ ( )) ∞ ( )∞ (3.5) Λ,Mp 2, R SU 1,1 σiλ, dλ ⊕ σiλ, dλ ⊕ σ 3k 1. ( ) 0 + 0 − k=1 − − ( ( ))S = ? We shall find the spectral decompositionS of the CasimirS element C on . We have ι C H2 E+E−+E−E++ 2 as element in the envelopping algebra of sp n, C , and Λ ι C Λ E+ Λ E− + Λ H 2 F ( ) = Λ E− Λ E+ + ( ) as operator on . To ease notation we write it just as C. The decomposition 2 ( ) ( ( )) = ( ) ( ) is done in several steps. (By) general( ) abstract theory the operatorF C has a well-defined self-adjoint extension on and k on any subspace U 1 of fixed weight k. ( ) F FS 1 1 Lemma 3.3. The Casimir element −C on − U 1 has continuous spectrum 2 , ∞ . ( ) F S H2 H2 [ ) Proof. The Casimir operator C E+E− + E−E+ + 2 2E+E− − H + 2 acts on any element 1 − − f U− 1 of weight 1, Λ H f f, as ( ) = = ∈PS ( ) = 1 3 Cf 2Λ E Λ E f + f + f 2Λ E Λ E f + f. + − 2 + − 2

1 = ( ) ( ) = (Im z) ( ) The subspace has an orthogonal basis W z 1 ( 2) with U− 1 m m m ( ) ( 3 ) ( 3 ) FS ( ) = 2 2 1 m! (3.6) Wm 1 2 π 3 m 3 m ( ) Y Y = 2 by Lemma 3.1. We compute the action of Λ E+(Λ) E(− )on Wm . We have, πΛ E− −∂2 + 2 π 3z1z2, and by straightforward computations, ( ) ( ) { } ( ) = √ 2 m 2 2 π 3 πΛ E− Im z −∂2 + π 3z1z2 z1z2 3 3 √ m 2 2 m 1 2 ( ) ( ) =− ( 3m 3m − 1 I)Œz √z2− + 3‘I + z z2− 1 −32m m − Imz 2 + 32Im 1 z z 2. = ( )( 3 ) 2−( ) + 2−( ) = ( ) ( ) 10 GENKAI ZHANG

2 3 2 2 1 π z1z2 1 − 1 2 + m m m Acting by πΛ E+ 3∂ ∂ π z2 we find, writing I z I z I − z 3√3 I − , that 2 √ π Λ E+ Λ(E−)I =m ( ) = ( ) ( ) = 2 1 m 2 2 m 1 2 2 2 2 1 m 2 2 m 1 2 − 3∂1∂2 −3 m m − I z + 3 I z z + π z −3 m m − I z + 3 I z z ( ) ( ) 3 2− + 2− 2 3 2− + 2− √ 2 2 = 3‹ ( 1 ) 2 π m 1( ) 3  ‹ 1 (π m ) ( )  − 3 −3 m m − m m − I − + 3 m + 1 m + I 3 3 3 3 3 3 3 √ 1 = + π23Œ2 −m(m − )Im(+ Im 1) √ ( )( ) √ ‘ 3 + 2 2 m 1 m m 1 3 π αm‹I +( + βmI) + γmI − with the leading coefficient α 1, the last coefficient = ( m ) 1 2 γ m2 m − m − = m 3 3 and the middle = 1 ( )( 1 ) 1 β − m + 1 m + − m m − − 2m + m + . m 3 3 3 Thus 1 = ( )( ) ( 1 ) = ( 1 ) 2 Λ E Λ E Im Im 1 − 2m2 + m + Im + m2 m − m − Im 1. 32 + − + 3 3 3 − Im z 1 2 1 2 1 2 Writing in terms( of) W(m z) = 1 ( 2) (and using 3) k 1 3 k (1 3 )(k 3 k k)+ 3 k + 3 for 3 m 3 m + + k m, m + 1, this becomes ( ) ( ) ( ) = ( ) ( ) = ( ) ( ) ( )( ) 1 1 2 1 + + − 2 + + + 2 = 2 π E+ π E− Wm m m Wm 1 2m m Wm m Wm 1. 3 3 3 + 3 − This is exact( the) same( recursion) = ( relation)( as (1.9)) for the con( tinuous dual) Hahn polynomials for 1 2 a 0, b ,c ,d 1; 3 3 moreover square norms (3.6) and (1.8)= are same.= In= other= words, Wm → Wm is a unitary operator 2 2 intertwining −Λ E+ Λ E− with the multiplication operator by 1 + x on the space L R+, µ . È 3 Thus −Λ E+ Λ E− has continuous spectrum 1, ∞ . But −Cf −2Λ E+ Λ E− f − 2 f so −C 1 ( ) ( ) ( ̃) has spectrum 2 , ∞ . This finishes the proof. ( ) ( ) [ ) = ( ) ( ) 0 1 Lemma 3.4. The[ Casimir) element −C on U 1 has continuous spectrum 2 , ∞ . ( ) Proof. This is proved by similar computationsF S as the above lemma. We consider[ the) action of C 0 0 0 on U 1 U 1 , the subspace of polynomials f of weight 0. If f 1 then Λ H f 0 ( ) ( ) U and ( ) P S ⊂F S ∈ PS ( ) = Cf 2Λ E+ Λ E− f. m 0 I z z2 The space has an orthogonal basis given by the polynomials W z 2 ( )4 , with U 1 m m m ( ) = ( ) ( ) 3 3 square norms computed in Lemma 3.1, ( ) ( ) FS 2 ( ) = 2 1 m! Wm 2 4 . π 3 m 3 m ( ) Y Y = ( ) ( ) BRANCHING OF METAPLECTIC REPRESENTATION OF Sp(2, R) UNDER ITS PRINCIPAL SL(2, R)-SUBGROUP11

We compute the action of C on Imz2 and find

1 m m 1 2 2 m 2 1 1 m 1 Λ E Λ E I z2 I z z2 − 2m + 2m + I z2 + m m − m + I z2 . 32 + − + 3 3 3 − Written( in) terms( )( of Wm)this = is ( ) ( )( ) ( )( )( ) 1 2 4 2 + + − 2 + + + 2 2 Λ E+ Λ E− Wm m m Wm 1 2m 2m Wm m Wm 1. 3 3 3 + 3 − This is the same( ) recursion( ) formula= ( for)( continuous) Hahn( polynomials in) for 1 1 5 1 a , b ,c ,d , 2 6 6 4 2 1 2 and it follows that the Λ E+ Λ E− with= the= multiplication= = operator by − d+x − 4 +x on 2 L R+, µ . This gives the spectrum of Λ E+ Λ E− and the Casimir element as claimed.  ( ) ( ) ( ) = ( ) We( studỹ) then the appearance of lowest( or) highest( ) weight representations. Lemma 3.5. There exists no lowest weight unitary representation of SU 1, 1 in Λ, . The only highest weight unitary representations of SU 1, 1 in Λ, are of the form σ 3k 1, k 0, − − and they all appear of multiplicity one. ( ) ( F) ( ) ( F) > Proof. Let f be a lowest weight vector, i.e., f is a weight vector and Λ E− f 0. Now 0 C E E t ( ) = − + B 0 with B and C as in (2.2). Thus the lowest= weight( ) = vector  condition for f becomes

1 2 − ∂ f z + π 3z1z2f 0. π 2 √ k k Then the lowest degree term f0 of f satisfies( )( ∂)2fo 0 and must= be of the form f0 z1 or f0 z1 z2 up to non-zero constants. But these vectors have negative weights under Λ H and any unitary lowest weight irreducible representation of su 1=, 1 has positive lowest weight,= consequently= lowest weight representations do not appear. ( ) ( ) Next we shall find all highest weight vectors in the space . So let σν be an irreducible representation for the Lie algebra su 1, 1 appearing in Λ with highest weight vector f of weight −ν, ν 0. F Expand f as sum of homogeneous( polynomials) > ∞ f pm k, m=0 + with pm k of homogeneity m + k and k being= Q the lowest degree. The highest weight condition + Λ E+ f 0 becomes 1 π (3.7)( ) = B∂,∂ f Cz,z f, 2π 2 i.e. 1 ( π ) = ( ) B∂,∂ pm k Cz,z pm k 4, m 0, 1,.... 2π + 2 + − ( ) = ( ) = 12 GENKAI ZHANG

Thus pm k 4 determines pm k up to B-harmonic polynomials of degree m + k. Hence the sum + − + l∞=0 pk 4l in f satisfies the equation (3.7) and it starts with the lowest degree pk of f. But since + we are finding all solutions we can assume f l∞=0 pk 4l. To ease notation we write this as ∑ + f l∞=0 fl, fl pk 4l. + We have then = ∑ = ∑ = 3 2 ⋯ (3.8) 2 ∂1∂2fl 1 z2 fl,l 0, 1, , √π +

= = k and the leading term f0 satisfies ∂1∂2f0 0, namely 2 3∂1∂2f0 0. So f0 z1 , k 0 or k f0 z2 ,k 0. √ k = = = ≥ If f0 z2 then Λ H f0 −1 + k f0, it is of nonnegative weight −1 + k and this contradicts our= assumption> that f is of negative weight, so this case is excluded. = k ( ) = ( ) Let f0 z1 , k 0. The weight of f is ν −1 − 3k. We claim that f is uniquely solved by

l = ≥ = k ∞ I z (3.9) f z k!z1 . l=0 l! l + k ! ( ) ( ) = Q Now the action of U 1 ∶ ι U 1 on the space of( m of) homogeneous polynomials of degree m is multiplicity free for any m. Thus that fl is of weight −k − 1 and of degree k + 4l implies that ( ) = ( ( )) P l k fl alI z z1 .

Note that = ( )

2 3 l 1 k 3 π l 1 l k 1 3 l 1 ∂1∂2 I z + z1 + ∂1∂2 z1+ + z2( + ) √π2 √π2 3 3 2 ( ( ) ) = 3( π√ )l 1 ( ) l k 3l 2 + l + k + 1 3 l + 1 z1+ z2 + √π2 3 3 2 = π( √ 3) l ( ) ( 2)( ) z1z2 l + k + 1 l + 1 z2 3 3 2 l = (l +√k + 1 )l(+ 1 z2 I )(z . )

The recursive relation (3.8) becomes= ( )( ) ( ) 1 al 1 al + l + k + 1 l + 1 = and is solved by ( )( ) k! a l l! k + l ! = This proves our claim. ( ) BRANCHING OF METAPLECTIC REPRESENTATION OF Sp(2, R) UNDER ITS PRINCIPAL SL(2, R)-SUBGROUP13

Finally we prove that f if k 0. The square norm of f is 1 1 2 1 2 l k 2 + k! 2 2 ∈FI z1 > 2 2 l! l k ! l l l l! k + l ! l l! k + l ! 3 3 2 1 ( ) Q Y Y = Q l + k ! 3 l 3( l)( ) ( ) ( ) ( (( ) ) (( ) ) 2 l l! k + l ! ( 2) ( 1) ( ) 2 1 = Q 3 l 3 l Γ k + 1 Γ 3 + l Γ 3 + l k! (( ) ) k! 2 1 l 1 l k + l ! Γ 3 Γ 3 l Γ l + 1 Γ k + l + 1 ( ) ( ) ( ) ( ) ( ) = Q1 = Q k (1 ) ( ) ( ) ( ) ( ) ( ) l l + which is convergent iff k 1. Here≍ weQ have used the known fact that Γ a + l 1 ≥ b a . Γ b + l l − ( ) This finished the proof. ≍  ( ) We can now prove Theorem 3.2. Proof. It follows from Lemmas 3.3, 3.4 and 3.5 and the right hand side in (3.5) are subrepre- sentation of Λ, , SU 1, 1 . We have to prove that it gives full decomposition. The proof relies some general and abstract arguments. Suppose σ is an irreducible unitary representation of SU 1, 1 that( appearF discretely( )) in Λ, ,Mp 2, R . If σ is spherical then the unique spherical vector of σ is an eigenvector of C in 0. But C has no discrete spectrum in 0 by Lemma 3.4, thus(σ will) not appear. Similarly non-spherical( F ( principal)) series or complementary series will not appear discretely, by Lemma 3.3. FurthermoreF in Lemma 3.5 lowest weightF representations are excluded and all highest ones are found. Let ∆ σdσ be the continuous part of the decomposition, namely Let ∫ ∞ SU 1,1 σdσ ⊕ ⊕ σ 3k 1. ∆ ( ) k=1 − − This is also a spectral decompositionFS of= theS CasimirQ element. The eigenvalue of Casimir on 1 2 highest or lowest weight representations takes discrete eigenvalues 2 ν − ν, ν ±1, ±2, ⋯, thus they will not appear as a continuous spectra. The only discrete components are then σ 3k 1,k = − − 1, 2,...,. The weights in each σ 3k 1 are of the form −3k − 1 − 2m −3k − 1, m 0, 1, 2, ⋯. 0 1 − − = Thus the weight spaces , − are then orthogonal to the discrete component and contained in ≤ = the continuous part ∆ σdσ⊕. But the irreducible representations σ in the continuous part all are even or odd principal seriesF andF then each contains weight vectors 0 or 1, a. e. with respect to dσ. ∫ 0 1 0 1 Then spectra of C in ∆ σdσ are then the same as those on ∆ σdσ ∩ + − + − , which are given in Lemmas 3.3 and 3.4. This completes the proof.  ∫ ∫ (F F )=(F F ) The even part Mp 2, R , Λ+, of the metaplectic representation consists of even holo- + morphic functions, the corresponding discrete components correspond to the the highest weight vector (3.9) with k( 2l(even.) We haveF ) thus

= 14 GENKAI ZHANG

Corollary 3.6. The discrete components appearing in the restriction of the even part of the meta- plectic representation Λ+, ,Mp 2, R to SU 1, 1 are precisely the highest weight represen- + tations of weights −6l − 1, l 1. ( F ( )) ( ) Remark 3.7. Part of the decomposition≥ of the even part of the metaplectic representation Λ+ can also be found using its realization as reproducing kernel space. We give a very brief de- scription. The representation Λ+ can be realize [6] on the Siegel bounded symmetric domain D G K Sp 2, R U 2 as a Hilbert space of holomorphic function with reproducing 1 2 kernel det 1 − zw∗ − . It is a subspace of holomorphic sections of a line bundle over the Siegel domain= ~Sp =2, R (U 2)~. Consider( ) the the hyperbolicH disc ∆ SU 1, 1 U 1 z C; z 1 ( ) 2 2 1 and the Hilbert space L D, 1 − z − dm z with the natural G0 SU 1, 1 action as ( )~ ( ) = ( )~ ( )={ ∈ S S < } ∶ → + + + 1 (g G( 0 fS Sz ) f (az)) b cz d cz d=− . ( ) 1 2 C 2 This is also the L -space of∈ the half-bundle( ) (( dz )(of the cotangent))( ) bundle. The pull-back of ι, ∶ → 2 − 2 1 R ι∗ L D,( )1 z − dm z 2 − 2 1 defines an intertwining map R=to LH ∆, 1( z( − SdmS ) z .( The)) operator RR∗ is a Berezin transform on ∆ and its spectral symbol can be computed using the method in [11]. The spectral density of the decomposition involves( then( theS symbolS ) of( the))Berezin transform and the Harish- Chandra c-function [10]. However the restriction map R here is not injective and it will produce only the continuous part, and other discrete components can be constructed using covariant dif- ferentiation; this requires rather subtle and detailed computations. It is interesting to notice that the solution (3.9) for k 0 still defines a holomorphic function, which is a highest weight vector but not in the Fock space. This can also be explained by another fundamental fact about the Hardy space of holomorphic= functions on the unit disc. This space 2 2 1 L ∆, 1− z − dm z ,G0 has no discrete component. However the Hardy on the unit disc is 2 2 1 invariant under the same action but it is not a subspace L D, 1− z − dm z . So the solution ((3.9)( plays( theS S ) role of( a)) Hardy) space function whereas the restriction of the reproducing kernel 2 2 1 ( ( S S ) ( )) space to D plays the role of L D, 1 − z − dm z . It is also possible to solve (3.7) for general n 2 by using B-harmonic polynomials. The computations for the Fock space( norm( seemS S ) rather( demanding.)) ≥ We observe that the proof of Lemmas 3.3 and 3.4 actually also provides generalized eigen- functions for the Casimir operator as a sum of bi-orthogonal polynomials

∞ 2 Ψx z em z em x m=0 2 Wm z Wm x ( ) = Q ( )̃ ( ) where em z ( ) , em È ( ) . It is easy to prove that this series is convergent point-wise Wm Wm Y Y È for all xR and z C2. We haveY notY been able to compute the sum. There exist several formulas [7, ( ) = ̃ = 2 m 2 pp. 196-199] for generating functions m cmWm x t of the polynomials cmWm x however ∈ they are not related to the eigenfunctions Ψx z . ∑ È ( ) È ( ) ( ) BRANCHING OF METAPLECTIC REPRESENTATION OF Sp(2, R) UNDER ITS PRINCIPAL SL(2, R)-SUBGROUP15

It might be possible to find all discrete components for Λ, ,Mp n, R for general n 3 by refining the techniques in Lemma 3.5. It would also be an interesting problem to study the branching problems of minimal representations for other sp( litF real simple( )) Lie groups, such≥ as quaternonic representations of G2, under its principal SL 2, R -subgroup.

REFERENCES ( ) [1] M. Burger, A. Iozzi and A. Wienhard, Tight homomorphisms and Hermitian symmetric spaces , Geom. Funct. Anal. 19 (2009), no. 3, 678-721. pages 7 [2] B. Kostant, The principal three-dimensional subgroup and the Betti numbers of a complex simple Lie group, Amer. J. Math. 81 (1959), 973-1032. pages 1, 7 [3] J. Faraut and A. Koranyi, Analysis on symmetric cones, Oxford University Press, Oxford, 1994. pages 2 [4] J. Faraut and A. Koranyi, Function spaces and reproducing kernels on bounded symmetric domains, J. Funct. Anal. 88 (1990), 64–89. pages 2 [5] Gerald B. Folland, Harmonic analysis in phase space, Annals of Mathematics Studies, vol. 122, Princeton University Press, Princeton, NJ, 1989. pages 2, 3 [6] M. Kashiwara and M. Vergne, On the Segal-Shale-Weil representation and harmonic polynomials, 44 (1978), Inven. Math., 1-47. pages 3, 14 [7] R. Koekoek, P. Lesky and R. Swarttouw, Hypergeometric orthogonal polynomials and their q-analogues, Springer, Monographs in Mathematics, 2010. pages 5, 14 [8] O. Loos, Bounded symmetric domains and Jordan pairs, University of California, Irvine, 1977. pages 2 [9] B. Ørsted and J. Vargas, Branching problems for semisimple Lie groups and reproducing kernels, C. R. Math. Acad. Sci. Paris 357 (2019), no. 9, 697-707. pages 2 [10] G. Zhang, Branching coefficients of holomorphic representations and Segal-Bargmann transform. J. Funct. Anal. 195 (2002), no. 2, 306-349. pages 1, 14 [11] , Berezin transform on real bounded symmetric domains, Trans. Amer. Math. Soc. 353 (2001), 3769– 3787. pages 1, 14 [12] , Tensor products of minimal holomorphic representations. Represent. Theory 5 (2001), 164-190. pages 1

MATHEMATICAL SCIENCES, CHALMERS UNIVERSITY OF TECHNOLOGY AND MATHEMATICAL SCIENCES, GOTEBORG¨ UNIVERSITY, SE-412 96 GOTEBORG¨ , SWEDEN Email address: [email protected]