MATRIX COEFFICIENTS, COUNTING AND PRIMES FOR ORBITS OF GEOMETRICALLY FINITE GROUPS
AMIR MOHAMMADI AND HEE OH
Abstract. Let G := SO(n, 1)◦ and Γ < G be a geometrically finite Zariski dense subgroup with critical exponent δ bigger than (n − 1)/2. Under a spectral gap hypothesis on L2(Γ\G), which is always satisfied when δ > (n − 1)/2 for n = 2, 3 and when δ > n − 2 for n ≥ 4, we obtain an effective archimedean counting result for a discrete orbit of Γ in a homogeneous space H\G where H is the trivial group, a symmetric subgroup or a horospherical subgroup. More precisely, we show that for any effectively well-rounded family {BT ⊂ H\G} of compact subsets, there exists η > 0 such that 1−η #[e]Γ ∩ BT = M(BT ) + O(M(BT ) ) for an explicit measure M on H\G which depends on Γ. We also apply the affine sieve and describe the distribution of almost primes on orbits of Γ in arithmetic settings. One of key ingredients in our approach is an effective asymptotic for- mula for the matrix coefficients of L2(Γ\G) that we prove by combining methods from spectral analysis, harmonic analysis and ergodic theory. We also prove exponential mixing of the frame flows with respect to the Bowen-Margulis-Sullivan measure.
Contents 1. Introduction 1 2. Matrix coefficients in L2(Γ\G) by ergodic methods 11 3. Asymptotic expansion of Matrix coefficients 12 4. Non-wandering component of Γ\ΓHat as t → ∞ 26 5. Translates of a compact piece of Γ\ΓH via thickening 32 6. Distribution of Γ\ΓHat and Transversal intersections 37 7. Effective uniform counting 45 8. Affine sieve 57 References 61
The authors are supported in part by NSF grants. 1 2 AMIR MOHAMMADI AND HEE OH
1. Introduction Let n ≥ 2 and let G be the identity component of the special orthogonal group SO(n, 1). As well known, G can be considered as the group of orienta- n tion preserving isometries of the hyperbolic space H . A discrete subgroup Γ of G is called geometrically finite if the unit neighborhood of its convex core1 has finite Riemannian volume. As any discrete subgroup admitting a finite n sided polyhedron as a fundamental domain in H is geometrically finite, this class of discrete subgroups provides a natural generalization of lattices in G. In particular, for n = 2, a discrete subgroup of G is geometrically finite if and only if it is finitely generated. In the whole introduction, let Γ be a torsion-free geometrically finite, Zariski dense, discrete subgroup of G. We denote by δ the critical exponent of Γ. Note that any discrete subgroup of G with δ > (n − 2) is Zariski dense in G. The main aim of this paper is to obtain effective counting results for discrete orbits of Γ in H\G, where H is the trivial group, a symmetric subgroup or a horospherical subgroup of G, and to discuss their applications in the affine sieve on Γ-orbits in an arithmetic setting. Our results are formulated under a suitable spectral gap hypothesis for L2(Γ\G) (see Def. 1.1 and 1.3). This hypothesis on Γ is known to be true if the critical exponent δ is strictly bigger than n − 2. Though we believe that the condition δ > (n − 1)/2 should be sufficient to guarantee this hypothesis, it is not yet known in general (see 1.2). For Γ lattices, i.e., when δ = n − 1, both the effective counting and applications to an affine sieve have been extensively studied (see [16], [17], [4], [44], [21], [42],[48], [20], etc. as well as survey articles [51], [49] [36], [37]). Hence our main focus is when Γ is of infinite co-volume in G.
1.1. Effective asymptotic of Matrix coefficients for L2(Γ\G). We be- gin by describing an effective asymptotic result on the matrix coefficients for L2(Γ\G), which is a key ingredient in our approach as well as of independent interest. When Γ is not a lattice, a well-known theorem of Howe and Moore 2 [27] implies that for any Ψ1, Ψ2 ∈ L (Γ\G), the matrix coefficient Z haΨ1, Ψ2i := Ψ1(ga)Ψ2(g)dg Γ\G decays to zero as a ∈ G tends to infinity (here dg is a G-invariant measure on Γ\G). Describing the precise asymptotic is much more involved. Fix a Cartan decomposition G = KAK where K is a maximal compact subgroup and A is a one-parameter subgroup of diagonalizable elements. Let M de- note the centralizer of A in K. The quotient spaces G/K and G/M can n 1 n be respectively identified with H and its unit tangent bundle T (H ), and
1 n n The convex core CΓ ⊂ Γ\H of Γ is the image of the minimal convex subset of H which contains all geodesics connecting any two points in the limit set of Γ. EFFECTIVE COUNTING 3 we parameterize elements of A = {at : t ∈ R} so that the right translation 1 n action of at in G/M corresponds to the geodesic flow on T (H ) for time t. n n We let {mx : x ∈ H } and {νx : x ∈ H } be Γ-invariant conformal densi- ties of dimensions (n − 1) and δ respectively, unique up to scalings. Each νx is a finite measure on the limit set of Γ, called the Patterson-Sullivan mea- sure viewed from x. Let mBMS, mBR, mBR∗ and mHaar denote, respectively, the Bowen-Margulis-Sullivan measure, the Burger-Roblin measures for the expanding and the contracting horospherical foliations, and the Liouville- 1 n measure on the unit tangent bundle T (Γ\H ), all defined with respect to the fixed pair of {mx} and {νx} (see Def. 2.1). Using the identification 1 n T (Γ\H ) = Γ\G/M, we may extend these measures to right M-invariant measures on Γ\G, which we will denote by the same notation and call them the BMS, the BR, the BR∗, the Haar measures for simplicity. We note that for δ < n − 1, only the BMS measure has finite mass [52]. In order to formulate a notion of a spectral gap for L2(Γ\G), denote by Gˆ and Mˆ the unitary dual of G and M respectively. A representation (π, H) ∈ Gˆ is called tempered if for any K-finite v ∈ H, the associated matrix coefficient function g 7→ hπ(g)v, vi belongs to L2+(G) for any > 0; non-tempered otherwise. The non-tempered part of Gˆ consists of the trivial representation, and complementary series representations U(υ, s − n + 1) ˆ n−1 parameterized by υ ∈ M and s ∈ Iυ, where Iυ ⊂ ( 2 , n − 1) is an interval depending on υ. This was obtained by Hirai [26] (see also [30, Prop. 49, 50]). Moreover U(υ, s−n+1) is spherical (i.e., has a non-zero K-invariant vector) if and only if υ is the trivial representation 1; see discussion in section 3.2. By the works of Lax-Phillips [40], Patterson [54] and Sullivan [62], if n−1 2 δ > 2 , U(1, δ − n + 1) occurs as a subrepresentation of L (Γ\G) with multiplicity one, and L2(Γ\G) possesses spherical spectral gap, meaning n−1 2 2 that there exists 2 < s0 < δ such that L (Γ\G) does not weakly contain any spherical complementary series representation U(1, s−n+1), s ∈ (s0, δ). The following notion of a spectral gap concerns both the spherical and non- spherical parts of L2(Γ\G).
Definition 1.1. We say that L2(Γ\G) has a strong spectral gap if (1) L2(Γ\G) does not contain any U(υ, δ − n + 1) with υ 6= 1; n−1 2 (2) there exist 2 < s0(Γ) < δ such that L (Γ\G) does not weakly contain any U(υ, s − n + 1) with s ∈ (s0(Γ), δ) and υ ∈ Mˆ .
n−1 2 n For δ ≤ 2 , the Laplacian spectrum of L (Γ\H ) is continuous [40]; this implies that there is no spectral gap for L2(Γ\G).
2for two unitary representations π and π0 of G, π is said to be weakly contained in π0 (or π0 weakly contains π) if every diagonal matrix coefficient of π can be approximated, uniformly on compact subsets, by convex combinations of diagonal matrix coefficients of π0. 4 AMIR MOHAMMADI AND HEE OH
Conjecture 1.2 (Spectral gap conjecture). If Γ is a geometrically finite and n−1 2 Zariski dense subgroup of G with δ > 2 , L (Γ\G) has a strong spectral gap. If δ > (n − 1)/2 for n = 2, 3, or if δ > (n − 2) for n ≥ 4, then L2(Γ\G) has a strong spectral gap (Theorem 3.27). Our main theorems are proved under the following slightly weaker spectral gap property assumption: 2 n−1 Definition 1.3. We say that L (Γ\G) has a spectral gap if there exist 2 < s0 = s0(Γ) < δ and n0 = n0(Γ) ∈ N such that (1) the multiplicity of U(υ, δ − n + 1) contained in L2(Γ\G) is at most dim(υ)n0 for any υ ∈ Mˆ ; 2 (2) L (Γ\G) does not weakly contain any U(υ, s−n+1) with s ∈ (s0, δ) and υ ∈ Mˆ .
The pair (s0(Γ), n0(Γ)) will be referred to as the spectral gap data for Γ. In the rest of the introduction, we impose the following hypothesis on Γ: L2(Γ\G) has a spectral gap.
Theorem 1.4. There exist η0 > 0 and ` ∈ N (depending only on the spectral ∞ gap data for Γ) such that for any real-valued Ψ1, Ψ2 ∈ Cc (Γ\G), as t → ∞, Z (n−1−δ)t Haar e Ψ1(gat)Ψ2(g)dm (g) Γ\G mBR(Ψ ) · mBR∗ (Ψ ) = 1 2 + O(S (Ψ )S (Ψ )e−η0t) |mBMS| ` 1 ` 2 2 where S`(Ψi) denotes the `-th L -Sobolev norm of Ψi for each i = 1, 2.
Remark 1.5. We remark that if either Ψ1 or Ψ2 is K-invariant, then The- n−1 orem 1.4 holds for any Zariski dense Γ with δ > 2 (without the spectral gap hypothesis), as the spherical spectral gap of L2(Γ\G) is sufficient to study the matrix coefficients associated to spherical vectors. † Let Hδ denote the sum of of all complementary series representations of 2 parameter δ contained in L (Γ\G), and let Pδ denote the projection operator 2 † 2 from L (Γ\G) to Hδ. By the spectral gap hypothesis on L (Γ\G), the main work in the proof of Theorem 1.4 is to understand the asymptotic of hatPδ(Ψ1),Pδ(Ψ2)i as t → ∞. Building up on the work of Harish-Chandra on the asymptotic behavior of the Eisenstein integrals (cf. [65], [66]), we first † obtain an asymptotic formula for hatv, wi for all K-finite vectors v, w ∈ Hδ (Theorem 3.23). This extension alone does not give the formula of the leading term of hatPδ(Ψ1),Pδ(Ψ2)i in terms of functions Ψ1 and Ψ2; however, an ergodic theorem of Roblin [56] and Winter [67] enables us to identify the main term as given in Theorem 1.4. EFFECTIVE COUNTING 5
1.2. Exponential mixing of frame flows. Via the identification of the n space Γ\G with the frame bundle over the hyperbolic manifold Γ\H , the right translation action of at on Γ\G corresponds to the frame flow for time t. The BMS measure mBMS on Γ\G is known to be mixing for the frame flows ([18], [67]). We deduce the following exponential mixing from Theorem 1.4: for a compact subset Ω of Γ\G, we denote by C∞(Ω) the set of all smooth functions on Γ\G with support contained in Ω.
Theorem 1.6. There exist η0 > 0 and ` ∈ N such that for any compact ∞ subset Ω ⊂ Γ\G, and for any Ψ1, Ψ2 ∈ C (Ω), as t → ∞, Z BMS Ψ1(gat)Ψ2(g)dm (g) Γ\G mBMS(Ψ ) · mBMS(Ψ ) = 1 2 + O(S (Ψ )S (Ψ )e−η0t) |mBMS| ` 1 ` 2 where the implied constant depends only on Ω.
For Γ convex co-compact, Theorem 1.6 for Ψ1 and Ψ2 M-invariant func- tions holds for any δ > 0 by Stoyanov [61], based on the approach developed by Dolgopyat [14]; however when Γ has cusps, this theorem seems to be new even for n = 2.
1.3. Effective equidistribution of orthogonal translates of an H- orbit. When H is a horospherical subgroup or a symmetric subgroup of G, we can relate the asymptotic distribution of orthogonal translates of a closed orbit Γ\ΓH to the matrix coefficients of L2(Γ\G). We fix a gener- alized Cartan decomposition G = HAK. We parameterize A = {at} as in section 1.1, and for H horospherical, we will assume that H is the expanding horospherical subgroup for at, that is, H = {g ∈ G : atga−t → e as t → ∞}. Haar PS Let µH and µH be respectively the H-invariant measure on Γ\ΓH de- fined with respect to {mx} and the skinning measure on Γ\ΓH defined with respect to {νx}, introduced in [52] (cf. (4.2)). PS Theorem 1.7. Suppose that Γ\ΓH is closed and that |µH | < ∞. There exist η0 > 0 and ` ∈ N such that for any compact subset Ω ⊂ Γ\G, any Ψ ∈ C∞(Ω) and any bounded φ ∈ C∞(Γ ∩ H\H), as t → ∞, Z (n−1−δ)t Haar e Ψ(hat)φ(h)dµH (h) h∈Γ\ΓH 1 = µPS(φ)mBR(Ψ) + O(S (Ψ) · S (φ)e−η0t) |mBMS| H ` ` with the implied constant depending only on Ω. PS For H horospherical, |µH | < ∞ is automatic for Γ\ΓH closed. For H symmetric (and hence locally isomorphic to SO(k, 1) × SO(n − k)), the cri- PS terion for the finiteness of µH has been obtained in [52] (see Prop. 4.15); PS in particular, |µH | < ∞ provided δ > n − k. 6 AMIR MOHAMMADI AND HEE OH
Letting YΩ := {h ∈ (Γ ∩ H)\H : hat ∈ Ω for some t > 0}, note that Z Z Haar Haar Ψ(hat)φ(h)dµH = Ψ(hat)φ(h)dµH YΩ PS since Ψ is supported in Ω. In the case when µH is compactly supported, YΩ turns out to be a compact subset and in this case, the so-called thickening method ([17], [29]) is sufficient to deduce Theorem 1.7 from Theorem 1.4, using the wave front property introduced in [17] (see [4] for the effective PS version). The case of µH not compactly supported is much more intricate to be dealt with. Though we obtain a thick-thin decomposition of YΩ with the thick part being compact and control both the Haar measure and the skinning measure of the thin part (Theorem 4.16), the usual method of thickening the thick part does not suffice, as the error term coming from the thin part overtakes the leading term. The main reason for this phe- Haar nomenon is because we are taking the integral with respect to µH as well as multiplying the weight factor e(n−1−δ)t in the left hand side of Theorem PS 1.7, whereas the finiteness assumption is made on the skinning measure µH . PS However we are able to proceed by comparing the two measures (at)∗µH Haar and (at)∗µH via the the transversal intersections of the orbits Γ\ΓHat with the weak-stable horospherical foliations (see the proof of Theorem 6.9 for more details). In the special case of n = 2, 3 and H horospherical, Theorem 1.7 was proved in [35], [34] and [41] by a different method.
1.4. Effective counting for a discrete Γ-orbit in H\G. In this subsec- tion, we let H be the trivial group, a horospherical subgroup or a symmetric subgroup, and assume that the orbit [e]Γ is discrete in H\G. Theorems 1.4 and 1.7 are key ingredients in understanding the asymptotic of the number #([e]Γ ∩ BT ) for a given family {BT ⊂ H\G} of growing compact subsets, as observed in [16]. Γ We will first describe a Borel measure MH\G = MH \G on H\G, depend- n ing on Γ, which turns out to describe the distribution of [e]Γ. Let o ∈ H be 1 n + − n the point fixed by K, X0 ∈ T (H ) the vector fixed by M, X0 ,X0 ∈ ∂(H ) the forward and the backward endpoints of X0 by the geodesic flow, respec- n tively and νo the Patterson-Sullivan measure on ∂(H ) supported on the limit set of Γ, viewed from o. Let dm denote the probability Haar measure of M. Definition 1.8. For H the trivial subgroup {e}, define a Borel measure Γ MG = MG on G as follows: for ψ ∈ Cc(G),
MG(ψ) := Z 1 δt + −1 − |mBMS| ψ(k1atmk2)e dνo(k1X0 )dtdmdνo(k2 X0 ). (K/M)×A+×M×(M\K) EFFECTIVE COUNTING 7
Definition 1.9. For H horospherical or symmetric, we have either G = HA+K or G = HA+K ∪ HA−K (as a disjoint union except for the identity ± element) where A = {a±t : t ≥ 0}. Γ Define a Borel measure MH\G = MH\G on H\G as follows: for ψ ∈ Cc(H\G),
MH\G(ψ) := PS |µH | R δt −1 − + |mBMS| A+×M×(M\K) ψ([e]atmk)e dtdmdνo(k X0 ) if G = HA K |µPS | P H,± R δt −1 ∓ |mBMS| A±×M×(M\K) ψ([e]a±tmk)e dtdmdνo(k X0 ) otherwise, PS where µH,− is the skinning measure on Γ ∩ H\H in the negative direction, as defined in (6.15).
Definition 1.10. For a family {BT ⊂ H\G} of compact subsets with MH\G(BT ) tending to infinity as T → ∞, we say that {BT } is effectively well-rounded with respect to Γ if there exists p > 0 such that for all small > 0 and large T 1: + − p MH\G(BT, − BT,) = O( ·MH\G(BT )) + − + where BT, = GBT G and BT, = ∩g1,g2∈G g1BT g2 if H = {e}; and BT, = − BT G and BT, = ∩g∈G BT g if H is horospherical or symmetric. Here G denotes a symmetric -neighborhood of e in G with respect to a left invariant Riemannian metric on G. Since any two left-invariant Riemannian metrics on G are Lipschitz equiv- alent to each other, the above definition is independent of the choice of a Riemannian metric used in the definition of G. See Propositions 7.11, 7.15 and 7.17 for examples of effectively well- rounded families. For instance, if G acts linearly from the right on a finite dimensional linear space V and H is the stabilizer of w0 ∈ V , then the family of norm balls BT := {Hg ∈ H\G : kw0gk < T } is effectively well-rounded. If Γ is a lattice in G, then MH\G is essentially the leading term of the invariant measure in H\G and hence the definition 1.10 is equivalent to the one given in [4], which is an effective version of the well-roundedness condition given in [17]. Under the additional assumption that H ∩ Γ is a lattice in H, it is known that if {BT } is effectively well-rounded, then
1−η0 #([e]Γ ∩ BT ) = Vol(BT ) + O(Vol(BT ) ) (1.11) for some η0 > 0, where Vol is computed with respect to a suitably normalized invariant measure on H\G (cf. [16], [17], [44], [20], [4]). We present a generalization of (1.11). In the next two theorems 1.12 and 1.14, we let {Γd : d ∈ I} be a family of subgroups of Γ of finite index such that Γd ∩ H = Γ ∩ H. We assume that {Γd : d ∈ I} has a uniform spectral gap in the sense that supd s0(Γd) < δ and supd n0(Γd) < ∞. 8 AMIR MOHAMMADI AND HEE OH
For our intended application to the affine sieve, we formulate our effective results uniformly for all Γd’s. Theorem 1.12. Let H be the trivial group, a horospherical subgroup or a PS symmetric subgroup. When H is symmetric, we also assume that |µH | < ∞. If {BT } is effectively well-rounded with respect to Γ, then there exists η0 > 0 such that for any d ∈ I and for any γ0 ∈ Γ,
1 1−η0 #([e]Γdγ0 ∩ BT ) = M (BT ) + O(M (BT ) ) [Γ:Γd] H\G H\G Γ where MH\G = MH\G and the implied constant is independent of Γd and γ0 ∈ Γ. See Corollaries 7.16 and 7.18 where we have applied Theorem 1.12 to sectors and norm balls. Remark 1.13. Theorem 1.12 can be used to provide an effective version of circle-counting theorems studied in [35], [41], [53] and [50] (as well as its higher dimensional analogues for sphere packings discussed in [51]). We also formulate our counting statements for bisectors in the HAK decomposition, motivated by recent applications in [8] and [9]. Let τ1 ∈ ∞ ∞ τ1,τ2 ∞ Cc (H) and τ2 ∈ C (K), and define ξT ∈ C (G) as follows: for g = hak ∈ HA+K, Z τ1,τ2 −1 ξT (g) = χA+ (a) · τ1(hm)τ2(m k)dH∩M (m) T H∩M + where χ + denotes the characteristic function of A = {at : 0 < t < log T } AT T 0 0 and dH∩M is the probability Haar measure of H ∩ M. Since hak = h ak 0 −1 0 τ1,τ2 implies that h = h m and k = m k for some m ∈ H ∩ M, ξT is well- defined.
Theorem 1.14. There exist η0 > 0 and ` ∈ N such that for any compact ∞ ∞ subset H0 of H which injects to Γ\G, any τ1 ∈ C (H0), τ2 ∈ C (K), any γ0 ∈ Γ and any d ∈ I, µ˜PS(τ ) · ν∗(τ ) X τ1,τ2 H 1 o 2 δ δ−η0 ξT (γ) = BMS T + O(S`(τ1)S`(τ2)T ) δ · [Γ : Γd] · |m | γ∈Γdγ0 ∗ R R −1 − PS where νo (τ2) = K M τ2(mk)dmdνo(k X0 ) and µ˜H is the skinning mea- sure on H with respect to Γ and the implied constant depends only on Γ and H0.
Theorem 1.14 also holds when τ1 and τ2 are characteristic functions of the so-called admissible subsets (see Corollary 7.21 and Proposition 7.11). We remark that unless H = K, Corollary 7.16, which is a special case of Theorem 1.12 for sectors in H\G, does not follow from Theorem 1.14, as the latter deals only with compactly supported functions τ1. For H = K, Theorem 1.14 was earlier shown in [7] and [64] for n = 2, 3 respectively. EFFECTIVE COUNTING 9
Remark 1.15. Non-effective versions of Theorems 1.7, 1.12, and 1.14 were obtained in [52] for a more general class of discrete groups, that is, any non-elementary discrete subgroup admitting finite BMS-measure.
1.5. Application to Affine sieve. One of the main applications of Theo- rem 1.12 can be found in connection with Diophantine problems on orbits of Γ. Let G be a Q-form of G, that is, G is a connected algebraic group ◦ defined over Q such that G = G(R) . Let G act on an affine space V via a Q-rational representation in a way that G(Z) preserves V (Z). Fix a non-zero vector w0 ∈ V (Z) and denote by H its stabilizer subgroup and set H = H(R). We consider one of the following situations: (1) H is a symmet- ric subgroup of G or the trivial subgroup; (2) w0G ∪ {0} is Zariski closed and H is a compact extension of a horospherical subgroup of G. In the case (1), w0G is automatically Zariski closed by [23]. Set W := w0G and w0G ∪ {0} respectively for (1) and (2). Let Γ be a geometrically finite and Zariski dense subgroup of G with n−1 δ > 2 , which is contained in G(Z). If H is symmetric, we assume that PS |µH | < ∞. For a positive integer d, we denote by Γd the congruence subgroup of Γ which consists of γ ∈ Γ such that γ ≡ e mod d. For the next two theorems 1.16 and 1.17 we assume that there exists a finite set S of primes that the family {Γd : d is square-free with no prime factors in S} has a uniform spectral gap. This property always holds if δ > (n − 1)/2 for n = 2, 3 and if δ > n − 2 for n ≥ 4 via the recent works of Bourgain,Gamburd, Sarnak ([6], [5]) and Salehi-Golsefidy and Varju [58] together with the classification of the unitary dual of G (see Theorem 8.2). Let F ∈ Q[W ] be an integer-valued polynomial on the orbit w0Γ. Salehi- Golsefidy and Sarnak [57], generalizing [6], showed that for some R > 1, the set of x ∈ w0Γ with F (x) having at most R prime factors is Zariski dense in w0G. The following are quantitative versions: Letting F = F1F2 ··· Fr be a factorization into irreducible polynomials in Q[W ], assume that all Fj’s are irreducible in C[W ] and integral on w0Γ. Let {BT ⊂ w0G : T 1} be an effectively well-rounded family of subsets with respect to Γ. Theorem 1.16 (Upper bound for primes). For all T 1,
Mw0G(BT ) {x ∈ w0Γ ∩ BT : Fj(x) is prime for j = 1, ··· , r} r . (log Mw0G(BT )) Theorem 1.17 (Lower bound for almost primes). Assume further that β maxx∈BT kxk Mw0G(BT ) for some β > 0, where k · k is any norm on V . Then there exists R = R(F, w0Γ, β) ≥ 1 such that for all T 1,
Mw0G(BT ) {x ∈ w0Γ ∩ BT : F (x) has at most R prime factors} r . (log Mw0G(BT )) 10 AMIR MOHAMMADI AND HEE OH
Observe that these theorems provide a description of the asymptotic dis- tribution of almost prime vectors, as BT can be taken arbitrarily.
Remark 1.18. In both theorems above, if all BT are K-invariant subsets, our hypothesis on the uniform spectral gap for the family {Γd} can be dis- posed again, as the uniform spherical spectral gap property proved in [58] and [6] is sufficient in this case. For instance, Theorems 1.16 and 1.17 can be applied to the norm balls δ/λ BT = {x ∈ w0G : kxk < T } and in this case Mw0G(BT ) T where λ denotes the log of the largest eigenvalue of a1 on the R-span of w0G. In order to present a concrete example, we consider an integral quadratic form Q(x1, ··· , xn+1) of signature (n, 1) and for an integer s ∈ Z, denote by WQ,s the affine quadric given by {x : Q(x) = s}.
As well-known, WQ,s is a one-sheeted hyperboloid if s > 0, a two-sheeted hyperboloid if s < 0 and a cone if s = 0. We will assume that Q(x) = s has a non-zero integral solution, so pick w0 ∈ WQ,s(Z). If s 6= 0, the stabilizer subgroup Gw0 is symmetric; more precisely, locally isomorphic to
SO(n − 1, 1) (if s > 0) or SO(n) (if s < 0) and if s = 0, Gw0 is a compact extension of a horospherical subgroup. By the remark following Theorem PS 1.7, the skinning measure µ is finite if n ≥ 3. For n = 2 and s > 0, Gw0 is Gw0 a one-dimensional subgroup consisting of diagonalizable elements, and µPS Gw0 2 is infinite only when the geodesic in H stabilized by Gw0 is divergent and 2 goes into a cusp of a fundamental domain of Γ in H ; in this case, we call w0 externally Γ-parabolic, following [52]. Therefore the following are special cases of Theorems 1.16 and 1.17: Corollary 1.19. Let Γ be a geometrically finite and Zariski dense subgroup n−1 of SOQ(Z) with δ > 2 . In the case when n = 2 and s > 0, we additionally assume that w0 is not externally Γ-parabolic. Fixing a K-invariant norm n+1 k · k on R , we have, for any 1 ≤ r ≤ n + 1, T δ (1) {x ∈ w0Γ: kxk < T, xj is prime for all j = 1, ··· , r} (log T )r ; (2) for some R > 1, T δ {x ∈ w Γ: kxk < T, x ··· x has at most R prime factors} . 0 1 r (log T )r The upper bound in (1) is sharp up to a multiplicative constant. The lower bound in (2) can also be stated for admissible sectors under the uniform spectral gap hypothesis (cf. Corollary 7.18). Corollary 1.19 was previously obtained in cases when n = 2, 3 and s ≤ 0 ([5], [33], [35],[34], [41]). To explain how Theorems 1.16 and 1.17 follow from Theorem 1.12, let
Γw0 (d) = {γ ∈ Γ: w0γ = w0 mod (d)} for each square-free integer d. Then Stab (w ) = Stab (w ) and the family {Γ (d)} admits a uni- Γw0 (d) 0 Γ 0 w0 form spectral gap property as Γd < Γw0 (d). Hence Theorem 1.12 holds EFFECTIVE COUNTING 11
for the congruence family {Γw0 (d): d is square-free, with no small primes}, providing a key axiomatic condition in executing the combinatorial sieve (see [28, 6.1-6.4], [25, Theorem 7.4], as well as [6, Sec. 3]). When an ex- plicit uniform spectral gap for {Γd} is known (e.g., [19], [43]), the number R(F, w0Γ) can also be explicitly computed in principle. The paper is organized as follows. In section 2, we recall the ergodic result of Roblin which gives the leading term of the matrix coefficients for L2(Γ\G). In section 3, we obtain an effective asymptotic expansion for the matrix coefficients of the complementary series representations of G (Theorem 3.23) as well as for those of L2(Γ\G), proving Theorem 1.4. In section 4, we study the reduction theory for the non-wandering component of Γ\ΓHat, describing its thick-thin decomposition; this is needed as Γ\ΓH has infinite Haar-volume in general. We will see that the non-trivial dynamics of Γ\ΓHat as t → ∞ can be seen only within a subset of finite PS-measure. In section 5, for φ compactly supported, we prove Theorem 1.7 using Theorem 1.4 via thickening. For a general bounded φ, Theorem 1.7 is obtained via a careful study of the transversal intersections in section 6. Theorem 1.6 is also proved in section 6. Counting theorems 1.12 and 1.14 are proved in section 7 and Sieve theorems 1.16 and 1.17 are proved in the final section 8.
2. Matrix coefficients in L2(Γ\G) by ergodic methods ◦ + n Throughout the paper, let G be SO(n, 1) = Isom (H ) for n ≥ 2, i.e., n the group of orientation preserving isometries of (H , d), and Γ < G be a n non-elementary torsion-free geometrically finite group. Let ∂(H ) denote n n the geometric boundary of H . Let Λ(Γ) ⊂ ∂(H ) denote the limit set of Γ, and δ the critical exponent of Γ, which is known to be equal to the Hausdorff dimension of Λ(Γ) [63]. n A family of measures {µx : x ∈ H } is called a Γ-invariant conformal n density of dimension δµ > 0 on ∂(H ), if each µx is a non-zero finite Borel n n n measure on ∂(H ) satisfying for any x, y ∈ H , ξ ∈ ∂(H ) and γ ∈ Γ, dµ y −δµβξ(y,x) γ∗µx = µγx and (ξ) = e , dµx −1 n where γ∗µx(F ) = µx(γ (F )) for any Borel subset F of ∂(H ). Here βξ(y, x) denotes the Busemann function: βξ(y, x) = limt→∞ d(ξt, y) − d(ξt, x) where ξt is a geodesic ray tending to ξ as t → ∞. We denote by {νx} the Patterson-Sullivan density, i.e., a Γ-invariant con- n formal density of dimension δ and by {mx : x ∈ H } a Lebesgue density, n i.e., a G-invariant conformal density on the boundary ∂(H ) of dimension (n − 1). Both densities are determined unique up to scalar multiples. t 1 n 1 n Denote by {G : t ∈ R} the geodesic flow on T (H ). For u ∈ T (H ), ± n we denote by u ∈ ∂(H ) the forward and the backward endpoints of the ± t n geodesic determined by u, i.e., u = limt→±∞ G (u). Fixing o ∈ H , the 12 AMIR MOHAMMADI AND HEE OH map + − u 7→ (u , u , s = βu− (o, π(u))) 1 n is a homeomorphism between T (H ) with n n n (∂(H ) × ∂(H ) − {(ξ, ξ): ξ ∈ ∂(H )}) × R. Using this homeomorphism, we define measuresm ˜ BMS, m˜ BR, m˜ BR∗ ,m ˜ Haar 1 n on T (H ) as follows ([11], [45], [63], [12], [56]): Definition 2.1. Set BMS δβ (o,π(u)) δβ (o,π(u)) + − (1) dm˜ (u) = e u+ e u− dνo(u )dνo(u )ds; BR (n−1)β (o,π(u)) δβ (o,π(u)) + − (2) dm˜ (u) = e u+ e u− dmo(u )dνo(u )ds; BR∗ δβ (o,π(u)) (n−1)β (o,π(u)) + − (3) dm˜ (u) = e u+ e u− dνo(u )dmo(u )ds; Haar (n−1)β (o,π(u)) (n−1)β (o,π(u)) + − (4) dm˜ (u) = e u+ e u− dmo(u )dmo(u )ds.
The conformal properties of {νx} and {mx} imply that these definitions n are independent of the choice of o ∈ H . We will extend these measures to n 1 n G; these extensions depend on the choice of o ∈ H and X0 ∈ To(H ). Let n 1 n K := StabG(o) and M := StabG(X0), so that H ' G/K and T (H ) ' G/M. Let A = {at : t ∈ R} be the one-parameter subgroup of diagonalizable t elements in the centralizer of M in G such that G (X0) = [M]at = [atM]. 1 n Using the identification T (H ) with G/M, we lift the above measures to G, which will be denoted by the same notation by abuse of notation, so that they are all invariant under M from the right. These measures are all left Γ-invariant, and hence induce locally finite Borel measures on Γ\G, which we denote by mBMS (the BMS-measure), mBR BR∗ Haar (the BR-measure), m (the BR∗ measure), m (the Haar measure) by abuse of notation. Let N + and N − denote the expanding and the contracting horospherical subgroups, i.e., ± N = {g ∈ G : atga−t → e as t → ±∞}. For g ∈ G, define ± ± n g := (gM) ∈ ∂(H ). We note that mBMS, mBR, and mBR∗ are invariant under A, N + and N − respectively and their supports are given respectively by {g ∈ Γ\G : g+, g− ∈ Λ(Γ)}, {g ∈ Γ\G : g− ∈ Λ(Γ)} and {g ∈ Γ\G : g+ ∈ Λ(Γ)}. The measure mHaar is invariant under both N + and N −, and hence under G, as N + and N − generate G topologically. That is, mHaar is a Haar measure of G. We consider the action of G on L2(Γ\G, mHaar) by right translations, which gives rise to the unitary action for the inner product: Z Haar hΨ1, Ψ2i = Ψ1(g)Ψ2(g)dm (g). Γ\G EFFECTIVE COUNTING 13
Theorem 2.2. Let Γ be Zariski dense. For any functions Ψ1, Ψ2 ∈ Cc(Γ\G),
BR BR∗ (n−1−δ)t m (Ψ1) · m (Ψ2) lim e hatΨ1, Ψ2i = . t→∞ |mBMS|
Proof. Roblin [56] proved this for M-invariant functions Ψ1 and Ψ2. His 1 n proof is based on the mixing of the geodesic flow on T (Γ\H ) = Γ\G/M. For Γ Zariski dense, the mixing of mBMS was extended to the frame flow on Γ\G, by [67]. Based on this, the proof given in [56] can be repeated verbatim to prove the claim (cf. [67]). 3. Asymptotic expansion of Matrix coefficients 3.1. Unitary dual of G. Let G = SO(n, 1)◦ for n ≥ 2 and K a maximal compact subgroup of G. Denoting by g and k the Lie algebras of G and K respectively, let g = k ⊕ p be the corresponding Cartan decomposition of g. Let A = exp(a) where a is a maximal abelian subspace of p and let M be the centralizer of A in K. Define the symmetric bi-linear form h·, ·i on g by 1 hX,Y i := B(X,Y ) (3.1) 2(n − 1) where B(X,Y ) = Tr(adXadY ) denotes the Killing form for g. The reason n for this normalization is so that the Riemannian metric on G/K ' H induced by h·, ·i has constant curvature −1. ij Let {Xi} be a basis for gC over C; put gij = hXi,Xji and let g be the (i, j) entry of the inverse matrix of (gij). The element X ij C = g XiXj is called the Casimir element of gC (with respect to h·, ·i). It is well-known that this definition is independent of the choice of a basis and that C lies in the center of the universal enveloping algebra U(gC) of gC. Denote by Gˆ the unitary dual, i.e., the set of equivalence classes of irre- ducible unitary representations of G. A representation π ∈ Gˆ is said to be tempered if for any K-finite vectors v1, v2 of π, the matrix coefficient func- 2+ tion g 7→ hπ(g)v1, v2i belongs to L (G) for any > 0. We describe the non-tempered part of Gˆ in the next subsection. 3.2. Standard representations and complementary series. Let α de- note the simple relative root for (g, a). The root subspace n of α has di- mension n − 1 and hence ρ, the half-sum of all positive roots of (g, a) with n−1 multiplicities, is given by 2 α. Set N = exp n. By the Iwasawa decompo- sition, every element g ∈ G can be written uniquely as g = kan with k ∈ K, a ∈ A and n ∈ N. We write κ(g) = k, exp H(g) = a and n(g) = n. For any g ∈ G and k ∈ K, we let κg(k) = κ(gk), and Hg(k) = H(gk) so that gk = κg(k) exp(Hg(k))n(gk). 14 AMIR MOHAMMADI AND HEE OH
Given a complex valued linear function λ on a, we define a G-representation U λ on L2(K) by the prescription: for φ ∈ L2(K) and g ∈ G,
λ (−(λ+2ρ)◦Hg−1 ) U (g)φ = e · φ ◦ κg−1 . (3.2) This is called a standard representation of G (cf. [65, Sec. 5.5]). Observe that the restriction of U λ to K coincides with the left regular representation 2 λ −1 of K on L (K): U (k1)f(k) = f(k1 k). If R denotes the right regular rep- resentation of K on L2(K), then R(m)U λ(g) = U λ(g)R(m) for all m ∈ M. In particular each M-invariant subspace of L2(K) for the right translation action is a G-invariant subspace of U λ. Following [65], for any υ ∈ Mˆ , we let Q(υ)L2(K) denote the isotypic R(M) submodule of L2(K) of type υ. Choosing a finite dimensional vector space, say, V on which M acts irreducibly via υ, it is shown in [65] that the υ-isotypic space Q(υ)L2(K) can be written as a sum of dim(υ) copies of Uυ(λ) where for each m ∈ M, U (λ) = f ∈ L2(K,V ): . υ f(km) = υ(m)f(k), for almost all k ∈ K
n−1 If λ ∈ ( + iR)α, then Uυ(λ) is unitary with respect to the inner product 2 R hf1, f2i = K hf1(k), f2(k)iV dk, and called a unitary principal series repre- sentation. These representations are tempered. A representation Uυ(λ) with n−1 λ∈ / ( 2 + iR)α is called a complementary series representation if it is uni- tarizable. For λ = rα, we will often omit α for simplicity. For n = 2, the ◦ complementary series representations of G = SO(2, 1) are U1(s − 1) with 1/2 < s < 1; in particular they are all spherical. For n ≥ 3, a representa- tion υ ∈ Mˆ is specified by its highest weight, which can be regarded as a n−1 sequence of 2 integers with j1 ≥ j2 ≥ · · · ≥ |j(n−1)/2| if n is odd, and as n−2 a sequence of 2 integers with j1 ≥ j2 ≥ · · · ≥ j(n−2)/2 ≥ 0 if n is even. In both cases, let ` = `(υ) be the largest index such that j` 6= 0 and put `(υ) = 0 if υ is the trivial representation. Then the complementary series n−1 representations are precisely Uυ(s − n + 1), s ∈ Iυ := ( 2 , (n − 1) − `), up to equivalence. In particular, the spherical complementary series representations are ex- hausted by {U1(s − n + 1) : (n − 1)/2 < s < n − 1}. The complementary representation Uυ(λ) contains the minimal K-type, say, συ with multiplicity one. The classification of Gˆ says that if π ∈ Gˆ is non-trivial and non-tempered, then π is (equivalent to) the unique irreducible subquotient of the comple- mentary series representation Uυ(s − n + 1), s ∈ Iυ, containing the K-type συ, which we will denote by U(υ, s − n + 1). This classification was obtained by Hirai [26]; see also [30, Prop. 49 and 50]) and [3]. Note that U(υ, s − n + 1) is spherical if and only if Uυ(s − n + 1) is spherical if and only if υ = 1. For convenience, we will call U(υ, s − n + 1) a complementary series representation of parameter (υ, s). EFFECTIVE COUNTING 15
Observe that the non-spherical complementary series representations exist n−1 only when n ≥ 4. For 2 < s < n − 1, we will set Hs := U(1, s − n + 1), i.e., the spherical complementary series representations of parameter s. Our normalization of the Casimir element C is so that C acts on Hs as the scalar s(s − n + 1). In order to study the matrix coefficients of complementary series repre- sentations, we work with the standard representations, which we first relate with Eisenstein integrals. 3.3. Generalized spherical functions and Eisenstein integrals. Fix a complex valued linear function λ on a, and the standard representation U λ. By the Peter-Weyl theorem, we may decompose the left-regular representa- 2 2 tion V = L (K) as V = ⊕σ∈Kˆ Vσ, where Vσ = L (K; σ) denotes the isotypic K-submodule of type σ, and Vσ ' dσ · σ where dσ denotes the dimension of σ. P 2 Set ΩK = 1 + ωK = 1 − Xi where {Xi} is an orthonormal basis of kC. It belongs to the center of the universal enveloping algebra of kC. By Schur’s lemma, ΩK acts on Vσ by a scalar, say, c(σ). Since ΩK acts as a skew-adjoint operator, c(σ) is real. Moreover c(σ) ≥ 1, see [65, p. 261], and ` ` kΩK vk = c(σ) kvk for any smooth vector v ∈ Vσ. Furthermore it is shown in [65, Lemma 4.4.2.3] that if ` is large enough, then X 2 −` dσ · c(σ) < ∞. (3.3) σ∈Kˆ For σ ∈ Kˆ and k ∈ K define
χσ(k) := dσ · Tr(σ(k)) where Tr is the trace operator. For any continuous representation W of K, and σ ∈ Kˆ , the projection operator from W to the σ-isotypic space W (σ) is given as follows: Z Pσ = χσ(k)W (k)dk. K
For υ ∈ Mˆ , we write υ ⊂ σ if υ occurs in σ|M , and we write υ ⊂ σ ∩ τ if υ occurs both in σ|M and τ|M . We remark that for σ ∈ Kˆ , given υ ∈ Mˆ occurs at most once in σ|M ([15]; [65, p.41]). For υ ⊂ σ, we denote by Pυ the projection operator from Vσ to the υ-isotypic subspace Vσ(υ) ' dσ · υ P so that any w ∈ Vσ can be written as w = υ⊂σ Pυ(w). By the theory of representations of compact Lie groups, we have for any f ∈ L2(K; σ) we have hf, χ¯σi = f(e). In the rest of this subsection, we fix σ, τ ∈ Kˆ . Define an M-module homomorphism T0 : Vσ → Vτ by X T0(w) = hPυ(w), Pυ(¯χσ)iPυ(¯χτ ). υ⊂σ∩τ 16 AMIR MOHAMMADI AND HEE OH
Set E := HomC(Vσ,Vτ ). Then E is a (τ, σ)-double representation space, a left τ and right σ-module. We put
EM := {T ∈ E : τ(m)T = T σ(m) for all m ∈ M}. λ λ λ Denote by Uσ and Uτ the representations of K obtained by restricting U |K to the subspace Vσ and Vτ respectively. Define Tλ ∈ E by Z λ λ −1 Tλ := Uτ (m)T0Uσ (m )dm M where dm denotes the probability Haar measure of M. It is easy to check that Tλ ∈ EM . R λ λ −1 λ(H(ak)) An integral of the form K Uτ (κ(ak))TλUσ (k )e dk is called an Eisenstein integral. Clearly, the matrix coefficients of the representation U λ are understood λ if we understand Pτ U (a)Pσ for all τ, σ ∈ Kˆ , which can be proved to be an Eisenstein integral: Theorem 3.4. For any a ∈ A, we have Z λ λ λ −1 λ(H(ak)) Pτ U (a)Pσ = Uτ (κ(ak))TλUσ (k )e dk. K ˆ 2 λ −1 P Proof. For • ∈ K and φ ∈ L (K; •), we write U• (k )φ = υ⊂• φk,υ, that λ −1 P is, φk,υ = Pυ(U• (k )φ). In particular, φ(k) = υ⊂• φk,υ(e) and λ −1 φk,υ(e) = hφk,υ, χ¯•i = hU• (k )φ, Pυ(¯χ•)i.
Let ϕ ∈ Vσ and ψ ∈ Vτ . For any g ∈ G, we have λ λ −1 X λ −1 λ −1 hUτ (κ(gk))T0Uσ (k )ϕ, ψi = hUσ (k )ϕ), Pυ(¯χσ)ihPυ(¯χτ ),Uτ (κ(gk)) ψi υ⊂σ∩τ X = ϕk,υ(e)ψκ(gk),υ(e). (3.5) υ⊂σ∩τ On the other hand, we have
Z −1 hU λ(a)ϕ, ψi = ϕ(κ(a−1k)ψ(k)e−(λ+2ρ)H(a k)dk K Z = ϕ(k)ψ(κ(ak))eλ(H(ak))dk K Z X X λ(H(ak)) = ( ϕk,υ(e))( ψκ(ak),υ(e))e dk. (3.6) K υ⊂σ υ⊂τ
We now claim that; if υ1 6= υ2, then Z λ(H(ak)) ϕk,υ1 (e)ψκ(ak),υ2 (e)e dk = 0 K To see this, first note that M and a commute, and hence H(amk) = H(ak), and κ(amk) = mκ(ak). We also note that
ϕk,υ1 ∈ Vσ(υ1), and ψκ(ak),υ2 ∈ Vτ (υ2). EFFECTIVE COUNTING 17
Now if υ1 6= υ2, then by Schur’s orthogonality of matrix coefficients, Z −1 −1 ϕk,υ1 (m )ψκ(ak),υ2 (m )dm = M Z λ λ 0 hUσ (m)ϕk,υ1 , Pυ1 (¯χσ)ihUτ (m)ψk ,υ2 , Pυ2 (¯χτ )idm = 0; M we get Z λ(H(ak)) ϕk,υ1 (e)ψκ(ak),υ2 (e)e dk K Z Z λ(H(amk)) = ( ϕmk,υ1 (e)ψκ(amk),υ2 (e)e dm)dk M\K M Z Z λ(H(ak)) −1 −1 = e ( ϕk,υ1 (m )ψκ(ak),υ2 (m )dm)dk = 0, M\K M implying the claim. Therefore, it follows from (3.5) and (3.6) that for any ϕ ∈ Vσ and ψ ∈ Vτ , λ λ hPτ U (a)Pσϕ, ψi = hU (a)ϕ, ψi Z X λ(H(ak)) = ϕk,υ(e)ψκ(gk),υ(e)e dk K υ⊂σ∩τ Z λ λ −1 λ(H(ak)) = hUτ (κ(ak))T0Uσ (k )ϕ, ψie dk K Z λ λ −1 λ(H(ak)) = hUτ (κ(ak))TλUσ (k )ϕ, ψie dk; K we have used κ(akm) = κ(ak)m and H(akm) = H(ak) for the last equality. It follows that Z λ λ λ −1 λ(H(ak)) Pτ U (a)Pσ = Uτ (κ(ak))TλUσ (k )e dk. K For the special case of τ = σ, this theorem was proved by Harish-Chandra (see [66, Thm. 6.2.2.4]), where T0 was taken to be T0(w) = (w, χ¯σ)¯χσ and R λ λ −1 Tλ = M Uσ (m)T0Uσ (m )dm. Lemma 3.7. For any λ ∈ C, 2 2 kTλk ≤ dσ · dτ where kTλk denotes the operator norm of Tλ. 2 ˆ Proof. Since kχυk ≤ dυ for any υ ∈ M, 2 X 2 X 4 kT0k ≤ kχυk ≤ dυ υ⊂σ∩τ υ⊂σ∩τ X 2 X 2 2 2 ≤ ( dυ · ( dυ) ≤ dσ · dτ . υ⊂σ υ⊂τ Since kTλk ≤ kT0k · kσk · kτk = kT0k, the claim follows. 18 AMIR MOHAMMADI AND HEE OH
3.4. Harish-Chandra’s expansion of Eisenstein integrals. Fix σ, τ ∈ Kˆ . Let E and EM to be as in the previous subsection. + Given T ∈ EM , r ∈ C, and at ∈ A , we investigate an Eisenstein integral: Z −1 (irα−ρ)(H(atk)) τ(κ(atk))Tirα−ρσ(k )e dk. K We recall the following fundamental result of Harish-Chandra:
Theorem 3.8. (Cf. [66, Theorem 9.1.5.1]) There exists a subset Oσ,τ of C, whose complement is a locally finite set, such that for any r ∈ Oσ,τ there exist uniquely determined functions c+(r), c−(r) ∈ HomC(EM ,EM ) such that for all T ∈ EM , Z −1 (irα−ρ)H(atk) ρ(at) τ(κ(atk))T σ(k )e dk K = Φ(r : at)c+(r)T + Φ(−r : at)c−(r)T
+ where Φ is a function on Oσ,τ ×A taking values in HomC(EM ,EM ), defined as in (3.12).
Let us note that, fixing T and at, the Eisenstein integral on the left hand side of the above is an entire function of r; see [66, Section 9.1.5]. Much of the difficulties lie in the fact that the above formula holds only for Oσ,τ but not for the entire complex plane, as we have no knowledge of which complementary series representations appear in L2(Γ\G). We need to un- R −1 (sα−2ρ)H(atk) derstand the Eisenstein integral K τ(κ(atk))Tsα−2ρσ(k )e dk n−1 for every s ∈ ( 2 , n − 1). We won’t be able to have as precise as a for- mula as Theorem 3.8 but will be able to determine a main term with an exponential error term. We begin by discussing the definition and properties of the functions Φ and c±.
3.4.1. The function Φ. As in [66, page 287], we will recursively define ratio- nal functions {Γ` : ` ∈ Z≥0} which are holomorphic except at a locally finite subset, say Sσ,τ . The subset Oσ,τ in Theorem 3.8 is indeed C−∪r∈Sσ,τ {±r}. More precisely, let l be the Lie algebra of the Cartan subalgebra (=the centralizer of A). Let Hα ∈ lC be such that B(H,Hα) = α(H) for all H ∈ lC. Let X ∈ g±α be chosen so that [X ,X ] = H and [H,X ] = ±α C α −α α α α(H)Xα. In particular, B(Xα,X−α) = 1. Write X±α = Y±α + Z±α where Y±α ∈ kC and Z±α ∈ pC. Letting ΩM denote the Casimir element of M, given S ∈ HomC(EM ,EM ), we define f(S) by
f(S)T = ST σ(ΩM ), for all T ∈ EM . n−1 We now define Γ` := Γ`(ir − 2 )’s in Q(aC) ⊗ HomC(EM ,EM ) by the following recursive relation (see [66, p. 286] for the def. of Q(aC)): Γ0 = I EFFECTIVE COUNTING 19 and X {`(2ir − n + 1) − `(` − n + 1) − f} Γ` = ((2ir−n+1)−2(`−2j))Γ`−2j j≥1 X X +8 (2j−1)τ(Yα)σ(Y−α)Γ`−(2j−1)−8 j {τ(YαY−α) + σ(YαY−α)} Γ`−2j. j≥1 j≥1
The set Oσ,τ consists of r’s such that {`(2ir − n + 1) − `(` − n + 1) − f} is invertible so that the recursive definition of the Γ`’s is meaningful.
Lemma 3.9. Fix any t0 > 0 and a compact subset ω ⊂ Oσ,τ . There exist bω (depending only on t0 and ω) and N0 > 1 (independent of σ, τ ∈ Kˆ ) such that for any r ∈ ω and ` ∈ N,
n−1 N0 N0 `t0 kΓ`(ir − 2 )k ≤ bωdσ dτ e . Proof. Our proof uses an idea of the proof of [66, Lemmas 9.1.4.3-4]. For n−1 s = ir − 2 , and T ∈ HomC(EM ,EM ), define 2 Λ`(T ) := −` + `(2s − n + 1) − f T.
For qσ and qτ which are respectively the highest weights for σ and τ, since qσ dσ with implied constant independent of σ ∈ Kˆ ,
max{kτ(Yα)σ(Y−α)k, kτ(YαY−α)k, σ(YαY−α)k} 2 2 0 3 3 ≤ c0 · (qσqτ + qσ + qσ)dσdτ ≤ c0dσdτ 0 for some c0, c0 > 0 independent of σ and τ. Hence for some c1 = c1(ω), for all r ∈ ω,
n−1 −1 3 3 X kΓ`(ir − 2 )k ≤ ` · kΛ` k · c1dσdτ kΓ`−jk. (3.10) j<`
2 −1 −t0 −1 3 3 Let N1 be an integer such that ` · kΛ` k · (1 − e ) c1dσdτ ≤ N1 for −1 −2 all ` ≥ 1. Since kΛ` k ` as ` → ∞ and the coefficients of f depend only on the eigenvalues of ΩM for those υ ∈ Mˆ contained in σ, we can take N2 N2 N1 = N1(ω) so that N1 ≤ c2dσ dτ for some N2 ≥ 1 and c2 = c2(ω) > 1 (independent of σ and τ). Set
n−1 −`t0 M(t0, ω) := max kΓ`(ir − 2 )ke . `≤N1,r∈ω
By (3.10) together with the observation that both N1 = N1(ω) and −1 max`≤N1 kΛ` k are bounded by a polynomial in dσ and dτ , we have M(t0, ω) ≤ N0 N0 bωdσ dτ for some N0 ≥ 1 and bω > 0. We shall now show by induction that for all r ∈ ω and for all ` ≥ 1,
n−1 `t0 kΓ`(ir − 2 )k ≤ M(t0, ω)e . (3.11) 20 AMIR MOHAMMADI AND HEE OH
First note that (3.11) holds for all ` ≤ N1 by the definition of M(t0, ω). Now for any N1 > N, suppose (3.11) holds for each ` < N. Then n−1 −1 2 −1 3 3 X n−1 kΓN (ir − 2 )k ≤ N (N kΛN kc1dσdτ ) kΓN−j(ir − 2 )k j −1 −t0 X (N−j)t0 ≤ N N1(1 − e )M(t0, ω) e j Nt0 ≤ M(t0, ω)e , finishing the proof. Following Warner (Cf. [66, Theorem 9.1.4.1]), we define irt X n−1 −`t Φ(r : at) = e Γ`(ir − 2 )e (3.12) `≥0 which converges for all large enough t by Lemma 3.9. − − 3.4.2. The function c±. Let N = exp(n ) be the root subspace corre- − sponding to −α, and dN − denote a Haar measure on N normalized so that R −2ρ(H(n)) N − e dN − (n) = 1. The following is due to Harish-Chandra; see [66, Thm. 9.1.6.1]. Theorem 3.13. For r ∈ Oσ,τ with =(r) < 0, c+(r) is holomorphic and given by Z −1 −(irα+ρ)(H(n)) c+(r)T = T σ(κ(n) )e dN − (n). N − R −(irα+ρ)H(n) The integral N − e dN − (n) is absolutely convergent iff =(r) < 0, shown by Gindikin and Karperlevic ([66, Coro.9.1.6.5]). Corollary 3.14. For any r ∈ Oσ,τ with =(r) < 0, the operator norm R (=(r)α−ρ)H(n) kc+(r)k is bounded above by N − e dN − (n). Proof. Since the operator norm kσ(k)k is 1 for any k ∈ K, the claim is immediate from Theorem 3.13. Proposition 3.15. Fix a compact subset ω contained in Oσ,τ ∩ {=(r) < 0}. There exist d1 = d1(ω) and N2 > 1 such that for any r ∈ ω, we have N2 N2 kc±(r)T±irα−ρk ≤ d1 · dτ dσ . R −(<(ir)α+ρ)H(n) Proof. By the assumption on ω, the integral N − e dN − (n) converges uniformly for all r ∈ ω. Hence the bound for c+(r)Tirα−ρ fol- lows from Corollary 3.14 together with Lemma 3.7. To get a bound for c−(r)T−irα−ρ, we recall that n−1 Z (−ir+ )t −1 (irα−ρ)(H(atk)) e 2 τ(κ(atk))T σ(k )e dk K −irt −irt = e Φ(r : at)c+(r)T + e Φ(−r : at)c−(r)T. EFFECTIVE COUNTING 21 −irt P n−1 −`t Then e Φ(−r : at) = I+ `≥1 Γ`(−ir− 2 )e , and applying Lemma 3.9 with t0 = 1, we get X n−1 −`t N0 N0 X `(1−t) kΓ`(−ir − 2 )e k ≤ bωdσ dτ e . `≥1 `≥1 N0 N0 P `(1−t0) Fix t0 > 0 so that bωdσ dτ `≥1 e < 1/2; then t0 log(dσdτ ). Now −irt0 0 Ar := e Φ(−r : at0 ) is invertible and for some N1 and bω, −1 0 N1 N1 kAr k ≤ bωdσ dτ . (3.16) R (irα−ρ)(H(at0 k)) Since the map k 7→ H(at0 k) is continuous, we have K |e |dk < dω for all r ∈ ω, and hence kc−(r)Tirα−ρk n−1 Z (−ir+ )t −1 2 0 −1 (irα−ρ)(H(at0 k)) ≤ kAr k · |e τ(κ(at0 k))Tirα−ρσ(k )e dk| K −1 −irt0 + kAr k · ke Φ(r : at0 )c+(r)Tirα−ρk −1 −1 ≤ kAr k · dω max kτ(κ(at0 k))Tirα−ρσ(k )k + kc+(r)Tirα−ρk . k∈K Hence the claim on kc−(r)Tirα−ρk now follows from (3.16), Lemma 3.7 and the bound for kc+(r)Tirα−ρk. 3.5. Asymptotic expansion of the matrix coefficients of the com- plementary series. Fix a parameter (n − 1)/2 < s0 < (n − 1), and recall that 2ρ = (n − 1)α. We apply the results of the previous subsections to the standard representation U (s0−n+1)α = U s0α−2ρ. The following theorem is a key ingredient of the proof of Theorem 3.30. Theorem 3.17. There exist η0 > 0 and N > 1 such that for any σ, τ ∈ Kˆ , for all t > 2, we have (s0−n+1)α (s0−n+1)t N N (s0−n+1−η0)t Pτ U (at)Pσ = e c+(rs0 )T(s0−n+1)α+O(dσ ·dτ e ), with the implied constant independent of σ, τ. Proof. Set rs := −i(s − ρ) ∈ C, for all s ∈ C. In particular, =(rs) < 0 for (n − 1)/2 < s < (n − 1). Fix t > 0, and define Ft : C → EM by sα−2ρ Ft(s) := Pτ U (at)Pσ. By Theorem 3.4, Z −1 (sα−2ρ)H(atk) Ft(s) = τ(κ(atk))Tsα−2ρσ(k )e dk. K As was remarked following Theorem 3.8, for each fixed t > 0, the function Ft(s) is analytic on C. Thus in view of Theorem 3.13, we have, whenever rs ∈ Oσ,τ and =(rs) < 0, (s−n+1)t Ft(s) − e c+(rs)Tsα−2ρ is analytic. (3.18) 22 AMIR MOHAMMADI AND HEE OH ˜ Recall the notation Sσ,τ , that is, Oσ,τ = C − ∪r∈Sσ,τ {±r}, and set Sσ,τ = {s : rs ∈ Sσ,τ }. Define (s−n+1)t Gt(s) = Ft(s) − e c+(rs)Tsα−2ρ. Indeed the map s 7→ Gt(s) is analytic on {s : =(rs) < 0} − S˜σ,τ . Since ˜ 0 ∪σ0,τ 0∈Kˆ ± Sσ0,τ 0 is countable, we may choose a small circle ω of radius at 0 most 1/2 centered at s0 such that {rs : s ∈ ω } ∩ ∪σ0,τ 0∈Kˆ ± Sσ0,τ 0 = ∅. Observe that the intersection of ω0 and the real axis is contained in the interval ((n − 1)/2, n − 1). Note that there exists η > 0 such that for all s ∈ ω0, (n − 1) − s0 + η < <(s) < s0 + 1 − η. (3.19) 0 Then Gt(s) is analytic on the disc bounded by ω . Hence by the maximum principle, we get kGt(s0)k ≤ max kGt(s)k. (3.20) s∈ω0 0 Since ω := {rs : s ∈ ω } ⊂ Oσ,τ , Theorems 3.4 and 3.8 imply that for all s ∈ ω0, we have (s−n+1)t X −`t n−1 Gt(s) = e ( e Γ`(irs − 2 )c+(rs)Tsα−2ρ) `≥1 −st X −`t n−1 + e ( e Γ`(−irs − 2 )c−(rs)Tsα−2ρ). `≥0 Fixing any t0 > 0, by Lemma 3.9, there exists b0 = b0(t0, ω) > 0 such that for all r ∈ ω, n−1 N0 N0 `t0 kΓ`(ir − 2 )k ≤ b0 · dσ · dτ · e . (3.21) By Proposition 3.15, for all r ∈ ω, N2 N2 kc±(r)T±irα−ρk ≤ d1 · dσ · dτ . P −`(t−t0) Let t > t0 + 1 so that `≥0 e ≤ 2. Then we have for any t > 0 and s ∈ ω0, X −`t n−1 k e Γ`(irs − 2 )c+(rs)Tsα−2ρk `≥1 N0+N2 N0+N2 −t t0 X −`(t−t0) ≤ ·dσ dτ · bσ,τ · e e e `≥0