IC/91/94 INTERNATIONAL CENTRE FOR THEORETICAL PHYSICS ARITHMETIC, GEOMETRY AND DYNAMICS IN THE UNIT TANGENT BUNDLE OF THE MODULAR ORBIFOLD Alberto Verjovsky INTERNATIONAL ATOMIC ENERGY AGENCY UNITED NATIONS EDUCATIONAL, SCIENTIFIC AND CULTURAL ORGANIZATION 1991 MIRAMARE - TRIESTE IC/90/94 INTRODUCTION International Atomic Energy Agency The modular group PSL(2, Z) and its action on the upper half-plane H together with and with its quotient, the modular orbifold, are fascinating mathematical objects. The study United Nations Educational Scientific and Cultural Organization of modular forms has been one of the classical and fruitful objects of study. If one considers INTERNATIONAL CENTRE FOR THEORETICAL PHYSICS the group PSL(2,R) and we take the quotient M = PSL(2,R)/PSL(2,Z) one obtains a three dimensional manifold that carries an enormous amount of arithmetic information. Of course this is not surprising as the elements of PSL(2,Z) are Mobius transformations given by matrices with columns consisting of lattice points with relatively prime integer coefficients and therefore M contains information about the prime numbers. On the other hand from the dynamical systems viewpoint, M is also very interesting—it has a flow which is an Anosov flow. The flow as well as its stable and unstable foliations correspond to three ARITHMETIC, GEOMETRY AND DYNAMICS IN THE UNIT one-parameter subgroups: the geodesic flow, and the stable and unstable horocyclic flows TANGENT BUNDLE OF THE MODULAR ORBIFOLD* (see the next section). All these flows preserve normalized Haar measure m, and are ergodic with respect to this measure. By a theorem of Dani [Da] the horocyclic flows have a curve •"(v)i (v > 0), of ergodic probability measures. These ergodic measures are supported in ALBERTO VERJOVSKYJ closed orbits of period y of the corresponding horocyclic flow. If we denote by my the measure corresponding to y > 0, then m, converges to m as y -» 0 ([Da] [Za]). We will International Centre for Theoretical Physics, Trieste, Italy. also give a proof of this. However what is most interesting for me as a dynamical systemist is the remarkable connection found by Don Zagier between the rate of approach of m, to m and the Riemann Hypothesis. Zagier found that the Riemann Hypothesis holds if and only if one can find a nonzero smooth function / with compact support on M such that 4 1/2 mv(f) = m(f) + oiy*' -') for all e > 0. He also proved that m,(/) = m(jF) + o(y ). In the Appendix we will review Zagier's approach as well as the extension given by Sarnak ([Sa]). The purpose of this paper is to analyze the dynamics and geometry of the horocyclic now to show that the exponent 1/2 is optimal for certain characteristic functions of sets ABSTRACT called "boxes". Of course this is very far from disproving the Riemann Hypothesis since such a characteristic function is not even continuous. By geometric means we reduce the The interplay between the geometry, dynamics and arithmetic of the modular orbifold analysis of the convergence of m, to ro to a lattice point counting. Thus the fact that are studied. The Riemann hypothesis looms upon this structure. the exponent cannot be made better than 1/2 is similar to the circle problem in which we count the number of lattice points with relatively prime coordinates inside a circle. If instead we take a smooth function / with compact support and take the sum, T.{y), of the values of the function over all lattice points (yn,ym) with y > 0 and (n,m) G Z then j/2£(y) will converge to the integral of / over the plane as y -* 0 and the error term will be oiy"), as y -t 0, for all a > 0. This follows by the Poisson summation formula using MIRAMARE - TRIESTE the fact that the Fourier transform of / decays very rapidly at infinity May 1991 We will use many properties of SL(2, R), its quotients by discrete subgroups and the two locally free actions of the proper real affine group on thse quotients. Good references for each subject are [A], [G], [L], [Ma] and [Ra]. 0. PRELIMINARIES *To be submitted for publication Let G := SL(2,R) denote the Lie group of 2 x 2 matrices of determinant one, with real fPermanent address: CINVESTAV Del IPN, Apdo. Postal 14-740, Mexico D.F., Mexico. coefficients. The Lie algebra of G, 0 := st(2,R) , consists of real 2x2 matrices of trace Typeset by zero. This Lie algebra has the standard basis: By euclidean translation to the origin and clockwise rotation of 90 degrees we have a trivialization $ : TjH -* H x S1 given by 0 0 A = B = 0 - c = 1 o To A, B and C correspond the left-invariant vector fields X, V, and Z respectively in where v is a hyperbolic unit tangent vector anchored at z 6 ^H. For example, using (0.2), SL(2,R).These vector fields induce, respectively, the nonsingular flows: -j, ,_» y(^ x) gives an explicit identification and it will be the one we will use here. If -v = f" M £ G, we will let 7(2) = -j denote the corresponding element in G. g,;G->G lc «J cz -(- a Using trivialization (0.2), we have Explicitly: (0.3) y(«,0) = (7{2),0-2arg(cz + d)) a b c d In this notation 8 is to be taken modulo 2ir, where the angle of a unit vector is measured from the vertical counter-clockwise. a b i *i (0.1) The three basic vector fields X, Y and Z descend toGs XjH and the flows induced by c d *([ ])- 0 lj' them correspond to the geodesic flow, unstable horocyclic flow and stable horocyclic flow, a b respectively. c d D- Geometrically these flows can be described as follows. Let z e HI and let vz £ XjH be a unit vector based at 2. This vector determines a unique oriented geodesic 7, as well + + To simplify notation, let us write: g := {gt}t€E, h := {hf}t&t and h •= {ht }tem- as two oriented horocycles C and C~ which pass through z are orthogonal to 7 and Consider the upper half-plane, H = {z = (x,y) := x + iy \ y > 0} C C equipped with the tangent to the real axis. Then v' := gt(vt) is the unit vector tangent to 7, following the 2 2 1 2 + metric ds = (l/y )(dx' + dy ). With this metric H is the hyperbolic plane with constant same orientation as 7 at the point at distance t from z. The vectors w = h+(vt) and + negative curvature minus one. w~ = hv(yt) are obtained by taking unit vectors tangent to C and C~, respectively, at G acts by isometries on H as follows: distances u and v respectively, and according to their orientations (see Figure 1). a b c d where z = x + iy, y > 0. The action is not effective and the kernel is the subgroup of order two, {/, —/}, consisting of the identity and its negative. Let G = 5/{1,-1} := PSL(2,R). Note that G is the group of Mobius transformations that preserve H and is in fact its full group of orientation- preserving isometries. The action of G on HI can be extended via the differential to the unit tangent bundle which we shall denote henceforth by T\R. If 7 £ G and y' denotes its differential acting on unit vectors, we have: Figure 1 *It is because of this geometric interpretation that the flows g, h+ and h~, defined originally in G are called geodesic and horocyclic flows respectively. Formulae (0.1) tell us H HI that the orbits of the respective flows are obtained by left-translations of the one-parameter subgroups where Px is the canonical projection. Naturally, "unit vector" refers to the hyperbolic metric. 0 e-t*l "Ho 1 • "" U Every one-parameter subgroup is conjugate in G to one of the above, and h+ is conjugate When F is co-compact both h+ and h are minimal flows on M(T)- In particular, the toft". horocyctic flows do not contain periodic orbits. This was proved by Hedlund [He]. It is Henceforth we will equip G with the left-invariant riemannian metric such that {X, Y, Z} also a consequence of a result of Plante JP1]. Suppose for instance that one orbit of h+ is is an oriented orthonormal framing. We will call this metric the standard metric. not dense. Then the closure of this orbit contains a non-trivial minimal set £ and Plante By the standard riemannian measure or Baar Measure, we will mean the measure, m, showed that E must be both a 2-torus and a global cross section for the geodesic flow, induced by the volume form £1 which takes the constant value one in the oriented framing implying that M(F) would be a torus bundle over S1. But this is impossible since under {X, Y,Z}. Since G is unimodular, the measure m is bi-invariant. the hypothesis, F cannot be solvable. Let J4.2(R) denote the proper a/pne group: It was shown by Furstenberg [Fu] that for F co-compact both h+ and h~ are strictly ergodic flows with m as their unique invariant measure (this also implies minimality of A2(R) = {T : R -» R | T(r) = ar + 6; a, 6 6 R, a > 0}. k+ and h~). Since the geodesic flow is transitive, it contains a set of the second Baire category of dense orbits and it also contains a countable number of periodic orbits whose Let us parametrize A2(R) by pairs (a, 6) with a,b 6 R, a > 0.