JOURNALOF MODERN DYNAMICS doi: 10.3934/jmd.2013.7.369 VOLUME 7, NO. 3, 2013, 369–394 SOME VIRTUALLY ABELIAN SUBGROUPS OF THE GROUP OF ANALYTIC SYMPLECTIC DIFFEOMORPHISMS OF A SURFACE JOHN FRANKS AND MICHAEL HANDEL (Communicated by Ralf Spatzier) ABSTRACT. We show that if M is a compact oriented surface of genus 0 and ! G is a subgroup of Symp¹ (M) that has an infinite normal solvable subgroup, then G is virtually abelian. In particular the centralizer of an infinite order ! f Symp¹ (M) is virtually abelian. Another immediate corollary is that if G is 2 ! a solvable subgroup of Symp¹ (M) then G is virtually abelian. We also prove a ! special case of the Tits Alternative for subgroups of Symp¹ (M). 1. INTRODUCTION r r If M is a compact connected oriented surface we let Diff 0(M) denote the C diffeomorphisms isotopic to the identity. In particular if r ! this denotes the r Æ subgroup of those which are real analytic. Let Symp¹(M) denote the subgroup of r r Diff 0(M) which preserve a smooth volume form ¹. We will denote by Cent (f ) r r r and Cent¹(f ) the subgroups of Diff 0(M) and Symp¹(M), respectively, whose r r elements commute with f . If G is a subgroup of Symp¹(M) then Cent¹(f ,G) will denote the the subgroup of G whose elements commute with f . In this article we wish to address the algebraic structure of subgroups of Symp¹1(M) and our results are largely limited to the case of analytic diffeomorphisms when M has genus 0. Our approach is to understand the possible dynamic behavior of such diffeomorphisms and use the structure exhibited to conclude information about subgroups. r DEFINITION 1.1. If N is a connected manifold, an element f Diff (N) will be 2 said to have full support provided N Fix(f ) is dense in N. We will say that à f has support of finite type if both Fix(f ) and N Fix(f ) have finitely many à components. We will say that a subgroup G of Diff r (N) has full support of finite type if every nontrivial element of G has full support of finite type. The primary case of interest for this article is analytic diffeomorphisms, but we focus on groups with full support of finite type to emphasize the properties Received December 2, 2012; revised September 18, 2013. 2010 Mathematics Subject Classification: Primary: 37C85; Secondary: 37E30, 37C40. Key words and phrases: Surface diffeomorphism groups, area-preserving, entropy. JF: Supported in part by NSF grant DMS0099640. MH: Supported in part by NSF grant DMS0103435 and a PSC-CUNY grant. INTHEPUBLICDOMAINAFTER 2041 369 ©2013 AIMSCIENCES 370 JOHN FRANKSAND MICHAEL HANDEL we use, which are dynamical rather than analytic in nature. The following result shows that we include Diff !(M). ! PROPOSITION 1.2. If N is a compact connected manifold, the group Diff (N) has full support of finite type. Proof. If f is analytic and nontrivial the set Fix(f ) is an analytic set in N and has no interior and the set N Fix(f ) is a semianalytic set. Hence Fix(f ) has à finitely many components and N Fix(f ) has finitely many components (see [4, à Corollary 2.7] for both these facts). The following result is due to Katok [12], who stated it only in the analytic case. For completeness we give a proof in Section3 but the proof we give is essentially the same as the analytic case. 2 PROPOSITION 1.3 (Katok [12]). Suppose G is a subgroup of Diff (M) which has full support of finite type and f Diff 2(M) has positive topological entropy. Then 2 the centralizer of f in G is virtually cyclic. Moreover, every infinite-order element of this centralizer has positive topological entropy. A corollary of the proof of this result is the following. 2 COROLLARY 1.4. Suppose f ,g Diff (M) have infinite order and full support of 2 finite type. If f g g f , then f has positive topological entropy if and only if g Æ has positive entropy. We remark that in contrast to our subsequent results Proposition 1.3 and its corollary make no assumption about preservation of a measure ¹. We can now state our main result, the proof of which is given in Section6. THEOREM 1.5. Suppose M is a compact oriented surface of genus 0 and G is a subgroup of Symp¹1(M) which has full support of finite type, e.g., a subgroup ! of Symp¹ (M). Suppose further that G has an infinite normal solvable subgroup. Then G is virtually abelian. There are several interesting corollaries of this result. To motivate the first we recall the following result. ! 1 THEOREM 1.6 (Farb–Shalen [6]). Suppose f Diff 0 (S ) has infinite order. Then ! 1 2 the centralizer of f in Diff 0 (S ) is virtually abelian. There are a number of instances in which results about the algebraic struc- 1 ture of Diff(S ) have close parallels for Symp¹(M). By analogy with the result of Farb and Shalen above, it is natural to ask the following question. 2 QUESTION 1.7. Suppose M is a compact surface and G is a subgroup of Diff 0(M) with full support of finite type and f G has infinite order. Is the centralizer of f 2 in G always virtually abelian? The following result gives a partial answer to this question. JOURNALOF MODERN DYNAMICS VOLUME 7, NO. 3 (2013), 369–394 VIRTUALLY ABELIAN SUBGROUPS OF ANALYTIC SYMPLECTIC DIFFEOMORPHISMS OF S2 371 PROPOSITION 1.8. Suppose M is a compact surface of genus 0, H is a subgroup of Symp1(M) with full support of finite type and f H has infinite order, then ¹ 2 Cent¹1(f , H), the centralizer of f in H, is virtually abelian. In particular this applies when H Symp!(M). ½ ¹ Proof. If we apply Theorem 1.5 to the group G Cent1(f , H) and observe that Æ ¹ the cyclic group generated by f is a normal abelian subgroup, then we conclude that G is virtually abelian. Question 1.9 of [7] asks if any finite-index subgroup of MCG(§g ) can act faith- fully by area-preserving diffeomorphisms on a closed surface. The following corollary of Proposition 1.8 and Proposition 1.2 gives a partial answer in the special case of analytic diffeomorphisms and M of genus 0. COROLLARY 1.9. Suppose H is a finite-index subgroup of MCG(§ ) with g 2 g ¸ or Out(F ) with n 3, and G is a group with the property that every element n ¸ of infinite order has a virtually abelian centralizer. Then any homomorphism º: H G has nontrivial kernel. In particular this holds if G Symp!(S2). ! Æ ¹ Proof. Suppose at first that H is a finite-index subgroup of MCG(§ ) with g 2. g ¸ Let ®1,®2 and ®3 be simple closed curves in M such that ®1 and ®2 are disjoint from ®3 and intersect each other transversely in exactly one point. Let Ti be a Dehn twist around ® with degree chosen so that T H. Then T and T i i 2 1 2 commute with T3 but no finite-index subgroup of the group generated by T1 and T2 is abelian. It follows that ¹ cannot be injective. Suppose now that H is a finite-index subgroup of Out(F ) with n 3. Let n ¸ A,B and C be three of the generators of Fn. Define © Aut(Fn) by B BA, 1 2 7! 1 define ª Aut(F ) by C CB AB ¡ A and define £ Aut(F ) by C C AB AB ¡ , 2 n 7! 2 n 7! where © fixes all generators other than B and where ª and £ fix all generators other than C. Let Á,à and θ be the elements of Out(Fn) determined by ©,ª and £ respectively. It is straightforward to check that © commutes with ª and k 1 k £ and that for all k 0, ª (which is defined by C C(B AB ¡ A) ) does not k È 7! 1 k k k commute with £ (which is defined by C C(AB AB ¡ ) ). Moreover, [£ ,ª ] 7! is not a nontrivial inner automorphism because it fixes both A and B. It follows that [θk ,Ãk ] is nontrivial for all k and hence that no finite-index subgroup of the group generated by à and θ, and hence no finite-index subgroup of Cent(Á), the centralizer of Á in Out(Fn), is abelian. The proof now concludes as above. Another interesting consequence of Theorem 1.5 is the following result. PROPOSITION 1.10. Suppose M is a compact surface of genus 0 and G is a solv- able subgroup of Symp¹1(M) with full support of finite type, then G is virtually abelian. The Tits Alternative (see J. Tits [17]) is satisfied by a group G if every subgroup (or by some definitions every finitely generated subgroup) of G is either virtually solvable or contains a nonabelian free group. This is a deep property known for finitely generated linear groups, mapping-class groups [10, 13], and the outer JOURNALOF MODERN DYNAMICS VOLUME 7, NO. 3 (2013), 369–394 372 JOHN FRANKSAND MICHAEL HANDEL automorphism group of a free group [2, 3]. It is an important open question for ! 1 1 Diff (S ) (see [6]). It is known that this property is not satisfied for Diff 1(S ) (again see [6]).