DISCRETE AND CONTINUOUS Website: http://aimSciences.org DYNAMICAL SYSTEMS Volume 17, Number 2, February 2007 pp. 423–439 ON THE BURNSIDE PROBLEM IN Diff (M) Julio C. Rebelo Dept. de Matematica, PUC-Rio R. Marques de S. Vicente 225 Rio de Janeiro RJ CEP 22453-900 Brazil Ana L. Silva Dept. de Matematica, CCE UEL Campus Universitario, Caixa Postal 6001 Londrina PR CEP 86051-990 Brazil Dedicated to C. Gutierrez and to M. A. Teixeira on the occasion of their 60th birthday. Abstract. In this paper we obtain some non-linear analogues of Schur’s the- orem asserting that a finitely generated subgroup of a linear group all of whose elements have finite order is, in fact, finite. The main result concerns groups of symplectomorphisms of certain manifolds of dimension 4 including the torus T4. 1. Introduction. The main result of this paper reads as follows: Theorem A: Let M be a compact 4-dimensional symplectic manifold and denote by Symp (M) the group of symplectomorphisms of M (say of class C2). Suppose that the fundamental class in H4(M, Z) is a product of classes in H1(M, Z). Then any finitely generated subgroup G ⊂ Symp (M) having only elements of finite order is, in fact, finite. In the proof of Theorem A, an important role is played by Theorem B below. Consider now a compact (oriented) manifold M and let Diff2(M) be the group of orientation-preserving C2-diffeomorphisms of M. If µ is a probability measure on 2 2 M, denote by Diffµ the subgroup of Diff (M) consisting of those diffeomorphisms preserving µ, then one has: 2 Theorem B: Let M, µ and Diff µ(M) be as above. Suppose that the fundamental class in Hn(M, Z) is a product of elements in H1(M, Z). Suppose in addition that the mapping class group of M is finite. Then any finitely generated subgroup G of 2 Diffµ(M) all of whose elements have finite order is, in fact, finite. Remark: Naturally we can dispense with the condition regarding the mapping class group of M if we assume directly that G consists of diffeomorphisms isotopic to the identity. Some comments on the above statements are necessary. First the assumption concerning the finiteness of the mapping class group of M made in Theorem B is definitely strong, especially because mapping class groups are notoriously hard to 2000 Mathematics Subject Classification. 53D25, 37C50 . Key words and phrases. Burnside problem . 423 424 JULIO C. REBELO AND ANA L. SILVA be computed. The reader who feels uncomfortable with this condition may sim- ply regard Theorem B as a technical statement used in the proof of Theorem A (further motivation for Theorem A will be discussed in the sequel). Still we would like to point out that in dimension 3 there is a good number of examples of man- ifolds having finite mapping class group. These include Haken manifolds or, more generally, manifolds carrying genuine laminations, hyperbolic manifolds and some Seifert manifolds (for terminology and proofs the reader may check [7] and references therein). It should also be emphasized that our statements aim at manifolds whose Euler characteristic e(M) vanishes. Indeed, when e(M) 6= 0, these statements can easily be greatly generalized as explained in Theorem (3.1). The corresponding argument is similar in “spirit” to those employed in [8] and [6]. On the other hand little seems to be known about manifolds verifying e(M) = 0 except for dimensions 1 and 2. Let us also point out that throughout this paper we work with C2 diffeomor- phisms. This is naturally required by the Hodge theoretic arguments (cf. Section 2) and we have not searched for the best condition in terms of regularity. Let us now situate the above theorems with respect to other works. In the present days, linear representation theory of groups is a classical and well-developed sub- ject that appears in different aspects of mathematics. However, there are also many relevant cases where a group acts through automorphisms on some manifold and this action is not “linear” i.e. it does not embed in a finite dimensional Lie group acting on M. This suggests us to consider a “non-linear” theory of representations where the target group is no longer a matrix group (or a finite dimensional Lie group) but the whole group of diffeomorphisms of some compact manifold M. Here the amount of regularity of the diffeomorphisms in question is not specified so that we may consider real analytic ones if necessary. This problem is equally interesting if restricted to the groups of diffeomorphisms of M that preserve a volume form or a symplectic structure. Indeed our attention is focused on groups of symplecto- morphisms due to two main reasons. The first one is that these groups are really huge and “essentially” non-linear which is necessary to be coherent with the above proposal. On the other hand, there is evidence that they share deep properties with linear groups, at least on the connected component of the identity. This is particu- larly clear from the work of Polterovich (cf. [18]) who has intensively investigated their structure. The point of view of (cf. [18]), however, is rather different from ours. The possibility of studying homomorphisms from abstract groups to a group of the form Diff (M) was discussed for example in [2]. This paper seems to have led E. Ghys to suggest to the first author that it would be interesting to consider actions of “torsion groups”. More precisely, a finitely generated, infinite, group all of whose elements have finite order is going to be called a Burnside group. The idea of E. Ghys was to discuss possible Burnside group actions on compact manifolds. As mentioned, his motivation apparently stemmed from [2] where the authors have suggested that a “generic” group may not act on a compact manifold. According to the works of Champetier and Ol’shanskii, Burnside groups are “generic” in the topological sense, i.e. they form a Gδ-dense set for a suitable topology on the space of finitely generated groups (cf. [10]). Nonetheless they are not “generic” in the probabilistic sense (see [10] for different notions of genericity used in Group Theory). Yet one first answer to the question of [2] might be achieved by showing ON THE BURNSIDE PROBLEM 425 that a Burnside group does not act on any compact manifold. Our theorems then can be viewed as a contribution to this problem in higher dimensions. Remark: After a classical result of H¨older stating that any non-Abelian group acting on the circle has points with non-trivial stabilizers, it immediately follows that a Burnside group cannot act faithfully on the circle. In fact, a diffeomorphism of finite order of the circle possessing fixed points must coincide with the identity. On the other hand, our main motivation to approach the question concerning the existence of an effective Burnside group action on a compact manifold comes, in an indirect way, from the Zimmer program as explained below. Note that years before [2], Zimmer had already proposed the program of studying homomorphisms from lattices of semi-simple Lie groups to Diff (M) with a view to obtaining “non-linear” versions of Margulis’s superrigidity theorems. Due to the difficulties involved and, to some extent, to the lack of suitable techniques to deal with them, definite progress in this program has been accomplished only in dimension 1 (cf. [9], [16], [25]). In view of the difficulties in carrying out the programs mentioned above, in this paper we turn to a question that, at least in principle, should be much simpler. Besides restricting attention to the symplectic category, we also propose to study infinite groups having “many” torsion elements. Of course, the prototype of the groups we have in mind are Burnside groups again. As a matter of fact, the quest for the properties of Burnside groups, in particular their very existence, is closely related to the development of Combinatorial Group Theory. Historically they have preceded the deep results on the structure of arithmetic subgroups and more general lattices in semi-simple Lie groups. Hence, it seemed to us that might be more reasonable to begin the study of “non-linear” representations of groups by considering possible embeddings of Burnside groups in the group of symplectomorphisms of a compact manifold. In this way, we are trying to obtain the non-linear analogue of a classical result due to Schur, cf. [23], rather than searching for the non-linear analogues of the very hard theorems of Margulis (the Zimmer program). This added to the independent interest of this study raised by Ghys observation quoted above. A consequence of our point of view is that, in some sense, this paper lies in the borderline between Dynamics and Topology. In fact, actions of infinite groups are associated to Dynamics since their orbits are not compact. On the other hand, actions of finite groups is a classical subject of Topology. Since our infinite groups have elements of finite order their possible actions have a significative topological component. To close this Introduction let us present a short description of the contents of the paper. The first ingredient needed is a criterion to guarantee that a given diffeomorphism coincides with the identity. We choose here one due to S. Weinberger ([20] and [24]) which is described in Section 2 along with examples of Burnside group actions on open manifolds. The second ingredient is the Flux homomorphism which is defined on the con- nected component of the identity of Diffµ(M) (for a given probability µ).