CLOSED AND TRANSITIVE TRANSFORMATION GROUPS OF A SURFACE FERRY H. KWAKKEL Abstract. The purpose of this paper is to survey the structure of closed and transitive trans- formation groups acting on a closed surface. In particular, we prove a number of relations between groups acting on the sphere that contain the rotation group, together with a diagram of how these groups are connected. In addition, we describe transformation groups acting on the torus and higher genus surfaces. 1. Introduction and statement of results 1.1. History and definitions. Let Homeo(M) be the group of orientation preserving home- omorphisms of a closed orientable topological surface M, with M either the sphere S2, the torus T2, or a higher genus surface S of genus at least two. The uniform topology defined on Homeo(M), or a subgroup of Homeo(M), is the topology induced by the topological distance dC0 between two homeomorphisms f, g ∈ Homeo(M), defined by (1.1) dC0 (f, g) = sup d(f(p), g(p)) = max d(f(p), g(p)), p∈M p∈M with d(·, ·) the metric defined on M, which is the standard spherical metric inducing the Lebesgue measure in the case of the sphere S2. The topology on Homeo(M) is defined to be the uniform topology. A subgroup G of Homeo(M) is said to be closed if it is a closed subset of Homeo(M) as a topological subspace in the uniform topology, and G is said to be transitive if for any two given p,q ∈ M, there exists a h ∈ G, such that h(p) = q. A closed transitive subgroup is said to be minimal, respectively maximal, if the group is minimal, respectively maximal, relative to inclusion of subgroups, with respect to the property of being a closed and transitive proper subgroup of Homeo(M). Since each such surface admits the group of area-preserving homeomorphisms homotopic to the identity which is a closed and transitive maximal proper arXiv:1502.04111v1 [math.GT] 13 Feb 2015 subgroup of the group of homeomorphisms homotopic to the identity of that surface, see Theorem A below, it follows that for every closed surface M, there exists at least one maximal closed and transitive subgroup other than the full group of homeomorphisms of M homotopic to the identity. The general problem presented is to classify the closed and transitive subgroups, modulo conjugation, of the homeomorphism group Homeo(M) of a closed surface and find its maximal subgroups. This problem relates to the classification of Lie group actions on a closed surface, initiated by classical works of Lie [9] and further classified by Mostow [10]. Classifying closed and transitive subgroups of the homeomorphism group of a closed manifold is a particular problem in the study of general homeomorphism groups of manifolds, see [5] by Fisher. 2010 Mathematics Subject Classification. 22F05 (primary), and 51H20 (secondary). Key words and phrases. transformation groups, Lie groups acting on a surface, surface homeomorphisms. 1 2 FERRY H. KWAKKEL In dimension one, subgroups of the homeomorphism group of the circle have been studied by Ghys [3], and in particular closed and transitive subgroups of the circle by Giblin-Markovic [4], answering an interesting question posed in [3]. The results in [4] include that a closed and transitive subgroup of Homeo(S1), with S1 the circle, that properly extends the rotation group Rot(S1) is either the group M¨obius(S1) of M¨obius transformations, or one of its cyclic covers, or the full homeomorphism group, and one of its cyclic covers. In particular, the group M¨obius(S1) acting on the circle is a maximal subgroup in Homeo(S1). 1.2. Statement of results. In what follows, denote RP2 the projective plane, Lin(RP2) the group of projective mappings, Ant(S2) the centralizer of Homeo(S2) with respect to the antipo- 2 2 dal action on the sphere, M¨obius(S ) the group of M¨obius transformations, and Homeoλ(S ) ⊂ 2 2 2 Homeo(S ) and Antλ(S ) ⊂ Ant(S ) the corresponding subgroups of area-preserving homeo- morphisms. A closed subgroup G ⊆ Homeo(S2), with S2 the two-sphere, that properly extends the rotation group is called a homogeneous transformation group. We present the following diagram of homogeneous groups. M¨obius(S2) 6 ⋆ 2 Homeoλ(S ) 2 5 ⋆ 2 2 ⋆ $ 2 Rot(S ) / Antλ(S ) / Homeo(S ) ; C ⋆ ⋆ ) , Ant(S2) ⋆ O ( Lin(S2) Each proper subgroup G ⊂ Homeo(S2) appearing in the diagram is a group satisfying (a combination of) symmetry conditions, with the symmetry conditions being (i) preserving circles (M¨obius(S2)), (ii) preserving geodesics (Lin(S2)), (iii) preserving the antipodal action (Ant(S2)) 2 2 and (iv) preserving area (Homeoλ(S )). For example, the group Rot(S ) preserves all these 2 symmetries and the group Antλ(S ) is the group that satisfies both symmetries (iii) and (iv), but not (i) and (ii). An arrow G −→ H between two groups G, H in the diagram is defined to be complete if G is maximal in H, in the sense that any closed group G ⊂ K ⊆ H that has the property that all the symmetries that G has, but that H does not have, are not respected by K, then it holds that K = H. We prove the following set of implications in terms of the subgroups in the diagram. Theorem A. Given the homogeneous transformation groups G ⊆ Homeo(S2) in the diagram, each arrow marked with ⋆ is complete and conversely each arrow not present between two groups in the diagram, other than Rot(S2) −→ Homeo(S2), corresponds to an intersection of groups already present in the diagram. CLOSED AND TRANSITIVE TRANSFORMATION GROUPS OF A SURFACE 3 A similar result in the setting of area-preserving homeomorphisms has been obtained recently by Le Roux in [8]. We state the following. Problem B. Classify all homogeneous transformation groups and their relations. Current work of the author includes a careful study of the properties of the isotopy subgroup of a homogeneous transformation group in Homeo(S2), and further relations in the diagram. Further, we remark the following. Call a closed and transitive group G ⊂ Homeo(M), with M the sphere, torus, or higher genus surface, an exotic group if the group G does not contain a continuous arc of homeomorphisms. It is an interesting problem to determine whether there exist such exotic closed and transitive groups. In the remainder, we first discuss several results about Lie group actions on a surface. After this section, we proceed with the main result about the spherical groups, and close the paper with a list of groups in the case of the torus and higher genus surfaces. 1.3. Lie group actions on a surface. Since Lie groups, both finite-dimensional and infinite- dimensional, are particularly well understood, a correspondence between homeomorphism groups and Lie group actions is of interest, and we detail how the groups described in this paper can be given the interpretation of a Lie group, modulo passing to a finite degree regular cover of the surface on which the closed and transitive group is defined. In the case of the sphere, the group Ant(S2) ⊂ Homeo(S2) is the group of homeomorphisms acting on the projective plane RP2 and so with the subgroups of Ant(S2). The pointwise classification of Lie groups by Mostow has been used by Belliart in [1] to classify the finite-dimensional Lie group actions of a surface without fixed points, which are the following (1) spherical case: the orthogonal action SO(3, R), the complex-projective action PGL(2, C), and the real-projective action PGL(3, R), (2) torus case: the circle-action (x,y) 7→ (x + α, y)mod Z2, (3) higher genus case: none. The possible pointwise GL(2, R) actions of a mapping is described by the Lie subgroups of the Lie group GL(2, R), namely (i) the orthogonal group O(2, R) ⊂ GL(2, R), (ii) the special linear group SL(2, R) ⊂ GL(2, R), (iii) the Borel subgroup U(2, R) ⊂ GL(2, R) of upper triangular matrices. On a given surface M, after passing to the C0-closure, these pointwise Lie group actions inte- grate globally to (i) the finite-dimensional group of conformal homeomorphisms, (ii) the infinite- dimensional group of area-preserving homeomorphisms, and (iii) the infinite-dimensional group of homeomorphisms that fiber over the circle S1, in other words, the skew-product homeomor- phisms of the form g(x,y) = (ϕ(x), ψ(x,y)). Acknowledgement. The author worked on the main ideas presented in this paper during a two year research position at IMPA, Rio de Janeiro, Brazil, in the years 2010 and 2011, and wishes to thank these institutions for their hospitality. Further, he would like to thank Etienne Ghys for a useful discussion about the ideas in this paper at IMPA, and Fabio Tal for several useful discussions. 4 FERRY H. KWAKKEL 2. The case of the sphere Closed and transitive subgroups of the homeomorphism group of the sphere that do not arise as extensions of the rotation group are called inhomogenous groups. These groups are equivalently characterized as closed and transitive groups that do not contain a compact and transitive kernel group. It is a natural question whether or not there exist such inhomogeneous groups on the sphere. On the torus, the group of Hamiltonian homeomorphisms is an example of such a inhomogeneous group. The homogeneous groups by definition, and the known inhomogeneous groups on the sphere, torus, and higher genus surfaces, are known to contain isotopies. Our focus in this section will be to consider homogeneous groups.