Annals of Mathematics, 161 (2005), 1–59
The Tits alternative for Out(Fn) II: A Kolchin type theorem
By Mladen Bestvina, Mark Feighn, and Michael Handel*
Abstract
This is the second of two papers in which we prove the Tits alternative for Out(Fn).
Contents 1. Introduction and outline
2. Fn-trees 2.1. Real trees 2.2. Real Fn-trees 2.3. Very small trees 2.4. Spaces of real Fn-trees 2.5. Bounded cancellation constants 2.6. Real graphs 2.7. Models and normal forms for simplicial Fn-trees 2.8. Free factor systems 3. Unipotent polynomially growing outer automorphisms 3.1. Unipotent linear maps 3.2. Topological representatives 3.3. Relative train tracks and automorphisms of polynomial growth 3.4. Unipotent representatives and UPG automorphisms 4. The dynamics of unipotent automorphisms 4.1. Polynomial sequences 4.2. Explicit limits 4.3. Primitive subgroups 4.4. Unipotent automorphisms and trees 5. A Kolchin theorem for unipotent automorphisms 5.1. F contains the suffixes of all nonlinear edges 5.2. Bouncing sequences stop growing 5.3. Bouncing sequences never grow
*The authors gratefully acknowledge the support of the National Science Foundation. 2 MLADEN BESTVINA, MARK FEIGHN, AND MICHAEL HANDEL
5.4. Finding Nielsen pairs 5.5. Distances between the vertices 5.6. Proof of Theorem 5.1 6. Proof of the main theorem References
1. Introduction and outline
Recent years have seen a development of the theory for Out(Fn), the outer automorphism group of the free group Fn of rank n, that is modeled on Nielsen- Thurston theory for surface homeomorphisms. As mapping classes have either exponential or linear growth rates, so free group outer automorphisms have either exponential or polynomial growth rates. (The degree of the polynomial can be any integer between 1 and n−1; see [BH92].) In [BFH00], we considered individual automorphisms with primary emphasis on those with exponential growth rates. In this paper, we focus on subgroups of Out(Fn) all of whose elements have polynomial growth rates. To remove certain technicalities arising from finite order phenomena, we restrict our attention to those outer automorphisms of polynomial growth ∼ n whose induced automorphism of H1(Fn; Z) = Z is unipotent. We say that such an outer automorphism is unipotent. The subset of unipotent outer auto- morphisms of Fn is denoted UPG(Fn) (or just UPG). A subgroup of Out(Fn) is unipotent if each element is unipotent. We prove (Proposition 3.5) that any polynomially growing outer automorphism that acts trivially in Z/3Z- homology is unipotent. Thus every subgroup of polynomially growing outer automorphisms has a finite index unipotent subgroup. The archetype for the main theorem of this paper comes from linear groups. A linear map is unipotent if and only if it has a basis with respect to which it is upper triangular with 1’s on the diagonal. A celebrated theorem of Kolchin [Ser92] states that for any group of unipotent linear maps there is a basis with respect to which all elements of the group are upper triangular with 1’s on the diagonal. There is an analogous result for mapping class groups. We say that a map- ping class is unipotent if it has linear growth and if the induced linear map on first homology is unipotent. The Thurston classification theorem implies that a mapping class is unipotent if and only if it is represented by a composition of Dehn twists in disjoint simple closed curves. Moreover, if a pair of unipotent mapping classes belongs to a unipotent subgroup, then their twisting curves cannot have transverse intersections (see for example [BLM83]). Thus every unipotent mapping class subgroup has a characteristic set of disjoint simple closed curves and each element of the subgroup is a composition of Dehn twists along these curves. THE TITS ALTERNATIVE FOR Out(Fn) II 3
Our main theorem is the analogue of Kolchin’s theorem for Out(Fn). Fix once-and-for-all a wedge Rosen of n circles and permanently identify its fun- damental group with Fn. A marked graph (of rank n) is a graph equipped with a homotopy equivalence from Rosen; see [CV86]. A homotopy equiva- lence f : G → G on a marked graph G induces an outer automorphism of the fundamental group of G and therefore an element O of Out(Fn); we say that f : G → G is a representative of O. Suppose that G is a marked graph and that ∅ = G0 G1 ··· GK = G is a filtration of G where Gi is obtained from Gi−1 by adding a single edge Ei. A homotopy equivalence f : G → G is upper triangular with respect to the filtration if each f(Ei)=viEiui (as edge paths) where ui and vi are closed paths in Gi−1. If the choice of filtration is clear then we simply say that f : G → G is upper triangular. We refer to the ui’s and vi’s as suffixes and prefixes respectively. An outer automorphism is unipotent if and only if it has a representative that is upper triangular with respect to some filtered marked graph G (see Section 3). For any filtered marked graph G, let Q be the set of upper triangular homotopy equivalences of G up to homotopy relative to the vertices of G.By Lemma 6.1, Q is a group under the operation induced by composition. There is a natural map from Q to UPG(Fn). We say that a unipotent subgroup of Out(Fn)isfiltered if it lifts to a subgroup of Q for some filtered marked graph. We denote the conjugacy class of a free factor F i by [[F i]]. If F 1 ∗F 2 ∗···∗ F k is a free factor, then we say that the collection F = {[[F 1]], [[F 2]],...,[[F k]]} is a free factor system. There is a natural action of Out(Fn) on free factor systems and we say that F is H-invariant if each element of the subgroup H fixes F. A (not necessarily connected) subgraph K of a marked real graph determines a free factor system F(K). A partial order on free factor systems is defined in subsection 2.8. We can now state our main theorem.
Theorem 1.1 (Kolchin theorem for Out(Fn)). Every finitely generated unipotent subgroup H of Out(Fn) is filtered. For any H-invariant free factor system F, the marked filtered graph G can be chosen so that F(Gr)=F for some filtration element Gr. The number of edges of G can be taken to be 3n − bounded by 2 1 for n>1.
It is an interesting question whether or not the requirement that H be finitely generated is necessary or just an artifact of our proof.
Question. Is every unipotent subgroup of Out(Fn) contained in a finitely generated unipotent subgroup? 4 MLADEN BESTVINA, MARK FEIGHN, AND MICHAEL HANDEL
Remark 1.2. In contrast to unipotent mapping class subgroups which are all finitely generated and abelian, unipotent subgroups of Out(Fn) can be quite large. For example, if G is a wedge of n circles, then a filtration on G corre- sponds to an ordered basis {e1,...,en} of Fn and elements of Q correspond to automorphisms of the form ei → aieibi with ai,bi ∈e1,...,ei−1. When n>2, the image of Q in UPG(Fn) contains a product of nonabelian free groups. This is the second of two papers in which we establish the Tits alternative for Out(Fn).
Theorem (The Tits alternative for Out(Fn)). Let H be any subgroup of Out(Fn). Then either H is virtually solvable, or contains a nonabelian free group. For a proof of a special (generic) case, see [BFH97a]. The following corollary of Theorem 1.1 gives another special case of the Tits alternative for Out(Fn). The corollary is then used to prove the full Tits alternative.
Corollary 1.3. Every unipotent subgroup H of Out(Fn) either contains a nonabelian free group or is solvable.
Proof. We first prove that if Q is defined as above with respect to a marked filtered graph G, then every subgroup Z of Q either contains a nonabelian free group or is solvable. Let i ≥ 0 be the largest parameter value for which every element of Z restricts to the identity on Gi−1.Ifi = K + 1, then Z is the trivial group and we are done. Suppose then that i ≤ K. By construction, each element of Z satisfies Ei → viEiui where vi and ui are paths (that depend on the element of Z)inGi−1 and are therefore fixed by every element of Z. The suffix map S : Z→Fn, which assigns the suffix ui to the element of Z, is therefore a homomorphism. The prefix map P : Z→Fn, which assigns the inverse of vi to the element of Z, is also a homomorphism. If the image of P×S: Z→Fn × Fn contains a nonabelian free group, then so does Z and we are done. If the image of P×Sis abelian then, since Z is an abelian extension of the kernel of P×S, it suffices to show that the kernel of P×Sis either solvable or contains a nonabelian free group. Upward induction on i now completes the proof. In fact, this argument shows that Z 3n − is polycyclic and that the length of the derived series is bounded by 2 1 for n>1. For H finitely generated the corollary now follows from Theorem 1.1. When H is not finitely generated, it can be represented as the increasing union of finitely generated subgroups. If one of these subgroups contains a nonabelian free group, then so does H, and if not then H is solvable with the 3n − length of the derived series bounded by 2 1. THE TITS ALTERNATIVE FOR Out(Fn) II 5
Proof of the Tits alternative for Out(Fn). Theorem 7.0.1 of [BFH00] asserts that if H does not contain a nonabelian free group then there is a finite index subgroup H0 of H and an exact sequence
1 →H1 →H0 →A→1 with A a finitely generated free abelian group and with H1 a unipotent sub- group of Out(Fn). Since H1 does not contain a nonabelian free group, by Corollary 1.3, H1 is solvable. Thus, H0 is solvable and H is virtually solvable.
In [BFH04] we strengthen the Tits alternative for Out(Fn) further by proving:
Theorem (Solvable implies abelian). A solvable subgroup of Out(Fn) has a finitely generated free abelian subgroup of index at most 35n2 .
Emina Alibegovi´c [Ali02] has since provided an alternate shorter proof. The rank of an abelian subgroup of Out(Fn)is≤ 2n − 3 for n>1 [CV86]. We reformulate Theorem 1.1 in terms of trees, and it is in this form that we prove the theorem. There is a natural right action of the automorphism group of Fn on the set of simplicial Fn-trees produced by twisting the action. See Section 2 for details. If we identify trees that are equivariantly isomorphic then this action descends to give an action of Out(Fn). A simplicial Fn-tree is nontrivial if there is no global fixed point. If T is a simplicial real Fn-tree with trivial edge stabilizers, then the set of conjugacy classes of nontrivial vertex stabilizers of T is a free factor system denoted F(T ). The reformulation is as follows.
Theorem 5.1. For every finitely generated unipotent subgroup H of Out(Fn) there is a nontrivial simplicial Fn-tree T with all edge stabilizers trivial that is fixed by all elements of H. Furthermore, there is such a tree with exactly one orbit of edges and if F is any maximal proper H-invariant free factor system then T may be chosen so that F(T )=F.
Such a tree can be obtained from the marked filtered graph produced by Theorem 1.1 by taking the universal cover and then collapsing all edges except for the lifts of the highest edge EK . For a proof of the reverse implication, namely that Theorem 5.1 implies Theorem 1.1, see Section 6. Along the way we obtain a result that is of interest in its own right. The necessary background material on trees may be found in Section 2, but also we give a quick review here. Simplicial Fn-trees may be endowed with metrics by equivariantly assigning lengths to edges. Given a simplicial real Fn-tree T and an element a ∈ Fn, the number T (a) is defined to be the infimum 6 MLADEN BESTVINA, MARK FEIGHN, AND MICHAEL HANDEL of the distances that a translates elements of T . It is through these length functions that the space of simplicial real Fn-trees is topologized. Again there is a natural right action of Out(Fn). We will work in the Out(Fn)-subspace T consisting of those nontrivial simplicial real trees that are limits of free actions.
Theorem 1.4 Suppose T ∈T and O∈UPG(Fn). There is an integer d = d(O,T) ≥ 0 such that the sequence {T Ok/kd} converges to a tree T O∞ ∈T.
This is proved in Section 4 as Theorem 4.22, which also contains an explicit description of the limit tree in the case that d(O,T) ≥ 1. Section 5 is the heart of the proof of Theorem 5.1. For notational sim- plicity, let us assume that H is generated by two elements, O1 and O2. Given T ∈T, let Elliptic(T ) be the subset of Fn consisting of elements fixing a point of T . Elements of Elliptic(T ) are elliptic. Choose T0 ∈T such that T0 has trivial edge stabilizers and such that Elliptic(T0)isH-invariant and maximal, i.e. such that if T ∈T has trivial edge stabilizers, if Elliptic(T )isH-invariant, and if Elliptic(T0) ⊂ Elliptic(T ), then Elliptic(T0) = Elliptic(T ). We prove that T0 satisfies the conclusions of Theorem 5.1 but not by a di- rect analysis of T0. Rather, we consider the “bouncing sequence” {T0,T1,T2, ···} T O∞ in defined inductively by Ti+1 = Ti i+1 where the subscripts of the outer automorphisms are taken mod 2. We establish properties of Ti for large i and then use these to prove that T0 is the desired tree. The key arguments in Section 5 are Proposition 5.5, Proposition 5.7, and Proposition 5.13. They focus not on discovering “ping-pong” dynamics (H may well contain a nonabelian free group), but rather on constructing an element in H of exponential growth. The connection to the bouncing sequence is as O∞O∞ O∞ follows. Properties of the tree Tk = T0 1 2 ... k−1 are reflected in the dy- O ON1 ON2 ONk−1 namics of the ‘approximating’ outer automorphism (k)= 1 2 ... k−1 where N1 N2 ··· Nk−1 1. We verify properties of Tk by proving that if the property did not hold, then O(k) would have exponential growth. After the breakthrough of E. Rips and the subsequent successful applica- tions of the theory by Z. Sela and others, it became clear that trees were the right tool for proving Theorem 1.1. Surprisingly, under the assumption that H is finitely generated (which is the case that we are concerned with in this paper and which suffices for proving that the Tits alternative holds), we only work with simplicial real trees and the full scale R-tree theory is never used. However, its existence gave us a firm belief that the project would succeed, and, indeed, the first proof we found of the Tits alternative used this theory. In a sense, our proof can be viewed as a development of the program, started by Culler-Vogtmann [CV86], to use spaces of trees to understand Out(Fn)in much the same way that Teichm¨uller space and its compactification were used by Thurston and others to understand mapping class groups. THE TITS ALTERNATIVE FOR Out(Fn) II 7
2. Fn-trees
In this section, we collect the facts about real Fn-trees that we will need. This paper will only use these facts for simplicial real trees, but we sometimes record more general results for anticipated later use. Much of the material in this section can be found in [Ser80], [SW79], [CM87], or [AB87].
2.1. Real trees. An arc in a topological space is a subspace homeomorphic to a compact interval in R. A point is a degenerate arc. A real tree is a metric space with the property that any two points may be joined by a unique arc, and further, this arc is isometric to an interval in R (see for example [AB87] or [CM87]). The arc joining points x and y in a real tree is denoted by [x, y]. A branch point of a real tree T is a point x ∈ T whose complement has other than 2 components. A real tree is simplicial if it is equipped with a discrete subspace (the set of vertices) containing all branch points such that the edges (closures of the components of the complement of the set of vertices) are compact. If the subspace of branch points of a real tree T is discrete, then it admits a (nonunique) structure as a simplicial real tree. The simplicial real trees appearing in this paper will come with natural maps to compact graphs and the vertex sets of the trees will be the preimages of the vertex sets of the graphs. For a real tree T , a map σ : J → T with domain an interval J is a path in T if it is an embedding or if J is compact and the image is a single point; in the latter case we say that σ is a trivial path. If the domain J of a path σ is compact, define the inverse of σ, denoted σ or σ−1,tobeσ ◦ ρ where ρ : J → J is a reflection. We will not distinguish paths in T that differ only by an orientation- preserving change of parametrization. Hence, every map σ : J → T with J compact is properly homotopic rel endpoints to a unique path [σ] called its tightening. If σ : J → T is a map from the compact interval J to the simplicial real tree T and the endpoints of J are mapped to vertices, then the image of [σ], if nondegenerate, has a natural decomposition as a concatenation E1 ···Ek where each Ei,1≤ i ≤ k, is a directed edge of T . The sequence E1 ···Ek is called the edge path associated to σ. We will identify [σ] with its associated edge path. This notation extends naturally if the domain of the path is a ray or the entire line and σ is an embedding. A path crosses an edge of T if the edge appears in the associated edge path. A path is contained in a subtree if it crosses only edges of the subtree. A ray in T is a path [0, ∞) → T that is an embedding.
2.2. Real Fn-trees. By Fn denote a fixed copy of the free group with basis {e1,...,en}. A real Fn-tree is a real tree equipped with an action of Fn by 8 MLADEN BESTVINA, MARK FEIGHN, AND MICHAEL HANDEL isometries. It is minimal if it has no proper Fn-invariant subtrees. If H is a subgroup of Fn then FixT (H) denotes the subset of T consisting of points that are fixed by each element of H.Ifa ∈ Fn, then FixT (a):=FixT (a). If X ⊂ T , then StabT (X) is the subgroup of Fn consisting of elements that leave X invariant. If x ∈ T , then StabT (x) := StabT ({x}). The symbol ‘[[·]]’ denotes ‘conjugacy class’. Define
Point(T ):={[[StabT (x)]] | x ∈ T, StabT (x) = 1} and
Arc(T ):={[[StabT (σ)]]
| σ is a nondegenerate arc in T, StabT (σ) = 1}.
The length function of a real Fn-tree T assigns to a ∈ Fn the number