TRANSACTIONS OF THE AMERICAN MATHEMATICAL SOCIETY Volume 361, Number 4, April 2009, Pages 2085–2107 S 0002-9947(08)04767-3 Article electronically published on November 5, 2008 THE COKERNEL OF THE JOHNSON HOMOMORPHISMS OF THE AUTOMORPHISM GROUP OF A FREE METABELIAN GROUP TAKAO SATOH Dedicated to Professor Shigeyuki Morita on the occasion of his 60th birthday Abstract. In this paper, we determine the cokernel of the k-th Johnson ho- momorphisms of the automorphism group of a free metabelian group for k ≥ 2 and n ≥ 4. As a corollary, we obtain a lower bound on the rank of the graded quotient of the Johnson filtration of the automorphism group of a free group. Furthermore, by using the second Johnson homomorphism, we determine the image of the cup product map in the rational second cohomology group of the IA-automorphism group of a free metabelian group, and show that it is iso- morphic to that of the IA-automorphism group of a free group which is already determined by Pettet. Finally, by considering the kernel of the Magnus repre- sentations of the automorphism group of a free group and a free metabelian group, we show that there are non-trivial rational second cohomology classes of the IA-automorphism group of a free metabelian group which are not in the image of the cup product map. 1. Introduction Let G be a group and ΓG(1) = G,ΓG(2), ... its lower central series. We denote by Aut G the group of automorphisms of G.Foreachk ≥ 0, let AG(k)bethe group of automorphisms of G which induce the identity on the quotient group G/ΓG(k + 1). Then we obtain a descending central filtration Aut G = AG(0) ⊃AG(1) ⊃AG(2) ⊃··· of Aut G, called the Johnson filtration of Aut G. This filtration was introduced in 1963 with a pioneer work by S. Andreadakis [1]. For each k ≥ 1, set LG(k):= k ab ΓG(k)/ΓG(k +1)andgr (AG)=AG(k)/AG(k +1).LetG be the abelianization of G. Then, for each k ≥ 1, an Aut Gab-equivariant injective homomorphim k ab τk :gr (AG) → HomZ(G , LG(k +1)) is defined. (For a definition, see Subsection 2.1.2.) This is called the k-th Johnson homomorphism of Aut G. Historically, the study of the Johnson homomorphism was begun in 1980 by D. Johnson [17]. He studied the Johnson homomorphism of a mapping class group of a closed oriented surface, and determined the abelianization Received by the editors May 17, 2007. 2000 Mathematics Subject Classification. Primary 20F28; Secondary 20J06. Key words and phrases. Automorphism group of a free metabelian group, Johnson homomor- phism, second cohomology group, Magnus representation. c 2008 American Mathematical Society 2085 License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 2086 TAKAO SATOH of the Torelli group. (See [18].) There is a broad range of remarkable results for the Johnson homomorphisms of a mapping class group. (For example, see [14] and [25].) M Let Fn be a free group of rank n with basis x1,...,xn and Fn the free metabelian M group of rank n.NamelyFn is the quotient group of Fn by the second derived series M [[Fn,Fn], [Fn,Fn]] of Fn. ThenbothabelianizationsofFn and Fn are a free abelian group of rank n, denoted by H. Fixing a basis of H induced from x1,...,xn,wecan ab M identify Aut G with GL(n, Z)forG = Fn and Fn . For simplicity, throughout this L A k A L A paper we write Γn(k), n(k), n(k)andgr ( n)forΓFn (k), Fn (k), Fn (k)and k A M LM AM k AM gr ( Fn ) respectively. Similarly, we write Γn (k), n (k), n (k)andgr( n ) k for Γ M (k), L M (k), A M (k)andgr(A M ) respectively. The first aim of this Fn Fn Fn Fn paper is to determine the GL(n, Z)-module structure of the cokernel of the Johnson M ≥ homomorphisms τk of Aut Fn for n 4 as follows: Theorem 1. For k ≥ 2 and n ≥ 4, Tr M → k AM −τ→k ∗ ⊗ LM −−−k→ k → 0 gr ( n ) H Z n (k +1) S H 0 is a GL(n, Z)-equivariant exact sequence. k M Here S H is the symmetric product of H of degree k,andTrk is a certain GL(n, Z)-equivariant homomorphism called the Morita trace introdued by S. Morita [24]. (For definition, see Subsection 3.2.) k From Theorem 1, we can give a lower bound on the rank of gr (An)fork ≥ 2 and n ≥ 4. The study of the Johnson filtration of Aut Fn was begun in the 1960s k by Andreadakis [1] who showed that for each k ≥ 1andn ≥ 2, gr (An)isafree abelian group of finite rank, and that A2(k) coincides with the k-th lower central series of the inner automorphism group Inn F2 of F2. Furthermore, he [1] computed k k rankZ gr (A2) for all k ≥ 1. However, the structure of gr (An) for general k ≥ 2 and n ≥ 3 is much more complicated. Set τk,Q = τk ⊗ idQ, and call it the k-th rational Johnson homomorphism. For any Z-module M,wedenoteM ⊗Z Q by the Q symbol obtained by attaching a subscript Q to M, like MQ and M .Forn ≥ 3, 2 A the GL(n, Z)-module structure of grQ( n) is completely determined by Pettet [31]. 3 A ≥ ≥ In our previous paper [33], we determined those of grQ( n)forn 3. For k 4, k A the GL(n, Z)-module structure of grQ( n) is not determined. Furthermore, even its dimension is also unknown. → M Let νn :AutFn Aut Fn be a natural homomorphism induced from the action M of Aut Fn on Fn . By notable works due to Bachmuth and Mochizuki [5], it is known that νn is surjective for n ≥ 4. They [4] also showed that ν3 is not surjective. In k A → k AM Subsection 3.1, we see that the homomorphism νn,k :gr ( n) gr ( n ) induced from νn is also surjective for n ≥ 4. Hence we have Corollary 1. For k ≥ 2 and n ≥ 4, n + k − 1 n + k − 1 rank (grk(A )) ≥ nk − . Z n k +1 k We should remark that in general, equality does not hold, since for instance 3 4 2 rankZ gr (An)=n(3n − 7n − 8)/12, which is not equal to the right hand side of the inequality above. Next, we consider the second cohomology group of the IA-automorphism group of the free metabelian group. Here the IA-automorphism group IA(G) of a group G License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use THE COKERNEL OF THE JOHNSON HOMOMORPHISMS 2087 is defined to be a group which consists of automorphisms of G which trivially act on A M the abelianization of G. By the definition, IA(G)= G(1). We write IAn and IAn M ∗ for IA(Fn)andIA(Fn ) for simplicity. Let H := HomZ(H, Z) be the dual group M ≥ of H. Then we see that the first homology group of IAn for n 4 is isomorphic ∗ ⊗ 2 → M to H Z Λ H in the following way. Let νn,1 :IAn IAn be the restriction of νn to IAn. Bachmuth and Mochizuki [5] showed that νn,1 is surjective for n ≥ 4. This fact sharply contrasts with their previous work [4], which shows there are M infinitely many automorphisms of IA3 which are not contained the image of ν3,1. On the other hand, by independent works of Cohen-Pakianathan [9, 10], Farb [11] ∼ ∗ 2 and Kawazumi [19], H1(IAn, Z) = H ⊗Z Λ H for n ≥ 3. Since the kernel of νn,1 is M M ∼ ∗ ⊗ 2 contained in the commutator subgroup of IAn ,wehaveH1(IAn , Z) = H Z Λ H for n ≥ 4. (See Subsection 2.3.) In general, however, there are few results for M computation of the (co)homology groups of IAn of higher dimensions. In this paper we determine the image of the cup product map in the rational second cohomology M group of IAn , and show that it is isomorphic to that of IAn, using the second 2 1 2 Johnson homomorphism. Namely, let ∪Q :ΛH (IAn, Q) → H (IAn, Q)and ∪M 2 1 M → 2 M Q :Λ H (IAn , Q) H (IAn , Q) be the rational cup product maps of IAn and M IAn respectively. In Subsection 4.2, we show ≥ ∗ ∪M → ∪ Theorem 2. For n 4, νn,1 :Im( Q ) Im( Q) is an isomorphism. Here we should remark that the GL(n, Z)-module structure of Im(∪Q)iscom- pletely determined by Pettet [31] for any n ≥ 3. M Now, for the study of the second cohomology group of IAn ,itisalsoanimpor- ∪M tant problem to determine whether the cup product map Q is surjective or not. For the case of IAn, it is still not known whether ∪Q is surjective or not. In the ∪M last section, we prove that the rational cup product map Q is not surjective for n ≥ 4. It is easily seen that Kn is an infinite subgroup of IAn,sinceKn contains the second derived series of the inner automorphism group of a free group Fn.The structure of Kn is, however, very complicated. For example (finitely or infinitely many) generators and the abelianization of Kn are still not known.