The Base Group of a Finite Wreath Product

PROCEEDINGS OF THE AMERICAN MATHEMATICAL SOCIETY Volume 00, Number 0, Pages 000–000 S 0002-9939(XX)0000-0

THE BASE GROUP OF A FINITE WREATH PRODUCT

BEN BREWSTER, D. S. PASSMAN, AND ELIZABETH WILCOX

Abstract. Let W = G o H denote the permutational wreath product of the ﬁnite group G by H ⊆ Symn, and suppose that H is transitive on {1, 2, . . . , n} or at least that it acts faithfully on all its orbits. Our main result determines all those wreath products W which have a non-characteristic base subgroup B = Q G. In addition, we brieﬂy discuss those automorphisms of W that move this base.

1. Introduction

In the following, let G and H be finite groups with H ⊆ Symn, and we let W = G o H denote the permutational wreath product of G by H. Then W has a normal subgroup B = Q G, its base group, which is a direct product of n copies of G, and W = B o H where H acts on B by naturally permuting its n factors. This wreath product construction provides a rich source of examples of finite groups and intuition regarding general group-theoretic structures. For example, [Hu, Satz I.15.9] shows that any extension group can be embedded in a suitable wreath product. Since these groups occur so frequently, it is certainly of interest to determine their automorphism groups. Indeed, there is a rich literature on this subject when the base subgroup B is characteristic in W . See for example [B], [Ha], and [M]. The question of when the base group is characteristic has also been considered. For simplicity, we state the following two results in the context of finite groups. To start with, we say that a group is odd-dihedral if it is the split extension A o hxi, where A is an abelian group of odd order and where x has order 2 and acts on A by inverting all its elements. Furthermore, the wreath product G o H is said to be standard if H is a regular subgroup of Symn. Theorem 1.1. [N, Theorem 9.1]. Let W = G o H be a standard wreath product with G and H finite. Then the base B = Q G is characteristic in W unless G is ∼ an odd-dihedral group, n = 2, and H = Sym2 = C2. Under less stringent hypotheses, there is the somewhat weaker Theorem 1.2. [B, Theorem 1]. If the base group of the wreath product W = G o H is not characteristic in W , then G is either an elementary abelian 2-group or a dihedral group, and H has a non-trivial normal elementary abelian 2-subgroup. Actually, the proof of Theorem 1.2 seems to be incomplete, and indeed there is some confusion in the mathematical community over this issue. While the result is stated again in [KBF] without proof, [M, page 101] asserts that no such general

2000 Mathematics Subject Classiﬁcation. 20E22, 20E36.

c XXXX American Mathematical Society 1 2 BEN BREWSTER, D. S. PASSMAN, AND ELIZABETH WILCOX result is known. In this paper, we settle the question definitively in the case of finite groups. Specifically, we prove Theorem 1.3. Let W = G o H be the wreath product of the finite group G by H ⊆ Symn and assume that H is transitive or at least that it acts faithfully on each orbit. Then the base group B = Q G fails to be characteristic in W if and only if G is an odd-dihedral group and H = C2 o Y , where C2 = Sym2, n is even, and Y ⊆ Symn/2. In this case, the base group of

W = G o H = G o (C2 o Y ) = (G o C2) o Y Q given by N = (G o C2) is characteristic in W . This will be proved in Section 3. Note that the latter fact about N being characteristic allows one to compute Aut(W ) by switching from the non-characteristic base B of W = G o H to the characteristic base N of W = (G o C2) o Y . Much of this research was begun during the second author’s recent visit to the Mathematics Department of SUNY-Binghamton. The results of this paper will constitute part of the third author’s forthcoming dissertation.

2. Arbitrary Actions

Again, G and H are finite groups with H ⊆ Symn, and W = G o H denotes the permutational wreath product of G by H. In particular, W is the split extension Q W = B o H where the base subgroup B = G is a direct product of n copies of G and where H acts on B by naturally permuting its n factors. For convenience, we write g = |G| and we say that x is a coordinate element of B if it is contained in precisely one of the G direct factors. Obviously, B is generated by its coordinate elements. Of course, if either G or H is the identity group, then B = h1i or W is clearly characteristic in W . Thus it suffices to assume throughout that G and H are both nonidentity groups. As a consequence, H acts faithfully on B, and indeed H =∼ W/B acts faithfully as a permutation group on the G factors of B. Our first goal is to show that O2(B), the smallest normal subgroup of B with factor group a 2-group, is characteristic in W . To this end, for any x ∈ W , let κ(x) denote the number of W -conjugates of x that do not commute with x, and set K = {x ∈ W | κ(x) < g}. Obviously, K is a characteristic subset of W . Lemma 2.1. With the above notation, we have i. If x is a coordinate element of B, then x ∈ K. ii. If x ∈ K, then x2 ∈ B. Proof. (i) If x is a coordinate element of B, then clearly all W -conjugates xw of x are also coordinate elements. If x and xw lie in different factors of B, then x and xw commute. Thus the conjugates counted in κ(x) are all contained in a single factor G containing x. Indeed, this subset is then contained in G \{x} and has size at most g − 1. (ii) Suppose x has a t-cycle, say (1 2 ··· t), in its permutation action on the factors of B, and let this correspond to the direct factor T of B given by T = G1 × G2 × · · · × Gt. If u ∈ CT (x), then the G1-entry of u uniquely determines u. Indeed, if u = (u1, u2, . . . , ut) ∈ T and if x = (x1, x2, . . . , xn)τ ∈ BH, then THE BASE GROUP OF A FINITE WREATH PRODUCT 3

x xt x1 x2 xt−1 x1 u = (ut , u1 , u2 , . . . , ut−1 ), and thus u ∈ CT (x) if and only if u2 = u1 , u3 = x2 xt−1 xt u2 , . . . , ut = ut−1 and u1 = ut . Thus |CT (x)| ≤ g and hence x has |T : CT (x)| ≥ gt−1 conjugates under T . Now if v ∈ T , then xv = x[x, v] and the commutator [x, v] is contained in T since T/ hT, xi. Thus the set of commutators [x, T ] has size ≥ gt−1. Furthermore, x commutes with xv if and only if it commutes with [x, v]. It follows that conjugating x by elements of T gives rise to at least [x, T ] \ CT (x) elements counted in κ(x), and hence κ(x) ≥ gt−1 − g. Finally, if t ≥ 3, then gt−1 − g ≥ g2 − g ≥ g, since g ≥ 2. Thus κ(x) ≥ g, contradicting the fact that x ∈ K. We conclude that in its permutation action on the factors of B, x can only be a product of 2-cycles. In particular, x2 acts trivially 2 and hence x ∈ B. With this, we can prove Proposition 2.2. If B is the base group of the wreath product W = G o H, then 2 Q 2 O (B) = O (G) is characteristic in W . Proof. Let S = hx2 | x ∈ Gi. Then S/G with G/S being an elementary abelian 2-group. Now deﬁne K = hx2 | x ∈ Ki, so that K is a characteristic subgroup of W . By part (ii) of the previous lemma, we see that K ⊆ B, and by part (i), we have K ⊇ S ×S ×· · ·×S. Thus, Q S ⊆ K ⊆ B = Q G, and since G/S is a 2-group, 2 2 so is B/K. It follows that O (B) = O (K) is characteristic in W . We will sharpen this result below. To start with, recall that a group is odd- dihedral if it is of the form A o hxi, where A is an abelian group of odd order and where x has order 2 and acts on A by inverting all its elements. The two results stated below are standard group-theoretic facts. We include their elementary proofs for the convenience of the reader. Of course, part (i) is a special case of the existence of Frobenius kernels. Lemma 2.3. Let D be a ﬁnite group. i. If D contains a self-centralizing subgroup of order 2, then D is odd-dihedral. ii. If more than 3/4 of the elements of D have order ≤ 2, then D is an elementary abelian 2-group. Proof. (i) Let x be an element of order 2 in D with hxi self-centralizing. If X denotes the conjugacy class of x, then |X| = d/2, where d = |D|. If A = D \ X, then |A| = d/2 and 1 ∈ A. The goal is to show that A is a subgroup of D. To this end, let u ∈ X and suppose that uX ∩ X 6= ∅. Then there exists v ∈ X with uv ∈ X, so u2 = v2 = (uv)2 = 1, and hence u and v commute. But u is a conjugate of x, so hui is self-centralizing. Therefore v = u and uv = u2 = 1, contradicting the fact that uv ∈ X. It follows that uX ∩ X = ∅ for all u ∈ X, and hence uX ⊆ A. By size considerations, we have uX = A and hence XX = A. Similarly, Xu = A, so AA = Xu·uX = XX = A, and A is indeed a subgroup of index 2 in D. Then A/D and D = A o hxi. Finally, using A = xX, we see that every element of A is of the form a = xy for some y ∈ X. Thus ax = yx = (xy)−1 = a−1, so x inverts all elements of A. It follows that A is abelian and, since hxi is self-centralizing, we see that A contains no element of order 2. Thus |A| is odd and D is odd-dihedral. (ii) Let E = {x ∈ D | x2 = 1} and again write d = |D|. If u ∈ E, then by assumption |E| = |uE| > (3/4)·d, so |uE ∩ E| > d/2. In other words, there exists 4 BEN BREWSTER, D. S. PASSMAN, AND ELIZABETH WILCOX

V ⊆ E with |V | > d/2 and with uV ⊆ E. Now if v ∈ V , then u2 = v2 = (uv)2 = 1 and hence u and v commute. Thus CD(u) ⊇ V and, since CD(u) is a subgroup of D and |V | > d/2, we have CD(u) = D. We have therefore shown that E is central in D and clearly E generates D since |E| > d/2. With this, we conclude easily that D is an elementary abelian 2-group. In order to prove the next result, we tighten the bound for κ(x) given in the definition of K. Indeed, we set L = {x ∈ W | κ(x) < g/4}. Again, L is a characteristic subset of W containing 1, and clearly L ⊆ K. Proposition 2.4. If the base group B = Q G of the wreath product W = G o H is not characteristic in W , then G is either an elementary abelian 2-group or it is odd-dihedral. Proof. Let L = hx | x ∈ Li. Since L is a characteristic subgroup of W and, since B is not characteristic, we have B 6= L. Thus either B 6⊆ L or L 6⊆ B, and we consider these two cases separately. Case 1. If B 6⊆ L, then G is odd-dihedral. Proof. Since B 6⊆ L, there must exist some coordinate element x of B with x∈ / L and hence x∈ / L. As we observed, all W -conjugates xw of x are coordinate elements of B and hence each belongs to a single factor of B. Furthermore, if x and xw lie in different factors, then x and xw commute. Thus the conjugates counted in κ(x) are all contained in the single factor G of B that contains x. In addition, it is easy to see that all W -conjugates of x contained in G are in fact G-conjugates of x. Thus, the number of W -conjugates of x contained in G is precisely |G : CG(x)|. Since x commutes with itself, we see that κ(x) < |G : CG(x)|. On the other hand, since x∈ / L, we have κ(x) ≥ g/4. Thus |G : CG(x)| > |G|/4 and |CG(x)| < 4. In other words, |CG(x)| = 2 or 3, so x has order 2 or 3 and CG(x) = hxi. If x has order 2 then, by part (i) of the previous lemma, G is odd-dihedral, and we are done. Thus, it suffices to assume that all coordinate elements of B of order 2 are contained in L. In particular, x must have order 3. −1 If NG(hxi) = hxi, then x is not conjugate to x . Now x has g/3 conjugates and so does x−1. Since O2(G) contains all elements of G of odd order, this yields |O2(G)| ≥ 2g/3. Thus G = O2(G) and B = O2(B) is characteristic in W by Proposition 2.2, a contradiction. We must therefore have NG(hxi) properly larger ∼ than hxi and, since |CG(x)| = 3, we conclude that NG(hxi) = Sym3. But then x = uv is a product of two elements of order 2 and, since u, v ∈ L, we have x ∈ L, again a contradiction. We conclude that G must be odd-dihedral in this case. Case 2. If L 6⊆ B, then G is an elementary abelian 2-group. Proof. Write E = {u ∈ G | u2 = 1} and let |E| = e. Now, by assumption, there exists an element x ∈ L with x∈ / B. Thus x acts nontrivially as a permutation on the factors of B and, since L ⊆ K, Lemma 2.1(ii) implies that x has only 2- cycles and 1-cycles in its cycle decomposition with at least one 2-cycle. Suppose T = G × G is a direct factor of B corresponding to one such 2-cycle. We take a closer look at the action of x on T . THE BASE GROUP OF A FINITE WREATH PRODUCT 5

For convenience, to study this action, we write x = (u, v)τ where τ ∈ H is the transposition interchanging the ﬁrst and second G factors, and where (u, v) ∈ T is the T -component of the B-part of x in W = B o H. Now if (r, s) ∈ T , then (r, s)x = (ru, sv)τ = (sv, ru).

v u Thus (r, s) ∈ CT (x) if and only if s = r and r = s. As usual, this shows that r determines s. Furthermore, r = sv = (ru)v = ruv, so the possibilities for r are limited to elements of CG(uv). In particular, if uv is not central in G, then |CT (x)| ≤ |CG(uv)| ≤ g/2 and x 2 has |T : CT (x)| ≥ g /(g/2) = 2g conjugates under T . Furthermore, by considering the commutators [x, T ] ⊆ T , as in the proof of Lemma 2.1(ii), we see that at most |CT (x)| ≤ g/2 of these can commute with x. Thus κ(x) ≥ 2g − (g/2) = 3g/2, a contradiction since x ∈ L. Thus, uv is central in G. It follows that if r ∈ G is u v uv arbitrary and if we set s = r , then s = r = r, so (r, s) ∈ CT (x). Next, we need to understand what the commutators in [x, T ] look like. These are of course given by −1 −1 −v −u −v −u [x, (r, s)] = x (r, s) x·(r, s) = (s , r )(r, s) = (s r, r s) = (r0, s0). v Thus, if this commutator also belongs to CT (x), then r0 = s0, so −v v −u v −uv v −1 v −1 s r = r0 = s0 = (r s) = r s = r s = r0 . u In other words, r0 ∈ E. Thus all elements of the set [x, T ] ∩ CT (x) look like (r0, r0 ) with r0 ∈ E, and therefore this set has size at most e = |E|. 2 Since [x, T ] has size |T : CT (x)| = g /g = g, it follows that

κ(x) ≥ [x, T ] − [x, T ] ∩ CT (x) ≥ g − e. But x ∈ L, so g/4 > κ(x) ≥ g − e and hence e > 3g/4. We can now conclude from Lemma 2.3(ii) that G is an elementary abelian 2-group, as required.

This clearly completes the proof of the proposition.

Notice that, if B is not characteristic in W , then O2(B) = O(B) is the largest normal subgroup of B of odd order. As an immediate consequence, we have Corollary 2.5. If the base group B = Q G of the wreath product W = G o H is not characteristic in W , then W has a characteristic subgroup M properly larger than B such that M/B is generated by elementary abelian normal 2-subgroups of W/B.

2 Q 2 Proof. In view of Proposition 2.1, O (B) = O (G) is characteristic in W and hence properly smaller than B. Since W/O2(B) = (G/O2(G)) o H with base group B/O2(B), it now clearly suﬃces to assume that O2(B) = h1i, and thus by Propo- sition 2.4, G and B are elementary abelian 2-groups. Now let M be generated by all the elementary abelian normal 2-subgroups of W . Then M is characteristic in W , M ⊇ B and M/B is generated by elementary abelian normal 2-subgroups of W/B. Since B is not characteristic in W , we must have M 6= B. We now come to the main result of this section. We oﬀer it in a form more precise than what is really needed here because future applications may require these additional details. In the following, we will think of a group of order 2 as being an odd-dihedral group, with odd part having order 1. 6 BEN BREWSTER, D. S. PASSMAN, AND ELIZABETH WILCOX

Theorem 2.6. Let W = G o H be the wreath product of the group G by H ⊆ Symn. If the base group B = Q G is not characteristic in W , then G is an odd-dihedral group. Furthermore, if σ is an automorphism of W with B 6= Bσ, then BBσ = B o Hσ = G o Hσ, where Hσ is a nonidentity normal elementary abelian 2-subgroup of H that is generated by commuting transpositions. Moreover, Hσ moves all points that are contained in faithful orbits of H. Proof. Let us first assume that G is an elementary abelian 2-group of order g. Then |B| = gn and B is a self-centralizing normal elementary abelian 2-subgroup of W . Since B is not characteristic in W , there exists an automorphism σ of W with C = Bσ 6= B, and fix any such σ. Then C is also a self-centralizing elementary abelian normal 2-subgroup of W of order gn. Notice that C = C/(C ∩ B) is a ∼ nonidentity normal abelian subgroup of W/B = H ⊆ Symn. Say C has t orbits O1, O2,..., Ot and let these orbits have sizes m1, m2, . . . , mt, respectively. Then m1 + m2 + ··· + mt = n. Furthermore, we cannot have all mi = 1, since otherwise C acts trivially, so C = h1i and C = B. We now obtain an upper bound for |C|, and we start with |C ∩ B|. We know that C is self-centralizing, so CW (C) = C and hence CB(C) = CW (C)∩B = C ∩B. Furthermore, C permutes the factors of B = Q G in t orbits. Since any element of CB(C) is uniquely determined by its first G-coordinate in each of the t orbits, it is t t clear that |C ∩ B| = |CB(C)| ≤ |G| = g .

Next, let COi denote the subgroup of C ﬁxing all points in Oi. Then C/COi is a transitive abelian group of permutations on Oi, so C/CO is regular in this action. i T Q Hence |C : CO | = |C/CO | = |Oi| = mi, and it follows that |C : CO | ≤ mi. T i i T i i i But CO is the subgroup of C ﬁxing all points, so CO = C ∩ B, and we i i Q i i conclude that |C : C ∩ B| ≤ i mi. t Q Q It follows that |C| = |C ∩ B|·|C : C ∩ B| ≤ g i mi = i g·mi and hence

Y n Y mi g·mi ≥ |C| = g = g . i i

mi Q In particular, if we set fi = g·mi/g , then i fi ≥ 1. Notice that, if mi = 1, then fi = 1. Thus, since not all mi are equal to 1, there must exist some mi ≥ 2 with mi−1 mi−1 fi ≥ 1. For any such mi, we have mi ≥ g ≥ 2 . Hence, we see first that mi = 2 and then that g = 2 and fi = 1. We conclude that |G| = g = 2 and G is odd-dihedral, as required. Furthermore, we see that no fi is strictly larger than 1. Thus, the fi are all equal to 1, and we conclude that each mi is either 1 or 2. Q Since i fi = 1, it follows that the inequalities above must all be equalities, and Q T in particular, |C| = |C : C ∩ B| = i mi. Since |C : COi | = mi and i COi = h1i, this tells us that the latter intersection describes a direct sum decomposition of C. T More precisely, if we let C 0 = C denote the subgroup of C fixing all points Oi j6=i Oj Q not in the orbit O , then C = C 0 . Note that |C 0 | = m , so C 0 is the identity i i Oi Oi i Oi if mi = 1 and is generated by a transposition on Oi if mi = 2. Since H =∼ W/B ⊇ C = C/(C ∩ B), the result essentially follows in this case, ∼ σ with Hσ 6= h1i being the subgroup of H corresponding to C = BC/B = BB /B. Clearly, BC = B o Hσ = G o Hσ. Furthermore, if H is faithful on any orbit Ω ⊆ {1, 2, . . . , n}, then h1i= 6 Hσ /H can fix no point in that orbit. Finally, if G is not an elementary abelian 2-group, then by Proposition 2.4, G is 2 Q 2 odd-dihedral. Furthermore, by Proposition 2.1, O (B) = O (G) is characteristic in W , and W/O2(B) = (G/O2(G)) o H with base group B/O2(B). Now let σ be THE BASE GROUP OF A FINITE WREATH PRODUCT 7 an automorphism of W that does not stabilize B. Then Bσ contains the charac- 2 σ 2 2 2 σ teristic subgroup O (B), so (BB )/O (B) = B/O (B) B/O (B) and σ is an automorphism of W/O2(B) moving its base group. Since G/O2(G) is a group of σ 2 2 order 2, the preceding work implies that (BB )/O (B) = (B/O (B)) o Hσ, where Hσ is a nonidentity normal subgroup of H with the appropriate properties. Thus σ again BB = B o Hσ = G o Hσ, and the result follows.

3. Transitive Actions The results of the last section were quite general in that no assumption was made on the nature of the permutation action of H ⊆ Symn. As is to be expected, more can be said about the structure of W when we have additional information on this action. For example, one obvious consequence of Theorem 2.6 is

Corollary 3.1. Let W = GoH be the wreath product of the group G by H ⊆ Symn. Q If H is contained in Altn, then the base group B = G is characteristic in W . Furthermore, we have the following extension of Theorem 1.1.

Corollary 3.2. If H ⊆ Symn is either semiregular or primitive, and if the base Q ∼ group B = G is not characteristic in W = G o H, then n = 2, H = C2 = Sym2, and G is odd-dihedral. Proof. By Theorem 2.6, G is odd-dihedral and H has a normal abelian subgroup h1i= 6 Hσ generated by transpositions. If H is semiregular, then Hσ is certainly semiregular. On the other hand, if H is primitive, then Hσ is a transitive abelian subgroup, and hence Hσ is regular. But Hσ contains a transposition, so semiregu- ∼ larity or regularity implies that n = 2 and hence H = C2 = Sym2. Our next goal is to study the situation when H is transitive, or at least when H acts faithfully on all its orbits. This will, of course, lead to our main result. To start with, we need

Lemma 3.3. Let T be an elementary abelian 2-subgroup of S = Symn that moves all points and is generated by transpositions. Then n = 2m is even, |T | = 2m and

NS(T ) = T o Y = C2 o Y, ∼ where Y = Symm transitively permutes the generators of T by conjugation, Fur- thermore, C2 = Sym2 is generated by any transposition in T . Proof. Since T is a 2-group ﬁxing no point, if follows that all its orbits are even and hence n = 2m is even. Furthermore, we can clearly assume that T is generated by the m commuting transpositions (1 m + 1), (2 m + 2),..., (m m + m) and hence m |T | = 2 . Let use write ∆i = {i, m + i} for i = 1, 2, . . . , m, so that these subsets form a partition of Ω = {1, 2, . . . , n}. Now it is easy to see that the above listed transpositions are the only transpositions contained in T . Thus Q = NS(T ) permutes these generators by conjugation. Indeed, Q is the set of all permutations that permute these generators. Note also y that, for y ∈ Symn, we have (i m + i) = (j m + j) if and only if ∆iy = ∆j. In particular, Q is precisely the set of all elements of Symn that permute the blocks ∆1, ∆2,..., ∆m, and we let the homomorphism θ : Q → Symm describe this permutation action. 8 BEN BREWSTER, D. S. PASSMAN, AND ELIZABETH WILCOX

∼ Now let Y = Symm be the subgroup of S = Symn consisting of all permutations that act the same on {1, 2, . . . , m} and on {m + 1, m + 2, . . . , m + m}. Specifically, if y ∈ Symm acts on {1, 2, . . . , m}, then y ∈ Y if and only if iy = j implies that (m + i)y = m + j. Now, it is clear that Y permutes the blocks ∆1, ∆2,..., ∆m, that it is faithful in this action, and that it acts as the full symmetric group on the set of blocks. In particular, Y ⊆ Q and θ(Y ) = θ(Q). Furthermore, the kernel of θ is clearly generated by all transpositions (i m + i), and hence ker θ = T . It follows ∼ that TY = Q and T ∩ Y = h1i, so Q = T o Y , where Y = Symm transitively permutes the m generators of T by conjugation. With this, we see that Q = C2 o Y , where C2 is any subgroup of T generated by a single transposition. In order to obtain both directions of our main result, we will need to show that base groups of certain wreath products can move. For this, it will be necessary to prove that certain automorphisms exist. The following few comments are standard. Suppose W = V o D is a semidirect product and let σ : D → D and τ : V → V be automorphisms of the two defining subgroups. Then we let θ : W → W be given by θ(vd) = τ(v)σ(d) for all v ∈ V and d ∈ D, so that θ extends both σ and τ. The question of interest here is to determine when θ defines an automorphism of W . Since θ is certainly a one-to-one correspondence, we need only check to see when multiplication is preserved. −1 −1 d1 −1 −1 Since v1d1 ·v2d2 = v1v2 ·d1 d2 , we see that θ preserves multiplication if and d1 σ(d1) only if τ(v2 ) = τ(v2) , or equivalently if and only if (∗) τ(vd) = τ(v)σ(d) for all d ∈ D and v ∈ V . Note that if (∗) holds for all v ∈ V and for two elements of D, then it holds for the product of the latter two elements. Thus, it suffices to check (∗) on generators of D. Furthermore, for each such generator, since both sides of (∗) are multiplicative in v, we need only check the equation on generators of V . In other words, θ is an automorphism of W if and only if (∗) holds for all generators d ∈ D and all generators v ∈ V . ∼ The groups of interest to us have D = D8, the dihedral group of order 8. Here 2 2 4 D8 = gp h x, t | x = 1, t = 1, (xt) = 1 i and, of course, z = (xt)2 is the unique nonidentity central element of this group.

Lemma 3.4. Let W = V o D8 with V an abelian group of odd order, and let D8 z −1 be described as above with hzi = Z(D8). Assume that v = v for all v ∈ V . x x −1 i. If V1 = {v ∈ V | v = v} and V2 = {v ∈ V | v = v }, then V = V1 × V2 t ∼ is the direct product of these two subgroups with V1 = V2. Thus V1 = V2 as abelian groups, with isomorphism class uniquely determined by V . ii. Any automorphism σ of D8 extends to an automorphism θ of W . Proof. Note that hx, zi = {1, x, y, z} is a normal elementary abelian subgroup of t D = D8 of order 4, with y = xz = x . (i) Since V is an abelian group of odd order and x has order 2, it follows that V = V1 × V2 with subgroups V1 and V2 as given above. Of course, V1 = CV (x). Furthermore, since z inverts all elements of V and since y = xz = zx, we see that t t t t V2 = CV (y). But x = y, so V1 = CV (x) = CV (x ) = CV (y) = V2. In particular, ∼ V1 = V2 as abelian groups. Thus, since V = V1 × V2, the Fundamental Theorem THE BASE GROUP OF A FINITE WREATH PRODUCT 9 of Abelian Groups implies that the isomorphism class of V1 and V2 is uniquely determined by that of V . (ii) Given any automorphism σ of D = D8, we need to construct an appropriate automorphism τ of V so that equation (∗) is satisfied. To start with, by the above, t we have V = V1 × V2 with V1 = V2. Furthermore, 2 2 4 D8 = gp h σ(x), σ(t) | σ(x) = 1, σ(t) = 1, σ(x)σ(t) = 1 i. Of course, σ(z) = z since z is the unique nonidentity central element of D. Thus σ(z) = z acts by inversion on V , so a second application of part (i) implies that σ(x) σ(x) −1 V = V3 × V4, where V3 = {v ∈ V | v = v} and V4 = {v ∈ V | v = v }. In σ(t) addition, we have V3 = V4. Now V1 and V3 are isomorphic as abelian groups, so we can let τ : V1 → V3 be t σ(t) any such isomorphism. Moreover, using V1 = V2 and V3 = V4, we can define the t σ(t) isomorphism τ : V2 → V4 by τ(v ) = τ(v) for all v ∈ V1. Combining these, we obtain an isomorphism τ : V = V1 × V2 → V3 × V4 = V . We need only check equation (∗) for v ∈ V1 or V2 and for the generators x and t of D. Since τ(V1) = V3 and τ(V2) = V4, the formulas for x are obvious. Next, if t σ(t) t v ∈ V1, then by definition we have τ(v ) = τ(v) . Finally, if v ∈ V2, then v = v1 t 2 for some v1 ∈ V1 and, indeed, v1 = v since t = 1. Thus σ(t) t σ(t) σ(t)2 t τ(v) = τ(v1) = τ(v1) = τ(v1) = τ(v ), and (∗) holds in this final case. It follows that if θ : W → W extends both σ and τ, then θ is the required automorphism of W . With this, we can quickly prove

Example 3.5. If G is an odd-dihedral group and if C = Sym2, then G o C does not have a characteristic base B.

Proof. First, note that W = GoC = B ohti, where B = G1 ×G2, the base group of t W , is the direct product of two copies of G. Furthermore, t has order 2 and G1 = G2. Since G is odd-dihedral, we have G1 = A1 o hxi, where A1 is a normal abelian subgroup of odd order, x has order 2, and x acts on A1 by inverting all elements. t t t Since G2 = G1, we also have G2 = A2 o hyi, where A2 = A1 and y = x . Using the fact that G1 and G2 commute, it follows that W = V o D, where V = A1 × A2 is an abelian group of odd order and where D = hx, y, ti. Now hx, yi is a normal elementary abelian subgroup of D of order 4, so D = hx, yiohti with xt = y. Thus D is the dihedral group of order 8 and indeed D = gp h x, t | x2 = 1, t2 = 1, (xt)4 = 1 i. Furthermore, B = V o hx, yi. Since x inverts A1 and centralizes A2, and since y inverts A2 and centralizes A1, it follows that z = xy inverts all elements of V . Thus W satisﬁes the hypotheses of the preceding lemma. In particular, if σ is the obvious automorphism of D that interchanges x and t, then by Lemma 3.4(ii), σ extends to an automorphism θ of W . But x ∈ B and θ(x) = t∈ / B, so θ does not stabilize B and hence B is not characteristic in W . Combining all of this, we obtain our characterization in the transitive case, namely Theorem 1.3, which we have restated below for convenience. Of course, the statement and proof implicitly use the associativity of wreath products as described in [Hu, Hilfsatz I.15.4]. This result also yields a more precise formulation of Corollary 2.5 for such groups, since we can take M to be the group N given below. 10 BEN BREWSTER, D. S. PASSMAN, AND ELIZABETH WILCOX

Theorem 3.6. Let W = G o H be the wreath product of the ﬁnite group G by H ⊆ Symn and assume that H is transitive or at least that it acts faithfully on each orbit. Then the base group B = Q G fails to be characteristic in W if and only if G is an odd-dihedral group and H = C2 o Y , where C2 = Sym2, n is even, and Y ⊆ Symn/2. In this case, the base group of

W = G o H = G o (C2 o Y ) = (G o C2) o Y Q given by N = (G o C2) is characteristic in W . Proof. Suppose ﬁrst that B is not characteristic in W . Then by Theorem 2.6, G is odd-dihedral and H ⊆ Symn = S has a normal elementary abelian 2-subgroup T generated by transpositions and moving all points of Ω = {1, 2, . . . , n}. Thus by m ∼ Lemma 3.3, n = 2m is even, |T | = 2 , and Q = NS(T ) = T o Y where Y = Symm. Since T ⊆ H ⊆ Q = TY , we then have H = T o Y0, where we set Y0 = Y ∩ H. Q Thus W = B o H = (B o T ) o Y0 and we note that B o T = (G o C2), where the various C2 groups correspond to the n/2 transpositions in T . With this, we see that W = (B o T ) o Y0 = (G o C2) o Y0 = G o (C2 o Y0). Conversely, suppose W = Go(C2 oY ) = (GoC2)oY . By the previous example, we know that GoC2 admits an automorphism σ that moves its base group G×G. Then σ extends to an automorphism of W = (G o C2) o Y that is the identity on Y , and Q clearly σ moves the base group B = G of W = G o H. Finally, note that G o C2 Q is not odd-dihedral, so N = (G o C2) is characteristic in W = (G o C2) o Y . As we indicated earlier, the automorphism group of the wreath product W = G o H can be computed using the techniques of [B] and [Ha] if the base group B = Q G is characteristic. On the other hand, if B is not characteristic and if H is transitive, then the previous theorem asserts that G is odd-dihedral, H = C2 o Y and W = (G o C2) o Y . Furthermore, in this new representation of W as a wreath Q product, the base group N = G o C2 is now characteristic. Therefore the known methods are again applicable to this situation. Obviously, these techniques require that we know the automorphism group of G o C2 when G is odd-dihedral and we compute this below. ∼ If G = C2, then G o C2 = D8, the dihedral group of order 8, and Aut(D8) is well known to be isomorphic to D8. On the other hand, if G 6= C2, then G has trivial center and hence G o C2 also has trivial center. Thus G o C2 embeds naturally in Aut(G o C2) as the group of inner automorphisms, and we implicitly use this embedding in the statement and proof of the following result. Since parts of the argument below are reminiscent of the proof of Example 3.5, we only sketch those particular aspects.

Proposition 3.7. Let G = A o hxi be an odd-dihedral group with A 6= h1i, write W = G o C2 and set W = Aut(W ). Then we have 2 ∼ i. W = V o D, where V = A × A = O (W ) and where D = D8 is a Sylow 2-subgroup of W . Furthermore, NV (D) = h1i, so V regularly permutes the Sylow 2-subgroups of W by conjugation. ∼ ii. The natural homomorphism ϕ : Aut(W ) → Aut(W/V ) = D8 is surjective. iii. If K/W is the kernel of the homomorphism ϕ, then K = V o NK (D), where NK (D) is isomorphic to Aut(A) acting diagonally on V . iv. |Aut(W )| = |W | = |W |·|Aut(A)|. THE BASE GROUP OF A FINITE WREATH PRODUCT 11

Proof. (i) We have already observed in Example 3.5 that W = V o D, and the Sylow theorems imply that V transitively permutes the Sylow 2-subgroups of W by conjugation. Note that V/W and V ∩ D = h1i, so clearly NV (D) = CV (D). Since V is abelian and W = VD, it then follows that CV (D) ⊆ Z(W ) = h1i and hence NV (D) = h1i. With this, we see that the conjugation action of V on the set of Sylow 2-subgroups of W is regular. (ii) Since V = O2(W ) is a characteristic subgroup of W , the natural homo- ∼ ∼ morphism ϕ : W = Aut(W ) → Aut(W/V ) exists and since W/V = D = D8, we ∼ have Aut(W/V ) = D8. Furthermore, we know that Z(D) = hzi acts in a dihedral manner on V , so Lemma 3.4(ii) implies that ϕ is surjective. (iii) Note that V is characteristic in W , so V/W and hence V/K. Furthermore, K acts on W and permutes its Sylow 2-subgroups. Since V is transitive in this action, we conclude that K = V NK (D). But V ∩ NK (D) = NV (D) = h1i, so K is the split extension K = V o NK (D). Now it is clear that the elements of NK (D) are precisely the automorphisms of W that stabilize and act trivially on D. In particular, since W = V o D, equation (∗) implies that these automorphisms correspond precisely to the automorphisms τ of V with (∗∗) τ(vd) = τ(v)d for all v ∈ V and d ∈ D. Now (∗∗) implies that τ stabilizes CV (d) for each d ∈ D, so τ acts on each factor of V = A×A. Indeed, since there exists an element t ∈ D that interchanges these two factors, it follows that τ corresponds to an automorphism of A that acts diagonally on V . Conversely, it is easy to check that every such ∼ diagonal automorphism satisﬁes (∗∗), so we conclude that NK (D) = Aut(A) acting diagonally on V . (iv) It follows from the above that |K| = |V |·|Aut(A)|, so |Aut(W )| = |W | = |W : K|·|K| = 8·|V |·|Aut(A)| = |W |·|Aut(A)|, as required. References [B] Yu. V. Bodnarchuk, Structure of the automorphism group of a nonstandard wreath product, Ukrainian Math. J., 36 (1984), 128–133. [Ha] A. M. Hassanabadi, Automorphisms of permutational wreath products, J. Australian Math. Soc. (Series A), 26 (1978), 198–208. [Hu] B. Huppert, Endliche Gruppen I, Springer-Verlag, Berlin-New York 1967. [KBF] L. A. Kalu˘znin,P. M. Beleckij, and V. Z. Fejnberg, Kranzprodukte, B. G. Teubner, Leipzig, 1987. [M] J. D. P. Meldrum, Wreath Products of Groups and Semigroups, Longman, Harlow, 1995. [N] P. M. Neumann, On the structure of standard wreath products of groups, Math. Z., 84 (1964), 343–373.

Department of Mathematics, SUNY-Binghamton, Binghamton, NY 13902 E-mail address: [email protected]

Department of Mathematics, University of Wisconsin-Madison, Madison, WI 53706 E-mail address: [email protected]

Department of Mathematics, SUNY-Binghamton, Binghamton, NY 13902 E-mail address: [email protected]