JOURNAL OF ALGEBRA 21, 17-50 (1972) 2-Transitive Groups in Which the Stabilizer of Two Points is Cyclic1 University of Illinois at Chicago Circle, Chicago, Illinois, MICHAEL E. O’NAV, Rutgers University, New Brunswick, New ]ersey, GARY &I. SEITZ* Vnivcrsity of Oregon, Eu,Tene, Oregon Communicated by Richard Brauer Received June 29, 1970 1. ISTRODUCTION The purpose of this paper is to prove the following result: ‘hEOREM 1.1. Let G be a finite group 2-transitive on a set R. Suppose that the stabilizer GOB qf two di.stizzct points a, /I E Q is cyclic, and that G Jzas no regukzr normal subgroup. Then G is ooze of the following groups in its usual 2-transitive representation: PSL(2, q), PGL(2, q), Sz(q), PSU(3, q), PGU(3, q) or a Rroup of Ree type. WC note also that the convcrsc of Theorem 1.1 is valid, i.e., in the usual 2-transitive representations of PSL(2, s), PW2, q), Sz(q), PSU(3, q), PGU(3, q), and the groups of Ree type, the stabilizer of two points is cyclic. I-kc S,-(q) is a Suzuki group [25]. Groups of Ret type will be &fined in Section 2. As with the classification of Zassenhaus groups by Zassenhaus [30], Feit [9], Ito [17], and Suzuki [25], this theorem characterizes several families of 2-transitive groups, without regard for the parity of the characteristic. ‘Presented to the American Wlathematical Society in San Antonio, Texas, on January 24, 1970. * Research supported in palt by NSF Grant GP-9584. 17 0 1972 by Academic Press, Inc. 48x/21/1-2 18 KANTOR, O’NAN, AND SEITZ Instead, however, of placing restrictions on the action of G on Q, we have specified the structure of the relatively small subgroup G,, of G. This is analogous to the classification of 2-transitive groups G in which GolB has odd order (Suzuki [26], Bender [2, 31). Th ese latter results are required in our proof. Two special cases of Theorem 1.1, needed in its proof, are due to Suzuki [27] and O’Nan [20]. Th eir results characterize PGU(3, Q) and PSU(3, q), 4 odd, by requiring that the structure of G, be very much like that of the corresponding subgroups of these 2-transitive groups. Another very special case of Theorem 1.1 is due to Ito [lg], who considered 2-transitive groups G in which 1 G,, 1 = 2. Although we have no comparable hold on 1 G,, 1, the arguments in cases A and C of Section 5 were motivated by Ito’s methods. Our arguments are surprisingly elementary, as a result of which the main part of the proof of Theorem 1.1 (in Section 5) is reasonably self-contained. The proof proceeds as follows. As already noted, by using results of Suzuki [26] and Bender [2, 31, we may assume G,, has an involution t. Let d be the set of fixed points of t. In Section 4 we observe that the permutation group C(QA induced by C(t) on d is a 2-transitive group in which the stabilizer of two points is cyclic. Moreover, using a simple counting argument we obtain an important and useful relationship between n = I Q /, I d j, and G,, . By induction, C(t)A either has a regular normal subgroup or is one of the groups we are characterizing. Also, n is either odd or even. Taking each of these possibilities into account, we are led to four cases, which are dealt with in Section 5. In the course of the proof we show that, since G has no regular normal subgroup and ] Gas / is even, n cannot be odd. In case A, we assume that n is odd and that C(t)A has a regular normal elementary abelian p-subgroup, with p an odd prime number. By studying p-subgroups and 2-subgroups of G, we obtain a contradiction. In case B, n is odd and C(t)d h as no regular normal subgroup, so that C(t)4 is PSL(2, 29, Sx(2f), PSU(3,2f), or PGU(3,2f), where 2f > 4. In case C, n is even and C(t)d has a regular normal elementary abelian 2-sub- group. In these cases, as C(ty has a relatively large elementary abelian 2-subgroup, we study the preimage in C(t) of this 2-group. In case D we consider the situation where n is even and C(t)” has no regular normal subgroup. We proceed in two steps. It is first shown that either G is of Ree type or that G very closely resembles PSU(3, q) or PGU(3, q), q odd: C(t)d is PGL(2, q), n = q3 + 1, G, has a normal subgroup regular on Q - 01, and the Sylow 2-subgroups of G are as they should be. We then use an argument of Brauer [l, Chap. 61 to show that ] Gorp 1 = q2 - 1 or (q2 - l)/(q + 1, 3). Th e a f orementioned results of Suzuki [27] and O’Nan [20] now imply that G is PGU(3, q) or PSU(3, q). 2-TRANSITIVE C.ROUPS !9 Section 2 contains properties of PSZ42, q), Sa(y), PSLI(3, q), and groups of Ree type. The definition of groups of Ree type is based on a paper of Rcc [23]. ‘There is an error in that paper which has been corrcctcd by Harada [I 51. In Theorem 2.3 we observe that Thcorcm 1 .I also yields a correction to Ret’s paper and a result even stronger than that of Rcc and that of Harada. 12’ofation. All groups will be finite. If G is a group and XC G, then IVY and C,(X), or simply r\‘(X) and C(X), are the normalizer and centralizer of X, respectively. Aut(G) is the automorphism group of G, Gil) is the derived group of G, O(G) is the largest normal subgroup of G of odd order, and G# : G - {I). If G is ap-group Q,(G) - (x 1 x E G and .P = 1) and U*(C) = (xv 1 x E G). WC use Wiclandt’s notation for permutation groups [28]. If G is a permuta- tion group on Q and OLE Q, then G, is the stabilizer of LY. If X c G, d C: Q and dx = A, then Xd denotes the set of permutations induced by X on A. G is said to be semiregular on R if only I E G fixes a point of R. G is regular on J? if it is transitive and semiregular on .Q. An involution t E G will be called regular if (t) is semiregular on Q. 2. PK~PERTIES 01: THE GROUPS In this section we state those properties of the groups listed in Theorem 1.1 which will be needed in the proof. LEMMA 2.1. Let G be PSL(2, 29, Sz(Zf), PSc’(3, 29 or PGU(3, 29, f > 1, in its usual 2-transitive permutation representation on a set Q. Let %pEQ,a;jlb. (i) G has a simple normul subgroup of index 1 or 3. (ii) If S is a Sylow 2-s&group of G, then S jixes some point 0: E Q, S CI G, , and S is regular on Q - a. (iii) Z(S) = Q,(S) 11~s order 2f, and S has exponent 2 OY 4. (iv) GZB is cyclic and has a subgroup A of order 2’ -- 1 which is reguiar on Z(S)“. (v) A is the set of inverted elements of each involution (@)... infer- changirrg a and p. (vi) If G is PGU(3, 29 OY PSU(3, 29, C(Z(S)),,? has order 2’ f I OY (2’ + 1)/3, andfixes 2f + 1 points. (vii) Zf t E Z(S)*, then C(t) = C(Z(S)). (viii) If G is Sz(29, f is odd. 20 KAXTOR, O'NAN, hi\‘D SEITZ For 4 odd, all of the properties of PGL(2, q) which will be ncedcd are well-known. The following lemma will motivate much of Section 5, case D. LEMMA 2.2. Let G he PGU(3, q), q odd, in its usual 2-transitive representa- tion on a set G. Let LY,p f JI, OL + p. (i) G has a simple normal subgroup ‘14 = PSU(3, q) of index (q + 1,3). (ii) All involutions of G ure conjugate and jix q + 1 points. (iii) Gorsis cyclic of order q2 - 1, and 1 M,, 1 = (q2 - l)/(q + 1, 3). (iv) G, has a normal subgroup Q of order q3, regular on R - (Y. (v) Z(Q) = O(Q) has order q, and G,, is jixed-point-jree on Q;@(Q). (vi) If t is an involution in G,, , then C.w(t) has a normal subgroup Co(t) isomorphic to SL(2, q). (vii) Z(Q) = Co(t). (viii) The Sylow 2-subgroups of G are quasi-dihedral if q -s I (mod 4) OY mreuthed Z,, 1 Z, if q -2 3 (mod 4). In [23], Ree considered groups G satisfying the following conditions: (Rl) G is a group 2-transitive on a finite set s2, [ J2 1 = n. (R2) If (Y,,!? E J2, o! f /3, G, has a unique element t f- 1 fixing more than 2 points. (R3) All involutions in G are conjugate. A family of groups staisfying (R I)-( R3) were- discovered earlier by Ree [22].