Primitive Permutation Groups with Soluble Stabilizers and Applications to Graphs Hua Zhang School of Mathematics and Statistics A thesis submitted for the degree of Doctor of Philosophy of the University of Western Australia February 2011 September 13, 2011 Dedicated to my parents ii Statement The results in this thesis are my own except where otherwise stated. iii Acknowledgements First and foremost, I would like to express my sincere gratitude to my supervi- sor, Professor Cai Heng Li, both for his constant and patient assistance and advice throughout the preparation of the thesis, for his continuous encouragement, and for all his kindly help and emotional support during my stay here in Australia. I would like to express my sincere gratitude to my supervisor, Winthrop Pro- fessor Cheryl E. Praeger. As one of the most influential mathematician in the world, she has led the research direction of many of her Ph.D students, and each of them benefits from her rich experience and vast knowledge. Her friendliness and long-term support will be most appreciated. It is a tremendous fortune to do my Ph.D under the supervision of the two leading mathematicians. I gratefully acknowledge the financial support of the Scholarships for Interna- tional Research Fees (SIRF) from the University of Western Australia. I am deeply indebted to my wife Yan Cao and my son Yucheng Zhang for their understanding, unlimited love and support. I am very grateful to Dr Jian Ping Wu, for her long-term friendship, so much help and spiritual support. I would like to thank Associated Professor Michael Giudici and Associated Pro- fessor John Bamberg for their friendship, constructive suggestions for the thesis, and the assistance of computation. iv Abstract In the past 60 years interactions between group theory and the theory of graphs have greatly stimulated the development of each other, especially the theory of symmetric graphs (or more generally vertex-transitive graphs) has almost developed in parallel with the theory of permutation groups. The aim of this thesis is to make an effort, both in terms of pure research and broader value, to solve some challenging problems in these two fields. First we considered the problem of classifying finite primitive permutation groups with soluble stabilizers. This problem has a very long history. By the O'Nan- Scott Theorem, such groups are of type affine, almost simple, and product action. We reduced the product action type to the almost simple type, and for the latter, a complete classification was given, which forms the main result of the thesis. The main result was then used to solve some problems in algebraic graph theory. The first application of which is to classify edge-primitive s-arc-transitive graphs for s ≥ 4. After undertaking a general study on the local structures of 2-path-transitive graphs, we presented the second application by classifying finite vertex-primitive and vertex-biprimitive 2-path-transitive graphs. The result then helps us to be able to construct some new half-transitive graphs. Another application of the main result is that a complete classification of finite vertex- biprimitive edge-transitive tetravalent graphs is given (recall that for the cubic case, the classification was given by Ivanov and Iofinova in a highly cited article in 1985). v Publications arising from this thesis 1, C. H. Li and H. Zhang, The finite primitive groups with soluble sta- bilizers, and s-arc-transitive graphs with s ≥ 4, Proc. London. Math. Soc. (3)103 (2011), 441-472. (based on Chapter 4 and Chapter 5). 2, C. H. Li and H. Zhang, On 2-path-transitive graphs, submitted to J. Graph Theory (based on Chapter 6). 3, C. H. Li and H. Zhang, Finite vertex-primitive and vertex-biprimitive 2-path-transitive graphs, submitted to J. Algebraic Combin. (based on Chapter 7). 4, C. H. Li and H. Zhang, Finite vertex-biprimitive edge-transitive tetrava- lent graphs, submitted to J. Combin. Theory, Ser. B (based on Chapter 8). vi Contents 1 Introduction1 1.1 Introduction..................................1 1.2 Literature review...............................3 1.3 Main results and the structure of this thesis................8 2 Preliminaries 10 2.1 Permutation groups............................. 10 2.2 Orbits, stabilizers, and transitivity..................... 12 2.3 Blocks, primitivity, and the O'Nan-Scott Theorem............ 14 2.3.1 The O'Nan-Scott Theorem..................... 15 2.4 More on almost simple groups........................ 18 2.5 The subgroup structure of the classical groups............... 20 3 Graphs 22 3.1 Basic concepts of graphs........................... 22 3.2 Symmetries of graphs............................ 23 3.3 General constructions of graphs....................... 26 4 Finite primitive groups with soluble stabilizers. 31 4.1 The main result................................ 31 4.2 A reduction for the proof of Theorem 4.1.1................ 33 4.3 Subgroups of classical groups of Lie type.................. 35 4.3.1 Preliminaries............................. 36 4.3.2 Soluble maximal Ci-subgroups.................... 37 4.3.3 The maximality............................ 43 4.4 Subgroups of linear groups.......................... 45 4.5 Subgroups of symplectic groups....................... 51 4.6 Subgroups of unitary groups......................... 53 4.7 Subgroups of orthogonal groups....................... 56 4.8 Tables..................................... 58 1 Contents 5 Finite edge-primitive s-arc-transitive graphs with s ≥ 4 66 5.1 The classification............................... 66 5.2 A reduction for the proof of Theorem 5.1.1................ 67 5.2.1 s-Arc transitive graphs........................ 67 5.2.2 Edge-primitive graphs........................ 69 5.3 Proof of Theorem 5.1.1............................ 70 6 Finite 2-path-transitive graphs 74 6.1 The problem and the results......................... 74 6.2 Proofs of Theorems 6.1.1 and 6.1.2..................... 76 6.3 Line graphs.................................. 80 6.4 Proof of Theorem 6.1.4............................ 81 7 Finite vertex-primitive and vertex-biprimitive 2-path-transitive graph- s. 86 7.1 The main results............................... 86 7.2 A reduction.................................. 88 7.3 Examples................................... 90 7.4 Proof of the main theorems......................... 96 7.4.1 Proof of Theorem 7.1.1....................... 96 7.4.2 Proof of Theorem 7.1.2....................... 98 7.4.3 Proof of Theorem 7.1.3....................... 100 7.5 The numbers of the non-isomorphic graphs................ 100 8 Finite vertex-biprimitive edge-transitive tetravalent graphs 104 8.1 The classification............................... 104 8.2 Examples................................... 106 8.3 Local structure of edge-transitive tetravalent graphs........... 109 8.4 A reduction.................................. 111 8.5 Almost simple type.............................. 114 8.6 Proof of Theorem 8.1.1............................ 120 9 Bibliography 124 2 Contents Nomenclature Groups Fq Field with q elements G; H; K Groups n Arbitary soluble group of order n Zn Cyclic group of order n G0 Derived group of G G:H Extension of G by H G:H Split extension of G by H jG:Hj Set of right cosets of H in G NG(H) Normalizer of H in G CG(H) Centralizer of H in G Out(G) Outer automorphism group of G Sn Symmetric group of degree n An Alternating group of degree n Gα Stabilizer of α in G G(∆) Pointwise stabilizer of ∆ in G G∆ Setwise stabilizer of ∆ in G AGLn(q) Affine groups of dimension n over Fq PSLn(q); PSUn(q) Classical groups of Lie type of dimension n over Fq 3 Contents Graphs Γ ; Σ Graphs V Γ Vertex set of Γ EΓ Edge set of Γ AΓ Arc set of Γ AutΓ Full automorphism group of Γ V al(Γ ) Valency of a regular graph Γ Cn Cycle of degree n Kn Complete graph of degree n Kn;m Complete bipartite graph of degree n + m Γ (α) Neighborhood of α [1] Gα Kernel of Gα acting on Γ (α) Γ(α) Gα Induced permutation group of Gα on Γ (α) ΓN Quotient graph of Γ with respect to normal subgroup N Cos(G; H; HgH) Coset graph Cos(G; L; R) Double coset graph 4 Chapter 1 Introduction 1.1 Introduction In the past 60 years interactions between group theory and the theory of graphs have greatly stimulated the development of each other, especially the theory of symmetric graphs (or more generally vertex-transitive graphs) has almost developed in parallel with the theory of permutation groups. The study of symmetric graphs has long been one of the main themes in algebraic graph theory. In early work in [95, 99, 103, 119, 120, 128, 133], finite group theory was used in the study of vertex-transitive graphs and many surprising results were obtained; while in [95, 106, 124, 129] ideas derived from graph theory were used to obtain strong and deep results about permutation groups, and new descriptions of some finite simple groups, and more importantly some new and interesting classes of permutation groups were found. Following the Classification of Finite Simple Groups (henceforth referred to as CFSG), together with the O'Nan-Scott Theorem about the finite primitive groups, more and more classes of symmetric graphs have been completely determined. In recent years symmetric graphs with additional properties, such as vertex-primitive or quasiprimitive graphs, locally primitive graphs, and s-arc-transitive graphs for s ≥ 2, just to name a few, have been extensively studied, see [41, 43, 44, 68, 69, 95, 97]. However, many challenging problems in this field still remain unsettled. The aim of this thesis is to make an effort towards this direction. The study will mainly be centered around the following aspects: • adding new information in permutation group theory, and • extending the knowledge of symmetric graphs. By definition a graph Γ is called symmetric if its automorphism group acts transi- tively on the set of the arcs. For some positive integer s, Γ is called s-arc-transitive if the automorphism group acts transitively on the set of s-arcs.