On the Number Field Sieve: Polynomial Selection and Smooth Elements in Number Fields Nicholas Vincent Coxon BSc (hons) A thesis submitted for the degree of Doctor of Philosophy at The University of Queensland in June 2012 School of Mathematics and Physics Abstract The number field sieve is the asymptotically fastest known algorithm for factoring large integers that are free of small prime factors. Two aspects of the algorithm are considered in this thesis: polynomial selection and smooth elements in number fields. The contributions to polynomial selection are twofold. First, existing methods of polynomial generation, namely those based on Montgomery's method, are extended and tools developed to aid in their analysis. Second, a new approach to polynomial generation is developed and realised. The development of the approach is driven by results obtained on the divisibility properties of univariate resultants. Examples from the literature point toward the utility of applying decoding algorithms for algebraic error-correcting codes to problems of finding elements in a ring with a smooth representation. In this thesis, the problem of finding algebraic integers in a number field with smooth norm is reformulated as a decoding problem for a family of error-correcting codes called NF-codes. An algorithm for solving the weighted list decoding problem for NF-codes is provided. The algorithm is then used to find algebraic integers with norm containing a large smooth factor. Bounds on the existence of such numbers are derived using algorithmic and combinatorial methods. ii Declaration by the Author This thesis is composed of my original work, and contains no material previously published or written by another person except where due reference has been made in the text. I have clearly stated the contribution by others to jointly-authored works that I have included in my thesis. I have clearly stated the contribution of others to my thesis as a whole, including statistical assistance, survey design, data analysis, significant technical procedures, professional editorial advice, and any other original research work used or reported in my thesis. The content of my thesis is the result of work I have carried out since the commencement of my research higher degree candidature and does not include a substantial part of work that has been submitted to qualify for the award of any other degree or diploma in any university or other tertiary institution. I have clearly stated which parts of my thesis, if any, have been submitted to qualify for another award. I acknowledge that an electronic copy of my thesis must be lodged with the University Library and, subject to the General Award Rules of The University of Queensland, immediately made available for research and study in accordance with the Copyright Act 1968. I acknowledge that copyright of all material contained in my thesis resides with the copyright holder(s) of that material. Where appropriate I have obtained copyright permission from the copyright holder to reproduce material in this thesis. iii Publications During Candidature [47] Nicholas Coxon. On nonlinear polynomial selection for the number field sieve. ArXiv e-Print archive, arXiv:1109.6398 [math.NT], September 2010. http://arxiv.org/abs/1109.6398. [48] Nicholas Coxon. List decoding of number field codes. Designs, codes and cryptography, 2013. doi:10.1007/s10623-013-9803-x. Publications Included in the Thesis Publications [47] and [48] have been incorporated into Chapter 3 and Chapter 5, respectively. Contributions by Others to the Thesis As supervisor, Victor Scharaschkin helped guide the research presented in the thesis. There has been no contribution by others to the writing of the thesis. Statement of Parts of the Thesis Submitted to Qualify for the Award of Another Degree None. iv Acknowledgements I am greatly indebted to Victor Scharaschkin for his supervision during my candidature. The support he has extended to me, and his informed guidance and advice, have been critically important through- out my postgraduate and undergraduate studies. I would like to thank Gary Carter, Eric Mortenson and Graham Norton for helpful discussions and their comments on draft chapters of this thesis. I would like to acknowledge financial support of the Australian Postgraduate Award. I have had the pleasure of working with and learning from many of the staff and students of the Mathematics Department. I am proud to call many of you my friends, and wish you all the best in your future endeavours. It would not have been possible to write this thesis without the help and encouragement of my fellow postgraduate students. In particular, I would like to thank Tristan Dunning, Peter Finch, Samuel Hambleton, Dejan Jovanovic, Geoff Martin, Thomas McCourt and Sian Stafford with whom daily conversations have been constructive, disruptive when needed most, and always greatly appreciated. To all the friends with whom I have shared many adventures whilst working on this thesis, thank you. Finally, to my family, thank you for always being there with love and support. v Keywords Integer factorisation, number field sieve, polynomial selection, smooth numbers, list decoding, number field codes, geometry of numbers, resultants. Australian and New Zealand Standard Research Classifications (ANZSRC) ANZSRC code: 010101, Algebra and Number Theory, 89:939% ANZSRC code: 080201, Analysis of Algorithms and Complexity, 10:061% Fields of Research (FoR) Classification FoR code: 0101, Pure Mathematics, 100:000% vi Contents List of Tables ix Nomenclature xi 1 Introduction 1 1.1 Congruence of Squares Factoring . .1 1.1.1 The Morrison{Brillhart Approach . .2 1.1.2 Finding Smooth Residues . .3 1.1.3 Complexity Estimates . .5 1.2 The Number Field Sieve . .6 1.2.1 Outline of the Number Field Sieve . .7 1.2.2 Complexity Estimates . 11 1.2.3 The Polynomial Selection Problem . 13 1.2.4 Smooth Elements in Number Fields . 14 1.3 Outline of the Thesis . 15 2 Preliminaries on Polynomial Selection 17 2.1 Quantifying Properties which Influence Polynomial Yield . 17 2.1.1 Quantifying Size Properties: Skewed Polynomial Norms . 18 2.1.2 Quantifying Root Properties . 22 2.1.3 Ranking Polynomials . 27 2.2 Number Field Sieve Polynomial Generation . 30 2.2.1 The Montgomery{Murphy Algorithm . 31 2.2.2 Kleinjung's Algorithm . 34 2.2.3 Nonlinear Algorithms . 40 vii 2.2.4 A Lower Bound on Polynomial Generation . 42 3 Nonlinear Polynomial Selection 45 3.1 Preliminaries on Lattices . 46 3.2 Nonlinear Polynomial Selection . 48 3.2.1 The Orthogonal Lattice . 48 3.2.2 Nonlinear Polynomial Generation in Detail . 52 3.2.3 Existing Algorithms . 54 3.3 Length d + 1 Construction Revisited . 57 3.3.1 Parameter Selection for Algorithm 3.3.3 . 59 3.4 The Koo{Jo{Kwon Length d + 2 Construction Revisited . 61 3.4.1 Parameter Selection for Algorithm 3.4.2 . 63 4 An Approach to Polynomial Selection 65 4.1 Overview of the Approach . 66 4.2 Divisibility Properties of Univariate Resultants . 69 4.2.1 Definition and Properties of Resultants . 70 4.2.2 Proof of Lemma 4.1.1 . 72 4.3 Combinatorial Bounds on Polynomial Selection . 79 4.4 An Initial Algorithm . 83 4.4.1 Parameter Selection for Algorithm 4.4.2 . 86 4.4.2 Algorithmic Bounds on Polynomial Selection . 93 4.5 Future Directions: Improvements and Generalisations . 95 4.5.1 Special-q ....................................... 96 4.5.2 Lattice Construction . 98 4.5.3 A Multivariate Generalisation . 105 5 Smooth Elements in Number Fields 111 5.1 Review of NF-codes . 113 5.2 Combinatorial Bounds on List Decoding . 115 5.3 Weighted List Decoding of NF-codes . 116 5.3.1 Additional Notes on Implementing Algorithm 5.3.2 . 119 viii 5.3.2 Analysis of the Decoding Algorithm . 120 5.3.3 Decoding with Integral Lattices . 123 5.3.4 Parameter Selection for Algorithm 5.3.2 . 124 5.4 Smooth Algebraic Integers in Number Fields . 127 5.4.1 Finding Smooth Algebraic Integers in Number Fields . 128 5.4.2 Bounds on Smooth Algebraic Integers in Number Fields . 131 6 Conclusions and Future Research 135 References 139 Appendix 153 A Appendices for Chapter 5 155 A.1 Number Field Codes with Known Rate . 155 A.2 Decoding with Nonzero Shift Parameters . 157 ix List of Tables 4.1 Bounds for Example 4.3.5. 82 4.2 Parameters for Example 4.4.10 . 92 4.3 Bounds for Example 4.4.12. 95 4.4 Parameters for Example 4.5.3 . 98 5.1 Bounds for Example 5.4.13 . 133 5.2 Bounds for Example 5.4.14 . 133 x Nomenclature The following is a list of abbreviations and symbols that appear in the thesis. Where possible, a page number is provided to indicate where an abbreviation or symbol first appears. The list of symbols is incomplete, with only those symbols that are not defined elsewhere in the thesis, or that appear in multiple chapters, being listed. All remaining symbols are possibly used for several different purposes throughout the thesis, but are always used consistently within the confines of a single chapter, where the relevant definition is found. List of Abbreviations CFRAC Continued fractions method, p. 1. GP Geometric progression, p. 41. NFS Number field sieve, p. 1. QS Quadratic Sieve, p. 1. SNFS Special number field sieve, p. 1. List of Symbols Z, Q, R, C The set of integers, rationals, real numbers and complex numbers, respectively. Fq The finite field of order q, where q is a prime power. Z=nZ The ring of integers modulo n. A× The group of units of a ring A. GLn(A) The general linear group of n × n invertible matrices with entries in a ring A.