The Mathematics of Ciphers Number Theory and RSA Cryptography S. C. Coutinho Department of Computer Science Federal University of Rio de Janeiro Rio de Janeiro, Brazil CRC Press Taylor & Francis Group Boca Raton London New York CRC Press is an im p rin t of the Taylor & Francis Group, an informa business AN AK PETERS BOOK Editorial, Sales, and Customer Service Office A K Peters, Ltd. 888 Worcester Street, Suite 230 Wellesley, MA 02482 Copyright © 1999 by A K Peters, Ltd. All rights reserved. No part of the material protected by this copyright notice may be reproduced or utilized in any form, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without written permis­ sion from the copyright owner. Library of Congress Catalogmg-in-Publication Data Coutinho, S. C. [Numeros inteiros e criptografia RSA. English] The mathematics of ciphers : number theory and RSA cryptography / S.C. Coutinho. p. cm. Includes bibliographical references and index. ISBN 1-56881-082-2 1. Number theory. 2. Cryptography. I. Title QA241 .c69513 1998 512' .7—dc21 98-49611 CIP For Andrea and Daniel And I wish the Reader also to take notice, that in writing of it I have made myself a recreation of a recreation; and that it might prove so to him, and not read dull and tediously, I have in sev­ eral places mixt (not any scurrility, but) some innocent, harmless mirth; of which, if thou be a severe, sowre-complexion d man, then I here disallow thee to be a competent judge; for Divines say, There are offences given, and offences not given but taken. Izaak Wolton in “The Compleat Angler” Contents Preface xiii Introduction 1 1. Cryptography 1 2. The RSA cryptosystem 3 3. Computer algebra 5 4. The Greeks and the integers 7 5. Fermat, Euler, and Gauss 8 6. The problems of number theory 11 7. Theorems and proofs 12 Chapter 1. Fundamental algorithms 17 1. Algorithms 17 2. Division algorithm 19 3. Division theorem 21 4. The Euclidean algorithm 22 5. Proof of the Euclidean algorithm 25 6. Extended Euclidean algorithm 27 7. Exercises 30 Chapter 2. Unique factorization 33 1. Unique factorization theorem 33 2. Existence of the factorization 34 3. Efficiency of the trial division algorithm 36 4. Fermat’s factorization algorithm 37 5. Proof of Fermat’s algorithm 38 6. A fundamental property of primes 40 7. The Greeks and the irrational 41 8. Uniqueness of factorization 43 9. Exercises 45 Chapter 3. Prime numbers 49 1. Polynomial formulae 49 2. Exponential formulae: Mersenne numbers 51 ix X Contents 3. Exponential formulae: Fermat numbers 53 4. The primorial formula 54 5. Infinity of primes 55 6. The sieve of Erathostenes 57 7. Exercises 61 Chapter 4. Modular arithmetic 63 1. Equivalence relations 63 2. The congruence relation 66 3. Modular arithmetic 68 4. Divisibility criteria 71 5. Powers 72 6. Diophantine equations 73 7. Division modulo n 75 8. Exercises 77 Chapter 5. Induction and Fermat 79 1. Hanoi! Hanoi! 79 2. Finite induction 83 3. Fermat’s theorem 86 4. Counting roots 88 5. Exercises 92 Chapter 6. Pseudoprimes 95 1. Pseudoprimes 95 2. Carmichael numbers 97 3. Miller’s test 100 4. Primality testing and computer algebra 103 5. Exercises 104 Chapter 7. Systems of Congruences 107 1. Linear equations 107 2. An astronomical example 108 3. The Chinese remainder algorithm: Co-prime moduli 110 4. The Chinese remainder algorithm: General case 113 5. Powers, again 114 6. On sharing secrets 116 7. Exercises 118 Chapter 8. Groups 121 1. Definitions and examples 121 2. Symmetries 122 3. Interlude 126 4. Arithmetic groups 129 5. Subgroups 132 6. Cyclic subgroups 133 Contents xi 7. Finding subgroups 134 8. Lagrange’s theorem 136 9. Exercises 138 Chapter 9. Mersenne and Fermat 141 1. Mersenne numbers 141 2. Fermat numbers 143 3. Fermat, again 145 4. The Lucas-Lehmer test 146 5. Exercises 149 Chapter 10. Primality tests and primitive roots 151 1. Lucas’s test 151 2. Another primality test 154 3. Carmichael numbers 155 4. Preliminaries 156 5. Primitive roots 157 6. Computing orders 158 7. Exercises 160 Chapter 11. The RSA cryptosystem 163 1. On first and last things 163 2. Encryption and decryption 164 3. Why does it work? 166 4. Why is it secure? 167 5. Choosing the primes 168 6. Signatures 170 7. Exercises 171 Coda 175 Appendix. Roots and powers 179 1. Square roots 179 2. Power algorithm 180 Bibliography 183 Index of the main algorithms 189 Index of the main results 191 Index 192 Preface This book will take you on a journey whose final destination is the celebrated Rivest, Shamir, and Adleman (RSA) public key cryptosystem. But it will be a leisurely journey, with many stops to appreciate the scenery and contemplate sites of historical interest. In fact, the book is more concerned with mathematics than with cryptogra­ phy. Although the working of the RSA cryptosystem is described in detail, we will not be concerned with details of its implementation. Instead, we concen­ trate on the mathematical problems it poses, which are related to the factorization of integers, and to determining whether a given integer is prime or composite. These are, in fact, among the oldest questions in the area of mathematics known as number theory, which has been a wonderful source of very challenging prob­ lems since antiquity. Among the mathematicians that worked in the theory of numbers we find Euclid, Fermat, Euler, Lagrange, Legendre, Gauss, Riemann, and, in more recent times, Weil, Deligne, and Wiles. The way number theory is presented in this book differs in some important respects from the classical treatment of most older books. Thus we emphasize the algorithmic aspects everywhere, not forgetting to give complete mathematical proofs of all the algorithms that appear in the book. Of course number theory has been permeated by algorithms since the time of Euclid, but these had been, until very recently, somewhat out of fashion. We take the algorithmic approach very seriously. Hence, Euclid’s proof of the infinity of primes is preceded by a discussion of the primorial formula for primes, and the existence of primitive roots modulo a prime number is proved using the algorithm Gauss invented for computing the roots. Hence, this is really a book about algorithmic number theory and its appli­ cations to RSA cryptography. But, although the book has a very sharp focus, the presentation is by no means narrow. Indeed, we have not always followed the most direct path, but chosen instead the one that promised to throw most light onto the subject. Hence the introduction of groups, which allow us to give a unified treatment of several factorization methods and primality tests in Chap­ ters 9 and 10. Our excursion into group theory will take us as far as Lagrange’s theorem, and includes a discussion of groups of symmetry. This book grew out of lecture notes aimed at first-year students of computer science. Some of the features of the book are really a consequence of the poor xiii xiv Preface background knowledge of the students. Thus very little previous knowledge of mathematics is required. Indeed, we never use anything beyond geometrical progressions and the binomial theorem. Also, although the book is mainly con­ cerned with algorithms, no knowledge of programming is assumed. However, given the subject matter of the book, it is to be expected that many readers will be computer literate. Thus every chapter ends with some (optional) programming exercises that illustrate the algorithms presented in the text. Many of these are aimed at generating numerical data to be compared with some known conjecture or formula of number theory. Thus they are of the sort that is sometimes called a mathematical experiment. A comment on style: Books on mathematics are sometimes written as a dry sequence of definitions, theorems, and proofs. This style goes back to Euclid’s Elements, and it became the standard way to present mathematics in the late twentieth century. But we should not forget that this marmoreal style did not prevail even among the Greek mathematicians. Thus Archimedes tells his readers of his difficulties and wrong turns, and he even warns his readers against propositions that he used and then discovered to be false. In this book I prefer to follow Archimedes’ example rather than Euclid’s. This choice affected the style of the exposition in many respects. First, the historical comments are part of the text, and not isolated notes, and range from the origins of group theory to mere tales of eccentricity. Second, the algorithms are described as instructions in plain English, and I made no effort to optimize them, unless that would promote understanding. Five years of teaching this material have convinced me that this is no obstacle to programming the algorithms for those with the necessary background. Another characteristic of the book that should be mentioned is that the most important theorems and algorithms are referred to by name. Most of these names are classical, and have been used to identify the results for decades, even centuries. Where a name wasn’t already available I made one up. Some, like the primitive root theorem, will be immediately identified by those who are familiar with the subject; some may be harder to identify. Since this makes dipping in more difficult, I have provided a separate index of all the main theorems and algorithms, which also includes a short description of their content. The results that do not have special names are referred to by the chapter and section where they can be found.