Modern Computer Arithmetic (Version 0.5. 1)
Total Page:16
File Type:pdf, Size:1020Kb
Modern Computer Arithmetic Richard P. Brent and Paul Zimmermann Version 0.5.1 arXiv:1004.4710v1 [cs.DS] 27 Apr 2010 Copyright c 2003-2010 Richard P. Brent and Paul Zimmermann This electronic version is distributed under the terms and conditions of the Creative Commons license “Attribution-Noncommercial-No Derivative Works 3.0”. You are free to copy, distribute and transmit this book under the following conditions: Attribution. You must attribute the work in the manner specified • by the author or licensor (but not in any way that suggests that they endorse you or your use of the work). Noncommercial. You may not use this work for commercial purposes. • No Derivative Works. You may not alter, transform, or build upon • this work. For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to the web page below. Any of the above conditions can be waived if you get permission from the copyright holder. Nothing in this license impairs or restricts the author’s moral rights. For more information about the license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/ Contents Contents iii Preface ix Acknowledgements xi Notation xiii 1 Integer Arithmetic 1 1.1 RepresentationandNotations . 1 1.2 AdditionandSubtraction . .. 2 1.3 Multiplication . 3 1.3.1 Naive Multiplication . 4 1.3.2 Karatsuba’s Algorithm . 5 1.3.3 Toom-Cook Multiplication . 7 1.3.4 UseoftheFastFourierTransform(FFT) . 8 1.3.5 Unbalanced Multiplication . 9 1.3.6 Squaring.......................... 12 1.3.7 Multiplication by a Constant . 15 1.4 Division .............................. 15 1.4.1 Naive Division . 16 1.4.2 Divisor Preconditioning . 18 1.4.3 Divide and Conquer Division . 19 1.4.4 Newton’sMethod..................... 23 1.4.5 Exact Division . 23 1.4.6 Only Quotient or Remainder Wanted . 24 1.4.7 Division by a Single Word . 25 1.4.8 Hensel’s Division . 26 iii iv Modern Computer Arithmetic, version 0.5.1 of April 28, 2010 1.5 Roots ............................... 27 1.5.1 SquareRoot........................ 27 1.5.2 k-thRoot ......................... 29 1.5.3 ExactRoot ........................ 30 1.6 GreatestCommonDivisor . 31 1.6.1 NaiveGCD ........................ 32 1.6.2 ExtendedGCD ...................... 35 1.6.3 Half Binary GCD, Divide and Conquer GCD . 36 1.7 BaseConversion.......................... 40 1.7.1 Quadratic Algorithms . 41 1.7.2 Subquadratic Algorithms . 41 1.8 Exercises.............................. 43 1.9 NotesandReferences . 48 2 Modular Arithmetic and the FFT 51 2.1 Representation .......................... 51 2.1.1 Classical Representation . 51 2.1.2 Montgomery’sForm . 52 2.1.3 ResidueNumberSystems . 53 2.1.4 MSBvsLSBAlgorithms . 53 2.1.5 Link with Polynomials . 53 2.2 Modular Addition and Subtraction . 54 2.3 TheFourierTransform . 54 2.3.1 TheoreticalSetting . 55 2.3.2 TheFastFourierTransform . 56 2.3.3 The Sch¨onhage-Strassen Algorithm . 60 2.4 Modular Multiplication . 63 2.4.1 Barrett’s Algorithm . 63 2.4.2 Montgomery’s Multiplication . 65 2.4.3 McLaughlin’s Algorithm . 68 2.4.4 SpecialModuli ...................... 70 2.5 Modular Division and Inversion . 70 2.5.1 SeveralInversionsatOnce . 72 2.6 ModularExponentiation . 74 2.6.1 Binary Exponentiation . 75 2.6.2 Exponentiation With a Larger Base . 76 2.6.3 Sliding Window and Redundant Representation . 77 2.7 ChineseRemainderTheorem. 79 Modern Computer Arithmetic v 2.8 Exercises.............................. 81 2.9 NotesandReferences . 83 3 Floating-Point Arithmetic 85 3.1 Representation .......................... 85 3.1.1 RadixChoice ....................... 87 3.1.2 ExponentRange ..................... 87 3.1.3 SpecialValues....................... 88 3.1.4 SubnormalNumbers . 88 3.1.5 Encoding ......................... 90 3.1.6 Precision: Local, Global, Operation, Operand . 91 3.1.7 LinktoIntegers...................... 92 3.1.8 Ziv’s Algorithm and Error Analysis . 93 3.1.9 Rounding ......................... 94 3.1.10 Strategies ......................... 98 3.2 Addition,Subtraction,Comparison . 99 3.2.1 Floating-Point Addition . 99 3.2.2 Floating-Point Subtraction . 101 3.3 Multiplication . 103 3.3.1 Integer Multiplication via Complex FFT . 107 3.3.2 TheMiddleProduct . .109 3.4 Reciprocal and Division . 110 3.4.1 Reciprocal.........................111 3.4.2 Division . 115 3.5 SquareRoot............................120 3.5.1 ReciprocalSquareRoot . .121 3.6 Conversion.............................124 3.6.1 Floating-Point Output . 124 3.6.2 Floating-Point Input . 127 3.7 Exercises..............................128 3.8 NotesandReferences . .130 4 Elementary and Special Function Evaluation 135 4.1 Introduction............................135 4.2 Newton’sMethod .........................136 4.2.1 Newton’sMethodforInverse Roots . 138 4.2.2 Newton’s Method for Reciprocals . 138 4.2.3 Newton’s Method for (Reciprocal) Square Roots . 139 vi Modern Computer Arithmetic, version 0.5.1 of April 28, 2010 4.2.4 Newton’s Method for Formal Power Series . 140 4.2.5 Newton’s Method for Functional Inverses . 141 4.2.6 Higher Order Newton-like Methods . 142 4.3 ArgumentReduction . .143 4.3.1 Repeated Use of a Doubling Formula . 145 4.3.2 LossofPrecision .....................145 4.3.3 GuardDigits .......................146 4.3.4 Doubling versus Tripling . 147 4.4 PowerSeries............................148 4.4.1 Direct Power Series Evaluation . 152 4.4.2 Power Series With Argument Reduction . 152 4.4.3 Rectangular Series Splitting . 153 4.5 AsymptoticExpansions. .157 4.6 ContinuedFractions . .163 4.7 RecurrenceRelations . .165 4.7.1 Evaluation of Bessel Functions . 166 4.7.2 Evaluation of Bernoulli and Tangent numbers . 167 4.8 Arithmetic-GeometricMean . .171 4.8.1 Elliptic Integrals . 172 4.8.2 First AGM Algorithm for the Logarithm . 173 4.8.3 ThetaFunctions. .174 4.8.4 Second AGM Algorithm for the Logarithm . 175 4.8.5 TheComplexAGM. .. .. .177 4.9 Binary Splitting . 178 4.9.1 A Binary Splitting Algorithm for sin, cos . 180 4.9.2 The Bit-Burst Algorithm . 181 4.10 ContourIntegration. .184 4.11Exercises..............................185 4.12 NotesandReferences . .194 5 Implementations and Pointers 201 5.1 SoftwareTools...........................201 5.1.1 CLN............................201 5.1.2 GNUMP(GMP).....................202 5.1.3 MPFQ...........................203 5.1.4 MPFR...........................203 5.1.5 Other Multiple-Precision Packages . 203 5.1.6 ComputationalAlgebraPackages . 204 Modern Computer Arithmetic vii 5.2 Mailing Lists . 206 5.2.1 The BNIS Mailing List . 206 5.2.2 TheGMPLists ......................206 5.2.3 TheMPFRList......................206 5.3 On-lineDocuments . .. .. .206 Bibliography 209 Index 230 Summary of Complexities 247 Preface This is a book about algorithms for performing arithmetic, and their imple- mentation on modern computers. We are concerned with software more than hardware — we do not cover computer architecture or the design of computer hardware since good books are already available on these topics. Instead we focus on algorithms for efficiently performing arithmetic operations such as addition, multiplication and division, and their connections to topics such as modular arithmetic, greatest common divisors, the Fast Fourier Transform (FFT), and the computation of special functions. The algorithms that we present are mainly intended for arbitrary-precision arithmetic. That is, they are not limited by the computer wordsize of 32 or 64 bits, only by the memory and time available for the computation. We consider both integer and real (floating-point) computations. The book is divided into four main chapters, plus one short chapter (es- sentially an appendix). Chapter 1 covers integer arithmetic. This has, of course, been considered in many other books and papers. However, there has been much recent progress, inspired in part by the application to public key cryptography, so most of the published books are now partly out of date or incomplete. Our aim is to present the latest developments in a concise manner. At the same time, we provide a self-contained introduction for the reader who is not an expert in the field. Chapter 2 is concerned with modular arithmetic and the FFT, and their applications to computer arithmetic. We consider different number represen- tations, fast algorithms for multiplication, division and exponentiation, and the use of the Chinese Remainder Theorem (CRT). Chapter 3 covers floating-point arithmetic. Our concern is with high- precision floating-point arithmetic, implemented in software if the precision provided by the hardware (typically IEEE standard 53-bit significand) is in- adequate. The algorithms described in this chapter focus on correct rounding, extending the IEEE standard to arbitrary precision. x Modern Computer Arithmetic, version 0.5.1 of April 28, 2010 Chapter 4 deals with the computation, to arbitrary precision, of functions such as sqrt, exp, ln, sin, cos, and more generally functions defined by power series or continued fractions. Of course, the computation of special functions is a huge topic so we have had to be selective. In particular, we have con- centrated on methods that are efficient and suitable for arbitrary-precision computations. The last chapter contains pointers to implementations, useful web sites, mailing lists, and so on. Finally, at the end there is a one-page Summary of Complexities which should be a useful aide-m´emoire. The chapters are fairly self-contained, so it is possible to read them out of order. For example, Chapter 4 could be read before Chapters 1–3, and Chapter 5 can be consulted at any time. Some topics, such as Newton’s method, appear in different guises in several chapters. Cross-references are given where appropriate. For details that are omitted we give pointers in the Notes and References sections of each chapter, as well as in the bibliography. We have tried, as far as possible, to keep the main text uncluttered by footnotes and references, so most references are given in the Notes and References sections. The book is intended for anyone interested in the design and implemen- tation of efficient algorithms for computer arithmetic, and more generally efficient numerical algorithms.