Numerical Computation Guide Sun™ Studio 9 Sun Microsystems, Inc. www.sun.com Part No. 817-6702-10 July 2004, Revision A Submit comments about this document at: http://www.sun.com/hwdocs/feedback Copyright © 2004 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, California 95054, U.S.A. All rights reserved. U.S. Government Rights - Commercial software. Government users are subject to the Sun Microsystems, Inc. standard license agreement and applicable provisions of the FAR and its supplements. Use is subject to license terms. This distribution may include materials developed by third parties. Parts of the product may be derived from Berkeley BSD systems, licensed from the University of California. UNIX is a registered trademark in the U.S. and in other countries, exclusively licensed through X/Open Company, Ltd. Sun, Sun Microsystems, the Sun logo, Java, and JavaHelp are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and other countries.All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. in the U.S. and other countries. Products bearing SPARC trademarks are based upon architecture developed by Sun Microsystems, Inc. This product is covered and controlled by U.S. Export Control laws and may be subject to the export or import laws in other countries. Nuclear, missile, chemical biological weapons or nuclear maritime end uses or end users, whether direct or indirect, are strictly prohibited. Export or reexport to countries subject to U.S. embargo or to entities identified on U.S. export exclusion lists, including, but not limited to, the denied persons and specially designated nationals lists is strictly prohibited. DOCUMENTATION IS PROVIDED "AS IS" AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID. Copyright © 2004 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, California 95054, Etats-Unis. Tous droits réservés. L’utilisation est soumise aux termes de la Licence. Cette distribution peut comprendre des composants développés par des tierces parties. Des parties de ce produit pourront être dérivées des systèmes Berkeley BSD licenciés par l’Université de Californie. UNIX est une marque déposée aux Etats-Unis et dans d’autres pays et licenciée exclusivement par X/Open Company, Ltd. Sun, Sun Microsystems, le logo Sun, Java, et JavaHelp sont des marques de fabrique ou des marques déposées de Sun Microsystems, Inc. aux Etats-Unis et dans d’autres pays.Toutes les marques SPARC sont utilisées sous licence et sont des marques de fabrique ou des marques déposées de SPARC International, Inc. aux Etats-Unis et dans d’autres pays. Les produits portant les marques SPARC sont basés sur une architecture développée par Sun Microsystems, Inc. Ce produit est soumis à la législation américaine en matière de contrôle des exportations et peut être soumis à la règlementation en vigueur dans d’autres pays dans le domaine des exportations et importations. Les utilisations, ou utilisateurs finaux, pour des armes nucléaires,des missiles, des armes biologiques et chimiques ou du nucléaire maritime, directement ou indirectement, sont strictement interdites. Les exportations ou réexportations vers les pays sous embargo américain, ou vers des entités figurant sur les listes d’exclusion d’exportation américaines, y compris, mais de manière non exhaustive, la liste de personnes qui font objet d’un ordre de ne pas participer, d’une façon directe ou indirecte, aux exportations des produits ou des services qui sont régis par la législation américaine en matière de contrôle des exportations et la liste de ressortissants spécifiquement désignés, sont rigoureusement interdites. LA DOCUMENTATION EST FOURNIE "EN L’ÉTAT" ET TOUTES AUTRES CONDITIONS, DECLARATIONS ET GARANTIES EXPRESSES OU TACITES SONT FORMELLEMENT EXCLUES, DANS LA MESURE AUTORISEE PAR LA LOI APPLICABLE, Y COMPRIS NOTAMMENT TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE, A L’APTITUDE A UNE UTILISATION PARTICULIERE OU A L’ABSENCE DE CONTREFAÇON. Contents Before You Begin xiii Who Should Use This Book xiii How This Book Is Organized xiii Typographic Conventions xiv Shell Prompts xv Accessing Sun Studio Software and Man Pages xv Accessing Compilers and Tools Documentation xviii Accessing Related Solaris Documentation xx Resources for Developers xx Contacting Sun Technical Support xxi Sun Welcomes Your Comments xxi 1. Introduction 1–1 1.1 Floating-Point Environment 1–1 2. IEEE Arithmetic 2–1 2.1 IEEE Arithmetic Model 2–1 2.1.1 What Is IEEE Arithmetic? 2–1 2.2 IEEE Formats 2–3 2.2.1 Storage Formats 2–3 iii 2.2.2 Single Format 2–3 2.2.3 Double Format 2–6 2.2.4 Double-Extended Format (SPARC) 2–8 2.2.5 Double-Extended Format (x86) 2–11 2.2.6 Ranges and Precisions in Decimal Representation 2–14 2.2.7 Base Conversion in the Solaris Environment 2–17 2.3 Underflow 2–18 2.3.1 Underflow Thresholds 2–19 2.3.2 How Does IEEE Arithmetic Treat Underflow? 2–19 2.3.3 Why Gradual Underflow? 2–20 2.3.4 Error Properties of Gradual Underflow 2–21 2.3.5 Two Examples of Gradual Underflow Versus Store 0 2–24 2.3.6 Does Underflow Matter? 2–25 3. The Math Libraries 3–1 3.1 Standard Math Library 3–1 3.2 Additional Math Libraries 3–2 3.2.1 Sun Math Library 3–2 3.2.2 Optimized Libraries 3–4 3.2.3 Vector Math Library (SPARC only) 3–5 3.2.4 libm9x Math Library 3–6 3.3 Single, Double, and Long Double Precision 3–8 3.4 IEEE Support Functions 3–9 3.4.1 ieee_functions(3m) and ieee_sun(3m) 3–9 3.4.2 ieee_values(3m) 3–11 3.4.3 ieee_flags(3m) 3–13 3.4.4 ieee_retrospective(3m) 3–15 3.4.5 nonstandard_arithmetic(3m) 3–17 3.5 C99 Floating Point Environment Functions 3–17 iv Numerical Computation Guide • July 2004 3.5.1 Exception Flag Functions 3–18 3.5.2 Rounding Control 3–18 3.5.3 Environment Functions 3–19 3.6 Implementation Features of libm and libsunmath 3–20 3.6.1 About the Algorithms 3–21 3.6.2 Argument Reduction for Trigonometric Functions 3–21 3.6.3 Data Conversion Routines 3–22 3.6.4 Random Number Facilities 3–22 4. Exceptions and Exception Handling 4–1 4.1 What Is an Exception? 4–2 4.1.1 Notes for Table 4-1 4–4 4.2 Detecting Exceptions 4–5 4.2.1 ieee_flags(3m) 4–5 4.2.2 C99 Exception Flag Functions 4–8 4.3 Locating an Exception 4–9 4.3.1 Using the Debuggers to Locate an Exception 4–10 4.3.2 Using a Signal Handler to Locate an Exception 4–18 4.3.3 Using libm9x.so Exception Handling Extensions to Locate an Exception 4–24 4.4 Handling Exceptions 4–31 A. Examples A–1 A.1 IEEE Arithmetic A–1 A.2 The Math Libraries A–3 A.2.1 Random Number Generator A–3 A.2.2 IEEE Recommended Functions A–6 A.2.3 IEEE Special Values A–10 A.2.4 ieee_flags — Rounding Direction A–12 A.2.5 C99 Floating Point Environment Functions A–14 Contents v A.3 Exceptions and Exception Handling A–19 A.3.1 ieee_flags — Accrued Exceptions A–19 A.3.2 ieee_handler — Trapping Exceptions A–22 A.3.3 ieee_handler — Abort on Exceptions A–31 A.3.4 libm9x.so Exception Handling Features A–31 A.3.5 Using libm9x.so With Fortran Programs A–38 A.4 Miscellaneous A–42 A.4.1 sigfpe — Trapping Integer Exceptions A–42 A.4.2 Calling Fortran From C A–43 A.4.3 Useful Debugging Commands A–47 B. SPARC Behavior and Implementation B–1 B.1 Floating-Point Hardware B–1 B.1.1 Floating-Point Status Register and Queue B–4 B.1.2 Special Cases Requiring Software Support B–6 B.2 fpversion(1) Function — Finding Information About the FPU B–11 C. x86 Behavior and Implementation C–1 D. What Every Computer Scientist Should Know About Floating-Point Arithmetic D–1 D.1 Abstract D–1 D.2 Introduction D–2 D.3 Rounding Error D–2 D.3.1 Floating-point Formats D–3 D.3.2 Relative Error and Ulps D–5 D.3.3 Guard Digits D–6 D.3.4 Cancellation D–8 D.3.5 Exactly Rounded Operations D–13 D.4 The IEEE Standard D–16 vi Numerical Computation Guide • July 2004 D.4.1 Formats and Operations D–17 D.4.2 Special Quantities D–22 D.4.3 NaNs D–23 D.4.4 Exceptions, Flags and Trap Handlers D–29 D.5 Systems Aspects D–34 D.5.1 Instruction Sets D–34 D.5.2 Languages and Compilers D–36 D.5.3 Exception Handling D–44 D.6 The Details D–45 D.6.1 Rounding Error D–46 D.6.2 Binary to Decimal Conversion D–55 D.6.3 Errors In Summation D–56 D.7 Summary D–58 D.8 Acknowledgments D–58 D.9 References D–59 D.10 Theorem 14 and Theorem 8 D–61 D.10.1 Theorem 14 D–61 D.10.2 Proof D–61 D.11 Differences Among IEEE 754 Implementations D–65 D.11.1 Current IEEE 754 Implementations D–67 D.11.2 Pitfalls in Computations on Extended-Based Systems D–68 D.11.3 Programming Language Support for Extended Precision D–74 D.11.4 Conclusion D–78 E.