Floating Points Numbers: How to Work with Them Correctly and Efficiently
Total Page:16
File Type:pdf, Size:1020Kb

Load more
Recommended publications
-
Decimal Rounding
Mathematics Instructional Plan – Grade 5 Decimal Rounding Strand: Number and Number Sense Topic: Rounding decimals, through the thousandths, to the nearest whole number, tenth, or hundredth. Primary SOL: 5.1 The student, given a decimal through thousandths, will round to the nearest whole number, tenth, or hundredth. Materials Base-10 blocks Decimal Rounding activity sheet (attached) “Highest or Lowest” Game (attached) Open Number Lines activity sheet (attached) Ten-sided number generators with digits 0–9, or decks of cards Vocabulary approximate, between, closer to, decimal number, decimal point, hundredth, rounding, tenth, thousandth, whole Student/Teacher Actions: What should students be doing? What should teachers be doing? 1. Begin with a review of decimal place value: Display a decimal in the thousandths place, such as 34.726, using base-10 blocks. In pairs, have students discuss how to read the number using place-value names, and review the decimal place each digit holds. Briefly have students share their thoughts with the class by asking: What digit is in the tenths place? The hundredths place? The thousandths place? Who would like to share how to say this decimal using place value? 2. Ask, “If this were a monetary amount, $34.726, how would we determine the amount to the nearest cent?” Allow partners or small groups to discuss this question. It may be helpful if students underlined the place that indicates cents (the hundredths place) in order to keep track of the rounding place. Distribute the Open Number Lines activity sheet and display this number line on the board. Guide students to recall that the nearest cent is the same as the nearest hundredth of a dollar. -
Scilab Textbook Companion for Numerical Methods for Engineers by S
Scilab Textbook Companion for Numerical Methods For Engineers by S. C. Chapra And R. P. Canale1 Created by Meghana Sundaresan Numerical Methods Chemical Engineering VNIT College Teacher Kailas L. Wasewar Cross-Checked by July 31, 2019 1Funded by a grant from the National Mission on Education through ICT, http://spoken-tutorial.org/NMEICT-Intro. This Textbook Companion and Scilab codes written in it can be downloaded from the "Textbook Companion Project" section at the website http://scilab.in Book Description Title: Numerical Methods For Engineers Author: S. C. Chapra And R. P. Canale Publisher: McGraw Hill, New York Edition: 5 Year: 2006 ISBN: 0071244298 1 Scilab numbering policy used in this document and the relation to the above book. Exa Example (Solved example) Eqn Equation (Particular equation of the above book) AP Appendix to Example(Scilab Code that is an Appednix to a particular Example of the above book) For example, Exa 3.51 means solved example 3.51 of this book. Sec 2.3 means a scilab code whose theory is explained in Section 2.3 of the book. 2 Contents List of Scilab Codes4 1 Mathematical Modelling and Engineering Problem Solving6 2 Programming and Software8 3 Approximations and Round off Errors 10 4 Truncation Errors and the Taylor Series 15 5 Bracketing Methods 25 6 Open Methods 34 7 Roots of Polynomials 44 9 Gauss Elimination 51 10 LU Decomposition and matrix inverse 61 11 Special Matrices and gauss seidel 65 13 One dimensional unconstrained optimization 69 14 Multidimensional Unconstrainted Optimization 72 3 15 Constrained -
Extended Precision Floating Point Arithmetic
Extended Precision Floating Point Numbers for Ill-Conditioned Problems Daniel Davis, Advisor: Dr. Scott Sarra Department of Mathematics, Marshall University Floating Point Number Systems Precision Extended Precision Patriot Missile Failure Floating point representation is based on scientific An increasing number of problems exist for which Patriot missile defense modules have been used by • The precision, p, of a floating point number notation, where a nonzero real decimal number, x, IEEE double is insufficient. These include modeling the U.S. Army since the mid-1960s. On February 21, system is the number of bits in the significand. is expressed as x = ±S × 10E, where 1 ≤ S < 10. of dynamical systems such as our solar system or the 1991, a Patriot protecting an Army barracks in Dha- This means that any normalized floating point The values of S and E are known as the significand • climate, and numerical cryptography. Several arbi- ran, Afghanistan failed to intercept a SCUD missile, number with precision p can be written as: and exponent, respectively. When discussing float- trary precision libraries have been developed, but leading to the death of 28 Americans. This failure E ing points, we are interested in the computer repre- x = ±(1.b1b2...bp−2bp−1)2 × 2 are too slow to be practical for many complex ap- was caused by floating point rounding error. The sentation of numbers, so we must consider base 2, or • The smallest x such that x > 1 is then: plications. David Bailey’s QD library may be used system measured time in tenths of seconds, using binary, rather than base 10. -
Introduction & Floating Point NMMC §1.4.1, Atkinson §1.2 1 Floating
Introduction & Floating Point NMMC x1.4.1, Atkinson x1.2 1 Floating-point representation, IEEE 754. In binary, 11:01 = 1 × 21 + 1 × 20 + 0 × 2−1 + 1 × 2−2: Double-precision floating-point format has 64 bits: ±; a1; : : : ; a11; b1; : : : ; b52 The associated number is a1···a11−1023 # = ±1:b1 ··· b52 × 2 : If all the a are zero, then the number is `subnormal' and the representation is −1022 # = ±0:b1 ··· b52 × 2 If the a are all 1 and the b are all 0 then it's ±∞ (± 1/0). If the a are all 1 and the b are not all 0 then it's NaN (±0=0). Machine-epsilon is the difference between 1 and the smallest representable number larger than 1, i.e. = 2−52 ≈ 2 × 10−16. The smallest and largest positive numbers are ≈ 10±308. Numbers larger than the max are set to 1. 2 Rounding & Arithmetic. Round-toward-nearest rounds a number towards the nearest floating-point num- ber. For nonzero numbers x within the range of max/min, the relative errors due to rounding are jround(x) − xj ≤ 2−53 ≈ 10−16 = u jxj which is half of machine epsilon. Floating-point arithmetic: add, subtract, multiply, divide. IEEE 754: If you start with 2 exactly- represented floating-point numbers x and y and compute any arithmetic operation `op', the magnitude of the relative error in the result is less than half machine precision. I.e. there is some δ with jδj ≤u such that fl(x op y) = (x op y)(1 + δ): The exception is underflow/overflow, where the result is either larger or smaller than can be represented. -
1.2 Round-Off Errors and Computer Arithmetic
1.2 Round-off Errors and Computer Arithmetic 1 • In a computer model, a memory storage unit – word is used to store a number. • A word has only a finite number of bits. • These facts imply: 1. Only a small set of real numbers (rational numbers) can be accurately represented on computers. 2. (Rounding) errors are inevitable when computer memory is used to represent real, infinite precision numbers. 3. Small rounding errors can be amplified with careless treatment. So, do not be surprised that (9.4)10= (1001.0110)2 can not be represented exactly on computers. • Round-off error: error that is produced when a computer is used to perform real number calculations. 2 Binary numbers and decimal numbers • Binary number system: A method of representing numbers that has 2 as its base and uses only the digits 0 and 1. Each successive digit represents a power of 2. (… . … ) where 0 1, for each = 2,1,0, 1, 2 … 3210 −1−2−3 2 • Binary≤ to decimal:≤ ⋯ − − (… . … ) (… 2 + 2 + 2 + 2 = 3210 −1−2−3 2 + 2 3 + 22 + 1 2 …0) 3 2 1 0 −1 −2 −3 −1 −2 −3 10 3 Binary machine numbers • IEEE (Institute for Electrical and Electronic Engineers) – Standards for binary and decimal floating point numbers • For example, “double” type in the “C” programming language uses a 64-bit (binary digit) representation – 1 sign bit (s), – 11 exponent bits – characteristic (c), – 52 binary fraction bits – mantissa (f) 1. 0 2 1 = 2047 11 4 ≤ ≤ − This 64-bit binary number gives a decimal floating-point number (Normalized IEEE floating point number): 1 2 (1 + ) where 1023 is called exponent − 1023bias. -
Floating Point Representation (Unsigned) Fixed-Point Representation
Floating point representation (Unsigned) Fixed-point representation The numbers are stored with a fixed number of bits for the integer part and a fixed number of bits for the fractional part. Suppose we have 8 bits to store a real number, where 5 bits store the integer part and 3 bits store the fractional part: 1 0 1 1 1.0 1 1 $ 2& 2% 2$ 2# 2" 2'# 2'$ 2'% Smallest number: Largest number: (Unsigned) Fixed-point representation Suppose we have 64 bits to store a real number, where 32 bits store the integer part and 32 bits store the fractional part: "# "% + /+ !"# … !%!#!&. (#(%(" … ("% % = * !+ 2 + * (+ 2 +,& +,# "# "& & /# % /"% = !"#× 2 +!"&× 2 + ⋯ + !&× 2 +(#× 2 +(%× 2 + ⋯ + ("%× 2 0 ∞ (Unsigned) Fixed-point representation Range: difference between the largest and smallest numbers possible. More bits for the integer part ⟶ increase range Precision: smallest possible difference between any two numbers More bits for the fractional part ⟶ increase precision "#"$"%. '$'#'( # OR "$"%. '$'#'(') # Wherever we put the binary point, there is a trade-off between the amount of range and precision. It can be hard to decide how much you need of each! Scientific Notation In scientific notation, a number can be expressed in the form ! = ± $ × 10( where $ is a coefficient in the range 1 ≤ $ < 10 and + is the exponent. 1165.7 = 0.0004728 = Floating-point numbers A floating-point number can represent numbers of different order of magnitude (very large and very small) with the same number of fixed bits. In general, in the binary system, a floating number can be expressed as ! = ± $ × 2' $ is the significand, normally a fractional value in the range [1.0,2.0) . -
IEEE Standard 754 for Binary Floating-Point Arithmetic
Work in Progress: Lecture Notes on the Status of IEEE 754 October 1, 1997 3:36 am Lecture Notes on the Status of IEEE Standard 754 for Binary Floating-Point Arithmetic Prof. W. Kahan Elect. Eng. & Computer Science University of California Berkeley CA 94720-1776 Introduction: Twenty years ago anarchy threatened floating-point arithmetic. Over a dozen commercially significant arithmetics boasted diverse wordsizes, precisions, rounding procedures and over/underflow behaviors, and more were in the works. “Portable” software intended to reconcile that numerical diversity had become unbearably costly to develop. Thirteen years ago, when IEEE 754 became official, major microprocessor manufacturers had already adopted it despite the challenge it posed to implementors. With unprecedented altruism, hardware designers had risen to its challenge in the belief that they would ease and encourage a vast burgeoning of numerical software. They did succeed to a considerable extent. Anyway, rounding anomalies that preoccupied all of us in the 1970s afflict only CRAY X-MPs — J90s now. Now atrophy threatens features of IEEE 754 caught in a vicious circle: Those features lack support in programming languages and compilers, so those features are mishandled and/or practically unusable, so those features are little known and less in demand, and so those features lack support in programming languages and compilers. To help break that circle, those features are discussed in these notes under the following headings: Representable Numbers, Normal and Subnormal, Infinite -
Variables and Calculations
¡ ¢ £ ¤ ¥ ¢ ¤ ¦ § ¨ © © § ¦ © § © ¦ £ £ © § ! 3 VARIABLES AND CALCULATIONS Now you’re ready to learn your first ele- ments of Python and start learning how to solve programming problems. Although programming languages have myriad features, the core parts of any programming language are the instructions that perform numerical calculations. In this chapter, we’ll explore how math is performed in Python programs and learn how to solve some prob- lems using only mathematical operations. ¡ ¢ £ ¤ ¥ ¢ ¤ ¦ § ¨ © © § ¦ © § © ¦ £ £ © § ! Sample Program Let’s start by looking at a very simple problem and its Python solution. PROBLEM: THE TOTAL COST OF TOOTHPASTE A store sells toothpaste at $1.76 per tube. Sales tax is 8 percent. For a user-specified number of tubes, display the cost of the toothpaste, showing the subtotal, sales tax, and total, including tax. First I’ll show you a program that solves this problem: toothpaste.py tube_count = int(input("How many tubes to buy: ")) toothpaste_cost = 1.76 subtotal = toothpaste_cost * tube_count sales_tax_rate = 0.08 sales_tax = subtotal * sales_tax_rate total = subtotal + sales_tax print("Toothpaste subtotal: $", subtotal, sep = "") print("Tax: $", sales_tax, sep = "") print("Total is $", total, " including tax.", sep = ") Parts of this program may make intuitive sense to you already; you know how you would answer the question using a calculator and a scratch pad, so you know that the program must be doing something similar. Over the next few pages, you’ll learn exactly what’s going on in these lines of code. For now, enter this program into your Python editor exactly as shown and save it with the required .py extension. Run the program several times with different responses to the question to verify that the program works. -
Absolute and Relative Error All Computations on a Computer Are Approximate by Nature, Due to the Limited Precision on the Computer
4 2 0 −1 −2 • 5.75 = 4 + 1 + 0.5 + 0.25 = 1 × 2 + 1 × 2 + 1 × 2 + 1 × 2 = 101.11(2) 4 0 −1 • 17.5 = 16 + 1 + 0.5 = 1 × 2 + 1 × 2 + 1 × 2 = 10001.1(2) 2 0 −1 −2 • 5.75 = 4 + 1 + 0.5 + 0.25 = 1 × 2 + 1 × 2 + 1 × 2 + 1 × 2 = 101.11(2) Note that certain numbers are finite (terminating) decimals, actually are peri- odic in binary, e.g. 0.4(10) = 0.01100110011 ...(2) = 0.00110011(2) Lecture of: Machine numbers (a.k.a. binary floating point numbers) 24 Jan 2013 The numbers stored on the computer are, essentially, “binary numbers” in sci- entific notation x = ±a × 2b. Here, a is called the mantissa and b the exponent. We also follow the convention that 1 ≤ a < 2; the idea is that, for any number x, we can always divide it by an appropriate power of 2, such that the result will be within [1, 2). For example: 2 2 x = 5(10) = 1.25(10) × 2 = 1.01(2) × 2 Thus, a machine number is stored as: b x = ±1.a1a2 ··· ak-1ak × 2 • In single precision we store k = 23 binary digits, and the exponent b ranges between −126 ≤ b ≤ 127. The largest number we can thus represent is (2 − 2−23) × 2127 ≈ 3.4 × 1038. • In double precision we store k = 52 binary digits, and the exponent b ranges between −1022 ≤ b ≤ 1023. The largest number we can thus represent is (2 − 2−52) × 21023 ≈ 1.8 × 10308. -
Introduction to the Python Language
Introduction to the Python language CS111 Computer Programming Department of Computer Science Wellesley College Python Intro Overview o Values: 10 (integer), 3.1415 (decimal number or float), 'wellesley' (text or string) o Types: numbers and text: int, float, str type(10) Knowing the type of a type('wellesley') value allows us to choose the right operator when o Operators: + - * / % = creating expressions. o Expressions: (they always produce a value as a result) len('abc') * 'abc' + 'def' o Built-in functions: max, min, len, int, float, str, round, print, input Python Intro 2 Concepts in this slide: Simple Expressions: numerical values, math operators, Python as calculator expressions. Input Output Expressions Values In [...] Out […] 1+2 3 3*4 12 3 * 4 12 # Spaces don't matter 3.4 * 5.67 19.278 # Floating point (decimal) operations 2 + 3 * 4 14 # Precedence: * binds more tightly than + (2 + 3) * 4 20 # Overriding precedence with parentheses 11 / 4 2.75 # Floating point (decimal) division 11 // 4 2 # Integer division 11 % 4 3 # Remainder 5 - 3.4 1.6 3.25 * 4 13.0 11.0 // 2 5.0 # output is float if at least one input is float 5 // 2.25 2.0 5 % 2.25 0.5 Python Intro 3 Concepts in this slide: Strings and concatenation string values, string operators, TypeError A string is just a sequence of characters that we write between a pair of double quotes or a pair of single quotes. Strings are usually displayed with single quotes. The same string value is created regardless of which quotes are used. -
Numerical Computations – with a View Towards R
Numerical computations { with a view towards R Søren Højsgaard Department of Mathematical Sciences Aalborg University, Denmark October 8, 2012 Contents 1 Computer arithmetic is not exact 1 2 Floating point arithmetic 1 2.1 Addition and subtraction . .2 2.2 The Relative Machine Precision . .2 2.3 Floating-Point Precision . .3 2.4 Error in Floating-Point Computations . .3 3 Example: A polynomial 3 4 Example: Numerical derivatives 6 5 Example: The Sample Variance 8 6 Example: Solving a quadratic equation 8 7 Example: Computing the exponential 9 8 Ressources 10 1 Computer arithmetic is not exact The following R statement appears to give the correct answer. R> 0.3 - 0.2 [1] 0.1 1 But all is not as it seems. R> 0.3 - 0.2 == 0.1 [1] FALSE The difference between the values being compared is small, but important. R> (0.3 - 0.2) - 0.1 [1] -2.775558e-17 2 Floating point arithmetic Real numbers \do not exist" in computers. Numbers in computers are represented in floating point form s × be where s is the significand, b is the base and e is the exponent. R has \numeric" (floating points) and \integer" numbers R> class(1) [1] "numeric" R> class(1L) [1] "integer" 2.1 Addition and subtraction Let s be a 7{digit number A simple method to add floating-point numbers is to first represent them with the same exponent. 123456.7 = 1.234567 * 10^5 101.7654 = 1.017654 * 10^2 = 0.001017654 * 10^5 So the true result is (1.234567 + 0.001017654) * 10^5 = 1.235584654 * 10^5 But the approximate result the computer would give is (the last digits (654) are lost) 2 1.235585 * 10^5 (final sum: 123558.5) In extreme cases, the sum of two non-zero numbers may be equal to one of them Quiz: how to sum a sequence of numbers x1; : : : ; xn to obtain large accuracy? 2.2 The Relative Machine Precision The accuracy of a floating-point system is measured by the relative machine precision or machine epsilon. -
CS 542G: Preliminaries, Floating-Point, Errors
CS 542G: Preliminaries, Floating-Point, Errors Robert Bridson September 8, 2008 1 Preliminaries The course web-page is at: http://www.cs.ubc.ca/∼rbridson/courses/542g. Details on the instructor, lectures, assignments and more will be available there. There is no assigned text book, as there isn’t one available that approaches the material with this breadth and at this level, but for some of the more basic topics you might try M. Heath’s “Scientific Computing: An Introductory Survey”. This course aims to provide a graduate-level overview of scientific computing: it counts for the “breadth” requirement in our degrees, and should be excellent preparation for research and further stud- ies involving scientific computing, whether for another subject (such as computer vision, graphics, statis- tics, or data mining to name a few within the department, or in another department altogether) or in scientific computing itself. While we will see many algorithms for many different problems, some histor- ical and others of critical importance today, the focus will try to remain on the bigger picture: powerful ways of analyzing and solving numerical problems, with an emphasis on common approaches that work in many situations. Alongside a practical introduction to doing scientific computing, I hope to develop your insight and intuition for the field. 2 Floating Point Unfortunately, the natural starting place for virtual any scientific computing course is in some ways the most tedious and frustrating part of the subject: floating point arithmetic. Some familiarity with floating point is necessary for everything that follows, and the main message—that all our computations will make (somewhat predictable) errors along the way—must be taken to heart, but we’ll try to get through it to more fun stuff fairly quickly.