Lecture 3 Taylor Series and Floating Point Representation J. Chaudhry (Zeb) Department of Mathematics and Statistics University of New Mexico J. Chaudhry (Zeb) (UNM) Math/CS 375 1 / 30 What we’ll do: Taylor Series Floating point representation J. Chaudhry (Zeb) (UNM) Math/CS 375 2 / 30 Taylor All we can ever do is add and multiply p We can’t directly evaluate ex, cos(x), x What to do? Taylor Series approximation Taylor The Taylor series expansion of f (x) at the point x = c is given by (x - c)2 (x - c)n f (x) = f (c) + (x - c)f 0(c) + f 00(c) + ··· + f (n)(c) + ... 2! n! (x - c)k = f (k)(c) 1 k! = Xk 0 J. Chaudhry (Zeb) (UNM) Math/CS 375 3 / 30 Taylor Example Taylor The Taylor series expansion of f (x) about the point x = c is given by (x - c)2 (x - c)n f (x) = f (c) + (x - c)f 0(c) + f 00(c) + ··· + f (n)(c) + ... 2! n! (x - c)k = f (k)(c) 1 k! = Xk 0 Example (ex) We know e0 = 1, so expand about c = 0 to get (x - 0)2 f (x) = ex = 1 + (x - 0) · 1 + · 1 + ... 2 x2 x3 = 1 + x + + + ... 2! 3! J. Chaudhry (Zeb) (UNM) Math/CS 375 4 / 30 Taylor Approximation So 22 23 e2 = 1 + 2 + + + ... 2! 3! But we can’t evaluate an infinite series, so we truncate... Taylor Series Polynomial Approximation The Taylor Polynomial of degree n for the function f (x) about the point c is n (x - c)k p (x) = f (k)(c) n k! = Xk 0 Example (ex) In the case of the exponential x2 xn ex ≈ p (x) = 1 + x + + ··· + n 2! n! J. Chaudhry (Zeb) (UNM) Math/CS 375 5 / 30 Taylor Approximation Evaluate e2: Using 0th order Taylor series: ex ≈ 1 does not give a good fit. Using 1st order Taylor series: ex ≈ 1 + x gives a better fit. Using 2nd order Taylor series: ex ≈ 1 + x + x2=2 gives a a really good fit. 1 x=2; 2 pn=0; 3 for j=0:15 4 pn = pn + (x^j)/factorial(j); 5 err = exp(2)-pn 6 pause 7 end J. Chaudhry (Zeb) (UNM) Math/CS 375 6 / 30 Taylor Approximation (Important to know all three forms) Infinite Taylor Series Expansion (exact) (x - c)2 (x - c)n f (x) = f (c) + (x - c)f 0(c) + f 00(c) + ··· + f (n)(c) + ... 2! n! Finite Taylor Series Expansion (exact) (x - c)2 (x - c)n f (x) = f (c) + (x - c)f 0(c) + f 00(c) + ··· + f (n)(ξ), 2! n! where ξ lies between c and x but we don’t know its value. Finite Taylor Series Approximation (x - c)2 (x - c)n f (x) ≈ f (c) + (x - c)f 0(c) + f 00(c) + ··· + f (n)(c). 2! n! J. Chaudhry (Zeb) (UNM) Math/CS 375 7 / 30 Taylor Approximation Error How accurate is the Taylor series polynomial approximation? The n terms of the approximation are simply the first n terms of the exact expansion: x2 x3 ex = 1 + x + + + ... (1) 2! 3! x p2 approximation to e truncation error So the function f (x) can be| written{z as} the Taylor| {z Series} approximation plus an error (truncation) term: f (x) = pn(x) + En(x) where (x - c)n+1 E (x) = f (n+1)(ξ) n (n + 1)! where ξ lies between c and x, that is ξ 2 [c, x]. J. Chaudhry (Zeb) (UNM) Math/CS 375 8 / 30 Truncation Error The Taylor series expansion of sin (x) is x3 x5 x7 x9 sin (x) = x - + - + - ... 3! 5! 7! 9! If x 1, then the remaining terms are small. If we neglect these terms x3 x5 x7 x9 sin (x) = x - + - + - ... 3! 5! 7! 9! approximation to sin truncation error | {z } | {z } J. Chaudhry (Zeb) (UNM) Math/CS 375 9 / 30 Taylor Series: Computations ( ) = 1 How do we evaluate f x 1-x computationally? Taylor Series Expansion: (x - c)2 (x - c)n f (x) = f (c) + (x - c)f 0(c) + f 00(c) + ··· + f (n)(ξ), 2! n! Thus with c = 0 1 = 1 + x + x2 + x3 + ... 1 - x Second order approximation: 1 ≈ 1 + x + x2 1 - x J. Chaudhry (Zeb) (UNM) Math/CS 375 10 / 30 Taylor Errors How many terms do I need to make sure my error is less than 2 × 10-8 for x = 1=2? 1 = 1 + x + x2 + ··· + xn + xk 1 - x 1 = + k Xn 1 so the error at x = 1=2 is 1k (1=2)n+1 ex=1=2 = = 1 2 1 - 1=2 = + k Xn 1 = 2 · (1=2)n+1 < 2 × 10-8 then we need -8 n + 1 > ≈ 26.6 or log10(1=2) n > 26 J. Chaudhry (Zeb) (UNM) Math/CS 375 11 / 30 Taylor Series: Alternate Forms The Taylor series expansion of f (x) at the point x = c is given by Taylor (x - c)2 (x - c)n f (x) = f (c) + (x - c)f 0(c) + f 00(c) + ··· + f (n)(c) + ... 2! n! (x - c)k = f (k)(c) 1 k! = Xk 0 Substitute x = x + h and c = x, Taylor: Alternate h2 hn f (x + h) = f (x) + hf 0(x) + f 00(x) + ··· + f (n)(x) + ... 2! n! hk = f (k)(x) 1 k! = Xk 0 J. Chaudhry (Zeb) (UNM) Math/CS 375 12 / 30 Floating Point Numbers We’re familiar with base 10 representation of numbers: 1234 = 4 × 100 + 3 × 101 + 2 × 102 + 1 × 103 and .1234 = 1 × 10-1 + 2 × 10-2 + 3 × 10-3 + 4 × 10-4 we write 1234.1234 as an integer part and a fractional part: a3a2a1a0.b1b2b3b4 For some (even simple) numbers, there may be an infinite number of digits to the right: π = 3.14159 ... 1=9 = 0.11111 ... p 2 = 1.41421 ... J. Chaudhry (Zeb) (UNM) Math/CS 375 13 / 30 Other bases So far, we have just base 10. What about base β? binary (β = 2), octal (β = 8), hexadecimal (β = 16), etc In the β-system we have n k -k (an ... a2a1a0.b1b2b3b4 ... )β = akβ + bkβ 1 = = Xk 0 Xk 1 J. Chaudhry (Zeb) (UNM) Math/CS 375 14 / 30 Conversion from base-2 to base-10 Convert a base-2 number to base-10: (1001.101)2 = 1 × 23 + 0 × 22 + 0 × 21 + 1 × 20 + 1 × 2-1 + 0 × 2-2 + 1 × 2-3 = 8 + 0 + 0 + 1 + 0.5 + 0 + 0.125 = 9.62 1 = For other numbers, such as 5 0.2, an infinite length is needed. 0.2 ! .0011 0011 0011 ... So 0.2 is stored just fine in base-10, but needs infinite number of digits in base-2 !!! This is roundoff error in its basic form... J. Chaudhry (Zeb) (UNM) Math/CS 375 15 / 30 Numerical ”bugs” Obvious Software has bugs Not-SO-Obvious Numerical software has two unique bugs: 1 roundoff error 2 truncation error J. Chaudhry (Zeb) (UNM) Math/CS 375 16 / 30 Numerical Errors Roundoff Roundoff occurs when digits in a decimal point (0.3333...) are lost (0.3333) due to a limit on the memory available for storing one numerical value. Truncation Truncation error occurs when discrete values are used to approximate a mathematical expression (e.g. finite number of terms in Taylor series). J. Chaudhry (Zeb) (UNM) Math/CS 375 17 / 30 Uncertainty: well- or ill-conditioned? Errors in input data can cause uncertain results input data can be experimental or rounded. leads to a certain variation in the results well-conditioned: numerical results are insensitive to small variations in the input ill-conditioned: small variations lead to drastically different numerical calculations (a.k.a. poorly conditioned) J. Chaudhry (Zeb) (UNM) Math/CS 375 18 / 30 Our Job As numerical analysts, we need to 1 solve a problem so that the calculation is not susceptible to large roundoff error 2 solve a problem so that the approximation has a tolerable truncation error How? incorporate roundoff-truncation knowledge into I the mathematical model I the method I the algorithm I the software design awareness ! correct interpretation of results J. Chaudhry (Zeb) (UNM) Math/CS 375 19 / 30 Floating Points Normalized Floating-Point Representation Real numbers are stored as e x = ±(0.d1d2d3 ... dm)β × β d1d2d3 ... dm is the mantissa, e is the exponent e is negative, positive or zero the general normalized form requires d1 , 0 J. Chaudhry (Zeb) (UNM) Math/CS 375 20 / 30 Floating Point Example In base 10 1000.12345 can be written as 4 (0.100012345)10 × 10 0.000812345 can be written as -3 (0.812345)10 × 10 J. Chaudhry (Zeb) (UNM) Math/CS 375 21 / 30 Floating Point Suppose we have only 3 bits for a mantissa and a 2 bit exponent stored like .d1 d2 d3 e1 e2 All possible combinations give: 0002 = 0 ... × 2-1,0,1 1112 = 7 1 2 7 1 2 7 1 2 7 So we get 0, 16 , 16 ,..., 16 , 0, 4 , 4 ,..., 4 , and 0, 8 , 8 ,..., 8 . On the real line: J. Chaudhry (Zeb) (UNM) Math/CS 375 22 / 30 Overflow, Underflow computations too close to zero may result in underflow computations too large may result in overflow overflow error is considered more severe underflow can just fall back to 0 J. Chaudhry (Zeb) (UNM) Math/CS 375 23 / 30 Normalizing -1,0,1 If we use the normalized form in our 5-bit case, we lose 0.0012 × 2 .