ECE 3040 Lecture 8: Taylor Series Approximations I © Prof. Mohamad Hassoun This lecture covers the following topics: The Taylor series Taylor series expansion for some basic functions Evidence of digital machine rounding error Elementary function computation using Taylor series Rearrangement of functions to control rounding error Significance of the Taylor series expansion point a The Taylor Series The concept of a Taylor series was discovered by the Scottish mathematician James Gregory and formally introduced by the English mathematician Brook Taylor in 1715. Taylor’s series is of great value in the study of numerical methods and the implementation of numerical algorithms. In mathematics, a Taylor series is a representation of a function as an infinite sum of terms that are calculated from the values of the function's derivatives at a single point. The series is based on Taylor’s Theorem that states that any smooth function, 푓(푥), can be approximated, in the neighborhood of an expansion point a, as the polynomial: (푥 − 푎) (푥 − 푎)2 (푥 − 푎)푛 푓(푥) = 푓(푎) + 푓(1)(푎) + 푓(2)(푎) + ⋯ + 푓(푛)(푎) + ⋯ 1! 2! 푛! where 푓(푘)(푎) is the 푘th derivative, evaluated at 푥 = 푎. We may truncate the infinite series at the 푛th term and use the resulting 푛th degree polynomial 푝푛(푥) as an approximation for the original function 푓(푥); i.e., 푝푛(푥) ≅ 푓(푥): (푥 − 푎) (푥 − 푎)2 (푥 − 푎)푛 푝 (푥) = 푓(푎) + 푓(1)(푎) + 푓(2)(푎) + ⋯ + 푓(푛)(푎) 푛 1! 2! 푛! It can be shown that the truncation error, 푅푛 = 푓(푥) − 푝푛(푥), which is the sum of all series terms above the 푛th term, is given by: 푓(푛+1)(훾) 푅 = (푥 − 푎)푛+1 푛 (푛 + 1)! Where 훾 is an unknown value that lies between 푥 and 푎. Note that if |푥 − 푎| < 1, then only a small number of series terms would suffice to give very accurate approximation for 푓(푥) (note the factorial term in the denominator). If the Taylor series is centered at zero (i.e., the expansion point 푎 is set to zero), then that series is also called a Maclaurin series (or power series), named after the Scottish mathematician Colin Maclaurin who made extensive use of this special case of Taylor series in the 18th century. The power series is then, 푥 푥2 푥푛 푓(푥) = 푓(0) + 푓(1)(0) + 푓(2)(0) + ⋯ + 푓(푛)(0) + ⋯ 1! 2! 푛! It should be noted that the Maclaurin series (and, as a matter of fact, the Taylor series centered at any point 푎) of a polynomial function is finite, and is identical to the polynomial itself. Example. Find and compare the Taylor series approximation for 푓(푥) = sin(푥), at 푎 = 0. Solution: The derivatives of sin(푥) are: 푓(1)(푥) = cos(푥) , 푓(2)(푥) = − sin(푥) , 푓(3)(푥) = − cos(푥), 푓(4)(푥) = sin(푥), 푓(5) = cos(푥) , … Evaluating the derivatives at 0, we obtain 푓(1)(0) = 1, 푓(2)(0) = 0, 푓(3)(0) = −1, 푓(4)(0) = 0, 푓(5)(0) = 1 … which leads to the following polynomial approximations: 푝0(푥) = 푓(0) = sin(0) = 0 푥 푝 (푥) = 푓(0) + 푓(1)(0) = 0 + 푥(1) = 푥 1 1! 푥 푥2 푥 푝 (푥) = 푓(0) + 푓(1)(0) + 푓(2)(0) = 0 + 푥(1) + (0) = 푥 (same as 푝 ) 2 1! 2! 2 1 푥 푥2 푥3 푥3 푝 (푥) = 푓(0) + 푓(1)(0) + 푓(2)(0) + 푓(3)(0) = 푥 − 3 1! 2! 3! 3! 푥3 푝 (푥) = 푥 − (same as 푝 ) 4 3! 3 푥3 푥5 푝 (푥) = 푥 − + 5 3! 5! It is not surprising to see that only odd terms are present in the expansion, since sin(푥) is an odd function (has symmetry with respect to the origin). The following is a Matlab plot comparing the various 푝푛(푥) approximations of sin(푥): sin(푥) (red), 푝1(푥) (green), 푝3(푥) (magenta) and 푝5(푥) (blue). Note how the higher degree polynomial approximations are closer to sin(푥), over a wider range of 푥. Matlab has a convenient function for generating the Taylor polynomial: taylor(f,x,a,’order’,n). Here, f represents the function 푓(푥) defined symbolically, or as an anonymous function, x is the label for the independent variable, a is the point at which the expansion is to be performed (the default value is zero), and ‘order’,n generates an approximation up to and including the 푛 − 1 order term. If the order is not specified, then the default value 푛 = 6 is used. Here are few examples of the use of the taylor function. As would be expected, the Taylor series for a polynomial function is the function itself, regardless of the expansion point: The following is an 11-degree polynomial approximation for 푓(푥) = sin(푥). We may plot this 11-degree polynomial by first defining it as an anonymous function: Alternatively, we may utilize the sym2poly command which converts a symbolic polynomial to coefficient vector representation, and then use the polyval command to evaluate the polynomial, as follows: Taylor Series Expansion for Some Basic Functions The following is a list of Taylor/Maclaurin/power series expansions (at 푎 = 0) for several frequently encountered analytic functions. Notes: The power series can be used to generate infinite sums and their corresponding values. For example, setting 푥 = 1 in the series for 푒푥 leads to 푒1 = 푒 = 1 + 1 + 1 1 1 + + ⋯ or 푒 = ∑∞ . 2! 3! 푘=0 푘! (1 + 푥)푛 is an 푛-degree polynomial function in factored form. Hence, it is not surprising that its power series is an exact, finite polynomial of degree 푛. Here, the 푛 푛! binomial coefficient ( ) = . 푘 푘!(푛−푘)! 1 The last series gives rise to the geometric series ∑∞ 푎푘 = if −1 < 푎 < 1. 푘=0 1−푎 A power series can be expressed as infinite sum that is useful in programming. For example, cos(푥) can be expressed as, ∞ (−1)푛 cos(푥) = ∑ 푥2푛 (2푛)! 푛=0 that can be approximated by truncating the infinite series as follows: 푘 (−1)푛 cos(푥) ≅ ∑ 푥2푛 (2푛)! 푛=0 The following Matlab function cos_approx accepts a scalar 푥 and a positive, even integer 푚 and generates a table of values for the approximations of cos(푥) using polynomials of varying order: 0, 2, 4, … and 푚. The function also computes the absolute relative error. This is essentially programming Problem 7 in Lecture 6, but with an alternative expression for the approximating sum. Example from Electric Circuits The characteristic 푖 − 푣 relationship for a semiconductor diode can be expressed as 푖(푣) = 푎(푒푏푣 − 1) where 푖 is the current through the diode and 푣 is the voltage drop. The above equation is plotted in the following figure. The plot also depicts the graphical solution for the current when the voltage applied across the diode is 푣(푡) = 퐴푐표푠(휔푡). We may expand 푖(푣) employing a power series as (푏푣)2 (푏푣)3 푖(푣) = 푎 [(1 + 푏푣 + + + ⋯ ) − 1] 2 6 When the voltage 푣(푡) has small values we may approximate the current as 푎푏2 푖(푣) ≅ 푎푏푣 + 푣2 2 So, for the case 푣(푡) = 퐴cos(휔푡) with small 퐴 we obtain the current 푎푏2퐴2 푖(푡) ≅ 푎푏퐴cos(휔푡) + cos2(휔푡) 2 Now, we may (optionally) employ the trigonometric identity 1 1 cos2(푥) = + cos(2푥) 2 2 to the above approximation and obtain the truncated (Fourier series) form 푎푏2퐴2 푎푏2퐴2 푖(푡) ≅ + 푎푏퐴cos(휔푡) + cos(2휔푡) 4 4 The following plot compares the exact current 푖(푡) = 0.1(푒cos(푡) − 1) to its approximation 푖(푡) ≅ 0.025 + 0.1cos(푡) + 0.025cos(2푡). We have assumed 푎 = 0.1, 푏 = 1, 퐴 = 1 and 휔 = 1. [Your turn: repeat for 퐴 = 0.5, 1.5.] Your turn: Repeat the above example assuming a cubic expansion for 푖(푡). Evidence of Digital Machine Rounding Error Let us consider a polynomial and see what happens when we apply Taylor series expansion to it. Let 푓(푥) be simply equal to (푥 − 1)7. If we expand this function, we obtain a 7th degree polynomial: Let us now apply Taylor series expansion at 푎 = 0, with ‘order’ set to 21: As expected, the expansion is exact since 푓(푥) is a polynomial. Here is a plot, of the function (푥 − 1)7 and its expansion, for −4 < 푥 < 6 (the plots look identical): Let us now zoom-in and plot the two function forms in the neighborhood of 푥 = 1: The blue plot is for (푥 − 1)7 and the green one is for its expanded version: x^7 - 7*x^6 + 21*x^5 - 35*x^4 + 35*x^3 - 21*x^2 + 7*x – 1 For example, if 푥 = 1.01, then the two expressions will return, respectively: thus, resulting in a surprising absolute relative error of 42.11% What is the problem?! The two functions are identical and, theoretically, their plots must be identical. You are witnessing the machine round-off error in action. This is an error that is due to the inability of digital machines to represent some numbers perfectly. It is due to the finite number of binary bits available to represent decimal numbers. (Machine round-off errors are the topic of the Lecture 10). Elementary Function Computation using Taylor Series Many microprocessors have an adder circuit (build out of logic gates) which performs addition. They do not have additional logic circuits to perform other arithmetic operations (-, *, /), or elementary functions [such as sin(푥), cos(푥), 푒푥, ln(푥), etc].