Numerical Differentiation
Total Page:16
File Type:pdf, Size:1020Kb
Jim Lambers MAT 460/560 Fall Semester 2009-10 Lecture 23 Notes These notes correspond to Section 4.1 in the text. Numerical Differentiation We now discuss the other fundamental problem from calculus that frequently arises in scientific applications, the problem of computing the derivative of a given function f(x). Finite Difference Approximations 0 Recall that the derivative of f(x) at a point x0, denoted f (x0), is defined by 0 f(x0 + h) − f(x0) f (x0) = lim : h!0 h 0 This definition suggests a method for approximating f (x0). If we choose h to be a small positive constant, then f(x + h) − f(x ) f 0(x ) ≈ 0 0 : 0 h This approximation is called the forward difference formula. 00 To estimate the accuracy of this approximation, we note that if f (x) exists on [x0; x0 + h], 0 00 2 then, by Taylor's Theorem, f(x0 +h) = f(x0)+f (x0)h+f ()h =2; where 2 [x0; x0 +h]: Solving 0 for f (x0), we obtain f(x + h) − f(x ) f 00() f 0(x ) = 0 0 + h; 0 h 2 so the error in the forward difference formula is O(h). We say that this formula is first-order accurate. 0 The forward-difference formula is called a finite difference approximation to f (x0), because it approximates f 0(x) using values of f(x) at points that have a small, but finite, distance between them, as opposed to the definition of the derivative, that takes a limit and therefore computes the derivative using an “infinitely small" value of h. The forward-difference formula, however, is just one example of a finite difference approximation. If we replace h by −h in the forward-difference formula, where h is still positive, we obtain the backward-difference formula f(x ) − f(x − h) f 0(x ) ≈ 0 0 : 0 h Like the forward-difference formula, the backward difference formula is first-order accurate. 1 If we average these two approximations, we obtain the centered-difference formula f(x + h) − f(x − h) f 0(x ) ≈ 0 0 : 0 2h To determine the accuracy of this approximation, we assume that f 000(x) exists on the interval [x0 − h; x0 + h], and then apply Taylor's Theorem again to obtain f 00(x ) f 000( ) f(x + h) = f(x ) + f 0(x )h + 0 h2 + + h3; 0 0 0 2 6 f 00(x ) f 000( ) f(x − h) = f(x ) − f 0(x )h + 0 h2 − − h3; 0 0 0 2 6 where + 2 [x0; x0 + h] and − 2 [x0 − h; x0]. Subtracting the second equation from the first and 0 solving for f (x0) yields f(x + h) − f(x − h) f 000( ) + f 000( ) f 0(x ) = 0 0 − + − h2: 0 2h 12 000 000 000 By the Intermediate Value Theorem, f (x) must assume every value between f (−) and f (+) on the interval (−; +), including the average of these two values. Therefore, we can simplify this equation to f(x + h) − f(x − h) f 000() f 0(x ) = 0 0 − h2; 0 2h 6 where 2 [x0−h; x0+h]. We conclude that the centered-difference formula is second-order accurate. 00 This is due to the cancellation of the terms involving f (x0). Example Consider the function p 2 x2+x sin cos x−x f(x) = p : sin p x−1 x2+1 Our goal is to compute f 0(0:25). Differentiating, using the Quotient Rule and the Chain Rule, we obtain p p p x2+x x2+x h 2x+1 x2+1(sin x+1) i 2 sin cos p + 2 0 cos x−x cos x−x 2 x2+1(cos x−x) (cos x−x) f (x) = p − sin p x−1 x2+1 p p p 2 x−1 h x( x−1) i sin x +x cos p p p1 − cos x−x x2+1 2 x x2+1 (x2+1)3=2 p : sin2 p x−1 x2+1 Evaluating this monstrous function at x = 0:25 yields f 0(0:25) = −9:066698770: 2 An alternative approach is to use a centered difference approximation, f(x + h) − f(x − h) f 0(x) ≈ : 2h Using this formula with x = 0:25 and h = 0:005, we obtain the approximation f(0:255) − f(0:245) f 0(0:25) ≈ = −9:067464295; 0:01 which has absolute error 7:7 × 10−4. While this complicated function must be evaluated twice to obtain this approximation, that is still much less work than using differentiation rules to compute f 0(x), and then evaluating f 0(x), which is much more complicated than f(x). 2 While Taylor's Theorem can be used to derive formulas with higher-order accuracy simply by evaluating f(x) at more points, this process can be tedious. An alternative approach is to compute the derivative of the interpolating polynomial that fits f(x) at these points. Specifically, suppose we want to compute the derivative at a point x0 using the data (x−j; y−j);:::; (x−1; y−1); (x0; y0); (x1; y1);:::; (xk; yk); where j and k are known nonnegative integers, x−j < x−j+1 < ⋅ ⋅ ⋅ < xk−1 < xk, and yi = f(xi) for 0 i = −j; : : : ; k. Then, a finite difference formula for f (x0) can be obtained by analytically computing k the derivatives of the Lagrange polynomials fLn;i(x)gi=−j for these points, where n = j + k + 1, and the values of these derivatives at x0 are the proper weights for the function values y−j; : : : ; yk. If f(x) is n + 1 times continuously differentiable on [x−j; xk], then we obtain an approximation of the form k k X f (n+1)() Y f 0(x ) = y L0 (x ) + (x − x ); 0 i n;i 0 (n + 1)! 0 i i=−j i=−j;i6=0 where 2 [x−j; xk]. Among the best-known finite difference formulas that can be derived using this approach is the second-order-accurate three-point formula −3f(x ) + 4f(x + h) − f(x + 2h) f 000() f 0(x ) = 0 0 0 + h2; 2 [x ; x + 2h]; 0 2h 3 0 0 which is useful when there is no information available about f(x) for x < x0. If there is no information available about f(x) for x > x0, then we can replace h by −h in the above formula to obtain a second-order-accurate three-point formula that uses the values of f(x) at x0, x0 − h and x0 − 2h. Another formula is the five-point formula f(x − 2h) − 8f(x − h) + 8f(x + h) − f(x + 2h) f (5)() f 0(x ) = 0 0 0 0 + h4; 2 [x − 2h; x + 2h]; 0 12h 30 0 0 3 which is fourth-order accurate. The reason it is called a five-point formula, even though it uses the value of f(x) at four points, is that it is derived from the Lagrange polynomials for the five points x0 − 2h; x0 − h; x0; x0 + h, and x0 + 2h. However, f(x0) is not used in the formula because 0 L4;0(x0) = 0, where L4;0(x) is the Lagrange polynomial that is equal to one at x0 and zero at the other four points. If we do not have any information about f(x) for x < x0, then we can use the following five-point formula that actually uses the values of f(x) at five points, −25f(x ) + 48f(x + h) − 36f(x + 2h) + 16f(x + 3h) − 3f(x + 4h) f (5)() f 0(x ) = 0 0 0 0 0 + h4; 0 12h 5 where 2 [x0; x0 + 4h]: As before, we can replace h by −h to obtain a similar formula that 0 approximates f (x0) using the values of f(x) at x0; x0 − h; x0 − 2h; x0 − 3h, and x0 − 4h. The strategy of differentiating Lagrange polynomials to approximate derivatives can be used to approximate higher-order derivatives. For example, the second derivative can be approximated using a centered difference formula, f(x + h) − 2f(x ) + f(x − h) f 00(x ) ≈ 0 0 0 ; 0 h2 which is second-order accurate. 0 Example We will construct a formula for approximating f (x) at a given point x0 by interpolating f(x) at the points x0, x0 + h, and x0 + 2h using a second-degree polynomial p2(x), and then 0 0 approximating f (x0) by p2(x0). Since p2(x) should be a good approximation of f(x) near x0, especially when h is small, its derivative should be a good approximation to f 0(x) near this point. Using Lagrange interpolation, we obtain p2(x) = f(x0)L2;0(x) + f(x0 + h)L2;1(x) + f(x0 + 2h)L2;2(x); 2 where fL2;j(x)gj=0 are the Lagrange polynomials for the points x0, x1 = x0 + h and x2 = x0 + 2h. Recall that these polynomials satisfy 1 if j = k L (x ) = = : 2;j k jk 0 otherwise Using the formula for the Lagrange polynomials, 2 Y (x − xi) L2;j(x) = ; (xj − xi) i=0;i6=j we obtain (x − (x0 + h))(x − (x0 + 2h)) L2;0(x) = (x0 − (x0 + h))(x0 − (x0 + 2h)) 4 x2 − (2x + 3h)x + (x + h)(x + 2h) = 0 0 0 ; 2h2 (x − x0)(x − (x0 + 2h)) L2;1(x) = (x0 + h − x0)(x0 + h − (x0 + 2h)) x2 − (2x + 2h)x + x (x + 2h) = 0 0 0 ; −h2 (x − x0)(x − (x0 + h)) L2;2(x) = (x0 + 2h − x0)(x0 + 2h − (x0 + h)) x2 − (2x + h)x + x (x + h) = 0 0 0 : 2h2 It follows that 2x − (2x + 3h) L0 (x) = 0 2;0 2h2 2x − (2x + 2h) L0 (x) = − 0 2;1 h2 2x − (2x + h) L0 (x) = 0 2;2 2h2 0 0 We conclude that f (x0) ≈ p2(x0), where 0 0 0 0 p2(x0) = f(x0)L2;0(x0) + f(x0 + h)L2;0(x0) + f(x0 + 2h)L2;0(x0) −3 2 −1 ≈ f(x ) + f(x + h) + f(x + 2h) 0 2h 0 h 0 2h 3f(x ) + 4f(x + h) − f(x + 2h) ≈ 0 0 0 : 2h Using Taylor's Theorem to write f(x0 + h) and f(x0 + 2h) in terms of Taylor polynomials centered 2 at x0, it can be shown that the error in this approximation is O(h ), and that this formula is exact when f(x) is a polynomial of degree 2 or less.