Chapter 4 Numerical Di↵erentiation and Integration 4.1 Numerical Di↵erentiation In this section, we introduce how to numerically calculate the derivative of a function. First, the derivative of the function f at x0 is defined as f x0 h f x0 f 1 x0 : lim p ` q´ p q. p q “ h 0 h Ñ This formula gives an obvious way to generate an approximation to f x0 : simply compute 1p q f x0 h f x0 p ` q´ p q h for small values of h. Although this way may be obvious, it is not very successful, due to our old nemesis round-o↵error. But it is certainly a place to start. 2 To approximate f x0 , suppose that x0 a, b ,wheref C a, b , and that x1 x0 h for 1p q Pp q P r s “ ` some h 0 that is sufficiently small to ensure that x1 a, b . We construct the first Lagrange ‰ Pr s polynomial P0,1 x for f determined by x0 and x1, with its error term: p q x x0 x x1 f x P0,1 x p ´ qp ´ qf 2 ⇠ x p q“ p q` 2! p p qq f x0 x x0 h f x0 h x x0 x x0 x x0 h p qp ´ ´ q p ` qp ´ q p ´ qp ´ ´ qf 2 ⇠ x “ h ` h ` 2 p p qq ´ for some ⇠ x between x0 and x1. Di↵erentiating gives p q f x0 h f x0 x x0 x x0 h f 1 x p ` q´ p q Dx p ´ qp ´ ´ qf 2 ⇠ x p q“ h ` 2 p p qq „ ⇢ f x0 h f x0 2 x x0 h p ` q´ p q p ´ q´ f 2 ⇠ x “ h ` 2 p p qq x x0 x x0 h p ´ qp ´ ´ qDx f 2 ⇠ x . ` 2 p p qq ` ˘ Deleting the terms involving ⇠ x gives p q f x0 h f x0 f 1 x0 p ` q´ p q. p q« h 53 54 CHAPTER 4. NUMERICAL DIFFERENTIATION AND INTEGRATION One difficulty with this formula is that we have no information about Dx f ⇠ x ,sothe p 2 p p qqq truncation error cannot be estimated. When x is x0, however, the coefficient of Dx f ⇠ x p 2 p p qqq is 0, and the formula simplifies to f x0 h f x0 h f 1 x0 p ` q´ p q f 2 ⇠ . (4.1) p q“ h ´ 2 p q For small values of h, the di↵erence quotient f x0 h f x0 2 can be used to approximate r p ` q´ p qs{ f x0 with an error bounded by M h 2, where M is a bound on f x for x between x0 1p q | | { | 2p q| and x0 h. This formula is known as the forward-di↵erence formula for h 0 and the ` ° backward-di↵erence formula if h 0. † Example 4.1.1. Use the forward-di↵erence formula to approximate the derivative of f x ln x p q“ at x0 1.8usingh 0.1, h 0.05, and h 0.01, and determine bounds for the approximation “ “ “ “ errors. Solution. The forward-di↵erence formula f 1.8 h f 1.8 p ` q´ p q h with h 0.1 gives “ ln 1.9 ln 1.8 0.64185389 0.58778667 ´ ´ 0.5496722. 0.1 “ 0.1 “ Because f x 1 x2 and 1.8 ⇠ 1.9, a bound for this approximation error is 2p q“´ { † † hf ⇠ h 0.1 | 2p q| | | 0.0154321. 2 “ 2⇠2 † 2 1.8 2 “ p q The approximation and error bounds when h 0.05 and h 0.01 are found in a similar manner “ “ and the results are shown in the following table. f 1.8 h f 1.8 h hf1.8 h p ` q´ p q | | p ` q h 2 1.8 2 p q 0.10.64185389 0.5406722 0.0154321 0.05 0.61518564 0.5479795 0.0077160 0.01 0.59332685 0.5540180 0.0015432 Since f x 1 x, the exact value of f 1.8 is 0.555,¯ and in this case the error bounds are quite 1p q“ { 1p q close to the true approximation error. To obtain general derivative approximation formulas, suppose that x0,x1, ,xn are n 1 ¨¨¨ ` distinct numbers in some interval I and that f Cn 1 I . Using the Lagrange interpolation P ` p q formula, we have n x x0 x xn n 1 f x f x L x p ´ q¨¨¨p ´ qf p ` q ⇠ x , p q“ p kq kp q` n 1 ! p p qq k 0 ÿ“ p ` q for some ⇠ x I,whereL x denotes the k-th Lagrange polynomial for f at x0,x1, ,xn. p qP kp q ¨¨¨ Di↵erentiating this expression gives n x x0 x xn n 1 x x0 x xn n 1 f 1 x f x L1 x Dx p ´ q¨¨¨p ´ q f p ` q ⇠ x p ´ q¨¨¨p ´ qDx f p ` q ⇠ x . p q“ p kq kp q` n 1 ! p p qq` n 1 ! p p qq k 0 „ ⇢ ÿ“ p ` q p ` q ” ı 4.1. NUMERICAL DIFFERENTIATION 55 We again have a problem estimating the truncation error unless x is one of the numbers xj.In this case, the last term vanishes and the formula becomes n n 1 n f p ` q ⇠ xj f 1 xj f x L1 xj p p qq xj x , (4.2) p q“ p kq kp q` n 1 ! p ´ kq k 0 k 0,k j ÿ“ p ` q “π‰ which is called an n 1 -point formula to approximate f xj . p ` q 1p q In general, using more evaluation points in (4.2) produces greater accuracy, although the number of functional evaluations and growth of round-o↵error discourages this somewhat. The most common formulas are those involving three and five evaluation points. We first derive some useful three-point formulas and consider aspects of their errors. Since x x1 x x2 2x x1 x2 L0 x p ´ qp ´ q L01 x ´ ´ . p q“ x0 x1 x0 x2 ùñ p q“ x0 x1 x0 x2 p ´ qp ´ q p ´ qp ´ q Similarly, we have 2x x0 x2 2x x0 x1 L1 x ´ ´ and L21 x ´ ´ . p q“ x1 x0 x1 x2 p q“ x2 x0 x2 x1 p ´ qp ´ q p ´ qp ´ q Hence, the equation (4.2) becomes (with n 2) “ 2xj x1 x2 2xj x0 x2 f 1 xj f x0 ´ ´ f x1 ´ ´ p q“ p q x0 x1 x0 x2 ` p q x1 x0 x1 x2 „p ´ qp ´ q⇢ „p ´ qp ´ q⇢ 2 (4.3) 2xj x0 x1 1 3 f x2 ´ ´ f p q ⇠j xj x , ` p q x x x x ` 6 p q p ´ kq „ 2 0 2 1 ⇢ k 0,k j p ´ qp ´ q “π‰ for each j 0, 1, 2, where the notation ⇠j indicates that this points depends on xj. “ Three-point formulas The formula from (4.3) become especially useful if the nodes are equally spaced, that is, when x1 x0 h and x2 x0 2h for some h 0. “ ` “ ` ‰ We will assume equally-spaced nodes throughout the remainder of this section. Using (4.3) with xj x0, x1 x0 h, and x2 x0 2h, it gives “ “ ` “ ` 2 1 3 1 h 3 f 1 x0 f x0 2f x1 f x2 f p q ⇠0 . p q“h ´2 p q` p q´2 p q ` 3 p q „ ⇢ Doing the same for xj x1 gives “ 2 1 1 1 h 3 f 1 x1 f x0 f x2 f p q ⇠1 , p q“h ´2 p q`2 p q ´ 6 p q „ ⇢ and for xj x2, “ 2 1 1 3 h 3 f 1 x2 f x0 2f x1 f x2 f p q ⇠2 . p q“h 2 p q´ p q`2 p q ` 3 p q „ ⇢ Since x1 x0 h and x2 x0 2h, these formulas can also be expressed as “ ` “ ` 2 1 3 1 h 3 f 1 x0 f x0 2f x0 h f x0 2h f p q ⇠0 , p q“h ´2 p q` p ` q´2 p ` q ` 3 p q „ ⇢ 56 CHAPTER 4. NUMERICAL DIFFERENTIATION AND INTEGRATION 2 1 1 1 h 3 f 1 x0 h f x0 f x0 2h f p q ⇠1 , p ` q“h ´2 p q`2 p ` q ´ 6 p q „ ⇢ and 2 1 1 3 h 3 f 1 x0 2h f x0 2f x0 h f x0 2h f p q ⇠2 . p ` q“h 2 p q´ p ` q`2 p ` q ` 3 p q „ ⇢ As a matter of convenience, the variable substitution x0 for x0 h is used in the middle equation ` to change this formula to an approximation for f x0 . A similar change, x0 for x0 2h,isused 1p q ` in the last equation. This gives three formulas for approximating f x0 : 1p q 2 1 h 3 f 1 x0 3f x0 4f x0 h f x0 2h f p q ⇠0 , p q“2h r´ p q` p ` q´ p ` qs ` 3 p q 2 1 h 3 f 1 x0 f x0 h f x0 h f p q ⇠1 , p q“2h r p ` q´ p ´ qs ´ 6 p q and 2 1 h 3 f 1 x0 3f x0 4f x0 h f x0 2h f p q ⇠2 .