MATH1070 4 Numerical Integr

Home , Rate of convergence, Trapezoidal rule

> 5. Numerical Integration Review of Interpolation

Find pn(x) with pn(xj) = yj, j = 0, 1, 2, . . . , n. Solution: Qn x−xj pn(x) = y0`0(x) + y1`1(x) + ... + yn`n(x), `k(x) = . j=1,j6=k xk−xj Theorem

Let yj = f(xj), f(x) smooth and pn interpolates f(x) at x0 < x1 < . . . < xn. For any x ∈ (x0, xn) there is a ξ such that

f (n+1)(ξ) f(x) − pn(x) = (x − x0)(x − x1) ... (x − xn) (n + 1)! | {z } ψ(x)

Example: n = 1, linear: f 00 (ξ) f(x) − p (x) = (x − x )(x − x ), 1 2 0 1 y1 − y0 p1(x) = (x − x0) + y0. x1 − x0

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.1 The Trapezoidal Rule Mixed rule

Find area under the curve y = f(x)

N−1 N−1 Z b Z xj+1 X X f(xj) + f(xj+1) f(x)dx = f(x)dx ≡ (x − x ) j+1 j 2 a j=0 xj j=0 | {z } interpolate f(x) on (xj, xj+1) and integrate exactly

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.1 The Trapezoidal Rule Trapezoidal Rule

Z xj+1 Z xj+1 f(x) ≈ p1(x)dx xj xj Z xj+1 f(xj+1) + f(xj) = (x − xj) + f(xj) dx xj xj+1 − xj f(x ) + f(x ) = (x − x ) j j+1 j+1 j 2

f(b) + f(a) T (f) = (b − a) (6.1) 1 2

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.1 The Trapezoidal Rule Another derivation of Trapezoidal Rule:

Seek: Z xj+1 f(x)dx ≈ wjf(xj) + wj+1f(xj+1) xj exact on polynomials of degree 1 (1, x).

Z xj+1 f(x) ≡ 1 : f(x)dx = xj+1 − xj = wj · 1 + wj+1 · 1 xj Z xj+1 2 2 xj+1 xj f(x) = x : xdx = − = wjxj + wj+1xj+1 xj 2 2

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.1 The Trapezoidal Rule

Example Approximate integral Z 1 dx I = 0 1 + x The true value is I = ln(2) =· 0.693147. Using (6.1), we obtain

1 1 3 T = 1 + = = 0.75 1 2 2 4

and the error is

· I − T1(f) = −0.0569 (6.2)

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.1 The Trapezoidal Rule To improve on approximation (6.1) when f(x) is not a nearly linear function on [a, b], break interval [a, b] into smaller subintervals and apply (6.1) on each subinterval. If the subintervals are small enough, then f(x) will be nearly linear on each one. Example

Evaluate the preceding example by using T1(f) on two subintervals of equal length.

For two subintervals, 1 1 2 2 1 Z 2 dx Z dx 1 1 + 1 + I = + =· 3 + 3 2 1 + x 1 1 + x 2 2 2 2 0 2 17 T = =· 0.70833 2 24 and the error · I − T2 = −0.0152 (6.3) 1 is about 4 of that for T1 in (6.2).

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.1 The Trapezoidal Rule We derive the general formula for calculations using n subintervals of b−a equal length h = n . The endpoints of each subinterval are then

xj = a + jh, j = 0, 1, . . . , n

Breaking the integral into n subintegrals

Z b Z xn I(f) = f(x)dx = f(x)dx a x0 Z x1 Z x2 Z xn = f(x)dx + f(x)dx + ... + f(x)dx x0 x1 xn−1 f(x ) + f(x ) f(x ) + f(x ) f(x ) + f(x ) ≈ h 0 1 + h 1 2 + ... + h n−1 n 2 2 2 The trapezoidal numerical integration rule 1 1 T (f)=h f(x )+f(x )+f(x )+···+f(x )+ f(x ) (6.4) n 2 0 1 2 n−1 2 n

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.1 The Trapezoidal Rule

With a sequence of increasing values of n, Tn(f) will usually be an increasingly accurate approximation of I(f). But which sequence of n should be used? If n is doubled repeatedly, then the function values used in each T2n(f) will include all earlier function values used in the preceding Tn(f). Thus, the doubling of n will ensure that all previously computed information is used in the new calculation, making the trapezoidal rule less expensive than it would be otherwise. f(x ) f(x ) T (f) = h 0 + f(x ) + 2 2 2 1 2

with b − a a + b h = , x = a, x = , x = b. 2 0 1 2 2 Also f(x ) f(x ) T (f) = h 0 + f(x ) + f(x ) + f(x ) + 4 4 2 1 2 3 2 with b − a 3a + b a + b a + 3b h = , x = a, x = , x = , x = , x = b 4 0 1 4 2 4 3 4 4

Only f(x1) and f(x3) need to be evaluated. 5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.1 The Trapezoidal Rule Example We give calculations of Tn(f) for three integrals 1 Z 2 I(1) = e−x dx =· 0.746824132812427 0 Z 4 (2) dx −1 · I = 2 = tan (4) = 1.32581766366803 0 1 + x Z 2π dx 2π I(3) = = √ =· 3.62759872846844 0 2 + cos(x) 3

n I(1) I(2) I(3) Error Ratio Error Ratio Error Ratio 2 1.55E-2 -1.33E-1 -5.61E-1 4 3.84E-3 4.02 -3.59E-3 37.0 -3.76E-2 14.9 8 9.59E-4 4.01 5.64E-4 -6.37 -1.93E-4 195.0 16 2.40E-4 4.00 1.44E-4 3.92 -5.19E-9 37,600.0 32 5.99E-5 4.00 3.60E-5 4.00 * 64 1.50E-5 4.00 9.01E-6 4.00 * 128 3.74E-6 4.00 2.25E-6 4.00 * The error for I(1),I(2) decreases by a factor of 4 when n doubles, for I(3) the answers for n = 32, 64, 128 were

correct up to the limits due to rounding error on the computer (16 decimal digits). 5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.1.1 Simpson’s rule

To improve on T1(f) in (6.1), use quadratic interpolation to approximate f(x) on [a, b]. Let P2(x) be the quadratic polynomial that interpolates f(x) a+b at a, c = 2 and b. Z b I(f) ≈ P2(x)dx (6.5) a Z b (x−c)(x−b) (x−a)(x−b) (x−a)(x−c) = f(a) + f(c) + f(b) dx a (a−c)(a−b) (c−a)(c−b) (b−a)(b−c) This can be evaluated directly. But it is easier with a change of variables. b−a Let h = 2 and u = x − a. Then Z b (x−c)(x−b) 1 Z a+2h dx = 2 (x − c)(x − b)dx a (a−c)(a−b) 2h a Z 2h 3 2h 1 1 u 3 2 2 h = 2 (u − h)(u − 2h)du = 2 − u h + 2h u = 2h 0 2h 3 2 0 3 and h a + b S (f) = f(a) + 4f( ) + f(b) (6.6) 2 3 2

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.1.1 Simpson’s rule Example Z 1 dx I = 0 1 + x

b−a 1 Then h = 2 = 2 and 1 2 1 25 S (f) = 2 1 + 4( ) + = =· 0.69444 (6.7) 2 3 3 2 36 and the error is · I − S2 = ln(2) − S2 = −0.00130 while the error for the trapezoidal rule (the number of function evaluations is the same for both S2 and T2) was · I − T2 = −0.0152.

The error in S2 is smaller than that in (6.3) for T2 by a factor of 12, a signiﬁcant increase in accuracy. 5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.1.1 Simpson’s rule

0.95

0.9

0.85

0.8

0.75

0.7

0.65

0.6

0.55

0.5 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Figure: An illustration of Simpson’s rule (6.6), y = f(x), y = P2(x) 5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.1.1 Simpson’s rule

The rule S2(f) will be an accurate approximation to I(f) if f(x) is nearly quadratic on [a, b]. For the other cases, proceed in the same manner as for the trapezoidal rule. b−a Let n be an even integer, h = n and deﬁne the evaluation points for f(x) by xj = a + jh, j = 0, 1, . . . , n

We follow the idea from the trapezoidal rule, but break [a, b] = [x0, xn] into larger intervals, each containing three interpolation node points:

Z b Z xn I(f) = f(x)dx = f(x)dx a x0 Z x2 Z x4 Z xn = f(x)dx + f(x)dx + ... + f(x)dx x0 x2 xn−2 h h ≈ [f(x ) + 4f(x ) + f(x )] + [f(x ) + 4f(x ) + f(x )] 3 0 1 2 3 2 3 4 h + ... + [f(x ) + 4f(x ) + f(x )] 3 n−2 n−1 n

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.1.1 Simpson’s rule

Simpson’s rule: h S (f) = (f(x )+4f(x )+2f(x )+4f(x )+2f(x ) (6.8) n 3 0 1 2 3 4 +···+2f(xn−2)+4f(xn−1)+f(xn))

It has been among the most popular numerical integration methods for more than two centuries.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.1.1 Simpson’s rule Example Evaluate the integrals 1 Z 2 I(1) = e−x dx =· 0.746824132812427 0 Z 4 (2) dx −1 · I = 2 = tan (4) = 1.32581766366803 0 1 + x Z 2π dx 2π I(3) = = √ =· 3.62759872846844 0 2 + cos(x) 3

n I(1) I(2) I(3) Error Ratio Error Ratio Error Ratio 2 -3.56E-4 8.66E-2 -1.26 4 -3.12E-5 11.4 3.95E-2 2.2 1.37E-1 -9.2 8 -1.99E-6 15.7 1.95E-3 20.3 1.23E-2 11.2 16 -1.25E-7 15.9 4.02E-6 485.0 6.43E-5 191.0 32 -7.79E-9 16.0 2.33E-8 172.0 1.71E-9 37,600.0 64 -4.87E-10 16.0 1.46E-9 16.0 * 128 -3.04E-11 16.0 9.15E-11 16.0 * For I(1),I(2), the ratio by which the error decreases approaches 16.

For I(3), the errors converge to zero much more rapidly 5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2 Error formulas

Theorem 2 Let f ∈ C [a, b], n ∈ N. The error in integrating Z b I(f) = f(x)dx a using the trapezoidal rule 1 1 Tn(f) = h 2 f(x0) + f(x1) + f(x2) + ··· + f(xn−1) + 2 f(xn) is given by

−h2(b − a) ET ≡ I(f) − T (f) = f 00(c ) (6.9) n n 12 n

b−a where cn is some unknown point in a, b, and h = n .

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2 Error formulas Theorem 2 Suppose that f ∈ C [a, b], and h = maxj(xj+1 − xj). Then

Z b X f(xj+1)+f(xj) b − a 2 00 f(x)dx − (xj+1 −xj) ≤ h max |f (x)|. 2 12 a≤x≤b a j

th Proof: Let Ij be the j subinterval and p1 = linear interpolant on Ij at xj, xj+1. f 00(x) f(x) − p1(x) = (x − xj)(x − xj+1) . 2 | {z } ψ(x) Local error:

Z xj+1 f(x ) + f(x ) f(x)dx − (x −x ) j j+1 = j+1 j 2 xj

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2 Error formulas proof

Z xj+1 f 00(ξ) = (x − x )(x − x ) 2! j j+1 xj Z xj+1 1 00 ≤ |f (ξ)||x − xj| |x − xj+1|dx 2 xj Z xj+1 1 00 ≤ max |f (x)| (x − xj)(xj+1 − x)dx a≤x≤b 2 xj 1 (x − x )3 = max |f 00(x)| j+1 j . 2 a≤x≤b 6 Hence

1 00 3 |local error| ≤ max |f (x)| · hj , hj = xj+1 − xj. 12 a≤x≤b

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2 Error formulas proof

Finally,

Z b X f(xj) + f(xj+1) |global error| = f(x)dx − (xj+1 −xj) 2 a j ! Z xj+1 X f(xj) + f(xj+1) X = f(x)dx − (xj+1 −xj) ≤ |local error| 2 j xj j n X 1 00 3 1 00 X 3 ≤ max |f (x)|(xj+1 − xj) = max |f (x)| (xj+1 − xj) 12 a≤x≤b 12 a≤x≤b | {z } j j=0 2 ≤h (xj+1−xj ) 2 n h 00 X ≤ max |f (x)| (xj+1 − xj) . 12 a≤x≤b j=0 | {z } b−a

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2 Error formulas Example Recall the example Z 1 dx I(f) = = ln 2 0 1 + x

1 00 2 Here f(x) = 1+x , [a, b] = [0, 1], and f (x) = (1+x)3 . Then by (6.9) h2 1 ET (f) = − f 00(c ), 0 ≤ c ≤ 1, h = . n 12 n n n

This cannot be computed exactly since cn is unknown. But 2 max |f 00(x)| = max = 2 0≤x≤1 0≤x≤1 (1 + x)3 and therefore h2 h2 |ET (f)| ≤ (2) = n 12 6 For n = 1 and n = 2 we have 1 2 T 1 · T 2 · | E1 (f) | ≤ = 0.167, | E2 (f) | ≤ = 0.0417. | {z } 6 | {z } 6 −0.0569 −0.0152

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2 Error formulas

A possible weakness in the trapezoidal rule can be inferred from the assumption of the theorem for the error. If f(x) does not have two continuous derivatives on [a, b], then Tn(f) does converge more slowly??

YES

for some functions, especially if the ﬁrst derivative is not continuous.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2.1 Asymptotic estimate of Tn(f)

The error formula (6.9) T −h2(b−a) 00 En (f) ≡ I(f) − Tn(f) = 12 f (cn) 00 can only be used to bound the error, because f (cn) is unknown.

This can be improved by a more careful consideration of the error formula.

A central element of the proof of (6.9) lies in the local error

Z α+h f(α) + f(α + h) h3 f(x)dx − h = − f 00(c) (6.10) α 2 12 for some c ∈ [α, α + h].

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2.1 Asymptotic estimate of Tn(f)

Recall the derivation of the trapezoidal rule Tn(f) and use the local error (6.10):

Z b Z xn T En (f) = f(x)dx − Tn(f) = f(x)dx − Tn(f) a x0 Z x1 f(x ) + f(x ) Z x2 f(x ) + f(x ) = f(x)dx − h 0 1 + f(x)dx − h 1 2 x0 2 x1 2 Z xn f(x ) + f(x ) + ··· + f(x)dx − h n−1 n xn−1 2 h3 h3 h3 = − f 00(γ ) − f 00(γ ) − · · · − − f 00(γ ) 12 1 12 2 12 n

with γ1 ∈ [x0, x1], γ2 ∈ [x1, x2], . . . γn ∈ [xn−1, xn], and

2 T h 00 00 En (f) = − hf (γ1) + ··· + hf (γn) , cn ∈ [a, b]. 12 | {z } 00 =(b−a)f (cn)

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2.1 Asymptotic estimate of Tn(f)

To estimate the trapezoidal error, observe that 00 00 hf (γ1) + ··· + hf (γn) is a Riemann sum for the integral

Z b f 00(x)dx = f 0(b) − f 0(a) (6.11) a The Riemann sum is based on the partition

[x0, x1], [x1, x2],..., [xn−1, xn] of [a, b].

As n → ∞, this sum will approach the integral (6.11). With (6.11), we ﬁnd an asymptotic estimate (improves as n increases)

−h2 ET (f) ≈ (f 0(b) − f 0(a)) =: EeT (f). (6.12) n 12 n

0 T As long as f (x) is computable, Een (f) will be very easy to compute.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2.1 Asymptotic estimate of Tn(f)

Example Z 1 dx Again consider I = . 0 1 + x 1 Then f 0(x) = − , and the asymptotic estimate (6.12) yields (1 + x)2 the estimate −h2 −1 −1 −h2 1 EeT = − = , h = n 12 (1 + 1) (1 + 0)2 16 n

and for n = 1 and n = 2

T 1 T · Ee1 = − 16 = −0.0625, Ee2 = −0.0156 · · I − T1 = −0.0569,I − T2 = −0.0152

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2.1 Asymptotic estimate of Tn(f) T −h2 0 0 The estimate Een (f) = 12 (f (b) − f (a)) has several practical T −h2(b−a) 00 advantages over the earlier formula (6.9) En (f) = 12 f (cn). 1 It conﬁrms that when n is doubled (or h is halved), the error decreases by a factor of about 4, provided that f 0(b) − f 0(a) 6= 0. This agrees with the results for I(1) and I(2).

2 (6.12) implies that the convergence of Tn(f) will be more rapid when f 0(b) − f 0(a) = 0. This is a partial explanation of the very rapid convergence observed with I(3)

3 (6.12) leads to a more accurate numerical integration formula T by taking Een (f) into account: h2 I(f) − T (f) ≈ − (f 0(b) − f 0(a)) n 12 h2 I(f) ≈ T (f) − (f 0(b) − f 0(a)) := CT (f), (6.13) n 12 n the corrected trapezoidal rule

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2.1 Asymptotic estimate of Tn(f) Example

(1) R 1 −x2 · Recall the integral I , I = 0 e dx = 0.74682413281243

n I − Tn(f) Een(f) CTn(f) I − CTn(f) Ratio 2 1.545E-4 1.533E-2 0.746698561877 1.26E-4 4 3.840E-3 3.832E-3 0.746816175313 7.96E-6 15.8 8 9.585E-4 9.580E-4 0.746823634224 4.99E-7 16.0 16 2.395E-4 2.395E-4 0.746824101633 3.12E-8 16.0 32 5.988E-5 5.988E-5 0.746824130863 1.95E-9 16.0 64 1.497E-5 1.497E-5 0.746824132690 2.22E-10 16.0

Table: Example of CTn(f) and Een(f) Note that the estimate h2e−1 1 EeT (f) = , h = n 6 n is a very accurate estimator of the true error. Also, the error in CTn(f) converges to zero at a more rapid rate than does the error for Tn(f). When n is doubled, the error in CTn(f) decreases by a factor of about 16.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2.2 Error formulae for Simpson’s rule Theorem Assume f ∈ C4[a, b], n ∈ N. The error in using Simpson’s rule is h4(b − a) ES(f) = I(f) − S (f) = − f (4)(c ) (6.14) n n 180 n

b−a with cn ∈ [a, b] an unknown point, and h = n . Moreover, this error can be estimated with the asymptotic error formula h4 EeS(f) = − (f 000(b) − f 000(a)) (6.15) n 180 Note that (6.14) says that Simpson’s rule is exact for all f(x) that are polynomials of degree ≤ 3, whereas the quadratic interpolation on which Simpson’s rule is based is exact only for f(x) a polynomial of degree ≤ 2. The degree of precision being 3 leads to the power h4 in the error, rather than the power h3, which would have been produced on the basis of the error in quadratic interpolation.

The higher power of h4, and the simple form of the method

that historically have caused Simpson’s rule to become the most popular numerical integration rule.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2.2 Error formulae for Simpson’s rule Example R 1 dx Recall (6.7) where S2(f) was applied to I = 0 1+x : 1 2 1 25 S (f) = 2 1 + 4( ) + = =· 0.69444 2 3 3 2 36 1 −6 24 f(x) = , f 3(x) = , f (4)(x) = 1 + x (1 + x)4 (1 + x)5 The exact error is given by h4 1 ES(f) = − f (4)(c ), h = n 180 n n

for some 0 ≤ cn ≤ 1. We can bound it by h4 2h4 |ES(f)| ≤ 24 = n 180 15 The asymptotic error is given by h4 −6 −6 h4 EeS(f) = − − = − n 180 (1 + 1)4 (1 + 0)4 32

S · For n = 2, Een = −0.00195; the actual error is −0.00130.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2.2 Error formulae for Simpson’s rule

The behavior in I(f) − Sn(f) can be derived from (6.15):

4 S h 000 000 Ee (f) = − f (b) − f (a) , n 180 i.e., when n is doubled, h is halved, and h4 decreases by of factor of 16.

S Thus, the error En (f) should decrease by the same factor, provided that f 000(a) 6= f 000(b). This is the error observed with integrals I(1) and I(2).

When f 000(a) = f 000(b), the error will decrease more rapidly, which is a partial explanation of the rapid convergence for I(3).

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2.2 Error formulae for Simpson’s rule

The theory of asymptotic error formulae

En(f) = Een(f) (6.16)

T S such as for En (f) and En (f), says that (6.16) will vary with the integrand f, which is illustrated with the two cases I(1) and I(2).

From (6.14) and (6.15) we infer that Simpson’s rule will not perform as well if f(x) is not four times continuously diﬀerentiable, on [a, b].

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2.2 Error formulae for Simpson’s rule

Example Use Simpson’s rule to approximate

Z 1 √ 2 I = xdx = . 0 3

n Error Ratio 2 2.860E-2 4 1.014E-2 2.82 8 3.587E-3 2.83 16 1.268E-3 2.83 32 4.485E-4 2.83 √ Table: Simpson’s rule for x The column ”Ratio” show the convergence is much slower.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2.2 Error formulae for Simpson’s rule

As was done for the trapezoidal rule, a corrected Simpson’s rule can be deﬁned: h4 CS (f) = S (f) − f 000(b) − f 000(a) (6.17) n n 180

This will usually will be more accurate approximation than Sn(f).

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2.3 Richardson extrapolation The error estimates for Trapezoidal rule (6.12) −h2 ET (f) ≈ f 0(b) − f 0(a) n 12 and Simpson’s rule (6.15) 4 S h 000 000 Ee (f) = − f (b) − f (a) n 180 are both of the form c I − I ≈ (6.18) n np b−a where In denotes the numerical integral and h = n . The constants c and p vary with the method and the function. With most integrands f(x), p = 2 for the trapezoidal rule and p = 4 for Simpson’s rule. There are other numerical methods that satisfy (6.18), with other value of p

and c. We use (6.18) to obtain a computable estimate of the error I − In, without needing to know c explicitly.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2.3 Richardson extrapolation

Replacing n by 2n c I − I ≈ (6.19) 2n 2pnp and comparing to (6.18) c 2p(I − I ) ≈ ≈ I − I 2n np n and solving for I gives the Richardson’s extrapolation formula

p p (2 − 1)I ≈ 2 I2n − In 1 I≈ (2pI − I ) ≡ R (6.20) 2p − 1 2n n 2n

R2n is an improved estimate of I, based on using In,I2n, p, and the assumption (6.18). How much more accurate than I2n depends on the validity of (6.18), (6.19).

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2.3 Richardson extrapolation

To estimate the error in I2n, compare it with the more accurate value R2n 1 I − I ≈ R − I = (2pI − I ) − I 2n 2n 2n 2p − 1 2n n 2n 1 I − I ≈ (I − I ) (6.21) 2n 2p − 1 2n n This is Richardson’s error estimate.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2.3 Richardson extrapolation Example Using the trapezoidal rule to approximate 1 Z 2 I = e−x dx =· 0.74682413281243 0 we have · · T2 = 0.7313702518,T4 = 0.7429840978 1 p Using (6.20) I ≈ 2p−1 (2 I2n − In) with p = 2 and n = 2, we obtain 1 1 I ≈ R = (4I − I ) = (4T − T ) =· 0.7468553797 4 3 4 2 3 4 2 The error in R4 is −0.0000312; and from a previous Table, R4 is more accurate than T32. To estimate the error in T4, use (6.21) to get 1 I − T ≈ (T − T ) =· 0.00387 4 3 4 2

The actual error in T4 is 0.00384; and thus (6.21) is a very accurate error estimate.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2.4 Periodic Integrands

Deﬁnition A function f(x) is periodic with period τ if

f(x) = f(x + τ), ∀x ∈ R (6.22)

and this relation should not be true with any smaller value of τ.

For example, f(x) = ecos(πx) is periodic with periodic τ = 2.

If f(x) is periodic and differentiable, then its derivatives are also periodic with period τ.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2.4 Periodic Integrands Consider integrating Z b I = f(x)dx a with trapezoidal or Simpson’s rule, and assume that b − a is and integer multiple of the period τ. Assume f(x) ∈ C∞[a, b] (has derivatives of any order).

Then for all derivatives of f(x), the periodicity of f(x) implies that

f (k)(a) = f (k)(b), k ≥ 0 (6.23)

If we now look at the asymptotic error formulae for the trapezoidal and Simpson’s rules, they become zero because of (6.23). T S Thus, the error formulae Een (f) and Een (f) should converge to zero more rapidly when f(x) is a periodic function, provided b − a is an integer multiple of the period of f.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2.4 Periodic Integrands

T S The asymptotic error formulae Een (f) and Een (f) can be extended to higher-order terms in h, using the Euler-MacLaurin expansion and the higher-order terms are multiples of f (k)(b) − f (k)(a) for all odd T integers k ≥ 1. Using this, we can prove that the errors En (f) and S En (f) converge to zero even more rapidly than was implied by the earlier comments for f(x) periodic.

Note that the trapezoidal rule is the preferred integration rule when we are dealing with smooth periodic integrands. The earlier results for the integral I(3) illustrate this.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2.4 Periodic Integrands

Example The ellipse with boundary

x2 y 2 + = 1 a b has area πab. For the case in which the area is π (and thus ab = 1), we study the variation of the perimeter of the ellipse as a and b vary.

The ellipse has the parametric representation (x, y) = (a cos θ, b sin θ), 0 ≤ θ ≤ 2π (6.24) By using the standard formula for the perimeter, and using the symmetry of the ellipse about the x-axis, the perimeter is given by s Z π dx2 dy 2 P = 2 + dθ 0 dθ dθ Z π p = 2 a2 sin2 θ + b2 cos2 θdθ 0

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2.4 Periodic Integrands Since ab = 1, we write this as

Z π r 1 2 2 2 P (b) = 2 2 sin θ + b cos θdθ 0 b 2 Z π p = (b4 − 1) cos2 θ + 1dθ (6.25) b 0 We consider only the case with 1 ≤ b < ∞. Since the perimeters for the two ellipses

x2 y 2 x2 y 2 + = 1 and + = 1 a b b a are equal, we can always consider the case in which the y-axis of the ellipse is larger than or equal to its x-axis; and this also shows

1 P = P (b), b > 0 (6.26) b

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2.4 Periodic Integrands The integrand of P (b)

2 1 f(θ) = (b4 − 1) cos2 θ + 1 2 b is periodic with period π. As discussed above, the trapezoidal rule is the natural choice for numerical integration of (6.25). Nonetheless, there is a variation in the behaviour of f(θ) as b varies, and this will aﬀect the accuracy

of the numerical integration.16

0 0 0.5 1 1.5 2 2.5 3 3.5 Figure:The graph of integrandπ/2 f(θ): b = 2, 5, 8

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2.4 Periodic Integrands

n b = 2 b = 5 b = 8 8 8.575517 19.918814 31.690628 16 8.578405 20.044483 31.953632 32 8.578422 20.063957 32.008934 64 8.578422 20.065672 32.018564 128 8.578422 20.065716 32.019660 256 8.578422 20.065717 32.019709

Table: Trapezoidal Rule Approximation of (6.25) Note that as b increases, the trapezoidal rule converges more slowly. This is due to the integrand f(θ) changing more rapidly as b increases. 1 For large b, f(θ) changes very rapidly in the vicinity of θ = 2 π; and this causes the trapezoidal rule to be less accurate than when b is smaller, near 1. To obtain a certain accuracy in the perimeter P (b), we must increase n as b increases.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.2.4 Periodic Integrands

35 z=P(b)

5 0 1 2 3 4 5 6 7 8 9 10 Figure: The graph of perimeter function P (b) for ellipse The graph of P (b) reveals that P (b) ≈ 4b for large b. Returning to (6.25), we have for large b π 2 Z 1 P (b) ≈ b4 cos2 θ 2 dθ b 0 2 Z π ≈ b2 | cos θ|dθ = 4b b 0 We need to estimate the error in the above approximation to know when we can use it to replace P (b); but it provides a way to avoid the integration of (6.25) for the most badly behaved cases.

5. Numerical Integration Math 1070 > 5. Numerical Integration > Review and more Review

Z xj+1 Z xj+1 f(x)dx ≈ pn(x)dx xj xj | {z } ≡Ij (0) (1) (n) pn(x) interpolates at xj , xj , . . . , xj points on [xj, xj+1]

Local error:

Z xj+1 Z xj+1 f (n+1)(ξ) f(x)dx − Ij = ψ(x)dx xj xj (n + 1)! (integrand is error in interpolation) (0) (1) (n) where ψ(x) = (x − xj )(x − xj ) ··· (x − xj ).

5. Numerical Integration Math 1070 > 5. Numerical Integration > Review and more

x 106 6

(x) 0 ψ

−2

−4

−6 x_j x_{j+1} x ≤ x ≤ x j j+1

5. Numerical Integration Math 1070 > 5. Numerical Integration > Review and more

Observation: If ξ is a point on (xj, xj+1), then

g(ξ) = g(x 1 ) + O(h) j+ 2 (if g0 is continuous) i.e.,

0 g(ξ) = g(x 1 ) + (ξ − x 1 ) g (η) j+ 2 j+ 2 | {z } ≤h | {z } O(h)

5. Numerical Integration Math 1070 > 5. Numerical Integration > Review and more

Local error: 1 Z xj+1 f (n+1)(ξ) ψ(x)dx (n + 1)! xj | {z } (n+1) =f (x 1 )+O(h) j+ 2 Z xj+1 1 (n+1) = f (x 1 ) ψ(x)dx j+ 2 (n + 1)! xj | {z } Dominant Term O(hn+2)

C (n+2) n+3 (n+1)! max |f |h

z Z xj+1 }| { 1 (n+2) + f (η(x)) (ξ − xj+ 1 ) ψ(x)dx (n + 1)! 2 xj | {z } take max out | {z } integrate | {z } Higher Order Terms O(hn+3)

5. Numerical Integration Math 1070 > 5. Numerical Integration > Review and more The dominant term: case N = 1, Trapezoidal Rule

ψ(x) = (x−x)(x−x ) j j+1 0.05

−0.05

−0.1

−0.15

−0.2 x_j x_{j+1}

5. Numerical Integration Math 1070 > 5. Numerical Integration > Review and more The dominant term: case N = 2, Simpson’s Rule

Z xj+1 ψ(x) = (x − xj)(x − x 1 )(x − xj+1) ⇒ ψ(x)dx = 0 j+ 2 xj

ψ(x) = (x−x)(x−x )(x−x ) j j+1/2 j+1 0.2

0.15

0.1

0.05

−0.05

−0.1

−0.15

−0.2 x_j x_{j+1/2} x_{j+1}

5. Numerical Integration Math 1070 > 5. Numerical Integration > Review and more The dominant term: case N = 3, Simpson’s 3/8’s Rule

Local error = O(h5)

ψ(x) = (x−x)(x−x )(x−x )(x−x ) j j+1/3 j+2/3 j+1 0.1

0.08

0.06

0.04

0.02

−0.02

−0.04

−0.06 x_j x_{j+1/3} x_{j+2/3} x_{j+1}

5. Numerical Integration Math 1070 > 5. Numerical Integration > Review and more The dominant term: case N = 4

Z ψ(x)dx = 0 ⇒ local error = O(h7)

0.4

0.3

0.2

0.1

−0.1

−0.2

−0.3

−0.4 x_j x_{j+1}

5. Numerical Integration Math 1070 > 5. Numerical Integration > Review and more Simpson’s Rule

Is exact on P2 (and P3 actually)

← x j+1/2

x = (x+ x )/2 j+1/2 j j+1 Seek:

Z xj+1 f(x)dx ≈ wjf(xj) + wj+1/2f(xj+1/2) + wj+1f(xj+1) xj

5. Numerical Integration Math 1070 > 5. Numerical Integration > Review and more Exact on 1, x, x2:

Z xj+1 1 : 1dx = xj+1 − xj = wj · 1 + wj+1/2 · 1 + wj+1 · 1, xj Z xj+1 2 2 xj+1 xj x : xdx = − = wjxj + wj+1/2xj+1/2 + wj+1xj+1, xj 2 2 Z xj+1 3 3 2 2 xj+1 xj 2 2 2 x : x dx = − = wjxj + wj+1/2xj+1/2 + wj+1xj+1. xj 3 3 3 × 3 linear system: 1 w = (x − x ); j 6 j+1 j 4 w = (x − x ); j+1/2 6 j+1 j 1 w = (x − x ). j+1 6 j+1 j

5. Numerical Integration Math 1070 > 5. Numerical Integration > Review and more

Theorem

h = max(xj+1 − xj), I(f) = Simpson’s rule approximation:

Z b b − a 4 (4) f(x)dx − I(f) ≤ h max f (x) . a 2880 a≤x≤b

Trapezoid rule versus Simpson’s rule

Cost in TR 2 function evaluations/integral × no. intervals 2 = = ; Cost in SR 3 function evaluations/integral × no. intervals 3

1 (reducible to 2 if storing the previous values) Accuracy in TR h2 b−a 240 = 12 = . Accuracy in SR 4 b−a h2 h 2880

1 6 6 E.g. for h = 100 ⇒ 2.4 × 10 , i.e. SR is more accurate than TR by factor of 2.4 × 10 .

5. Numerical Integration Math 1070 > 5. Numerical Integration > Review and more What if there is round-oﬀ error?

Suppose we use the method with

f(xj)computed = f(xj)true ± εj, εj = O(machine precision) | {z } =ε

Z b X f(xj+1)computed + f(xj)computed f(x)dx ≈ (x − x ) j+1 j 2 a j

X f(xj+1) ± εj+1 + f(xj) ± εj = (x − x ) j+1 j 2 j

X f(xj+1) + f(xj) X ±εj+1 ± εj = (x − x ) + (x − x ) j+1 j 2 j+1 j 2 j j | {z } | {z } value in exact arithmetic contribution of round-oﬀ error≤ε(b−a)

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3 Gaussian Numerical Integration

The numerical methods studied in the ﬁrst 2 sections were based on integrating 1 linear (trapezoidal rule) and 2 quadratic (Simpson’s rule) and the resulting formulae were applied on subdivisions of ever smaller subintervals.

We consider now a numerical method based on exact integration of polynomials of increasing degree; no subdivision of the integration interval [a, b] is used. Recall the Section 4.4 of Chapter 4 on approximation of functions.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3 Gaussian Numerical Integration

Let f(x) ∈ C[a, b]. Then ρn(f) denotes the smallest error bound that can be attained in approximating f(x) with a polynomial p(x) of degree ≤ n on the given interval a ≤ x ≤ b. The polynomial mn(x) that yields this approximation is called the minimax approximation of degree n for f(x)

max |f(x) − mn(x)| = ρn(f) (6.27) a≤x≤b

and ρn(f) is called the minimax error.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3 Gaussian Numerical Integration

Example Let f(x) = e−x2 for x ∈ [0, 1]

n ρn(f) n ρn(f) 1 5.30E-2 6 7.82E-6 2 1.79E-2 7 4.62E-7 3 6.63E-4 8 9.64E-8 4 4.63E-4 9 8.05E-9 5 1.62E-5 10 9.16E-10

Table: Minimax errors for e−x2 , 0 ≤ x ≤ 1 The minimax errors ρn(f) converge to zero rapidly, although not at a uniform rate.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3 Gaussian Numerical Integration

If we have a

numerical integration formula to integrate low- to moderate-degree polynomials exactly,

then the hope is that

the same formula will integrate other functions f(x) almost exactly,

if f(x) is well ”approximable” by such polynomials.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3 Gaussian Numerical Integration To illustrate the derivation of such integration formulae, we restrict ourselves to the integral Z 1 I(f) = f(x)dx. −1

The integration formula is to have the general form: (Gaussian numerical integration method)

n X In(f) = wjf(xj) (6.28) j=1 and we require that

the nodes {x1, . . . , xn} and weights {w1, . . . , wn} be so chosen that In(f) = I(f) for all polynomials f(x) of as large degree as possible.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3 Gaussian Numerical Integration Case n = 1 The integration formula has the form Z 1 f(x)dx ≈ w1f(x1) (6.29) −1

Using f(x) ≡ 1, and forcing (6.29) 2 = w1

Using f(x) = x 0 = w1x1 which implies x1 = 0. Hence (6.29) becomes Z 1 f(x)dx ≈ 2f(0) ≡ I1(f) (6.30) −1 This is the midpoint formula, and is exact for all linear polynomials. To see that (6.30) is not exact for quadratics, let f(x) = x2. Then the error in (6.30) is Z 1 2 x2dx − 2(0)2 = 6= 0, −1 3 hence (6.30) has degree of precision 1.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3 Gaussian Numerical Integration Case n = 2

The integration formula is

Z 1 f(x)dx ≈ w1f(x1) + w2f(x2) (6.31) −1

and it has four unspeciﬁed quantities: x1, x2, w1, w2. To determine these, we require (6.31) to be exact for the four monomials

f(x) = 1, x, x2, x3.

obtaining 4 equations

2 = w1 + w2 0 = w1x1 + w2x2 2 2 2 3 = w1x1 + w2x2 3 3 0 = w1x1 + w2x2

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3 Gaussian Numerical Integration Case n = 2

This is a nonlinear system with a solution √ √ 3 3 w = w = 1, x = − , x = (6.32) 1 2 1 3 2 3

and another one based on reversing the signs of x1 and x2. This yields the integration formula √ ! √ ! Z 1 3 3 f(x)dx ≈ f − + f ≈ I2(f) (6.33) −1 3 3

which has degree of precision 3 (exact on all polynomials of degree 3 and not exact for f(x) = x4).

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3 Gaussian Numerical Integration

Example Approximate

Z 1 I = exdx = e − e−1 =· 2.5304024 −1

Using √ ! √ ! Z 1 3 3 f(x)dx ≈ f − + f ≈ I2(f) −1 3 3 we get

√ √ − 3 − 3 · I2 = e 3 + e 3 = 2.2326961 · I − I2 = 0.00771

The error is quite small, considering we are using only 2 node points.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3 Gaussian Numerical Integration Case n > 2 We seek the formula (6.28) n X In(f) = wjf(xj) j=1 which has 2n points unspeciﬁed parameters x1, . . . , x,w1, . . . , wn, by forcing the integration formula to be exact for 2n monomials f(x) = 1, x, x2, . . . , x2n−1

In turn, this forces In(f) = I(f) for all polynomials f of degree ≤ 2n − 1. This leads to the following system of 2n nonlinear equations in 2n unknowns: 2 = w1 + w2 + ... + wn 0 = w1x1 + w2x2 + ... + wnxn 2 2 2 2 3 = w1x1 + w2x2 + ... + wnxn . (6.34) . 2 2n−2 2 2n−2 2n−1 = w1x1 + w2x2 + ... + wnxn 2n−1 2 2n−1 0 = w1x1 + w2x2 + ... + wnxn

The resulting formula In(f) has degree of precision 2n − 1.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3 Gaussian Numerical Integration

Solving this system is a formidable problem. The nodes {xi} and weights {wi} have been calculated and collected in tables for most commonly used values of n.

n xi wi 2 ± 0.5773502692 1.0 3 ± 0.7745966692 0.5555555556 0.0 0.8888888889 4 ± 0.8611363116 0.3478548451 ± 0.3399810436 0.6521451549 5 ± 0.9061798459 0.2369268851 ± 0.5384693101 0.4786286705 0.0 0.5688888889 6 ± 0.9324695142 0.1713244924 ± 0.6612093865 0.3607651730 ± 0.2386191861 0.4679139346 7 ± 0.9491079123 0.1294849662 ± 0.7415311856 0.2797053915 ± 0.4058451514 0.3818300505 0.0 0.4179591837 8 ± 0.9602898565 0.1012285363 ± 0.7966664774 0.2223810345 ± 0.5255324099 0.3137066459 ± 0.1834346425 0.3626837834

Table: Nodes and weights for Gaussian quadrature formulae

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3 Gaussian Numerical Integration

There is also another approach to the development of the numerical integration formula (6.28), using the theory of orthogonal polynomials. From that theory, it can be shown that the nodes {x1, . . . , xn} are the zeros of the Legendre polynomials of degree n on the interval [−1, 1]. Recall that these polynomials were introduced in Section 4.7. For example, 1 P (x) = (3x2 − 1) 2 2

and its roots are the nodes given√ in (6.32) √ 3 3 x1 = − 3 , x2 = 3 . Since the Legendre polynomials are well known, the nodes {xj} can be found without any recourse to the nonlinear system (6.34).

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3 Gaussian Numerical Integration The sequence of formulae (6.28) is called Gaussian numerical integration method. From its deﬁnition, In(f) uses n nodes, and it is exact for all R 1 polynomials of degree ≤ 2n − 1. In(f) is limited to −1 f(x)dx, an integral over [−1, 1]. But this limitation is easily removed. Given an integral Z b I(f) = f(x)dx (6.35) a introduce the linear change of variable b + a + t(b − a) x = , −1 ≤ t ≤ 1 (6.36) 2 transforming the integral to b − a Z 1 I(f) = fe(t)dt (6.37) 2 −1 with b + a + t(b − a) fe(t) = f 2

Now apply Ien to this new integral.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3 Gaussian Numerical Integration

Example Apply Gaussian numerical integration to the three integrals (1) R 1 −x2 (2) R 4 dx (3) R 2π dx I = 0 e dx, I = 0 1+x2 , I = 0 2+cos(x) , which were used as examples for the trapezoidal and Simpson’s rules.

All are reformulated as integrals over [−1, 1]. The error results are

n Error in I(1) Error in I(2) Error in I(3) 2 2.29E-4 -2.33E-2 8.23E-1 3 9.55E-6 -3.49E-2 -4.30E-1 4 -3.35E-7 1.90E-3 1.77E-1 5 6.05E-9 1.70E-3 -8.12E-2 6 -7.77E-11 2.74E-4 3.55E-2 7 7.89E-13 -6.45E-5 -1.58E-2 10 * 1.27E-6 1.37E-3 15 * 7.40E-10 -2.33E-5 20 * * 3.96E-7

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3 Gaussian Numerical Integration

If these results are compared to those of trapezoidal and Simpson’s rule, then Gaussian integration of I(1) and I(2) is much more eﬃcient than the trapezoidal rules. But then integration of the periodic integrand I(3) is not as eﬃcient as with the trapezoidal rule. These results are also true for most other integrals.

Except for periodic integrands, Gaussian numerical integration is usually much more accurate than trapezoidal and Simpson rules.

This is even true with many integrals in which the integrand does not have a continuous derivative.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3 Gaussian Numerical Integration Example Use Gaussian integration on Z 1 √ 2 I = xdx = 0 3 The results are n I − In Ratio 2 -7.22E-3 4 -1.16E-3 6.2 8 -1.69E-4 6.9 16 -2.30E-5 7.4 32 -3.00E-6 7.6 64 -3.84E-7 7.8 where n is the number of node points. The ratio column is deﬁned as I − I 1 2 n I − In and it shows that the error behaves like c I − I ≈ (6.38) n n3 for some c. The error using Simpson’s rule has an empirical rate of 1 convergence proportional to only n1.5 , a much slower rate than (6.38).

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3 Gaussian Numerical Integration

A result that relates the minimax error to the Gaussian numerical integration error. Theorem Let f ∈ C[a, b], n ≥ 1. Then, if we aply Gaussian numerical integration to R b I = a f(x)dx, the error In satisﬁes

|I(f) − In(f)| ≤ 2(b − a)ρ2n−1(f) (6.39)

where ρ2n−1(f) is the minimax error of degree 2n − 1 for f(x) on [a, b].

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3 Gaussian Numerical Integration

Example Using the table n ρn(f) n ρn(f) 1 5.30E-2 6 7.82E-6 2 1.79E-2 7 4.62E-7 3 6.63E-4 8 9.64E-8 4 4.63E-4 9 8.05E-9 5 1.62E-5 10 9.16E-10 apply (6.39) to 1 Z 2 I = e−x dx 0

For n = 3, the above bound implies

−x2 · −5 |I − I3| ≤ ρ5(e ) = 3.24 × 10 .

The actual error is 9.95E − 6.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3 Gaussian Numerical Integration

Gaussian numerical integration is not as simple to use as are the trapezoidal and Simpson rules, partly because the Gaussian nodes and weights do not have simple formulae and also because the error is harder to predict. Nonetheless, the increase in the speed of convergence is so rapid and dramatic in most instances that the method should always be considered seriously when one is doing many integrations. Estimating the error is quite diﬃcult, and most people satisfy themselves by looking at two or more succesive values. If n is doubled, then repeatedly comparing two successive values, In and I2n, is almost always adequate for estimating the error in In

I − In ≈ I2n − In

This is somewhat ineﬃcient, but the speed of convergence in In is so rapid that this will still not diminish its advantage over most other methods.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3.1 Weighted Gaussian Quadrature

A common problem is the evaluation of integrals of the form

Z b I(f) = w(x)f(x)dx (6.40) a with f(x) a “well-behaved” function and w(x) a possibly (and often) ill-behaved function. Gaussian quadrature has been generalized to handle such integrals for many functions w(x). Examples include

Z 1 f(x) Z 1 √ Z 1 1 √ , xf(x)dx, f(x) ln( )dx. 2 −1 1 − x 0 0 x The function w(x) is called a weight function.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3.1 Weighted Gaussian Quadrature

We begin by imitating the development given earlier in this section, and we do so for the special case of

Z 1 f(x) I(f) = √ dx 0 x

√1 in which w(x) = x . As before, we seek numerical integration formulae of the form

n X In(f) = wjf(xj) (6.41) j=1

and we require that the nodes {x1, . . . , xn} and the weights {w1, . . . , wn} be so chosen that In(f) = I(f) for polynomials f(x) of as large as possible.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3.1 Weighted Gaussian Quadrature Case n = 1 The integration formula has the form Z 1 f(x) √ dx ≈ w1f(x1) 0 x We force equality for f(x) = 1 and f(x) = x. This leads to equations

Z 1 1 w1 = √ dx = 2 0 x Z 1 x 2 w1x1 = √ dx = 0 x 3

Solving for w1 and x1, we obtain the formula Z 1 f(x) 1 √ dx ≈ 2f( ) (6.42) 0 x 3 and it has the degree of precision 1.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3.1 Weighted Gaussian Quadrature Case n = 2 The integration formula has the form Z 1 f(x) √ dx ≈ w1f(x1) + w2f(x2) (6.43) 0 x We force equality for f(x) = 1, x, x2, x3. This leads to equations Z 1 1 w1 + w2 = √ dx = 2 0 x Z 1 x 2 w1x1 + w2x2 = √ dx = 0 x 3 Z 1 2 2 2 x 2 w1x1 + w2x2 = √ dx = 0 x 5 Z 1 3 3 3 x 2 w1x1 + w2x2 = √ dx = 0 x 7 This has the solution √ √ x = 3 − 2 30 =· 0.11559, x = 3 + 2 30 =· 0.74156 1 7 35 √ 2 7 35 √ 1 · 1 · w1 = 1 + 18 30 = 1.30429, w2 = 1 − 18 30 = 0.69571 The resulting formula (6.43) has degree of precision 3.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3.1 Weighted Gaussian Quadrature Case n > 2

We seek formula (6.41), which has 2n unspeciﬁed parameters, x1, . . . , xn, w1, . . . , wn, by forcing the integration formula to be exact for the 2n monomials f(x) = 1, x, x2, . . . , x2n−1.

In turn, this forces In(f) = I(f) for all polynomials f of degree ≤ 2n − 1. This leads to the following system of 2n nonlinear equations in 2n unknowns:

w1 + w2 + ... + wn = 2 2 w1x1 + w2x2 + ... + wnxn = 3 2 2 2 2 w1x + w2x + ... + wnx = (6.44) 1. 2 n 5 . 2n−1 2n−1 2n−1 2 w1x1 + w2x2 + ... + wnxn = 4n−1

The resulting formula In(f) has degree of precision 2n − 1. As before, this system is very diﬃcult to solve directly, but there are

alternative methods of deriving {xi} and {wi}. It is based on looking at the polynomials that are orthogonal with respect to the weight function √1 w(x) = x on the interval [0, 1].

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3.1 Weighted Gaussian Quadrature

Example We evaluate Z 1 cos(πx) I = √ dx =· 0.74796566683146 0 x

using (6.42) Z 1 f(x) 1 √ dx ≈ 2f( ) ≡ 1.0 0 x 2 and (6.43)

1 Z f(x) · √ dx ≈ w1f(x1) + w2f(x2) = 0.740519 0 x

· I2 is a reasonable estimate of I, with I − I2 = 0.00745.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.3.1 Weighted Gaussian Quadrature A general theory can be developed for the weighted Gaussian quadrature n Z b X I(f) = w(x)f(x)dx ≈ wjf(xj) = In(f) (6.45) a j=1

It requires the following assumptions for the weight function w(x):

1 w(x) > 0 for a < x < b;

2 For all integers n ≥ n, Z b w(x)|x|ndx < ∞ a These hypotheses are the same as were assumed for the generalized least squares approximation theory following Section 4.7 of Chapter 4. This is not accidental since both Gaussian quadrature and least squares approximation theory are dependent on the subject of orthogonal polynomials. The node points {xj} solving the system (6.44) are the zeros of the degree n orthogonal √1 polynomial on [a, b] with respect to the weight function w(x) = x . For the generalization (6.45), the nodes {xi} are the zeros of the degree n orthogonal polynomial on [a, b] with respect to the weight function w(x).

5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement Gauss’s idea:

The optimal abscissas of the κ−point Gaussian quadrature formulas are precisely the roots of the orthogonal polynomial for the same interval and weighting function. Z b X Z xj+1 f(x)dx = f(x)dx a j xj | {z } composite formula Z 1 X xj+1 − xj xj+1 + xj xj+1 − xj = f t + dt 2 2 2 j −1 | {z } κ R 1 X = −1 g(t)dt ≈ w`g(q`) `=1 | {z } κ point Gauss Rule for max accuracy

w1, . . . , wκ: weights, q1, . . . , qκ: quadrature points on (−1, 1). 2 2κ−1 Exact on polynomials p(x) ∈ P2κ−1, i.e., 1, t, t , . . . , t . 5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement

2 3 4 5 Example: 3 point Gauss, exact on P5 ⇔ exact on 1, t, t , t , t , t

Z +1 g(t)dt ≈ w1g(q1) + w2g(q2) + w3g(q3) −1 R 1 −1 1dt = 2 = w1 · 1 + w2 + w3 R 1 −1 t dt = 0 = w1q1 + w2q2 + w3q3 R 1 2 2 2 2 2 −1 t dt = 3 = w1q1 + w2q2 + w3q3 R 1 3 3 3 3 −1 t dt = 0 = w1q1 + w2q2 + w3q3 R 1 4 2 4 4 4 −1 t dt = 5 = w1q1 + w2q2 + w3q3 R 1 5 5 5 5 −1 t dt = 0 = w1q1 + w2q2 + w3q3

Guess: q1 = −q3, q2 = 0(q1 ≤ q2 ≤ q3), w1 = w3.

5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement

2 3 4 5 Example: 3 point Gauss, exact on P5 ⇔ exact on 1, t, t , t , t , t

With this guess:  2w1 + w2 = 2  2 2w1q1 = 2/3 ,  4 2w1q1 = 2/5 hence

r3 r3 q = − , q = 1 5 3 5 5 5 8 w = , w = , w = 1 9 3 9 2 9 A. H. Stroud and D. Secrest: ”Gaussian Quadrature Formulas”. Englewood Cliﬀs, NJ: Prentice-Hall, 1966.

5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement

1 The idea of Gauss Gauss-Lobatto

Z 1 g(t)dt = w1g(−1) + w2g(q2) + ··· + wk−1g(k−1) + wkg(1) −1

k − 2 nodes locates as k-point formula; is accurate P2k−3. (Order decreased by 2 beside the Gauss quadrature formula).

5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement Adaptive Quadrature

Problem R b Given a f(x)dx and ε−preassigned tolerance compute Z b I(f) ≈ f(x)dx a with (a) to assured accuracy

Z b

f(x)dx − I(f) < ε a (b) at minimal / near minimal cost (no. function evaluations)

Strategy: LOCALIZE!

5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement Localization Theorem P R xj+1 Let I(f) = Ij(f) where Ij(f) ≈ f(x)dx. j xj

Z xj+1 ε(x − x ) If f(x)dx − I (f) < j+1 j (= local tolerance), j b − a xj Z b

then f(x)dx − I(f) < ε(= tolerance) a

Z b Z xj+1 X X Proof: f(x)dx − I(f) = | f(x)dx − I(f)|

a j xj j ! X Z xj+1 X Z xj+1 = | f(x)dx − Ij(f) | ≤ | f(x)dx − Ij(f)| j xj j xj

X ε(xj+1 − xj) ε X ε = = (x − x ) = (b − a) = ε. b − a b − a j+1 j b − a j j 5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement Need:

Estimator for local error and Strategy when to cut h to ensure accuracy? when to increase h to ensure minimal cost? One approach: halving and doubling! Recall: Trapezoidal rule

f(xj )+f(xj+1) Ij ≈ (xj+1 − xj) 2 . A priori estimate:

Z xj+1 3 (xj+1 − xj) 00 f(x)dx − Ij = f (sj) xj 12

for some sj in (xj, xj+1).

5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement

Step 1: compute Ij

f(xj+1)+f(xj ) Ij = 2 (xj+1 − xj)

Step 2: cut interval in half + reuse trapezoidal rule

f(xj) + f(x 1 ) f(xj+1) + f(x 1 ) j+ 2 j+ 2 Ij = (xj+ 1 −xj) + (xj+1 −xj+ 1 ) 2 2 2 2 Error estimate: Z xj+1 h3 j 00 st f(x)dx − Ij = f (ξj) = ej 1 use of trapezoid rule xj 12 Z xj+1 3 3 (hj/2) 00 (hj/2) 00 nd f(x)dx − Ij = f (η1) + f (η2) 2 use of TR xj 12 12 1 h3 = j f 00(ξ ) +O(h4) 4 12 j j | {z } ej 5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement

ej = 4ej + Higher Order Terms Substracting

4 Ij − Ij Ij − Ij = 3ej + O(h ) =⇒ ej = + Higher Order Terms 3 | {z } O(h4)

5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement

4 points Gauss: exact on P7 local error: O(h9) global error: O(h8) A priori estimate: Z xj+1 9 (8) f(x)dx − Ij = C(xj+1 − xj) f (ξj) xj Z xj+1 9 (8) f(x)dx − Ij = Chj f (ξj) xj Z xj+1 9 9 hj (8) 0 hj (8) 00 f(x)dx − Ij = C f (ξj) + C f (ξj ) xj 2 2 | {z } = C h9f (8)(ξ )+O(h10) 28 j j 10 ⇒ Ij − Ij = 225ej + O(h )

Ij − Ij ⇒ ej = + Higher Order Terms 225 | {z } O(h10) 5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement Algorithm

Input: a, b, f(x) upper error tolerance: εmax initial mesh width: h Initialize: Integral = 0.0 xL = a 1 εmin = 2k+3 εmax * xR = xL + h ( If xR > b, xR ← b do integral 1 more time and stop

Compute on xL, xR: I, I and EST

I − I EST = (if exact on Pk) 2k+1 − 1

5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement ’error is just right’: h h If εmin b−a < EST < εmax b−a Integral ← Integral + I xL ← xR go to * ’error is too small’: εmin If EST ≤ b−a h Integral ← Integral + I xL ← xR h ← 2h go to * ’error is too big’: εmax If EST ≥ b−a h h ← h/2.0 go to * STOP END 5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement

Trapezium rule

Z xj+1 f(xj+1) + f(xj) f(x)dx − (xj+1 − xj) xj 2 Z xj+1 Z xj+1 f 00(ξ) f(x) − p (x)dx = (x − x )(x − x ) dx 1 2 j j+1 xj xj | {z } ψ(x) f 00(x) Z xj+1 Z xj+1 = ψ(x)dx +O(h) ψ(x)dx 2 xj xj | {z } integrate exactly

5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement The mysteries of ψ(x)

x 105 1.5

0.5

(x) 0 ψ

−0.5

−1

−1.5 x_j q_1 q_2 q_3 q_4 q_5 q_6 q_7 x ≤ x ≤ x j j+1

ψ(x) = (x − q1)(x − q2) ··· (x − q7)

5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement Error in k + 1 point quadrature

f k+1(ξ) pk(x) interpolates f(x) =⇒ f(x) − pk(x) = (k+1)! ψ(x)

(xj ≤) q1 < q2 < . . . < qk+1 (≤ xj+1)

Z xj+1 Z xj+1 Z xj+1 ψ(x) (k+1) f(x)dx − pk(x)dx = f (ξ)dx xj xj xj (k + 1)! | {z } | {z } true approx

5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement 1. A simple error bound

Ignoring oscillation of ψ(x):

k+1 Z xj+1 k+1 max |f | max |f | k+2 |error| ≤ |ψ(x)|dx ≤ |xj+1−xj| (k + 1)! xj (k + 1)!

x | {z } x R j+1 k+1 R j+1 = xj |x−q1|···|x−qk+1|≤h xj dx x 105 1.5

0.5 (x)) ψ

0 (x) and abs( ψ

−0.5

−1

−1.5 x_j q_1 q_2 q_3 q_4 q_5 q_6 q_7 x ≤ x ≤ x j j+1

5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement 2. Analysis without cancelation

xj < ξ < x < xj+1 Lemma

Let ξ, x ∈ (xj, xj+1). Then

f (k+1)(ξ) = f (k+1)(x) + (ξ − x)f k+2(η) MVT

for some η between ξ and x, and |ξ − x| ≤ xj+1 − xj ≤ h.

5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement 2. Analysis without cancelation |error| ≤ |true − approx| 1 Z xj+1 h i = | ψ(x) f (k+1)(x) +(ξ − x)f (k+2)(η) dx| (k + 1)! xj | {z } | {z } fixed O(h) f (k+1)(x) Z xj+1 ≤ | ψ(x)dx | (k + 1)! xj | {z } xj+1 if R =0 xj ψ(x)dx=0 1 Z xj+1 + | f (k+2)(η)(ξ − x)ψ(x)| (k + 1)! xj max |f (k+2)| Z xj+1 ≤ |ξ − x| |ψ(x)| dx (k + 1)! xj | {z } | {z } ≤h ≤hk+1 hk+3 max |f (k+2)(x)| ≤ The error for Simpson’s rule, i.e. cancelation. (k + 1)!

5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement

ψ(x) interpolates zero at k + 1 points (deg ψ(x) = k + 1) Lemma

The general result of p(q`) = 0, ` = 1, . . . , k + 1, p ∈ Pk+1 is p(x) = Constant · ψ(x).

5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement Questions:

1) How to pick the points q1, . . . , qk+1 so that

Z 1 g(x)dx ≈ w1g(q1) + ... + wk+1g(qk+1) (6.46) −1

integrates Pk+m exactly? 2) What does this imply about the error?

Remark R 1 If m < 1, pick q1, . . . , qk+1 so that −1 ψ(x)dx = 0 and then the error converges O(hk+3).

m=2

5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement Step 1

Let r1 be some ﬁxed point on [−1, 1]: −1 < q1 < q2 < . . . < r1 < . . . < qk < qk+1

g(r1) − pk(r1) pk+1(x) = pk(x) + ψ(x) (6.47) ψ(r1)

pk interpolates g(x) at q1, . . . , qk+1. Claim: pk+1 interpolates g(x) at k + 2 points q1, . . . , qk+1, r1. Suppose now that (6.46) is exact on Pk+1, then from (6.47) Z 1 Z 1 g(x)dx − pk+1(x) dx error in k + 2 quadrature rule, Ek+2 −1 −1 | {z } | {z } substitute (6.47) true Z 1 Z 1 Z 1 f(r1) − pk(r1) = g(x)dx − pk(x)dx − ψ(x) dx −1 −1 −1 ψ(r1) | {z } error in k + 1 quadrature rule ≡ Ek+1

5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement Step 1

Z 1 f(r1) − pk(r1) So Ek+2 = Ek+1 − ψ(x)dx ψ(r1) −1 Conclusion 1 R 1 If −1 ψ(x)dx = 0, then error in k + 1 point rule is exactly the same as if we had used k + 2 points.

5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement Step 2

Let r1, r2 be ﬁxed points in [−1, 1]. So interpolate at k + 3 points: q1, . . . , qk+1, r1, r2

g(r2) − pk(r2) pk+2(x) = pk(x) + ψ(x)(x − r1) (6.48) (r2 − r1)ψ(r2) g(r1) − pk(r1) + ψ(x)(x − r2) (r1 − r2)ψ(r1) Consider error in a rule with k + 1 + 2 points: Z 1 Z 1 error in k + 3 p. r. = g(x)dx − pk+2(x)dx −1 −1 Z 1 Z 1 Z 1 g(r2) − pk(r2) = g(x)dx − pk(x)dx − ψ(x)(x − r1)dx −1 −1 (r2 − r1)ψ(r2) −1 Z 1 g(r1) − pk(r1) − ψ(x)(x − r2)dx. (r1 − r2)ψ(r1) −1

5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement Step 2

R 1 R 1 So Ek+3 = Ek+1 + Const −1 ψ(x)(x − r1)dx + Const −1 ψ(x)(x − r2)dx Conclusion 2 R 1 R 1 If −1 ψ(x)dx = 0 and −1 xψ(x)dx = 0, then error in k + 1 point rule has the same error as k + 3 point rule.

5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement

...

Z 1 Z 1 1 Ek+1+m = Ek+1 + C0 ψ(x)dx + C1 ψ(x)x dx + ... (6.49) −1 −1 Z 1 m−1 + Cm ψ(x)x dx (6.50) −1 So Conclusion 3 R 1 j If −1 ψ(x)x dx = 0, j = 0, . . . , m − 1, then error is as good as using m extra points.

5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement Overview

Interpolating Quadrature Interpolate f(x) at q0, q1, q2, . . . , qk ⇒ pk(x)

f (k+1)(ξ) f(x) − p (x) = (x − q )(x − q ) ... (x − q ) k (k + 1)! 0 1 k Z 1 Z 1 Z 1 1 (k+1) f(x)dx − pk(x)dx = f (ξ)ψ(x)dx −1 −1 (k + 1)! −1 Gauss rules

pick q` to maximize exactness what is the accuracy

what are the q`’s?

5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement Overview

Interpolate at k + 1 + m points Interpolate at k + 1 points q0, . . . , qk, r1 . . . , rm q0, . . . , qk error error ⇓ ⇓ R 1 Ek+m = Ek + c0 −1 ψ(x) · 1dx R 1 +c1 −1 ψ(x) · xdx + ... R 1 m−1 +cm −1 ψ(x) · x dx Deﬁnition p(x) is the µ + 1st orthogonal polynomial on [−1, 1] (weight R 1 ` w(x) ≡ 1) if p(x) ∈ Pµ+1 and −1 p(x)x dx = 0, ` = 0, . . . , µ, i.e., R 1 −1 p(x)q(x)dx = 0∀q ∈ Pµ.

5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement Overview

Pick q0, q1, . . . , qk so R 1 −1 ψ(x) · 1dx = 0 R 1 deg m−1 ψ(x) · xdx = 0 Z 1 z}|{ −1 ⇔ ψ(x) q(x) dx = 0, ∀q ∈ P . m−1 . −1 |{z} R 1 m−1 deg k+1 −1 ψ(x) · x dx = 0 So, maximum accuracy if ψ(x) is the orthogonal polynomial of degree k + 1: Z 1 ψ(x)q(x)dx = 0 ∀q ∈ Pk ⇒ m − 1 = k, m = k + 1 −1 So, the Gauss quadrature points are the roots of the orthogonal polynomial.

5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement Overview

Adaptivity

I = I1 + I2 Trapezium rule’s local error = O(h3)

Z xj+1 f(x)dx − I = 4e ⇐ e ≈ 8e1, e ≈ 8e2 so, e ≈ 4e xj Z xj+1 f(x)dx − I = e = e1 + e2 xj I − I I − I = 3e (+ Higher Order Terms) ⇒ e = (+ Higher Order Terms) 3

5. Numerical Integration Math 1070 > 5. Numerical Integration > Supplement Overview

Final Observation

T rue − Approx ≈ 4e ) 2 equations, 2 unknowns: e, T rue T rue − Approx ≈ e So, can solve for e + T rue Solving for T rue:

I − I 4 1 T rue ≈ I + e ≈ I + ≈ I − I (+ Higher Order Terms) 3 3 3

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.4 Numerical Diﬀerentiation

To numerically calculate the derivative of f(x), begin by recalling the deﬁnition of derivative f(x + h) − f(x) f 0(x) = lim x→0 h This justiﬁes using

0 f(x + h) − f(x) f (x) ≈ ≡ Dhf(x) (6.51) h

for small values of h. Dhf(x) is called a numerical derivative of f(x) with stepsize h.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.4 Numerical Diﬀerentiation

Example π Use Dhf to approximate the derivative of f(x) = cos(x) at x = 6 .

h Dn(f) Error Ratio 0.1 -0.54243 0.04243 0.05 -0.52144 0.02144 1.98 0.025 -0.51077 0.01077 1.99 0.0125 -0.50540 0.00540 1.99 0.00625 -0.50270 0.00270 2.00 0.003125 -0.50135 0.00135 2.00

Looking at the error column, we see the error is nearly proportional to h; when h is halved, the error is almost halved.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.4 Numerical Diﬀerentiation

To explain the behaviour in this example, Taylor’s theorem can be used to ﬁnd an error formula. Expanding f(x + h) about x, we get

h2 f(x + h) = f(x) + hf 0(x) + f 00(c) 2 for some c between x and x + h. Substituting on the right side of (6.51), we obtain

1 h2 D f(x) = f(x) + hf 0(x) + f 00(c) − f(x) h h 2 h = f 0(x) + f 00(c) 2 h f 0(x) − D f(x)= − f 00(c) (6.52) h 2 The error is proportional to h, agreeing with the results in the Table above.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.4 Numerical Differentiation For that example, π π h f 0 − D f = cos(c) (6.53) 6 h 6 2 1 1 where c is between 6 π and 6 π + h. 1 Let’s check that if c is replaced by 6 π, then the RHS of (6.53) agrees with the error column in the Table. As seen in the example, we use the formula (6.51) with a positive stepsize h > 0. The formula (6.51) is commonly known as the forward difference formula for the first derivative. We can formally replace h by −h in (6.51) to obtain the formula f(x) − f(x − h) f 0(x) ≈ , h > 0 (6.54) h This is the backward difference formula for the first derivative. A derivation similar to that leading to (6.52) shows that f(x) − f(x − h) h f 0(x) − = f 00(c)) (6.55) h 2 for some c between x and x − h. Thus, we expect the accuracy of the backward difference formula to be almost the same as that of the forward difference formula.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.4.1 Diﬀerentiation Using Interpolation

Let Pn(x) denote the degree n polynomial that interpolates f(x) at n + 1 node points x0, . . . , xn.

To calculate f 0(x) at some point x = t, use

0 0 f (t) ≈ Pn(t) (6.56) Many diﬀerent formulae can be obtained by 1 varying n and by

2 varying the placement of the nodes x0, . . . , xn relative to the point t of interest.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.4.1 Diﬀerentiation Using Interpolation As an especially useful example of (6.56), take

n = 2, t = x1, x0 = x1 − h, x2 = x1 + h.

Then (x − x )(x − x ) (x − x )(x − x ) P (x) = 1 2 f(x ) + 0 2 f(x ) 2 2h2 0 −h2 1 (x − x )(x − x ) + 0 1 f(x ) 2h2 2 2x − x − x 2x − x − x P 0(x) = 1 2 f(x ) + 0 2 f(x ) 2 2h2 0 −h2 1 2x − x − x + 0 1 f(x ) 2h2 2 x − x 2x − x − x x − x P 0(x ) = 1 2 f(x ) + 1 0 2 f(x ) + 1 0 f(x ) 2 1 2h2 0 −h2 1 2h2 2 f(x ) − f(x ) = 2 0 (6.57) 2h

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.4.1 Diﬀerentiation Using Interpolation The central diﬀerence formula

Replacing x0 and x2 by x1 − h and x1 + h, from (6.56) and (6.57) we obtain central diﬀerence formula

0 f(x1 + h) − f(x1 − h) f (x1) ≈ ≡ Dhf(x1), (6.58) 2h another approximation to the derivative of f(x).

It will be shown below that this is a more accurate approximation to 0 f (x) than is the the forward diﬀerence formula Dhf(x) of (6.51), i.e., f(x + h) − f(x) f 0(x) = ≡ D f(x). h h

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.4.1 Diﬀerentiation Using Interpolation

Theorem Assume f ∈ Cn+2[a, b]. Let x0, x1, . . . , xn be n + 1 distinct interpolation nodes in [a, b], and let t be an arbitrary given point in [a, b]. Then

(n+2) (n+1) 0 0 f (c1) 0 f (c2) f (t) − P (t) = Ψn(t) + Ψ (t) (6.59) n (n + 2)! n (n + 1)!

with Ψn(t) = (t − x0)(t − x1) ··· (t − xn)

The numbers c1 and c2 are unknown points located between the maximum and minimum of x0, x1, . . . , xn and t.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.4.1 Differentiation Using Interpolation To illustrate this result, an error formula can be derived for the central difference formula (6.58). Since t = x1 in deriving (6.58), we find that the first term on the RHS of (6.59) is zero. Also n = 2 and

Ψ2(x) = (x − x0)(x − x1)(x − x2) 0 Ψ2(x) = (x − x0)(x − x1) + (x − x0)(x − x2) + (x − x0)(x − x1) 0 2 Ψ2(x1) = (x1 − x0)(x1 − x2) = −h

Using this in (6.59), we get

f(x + h) − f(x − h) h2 f 0(x ) − 1 1 = − f 000(c ) (6.60) 1 2h 6 2

with x1 − h ≤ c2 ≤ x1 + h. This says that for small values of h, the central diﬀerence formula (6.58) should be more accurate that the earlier approximation (6.51), the forward diﬀerence formula, because the error term of (6.58) decreases more rapidly with h.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.4.1 Diﬀerentiation Using Interpolation

Example The earlier example f(x) = cos(x) is repeated using the central π diﬀerence fomula (6.58) (recall x1 = 6 ).

h Dn(f) Error Ratio 0.1 -0.49916708 -0.0008329 0.05 -0.49979169 -0.0002083 4.00 0.025 -0.49994792 -0.00005208 4.00 0.0125 -0.49998698 -0.00001302 4.00 0.00625 -0.49999674 -0.000003255 4.00

The results confirm the rate of convergence given in (6.58), and they illustrate that the central difference formula (6.58) will usually will be superior to the earlier approximation, the forward difference formula (6.51).

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.4.2 The Method of Undetermined Coefficients The method of undetermined coefficients is a procedure used in deriving formulae for numerical differentiation, interpolation and integration. We will explain the method by using it to derive an approximation for f 00(x). To approximate f 00(x) at some point x = t, write

00 (2) f (t) ≈ Dh f(t) ≡ Af(t + h) + Bf(t) + Cf(t − h) (6.61) with A, B and C unspeciﬁed constants. Replace f(t − h) and f(t + h) by the Taylor polynomial approximations

2 3 4 f(t − h) ≈ f(t) − hf 0(t) + h f 00(t) − h f 000(t) + h f (4)(t) 2 6 24 (6.62) 0 h2 00 h3 000 h4 (4) f(t + h) ≈ f(t) + hf (t) + 2 f (t) + 6 f (t) + 24 f (t) Including more terms would give higher powers of h; and for small values of h, these additional terms should be much smaller that the terms included in (6.62).

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.4.2 The Method of Undetermined Coefficients (2) Substituting these approximations into the formula for Dh f(t) and collecting together common powers of h give us 2 (2) 0 h 00 Dh f(t) ≈ (A + B + C) f(t) + h(A − C) f (t) + (A + C) f (t) | {z } | {z } 2 =0 =0 | {z } =1 h3 h4 + (A − C)f 000(t) + (A + C)f (4)(t) (6.63) 6 24 To have (2) 00 Dh f(t) ≈ f (t) for arbitrary functions f(x), it is necessary to require A + B + C = 0; coefficient of f(t) h(A − C) = 0; coefficient of f 0(t) h2 00 2 (A + C) = 1; coefficient of f (t) This system has the solution 1 2 A = C = ,B = − . h2 h2 5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.4.2 The Method of Undetermined Coefficients This determines f(t + h) − 2f(t) + f(t − h) D(2)f(t) = (6.64) h h2 (2) To determine an error error formula for Dh f(t), substitute 1 2 A = C = h2 ,B = − h2 into (6.63) to obtain h2 D(2)f(t) ≈ f 00(t) + f (4)(t). h 12 The approximation in this arises from not including in the Taylor polynomials (6.62) corresponding higher powers of h. Thus, f(t + h) − 2f(t) + f(t − h) −h2 f 00(t) − ≈ f (4)(t) (6.65) h2 12 This is accurate estimate of the error for small values of h. Of course, in a practical situation we would not know f (4)(t). But the error formula shows that the error decreases by a factor of about 4 when h is halved. This can lead to justify Richardson’s extrapolation to obtain an even more accurate estimate of the error and of f 00(t).

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.4.2 The Method of Undetermined Coeﬃcients

Example 1 Let f(x) = cos(x), t = 6 π, and use (6.64) to calculate 00 1 f (t) = − cos( 6 π).

2 h Dh(f) Error Ratio 0.5 -0.84813289 -1.789E-2 0.25 -0.86152424 -4.501E-3 3.97 0.125 -0.86489835 -1.127E-3 3.99 0.0625 -0.86574353 -2.819E-4 4.00 0.03125 -0.86595493 -7.048E-5 4.00

The results shown (see the ratio column) are consistent with the error formula (6.65) f(t + h) − 2f(t) + f(t − h) −h2 f 00(t) − ≈ f (4)(t). h2 12

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.4.2 The Method of Undetermined Coeﬃcients In the derivation of (6.65), the form (6.61) 00 (2) f (t) ≈ Dh f(t) ≡ Af(t + h) + Bf(t) + Cf(t − h) was assumed for the approximate derivative. We could equally well have chosen to evaluate f(x) at points other than those used there, for example,

f 00(t) ≈ Af(t + 2h) + Bf(t + h) + Cf(t)

Or, we could have chosen more evaluation points, as in

f 00(t) ≈ Af(t + 3h) + Bf(t + 2h) + Cf(t + h) + Df(t)

The extra degree of freedom could have been used to obtain a more accurate approximation to f 00(t), by forcing the error term to be proportional to a higher power of h. Many of the formulae derived by the method of undetermined coeﬃcients can also be derived by diﬀerentiating and evaluating a suitably chosen interpolation polynomial. But often, it is easier to visualize the desired formula as a combination of certain function values and to then derive the proper combination, as was done above for (6.64).

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.4.3 Effects of error in function evaluation The formulae derived above are useful for differentiating functions that are known analytically and for setting up numerical methods for solving differential equations. Nonetheless, they are very sensitive to errors in the function values, especially if these errors are not sufficiently small compared with the stepsize h used in the differentiation formula. To explore this, we analyze the effect of such errors in the formula (2) 00 Dh f(t) approximating f (t). (2) f(t+h)−2f(t)+f(t−h) Rewrite (6.64), Dh f(t) = h2 , as f(x ) − 2f(x ) + f(x ) D(2)f(x ) = 2 1 0 ≈ f 00(x ) h 1 h2 1

where x2 = x1 + h, x0 = x1 − h. Let the actual function values used in the computation be denoted by fb0, fb1, and fb2 with

f(xi) − fbi = i, i = 0, 1, 2

the errors in the function values.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.4.3 Eﬀects of error in function evaluation

Thus, the actual quantity calculated is

2 fb2 − 2fb1 + fb0 Db f(x1) = h h2

For the error in this quantity, replace fbj by f(xj) − j, j = 0, 1, 2 to obtain

00 2 00 [f(x2) − 2] − 2[f(x1) − 1] + [f(x0) − 0] f (x1)−Db f(x1) = f (x1) − h h2 h f(x ) − 2f(x ) + f(x ) i − 2 + = f 00(x ) − 2 1 0 + 2 1 0 1 h2 h2 | {z } (6.65) −h2 (4) = 12 f (t) 2 h (4) 2 − 21 + 0 ≈ − f (x1) + (6.66) 12 h2

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.4.3 Eﬀects of error in function evaluation

The errors 0, 1, 2 are generally random in some interval [−δ, δ].

If the values fb0, fb1, fb2 are experimental data, then δ is a bound on the experimental data.

Also, if these function values fbi are obtained from computing f(x) on a computer, then the errors j are combination of rounding or chopping errors and δ is a bound on these errors. In either case, (6.66)

2 00 2 h (4) 2 − 21 + 0 f (x1)−Db f(x1) ≈ − f (x1) + h 12 h2 yields the approximate inequality

2 00 (2) h (4) 4δ f (x1) − Db f(x1) ≤ f (x1) + (6.67) h 12 h2 This error bound suggests that as h → 0, 4δ the error will eventually increase, because of the ﬁnal term h2 . 5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.4.3 Eﬀects of error in function evaluation

Example

(2) 1 Calculate Dbh (x1) for f(x) = cos(x) at x1 = 6 π. To show the eﬀect of rounding errors, the values fbi are obtained by rounding f(xi) to six signiﬁcant digits; and the errors satisfy

−7 |i| ≤ 5.0 × 10 = δ, i = 0, 1, 2

(2) Other than these rounding errors, the formula Dbh f(x1) is calculated exactly. The results are 2 h Dbh(f) Error 0.5 -0.848128 -0.017897 0.25 -0.861504 -0.004521 0.125 -0.864832 -0.001193 0.0625 -0.865536 -0.000489 0.03125 -0.865280 -0.000745 0.015625 -0.860160 -0.005865 0.0078125 -0.851968 -0.014057 0.00390625 -0.786432 -0.079593

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.4.3 Eﬀects of error in function evaluation In this example, the bound (6.67), i.e., 2 f 00(x ) − D(2)f(x ) ≤ h f (4)(x ) + 4δ , 1 bh 1 12 1 h2 becomes 2 00 (2) h 1 4 −7 f (x1) − Db f(x1) ≤ cos π + (5 × 10 ) h 12 6 h2 2 × 10−6 =· 0.0722h2 + ≡ E(h) h2

For h = 0.125, the bound E(h) =· 0.00126, which is not too far oﬀ from actual error given in the table.

The bound E(h) indicates that there is a smallest value of h, call it h∗, below which the error will begin to increase. 0 To ﬁnd it, let E (h) = 0, with its root being h∗. · This leads to h∗ = 0.00726, which is consistent with the behaviour of the errors in the table.

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.4.3 Eﬀects of error in function evaluation

5. Numerical Integration Math 1070 > 5. Numerical Integration > 5.4.3 Eﬀects of error in function evaluation One must be very cautious in using numerical diﬀerentiation, because of the sensitivity to errors in the function values.

This is especially true if the function values are obtained empirically with relatively large experimental errors, as is common in practice. In this latter case, one should probably use a carefully prepared package program for numerical differentiation. Such programs take into account the error in the data, attempting to find numerical derivatives that are as accurate as can be justified by the data.

In the absence of such a program, one should consider producing a cubic spline function that approximates the data, and then use its derivative as a numerical derivative for the data. The cubic spline function could be based on interpolation; or better for data with large relatively errors, construct a cubic spline that is a least squares approximation to the data. The concept of least squares approximation is introduced in Section 7.1.

5. Numerical Integration Math 1070