Numerical Integration

Home , Gaussian quadrature, Numerical integration

MAT 2310 Computational Mathematics Wm C Bauldry

November, 2011

ICM 1 Contents

1. Numerical Integration 5. Gaussian Quadrature

2. Left- and Right Endpoint, 6. Gauss-Kronrod Quadrature and Midpoint Sums 7. Test Integrals 3. Trapezoid Sums 8. Exercises 4. Simpson’s Rule 9. Links and Others

Today

ICM 2 Numerical Integration What is Numerical Integration?

Numerical integration or (numerical) 2.5 quadrature is the calculation of a deﬁnite 2 1.5 integral using numerical formulas, not the 1 fundamental theorem. The Greeks stud- 0.5 ied quadrature: given a ﬁgure, construct a -1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 -0.5 square that has the same area. The two -1 most famous are Hippocrates of Chios’ Quadrature of the Lune (c. 450 BC) and Archimedes’ Quadrature of the Parabola (c. 250 BC). Archimedes used the method of exhaustion — a precursor of calculus — invented by Eudoxus. Squaring the circle is one of the classical problems of constructing a square with the area of a given circle – it was shown impossible by Lindemann’s theorem (1882).

ICM 3 Methods of Elementary Calculus Rectangle Methods Left endpoint sum Midpoint sum Right endpoint sum

n n n X X X An ≈ f(xk−1) ∆xk An ≈ f(mk) ∆xk An ≈ f(xk) ∆xk k=1 k=1 k=1 1 mk = 2 (xk−1 + xk)

(b−a)2 1 (b−a)3 1 (b−a)2 1 εn ≤ · M1 · εn ≤ · M2 · εn ≤ · M1 · 2 n 24 n2 2 n where M = max f (i)(x) R πf dx = 2 ≈ [1.984, 2.008, 1.984] i 0 n=10 ICM 4 Trapezoid Sums Trapezoid Sums Instead of the degree 0 rectangle approximations to the function, use a linear degree 1 approximation. The area of the trapezoid is given by 1 AT = 2 [f(xk−1) + f(xk)] ∆xk This gives an approximation for the integral Z b n X 1 f(x) dx ≈ 2 [f(xk−1) + f(xk)] ∆xk a k=1 [Midpoint: measure height at average x v. trapezoid: average the height measures] The formula is often written as " n−1 ! # X ∆xk T ≈ f(x ) + 2 f(x ) + f(x ) n 0 k n 2 k=1 Error for the trapezoid rule is (b − a)3 1 ε ≤ · M · n 12 2 n2 ICM 5 Sample Trapezoid

Example 1 1 1 Let f(x) = sin(x) + 2 sin(2x) − 4 sin(4x) + 16 sin(8x) over [0, π]. With an equipartition,

∆x = π/10 ≈ 0.314

Then

" 10 ! # X ∆x T = f(0)+ 2 f( k π) +f(π) 10 9 2 k=1 which gives

T10 = 1.984 with absolute error of 0.016.

ICM 6 Simpson’s Rule Simpson’s Rule We now move to a degree 2 approximation. The easiest way to have 3 data pts is to take the panels in pairs: instead of rectangle base [xi, xi+1], use [xi, xi+1, xi+2]. So we require an even number of panels. The area under the parabola is 1 h i AS = 3 f(xi) + 4f(xi+1) + f(xi+2) ∆x This gives a 2n-panel approximation for the integral " n # Z b X ∆x f(x) dx ≈ f(x ) + 4f(x ) + f(x ) 2k−2 2k−1 2k 3 a k=1 often written as ∆x S = [f(x ) + 4f(x ) + 2f(x ) + 4f(x ) + ··· + 4(f(x ) + f(x )] 2n 0 1 2 3 2n−1 2n 3 The error is bounded by (b − a)5 1 ε ≤ · M · n 180 4 n4 ICM 7 Sample Simpson

Example 1 1 1 Let f(x) = sin(x) + 2 sin(2x) − 4 sin(4x) + 16 sin(8x) over [0, π].

With a 10 panel equipartition,

∆x = π/10 ≈ 0.314

Then, with yi = f(xi),

1 S10 = 3 [y0 +4y1 +2y2 +···+4y9 +y10]∆x which gives

S10 = 2.000 with absolute error of 6.78 · 10−6.

ICM 8 A Maple Comparison

Approximating a Difﬁcult Integral Z 2 10−4 Consider dx. The integrand has a sharp peak at π/2. 1 2 −8 1 (x − 2 π) + 10 The exact value of the integral (FToC) is arctan(−10000 + 5000 π) + arctan(20000 − 5000 π) ≈ 3.14118447

Maple gives n = 50 n = 500 n = 5000  left 0.0497205  left 0.539346  left 3.42497  right 0.0497253  right 0.539345  right 3.42495        midpoint 3.0930000  midpoint 4.167060  midpoint 2.88089       trapezoid 0.0497228 trapezoid 0.539342 trapezoid 3.42498 Simpson 2.0785900 Simpson 2.957850 Simpson 3.06226

To achieve relative error below 1% requires n ≥ 8000.

ICM 9 Þe Charte Ob Anfeald Gang

Quadrature “Height” Error Bound

(b−a)2 1 Left end point f(xi) 2 · M1 · n

(b−a)2 1 Right end point f(xi+1) 2 · M1 · n

3 f 1 (x + x ) (b−a) · M · 1 Midpoint 2 i i+1 24 2 n2

3 1 (f(x ) + f(x )) (b−a) · M · 1 Trapezoid Rule 2 i i+1 12 2 n2

1 (b−a)5 1 Simpson’s Rule 3 (f(xi) + 4f(xi+1) + f(xi+2)) 180 · M4 · n4

(i) where Mi ≥ max f (x) .

ICM 10 Gaussian Quadrature

Johann Carl Friedrich Gauss About 1815, while Gauss was ﬁnishing constructing astronomical observat- ory, he wrote a paper1 on approximating integrals. Gauss’s technique was studied and extended by Christoffel in 1858. There are several ways to develop his method. We’ll use the easiest . . .

In Search of Improvements Write the rules we’ve seen as sums:

1 1 1 Left endpt: Ln= n f(x0) + n f(x1) + ··· + n f(xn−1) 1 1 1 Right endpt: Rn= n f(x1) + n f(x2) + ··· + n f(xn) 1 1 1 Midpoint:Mn= n f(xm1 ) + n f(xm2 ) + ··· + n f(xmn ) 1 1 1 1 Trapezoid: Tn= 2n f(x0) + n f(x1) + ··· + n f(xn−1) + 2n f(xn) 1 4 2 4 1 Simpson’s: Sn= 3n f(x0) + 3n f(x1) + 3n f(x2) + ··· + 3n f(xn−1) + 3n f(xn)

1 “Methodus nova integralium valores per approximationem inveniendi,” Werke, 3, K. Ges. Wissenschaft. Göttingen (1886) pp. 163–196. ICM 11 Patterns

Observe 1. Each of the formulas has the same form

An = w1 · f(x1) + w2 · f(x2) + ··· + wn · f(xn)

with different sets of weights wi and different sets of nodes xi.

2. Any closed interval can be mapped to and from [−1, 1], so we R 1 can focus only on −1 f(x) dx.

3. Gauss posed the question: Is there a “best choice” of weights {wi} and nodes {xi}?

4. The answer depends on what “best” means.

5. Since we have 2n ‘unknowns’ wi and xi, let’s look for a set that integrates a 2n − 1 degree polynomial exactly.

ICM 12 Sampling 3 Example (Third Degree)

Set n = 3. Determine the choice of wi and of xi so that Z 1 3 p X p x dx = wk · (xk) −1 k=1 exactly for p = 0, 1,..., 5 = 2 · 3 − 1.

The range for p gives us six equations:    w1 + w2 + w3 = 2       w x + w x + w x = 0   1 1 2 2 3 3   √ √   15 15  2 2 2 2   x1 = − 5 , x2 = 0, x3 = 5  w1x1 + w2x2 + w3x3 =   3 =⇒  w x3 + w x3 + w x3 = 0    1 1 2 2 3 3   w = 5 , w = 8 , w = 5    1 9 2 9 3 9  4 4 4 2   w1x1 + w2x2 + w3x3 = 5     5 5 5   w1x1 + w2x2 + w3x3 = 0 

√ √ 5 15 8 5 15 Our Gaussian quadrature is G3(f) = 9 f − 5 + 9 f(0) + 9 f 5 ICM 13 Testing Gauss

Random Polynomials Generate and test a random 5th degree polynomial. p := unapply(sort(randpoly(x, degree = 5), x), x) x → −7x5 + 22x4 − 55x3 − 94x2 + 87x − 56 G3 := 5/9*p(-sqrt(15)/5) + 8/9*p(0) + 5/9*p(sqrt(15)/5) 2488 − 15 Int(p(x), x = -1..1) = int(p(x), x = -1..1) Z 1 2488 p(x) dx = − 15 −1

Generate and test a random 7th degree polynomial. q := unapply(sort(randpoly(x, degree = 7), x), x) x → 97x7 − 73x6 − 4x5 − 83x3 − 10x − 62 int(q(x),x= -1..1)= 5/9*q(-sqrt(15)/5)+8/9*q(0)+5/9*q(sqrt(15)/5) 722 2662 7 = 25

ICM 14 Gaussian Properties Theorem Z 1

Let f have 2n continuous derivatives. Then for εn = Gn − f(x)dx , 2 −1 ε ≤ · M 2n n (2n + 1)!

(2n) where M2n ≥ max f (x) .

Values of Gaussian Weights and Nodes There are numerous sources online, e.g.,: 1. The classic Abramowitz and Stegun Handbook (see the entire book) 2. Holoborodko or Kamermans Or calculate the values directly: 1 dn 2 n Set Pn(x) = 2n n! · dxn x − 1 (the Legendre polynomials). Then

n 2 {xi}i=1 = {zeros of Pn} and wi = 2 0 2 (1 − xi )[Pn(xi)]

ICM 15 Aleksandr Kronrod

Kronrod’s Idea (1964) One difficulty in Gaussian quadrature is that increasing the number of nodes requires recomputing all the values of • nodes • weights • function evaluations Kronrod2 discovered he could interlace n + 1 new nodes with n original Gaussian nodes and have a rule of order 3n + 1. A 2n + 1 node Gaussian quadrature would have order 4n + 1, but with significant extra computation for an increase of only n in order over Kronrod’s method. (Bad news: calculating the nodes and weights is way beyond the scope of our class.) Gauss-Kronrod quadrature is used in the CASs Maple, Mathematica, Matlab, and Sage; it’s included in the QUADPACK library, the GNU Scientific Library, the NAG Numerical Libraries, andR. GK 7,15 is the basis of TI’s numerical integration in calculators.

2Kronrod, A. S. (1964.) “Integration with control of accuracy” (in Russian), Dokl. Akad. Nauk SSSR 154, 283–286. ICM 16 Gauss-Kronrod Quadrature in Practice

GK7,15 (1989) A widely used implementation is based on a Gaussian quadrature with 7 nodes. Kronrod adds 8 to total 15 nodes.

GK7,15 on [−1, 1] 7 Gauss-7 nodes Weights X 0.00000 00000 00000 0.41795 91836 73469 G7 = wk f(xk) ±0.40584 51513 77397 0.38183 00505 05119 k=1 ±0.74153 11855 99394 0.27970 53914 89277 15 ±0.94910 79123 42759 0.12948 49661 68870 X GK7,15 = wk f(xk) Kronrod-15 nodes Weights k=1 0.00000 00000 00000 0.20948 21410 84728 ±0.20778 49550 07898 K 0.20443 29400 75298 ε7,15 ≈ |G7 − GK7,15| ±0.40584 51513 77397 0.19035 05780 64785 3 ±0.58608 72354 67691 K 0.16900 47266 39267 or, in practice, use ±0.74153 11855 99394 0.14065 32597 15525 3/2 ±0.86486 44233 59769 K 0.10479 00103 22250 ≈ [200 |G7 −GK7,15|] ±0.94910 79123 42759 0.06309 20926 29979 ±0.99145 53711 20813 K 0.02293 53220 10529

3Kahaner, Moler, & Nash, Numerical Methods and Software, Prentice-Hall, 1989. ICM 17 GK Sample

Example 1 Z 2 Find e−x dx. −1 Using Maple gives: 7 X G7 = wkf(xk) = 1.49364828886941 k=1

15 X GK7,15 = wkf(xk) = 1.49364826562485 k=1

−8 ε7,15 = |G7 − GK7,15| = 2.324456 · 10

evalf(Int(f(x),x=-1..1) ≈ 1.49364826562485

ICM 18 An Exercise

Easy, but Hard Set f(x) = x − bxc on [0, 32/5]. Set n = 10. Find 1. The exact value 2. Left endpoint approximation 3. Right endpoint approximation 4. Midpoint approximation 5. Trapezoid rule approximation 6. Simpson’s rule approximation 7. Gauss 7 quadrature 8. Gaussian-Kronrod 7-15 quadrature

ICM 19 Test Integrals, I

Integrals for Testing Numerical Quadratures, I4 Lyness: Z 2 0.1 1. I(λ) = dx 2 1 (x − λ ) + 0.01

Piessens, de Doncker-Kapenga, Überhuber, & Kahaner:

1 1 2 α+1 1 α+1 Z 1 1 Z α ( ) +( ) 2. xα log dx = 5. x − 1 dx = 3 3 2 3 1+α 0 x (1 + α) 0

Z 1 4−α Z 1 π α 3. dx 6. x − dx π 2 0 −α 0 4 x − 4 + 16 (1− π )α+1+( π )α+1 = tan−1 (4 − π)4α−1 4 4 = 1+α + tan−1 π4α−1 Z +1 1 1 Z π 7. √ dx α α 2 −α 4. cos(2 sin(x)) dx = πJ0(2 ) −1 1 − x 1 + x + 2 0 = √ π (1+2−α)2−1

4From D. Zwillinger’s Handbook of Integration (A K Peters/CRC Press, 1992). ICM 20 Test Integrals, II

Integrals for Testing Numerical Quadratures, II Piessens, et al (continued):

Z π/2 Z 1 α α−1 cos(2 x) 8. sin (x) dx 10. p dx 0 0 x(1 − x) Γ2 α α−1 α−1 α−2 ( 2 ) = π cos(2 )J0(2 ) = 2 Γ(α)

Z ∞ −α Z 1 11. x2e−2 x dx = 23α+1 α−1 1 9. log dx = Γ(α) 0 0 x Z ∞ xα−1 (1 − α)π 12. dx = 2 α 0 (1 + 10x) 10 sin(πα) Berntsen, Espelid, & Sørevik:

Z 1 Z 1 1 −1/2 2|x−1/3| 13. x − 3 dx (singularity) 15. e dx (C0 function) 0 0

Z 1 Z 2 10−4 14. U(x) ex/2 dx (discontinuity) 16. dx (sharp peak) π 2 −1 1 −8 x − 2 +10

ICM 21 Problems

Exercises For each of the following functions, investigate the integrals using: left endpoint, midpoint, trapezoid, and Simpson’s rules Z x √ 1. S(x) = sin(t2/2) − π/(2x) dt 0 Z 2 0.1 2. Lyness’ integral I(λ) = 2 dx for λ = π/2 1 (x − λ ) + 0.01

1 Z 2 0.1 3. Modiﬁed Piessens’ integral x2 − π dx 16 −1

Investigate Gaussian and Gauss-Kronrod quadrature (after transforming the interval to [−1, 1]) of the integral Z 2 10−4 4. dx π 2 −8 1 x − 2 +10

ICM 22 Links and Others

More information: Dr John Matthews’ modules: Wikipedia entries: • Adaptive Simpson’s Rule • Newton-Cotes formulas • Monte Carlo Integration • Romberg’s method • Legendre Polynomials • Clenshaw-Curtis integration • Chebyshev Polynomials • Cubature

See also: MathWorld (Wolfram Research) and Interactive Educational Modules in Scientiﬁc Computing (U of I) Investigate: • Boole’s Rule • Maple’s evalf/Int command • Gauss-Kronrod Quadrature • Matlab’s quad command • Vandermonde Matrix • The Maple command • orthogonal polynomials ApproximateInt in the Student[Calculus1] package • Cubature

ICM 23