Derivation of Numerical Methods Using Computer Algebra∗
Total Page:16
File Type:pdf, Size:1020Kb
SIAM REVIEW c 1999 Society for Industrial and Applied Mathematics Vol. 41, No. 3, pp. 577–593 Derivation of Numerical Methods Using Computer Algebra∗ Walter Gandery Dominik Gruntzz Abstract. The use of computer algebra systems in a course on scientific computation is demonstrated. Various examples, such as the derivation of Newton's iteration formula, the secant method, Newton{Cotes and Gaussian integration formulas, as well as Runge{Kutta formulas, are presented. For the derivations, the computer algebra system Maple is used. Key words. numerical methods, computer algebra, Maple, quadrature formulas, nonlinear equations, finite elements, Rayleigh{Ritz, Galerkin method, Runge{Kutta method AMS subject classifications. 65-01, 68Q40, 65D32, 65H05, 65L60, 65L06 PII. S003614459935093X 1. Introduction. At ETH Z¨urich we have redesigned our courses on numerical analysis. We not only teach numerical algorithms but also introduce the students to computer algebra and make heavy use of computer algebra systems in both lectures and assignments. Computer algebra is used to generate numerical algorithms, to compute discretization errors, to derive convergence rates, to simplify proofs, to run examples, and to generate plots. We claim that it is easier for students to follow a derivation carried out with the help of a computer algebra system than one done by hand. Computer algebra systems take over the hard manual work, such as solving systems of equations. Students need not be concerned with all the details (and all the small glitches) of a manual derivation and can understand and keep an overview of the general steps of the derivation. A computer-supported derivation is also more convincing than a presentation of the bare results without any reasoning. Moreover, using computer algebra systems, rather complex numerical formulas can be derived|far more complex than what can be done in class by hand. For example, all useful Newton{Cotes rules can be computed without difficulty, in contrast to hand derivations, which usually end with Simpson's rule. We will demonstrate these claims with examples taken from our introductory courses in scientific computing. We start with formulas for solving nonlinear equa- tions (Newton's formula, the secant method, and a new formula based on inverse ∗Received by the editors July 7, 1998; accepted for publication (in revised form) January 21, 1999; published electronically July 27, 1999. http://www.siam.org/journals/sirev/41-3/35093.html yInstitute of Scientific Computing, ETH-Zentrum, CH-8092 Z¨urich, Switzerland (gander@inf. ethz.ch). zFachhochschule Aargau, Klosterzelgstrasse, CH-5210 Windisch, Switzerland (gruntz@fh-aargau. ch). 577 578 WALTER GANDER AND DOMINIK GRUNTZ interpolation) and show, in section 5, how Newton{Cotes rules can be derived. In the second part of the paper we show how computer algebra can be used to support more complex derivations such as Gauss quadrature rules (section 7), Runge{Kutta formulas (section 8), and the methods of Ritz and Galerkin (section 9). In our examples we use Maple V Release 5, but the computations could also be reproduced with, e.g., Mathematica, MuPad, or any other computer algebra system. 2. Newton’s Formula. One of the first formulas that students learn is Newton's iteration for solving a nonlinear equation f(x) = 0. Given an approximation xk for the root s, a better approximation, xk+1 = F (xk), can be obtained using the iteration function > F := x -> x - f(x) / D(f)(x); f(x) (2.1) F := x ! x − : D(f)(x) This iteration converges quadratically to a simple root s of f(x). This can be proven by computing the first derivative of F (x)atx = s, which is zero. > dF := D(F)(x); f(x)D(2)(f)(x) (2.2) dF := ; D(f)(x)2 > subs(f(s) = 0, D(F)(s)); 0; > subs(f(s) = 0, D(D(F))(s)); D(2)(f)(s) : D(f)(s) Arbitrary precision floating point arithmetic, which is provided by most computer algebra systems, can be used to demonstrate what quadratic convergence means \in real life." As an example, we compute the square root of 9 by using Newton's iteration 2 to solve equation x − 9 = 0 starting with x0 = 1. As expected, the number of correct digits doubles with each iteration. >f:=x->x^2-9: > Digits := 70: > xk := 1.0: to 8 do xk := F(xk); lprint(xk); od: 5.000000000000000000000000000000000000000000000000000000000000000000000 3.400000000000000000000000000000000000000000000000000000000000000000000 3.023529411764705882352941176470588235294117647058823529411764705882353 3.000091554131380178530556191348134584573128862439917601281757839322499 3.000000001396983862248478425258881977121085388846948134910070368766335 3.000000000000000000325260651745651330219868255523316826048221278205013 3.000000000000000000000000000000000000017632415262334312619531048058334 3.000000000000000000000000000000000000000000000000000000000000000000000. The convergence result is valid only for simple roots, because the first derivative of f appears in the denominator of (2.2); i.e., the result is valid only if f 0(s) =0.6 The behavior of Newton's iteration for an equation with a multiple root is the next topic we discuss. Let us assume that f(x) has a zero of multiplicity n at x = s.We therefore define f(x)tobe > f := x -> (x-s)^n * g(x); f := x ! (x − s)n g(x); DERIVATION OF NUMERICAL METHODS 579 where g(s) =6 0. Again we inspect the first derivative of F (x). If F 0(s) =6 0, then the iteration converges only linearly. > dF; (x − s)n n2 g(x) (x − s)n ng(x) (x − s)n n D(g)(x) (x − s)n g(x) − +2 (x − s)2 (x − s)2 x − s ! (x − s)n ng(x) 2 +(x − s)n (D(2))(g)(x) +(x − s)n D(g)(x) : x − s Taking the limit of the above expression for x ! s we obtain > limit(%, x=s); n − 1 : n We have just proven that Newton's iteration converges linearly with factor (n − 1)=n if f(x) has a zero of multiplicity n. Thus, e.g., convergence is linear with factor 1=2 for a double root. Newton's iteration also has a nice geometrical interpretation. Starting with the approximation xk, the next value xk+1 of the iteration is the intersection of the tangent to f(x)at[xk;f(xk)] with the x-axis. This property can also be proven with Maple. We set up an equation p(x)=ax+ b for the tangent line. p(x) must interpolate [xk;f(xk)] and must have the same derivative as f(x)atx = xk. With these two conditions the parameters a and b of the tangent p(x) are defined. > f := 'f': >p:=x->a*x+b: > solve({p(x[k]) = f(x[k]), D(p)(x[k]) = D(f)(x[k])}, {a, b}); fb = −D(f)(xk) xk + f(xk);a=D(f)(xk)g: We have claimed that the intersection of the tangent p(x) with the x-axis is the next Newton approximation. If the equation p(x) = 0 is solved, the iteration function (2.1) is obtained, proving that the geometrical interpretation is correct. > assign(%); > x[k+1] = expand(solve(p(t) = 0, t)); f(xk) xk+1 = xk − : D(f)(xk) 3. Secant Method. Another method we present to our students is the secant method. Compared to Newton's iteration, the secant method has the advantage that no derivatives are needed. The derivative that appears in Newton's formula is approximated by a finite difference. The interesting aspect we demonstrate with the help of Maple is the convergence rate of this method. The iteration function is given in (3.1). > F := (u, v) -> u - f(u) * (u-v) / (f(u) - f(v)); f(u)(u − v) F := (u; v) ! u − ; f(u) − f(v) > x[k+1] = F(x[k], x[k-1]); f(xk)(xk − xk−1) (3.1) xk+1 = xk − : f(xk) − f(xk−1) 580 WALTER GANDER AND DOMINIK GRUNTZ Using (3.1) we obtain for the error ek+1 = xk+1 − s the recurrence ek+1 = F (s + ek;s+ ek−1) − s: The right-hand side can be expanded into a multivariate Taylor series at ek = 0 and 0 00 ek+1 = 0. We assume that s is a simple root (f (s) =6 0) and also that f (s) =06 holds. We set f(s) = 0 and compute the first term of the Taylor series expansion: > f(s) := 0: > e2 = normal(readlib(mtaylor)(F(s + e1, s + e0) - s, [e0, e{1], 4)); 1 e0(D(2))(f)(s) e1 e2= : 2 D(f)(s) If we divide this leading coefficient by e0 and e1, we see that the limit of the quotient e2=(e0 e1) is a constant different from zero. We assume that the convergence coefficient p p p is p, substitute e2 = Ke1 and e1 = Ke0 , and divide by the constant K . > %/e1/e0; e2 1 (D(2))(f)(s) = ; e1 e0 2 D(f)(s) > simplify(subs(e2 = K*e1^p, e1 = K*e0^p, %/K^p), assume=positive); −p (2) 2 1 K (D )(f)(s) e0(p −p−1) = : 2 D(f)(s) This equation is valid for all errors e0. Since the right-hand side is constant, the left-hand side also must be independent of e0. This is the case only if the exponent of e0 is zero. This conditionp is an equation for p, whose solution gives the well-known convergence factor p =(1+ 5)=2 for the secant method. > solve(ln(lhs(%)), p); 1 p 1 1 1 p 5+ ; − 5: 2 2 2 2 4. A New Iteration Formula. Having considered the Newton and the secant methods to compute roots of a nonlinear equation, we now want to show how a new iteration method can be derived and analyzed. The new method is a combination of the Newton and the secant methods.