Chapter 6 Nonlinear Equations 6.1 The Problem of Nonlinear Root-finding In this module we consider the problem of using numerical techniques to find the roots of nonlinear x = 0 equations, f . Initially we examine the case where the nonlinear equations are a scalar function of a single independent variable, x. Later, we shall consider the more difficult problem n f x= 0 where we have a system of n nonlinear equations in independent variables, . x = 0 Since the roots of a nonlinear equation, f , cannot in general be expressed in closed form, we must use approximate methods to solve the problem. Usually, iterative techniques are used to start from an initial approximation to a root, and produce a sequence of approximations 0 1 2 x ;x ;x ;::: ; which converge toward a root. Convergence to a root is usually possible provided that the function x f is sufficiently smooth and the initial approximation is close enough to the root. 6.2 Rate of Convergence Let 0 1 2 x ;x ;x ;::: c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002 70 NONLINEAR EQUATIONS k k = x p be a sequence which converges to a root ,andlet . If there exists a number and a non-zero constant c such that k +1 lim = c; p (6.1) k k !1 j j p = 1; 2; 3 then p is called the order of convergence of the sequence. For the order is said to be linear, quadratic and cubic respectively. 6.3 Termination Criteria Since numerical computations are performed using floating point arithmetic, there will always be a finite precision to which a function can be evaluated. Let denote the limiting accuracy of the nonlinear function near a root. This limiting accuracy of the function imposes a limit on the accuracy, , of the root. The accuracy to which the root may be found is given by: : = (6.2) 0 jf j This is the best error bound for any root finding method. Note that is large when the first derivative 0 f j of the function at the root, j , is small. In this case the problem of finding the root is ill- conditioned. This is shown graphically in Figure 6.1. f x f x 2 2 1 2 2 1 2 2 x 0 1 large 2 small f 1 ill-conditioned well-conditioned FIGURE 6.1: When the sequence 0 1 2 x ;x ;x ;::: ; k k 1 k x x converges to the root, , then the differences x will decrease until . c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002 6.4 BISECTION METHOD 71 With further iterations, rounding errors will dominate and the differences will vary irregularly. k The iterations should be terminated and x be accepted as the estimate for the root when the following two conditions are satisfied simultaneously: k +1 k k k 1 x x x 1. x and (6.3) k k 1 x x : 2. < (6.4) k 1+jx j k x is some coarse tolerance to prevent the iterations from being terminated before is close to , k k 1 x i.e. before the step size x becomes “small”. The condition (6.4) is able to test the relative k x 1 offset when is much larger than 1 andmuchsmallerthan . In practice, these conditions are used in conjunction with the condition that the number of iterations not exceed some user defined limit. 6.4 Bisection Method 0 1 f x ;x Suppose that is continuous on an interval x . A root is bracketed on the interval 0 1 0 1 f f ;x x x x if and have opposite sign. Let 1 0 x x 2 0 = x + x (6.5) 2 0 1 ;x be the mid-point of the interval x . Three mutually exclusive possibilities exist: 2 f x =0 if then the root has been found; 2 0 2 1 f x f x ;x if has the same sign as then the root is in the interval x ; 0 2 2 1 x ;x f x f x if has the same sign as then the root is in the interval . In the last two cases, the size of the interval bracketing the root has decreased by a factor of two. The next iteration is performed by evaluating the function at the mid-point of the new interval. After k iterations the size of the interval bracketing the root has decreased to: 1 0 x x k 2 c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002 72 NONLINEAR EQUATIONS The process is shown graphically in Figure 6.2. The bisection method is guaranteed to converge to a root. If the initial interval brackets more than one root, then the bisection method will find one of them. f x 1 x 0 x x 2 x 3 x 4 x 5 x FIGURE 6.2: Since the interval size is reduced by a factor of two at each iteration, it is simple to calculate in k advance the number of iterations, , required to achieve a given tolerance, 0 , in the solution: 1 0 x x k = : log2 0 The bisection method has a relatively slow linear convergence. 6.5 Secant Method x The bisection method uses no information about the function values, f , apart from whether they are positive or negative at certain values of x. Suppose that k 1 k ; < f x f x k k 1 x Then we would expect the root to lie closer to x than . Instead of choosing the new estimate to lie at the midpoint of the current interval, as is the case with the bisection method, the secant k 1 k 1 x ;f x method chooses the x-intercept of the secant line to the curve, the line through k k ;f x f x and x . This places the new estimate closer to the endpoint for which has the smallest absolute value. The new estimate is: k k k 1 f x x x k +1 k x = x : (6.6) k k 1 f x f x c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002 6.6 REGULA FALSI 73 Note that the secant method requires two initial function evaluations but only one new function evaluation is made at each iteration. The secant method is shown graphically in Figure 6.3. f x 0 1 2 x x x 3 x x FIGURE 6.3: The secant method does not have the root bracketing property of the bisection method since k +1 k 1 k x x the new estimate, x , of the root need not lie within the bounds defined by and .As a consequence, the secant method does not always converge, but when it does so it usually does so faster than the bisection method. It can be shown that the order of convergence of the secant method is: p 1+ 5 1:618: 2 6.6 Regula Falsi Regula falsi is a variant of the secant method. The difference between the secant method and regula falsi lies in the choice of points used to form the secant. While the secant method uses k 1 k 1 ;f x the two most recent function evaluations to form the secant line through x and k k x x ;f , regula falsi forms the secant line through the most recent points that bracket the 0 root. The regula falsi steps are shown graphically in Figure 6.4. In this example, the point x remains active for many steps. The advantage of regula falsi is that like the bisection method, it is always convergent. How- ever, like the bisection method, it has only linear convergence. Examples where the regula falsi method is slow to converge are not hard to find. One example is shown in Figure 6.5. c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002 74 NONLINEAR EQUATIONS f x 0 1 x x 3 2 x x x FIGURE 6.4: f x 3 x 0 x 2 1 x x x FIGURE 6.5: c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002 6.7 NEWTON’S METHOD 75 6.7 Newton’s Method The methods discussed so far have required only function values to compute the new estimate of the root. Newton’s method requires that both the function value and the first derivative be evaluated. Geometrically, Newton’s method proceeds by extending the tangent line to the function at the current point until it crosses the x-axis. The new estimate of the root is taken as the abscissa of the zero crossing. Newton’s method is defined by the following iteration: k f x k +1 k x = x : (6.7) 0 k f x The update is shown graphically in Figure 6.6. f x 2 1 0 x x x x FIGURE 6.6: Newton’s method may be derived from the Taylor series expansion of the function: 2 x 0 00 + ::: x +x=f x+f xx + f x f (6.8) 2 x For a smooth function and small values of , the function is approximated well by the first two f x f x +x=0 x = terms. Thus implies that 0 . Far from a root the higher order terms are f x significant and Newton’s method can give highly inaccurate corrections.