Math 361S Lecture Notes Numerical solution of ODEs: Part I Jeffrey Wong April 2, 2019 Topics covered • Overview ◦ Brief review of ODEs ◦ Perspective for numerical methods • Euler's method ◦ Local and global truncation error; consistency ◦ Convergence; proof of convergence • Stability ◦ Stiff differential equations ◦ Regions of absolute stability ◦ Some implicit methods (trapezoidal method, backward euler) • Runge-Kutta methods ◦ (Briefly) Taylor series method ◦ Constructing higher-order one step methods ◦ Runge-Kutta methods: the general approach ◦ Example: RK4 (fixed step size) • Implicit one step methods ◦ General notes ◦ Implementation 1 1 Overview of goals In this section of the course we will consider the solution of ordinary differential equations. The tools we have developed so far - interpolation, differentiation and integtration - will serve as a foundation for constructing the algorithms. We will also make use of some linear algebra and non-linear equations (e.g. Newton's method) to solve some sub-problems. Our main goal here twofold. First, we seek to understand the fundamentals of numerical methods for ODEs: what are the numerical issues that arise, how to recognize and correct them, and the right perspective. If you are handed a nasty ODE, how do you approach writing a method to solve it? How can you argue that the answer you obtain is correct? Second, as ODEs are a synthesis of existing techniques, we will see how pieces can be combined and how the fundamental principles (error in interpolation, numerical stability of integration and so on) show up in practice. We'll also see a few more concepts: • How stability manifests in numerical solution to ODEs (and how it connects to the mathematical notion of stability from theory) • Managing trade-offs between accuracy and stability (the consequences of sacrificing one for the other are more severe for ODEs than the other problems we have seen) • Designing adaptive algorithms that 'just work' like ode45 in Matlab (we saw a bit of this with adaptive integration, but will explore it in detail here) 2 The problem to solve 2.1 Note on background See sections 7.1-7.3 of Moler's book or any standard text on ODEs for a review of ODEs. The bare minimum will be presented here. Essentially no ODE theory is required to solve ODEs numerically, but the theory does provide important intuition, so it will greatly enhance your understanding of the numerics. 2.2 Initial value problems Here, we will consider solving an initial value problem y0(t) = f(t; y); t 2 [a; b] (2.1a) y(t0) = y0: (2.1b) The equation (2.1a) is the ODE (ordinary differential equation) for y(t) and (2.1b) is the initial condition. We seek a function y(t) that satisfies (2.1a) for all t in the given interval and whose value at t0 is y0: The ODE is first-order (the highest derivative is first-order) and scalar (y(t) is a real- valued function). Throughout, we shall assume that 2 • f(t; y) is a continuous function in t and y • f has partial derivatives of all orders required for any derivation (mostly for Taylor expansions) It is a fundamental theorem that these conditions ensure a unique solution exists, at least in some interval around (t0; y0). It is not enough to ensure that a solution exists for all t; the solution may diverge. For our purposes, we will attempt to construct numerical solutions where the actual so- lution exists, so the theory is just there to ensure that the problem to solve is well-defined. 2.3 Slope fields and sensitivity Here we review a useful geometric interpretation of the ODE y0 = f(t; y): A solution (t; y(t)) forms a curve in the (t; y) plane. The ODE tells us the direction of the curve at any given point, since y(t) is parallel to (1; y0) = (1; f): In this sense, solutions to the ODE follow the 'slope field’, which is the vector field (1; f(t; y)) in the (t; y) plane. To find a solution to the IVP starting at (t0; y0), we may follow the slope field to construct the curve; this is the basis of the simplest numerical method that is detailed in the next section. The slope field gives geometric intuition for some important concepts for numerical methods. For instance, we may ask: if y(t) gets perturbed by some amount ∆y at time t0, how far apart are the original and perturbed curves after some time? Put another way: how sensitive is the ODE to changes in the initial condition? The picture is shown in Figure 3. Suppose we have two solutions x(t) and y(t) to the same ODE, 0 y = f(t; y); y(t0) = y0; 0 x = f(t; x); x(t0) = x0: Informally, the difference z = y − x satisfies @f z0 = f(t; y) − f(t; x) = (t; ξ)z @y 3 for some ξ, by the mean value theorem. Thus the size of @f=@y determines how fast the difference can change with time. Let @f L = max (t; y) : t2[a;b];y2R @y Then we have, informally, that 0 Lt jzj ≤ Ljzj =) jzj ≤ jz0je ignoring that the absolute value cannot be manipulated this way with a derivative. Thus L (the max. of the variation of f with respect to y) is the exponential rate, at worst, at which the two solutions can move apart. However, the bound is sometimes pessimistic. Taking absolute values discards informa- tion about the sign, so if z ∼ −Lz then the bound is the same, even though z then decays exponentially. This is shown in the figure. 4 4 3 2 2 1 0 0 0 1 2 0 1 2 Figure 1: Sketch of the difference in two solutions that start at nearby points (t0; x0) and 0 0 (t0; y0) and numerical examples for y = ty and y = −ty. 4 3 Numerical methods: the basics Summary of notation in this section: • t0; ··· ; tN : the points where the approximate solution is defined • yj = y(tj): the exact solution to the IVP • y~j the approximation at tn • τj: truncation error in obtainingy ~j • h or ∆t: the `step size' tj − tj−1 (if it is constant); otherwise hj or ∆tj Here we introduce Euler's method, and the framework to be used for better numerical methods later. We seek a numerical solution to the IVP 0 y = f(t; y); y(a) = y0 and suppose we wish to solve for y(t) up to a time1 t = b. The approximation will take the form of valuesy ~j defined on a grid a = t0 < t1 < ··· < tN = b such that y~j ≈ y(tj): For convenience, denote by yj the exact solution at tj and let the `error' at each point be ej = yj − y~j: It will be assumed that we have a free choice of the tj's. The situation is sketched in Figure 2. 3.1 Euler's method Assume for simplicity that h = tj − tj−1; the `time step' size, is constant (not necessary, juts convenient). Suppose that we have the exact value of y(t). To get y(t + h) from y(t), expand in a Taylor series and use the ODE to simplify the derivatives: y(t + h) = y(t) + hy0(t) + O(h2) = y(t) + hf(t; y) + O(h2): 1Here t is just the independent variable; time is a useful analogy since it suggests the direction (forward in time) in which the ODE is to be solved. 5 Figure 2: Numerical solution of an IVP forward in time from t = a to t = b. This gives, for any point tj, the formula y(tj + h) = y(tj) + hf(tj; y(tj)) + τj+1 (3.1) where τj+1 is the local truncation error defined below. We could derive a formula, but the important thing is that 2 τj+1 = O(h ): Dropping the error in (3.1) and iterating this formula, we get Euler's method: yej+1 = yej + hf(tj; yej): (3.2) The initial point is, ideally, ye0 = y0 since the initial value (at t0) is given. However, in practice, there may be some initial error in this quantity as well. Notice that the total error is not just the sum of the truncation errors, because f is evaluated at the approximation yen. The truncation error will propagate through the iteration, as a careful analysis will show. Definition (Local truncation error, or LTE): The local truncation error is the error incurred in obtainingy ~j when the previous data (yj−1 etc.) is known exactly. It is 'local' in the sense that it does not include errors created at previous steps. Another interpretation is this: The local truncation error is what is 'left over' when the exact solution is plugged into the approximation. For Euler's method, yej+1 = yej + hf(tj; yej) 6 if we plug in y(tj) instead ofy ~j, the LHS/RHS are not equal; the difference is the local truncation error: yj+1 = yj + hf(tj; yj) + τj+1: 3.2 Convergence Suppose we use Euler's method to generate an approximation (tj; yej); j = 0; ··· ;N to the solution y(t) in the interval [a; b] (with t0 = a and tN = b). The `error' in the approximation that matters in practice is the global error E = max jyj − yjj = max jejj 0≤j≤N e 0≤j≤N where ej = yj − yej is the error at tj. This is a measure of how well the approximation yej agrees with the true solution over the whole interval.2 Our goal is to show that, given an interval [a; b], the global error has the form p max jejj = O(h ) 0≤j≤N for some integer p, the order of the approximation.