Introduction to Finite Differences Lim |Uj − U(Xj, Tn)| = 0 H,4T→0

Also we frequently need N to represent other quantities and only seldom do we need a notation for the total number of timesteps.

n • U denotes the numerical approximation to u. Uj will denote the numerical solution at timestep n and n gridpoint j. If the numerical solution exactly agrees with the true solution, then Uj = u(xj, tn). One goal, though not the only one, of a numerical scheme is to make

n Introduction to Finite Differences lim |Uj − u(xj, tn)| = 0 h,4t→0

Explicit Euler solution Consider the heat equation on a finite interval subject to Dirichlet boundary conditions and arbitrary (i.e. user specified) initial conditions: ∂u ∂2u Basic methodology of finite-difference schemes - approximate the derivatives appearing in the partial dif- PDE: = , 0 ≤ x ≤ `, t > 0 ∂t ∂x2 ferential equation with combinations (differences) of the values on the grid. Here we consider the simplest BC: u(0, t) = γ0 and u(`, t) = γ`, t > 0 case. IC: u(x, 0) = u0(x), 0 ≤ x ≤ `. Approximate time derivative by the forward difference: Assume the γ’s are time-independent. n+1 ∂u ∂u U − U n | = (x , t ) ' j j ∂t j,n ∂t j n 4t Space-time discretization Approximate space-derivative by the second-order center difference:

2 2 2 n n n ∂ ∂ ∂ u ∂ u Uj−1 − 2Uj + Uj+1 We need a numerical representation for the function u(x, t) and the for operators ∂t and ∂x2 . | = (x , t ) ' ∂x2 j,n ∂x2 j n h2 In the simplest case the dependent variable u is represented by values on a uniform grid or lattice in space and time. The heat equation becomes: n+1 n n n n U − U U − − 2U + U j j = j 1 j j+1 4t h2

n+1 Solving for Uj : n+1 n n n n Uj = Uj + ν(Uj−1 − 2Uj + Uj+1)

where ν ≡ 4t/h2.

This is an explicit expression for the U’s at the n + 1st timestep in terms of the U’s at the nth timestep. Taking into account the BCs and IC we have the explicit, or forward, Euler scheme for the heat equation:

0 0 Uj = u (xj), 0 ≤ j ≤ J

γ0, j = 0 n+1 n n n n U = U + ν(U − − 2U + U ), 0 < j < J, j  j j 1 j j+1 • Divide the interval [0, `] into J equally spaced intervals of size 4x or h. Hence h = 4x = `/J.  γ`, j = J There are a total of J + 1 gridpoints labeled xj. For the uniform grid xj = j4x, j = 0, J  • n labels timesteps, tn denotes discrete time values, and 4t is the timestep. When needed, t0 and tf will denote the initial and ﬁnal times and N will denote the total number of time steps. Usually t0 = 0 At each new timestep, the dependent variable at each interior grid point is computed from values at three and tn = n4t. gridpoints at the preceding timestep. 1 Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley 3 4

More on ﬁnite-difference formulas Derivation of ﬁnite-difference formulas

∂ru Difference operators Suppose we want to approximate ∂xr to some order of accuracy as a weighted sum of values of u on a (possibly non-uniform) grid . . . , xj−2, xj−1, xj, xj+1, xj+2, . . .:

∂ru The following difference operators are frequently useful: (x ) ' w u ∂xr j k j+k Xk

• forward difference: where uj+k = u(xj+k). How are the weights wk chosen?

∆+tu(x, t) ≡ u(x, t + 4t) − u(x, t) The trick is to express all the uj+k = u(xj+k) as Taylor series expansions about point xj: ∆+xu(x, t) ≡ u(x + 4x, t) − u(x, t) uj = uj ∂u ∂2u (x − x )2 • backward difference: u = u + | (x − x ) + | j+1 j + . . . j+1 j ∂x j j+1 j ∂x2 j 2 2 2 ∆−tu(x, t) ≡ u(x, t) − u(x, t − 4t) ∂u ∂ u (xj−1 − xj) − − uj 1 = uj + |j(xj 1 − xj) + 2 |j + . . . ∆−xu(x, t) ≡ u(x, t) − u(x − 4x, t) ∂x ∂x 2 . . • central difference: 1 1 where |j means evaluated at xj. Multiplying each equation by corresponding weight and summing we have: δtu(x, t) ≡ u(x, t + 4t) − u(x, t − 4t) 2 2 r r ∂u ∂ u (xj+k − xj) 1 1 wkuj+k = uj wk + |j wk(xj+k − xj) + . . . r |j wk + . . . δxu(x, t) ≡ u(x + 4x, t) − u(x − 4x, t) ∂x ∂x r! 2 2 Xk Xk Xk Xk From this follows the second-order central difference: ∂ru 2 The left-hand-side is the sum that we want to approximate ∂xr . To achieve this for generic functions u, we δxu(x, t) ≡ u(x − 4x, t) − 2u(x, t) + u(x + 4x, t) ∂ru ∂ru must have all terms preceding ∂xr on the-right-side be zero and the coefﬁcient of ∂xr be one. In addition ∂ru we would like as many terms as possible following r also to be zero. This is satisﬁed by choosing the • ∂x central difference using double interval: weights such that: 1 1 ∆ u(x, t) ≡ (∆ + ∆− )u(x, t) = (u(x + 4x, t) − u(x − 4x, t)) 0x 2 +x x 2 wk = 0 Xk wk(xj+k − xj) = 0 Xk . . (x − x )r w j+k j = 1 k r! Xk (x − x )r+1 w j+k j = 0 k (r + 1)! Xk . .

These equations are linear in the weights and thus can be solved for the wk in terms of the differences xj+k − xj.

The number of gridpoints necessary to approximate a derivative depends on the order r of the derivative and the number of high-order terms in the Taylor series that one want to make zero. Frequently on uniform grids symmetries enter such that with q weights it is possible to satisfy more than q conditions.

Roundoff Error

Roundoff errors arise due to ﬁnite precision computations. The one rule with regard to roundoff is:

Error Analysis Where possible avoid subtracting nearly equal numbers. This rule implies, for example, that one should not take tan(x) for large x.

The analysis of errors in numerical schemes is important for the following reasons: Truncation Error

• It tells us where errors come from and where we should concentrate efforts to reduce errors. Truncation error T (x, t) is the error in approximating differential operators and PDEs by discrete represen- • It allows comparison of different schemes. tations such as ﬁnite differences.

• It provides a powerful basis for testing programs. For a PDE written as Fu = 0,

T (x, t) = F Au(x, t)

where F A is the approximation to the differential equation and u is an exact solution to the PDE.

Example: The truncation error for the explicit Euler scheme for the heat equation is

u(x, t + 4t) − u(x, t) u(x − h, t) − 2u(x, t) + u(x + h, t) T (x, t) = − 4t h2 ∂u ∂2u 4t ∂2u h2 ∂4u = (x, t) − (x, t) + (x, t) − (x, t) + . . . ∂t ∂x2 2 ∂t2 12 ∂x4 4t ∂2u h2 ∂4u = (x, t) − (x, t) + . . . 2 ∂t2 12 ∂x4 so 2 Th,4t(x, t) = O(4t) + O(h )

∂2u ∂4u We assume | ∂t2 | and | ∂x4 | are bounded in the space-time domain of interest. The truncation error goes to zero everywhere in the domain as 4t and h go to zero. Hence the explicit Euler approximation is consistent with the partial differential equation.

Discretization Error

n The discretization error ej at a point of the computational grid is the difference between the numerical and exact solutions:

n 0 where u and Uj satisfy the same initial conditions, i.e. Uj = u(xj, 0).

Frequently we are interested in:

f E ≡ maxj|ej|, for ﬁxed ﬁnal time tf also called the discretization error (though usage varies).

For the explicit Euler solution of the heat equation this can be bounded using the truncation error as long as ν ≤ 1/2.

f E ≤ tf T¯.

¯ n ¯ where T is a bound on the truncation error: |Tj | ≤ T .

Ef → 0, for 4t, h → 0 such that ν = 4t/h2 ≤ 1/2

The numerical solution is said to converge to the exact solution.

Our main interest is the scaling of discretization error with 4t and h:

Ef = |O(4t) + O(h2)| 1

We can investigate the final error Ef (at a fixed final time) as a function of 4t and h. To make this precise, we need a refinement path = (hi, 4ti), i = 0, 1, 2, . . . with hi and 4ti going to zero such that 2 f νi = 4ti/hi ≤ 1/2. We compute the error E at points on the refinement path and verify the scaling of Ef . In practice for testing it is better to vary only one of 4t and h while holding the other constant.

The scaling of the discretization error with 4t and h provides a valuable check on the correctness of a program.

1Note: This is slightly subtle in that the discretization error for one-step is 2nd order in 4t, i.e.

E1 ≤ 4t|O(4t) + O(h2)|

For fixed final time tf the number of timesteps N necessary to reach tf increases as 1/4t decreases (N ∼ 1/4t). This cancels the leading 4t in the one-step error. Because N ∼ 1/4t we cannot consider E = EN with fixed N but instead we consider a fixed final time and hence the notation Ef . Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley 10

We focus on λ real, but one can also consider complex λ.

Compare the exact solution with a variety of timestepping schemes

Stability and Implicit Methods

Motivation

In the analysis of discretization error in the explicit Euler method for the heat equation, we needed to assume ν = 4t/h2 ≤ 1/2 in order to obtain bounds on the error. Consider now what happens to the numerical solution using the explicit Euler method when ν = 4t/h2 > 1/2. After a relative small number of timesteps the solution develops a numerical instability which grows exponentially with the number of timesteps.

This purpose of this chapter is to understand why this happens and to introduce stable (implicit) methods to eliminate this problem.

ODEs

Most of the issues can be understood from the simplest ODE:

• Exact solution:

u(t) = u(0)eλt

• The explicit or forward Euler timestepping:

U n+1 − U n = λU n ⇒ U n+1 = (1 + λ4t)U n 4t

• The implicit or backward Euler timestepping:

U n+1 − U n 1 = λU n+1 ⇒ U n+1 = U n 4t 1 − λ4t

• The Crank-Nicolson scheme is given by an average of the explicit and implicit schemes: We make the following observations.

U n+1 − U n λU n + λU n+1 1 + (4t/2)λ = ⇒ U n+1 = U n 4t 2 1 − (4t/2)λ • All three schemes approximate the exact, exponential, relation for small 4t. • The forward Euler seems particularly bad for moderate and large 4t. Importantly note that |U n+1| > |U n| for 4t > 2. • All three schemes can be generalized to an arbitrary weighted average: • In many ways the backward Euler scheme appear to be the best because is never gets too far from the exact solution and has the correct limit for large 4t. Note, however, that for small 4t the backward U n+1 − U n 1 + (1 − θ)λ4t = (1 − θ)λU n + θλU n+1 ⇒ U n+1 = U n Euler scheme approximates the exact to the same accuracy as forward Euler. 4t 1 − θλ4t • The Crank-Nicolson scheme is a better approximation to the exact solution for small 4t than either of the other two methods. That is the Crank-Nicolson curve is seen to follow the exact solution better 0 ≤ θ ≤ 1 θ where . This is called the weighted average or -method. at small 4t. |U n+1| < |U n| for all 4t.

For small 4t expand each form in Taylor’s series to obtain: Aside on terminology: Implicit timestepping refers to the fact that U n+1 appears on the right-hand-side of this equation. Hence it is an implicit equation for U n+1, the value of u at the next timestep, in terms of U n, the known value at the current timestep. For this simple ODE it is trivial to solve for U n+1, but for general u(t + 4t) 1 1 ODES this would not be so. Exact: = eλ4t = 1 + λ4t + λ24t2 + λ34t3 + . . . u(t) 2 6 The terms forward and backward come about from a slightly different way of viewing these schemes. If U n+1 FE: = 1 + λ4t = 1 + λ4t the time appearing on the right-hand-side is taken as the reference point, then the derivative on the left-hand- U n side is a forward difference or a backward difference. U n+1 BE: = (1 − λ4t)−1 = 1 + λ4t + λ24t2 + . . . U n n+1 U 4 4 1 1 CN: = (1 − λ t )−1(1 + λ t ) = 1 + λ4t + λ24t2 + λ34t3 + . . . U n 2 2 2 4 One-step analysis Hence we see that FE and BE agree with Exact to O(4t) and makes an error O(4t2), whereas CN agrees with Exact to O(4t2) making an error O(4t3). Note that these are the single-timestep errors which are one For each of the three schemes let us consider the dependence of U n+1/U n on 4t and compare this with the power of 4t higher than the ﬁnal truncation or discretization error. exact expression for u(t + 4t)/u(t). The results are shown graphically for the case λ = −1. Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley 13 14

N-step analysis PDEs

Consider now the result of repeatedly timestepping each scheme: Notation U n = αnU 0 • Un will be the vector representing the numerical solution at timestep n:

n n n n n n n T U = (U0 , U1 , U2 , . . . , Uj , . . . , UJ−1, UJ ) This issue is, • L will be the matrix representing the ﬁnite-difference representation of a linear operator L. For now is |α| ≤ 1 whenever λ ≤ 0, i.e. are numerical solutions bounded whenever exact solutions are? ∂2 we consider L = ∂x2 and

• For BE and CN methods one can easily establish that when λ < 0, |α| ≤ 1 for 0 ≤ 4t < ∞.  . . .  ...... BE and CN are unconditionally stable because there is no requirement on 4t for stability. 1   L =  1 −2 1  h2   n  . . .  • For the FE case, however, it is evident that |α| exceeds one and thus |U | will diverge as n → ∞.  ......    This is the essence of the numerical instability illustrated at the beginning of the chapter.       The explicit Euler scheme is conditionally stable with stability limit     • I will denote the identity matrix. 2 4t ≤ −λ With this notation we can easily express the 3 timestepping schemes as follows:

Because instability occurs for α < −1, it is oscillatory in time with U n+1 and U n of opposite signs. This is one of the key signatures of numerical instabilities. Un+1 − Un FE: = L Un ⇒ Un+1 = (I + 4tL) Un 4t

n+1 n U − U − BE: = L Un+1 ⇒ Un+1 = (I − 4tL) 1 Un 4t

n+1 n n n+1 U − U L U + L U 4t − 4t CN: = ⇒ Un+1 = (I − L) 1 (I + L) Un 4t 2 2 2

All the above timestepping schemes are of the form:

n+1 −1 n n U = A− A+ U = A U . where A+ ≡ (I + (1 − θ)4tL) A− ≡ (I − θ4tL).

Stability analysis

All the above timestepping schemes are of the form:

Un+1 = A Un.

The stability analysis is reduced to determining whether the matrix A has any eigenvalues such that |α| > 1.

The von Neumann or Fourier method of stability analysis is based on choosing the eigenvectors of A to be trigonometric: φj = cos(βjh) Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley 15 16 or iβjh φj = e whichever is easier to work with. β is continuous and can take on any value. In this approach the corresponding values of α are simply computed.

The eigenvalues α of A are related to the eigenvalues λ of L:

FE: α = 1 + λ4t − BE: α = (1 − λ4t) 1 4t − 4t CN: α = (1 − λ ) 1(1 + λ ) 2 2

Find the eigenvalue of L: 1 λφ = (φ − − 2φ + φ ) j h2 j 1 j j+1 1 − λeiβjh = (eiβ(j 1)h − 2eiβjh + eiβ(j+1)h) h2 1 − λ = (e iβh − 2 + eiβh) h2 2 λ = (cos βh − 1) h2

The eigenvalues of A for the forward Euler method are: 24t α = 1 + 4tλ = 1 + (cos βh − 1) h2

Note that (cos βh − 1) ≤ 0. Hence the only possibility for instability occurs for α < −1. For any 4t, the minimum (most negative) value of α occurs for βh = π giving: 44t α = 1 − min h2

For explicit Euler scheme to be stable, αmin must be greater than −1. This is equivalent to the condition

h2 4t 1 4t ≤ or ν = ≤ 2 h2 2

• The explicit Euler scheme is conditionally stable.

• The reader can easily show that the BE and CN schemes are unconditionally stable.

Form of the numerical instability. As before the instability occurs for α < −1 and hence when instability n+1 n develops it is oscillatory in time with Uj and Uj of opposite signs. However, now there is the additional dependence on space given by the eigenvector φj = cos(βjh) (now I use the cosine form). The instability j arises ﬁrst from the eigenvector with βh = π or φj = cos(πj) = (−1) . This normal mode changes sign in space at every other gridpoint. Hence when the instability develops it is oscillatory in space as well as time. Again this is one of the key signatures of a numerical instability.

Next deﬁne the boundary operator Bn by:

n U0 γ0 U1 U1     Bn U = Bn = . . . . . .                 Boundary Conditions Bn sets the boundary values to the corresponding BC at time t = n4t. Note: B is not a linear operator if γ0 6= 0.

Using this operator the three timestepping schemes can be written: We shall consider the following boundary conditions on the interval [0, `]: ∂u FE: Un+1 = Bn+1 (I + 4tL) Un α (t)u(0, t) + β (t) (0, t) = γ (t), α , β ≥ 0 0 0 ∂x 0 0 0 ∂u − α (t)u(l, t) + β (t) (l, t) = γ (t), α , β ≥ 0 BE: Un+1 = (I − 4tL) 1 Bn+1 Un ` ` ∂x ` ` `

These boundary conditions are Dirichlet if β = 0 and Neumann if α = 0 and Robin if both α and β are 4t − 4t CN: Un+1 = (I − L) 1 Bn+1 (I + L) Un nonzero. 2 2

We consider methods for dealing with these cases which have the advantage that the same J + 1 spatial points xj with x0 = 0 and xJ = ` are used in each case. Neumann/Robin case: β = 0 Computationally for (1) and (1) there are two distinct cases: β = 0 and β 6= 0. 6

Without loss of generality we can write the left BC as: = 0 Dirichlet case: β ∂u (0, t) = α (t)u(0, t) + γ (t) ∂x 0 0

We consider in detail the left boundary only, the right boundary follows similarly. since β0 6= 0.

n n Without loss of generality we can write the left BC as: In this case we do not know U0 directly from the boundary condition, and must ﬁnd U0 by imposing the PDE at the boundary. u(0, t) = γ0(t) ∂2u However, to use the second-order centered-difference approximation for 2 at grid point j = 0 would since α0 cannot be zero if β0 is. ∂x require “U−1”, i.e. 2 n n n δx 1 Eq. () dictates the numerical solution at x = 0 for all time: U = γ ≡ γ0(n4t). Thus U does not need 0 0 0 2 U0 = 2 (U−1 − 2U0 + U1) to be computed from the PDE and and we need not (and cannot) impose the PDE at j = 0. h h and U−1 does not exist. Here we consider a heuristic approach in which () is used where U−1 is obtained The following provides a simple implementation. Consider the heat equation and let L be the discretization from boundary condition (). of the second derivative. Put zeros in the rows of matrix L at Dirichlet boundary points To obtain U−1 from the BC use the second-order centered-difference representation for the ﬁrst derivative 0 0 at the boundary: 1 −2 1 ∂u 1 2   (0, t) = (U − U− ) + O(h ) 1 1 −2 1 ∂x 2h 1 1 L = 2  .  h  ..  Plugging this into the boundary condition () gives:       1   (U1 − U−1) = α0U0 + γ0   2h   2 So [LU]0 = 0, [LU]1 = (U0 − 2U1 + U2)/h . You may think of this as not imposing the PDE at the Solving for U−1: boundary. U−1 = U1 − 2hα0U0 − 2hγ0 17 Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley 19 20

Then Periodic BCs

2 δx 1 2 U0 = 2 (U1 − 2hα0U0 − 2hγ0 − 2U0 + U1) h h Consider periodic boundary conditions imposed on the grid with x0 = 0 and xJ = `. Then u(0, t) = u(`, t) 1 2 n n = (−2(1 + hαn)U n + 2U n) − γn gives U0 = UJ . The PDE is easily imposed at all grid points. For example, h2 0 0 1 h 0 2 ∂ u 1 n n n 2 (0, t) = (U− − 2U + U ) + O(h ) ∂x2 h2 1 0 1 Ln We deﬁne by: 1 n n n 2 n = 2 (UJ−1 − 2U0 + U1 ) + O(h ) −2(1 + hα0 ) 2 h 1 −2 1   1 1 −2 1 Ln Thus the matrix L has the structure: = 2  .  h  ..            1   n   L = and deﬁne the boundary operator B in this case to be: h2       −24t n   U0 U0 γ   h 0   U1 U1 0         and is not tridiagonal, nor is it banded. For implicit methods this is a problem because the work required to Bn U = Bn = + . . . . . .  . . .  invert A− = (I − 4tL) is proportional to the square of the bandwidth.                         However, A− is sparse – it contains mostly zero entries and it differs in only 6 places from a tridiagonal             matrix. To invert such a matrix, one can turn to the Sherman-Morrison formula. n n n n where α0 ≡ α0(n4t), and as before γ0 = γ0(n4t). L and B depend on n (time) if α and γ do. If The three timestepping schemes are then: Aˆ = A + u ⊗ v then −1 −1 ˆ −1 −1 (A · u) ⊗ (v · A ) A = A − − FE: Un+1 = Bn (I + 4tLn) Un v · A 1 · u

− ˆ −1 BE: Un+1 = (I − 4tLn+1) 1 Bn+1 Un If u and v are unit vectors, then A differs from A in exactly one entry. Hence, A− can be written in terms of the inverse of a tridiagonal matrix together with 6 applications of the Sherman-Morrison formula.

4t − 4t CN: Un+1 = (I − Ln+1) 1 Bn+1/2 (I + Ln) Un 2 2

where Bn+1/2 is deﬁned by:

−4t n n+1 U0 h (γ0 + γ0 ) U1 0 1     Bn+1/2 U = (Bn+1 + Bn) U = + 2 . . .  . . .                         

We will frequently write B for the appropriately deﬁned boundary condition operator. The forms of this operator, and the timestep n at which it is evaluated depends on the boundary conditions and the method under consideration. Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley 22

General Linear Parabolic Eq in 1D

Most of the difﬁculties encountered in going to the general case are in the analysis of the discretization error and stability. Implementing the general case can be a simple extension of the constant coefﬁcient heat equation.

Write the PDE as: ∂u (x, t) = L(x, t)u(x, t) + g(x, t) ∂t then discretize all quantities in space and time:

n n U0 g0 U n gn  1   1  u(x, t) → Un = L(x, t) → Ln g(x, t) → gn = . . . . . .             U n gn   J   J      n where gj ≡ g(jh, n4t).

n ∂2u ∂u The matrix L can be obtained from ﬁnite-difference formulas for ∂x2 and ∂x .

The three timestepping schemes are then:

FE: Un+1 = B {(I + 4tLn) Un + 4tgn}

− BE: Un+1 = (I − 4tLn+1) 1 B Un + 4tgn+1 4t − 4t 4t CN: Un+1 = (I − Ln+1) 1 B (I + Ln) Un + (gn + gn+1) 2 2 2 where B represents the appropriate boundary operator.

• Implicit methods are much more difﬁcult to apply to nonlinear terms. Consider timestepping ∂u = F (u) ∂t by implicit Euler method where F () is a nonlinear operator. Then:

Un+1 − Un = F(Un+1) ⇒ (I − 4tF)(Un+1) = Un. Nonlinearity 4t

Here (I − 4tF) is a nonlinear operator. Solving this equation is not simply a matrix inversion. Implicit methods are not often used in treating nonlinearities in PDEs. Generally some sacrifices must be made in order of accuracy and/or stability. Difficulties • Relatively smooth initial conditions evolve into solutions requiring very fine spatial resolution. For example, solutions to Burgers equation Some of the numerical difficulties encountered with nonlinear equations are: ∂u ∂u ∂2u = −u + ∂t ∂x ∂x2 can become locally very sharp.

• Testing is difﬁcult because exact solutions are rare.

Some possible methods

• Explicit Euler method can always be used:

Un+1 = Un + 4tF(Un).

While not favored because it is low-order accurate (O(4t)), it is nevertheless a good method for getting results quickly in a few cases.

• Bite the bullet and invert nonlinear operator, iteratively.

• Use Runge-Kutta or a multistep method.

We will consider one of the most common methods for treating nonlinear terms in PDEs. The nonlinear terms are treated explicitly, but more accurately than with the forward Euler method.

Multistep Methods for Nonlinear ODEs

Consider multistep methods for solving the nonlinear ODE:

du = f(u). dt

These schemes advance solution for time t to time t + 4t using not only f(U n) and (possibly) f(U n+1), − − but also f(U n 1), f(U n 2), . . . 23 Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley 25 26

n Generally: where Nj(U ) is the numerical approximation to N (u) at grid point j. k n+1 n n+1−i U = U + 4t βif(U ) Solving for Un+1 we obtain: Xi=0 4t − 4t 4t − Un+1 = (I − L) 1 (I + L)Un + 3N(Un) − N(Un 1) n+1 2 2 2 If β0 = 0 then f(U ) is not used and the method is explicit. Of these the Adams-Bashforth schemes are the most common. With boundary conditions properly accounted for and with the possibility of an inhomogeneous term in the equation, this can be written: Some cases are:

k = 1, β1 = 1 explicit Euler 3 1 k = 2, β = , β = − 2nd order Adams − Bashforth 4t − 4t 4t − 4t 1 2 2 2 Un+1 = (I − Ln+1) 1 B (I + Ln) Un + 3N(Un) − N(Un 1) + (gn + gn+1) 55 59 37 9 2 2 2 2 k = 4, β1 = 24 , β2 = − 24 , β3 = 24 , β4 = − 24 4th order Adams − Bashforth

In detail the second order Adams-Bashforth scheme is: where B is the boundary condition operator appropriate to the type of BC applied.

4t − f 2 U n+1 = U n + 3f(U n) − f(U n 1) The contribution to the global error E from time discretization is O(4t ). 2 The one-step error is O(4t3) and discretization error is one power of 4t smaller, so E f = O(4t2)). The scheme is not unconditionally stable. There is a maximum 4t, determined from N and U. For most parabolic PDEs this restriction is less severe than the restriction 4t/h2 ≤ 1/2. Comments:

• The Adams-Bashforth methods give higher-order accuracy in time and yet are explicit.

• Numerical stability is an issue and for large k the timestep restriction is too severe for the schemes to be practical.

• By saving previous values of f(U n), one need only evaluate f once per timestep. This is signiﬁcant for PDEs and is a signiﬁcant advantage over other high-order methods such as Runge-Kutta in which multiple function evaluations are required at each timestep.

• The ﬁrst timestep(s) cannot be multistep. Some number of steps must be made at lower-order accuracy or using another method, such as Runge-Kutta. When using second-order Adams-Bashforth, it is common to take one explicit Euler step at the beginning. This has no importance for the global discretizaton error.

Nonlinear PDEs

Write the PDE as: ∂u = L · u + N (u) ∂t where L and N are linear and nonlinear operators respectively.

One can then treat the linear operator using Crank-Nicolson as before and treat the nonlinear operator using 2nd order Adams-Bashforth. This has the advantage of achieving overall O(4t2) accuracy and allowing reasonable size for 4t. In detail:

For a given number of grid points N = J we have h = O(1/N) so − Ef = O(N 2)

Comments

Introduction to Spectral Methods • The accuracy of spectral methods translates into fewer unknowns and thus greater speed and less memory for the same accuracy compared with FD methods. • Given fast transforms, implicit methods can be efficiently implemented. • Spectral methods typically produce smaller artificial dissipation and dispersion in comparison to FD The essential idea is to approximate the solution u(x, t) by an expansion in a finite set of spectral modes methods. φm(x) and their time-dependent amplitudes am(t): • Finite-difference and finite-element methods are generally more flexible. They are able to handle M variable coefficients, general boundary conditions, free-boundaries etc. In higher dimensions they are u(x, t) ' am(t)φm(x) able to handle irregular geometries more easily than spectral methods. m=0 X Trigonometric functions and orthogonal polynomials (Chebyshev or Legendre polynomials) are the most frequently used. The most natural indexing of modes is not always from 0 to M, and so the summation limits will often not be given, but it is understood to be finite. Fourier Pseudospectral Timestepping

Spatial derivatives of u(x, t) are then expressed in terms of spatial derivatives of the φ , e.g. m Pseudospectral methods can be applied to PDEs with inhomogeneities and nonlinearities. In the pseu- 2 ∂ u 00 dospectral approach we have both a spatial grid (collocation points) and a spectral expansion. The numerical (x, t) ' am(t)φ (x) ∂x2 m solution can either be represented by the amplitudes am or by the values on the grid Uj. Operations such as m X differentiation are best carried out in spectral space (because the modes φm(x) can be differentiated exactly and the derivative operator often local in spectral space). Complications arising from inhomogeneities and iβmx So in the trigonometric case φm = e nonlinearities are best handled in physical space where they are local. ∂2u (x, t) ' −β2a (t)φ (x) We consider in detail how to implement pseudospectral timestepping for problems with periodic boundary ∂x2 m m m conditions for which Fourier modes are the appropriate choice. X and we have “exactly” differentiated our approximation.

Amplitudes and Grid values • Orthogonal functions are to spectral methods as Taylor series are to ﬁnite-difference methods.

Let U denote the vector of values on a uniform grid xj = jh where h = `/J. Let a denote the vector of The treatment of time is the generally the same in both methods. amplitudes. The spectral expansion relates these

J−1 i2πmj/J Advantages and Disadvantages Uj = ame j = 0, 1, . . . , J − 1, m=0 X

The greatest advantage of spectral over ﬁnite difference methods is the greater spatial accuracy. For smooth Hence solutions, the discretization error of a spectral method decreases faster than any power of the resolution M:

f −k E = O(M ) for every k. J−1 1 − a = U e i2πmj/J , m = 0, 1, . . . , J − 1, The approximation is said to be exponentially accurate. m J j Xj=0 Recall, for second-order ﬁnite differences we found: Thus the amplitudes and grid values are simply related by the Discrete Fourier Transform (DFT) which shall Ef = O(h2) be written: 27 Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley 29 30

We can write our timestepping scheme as:

a = DFT U −1 ˆ −1 U = a DFT n A+ n 1 DFT n 1 DFT Un −→ a −→ a + −→ U +

There are a variety of conventions for ordering the Fourier amplitudes am because we are generally inter- r i Backward Euler and Crank-Nicolson methods can be treated in exactly the same way. Note for example that ested in real-to-complex DFTs taking J real values Uj on a grid to J complex amplitude am = am + iam. ∗ for backward Euler: The property am = a−m implies that the complex amplitudes are actually described by J real quantities − − and the highest spatial frequency is J/2. Un+1 = DFT 1 (I − 4tLˆ) 1 DFT Un − −1 1 n r i = DFT Aˆ− DFT U We use a generic ordering (storage) for am and am and let a denote the real vector: r r i r i r i r T a = (a0, a1, a1, a2, a2, . . . , aJ/2−1, aJ/2−1, aJ/2) where

2 i i 1 + 4tβ0 Necessarily a0 = aJ/2 = 0 and these are not included in the list. a is a real vector of length J. ..  .  2 In practice proper account must be taken of the ordering actually produced by the discrete Fourier transform 1 + 4tβm 2 Aˆ− ≡   = diag{1 + 4tβ } function used. The amplitudes are generally not normalized and require an additional normalization (such  1 + 4tβ2  m  m  as division by J).  ..   .     1 + 4tβ2   J/2    Heat Equation −1 Since Aˆ− is diagonal, ﬁnding its inverse is trivial: [Aˆ− ]jj = 1/Aˆ −jj. Consider timestepping the heat equation Crank-Nicolson timestepping follows similarly. ∂u ∂2u = Lu = ∂t ∂x2 Note that L, A+, etc. are diagonal only when the boundary conditions are periodic, permitting the use of by the forward Euler method Fourier expansions. For Dirichlet boundary conditions, for example, polynomial expansions are appropriate; ∂u Un+1 − Un ' = LUn the resulting matrices representing differential operators are banded or structured, but not diagonal. ∂t 4t then as usual this becomes: Nonlinearity in the Pseudospectral Approach n+1 n n U = (I + 4tL)U = A+U Consider the PDE: ∂u L and A+ are not the tridiagonal matrices from the ﬁnite-difference approach. = Lu + N (u) ∂t The operators are applied in spectral space, with L and N () linear and nonlinear operators, respectively. (Here we shall assume that N () does not ∂u ∂2u − depend on , 2 , etc. although this assumption in not essential.) Un+1 = DFT 1 (I + 4tLˆ) DFT Un ∂x ∂x −1 n = DFT Aˆ+ DFT U Discretize space and time in the usual way and timestep with Crank-Nicolson/Adams-Bashforth method:

n+1 n where they are diagonal U − U 1 1 − = LUn+1 + LUn + 3N(Un) − N(Un 1) 1 − 4tβ2 4t 2 2 0 .. Un  .  where Nj( ) is the numerical approximation to N (u) at grid point j. 2 ˆ 1 − 4tβm 2 A+ ≡  2  = diag{1 − 4tβm} Un+1  1 − 4tβm  Solving for we obtain:  .   ..    n+1 4t −1 4t n 4t n n−1  2  U = (I − L) (I + L)U + 3N(U ) − N(U )  1 − 4tβ   J/2  2 2 2   Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley 31 32

We now wish to act with the linear operators in spectral space. Apply discrete Fourier Transform and its inverse to the right-hand-side to obtain:

− 4t − 4t 4t − Un+1 = DFT 1 DFT (I − L) 1 (I + L)Un + 3N(Un) − N(Un 1) 2 2 2

− 4t − 4t 4t − Un+1 = DFT 1 (I − Lˆ) 1 (I + Lˆ) DFT Un + DFT 3N(Un) − N(Un 1) 2 2 2 where hats denote linear operators in spectral space.