Fourier Transforms One May Think of (5) As Arising from (1) by Replacing the Integral by a Discrete (Trapezoid) Approximation

J−1 2 DST: a = U sin(πmj/J), m = 1, 2, . . . , J − 1 (5) m J j Xj=1 J−1 Inverse DST: Uj = am sin(πmj/J), j = 1, 2, . . . , J − 1 (6) m=1 X Fourier Transforms One may think of (5) as arising from (1) by replacing the integral by a discrete (trapezoid) approximation.

Note that if one included the cases m = 0 and m = J in (5) one would ﬁnd that a0 = aJ = 0. Hence there is not reason to include these. Similarly, U0 and UJ would necessarily be zero by (6). Hence it is not possible to recover non-zero values of U0 and UJ via the inverse DST and these are excluded. In this chapter I provide a summary of various transform pairs. Notation varies in the literature and I present here a self-consistent treatment. The proof that the above pairs are inverses is easy given the orthogonality relation: J−1 πmj πm0j J sin( ) sin( ) = δ 0 (7) J J 2 mm Xj=1 Sine Transform πmj Note that m and j play exactly the same role in sin( J ) so that the discrete analog of the completeness relation (4) holds as a direct consequence of (7) by relabelling j and m: Given a function u(x) on the interval [0, `], the sine transform and its inverse are given by: J−1 πmj πmj0 J sin( ) sin( ) = δjj0 (8) ` J J 2 2 m=1 Sine Transform: am = u(x) sin(πmx/`)dx (1) X ` 0 Equations (7) and (8) are the same thing. ∞Z

Inverse Sine Transform: u(x) = am sin(πmx/`) (2) m=1 X Cosine Transform

To prove that these are indeed inverses of one another, one uses the orthogonality relation for sine functions. Given a function u(x) on the interval [0, `], the cosine transform and its inverse are given by: ` ` 0 ` 2 sin(πmx/`) sin(πm x/`)dx = δmm0 (3) Cosine Transform: am = u(x) cos(πmx/`)dx (9) 0 2 ` 0 Z Z ∞ a Inverse Cosine Transform: u(x) = 0 + a cos(πmx/`) (10) which is easy to prove by direct integration. One can also use the completeness relation: 2 m m=1 ∞ X ∞ 0 ` 0 = wmam cos(πmx/`) (11) sin(πmx/`) sin(πmx /`) = δ(x − x ) (4) 2 m=0 m=1 X X where the weight function wm is given by: 1 which is not easy to prove. 2 if j = 0 wj = (12) (1 if j > 0

The proof that these are indeed inverses uses the orthogonality relation for cosine functions. Discrete Sine Transform ` 0 ` cos(πmx/`) cos(πm x/`)dx = δ 0 (13) 2 mm Z0 There is a discrete form of the sine transformation, the discrete sine transform (DST). In this case we are which again is easy to prove by direct integration. These is also a difﬁcult to prove completeness relation: ∞ given a set of value Uj for j = 1, . . . , J − 1 generally thought of as a discrete sampling of a function 0 ` 0 u(x) on [0, `], i.e. U = u(jh), which h = `/J. For the DST the function is sampled only on the interior w cos(πmx/`) cos(πmx /`) = δ(x − x ) (14) j m 2 m=0 j = 1, 2, . . . , J − 1. Then the discrete sine transform (DST) and its inverse are given by: X 33 Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley 35 36

Discrete Cosine Transform One can also use the completeness relation:

∞ − 0 0 ei2πmx/`e i2πmx /` = `δ(x − x ) (23) For the discrete cosine transform (DST) we consider a set of value Uj for j = 0, . . . , J again generally m=−∞ thought of as a discrete sampling of a function u(x) on [0, `]. This time however we sample the function at X the end points j = 0 and j = J. Then the discrete cosine transform (DST) and its inverse are given by: which is not easy to prove.

J 2 DCT: a = w U cos(πmj/J) (15) In the case which u(x) is a real-valued function, the am will still be complex, but the following symmetry m J j j j=0 holds: X ∗ J a−m = am (24) Inverse DCT: Uj = wmam cos(πmj/J) (16) m=0 where ∗ denotes complex conjugation. Proof: X (17) ∗ ` ` ` ∗ 1 −i2πmx/` 1 ∗ i2πmx/` 1 −i2π(−m)x/` am = u(x)e dx = u (x)e dx = u(x)e dx = a−m where now the weight function is: ` ` ` Z0 Z0 Z0 1 2 if j = 0, J, wj = (18) (1 if 0 < j < J Using this fact, it is possible to re-write the FT pair in such a way that all quantities are real:

∞ A One may think of (15) as arising from (9) by replacing the integral by a trapezoid approximation. The actual Inverse Fourier Transform: u(x) = 0 + (A cos(2πmx/`) + B sin(2πmx/`)) 2 m m proof that the above pairs are inverses comes from the following identity. m=1 X J πm0j πmj J with: w w cos( ) cos( ) = δ 0 (19) j m J J 2 mm j=0 2 ` X Fourier Transform: A = 2ar = u(x) cos(2πmx/`)dx m m ` Note that as the the DST, m and j play exactly the same role in this equation so that the discrete analog of Z0 2 ` the completeness relation (14) holds as a direct consequence of (19) by relabelling j and m. B = −2ai = u(x) sin(2πmx/`)dx m m ` Z0 r i r i where am and am are the real and imaginary parts of am: am = am + iam. Fourier Transform Hence the FT of a real-valued function is equivalent to taking both sine and cosine transforms on the interval [0, `] but with argument 2πmx/` rather than πmx/`. Given a function u(x) on the interval [0, `], the Fourier transform (FT) and its inverse are given by: The details are left for the reader, but the ﬁrst few lines of the derivation are: ` 1 −i2πmx/` Fourier Transform: am = u(x)e dx (20) ∞ −∞ ` 0 i2πmx/` i2πmx/` Z∞ u(x) = a0 + ame + ame Inverse Fourier Transform: i2πmx/` m=1 m=−1 u(x) = ame (21) X∞ X∞ −∞ m= i2πmx/` −i2πmx/` X = a0 + ame + a−me m=1 m=1 In this case the function u(x) is often allowed to be complex, but we shall consider only the case of real X∞ X u(x). i2πmx/` ∗ −i2πmx/` = a0 + ame + ame m=1 One has the following orthogonality relation for complex exponentials: X

` i2πmx/` −i2πm0x/` e e dx = `δmm0 (22) Discrete Fourier Transform Z0 which as usual is easy to prove by direct integration. For the discrete Fourier transform (DFT) we again consider a set of value Uj for j = 0, . . . , J thought of as ` ∗ 0 i2πmx/` (Notice that here we have 0 φm(x)φm0 (x)dx = `δmm , where φm(x) = e .) a discrete sampling of a function u(x) on [0, `] including end points j = 0 and j = J. In this case we are Numerical methods for PDESR Copyright (C) 2002-2005 Dwight Barkley Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley 37 38 going to consider several forms for the DFT. The ﬁrst is given by: This is a non-standard form for the DFT. I give it here because it is the form one might guess from the continuous FT (20)-(21). J−1 1 −i2πmj/J DFT: am = Uje , m = −J/2 + 1, −J/2 + 2, . . . , J/2, (25) J Probably the most common form for the DFT is given by using the periodicity in am to shift the sum in (26) j=0 X and obtain: J/2 i2πmj/J J−1 Inverse DFT: Uj = ame j = 0, 1, . . . , J − 1, (26) 1 −i2πmj/J − DFT: am = Uje , m = 0, 1, . . . , J − 1, (32) m=XJ/2+1 J Xj=0 where we have assumed J is even. This is the form we shall consider most frequently in this course. J−1 i2πmj/J Inverse DFT: Uj = ame j = 0, 1, . . . , J − 1, (33) m=0 The proof that the above pairs are inverses comes from the orthogonality of the complex exponentials: X This form has the advantage that both sums are over the same range. However, it has the disadvantages that J−1 −i2πmj/J i2πm0j/J (1) it is not a natural choice given the continuous FT, (2) more importantly, we will consider real Uj and this e e = Jδmm0 (27) form does not make it as obvious that Uj obtained by the inverse transformation will be real. Xj=0 As with the other discrete transforms, this is also equivalent to the completeness relation in the continuous Most libraries that perform discrete Fourier transforms say they compute transforms of type (32)-(33). They case. are all equivalent of course.

∗ Other standard and non-standard forms of the DFT can be derived by noting that if (25) is used to deﬁne am Finally, we consider the sine/cosine forms of the DFT. As in the continuous case: Uj real implies a−m = am. for all integer m, then the am are periodic with period J. Similarly, (26) gives periodic Uj with period J. Then (26) gives: J/2 am+kJ = am and Uj+kJ = Uj i2πmj/J Uj = ame (34) − for all integer k. Proof: m=XJ/2+1 J/2−1 −(J/2−1) J−1 i2πmj/J i2πmj/J i2π(J/2)j/J 1 − = a0 + ame + ame + a e (35) a = U e i2π(m+kJ)j/J J/2 m+kJ J j m=1 m=−1 j=0 X X X J/2−1 J−1 i2πmj/J ∗ −i2πmj/J iπj 1 − − = a + a e + a e + a e (36) = U e i2πmj/J e i2πkj 0 m m J/2 J j m=1 j=0 X X J/2−1 J−1 r i 1 − = a + 2a cos(2πmj/J) + (−2a ) sin(2πmj/J) + a cos(πmj/(J/2)) (37) = U e i2πmj/J = a 0 m m J/2 J j m m=1 j=0 X X J/2−1 A AJ/2 = 0 + (A cos(2πmj/J) + B sin(2πmj/J)) + cos(πmj/(J/2)) (38) 2 m m 2 m=1 Using this, the following is an equivalent form for the DFT and its inverse: X So J 1 −i2πmj/J J/2 DFT: am = wjUje , m = −J/2, −J/2 + 1, . . . , J/2, (28) J Inverse Fourier Transform: j=0 Uj = (wmAm cos(2πmj/J) + Bm sin(2πmj/J)) X m=0 J/2 X Inverse DFT: U = wˆ a ei2πmj/J j = 0, 1, . . . , J − 1, where j m m (29) 1 m=−J/2 2 if m = 0, J/2, X wm = (39) (1 if 0 < m < J/2 where: 1 and Am and Bm are given by: 2 if j = 0, J, wj = (30) J (1 if 0 < j < J 2 Fourier Transform: A = 2ar = w U cos(2πmj/J) m m J j j and j=0 1 if m = −J/2, J/2, X wˆ = 2 J−1 m (31) i 2 (1 if −J/2 < m < J/2 Bm = −2a = Uj sin(2πmj/J) m J j=1 In effect, since U0 = UJ , U0 in (25) can been replaced by (U0 + UJ )/2. Similarly for (29) since a−J/2 = X aJ/2. Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley 39 40

Final remark on counting. It is important to realize that the DFT is simply a change of bases. In the case J of complex Uj it is a change of bases in C . In the case of real Uj, our case, it is equivalent to a change of bases in RJ . That the counting is correct for this real case (so called real-to-complex-transform) can be seen by the as follows. Referring to (25), given J real values Uj, j = 0, 1, . . . , J − 1, we obtain J complex amplitudes am, m = −J/2 + 1, −J/2 + 2, . . . , J/2. However, there is a symmetry in the am, namely ∗ a−m = am. Hence the negative half of the spectrum: a−1, a−2, . . . , a−J/2+1 is known given the positive half and thus is should not be counted. This leaves us with the J/2 + 1 values a0, a1, . . . , aJ/2. However, ∗ a0 is necessarily real because a0 = a−0 = a0. Similarly aJ/2 is necessarily real (try to show this). Hence there are precisely J distinct real quantities in the am m = −J/2 + 1, −J/2 + 2, . . . , J/2. These are r r i r i r a0, a1, a1, . . . , aJ/2−1, aJ/2−1, aJ/2.

Two and More Space Dimensions

The heat equation in two space dimensions is: ∂u = ∇2u ∂t where ∂2 ∂2 ∇2u = + u, in Cartesian coordinates ∂x2 ∂y2 ∂2 1 ∂ 1 ∂2 = + + u, in polar coordinates ∂r2 r ∂r r2 ∂θ2

The ﬁnite-difference approach to such an equation is, in principle, very similar to that used in one space dimension:

2 Discretize u and spatial operator(s): u(x, y) → Ujk, ∇ → L

∂ Discretize time: ∂t → FE, BE, CN, . . .

Solve resulting algebraic equations efﬁciently

New difﬁculties:

1. Geometry and Boundary conditions

2. Grids

3. Solving large linear systems of equations.

– Use your pencil or your favorite algebraic manipulation package. Choose a suitable function v and work out the effect of one time step on this function. Verify that the code does as expected over one time step. Often very trivial functions such as linear or constant functions can provide limited testing. These are surprisingly good at ﬁnding mistakes in boundary conditions.

These are powerful, very common, yet largely un-publicized method of testing. Comments on Testing and Software Design • Benchmarks. Test against benchmarks and other code if available. • Test parts of your program that cannot possibly be wrong. Suggestions for Testing • Leave testing code in place. You will need if more often than you think. Keep in a separate ﬁle if you wish. • Test parts of your program that cannot possibly be wrong. For testing strive for complete clarity, not cleverness. Computational cost is not an issue for testing. Produce a documented test suit which can be easily re-run frequently. • Test scaling. Even without exact solutions one can test convergence rates. Use a very high resolution solution as “exact” even if it may not be so. Software Issues • If possible test linear and nonlinear part of the code separately.

• Linear Operators. Some thought to consider in writing programs to solve PDEs. Generally straightforward but here are some hints.

– Build up your operators in pieces that can be tested independently if necessary. • Speed vs readability/maintainability. – It is usually easy to choose some test functions v and verify that, for example, (I + 4tL)V Computational speed may be sacrificed for readability, reliability, maintainability, and development converges to (1 + 4tL)v with the correct scaling. time. This is especially true during initial development. Start with good readability/maintainability. Strive for efficiency in algorithms, but do not necessarily strive for the absolute minimum number of – Verify that all linear operators are linear. − floating point operations per time step. – Verify matrix inverses. Note that (I + 4tL) and (I − 4tL) 1 are inverses of one another apart Later when computation speed becomes a real limit (particularly for ”production” runs) you can im- from the sign of 4t. A forward Euler time step of size 4t followed by a backward Euler time prove efficiency and reduce computations to the minimum number of operations. Improvement can step with (−4t) should be the identity: usually made in a working code. The slow-but-sure code then can serve as a benchmark for the − U = (I − (−4t)L) 1 (I + 4tL)U improved code. It is a good idea to comment known inefficiencies while you write code. for any U. This will verify that the explicit and implicit parts of a code are at least consistent. I strongly recommend this test. Note, speed in not simply proportional to the number of computations performed. Memory access is a major issue. • Exact solutions. • My rule of thumb: a factor of 2 in speed is not worth worrying about if it means a large sacrifice in Exact solutions are great, but even without one you can (usually) test in at least two ways. any of the other factors: readability, reliability, maintainability. – Change the problem. • Generality vs readability/maintainability. Given a nonlinear equation Do not try solve all problems with one code (especially if you write object oriented code). In particular ∂u = Lu + N (u), do not try to solve problems you will never encounter. ∂t choose a suitable test function uT for which one can analytically compute • Libraries and portability: when to use BLAS/LAPACK libraries and when to write vs self-contained codes. g = −(Lu + N (u )). T T Using local libraries (libraries specific the hardware) has the advantage of speed. Furthermore using Add this constant forcing function to the right-hand-side and solve the PDE: the libraries in general can reduce errors and also reduce debugging – there are no bugs in BLAS and you need not look there for errors. ∂u = Lu + N (u) + g. The advantage of self-contained codes written in standard C++ is that you are guaranteed portability ∂t and porting is generally easier. uT will necessarily be an exact steady solution to this equation. 43 Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley