Appendix A Taylor’s Theorem A.1 Single Variable The single most important result needed to develop an asymptotic approx- imation is Taylor’s theorem. The single variable version of the theorem is below. Theorem A.1. Given a function f(x) assume that its (n + 1)st derivative (n+1) f (x) is continuous for xL < x < xR. In this case, if a and x are points in the interval (xL, xR) then 1 1 f(x) = f(a)+(x−a)f 0(a)+ (x−a)2f 00(a)+···+ (x−a)nf (n)(a)+R , 2 n! n+1 (A.1) where the remainder is 1 R = (x − a)n+1f (n+1)(η), (A.2) n+1 (n + 1)! and η is a point between a and x. There are different, but equivalent, ways to write the above result. One is 1 1 f(x + h) = f(x) + hf 0(x) + h2f 00(x) + ··· + hnf (n)(x) + R , (A.3) 2 n! n+1 The requirement here is that x and x + h are points in the interval (xL, xR). A.2 Two Variables The two-variable version of the expansion in (A.3) is M.H. Holmes, Introduction to the Foundations of Applied Mathematics, 441 Texts in Applied Mathematics 56, DOI 10.1007/978-0-387-87765-5 BM1, c Springer Science+Business Media, LLC 2009 442 A Taylor’s Theorem 1 1 f(x + h, t + k) = f(x, t) + Df(x, t) + D2f(x, t) + ··· + Dnf(x, t) + R . 2 n! n+1 (A.4) where ∂ ∂ D = h + k . ∂x ∂t Writing this out, through quadratic terms, yields f(x + h, t + k) = f(x, t) + hfx(x, t) + kft(x, t) 1 1 + h2f (x, t) + hkf (x, t) + k2f (x, t) + ··· . 2 xx xt 2 tt The subscripts in the above expression denote partial differentiation. So, for example, ∂2f f = . xt ∂x∂t It is assumed that the function f has continuous partial derivatives up through order n + 1. The above expansion can be expressed in a form similar to the one in (A.1), and the result is f(x, t) = f(a, b) + (x − a)fx(a, b) + (t − b)ft(a, b) 1 1 + (x − a)2f (a, b) + (x − a)(t − b)f (a, b) + (t − b)2f (a, b) 2 xx xt 2 tt + ··· . A.3 Multivariable Versions For more than two variables it is convenient to use vector notation. In this case (A.4) takes the form 1 1 f(x + h) = f(x) + Df(x) + D2f(x) + ··· + Dnf(x) + R , 2 n! n+1 where x = (x1, x2, ··· , xk), h = (h1, h2, ··· , hk) and D = h · ∇ ∂ ∂ ∂ = h1 + h2 + ··· + hk . ∂x1 ∂x2 ∂xk Writing this out, through quadratic terms, yields 1 f(x + h) = f(x) + h · ∇f(x) + hT Hh + ··· , 2 A.3 Multivariable Versions 443 where H is the Hessian and is given as ∂2f ∂2f ∂2f 2 ··· ∂x1 ∂x2∂x1 ∂xk∂x1 ∂2f ∂2f ∂2f ··· ∂x ∂x ∂x2 ∂x ∂x 1 2 2 k 2 H = . . .. . ∂2f ∂2f ∂2f ··· 2 ∂x1∂xk ∂x2∂xk ∂xk Taylor’s theorem can also be extended to vector functions, although the formulas are more involved. To write down the expansion through the linear terms, assume that f(x) = (f1(x), f2(x), . , fm(x)) and x = (x1, x2, . , xk). In this case, f(x + h) = f(x) + (∇f)h + ··· , where ∂f ∂f ∂f 1 1 ··· 1 ∂x1 ∂x2 ∂xk ∂f2 ∂f2 ∂f2 ··· ∂x1 ∂x2 ∂xk ∇f = . . . .. ∂f ∂f ∂f m m ··· m ∂x1 ∂x2 ∂xk Appendix B Fourier Analysis B.1 Fourier Series It is assumed here that the function f(x) is piecewise continuous for 0 ≤ x ≤ `. Recall that this means f(x) is continuous on the interval 0 ≤ x ≤ ` except at a finite number of points within the interval at which the function has a jump discontinuity. The Fourier sine series for f(x) is defined as ∞ X S(x) = βn sin(λnx), (B.1) n=1 where λn = nπ/` and 2 Z ` βn = f(x) sin(λnx)dx. (B.2) ` 0 The Fourier cosine series for f(x) is defined as ∞ 1 X C(x) = α + α cos(λ x), (B.3) 2 0 n n n=1 where 2 Z ` αn = f(x) cos(λnx)dx. (B.4) ` 0 A certain amount of smoothness is required of the function f(x) so the above series are defined. For example, f(x) must be smooth enough that the in- tegrals in (B.2) and (B.4) exist. Certainly assuming f(x) is continuous is enough for the integrals, but, unfortunately, this is not enough to guarantee that the series in (B.1) and (B.3) converge. They will converge, however, if M.H. Holmes, Introduction to the Foundations of Applied Mathematics, 445 Texts in Applied Mathematics 56, DOI 10.1007/978-0-387-87765-5 BM2, c Springer Science+Business Media, LLC 2009 446 B Fourier Analysis f(x) and f 0(x) are piecewise continuous. The question naturally arises as to what they converge to, and for this we have the following result. Theorem B.1. Assume f(x) and f 0(x) are piecewise continuous for 0 ≤ x ≤ `. On the interval 0 < x < `, the Fourier sine series, and the Fourier cosine series, converge to f(x) at points where the function is continuous, 1 and they converge to 2 (f(x+) + f(x−)) at points where the function has a jump discontinuity. At the endpoints, S(0) = S(`) = 0, while C(0) = f(0) and C(`) = f(`). When using a Fourier series to solve a differential equation one usually needs the expansion of the solution as well as its derivatives. The problem is that it is not always possible to obtain the series for f 0(x) by differentiating the series for f(x). For example, given a sine series as in (B.1) one might be tempted to conclude that ∞ 0 X S (x) = βnλn cos(λnx). n=1 The issue is that the differentiation has resulted in λn appearing in the coef- ficient. As an example, for the function 1 if 0 ≤ x ≤ 1 f(x) = 2 if 1 < x ≤ 2, one finds that 2 β λ = [1 − 2(−1)n + cos(nπ/2)] . n n ` The general term βnλn cos(λnx) of the series does not converge to zero as n → ∞, and this means that the series does not converge. Consequently, additional restrictions must be imposed on f(x) to guarantee convergence. 2 Basically what are needed are conditions that will give us βn = O(1/n ), and this brings us to the next result. Theorem B.2. Assume f(x) is continuous, with f 0(x) and f 00(x) piecewise continuous, for 0 ≤ x ≤ `. If f(x) is expanded in a cosine series then the series for f 0(x) can be found by differentiating the series for f(x). If f(x) is expanded in a sine series, and if f(0) = f(`) = 0, then the series for f 0(x) can be found by differentiating the series for f(x). The question of convergence for integration is much easier to answer. As long as the Fourier series of f(x) converges then the series for the integral of f can be found by simply integrating the series for f. B.2 Fourier Transform 447 B.2 Fourier Transform To derive the formula for the Fourier transform from the Fourier series, it is convenient to use the symmetric interval −` < x < `. Generalizing (B.2) and (B.3), the Fourier series of a continuous function f(x) is ∞ 1 X f(x) = α + [α cos(λ x) + β sin(λ x)] , 2 0 n n n n n=1 where λn = nπ/`, 1 Z ` αn = f(x) cos(λnx)dx, ` −` and 1 Z ` βn = f(x) sin(λnx)dx. ` −` 1 iθ −iθ 1 iθ −iθ By using the identities cos(θ) = 2 (e + e ) and sin(θ) = 2i (e − e ), the Fourier series can be written in exponential form as ∞ X iλnx f(x) = γne , n=−∞ where 1 Z ` −iλnx¯ γn = f(¯x)e dx.¯ 2` −` Combining these two expressions ∞ X 1 Z ` f(x) = f(¯x)eiλn(x−x¯)dx.¯ 2` n=−∞ −` The sum in the above equation is reminiscent of the Riemann sum used to π define integration. To make this more evident, let ∆λ = λn+1 −λn = ` . With this ∞ X 1 Z ` f(x) = f(¯x)eiλn(x−x¯)dx∆λ.¯ 2π n=−∞ −` The argument originally used by Fourier is that in the limit of ` → ∞, the above expression yields 1 Z ∞ Z ∞ f(x) = f(¯x)eiλ(x−x¯)dxdλ.¯ 2π −∞ −∞ Fourier then made the observation that the above equation can be written as f(x) = F −1(F(f)), where F is the Fourier transform defined in Section 7.2.5. With this, the Fourier transform was born. 448 B Fourier Analysis To say that the above derivation is heuristic would be more than generous. However, it is historically correct, and it does show the origin of the Fourier transform and its inverse. The formal proof of the derivation can be found in Weinberger [1995]. Appendix C Stochastic Differential Equations The steps used to solve the Langevin equation look routine, and the solu- tions in (4.85) and (4.86) are not particularly remarkable. However, on closer inspection, the randomness of the forcing function raises some serious mathe- matical questions. An example of R is shown in Figure 4.27 using 400 points along the t-axis.