UNIVERSITY OF CAMBRIDGE CENTRE FOR MATHEMATICAL SCIENCES DEPARTMENT OF APPLIED MATHEMATICS &THEORETICAL PHYSICS Numerical analysis and orthogonal polynomials CHAPTER 2: Orthogonal expansions Arieh Iserles OPTRIM, Madrid October 2017 1 List of topics: 2.1 A primer of spectral methods 2.2 Fast Legendre Transform via hypergeometric identities 2.3 Connection coefficients 2.4 Fast Legendre Transform via perturbed Chebyshev grid 2.5 Fast Multipole Method (FMM) 2.6 FMM applied to fast transforms 2 2.1 A primer of spectral methods Classical methods for differential equations: Finite differences: Discretise your PDE on a grid, approximating deriva- tives by finite linear combinations of nodal values. Finite elements: Approximate the solution as a linear combination of func- tions with small support and either minimise a variational functional (the Ritz method) or determine a projection of the weak solution (the Galerkin method). The end-product in each case is a large system (linear or nonlinear) of sparse algebraic equations: solving this system represents the real nu- merical price tag of a method. An alternative paradigm: Expand the solution in an orthogonal system so that convergence occurs very rapidly. Although the outcome is a dense algebraic system, typically it is considerably smaller. 3 ∗ We seek an orthogonal basis fφng of the ambient space s.t. 1. The expansion coefficients Z b f(x)φm(x) dx f^ = a m b Z 2 φm(x) dx a converge rapidly; 2. The first n coefficients f^m can be evaluated rapidly: this typically means in O(n(log n)q) operations for some small q. 3. Let f^ 2 be the vector of the coefficients. We say that D is a differ- C 0 entiation matrix if f^ = Df^. It need be known explicitly, it should be cheap to form Dv for a vector v and, in an ideal world, we want D to be skew-Hermitian. ∗We assume for simplicity that the ambient space is L(a; b). 4 4. The basis functions fφng match required boundary conditions (e.g. Dirichlet, Neumann or periodic). Example I The diffusion equation ut = uxx for −1 ≤ x ≤ 1, t ≥ 0, with periodic boundary conditions. iπnx We choose the Fourier basis φn = e , n 2 Z – in other words, repre- sent N X iπnx u(x; t) ≈ u^n(t)e n=−N+1 0 2 2 for some N 1. Trivially, we end up with u^n = −π n u^n, in other words with N X iπnx−π2n2t u(x; t) ≈ αne ; m=−N+1 where the αns are given by the Fourier expansion of the initial condition – this can be done in O(N log N) operations by Fast Fourier Transform (FFT). 5 In this case the spectral method is nothing else but separation of variables, although in general this need not be true. Let us examine points 1–4: 1. The αns converge for a smooth function at a spectral speed (i.e., their size is asymp- totically smaller than a reciprocal of any polynomial in jnj). This means that we need a fairly small N. 2. αn for −N + 1 ≤ n ≤ N can be calculated in O(N log N) flops by FFT. 0 3. The differentiation matrix D is diagonal and skew-Hermitian, Dn;n = −iπnDn;n. 4. Periodicity is matched by the basis functions. Example II The diffusion equation ut = uxx for −1 ≤ x ≤ 1, t ≥ 0, with Dirichlet boundary conditions. 1 The obvious choice is a Chebyshev basis fTngn=0, whereby 1 Z π 2 Z π u^0 = f(cos θ) dθ; u^n = f(cos θ) cos nθ dθ; n 2 N: π 0 π 0 6 n−1 n 0 X 0 X T2n = 4n T2`+1; T2n+1 = (2n + 1)T0 + 2(2n + 1) T2` `=0 `=1 implies n−1 00 3 X 2 2 T2n =4 n T0 + 8n (n − k )T2k; k=1 n−1 00 X T2n+1 = 4(2n + 1) (n − k)(n + k + 1)T2k+1: k=0 PN Consequently, letting u(x; t) ≈ n=0 u^n(t)Tn(x), we have bN=2c 0 X 3 u^0 =4 k u^2k; k=0 bN=2c 0 X 2 2 u^2n =8 k(k − n )^u2k; n ≥ 1; k=n+1 bN=2c−1 0 X u^2n+1 =4 (2k + 1)(k − n)(k + n + 1)^u2k+1: k=n+1 7 A checklist: 1. An expansion of a smooth function in Chebyshev polynomials converges at spectral speed. 2. Chebyshev expansion of initial condition can be computed with Fast Cosine Transform in O(N log N) operations. 3. The differentiation matrix of any polynomial basis is necessarily lower-triangular. A naive product by a vector is expensive and, worse, lower triangular is as far from skew- Hermitian as they come – and this is bad news for numerical stability – we might be compelled to solve the ODE using excessively small time steps! 4. We can match Dirichlet (or Neumann, Robin, . ) boundary conditions with Chebyshev polynomials. Point 3 is a serious impediment toward use of Chebyshev spectral methods for parabolic PDEs (although there are clever ways to overcome it), but neither for elliptic not hyperbolic PDEs. 8 Example III Consider the spectral problem for a Fredholm operator, Z 1 f(x)K(y − x) dx = λf(y); −1 ≤ y ≤ 1: −1 This is a compact operator hence it has a point spectrum with at most a single accumulation point at the origin. The finite section method approxi- mates 1 X f(x) ≈ αnφn(x); n=0 1 where fφngn=0 is an orthonormal basis of L[−1; 1]. It is easy to prove that 1 X Z 1 Z 1 αn φn(x)φm(y)K(y − x) dx dy = λαm; m 2 Z+; 1 1 n=0 − − i.e. the algebraic eigenvalue problem A[1]α = λα, where [1] Z 1 Z 1 Am;n = φn(x)φm(y)K(y − x) dx dy; m; n 2 Z+: −1 −1 9 Denoting by A[N] the (N + 1) × (N + 1) section of A[1] (i.e., restricting the range of m; n to 0; : : : ; n) we obtain the finite-dimensional eigenvalue problem A[N]α[N] = λα[N], which can be solved by standard software. Clearly, the faster the convergence, the smaller we need to take N, the smaller the eigen- value problem. Which basis to choose? In this case the nature of the basis is dictated by the problem itself: we need to expand in Legendre polynomials. 1. The bivariate expansion converges at a spectral speed. 2. Just now we have no fast way of computing a Fast Legendre Transform but watch this space! 3. A differentiation matrix is irrelevant in this setting. 4. Using Legendre polynomials, we have no problems with boundary con- ditions. 10 Speed of convergence 2 We will examine convergence for two functions: e−x and ecos πx, both in [−1; 1]: note that the second is periodic. e−x2 : N = 16 N = 32 N = 64 Fourier N = 128 N = 256 Chebyshev log P f^ − f log jf^ j 11 10 jn|≤N n n 10 n ecos πx: N = 16 N = 32 N = 64 Fourier N = 128 N = 256 Chebyshev P ^ ^ log10 jn|≤N fn n − f log10 jfnj 12 Comments: 2 A. Once e−x , which is not periodic, is approximated by Fourier, the error decays like O jNj−1 : very slowly. Otherwise, in all instances the error decays spectrally. B. Once the error decays spectrally, we need a fairly small number of terms – the outcome is a very small algebraic system. Our presentation is exclusively for linear problems but spectral methods can deal with nonlinear equations, e.g. by exploiting the identities 1 0 1 1 X X inx f(x)g(x) = @ f^m^gn−mAe (convolution), n=−∞ m=−∞ 1 TmTn = (T + T ): 2 jm−nj m+n They can also be easily generalised by tensor products to parallelepipeds.∗ ∗ m More exotic domains in R , m ≥ 2, are much more complicated. 13 2.2 Fast Legendre Transform via hypergeometric identities We can do Chebyshev expansions (of the first and the second kind) fast – what about Legendre polynomials (and, with greater generality, Jacobi polynomials)? There are several algorithms and the first is based on a very strange hypergeometric identity. bn=2c xn 1 2n + 1 − 4m Step I: = X P (x): ! 2n 3 n−2m n m=0 m!(2)n−m Step II: Let f be analytic within the Bernstein ellipse n1 iθ −1 −iθ o Br = 2(re + r e ): −π ≤ θ ≤ π ; r 2 (0; 1): Then 1 (n+2m) 1 X f (0) f^n = (n + ) ; n 2 : 2 n+2m 1 Z+ 2 m!( )n+m+1 m=0 2 14 Step III: If γ is a simple, closed, positively oriented Jordan curve in Br n [−1; 1], then by the Cauchy integral theorem Z ( ) ^ cn f z fn = 'n(z) dz; n 2 Z+; 2πi γ zn+1 where " # n! n+1; n+2; 1 cn = ;'n(z) = F 2 2 : n 1 2 1 + 3; 2 2 (2)n n 2 z Step IV: 'n is a slowly-convergent Taylor series. Instead use the nonlin- ear transformation " 1 # " # a; a + ; 2 1 −2a 2a; 2a − c + 1; ζ 2F1 2 2ζ − ζ = (1 − ζ) 2F1 ; c; 2 c; 2 − ζ where Re ζ < 1. 15 1 3 Set a = 2(n + 1), c = n + 2, whereby 1 " 1 # 2 −1=2 n + 1; 2; ζ 'n((2ζ − ζ ) ) = F : 1 n+12 1 n + 3; 2 (1 − 2ζ) 2 − ζ Choose γ as Bρ, ρ 2 (r; 1) (therefore Bρ lives inside Br).