A fast and well-conditioned spectral method: The US method Alex Townsend University of Oxford Leslie Fox Prize, 24th of June 2013 Partially based on: S. Olver & T., A fast and well-conditioned spectral method, to appear in SIAM Review, (2013). Alex Townsend, University of Oxford 24th of June 2013 0/26 The problem Problem: Solve linear ordinary differential equations (ODEs) d N u du aN (x) + ··· + a1(x) + a0(x)u = f (x); x 2 [−1; 1] dx N dx R 1 with K linear boundary conditions (Dirichlet, Neumann, −1 u = c, etc.) Assumptions: a0;:::; aN ; f are continuous with bounded variation. The leading term aN (x) is non-zero on the interval. The solution exists and is unique. Main philosophy: Replace a0;:::; aN ; f by polynomial approximations, and solve for a polynomial approximation for u. The spectral method finds the coefficients in a Chebyshev series in O(m2n) operations: n−1 X −1 u(x) ≈ αk Tk (x); Tk (x) = cos(k cos x): k=0 Alex Townsend, University of Oxford 24th of June 2013 1/26 Overview 6 4 degree(u) = 51511 2 u(x) Conventional wisdom 0 A comparison −2 −4 First-order ODEs −1 −0.5 0 0.5 1 x Higher-order ODEs 1 Stability and convergence Fast linear algebra 0.5 PDEs y 0 Conclusion −0.5 −1 −1 −0.5 0 0.5 1 x Alex Townsend, University of Oxford 24th of June 2013 2/26 Quotes about spectral methods \It is known that matrices generated by spectral methods can be dense and ill-conditioned." [Chen 2005] \The idea behind spectral methods is to take this process to the limit, at least in principle, and work with a differentiation formula of infinite order and infinite bandwidth, i.e., a dense matrix." [Trefethen 2000] \The best advice is still this: use pseudospectral (collocation) methods instead of spectral (coefficient), and use Fourier series and Chebyshev polynomials in preference to more exotic functions." [Boyd 2003] The Ultraspherical spectral method (US method) can result in almost banded, well-conditioned linear systems. Alex Townsend, University of Oxford 24th of June 2013 3/26 Chebyshev spectral methods u00(x) + cos(x)u(x) = 0; u(±1) = 0: Collocation Coefficients US method 1010 1010 4 Condition 105 105 3 number 100 100 2 0 50 100 150 200 0 50 100 150 200 0 50 100 150 200 Density Collocation Coefficient US method Matrix structure Dense Banded from below Banded + rank-K Condition number O(n2N ) O(n2N ) O(n2(D−1)) Problem sizes n ≤ 5;000 n ≤ 5;000 n ≤ 70;000 Operation count O(n3) O(mn2) O(m2n) Alex Townsend, University of Oxford 24th of June 2013 4/26 First-order differential equations Differentiation Multiplication Operator Operator u′(x) + a(x)u(x) = f(x) (D + SM[a]) u = Sf Jacobi with w(x)=(1−x)α(1+x)β 1.5 D : T ! C(1) 1 (1) 0.5 C M[a]: T ! T β 0 L Differentiation −0.5 Conversion Jacobi Hypergeometric functions T (1) Ultraspherical S : T ! C −1 Hypergeometric functions −1.5 −1.5 −1 −0.5 0 0.5 1 1.5 α Alex Townsend, University of Oxford 24th of June 2013 5/26 First-order operators Differentiation operator: D : T ! C(1), [DLMF] 00 1 1 ( (1) 2 dTk kCk−1; k ≥ 1; B C = D = B 3 C : dx 0; k = 0; B C @ : A :: Conversion operator: S : T ! C(1), [DLMF] 0 1 1 8 1 (1) (1) 1 0 − 2 C − C ; k ≥ 2; 1 1 > 2 k k−2 0 − <> B 2 2 C 1 (1) B C Tk = S = B 1 :: C : 2 C1 ; k = 1; 0 : > B 2 C :> (1) @ : : A C0 ; k = 0; :: :: Multiplication operator: M[a]: T ! T, [DLMF] 1 T T = T + T ; j; k ≥ 0: j k 2 jj−kj j+k Alex Townsend, University of Oxford 24th of June 2013 6/26 First-order multiplication operator 1 1 T T = T + T j k 2 jj−kj 2 j+k 0 1 0 1 2a0 a1 a2 a3 ::: 0 0 0 0 ::: B :: C B ::C B a1 2a0 a1 a2 :C Ba1 a2 a3 a4 : C B C B C 1 B :: C 1 B ::C M[a] = B a2 a1 2a0 a1 :C + Ba2 a3 a4 a5 : C 2 B C 2 B C B :: C B ::C B a3 a2 a1 2a0 :C Ba3 a4 a5 a6 : C @ : : : : : A @ : : : :: A : :: :: :: :: : :: :: :: :: | {z } | {z } Toeplitz Hankel + rank-1 Multiplication is not a dense operator in finite precision. It is m-banded: 1 m X X a(x) = ak Tk (x) = ~ak Tk (x) + O(); k=0 k=0 where ~ak are aliased Chebyshev coefficients. Alex Townsend, University of Oxford 24th of June 2013 7/26 Imposing boundary conditions Dirichlet: u(−1) = c: k B = (1; −1; 1; −1;:::; ) ; Tk (−1) = (−1) : Neumann: u0(1) = c: 2 B = (0; 1; 4; 9;:::; ) ; Tk (1) = k : R 1 0 Other conditions: −1 u = c, u(0) = c, or u(0) + u (π=6) = c. Imposing boundary conditions by boundary bordering: u0(x)+a(x)u(x) = f ; B c = : Bu = c; D + SM[a] f The finite section can be done exactly by projection Pn = (In ; 0): T ! ! BPn c T T T = : Pn−1DPn + Pn−1SPn+mPn+mM[a]Pn Pn−1f Alex Townsend, University of Oxford 24th of June 2013 8/26 First-order examples Example 1 u0(x) + x 3u(x) = 100 sin(20;000x 2); u(−1) = 0: The exact solution is 4 Z x 4 − x t 2 u(x) = e 4 100e 4 sin(20;000t )dt : −1 1.2 1 N = ultraop(@(x,u) ··· ); 0.8 N.lbc = 0; u = N n f; degree(u) = 20391 0.6 Adaptively selects the u(x) 0.4 discretisation size. time = 15.5s 0.2 Forms a chebfun object 0 [Chebfun V4.2]. −0.2 −15 −1 −0.5 0 0.5 1 ku~ − uk = 1:5 × 10 . x 1 Alex Townsend, University of Oxford 24th of June 2013 9/26 First-order examples Example 2 1 u0(x) + u(x) = 0; u(−1) = 1: 1 + 50;000x 2 The exact solution with a = 50;000 is p p tan−1( ax) + tan−1( a) u(x) = exp − p : a −5 1.005 10 1 −10 degree(u) = 5093 10 0.995 u(x) −15 Absolute error 10 0.99 Old chebop Preconditioned collocation −20 US method 0.985 10 −1 −0.5 0 0.5 1 −1 −0.5 0 0.5 1 x x Alex Townsend, University of Oxford 24th of June 2013 10/26 Higher-order linear ODEs (λ) Higher-order diff operators: Dλ : T ! C , [DLMF] 0 λ times 1 (λ) ( z }| { dC (λ+1) B0 ··· 0 λ C k 2λCk−1 ; k ≥ 1; λ−1 B C = Dλ = 2 (λ−1)! B C : dx 0; k = 0; B λ + 1 C @ : A :: (λ) (λ+1) Higher-order conversion operators: Sλ : C ! C , [DLMF] 8 0 1 λ (λ+1) (λ+1) 1 − λ > λ+k Ck − Ck−2 ; k ≥ 2; λ+2 <> λ λ (λ) λ (λ+1) B − C Ck = C ; k = 1; Sλ = B λ+1 λ+3 C : > λ+1 1 @ : : A :> (λ+1) :: :: C0 ; k = 0; Jacobi with w(x)=(1−x)α(1+x)β 2.5 Multiplication operators: 2 C(2) λ λ 1.5 Mλ[a]: C ! C ; β 1 Differentiation (1) (1) (1) M1[a]: C ! C ; 0.5 C Conversion L 0 M1[a] = Toeplitz + Hankel: Jacobi Ultraspherical T−0.5 −0.5 0 0.5 1 1.5 2 2.5 α Alex Townsend, University of Oxford 24th of June 2013 11/26 Construction of ultraspherical multiplication operators (λ) (λ) (λ) Mλ[a]: C ! C represents multiplication with a(x) in C . If m X (λ) a(x) = aj Cj (x); j=0 then it can be shown that Mλ[a] is m-banded with k X λ (Mλ[a])j;k = a2s+j−k cs (k; 2s + j − k); j; k ≥ 0; s=max(0;k−j) where (to prevent overflow issues) we have s−1 j−s−1 j + k + λ − 2s Y λ + t Y λ + t cλ(j; k) = × × s j + k + λ − s 1 + t 1 + t t=0 t=0 s−1 j−s−1 Y 2λ + j + k − 2s + t Y k − s + 1 + t × × : λ + j + k − 2s + t k − s + λ + t t=0 t=0 λ For efficiency, we use a recurence relation to generate cs (j; k). Alex Townsend, University of Oxford 24th of June 2013 12/26 A high-order example d N u d N−1u aN (x) + aN−1(x) + ··· + a0(x)u = f (x); Bu = c; dx N dx N−1 B c N N−1 0 = MN [a ]DN + SN−1MN−1[a ]DN−1 + ··· + SN−1 ···S0M0[a ] SN−1 ···S0f Example 3: High-order ODE u(10)(x) + cosh(x)u(8)(x) + cos(x)u(2)(x) + x 2u(x) = 0 u(±1) = 0; u0(±1) = 1; u(k)(±1) = 0; k = 2; 3; 4: 0 0.5 10 0.4 −5 0.3 10 0.2 1 2 −10 0.1 || 10 1 2 n Z u 2 0 − u(x) (~u(x) +u ~(−x)) n+1 −15 −0.1 u 10 || −1 −0.2 −20 −14 −0.3 10 = 1:3 × 10 : −0.4 −25 −0.5 10 −1 −0.5 0 0.5 1 20 40 60 80 100 120 140 x n Alex Townsend, University of Oxford 24th of June 2013 13/26 Airy equation and regularity preserving Example 4 Consider Airy's equation for > 0, u00(x) − xu(x) = 0; u(−1) = Ai −p3 1/ ; u(1) = Ai p3 1/ : Relative error in derivative at x=1/2 5 10 10 10 Ai(1) = 10−9 Ai(20) 0 (100) 108 10 Ai 6 10 −4 −5 = 10 10 104 Relative error Condition Number −10 10 102 = 1 −15 0 10 10 0.5 1 1.5 2 2.5 3 0 50 100 150 200 n x 104 n The US method can accurately compute high derivatives of functions.