Function approximation: Fourier, Chebyshev, Lagrange ¾ Orthogonal functions ¾ Fourier Series ¾ Discrete Fourier Series ¾ Fourier Transform: properties ¾ Chebyshev polynomials ¾ Convolution ¾ DFT and FFT Scope: Understanding where the Fourier Transform comes from. Moving from the continuous to the discrete world. The concepts are the basis for pseudospectral methods and the spectral element approach. Orthogonal functions 1 Fourier Series: one way to derive them The Problem we are trying to approximate a function f(x) by another function gn (x) which consists of a sum over N orthogonal functions Φ(x) weighted by some coefficients an . N f()()() x≈ gN x =∑ i a i Φ x i=0 sn functiolanogOrtho 2 The Problem ... and we are looking for optimal functions in a least squares (l2 ) sense ... b 1 / 2 ⎡ 2 ⎤ f x()()()() g− x = f{} x − g x=Min dx ! N 2 ⎢∫ N ⎥ ⎣a ⎦ ... a good choice for the basis functions Φ(x) are orthogonal functions. are orthogonal functions? What Two functions f and g are said to be orthogonal in the interval [a,b] if b f( x∫ ) g ( x )= dx 0 a How is this related to the more conceivble concept of orthogonal a vectors? Let us look at the original definition of integrals: sn functiolanogOrtho 3 Orthogonal Functions b ⎛ N ⎞ (f ) x ( g ) x= dx limf⎜ xi g ( i Δ x )⎟ ( x ) ∫ N→ ∞∑ a ⎝ i=1 ⎠ ... where x0 =a and xN =b, and xi-x i-1 =Δx ... If we interpret f(xi) and g(xicomponents of an N component ) as the ith vector, then this sum corresponds directly to a scalar product of vectors. The vanishing of the scalar product is the condition for orthogonality of vectors (or functions). f• g = f g0 = g i i∑ i i fi i i sn functiolanogOrtho 4 Periodic functions Let us assume we have a piecewise continuous function of the form f( x 2+ π )= f ( x ) 40 30 f( x+ 2π ) = f ( x ) =2 x 20 10 0 -15 -10 -5 0 5 10 15 20 ... we want to approximate this function with a linear combination of 2π periodic functions: 1 , cos( ), sin( ),x cos( x 2 ), x sin( x 2nx ),..., nx cos( ), sin( ) 1 N f x⇒()() g ≈ xN a =0 ∑cos({} a +k ) kx+ k sin( b kx ) 2 k=1 sn functiolanogOrtho 5 Orthogonality ... are these functions orthogonal ? ⎧ 0 j≠ k π ⎪ cos(∫ jx ) cos( kx= dx⎨ 2 ) π j= k =0 − π ⎪ ⎩ π j= k >0 0⎧ j≠ k , j , k > 0 π ⎪ sin(∫ jx ) sin( kx= dx⎨ ) − π ⎪ ⎩π j= k >0 π cos(∫ jx ) sin( kx= dx )j≥ 00 , k > 0 − π ... YES, and these relations are valid for any interval of length 2π. this is an orthogNow we know that how can we obtain the onal basis, but coefficients for the basis functions? f(x)-g(x)from minimising sn functiolanogOrtho 6 Fourier coefficients optimal functions g(x) are given if (g ) x− ( f ) x = Minor ! ∂ g( ) x− ( f ) x = 0 n 2 ∂a k { n 2 } ... with the definition of g(x) we get ... 2 ⎡ π N ⎤ ∂ 2 ∂ ⎡ 1 ⎤ g()()n x− f x = a⎢ ⎢ 0 a+ ∑cos({}k kx )+ bk sin( kx− ) f⎥ x (⎥ ) dx ∂a 2 ∂a ∫ 2 k k ⎣⎢ − π ⎣ k =1 ⎦ ⎦⎥ leading to 1 N g xN()= a0 cos( +∑ a{}k ) kx+ k sin( b ) kx with 2 k =1 1 π a= ( f ) x cos( kx ) dx , 0k = , 1 ,...,N k ∫ π −π 1 π b= ( f ) x sin( kx ) dx , 1k ,= 2 ,...,N k ∫ π −π sn functiolanogOrtho 7 Fourier approximation of |x| ... Example ... f(), x= x −π ≤x ≤ π leads to the Fourier Serie 1 4⎧ cos(x cos( ) x 3cos( ) x 5 )⎫ g() x=π − ⎨ + + + ... ⎬ 2 π ⎩ 1 2 3 2 5 2 ⎭ .. and for n<4 g(x) looks like 4 3 2 1 0 -20 -15 -10 -5 0 5 10 sn functiolanogOrtho 8 Fourier approximation of x2 ... another Example ... (f ) x= 2 , x < 0x < π 2 leads to the Fourier Serie 4π 2 N ⎧ 4 4π ⎫ g() x cos(kx ) sin(kx ) N = +∑ ⎨ 2 − ⎬ 3 k =1 ⎩ k k ⎭ .. and for N<11, g(x) looks like 40 30 20 10 0 -10 -10 -5 0 5 10 sn functiolanogOrtho 9 discrete functionsFourier - ... what happens if we know our function f(x) only at the points 2π x = i i N it turns out that in this particular case the coefficients are given by 2 N a * = (f ) x cos( kx ) ,0k ,= 1 , 2 ,... k ∑ j j N j =1 2 N b * = (f ) x sin( kx ) ,1k ,= 2 , 3 ,... k ∑ j j N j =1 .. the so-defined Fourier polynomial is the unique interpolating function to the function f(xj) with N=2m 1 m −1 1 g x*() a=* a + cos(* kx )+ b* sin(+ kx* )cos( a ) kx m 0 ∑ {}k k m 2 k =1 2 sn functiolanogOrtho 10 Fourier - collocation points ... with the important property that ... * g()()m x i = fi x ... in our previous examples ... 3.5 3 2.5 2 1.5 1 0.5 0 -10 -5 0 5 f(x)=|x| => f(x) - blue ; g(x) - red; xi -‘+’ Orthogonal functions 11 Fourier series - convergence 2 f(x)=x => f(x) - blue ; g(x) - red; xi -‘+’ Orthogonal functions 12 Fourier series - convergence 2 f(x)=x => f(x) - blue ; g(x) - red; xi -‘+’ Orthogonal functions 13 Gibb’s phenomenon 2 f(x)=x => f(x) - blue ; g(x) - red; xi -‘+’ N = 16 N = 32 6 6 6 4 4 4 2 2 2 0 0 0 -2 -2 -2 -4 -4 -4 -6 -6 -6 0 0.5 1 1.5 0 0.5 1 1.5 0 0.5 N = 128 N = 256 6 6 4 4 The overshoot for equi- 2 2 spaced Fourier 0 0 interpolations is ≈14% of -2 -2 the step height. -4 -4 -6 -6 0 0.5 1 1.5 0 0.5 1 1.5 Orthogonal functions 14 Chebyshev polynomials We have seen that Fourier series are excellent for interpolating (and differentiating) periodic functions defined on a regularly spaced grid. In many circumstances physical phenomena which are not periodic (in space) and occur in a limited area. This quest leads to the use of Chebyshev polynomials. We depart by observing that cos(nϕ) can be expressed by a polynomial in cos(ϕ): cos( 2ϕ )= 22 ϕ cos− 1 cos( 3 )ϕ = 4 cosϕ3 − ϕ 3 cos cos( 4 )ϕ 8= cos4 ϕ − 2 8ϕ + cos 1 ... which leads us to the definition: Orthogonal functions 15 definitionpolynomials - Chebyshev cos( )nϕ = Tn (cos(ϕ T= n x))cos(= x ( ),ϕ ),x ∈ − [ 1 ,n 1∈ ], N for the Chebyshev polynomials T... n (x). Note that because of x=cos(ϕ) they are defined in the interval [-1,1] (which - however - can be extended to ℜ). The first polynomials are T(0 ) x= 1 T1() x= x 2 T(2 ) x= 2 x − 1 3 T(3 x )= 4 x − 3 x 4 2 T(4 ) x= 8 x − 8 x + 1 where ( ) 1Tn x ≤ forx∈ [ 1− , 1 ]n∈ and N0 sn functiolanogOrtho 16 Chebyshev polynomials - Graphical The first ten polynomials look like [0, -1] 1 0.5 0 T_n(x) -0.5 -1 0 0.2 0.4 0.6 0.8 x The n-th polynomial has extrema with values 1 or -1 at kπ x()ext =cos( ),0 ,k 1= , 2 , 3n ,..., k n Orthogonal functions 17 Chebyshev collocation points These extrema are not equidistant (like the Fourier extrema) k x(k) kπ x()ext =cos( ),0 ,k 1= , 2 , 3n ,..., k n Orthogonal functions 18 orthogonalityChebyshev polynomials - ... are the Chebyshev polynomials orthogonal? Chebyshev polynomials are an orthogonal set of functions in the interval [-1,1] with respect to the weight function 1 /− 1x2 such that ⎧ 0 for≠ k j⎫ 1 dx ⎪ ⎪ T()() x T x = π/ 2 for= k >0 j ,,k j∈ N ∫ k j 2 ⎨ ⎬ 0 −1 1 − x ⎪ ⎪ ⎩ π for= k =0 j ⎭ this can be easily... verified noting that x =cosϕ , dx = −sinϕdϕ ( )T xk cos(= kϕ ),j T (= )xϕ cos( j ) sn functiolanogOrtho 19 nterpolationiChebyshev polynomials - ... we are now faced with the same problem as with the Fourier series. We want to approximate a function f(x), this time not a periodical function but a function which is defined between [-1,1]. We are looking for gn (x) 1 n f x()() g≈ xn = c0()() T 0 + x∑ k k c T x 2 k =1 ... and we are faced with the problem, how we can determine the coefficients ck. Again we obtain this by finding the extremum (minimum) ∂ ⎡ 1 dx ⎤ g{}()() x− f2 x = 0 ⎢∫ n 2 ⎥ ∂ck ⎣−1 1− x ⎦ sn functiolanogOrtho 20 nterpolationiChebyshev polynomials - ... to obtain ... 2 1 dx c= f()() x T, x 0k ,= 1 , 2n ,..., k ∫ k 2 π −1 1 − x ... surprisingly these coefficients can be calculated with FFT techniques, noting that 2 π (cosc= ) f ϕ cosk ϕ d , ϕ = k 0 , 1n , 2 ,..., k ∫ π 0 ... and the fact that f(cosϕ) is a 2π-periodic function ... 1 π (cosc= ) f ϕ cosk ϕ d , ϕ = k 0 , 1n , 2 ,..., k ∫ π −π ... which means that the coefficients ck are the Fourier coefficients ak of the periodic function F(ϕ)=f(cos ϕ)! sn functiolanogOrtho 21 Chebyshev discrete functions- ... what happens if we know our function f(x) only at the points π x = cos i i N in this particular case the coefficients are given by 2 N c * = (cosf ϕ ) cos(k ϕ 0 ) ,k , 1= , 2 ,...N / 2 k ∑ j j N j =1 ..