OrthogonalOrthogonal functionsfunctions -- FFunctionunction ApproximationApproximation
- The Problem
The Problem
we are trying to approximate a function f(x) by another function gn(x) which consists of a sum over N orthogonal functions Φ(x) weighted by
some coefficients an. N f (x) ≈ gN (x) = ∑aiΦi (x) i=0
Numerical Methods in Geophysics Orthogonal Functions TheThe ProblemProblem
... and we are looking for optimal functions in a least squares (l2) sense ... b 1/2 ⎡ 2 ⎤ f (x)− g (x) = {}f (x)− g (x) dx = Min! N 2 ⎢∫ N ⎥ ⎣a ⎦
... a good choice for the basis functions Φ(x) are orthogonal functions. What are orthogonal functions? Two functions f and g are said to be orthogonal in the interval [a,b] if
b ∫ f (x)g(x)dx= 0 a How is this related to the more conceivable concept of orthogonal vectors? Let us look at the original definition of integrals:
Numerical Methods in Geophysics Orthogonal Functions OrthogonalOrthogonal FunctionsFunctions -- DDefinitionefinition
b ⎛ N ⎞ f (x)g(x)dx= lim⎜ fi (x)gi (x)∆x⎟ ∫ N→∞ ∑ a ⎝ i=1 ⎠
... where x0=a and xN=b, and xi-xi-1=∆x ... If we interpret f(xi) and g(xi) as the ith components of an N component vector, then this sum corresponds directly to a scalar product of vectors. The vanishing of the scalar product is the condition for orthogonality of vectors (or functions).
fi • gi = ∑ fi gi = 0 i g fi i
Numerical Methods in Geophysics Orthogonal Functions PeriodicPeriodic functionsfunctions
Let us assume we have a piecewise continuous function of the form f (x+2π) = f (x)
40 30 f (x+2π) = f (x) = x2 20 10
0 -15 -10 -5 0 5 10 15 20
... we want to approximate this function with a linear combination of 2π periodic functions:
1, cos( x ), sin( x ), cos( 2 x ), cos( 2 x ),..., cos( nx ), sin( nx ) 1 N ⇒ f (x) ≈ gN (x) = a0 +∑{}ak cos(kx) +bk sin(kx) 2 k=1
Numerical Methods in Geophysics Orthogonal Functions OrthogonalityOrthogonality ofof PeriodicPeriodic functionsfunctions
... are these functions orthogonal ?
⎧ 0 j ≠ k π ⎪ ∫ cos( jx ) cos( kx )dx = ⎨ 2π j = k = 0 − π ⎪ ⎩ π j = k > 0 ⎧ 0 j ≠ k , j , k > 0 π ⎪ ∫ sin( jx ) sin( kx )dx = ⎨ − π ⎪ ⎩π j = k > 0 π ∫ cos( jx ) sin( kx )dx = 0 j ≥ 0 , k > 0 − π
... YES, and these relations are valid for any interval of length 2π. Now we know that this is an orthogonal basis, but how can we obtain the coefficients for the basis functions?
from minimising f(x)-g(x)
Numerical Methods in Geophysics Orthogonal Functions FourierFourier coefficientscoefficients
optimal functions g(x) are given if g (x) − f (x) = Min ! or ∂ {g (x) − f (x) }= 0 n 2 ∂a k n 2 ... with the definition of g(x) we get ...
2 ⎡ π N ⎤ ∂ 2 ∂ ⎡ 1 ⎤ g n ( x ) − f ( x ) = ⎢ ⎢ a 0 + ∑ {}a k cos( kx ) + b k sin( kx ) − f ( x )⎥ dx ⎥ ∂a 2 ∂a ∫ 2 k k ⎣⎢ −π ⎣ k =1 ⎦ ⎦⎥
leading to (nice exercise) 1 N g N ( x) = a 0 + ∑ {}a k cos( kx ) + bk sin( kx ) with 2 k =1 1 π a = f ( x) cos( kx )dx , k = 0,1,..., N k ∫ π −π 1 π b = f ( x) sin( kx )dx , k = 1,2,..., N k ∫ π −π
Numerical Methods in Geophysics Orthogonal Functions FourierFourier approximationapproximation ofof |x||x|
... Example ... f (x) = x , − π ≤ x ≤ π
leads to the Fourier Serie
1 4 ⎧ cos( x ) cos( 3 x ) cos( 5 x ) ⎫ g ( x ) = π − ⎨ + + + ... ⎬ 2 π ⎩ 1 2 3 2 5 2 ⎭
.. and for n<4 g(x) looks like
4 3 2 1 0 -20 -15 -10 -5 0 5 10
Numerical Methods in Geophysics Orthogonal Functions FourierFourier approximationapproximation ofof xx22
... another Example ... f (x) = x 2 , 0 < x < 2π
leads to the Fourier Serie
4π 2 N ⎧ 4 4π ⎫ g ( x ) cos( kx ) sin( kx ) N = + ∑ ⎨ 2 − ⎬ 3 k =1 ⎩ k k ⎭
.. and for N<11, g(x) looks like
40
30
20
10
0
-1 0 -1 0 -5 0 5 10
Numerical Methods in Geophysics Orthogonal Functions FourierFourier -- ddiscreteiscrete functionsfunctions
... what happens if we know our function f(x) only at the points 2π x = i i N it turns out that in this particular case the coefficients are given by
2 N a * = f ( x ) cos( kx ), k = 0,1,2,... k ∑ j j N j =1 2 N b * = f ( x ) sin( kx ), k = 1,2,3,... k ∑ j j N j =1
.. the so-defined Fourier polynomial is the unique interpolating function to
the function f(xj) with N=2m
1 m −1 1 g * ( x) = a * + {}a * cos( kx ) + b * sin( kx ) + a * cos( kx ) m 0 ∑ k k m 2 k =1 2
Numerical Methods in Geophysics Orthogonal Functions FourierFourier -- ccollocationollocation pointspoints
... with the important property that ...
* g m (xi ) = f (xi )
... in our previous examples ...
3.5
3
2.5
2
1.5
1
0.5
0 -10 -5 0 5
f(x)=|x| => f(x) - blue ; g(x) - red; xi -‘+’
Numerical Methods in Geophysics Orthogonal Functions FourierFourier seriesseries -- cconvergenceonvergence
2 f(x)=x => f(x) - blue ; g(x) - red; xi -‘+’ N = 4 50 40
30 20 10
0 -10 -10 -5 0 5 10 N = 8 50 40
30 20 10
0 -10 -10 -5 0 5 10
Numerical Methods in Geophysics Orthogonal Functions FourierFourier seriesseries -- cconvergenceonvergence
2 f(x)=x => f(x) - blue ; g(x) - red; xi -‘+’ N = 16 50 40
30 20 10
0 -10 -10 -5 0 5 10
N = 32 50 40
30 20 10
0 -10 -10 -5 0 5 10
Numerical Methods in Geophysics Orthogonal Functions OrthogonalOrthogonal functionsfunctions -- GGibb’sibb’s phenomenonphenomenon
2 f(x)=x => f(x) - blue ; g(x) - red; xi -‘+’
N = 16 N = 32
6 6 6 4 4 4 2 2 2 0 0 0 -2 -2 -2 -4 -4 -4 -6 -6 -6 0 0.5 1 1.5 0 0.5 1 1.5 0 0.5 N = 128 N = 256
6 6 4 4 The overshoot for equi- 2 2 spaced Fourier 0 0 interpolations is ≈14% of -2 -2 the step height. -4 -4 -6 -6 0 0.5 1 1.5 0 0.5 1 1.5
Numerical Methods in Geophysics Orthogonal Functions ChebyshevChebyshev polynomialspolynomials
We have seen that Fourier series are excellent for interpolating (and differentiating) periodic functions defined on a regularly spaced grid. In many circumstances physical phenomena which are not periodic (in space) and occur in a limited area. This quest leads to the use of Chebyshev polynomials.
We depart by observing that cos(nϕ) can be expressed by a polynomial in cos(ϕ):
cos( 2ϕ ) = 2 cos 2 ϕ − 1 cos( 3ϕ ) = 4 cos 3 ϕ − 3 cos ϕ cos( 4ϕ ) = 8 cos 4 ϕ − 8 cos 2 ϕ + 1
... which leads us to the definition:
Numerical Methods in Geophysics Orthogonal Functions ChebyshevChebyshev polynomialspolynomials -- ddefinitionefinition
cos(nϕ) = Tn (cos(ϕ)) = Tn (x), x = cos(ϕ), x ∈[−1,1], n ∈ N
... for the Chebyshev polynomials Tn(x). Note that because of x=cos(ϕ) they are defined in the interval [-1,1] (which - however - can be extended to ℜ). The first polynomials are
T0 (x) =1
T1(x) = x 2 T2 (x) = 2x −1 3 T3 (x) = 4x − 3x 4 2 T4 (x) = 8x − 8x +1 where
Tn (x) ≤ 1 for x ∈[−1,1] and n ∈ N0
Numerical Methods in Geophysics Orthogonal Functions ChebyshevChebyshev polynomialspolynomials -- GGraphicalraphical
The first ten polynomials look like [0, -1]
1
0.5 ) x (
n 0 T_
-0.5
-1 0 0.2 0.4 0.6 0.8 x
The n-th polynomial has extrema with values 1 or -1 at
kπ x (ext ) = cos( ), k = 0,1,2,3,..., n k n
Numerical Methods in Geophysics Orthogonal Functions ChebyshevChebyshev collocationcollocation pointspoints
These extrema are not equidistant (like the Fourier extrema)
k
x(k)
kπ x (ext ) = cos( ), k = 0,1,2,3,..., n k n
Numerical Methods in Geophysics Orthogonal Functions ChebyshevChebyshev polynomialspolynomials -- oorthogonalityrthogonality
... are the Chebyshev polynomials orthogonal?
Chebyshev polynomials are an orthogonal set of functions in the interval [-1,1] with respect to the weight function 1/ 1− x2 such that
⎧ 0 for k ≠ j ⎫ 1 dx ⎪ ⎪ T (x)T (x) = π / 2 for k = j > 0 , k , j ∈ N ∫ k j 2 ⎨ ⎬ 0 −1 1 − x ⎪ ⎪ ⎩ π for k = j = 0 ⎭
... this can be easily verified noting that x = cosϕ, dx = −sinϕdϕ
Tk (x) = cos(kϕ), Tj (x) = cos( jϕ)
Numerical Methods in Geophysics Orthogonal Functions ChebyshevChebyshev polynomialspolynomials -- iinterpolationnterpolation
... we are now faced with the same problem as with the Fourier series. We want to approximate a function f(x), this time not a periodical function but a function which is defined between [-1,1].
We are looking for gn(x)
1 n f (x) ≈ g n (x) = c0T0 (x) + ∑ ck Tk (x) 2 k =1 ... and we are faced with the problem, how we can determine the
coefficients ck. Again we obtain this by finding the extremum (minimum)
∂ ⎡ 1 dx ⎤ {}g (x) − f (x) 2 = 0 ⎢∫ n 2 ⎥ ∂ck ⎣−1 1− x ⎦
Numerical Methods in Geophysics Orthogonal Functions ChebyshevChebyshev polynomialspolynomials -- iinterpolationnterpolation
... to obtain ... 2 1 dx c = f (x)T (x) , k = 0,1,2,..., n k ∫ k 2 π −1 1 − x
... surprisingly these coefficients can be calculated with FFT techniques, noting that 2 π c = f (cosϕ)cos kϕdϕ, k = 0,1,2,..., n k ∫ π 0 ... and the fact that f(cosϕ) is a 2π-periodic function ... 1 π c = f (cosϕ)cos kϕdϕ, k = 0,1,2,..., n k ∫ π −π
... which means that the coefficients ck are the Fourier coefficients ak of the periodic function F(ϕ)=f(cos ϕ)!
Numerical Methods in Geophysics Orthogonal Functions ChebyshevChebyshev -- ddiscreteiscrete functionsfunctions
... what happens if we know our function f(x) only at the points π x = cos i i N in this particular case the coefficients are given by 2 N c * = f (cos ϕ ) cos( kϕ ), k = 0,1,2,... N / 2 k ∑ j j N j =1 ... leading to the polynomial ...
1 m g * ( x) = c *T + c *T ( x) m 0 0 ∑ k k 2 k =1
... with the property
* g m ( x) = f ( x) at x j = cos( πj/N) j = 0,1,2,..., N
Numerical Methods in Geophysics Orthogonal Functions ChebyshevChebyshev -- ccollocationollocation pointspoints -- ||x|x|
f(x)=|x| => f(x) - blue ; gn(x) - red; xi -‘+’
N = 8 1
0.8
0.6 8 points 0.4
0.2
0 -1 -0 .8 -0 .6 -0 .4 -0 .2 0 0.2 0.4 0.6 0.8
N = 16 1
0.8
0.6 16 points 0.4
0.2
0 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8
Numerical Methods in Geophysics Orthogonal Functions ChebyshevChebyshev -- ccollocationollocation pointspoints -- ||x|x|
f(x)=|x| => f(x) - blue ; gn(x) - red; xi -‘+’
N = 32 1
0.8
0.6 32 points 0.4
0.2
0 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8
N = 128 1
0.8
128 points 0.6
0.4
0.2
0 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.
Numerical Methods in Geophysics Orthogonal Functions ChebyshevChebyshev -- ccollocationollocation pointspoints -- xx22
2 f(x)=x => f(x) - blue ; gn(x) - red; xi -‘+’ N = 8 1.2
1
0.8
8 points 0.6
0.4
0.2 The interpolating
0 function gn(x) was -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 N = 64 shifted by a small 1.2 amount to be
1 visible at all!
0.8
0.6
64 points 0.4
0.2
0 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
Numerical Methods in Geophysics Orthogonal Functions ChebyshevChebyshev vs.vs. FourierFourier -- nnumericalumerical
Chebyshev Fourier
N = 16 N = 1 35
0.8 30
25 0.6 20
15 0.4 10
0.2 5 0
0 -5 0 0.2 0.4 0.6 0.8 1 0 2
2 f(x)=x => f(x) - blue ; gN(x) - red; xi -‘+’
This graph speaks for itself ! Gibb’s phenomenon with Chebyshev?
Numerical Methods in Geophysics Orthogonal Functions ChebyshevChebyshev vs.vs. FourierFourier -- GGibb’sibb’s
Chebyshev Fourier
N = 16 N = 16 1.5 1.5
1 1
0.5 0.5
0 0
-0.5 -0.5
-1 -1
-1.5 -1.5 -1 -0.5 0 0.5 1 0 2
) f(x)=sign(x-π => f(x) - blue ; gN(x) - red; xi -‘+’
Gibb’s phenomenon with Chebyshev? YES!
Numerical Methods in Geophysics Orthogonal Functions ChebyshevChebyshev vs.vs. FourierFourier -- GGibb’sibb’s
Chebyshev Fourier
N = 64 N = 6 1.5 1.5
1 1
0.5 0.5
0 0
-0.5 -0.5
-1 -1
-1.5 -1.5 -1 -0.5 0 0.5 1 0 2 4
) f(x)=sign(x-π => f(x) - blue ; gN(x) - red; xi -‘+’
Numerical Methods in Geophysics Orthogonal Functions FourierFourier vs.vs. ChebyshevChebyshev
Fourier Chebyshev
2π π x = i collocation points xi = cos i i N N
periodic functions domain limited area [-1,1]
T ( x ) = cos( n ϕ ), cos( nx ), sin( nx ) basis functions n x = cos ϕ
* 1 * m g m (x) = a 0 2 * 1 * * interpolating g m ( x) = c0T 0 + c k Tk ( x) m −1 ∑ function 2 k =1 + {}a * cos( kx) + b * sin( kx) ∑ k k k =1 1 + a * cos( kx) 2 m
Numerical Methods in Geophysics Orthogonal Functions FourierFourier vs.vs. ChebyshevChebyshev (cont’d)(cont’d)
Fourier Chebyshev
N * 2 a = f ( x j ) cos( kx j ) N k ∑ * 2 N j =1 c = f (cos ϕ ) cos( kϕ ) coefficients k N ∑ j j 2 N j =1 b * = f ( x ) sin( kx ) k ∑ j j N j =1
• Gibb’s phenomenon for • limited area calculations discontinuous functions some properties • grid densification at boundaries • Efficient calculation via FFT • coefficients via FFT • infinite domain through periodicity • excellent convergence at boundaries
• Gibb’s phenomenon
Numerical Methods in Geophysics Orthogonal Functions