Rational Function Approximation

Rational function approximation

Rational function of degree N = n + m is written as

p(x) p + p x + + p xn r(x) = = 0 1 ··· n q(x) q + q x + + q xm 0 1 ··· m Now we try to approximate a function f on an interval containing 0 using r(x).

WLOG, we set q0 = 1, and will need to determine the N + 1 unknowns p0,..., pn, q1,..., qm.

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 240 Pad´eapproximation

The idea of Pad´eapproximation is to ﬁnd r(x) such that

f (k)(0) = r (k)(0), k = 0, 1,..., N

This is an extension of Taylor series but in the rational form.

i Denote the Maclaurin series expansion f (x) = i∞=0 ai x . Then

i m i n i ∞ a x q x P p x f (x) r(x) = i=0 i i=0 i − i=0 i − q(x) P P P If we want f (k)(0) r (k)(0) = 0 for k = 0,..., N, we need the − numerator to have 0 as a root of multiplicity N + 1.

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 241 Pad´eapproximation

This turns out to be equivalent to

k ai qk i = pk , k = 0, 1,..., N − Xi=0 for convenience we used convention p = = p = 0 and n+1 ··· N q = = q = 0. m+1 ··· N From these N + 1 equations, we can determine the N + 1 unknowns: p0, p1,..., pn, q1,..., qm

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 242 Pad´eapproximation

Example x Find the Pad´eapproximation to e− of degree 5 with n = 3 and m = 2. x Solution. We ﬁrst write the Maclaurin series of e− as

i x 1 2 1 3 1 4 ∞ ( 1) i e− = 1 x + x x + x + = − x − 2 − 6 24 ··· i! Xi=0 2 3 p0+p1x+p2x +p3x Then for r(x) = 2 , we need 1+q1x+q2x 1 1 1 x + x2 x3 + (1+q x+q x2) p +p x+p x2+p x3 − 2 − 6 ··· 1 2 − 0 1 2 3 to have 0 coeﬃcients for terms 1, x,..., x5.

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 243 530 CHAPTER 8 Approximation Theory

Find the Padé approximation to e x of degree 5 with n 3 and m 2. − = = Solution To ﬁnd the Padé approximation we need to choose p0, p1, p2, p3, q1, and q2 so that the coefﬁcients of xk for k 0, 1, ..., 5 are 0 in the expression = 2 3 x x 2 2 3 1 x (1 q1x q2x ) (p0 p1x p2x p3x ). − + 2 − 6 +··· + + − + + + ! " Expanding and collecting terms produces

5 1 1 1 2 1 x : q1 q2 0; x : q1 q2 p2; −120 + 24 − 6 = 2 − + = 4 1 1 1 1 x : q1 q2 0; x : 1 q1 p1; 24 − 6 + 2 = − + = 3 1 1 0 x : q1 q2 p3; x :1 p0. −6 + 2 − = = To solve the system in Maple, we use the following commands: eq1: 1 q1 p1: = −1 + = eq2: 2 q1 q2 p2: = −1 1+ = eq3: 6 2 q1 q2 p3: = −1 +1 −1 = eq4: 24 6 q1 2 q2 0: = −1 1+ 1= eq5: 120 24 q1 6 q2 0: solve(=eq−1, eq+2, eq3, eq−4, eq5=, q1, q2, p1, p2, p3 ) { } { } This gives Pad´eapproximation 3 3 1 2 1 p1 , p2 , p3 , q1 , q2 = −5 = 20 = −60 = 5 = 20 # $ So the Padé approximation is Solution. (cont.) By solving this, we get p0, p1, p2, q1, q2 and 1 3 x 3 x2 1 x3 hence r(x) − 5 + 20 − 60 . = 31 2 x3 12x2 1 3 1 x +5 x20 x r(x) = − 5 + 20+ − 60 Table 8.10 lists values of r(x) and P5(x)1, the + ﬁfth2 x Maclaurin+ 1 x2 polynomial. The Padé approximation is clearly superior in this example. 5 20

Table 8.10 x x x xe− P (x) e− P (x) r(x) e− r(x) 5 | − 5 | | − | 8 9 0.2 0.81873075 0.81873067 8.64 10− 0.81873075 7.55 10− × 6 × 7 0.4 0.67032005 0.67031467 5.38 10− 0.67031963 4.11 10− × 5 × 6 0.6 0.54881164 0.54875200 5.96 10− 0.54880763 4.00 10− × 4 × 5 0.8 0.44932896 0.44900267 3.26 10− 0.44930966 1.93 10− × 3 × 5 1.0 0.36787944 0.36666667 1.21 10− 0.36781609 6.33 10− × ×

whereMapleP5 can(x) also is Maclaurin be used directly polynomial to compute a of Padé degree approximation. 5 for comparison. We ﬁrst compute the Maclaurin series with the call series(exp( x), x) − to obtain Numerical Analysis I – Xiaojing Ye, Math1 & Stat,1 Georgia State1 University1 244 1 x x2 x3 x4 x5 O(x6) − + 2 − 6 + 24 − 120 + The Padé approximation r(x) with n 3 and m 2 is found using the command = =

Copyright 2010 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Chebyshev rational function approximation

To obtain more uniformly accurate approximation, we can use Chebyshev polynomials Tk (x) in Pad´eapproximation framework.

For N = n + m, we use

n p T (x) r(x) = k=0 k k m q T (x) Pk=0 k k

where q0 = 1. Also write f (x)P using Chebyshev polynomials as

∞ f (x) = ak Tk (x) Xk=0

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 245 Chebyshev rational function approximation

Now we have m n ∞ a T (x) q T (x) p T (x) f (x) r(x) = k=0 k k k=0 k k − k=0 k k − m q T (x) P P k=0 k k P

We again seek for p0,..., pn, q1,...,P qm such that coeﬃcients of 1, x,..., xN are 0.

To that end, the computations can be simplified due to 1 Ti (x)Tj (x) = Ti+j (x) + T i j (x) 2 | − | Also note that we also need to compute Chebyshev series coefficients ak first.

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 246 8.4 Rational Function Approximation 535

The solution to this system produces the rational function

1.055265T0(x) 0.613016T1(x) 0.077478T2(x) 0.004506T3(x) rT (x) − + − . = T0(x) 0.378331T1(x) 0.022216T2(x) + + We found at the beginning of Section 8.3 that 2 3 T0(x) 1, T1(x) x, T2(x) 2x 1, T3(x) 4x 3x. = = = − = − ChebyshevUsing these to rational convert to an function expression involving approximation powers of x gives 0.977787 0.599499x 0.154956x2 0.018022x3 rT (x) − + − . = 0.977784 0.378331x 0.044432x2 + + Table 8.11 lists values of rT (x) and, for comparison purposes, the values of r(x) obtained Examplein Example 1. Note that the approximation given by r(x) is superior to that of rT (x) for 5 6 x 0.2 and 0.4, but thatx the maximum error for r(x) is 6.33 10− compared to 9.13 10− Approximate= e− using the Chebyshev rational× approximation of× for rT (x). degree n = 3 and m = 2. The result is rT (x).

Table 8.11 x x x xe− r(x) e− r(x) r (x) e− r (x) | − | T | − T | 9 6 0.2 0.81873075 0.81873075 7.55 10− 0.81872510 5.66 10− × 7 × 6 0.4 0.67032005 0.67031963 4.11 10− 0.67031310 6.95 10− × 6 × 6 0.6 0.54881164 0.54880763 4.00 10− 0.54881292 1.28 10− × 5 × 6 0.8 0.44932896 0.44930966 1.93 10− 0.44933809 9.13 10− × 5 × 6 1.0 0.36787944 0.36781609 6.33 10− 0.36787155 7.89 10− × ×

whereTher Chebyshev(x) is the approximation standard Pad´eapproximation can be generated using Algorithm shown 8.2. earlier.

ALGORITHM Chebyshev Rational Approximation 8.2 To obtain the rational approximation

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia Staten University 247 k 0 pkTk(x) rT (x) m= = k 0 qkTk(x) ! = for a given function f(x): !

INPUT nonnegative integers m and n.

OUTPUT coefficients q0, q1, ..., qm and p0, p1, ..., pn. Step 1 Set N m n. = + 2 π Step 2 Set a0 f(cos θ) dθ;(The coefficient a0 is doubled for computational = π "0 efficiency.) For k 1, 2, ..., N m set = + 2 π ak f(cos θ) cos kθ dθ. = π "0 (The integrals can be evaluated using a numerical integration procedure or the coefficients can be input directly.)

Step 3 Set q0 1. = Step 4 For i 0, 1, ..., N do Steps 5–9. (Set up a linear system with matrix B.) =

Recall the Fourier series uses a set of 2n orthogonal functions with respect to weight w 1 on [ π, π]: ≡ − 1 φ (x) = 0 2 φk (x) = cos kx, k = 1, 2,..., n φ (x) = sin kx, k = 1, 2,..., n 1 n+k −

We denote the set of linear combinations of φ0, φ1, . . . , φ2n 1 by − , called the set of trigonometric polynomials of degree n. Tn ≤

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 248 Trigonometric polynomial approximation

For a function f C[ π, π], we want to ﬁnd S of form ∈ − n ∈ Tn n 1 a − S (x) = 0 + a cos nx + (a cos kx + b sin kx) n 2 n k k Xk=1 to minimize the least squares error

π 2 E(a0,..., an, b1,..., bn 1) = f (x) Sn(x) dx − π | − | Z−

Due to orthogonality of Fourier series φ0, . . . , φ2n 1, we get − 1 π 1 π ak = f (x) cos kx dx, bk = f (x) sin kx dx π π π π Z− Z−

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 249 Trigonometric polynomial approximation

Example Approximate f (x) = x for x [ π, π] using trigonometric | | ∈ − polynomial from n. T 1 π Solution. It is easy to check that a0 = π π x dx = π and − | | π R 1 2 k ak = x cos kx dx = 2 (( 1) 1), k = 1, 2,..., n π π | | πk − − Z− 1 π bk = x sin kx dx = 0, k = 1, 2,..., n 1 π π | | − Z− Therefore n π 2 ( 1)k 1 S (x) = + − − cos kx n 2 π k2 Xk=1

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 250 540 CHAPTER 8 Approximation Theory

Solution We first need to find the coefficients 1 π 1 0 1 π 2 π a0 x dx x dx x dx x dx π, = π π | | = −π π + π 0 = π 0 = !− !− ! ! 1 π 2 π 2 a x cos kx dx x cos kx dx ( 1)k 1 , k 2 = π π | | = π 0 = πk − − !− ! " # for each k 1, 2, ..., n, and = 1 π bk x sin kx dx 0, for each k 1, 2, ..., n 1. =π π | | = = − !−

That the bk’s are all 0 follows from the fact that g(x) x sin kx is an odd function for each k, and the integral of a continuous odd function over=| an| interval of the form a, a [− ] is 0. (See Exercises 13 and 14.) The trigonometric polynomial from n approximating f is therefore, T

π 2 n ( 1)k 1 Sn(x) − − cos kx. = 2 + π k2 k 1 $= TrigonometricThe polynomial ﬁrst few trigonometric approximation polynomials for f(x) x are shown in Figure 8.13. =| |

Sn(x)Figure for 8.13 the ﬁrst few n are shown below:

ϭ π y Խ x Խ

y ϭ S x ϭ π Ϫ 4 x Ϫ 4 x 3( ) 2 π cos 9π cos 3 y ϭ S x ϭ S x ϭ π Ϫ 4 x 1( ) 2( ) 2 π cos π y ϭ S x ϭ π 2 0( ) 2

Ϫπ π π π x Ϫ 2 2

The Fourier series for f is

k Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State Universityπ 2 ∞ ( 1) 1 251 S(x) lim Sn(x) − − cos kx. = n = 2 + π k2 →∞ k 1 $= Since cos kx 1 for every k and x, the series converges, and S(x) exists for all real numbers| x. | ≤

2m 1 If we have 2m paired data points (x , y ) − where x are { j j }j=0 j equally spaced on [ π, π], i.e., − j x = π + π, j = 0, 1,..., 2m 1 j − m − Then we can also seek for S such that the discrete least n ∈ Tn square error below is minimized:

2m 1 − 2 E(a0,..., an, b1,..., bn 1) = (yj Sn(xj )) − − Xj=0

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 252 Discrete trigonometric approximation

Theorem Deﬁne

2m 1 2m 1 1 − 1 − a = y cos kx , b = y sin kx k m j j k m j j Xj=0 Xj=0 Then the trigonometric S deﬁned by n ∈ Tn n 1 a − S (x) = 0 + a cos nx + (a cos kx + b sin kx) n 2 n k k Xk=1 minimizes the discrete least squares error

2m 1 − 2 E(a0,..., an, b1,..., bn 1) = (yj Sn(xj )) − − Xj=0

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 253 Fast Fourier transforms

The fast Fourier transform (FFT) employs the Euler formula ezi = cos z + i sin z for all z R and i = √ 1, and compute the ∈ − discrete Fourier transform of data to get

2m 1 2m 1 1 − − c ekxi, where c = y ekπi/m k = 0,..., 2m 1 m k k j − Xk=0 Xj=0

Then one can recover ak , bk R from ∈ ( 1)k ak + ibk = − ck C m ∈

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 254 Fast Fourier transforms

The discrete trigonometric approximation for 2m data points requires a total of (2m)2 multiplications, not scalable for large m.

The cost of FFT is only

3m + m log2 m = O(m log2 m)

For example, if m = 1024, then (2m)2 4.2 106 and ≈ × 3m + m log m 1.3 104. The larger m is, the more beneﬁt FFT 2 ≈ × gains.

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 255