Orthogonal Polynomials: Gram-Schmidt Process

1 Chebyshev Polynomials: w (x) = √ on [−1, 1] 1 − x2

Orthogonal Polynomials: Gram-Schmidt process

Thm: The set of polynomial functions {φ0, ··· , φn} deﬁned below on [a, b] is orthogonal with respect to the weight function w.

φ0 (x) = 1, φ1 (x) = x − B1, and for k ≥ 2

φk (x) = (x − Bk ) φk−1 (x) − Ck φk−2 (x) , with

R b 2 a x w (x) φj−1 (x) d x Bj = , j = 1, 2, ··· , n, R b 2 a w (x) φj−1 (x) d x R b a x w (x) φj−1 (x) φj−2 (x) d x Cj = , j = 2, 3, ··· , n. R b 2 a w (x) φj−2 (x) d x Orthogonal Polynomials: Gram-Schmidt process

Thm: The set of polynomial functions {φ0, ··· , φn} deﬁned below on [a, b] is orthogonal with respect to the weight function w.

φ0 (x) = 1, φ1 (x) = x − B1, and for k ≥ 2

φk (x) = (x − Bk ) φk−1 (x) − Ck φk−2 (x) , with

R b 2 a x w (x) φj−1 (x) d x Bj = , j = 1, 2, ··· , n, R b 2 a w (x) φj−1 (x) d x R b a x w (x) φj−1 (x) φj−2 (x) d x Cj = , j = 2, 3, ··· , n. R b 2 a w (x) φj−2 (x) d x

1 Chebyshev Polynomials: w (x) = √ on [−1, 1] 1 − x2 I Let x = cos (θ) , θ ∈ [0, π]. Then φj (x) = cos (jθ) , j = 0, 1. I Induction hypothesis with x = cos (θ) , θ ∈ [0, π]: cos(jθ) φj (x) = 2j−1 for j = 2, ··· , n − 1 . I By Gram-Schmidt, for n ≥ 2

φn (x) = (x − Bn) φn−1 (x) − Cn φn−2 (x) , with

R 1 2 −1 x w (x) φn−1 (x) d x Bn = , R 1 2 −1 w (x) φn−1 (x) d x R 1 −1 x w (x) φn−1 (x) φn−2 (x) d x Cn = . R 1 2 −1 w (x) φn−2 (x)

φn (x) = (x − Bn) φn−1 (x) − Cn φn−2 (x) , with

R 1 2 −1 x w (x) φn−1 (x) d x Bn = , R 1 2 −1 w (x) φn−1 (x) d x R 1 −1 x w (x) φn−1 (x) φn−2 (x) d x Cn = . R 1 2 −1 w (x) φn−2 (x)

§8.3 Chebyshev Polynomials/Power Series Economization Chebyshev: Gram-Schmidt for orthogonal polynomial functions {φ , ··· , φ } on [−1, 1] with weight function w (x) = √ 1 . 0 n 1−x2 R 1 √ x d x −1 1−x2 I φ0 (x) = 1; φ1 (x) = x − B1, with B1 = 1 = 0. R √ 1 d x −1 1−x2 I Let x = cos (θ) , θ ∈ [0, π]. Then φj (x) = cos (jθ) , j = 0, 1. I Induction hypothesis with x = cos (θ) , θ ∈ [0, π]: cos(jθ) φj (x) = 2j−1 for j = 2, ··· , n − 1 . §8.3 Chebyshev Polynomials/Power Series Economization Chebyshev: Gram-Schmidt for orthogonal polynomial functions {φ , ··· , φ } on [−1, 1] with weight function w (x) = √ 1 . 0 n 1−x2 R 1 √ x d x −1 1−x2 I φ0 (x) = 1; φ1 (x) = x − B1, with B1 = 1 = 0. R √ 1 d x −1 1−x2 I Let x = cos (θ) , θ ∈ [0, π]. Then φj (x) = cos (jθ) , j = 0, 1. I Induction hypothesis with x = cos (θ) , θ ∈ [0, π]: cos(jθ) φj (x) = 2j−1 for j = 2, ··· , n − 1 . I By Gram-Schmidt, for n ≥ 2

φn (x) = (x − Bn) φn−1 (x) − Cn φn−2 (x) , with

R 1 2 −1 x w (x) φn−1 (x) d x Bn = , R 1 2 −1 w (x) φn−1 (x) d x R 1 −1 x w (x) φn−1 (x) φn−2 (x) d x Cn = . R 1 2 −1 w (x) φn−2 (x) 1 cos (θ) cos2 ((n − 1)θ) = cos (θ) (1 + cos (2(n − 1)θ)) 2 1 1 = cos (θ) + (cos ((2n − 3)θ) + cos ((2n − 1)θ)) , so 2 2 Z π cos (θ) cos2 ((n − 1)θ) d θ 0 Z π 1 1 = cos (θ) + (cos ((2n − 3)θ) + cos ((2n − 1)θ)) d θ = 0, Bn = 0. 0 2 2

Chebyshev Polynomials: Compute Bn cos(jθ) Let x = cos (θ) , θ ∈ [0, π]. Then φj (x) = 2j−1 , 1 1 d x = −sin (θ) d θ, w (x) = √ = . 1 − x2 sin (θ)

2 R π cos(θ)cos ((n−1)θ) sin (θ) d θ B = 0 sin(θ) n R π cos2((n−1)θ) 0 sin(θ) sin (θ) d θ R π cos (θ) cos2 ((n − 1)θ) d θ = 0 , where R π 2 0 cos ((n − 1)θ) d θ Z π cos (θ) cos2 ((n − 1)θ) d θ 0 Z π 1 1 = cos (θ) + (cos ((2n − 3)θ) + cos ((2n − 1)θ)) d θ = 0, Bn = 0. 0 2 2

Chebyshev Polynomials: Compute Bn cos(jθ) Let x = cos (θ) , θ ∈ [0, π]. Then φj (x) = 2j−1 , 1 1 d x = −sin (θ) d θ, w (x) = √ = . 1 − x2 sin (θ)

2 R π cos(θ)cos ((n−1)θ) sin (θ) d θ B = 0 sin(θ) n R π cos2((n−1)θ) 0 sin(θ) sin (θ) d θ R π cos (θ) cos2 ((n − 1)θ) d θ = 0 , where R π 2 0 cos ((n − 1)θ) d θ 1 cos (θ) cos2 ((n − 1)θ) = cos (θ) (1 + cos (2(n − 1)θ)) 2 1 1 = cos (θ) + (cos ((2n − 3)θ) + cos ((2n − 1)θ)) , so 2 2 Z π cos (θ) cos2 ((n − 1)θ) d θ 0 Z π 1 1 = cos (θ) + (cos ((2n − 3)θ) + cos ((2n − 1)θ)) d θ = 0, 0 2 2 Chebyshev Polynomials: Compute Bn cos(jθ) Let x = cos (θ) , θ ∈ [0, π]. Then φj (x) = 2j−1 , 1 1 d x = −sin (θ) d θ, w (x) = √ = . 1 − x2 sin (θ)

2 R π cos(θ)cos ((n−1)θ) sin (θ) d θ B = 0 sin(θ) n R π cos2((n−1)θ) 0 sin(θ) sin (θ) d θ R π cos (θ) cos2 ((n − 1)θ) d θ = 0 , where R π 2 0 cos ((n − 1)θ) d θ 1 cos (θ) cos2 ((n − 1)θ) = cos (θ) (1 + cos (2(n − 1)θ)) 2 1 1 = cos (θ) + (cos ((2n − 3)θ) + cos ((2n − 1)θ)) , so 2 2 Z π cos (θ) cos2 ((n − 1)θ) d θ 0 Z π 1 1 = cos (θ) + (cos ((2n − 3)θ) + cos ((2n − 1)θ)) d θ = 0, Bn = 0. 0 2 2 cos (θ) cos ((n − 1)θ) cos ((n − 2)θ) 1 = cos ((n − 1)θ)(cos ((n − 1)θ) + cos ((n − 3)θ)) 2 1 = (1 + cos (2(n − 1)θ) + cos (2(n − 2)θ) + cos (2θ)) , so 4 Z π π 2 , for n = 2, cos (θ) cos ((n − 1)θ) cos ((n − 2)θ) d θ = π 0 4 , for n > 2.

Chebyshev Polynomials: Compute Cn (I) cos(jθ) Let x = cos (θ) , θ ∈ [0, π]. Then φj (x) = 2j−1 , 1 1 d x = −sin (θ) d θ, w (x) = √ = . 1 − x2 sin (θ)

R π cos(θ)cos((n−1)θ) cos((n−2)θ) sin (θ) d θ C = 0 sin(θ) n R π cos2((n−2)θ) 2 0 sin(θ) sin (θ) d θ R π cos (θ) cos ((n − 1)θ) cos ((n − 2)θ) d θ = 0 , where R π 2 2 0 cos ((n − 2)θ) d θ Z π π 2 , for n = 2, cos (θ) cos ((n − 1)θ) cos ((n − 2)θ) d θ = π 0 4 , for n > 2.

Chebyshev Polynomials: Compute Cn (I) cos(jθ) Let x = cos (θ) , θ ∈ [0, π]. Then φj (x) = 2j−1 , 1 1 d x = −sin (θ) d θ, w (x) = √ = . 1 − x2 sin (θ)

R π cos(θ)cos((n−1)θ) cos((n−2)θ) sin (θ) d θ C = 0 sin(θ) n R π cos2((n−2)θ) 2 0 sin(θ) sin (θ) d θ R π cos (θ) cos ((n − 1)θ) cos ((n − 2)θ) d θ = 0 , where R π 2 2 0 cos ((n − 2)θ) d θ cos (θ) cos ((n − 1)θ) cos ((n − 2)θ) 1 = cos ((n − 1)θ)(cos ((n − 1)θ) + cos ((n − 3)θ)) 2 1 = (1 + cos (2(n − 1)θ) + cos (2(n − 2)θ) + cos (2θ)) , so 4 Chebyshev Polynomials: Compute Cn (I) cos(jθ) Let x = cos (θ) , θ ∈ [0, π]. Then φj (x) = 2j−1 , 1 1 d x = −sin (θ) d θ, w (x) = √ = . 1 − x2 sin (θ)

R π cos(θ)cos((n−1)θ) cos((n−2)θ) sin (θ) d θ C = 0 sin(θ) n R π cos2((n−2)θ) 2 0 sin(θ) sin (θ) d θ R π cos (θ) cos ((n − 1)θ) cos ((n − 2)θ) d θ = 0 , where R π 2 2 0 cos ((n − 2)θ) d θ cos (θ) cos ((n − 1)θ) cos ((n − 2)θ) 1 = cos ((n − 1)θ)(cos ((n − 1)θ) + cos ((n − 3)θ)) 2 1 = (1 + cos (2(n − 1)θ) + cos (2(n − 2)θ) + cos (2θ)) , so 4 Z π π 2 , for n = 2, cos (θ) cos ((n − 1)θ) cos ((n − 2)θ) d θ = π 0 4 , for n > 2. 1 cos2 ((n − 2)θ) = (1 + cos (2(n − 2)θ)) , so 2 Z π 2 π, for n = 2, 1 cos ((n − 2)θ) d θ = π Cn = 4 . 0 2 , for n > 2.

I Induction on φn (x): With x = cos (θ), cos (θ) cos ((n − 1)θ) 1 cos ((n − 2)θ) φ (x) = − n 2n−2 4 2n−3 cos (n θ) + cos ((n − 2)θ) cos ((n − 2)θ) = − 2n−1 2n−1 cos (n θ) = . 2n−1

I Chebyshev Polynomials: T0 (x) = 1, T1 (x) = cos (θ), Tn (x) = cos (n θ) , n = 2, 3, ··· .

Chebyshev Polynomials: Compute Cn (II) R π cos (θ) cos ((n − 1)θ) cos ((n − 2)θ) d θ C = 0 , where n R π 2 2 0 cos ((n − 2)θ) d θ Z π 2 π, for n = 2, 1 cos ((n − 2)θ) d θ = π Cn = 4 . 0 2 , for n > 2.

I Chebyshev Polynomials: T0 (x) = 1, T1 (x) = cos (θ), Tn (x) = cos (n θ) , n = 2, 3, ··· .

Chebyshev Polynomials: Compute Cn (II) R π cos (θ) cos ((n − 1)θ) cos ((n − 2)θ) d θ C = 0 , where n R π 2 2 0 cos ((n − 2)θ) d θ 1 cos2 ((n − 2)θ) = (1 + cos (2(n − 2)θ)) , so 2 1 Cn = 4 .

I Chebyshev Polynomials: T0 (x) = 1, T1 (x) = cos (θ), Tn (x) = cos (n θ) , n = 2, 3, ··· .

Chebyshev Polynomials: Compute Cn (II) R π cos (θ) cos ((n − 1)θ) cos ((n − 2)θ) d θ C = 0 , where n R π 2 2 0 cos ((n − 2)θ) d θ 1 cos2 ((n − 2)θ) = (1 + cos (2(n − 2)θ)) , so 2 Z π 2 π, for n = 2, cos ((n − 2)θ) d θ = π 0 2 , for n > 2. I Induction on φn (x): With x = cos (θ), cos (θ) cos ((n − 1)θ) 1 cos ((n − 2)θ) φ (x) = − n 2n−2 4 2n−3 cos (n θ) + cos ((n − 2)θ) cos ((n − 2)θ) = − 2n−1 2n−1 cos (n θ) = . 2n−1

I Chebyshev Polynomials: T0 (x) = 1, T1 (x) = cos (θ), Tn (x) = cos (n θ) , n = 2, 3, ··· .

Chebyshev Polynomials: Compute Cn (II) R π cos (θ) cos ((n − 1)θ) cos ((n − 2)θ) d θ C = 0 , where n R π 2 2 0 cos ((n − 2)θ) d θ 1 cos2 ((n − 2)θ) = (1 + cos (2(n − 2)θ)) , so 2 Z π 2 π, for n = 2, 1 cos ((n − 2)θ) d θ = π Cn = 4 . 0 2 , for n > 2. I Chebyshev Polynomials: T0 (x) = 1, T1 (x) = cos (θ), Tn (x) = cos (n θ) , n = 2, 3, ··· .

I Induction on φn (x): With x = cos (θ), cos (θ) cos ((n − 1)θ) 1 cos ((n − 2)θ) φ (x) = − n 2n−2 4 2n−3 cos (n θ) + cos ((n − 2)θ) cos ((n − 2)θ) = − 2n−1 2n−1 cos (n θ) = . 2n−1 Chebyshev Polynomials: Compute Cn (II) R π cos (θ) cos ((n − 1)θ) cos ((n − 2)θ) d θ C = 0 , where n R π 2 2 0 cos ((n − 2)θ) d θ 1 cos2 ((n − 2)θ) = (1 + cos (2(n − 2)θ)) , so 2 Z π 2 π, for n = 2, 1 cos ((n − 2)θ) d θ = π Cn = 4 . 0 2 , for n > 2.

I Chebyshev Polynomials: T0 (x) = 1, T1 (x) = cos (θ), Tn (x) = cos (n θ) , n = 2, 3, ··· . Chebyshev Polynomials: T1 (x) through T4 (x) Chebyshev Polynomials: Zeros and extrema in [−1, 1]

Let x = cos (θ) , θ ∈ [0, π], T0 (x) = 1, and for n ≥ 1,

n−1 n Tn (x) = cos (nθ) = 2 x + lower order terms.

2k−1 I All zeros are simple and in (−1, 1): Tn cos 2n π = 0 for k = 1, ··· , n. k k I n + 1 local extrema on [−1, 1]: Tn cos n π = (−1) for k = 0, 1, ··· , n.

I Tn (x) is monotonic for |x| ≥ 1, 1 p n p n T (x) = x − x2 − 1 + x + x2 − 1 . n 2 k k Tn (x) Tn cos n π (−1) Proof: First we have ∈ Πen with = . 2n−1 2n−1 2n−1 1 Therefore min maxx∈[−1,1] |Pn (x)| ≤ . Pn(x)∈Πen 2n−1 1 Let Pn (x) ∈ Πen be a polynomial with maxx∈[−1,1] |Pn (x)| ≤ 2n−1 . T (x) Then Q (x) def= n − P (x) has degree at most n − 1, satisﬁes 2n−1 n k 1 k (−1)k Q cos π = − (−1)k P cos π ≥ 0. n 2n−1 n n Since 0 ≤ k ≤ n, Q (x) changes sign or reaches 0 at least n times, Tn(x) this can be true only if Q (x) ≡ 0, or Pn (x) = 2n−1 . Thus 1 max |P (x)| = . x∈[−1,1] n 2n−1

Chebyshev Polynomials: Min-Max Theorem Let Πen denote the set of all monic polynomials of degree n. Then

Tn (x) 1 min maxx∈[−1,1] |Pn (x)| = maxx∈[−1,1] = . Pn(x)∈Πen 2n−1 2n−1 1 Let Pn (x) ∈ Πen be a polynomial with maxx∈[−1,1] |Pn (x)| ≤ 2n−1 . T (x) Then Q (x) def= n − P (x) has degree at most n − 1, satisﬁes 2n−1 n k 1 k (−1)k Q cos π = − (−1)k P cos π ≥ 0. n 2n−1 n n Since 0 ≤ k ≤ n, Q (x) changes sign or reaches 0 at least n times, Tn(x) this can be true only if Q (x) ≡ 0, or Pn (x) = 2n−1 . Thus 1 max |P (x)| = . x∈[−1,1] n 2n−1

Chebyshev Polynomials: Min-Max Theorem Let Πen denote the set of all monic polynomials of degree n. Then

Tn (x) 1 min maxx∈[−1,1] |Pn (x)| = maxx∈[−1,1] = . Pn(x)∈Πen 2n−1 2n−1

k k Tn (x) Tn cos n π (−1) Proof: First we have ∈ Πen with = . 2n−1 2n−1 2n−1 1 Therefore min maxx∈[−1,1] |Pn (x)| ≤ . Pn(x)∈Πen 2n−1 Since 0 ≤ k ≤ n, Q (x) changes sign or reaches 0 at least n times, Tn(x) this can be true only if Q (x) ≡ 0, or Pn (x) = 2n−1 . Thus 1 max |P (x)| = . x∈[−1,1] n 2n−1

Chebyshev Polynomials: Min-Max Theorem Let Πen denote the set of all monic polynomials of degree n. Then

Tn (x) 1 min maxx∈[−1,1] |Pn (x)| = maxx∈[−1,1] = . Pn(x)∈Πen 2n−1 2n−1

k k Tn (x) Tn cos n π (−1) Proof: First we have ∈ Πen with = . 2n−1 2n−1 2n−1 1 Therefore min maxx∈[−1,1] |Pn (x)| ≤ . Pn(x)∈Πen 2n−1 1 Let Pn (x) ∈ Πen be a polynomial with maxx∈[−1,1] |Pn (x)| ≤ 2n−1 . T (x) Then Q (x) def= n − P (x) has degree at most n − 1, satisﬁes 2n−1 n k 1 k (−1)k Q cos π = − (−1)k P cos π ≥ 0. n 2n−1 n n Since 0 ≤ k ≤ n, Q (x) changes sign or reaches 0 at least n times, Tn(x) this can be true only if Q (x) ≡ 0, or Pn (x) = 2n−1 . Thus 1 max |P (x)| = . x∈[−1,1] n 2n−1 Since the set of polynomials of the form (Pn (x) − P (x)) /an is precisely Πen, it follows from Min-Max Theorem that T (x) (P (x) − P (x)) /a = n n n 2n−1 gives the best approximation. So the best approximation

a T (x) |a | P (x) = P (x) − n n , with |P (x) − P (x)| ≤ n . n−1 n 2n−1 n n−1 2n−1

Reducing the Degree of Approximating Polynomials Given an arbitrary nth-degree polynomial

n n−1 Pn (x) = an x + an−1 x + ··· a1 x + a0,

a (somewhat silly) problem is to ﬁnd a polynomial Pn−1 (x) of degree at most (n − 1) to best approximate Pn (x) on [−1, 1]: Pn−1 (x) = argminP(x)∈Πn−1 maxx∈[−1,1] |Pn (x) − P (x)| . Reducing the Degree of Approximating Polynomials Given an arbitrary nth-degree polynomial

n n−1 Pn (x) = an x + an−1 x + ··· a1 x + a0,

a (somewhat silly) problem is to ﬁnd a polynomial Pn−1 (x) of degree at most (n − 1) to best approximate Pn (x) on [−1, 1]: Pn−1 (x) = argminP(x)∈Πn−1 maxx∈[−1,1] |Pn (x) − P (x)| .

Since the set of polynomials of the form (Pn (x) − P (x)) /an is precisely Πen, it follows from Min-Max Theorem that T (x) (P (x) − P (x)) /a = n n n 2n−1 gives the best approximation. So the best approximation

a T (x) |a | P (x) = P (x) − n n , with |P (x) − P (x)| ≤ n . n−1 n 2n−1 n n−1 2n−1 Maclaurin Polynomial vs. Chebyshev Polynomial

Given function f (x) on [−1, 1], the nth-degree Maclaurin polynomial is n (j) def X f (0) P (x) = xj , n j! j=0

and the nth-degree Chebyshev approximation is

n def X Pbn (x) = αj Tj (x) , j=0

1 f (x) T (x) R √ j R π −1 2 d x f (cos (θ)) cos (j θ) d θ with α = 1−x = 0 . j 2 R π 2 1 Tj (x) cos (j θ) d θ R √ d x 0 −1 1−x2

§8.4 Rational Function Approximation

A rational function r of degree N has the form p (x) r (x) = , q (x)

where p (x) and q (x) are polynomials whose degrees sum to N. Pro Rational functions can better approximate a given function than polynomials. Con Denominator function q (x) may have un-wanted zeros to mess up the over-all approximation. I Must have q0 6= 0. Could set q0 = 1 or any non-zero value. I There must exist a continuous function h (x) such that

f (x) q (x) − p (x) xN+1 h (x) f (x) − r (x) = = , q (x) q (x)

i.e., f − r has a zero of multiplicity N + 1 at x = 0.

Pade´ Approximation (I)

Let r be a rational function of degree N = n + m of the form

n p (x) p0 + p1 x + ··· + pn x r (x) = = m . q (x) q0 + q1 x + ··· + qm x

Pade´ Approximation of given function f (x): Choose coeﬃcients p0, p1, ··· , pn and q0, q1, ··· , qm so that

f (k) (0) = r (k) (0) , k = 0, 1, ··· , N. Pade´ Approximation (I)

Let r be a rational function of degree N = n + m of the form

n p (x) p0 + p1 x + ··· + pn x r (x) = = m . q (x) q0 + q1 x + ··· + qm x

Pade´ Approximation of given function f (x): Choose coeﬃcients p0, p1, ··· , pn and q0, q1, ··· , qm so that

f (k) (0) = r (k) (0) , k = 0, 1, ··· , N.

I Must have q0 6= 0. Could set q0 = 1 or any non-zero value. I There must exist a continuous function h (x) such that

f (x) q (x) − p (x) xN+1 h (x) f (x) − r (x) = = , q (x) q (x)

i.e., f − r has a zero of multiplicity N + 1 at x = 0. Pade´ Approximation (II) P∞ i Assume Maclaurin series expansion f (x) = i=0 ai x . Let pj = 0 for j ≥ n + 1 and qk = 0 for k ≥ m + 1.

∞ !  ∞  ∞ ! X i X j X k f (x) q (x) − p (x) = ai x  qj x  − pk x i=0 j=0 k=0

∞    X X k N+1 =  ai qj  − pk  x = x h (x) . k=0 i+j=k

Pade´ equations:

min(k,m) X ak−j qj = pk , k = 0, 1, ··· n, j=0 min(k,m) X ak−j qj = 0, k = n + 1, ··· N. j=0 f (x) q (x) − p (x) f (x) − r (x) = q (x) P∞ Pm Pn ( i=0 ai Ti (x)) j=0 qj Tj (x) − ( k=0 pk Tk (x)) = q (x) P∞ Pn i,j=0 ai qj Ti (x) Tj (x) − ( k=0 pk Tk (x)) = , q (x) P∞ hk Tk (x) ===Goal k=N+1 . q (x)

Chebyshev Rational Function Approximation (I) P∞ Assume Chebyshev polynomial expansion f (x) = i=0 ai Ti (x). Choose Chebyshev polynomial expressions for p (x) and q (x) n ∞ X def X p (x) = pk Tk (x) = pk Tk (x) , k=0 k=0 m ∞ X def X q (x) = qj Tj (x) = qj Tj (x) , so that j=0 j=0 f (x) q (x) − p (x) f (x) − r (x) = q (x) P∞ Pm Pn ( i=0 ai Ti (x)) j=0 qj Tj (x) − ( k=0 pk Tk (x)) = q (x) P∞ Pn i,j=0 ai qj Ti (x) Tj (x) − ( k=0 pk Tk (x)) = , q (x) Chebyshev Rational Function Approximation (I) P∞ Assume Chebyshev polynomial expansion f (x) = i=0 ai Ti (x). Choose Chebyshev polynomial expressions for p (x) and q (x) n ∞ X def X p (x) = pk Tk (x) = pk Tk (x) , k=0 k=0 m ∞ X def X q (x) = qj Tj (x) = qj Tj (x) , so that j=0 j=0 f (x) q (x) − p (x) f (x) − r (x) = q (x) P∞ Pm Pn ( i=0 ai Ti (x)) j=0 qj Tj (x) − ( k=0 pk Tk (x)) = q (x) P∞ Pn i,j=0 ai qj Ti (x) Tj (x) − ( k=0 pk Tk (x)) = , q (x) P∞ hk Tk (x) ===Goal k=N+1 . q (x) Chebyshev rational function equations: m X 2 p0 = 2 a0 q0 + aj qj , j=1 min(k,m) m m−k X X X 2 pk = ak−j qj + aj+k qj + ai qi+k , 1 ≤ k ≤ n, j=0 j=0 i=0 min(k,m) m m−k X X X 0 = ak−j qj + aj+k qj + ai qi+k , n + 1 ≤ k ≤ N. j=0 j=0 i=0

Chebyshev Rational Function Approximation (II) 1 Chebyshev polynomial identity: T (x) T (x) = T (x) + T (x). i j 2 i+j |i−j| ∞  m  X 1 X a q T (x) T (x) = a q + a q T (x) i j i j  0 0 2 j j  0 i,j=0 j=1

∞    ∞  ∞ ! 1 X X X X + a q + a q + a q T (x) . 2  i j   j+k j  i i+k  k k=1 i+j=k j=0 i=0 Chebyshev Rational Function Approximation (II) 1 Chebyshev polynomial identity: T (x) T (x) = T (x) + T (x). i j 2 i+j |i−j| ∞  m  X 1 X a q T (x) T (x) = a q + a q T (x) i j i j  0 0 2 j j  0 i,j=0 j=1

∞    ∞  ∞ ! 1 X X X X + a q + a q + a q T (x) . 2  i j   j+k j  i i+k  k k=1 i+j=k j=0 i=0 Chebyshev rational function equations: m X 2 p0 = 2 a0 q0 + aj qj , j=1 min(k,m) m m−k X X X 2 pk = ak−j qj + aj+k qj + ai qi+k , 1 ≤ k ≤ n, j=0 j=0 i=0 min(k,m) m m−k X X X 0 = ak−j qj + aj+k qj + ai qi+k , n + 1 ≤ k ≤ N. j=0 j=0 i=0

§8.5 Trigonometric Polynomial Approximation 2 n I {φk (x)}k=0 orthogonal on [−π, π] for weight function w (x) ≡ 1. Z π φk (x) φj (x) d x = 0 for all k 6= j. i.e., −π Z π 1 Z π 1 cos (k x) d x = sin (k x) d x = 0, −π 2 −π 2 Z π 1 Z π cos (k x) cos (j x) d x = (cos ((k + j) x) + cos ((k − j) x)) d x =0, −π 2 −π Z π 1 Z π sin (k x) sin (j x) d x = (cos ((k − j) x) − cos ((k + j) x)) d x =0, −π 2 −π Z π 1 Z π sin (k x) cos (j x) d x = (sin ((k + j) x) + sin ((k − j) x)) d x =0. −π 2 −π

I Trigonometric polynomials of degree ≤ n: 1 φ (x) = , φ (x) = cos (k x) , for k = 1, ··· , n, 0 2 k φn+k (x) = sin (k x) , for k = 1, ··· , n. Z π 1 Z π cos (k x) cos (j x) d x = (cos ((k + j) x) + cos ((k − j) x)) d x =0, −π 2 −π Z π 1 Z π sin (k x) sin (j x) d x = (cos ((k − j) x) − cos ((k + j) x)) d x =0, −π 2 −π Z π 1 Z π sin (k x) cos (j x) d x = (sin ((k + j) x) + sin ((k − j) x)) d x =0. −π 2 −π

I Trigonometric polynomials of degree ≤ n: 1 φ (x) = , φ (x) = cos (k x) , for k = 1, ··· , n, 0 2 k φn+k (x) = sin (k x) , for k = 1, ··· , n.

2 n I {φk (x)}k=0 orthogonal on [−π, π] for weight function w (x) ≡ 1. Z π φk (x) φj (x) d x = 0 for all k 6= j. i.e., −π Z π 1 Z π 1 cos (k x) d x = sin (k x) d x = 0, −π 2 −π 2 Z π 1 Z π sin (k x) sin (j x) d x = (cos ((k − j) x) − cos ((k + j) x)) d x =0, −π 2 −π Z π 1 Z π sin (k x) cos (j x) d x = (sin ((k + j) x) + sin ((k − j) x)) d x =0. −π 2 −π

I Trigonometric polynomials of degree ≤ n: 1 φ (x) = , φ (x) = cos (k x) , for k = 1, ··· , n, 0 2 k φn+k (x) = sin (k x) , for k = 1, ··· , n.

2 n I {φk (x)}k=0 orthogonal on [−π, π] for weight function w (x) ≡ 1. Z π φk (x) φj (x) d x = 0 for all k 6= j. i.e., −π Z π 1 Z π 1 cos (k x) d x = sin (k x) d x = 0, −π 2 −π 2 Z π 1 Z π cos (k x) cos (j x) d x = (cos ((k + j) x) + cos ((k − j) x)) d x =0, −π 2 −π Z π 1 Z π sin (k x) sin (j x) d x = (cos ((k − j) x) − cos ((k + j) x)) d x =0, −π 2 −π I Trigonometric polynomials of degree ≤ n: 1 φ (x) = , φ (x) = cos (k x) , for k = 1, ··· , n, 0 2 k φn+k (x) = sin (k x) , for k = 1, ··· , n.

2 n I {φk (x)}k=0 orthogonal on [−π, π] for weight function w (x) ≡ 1. Z π φk (x) φj (x) d x = 0 for all k 6= j. i.e., −π Z π 1 Z π 1 cos (k x) d x = sin (k x) d x = 0, −π 2 −π 2 Z π 1 Z π cos (k x) cos (j x) d x = (cos ((k + j) x) + cos ((k − j) x)) d x =0, −π 2 −π Z π 1 Z π sin (k x) sin (j x) d x = (cos ((k − j) x) − cos ((k + j) x)) d x =0, −π 2 −π Z π 1 Z π sin (k x) cos (j x) d x = (sin ((k + j) x) + sin ((k − j) x)) d x =0. −π 2 −π I Given function f ∈ C[−π, π], continuous least squares approximation by trigonometric polynomials: Z π 2 minSn(x) (f (x) − Sn (x)) d x, where −π

n a0 X S (x) = + (a cos (k x) + b sin (k x)) . n 2 k k k=1

I Orthogonal trigonometric polynomials: 1 φ (x) = , φ (x) = cos (x) , for k = 1, ··· , n, 0 2 k φn+k (x) = sin (x) , for k = 1, ··· , n. I Orthogonal trigonometric polynomials: 1 φ (x) = , φ (x) = cos (x) , for k = 1, ··· , n, 0 2 k φn+k (x) = sin (x) , for k = 1, ··· , n.

I Given function f ∈ C[−π, π], continuous least squares approximation by trigonometric polynomials: Z π 2 minSn(x) (f (x) − Sn (x)) d x, where −π

n a0 X S (x) = + (a cos (k x) + b sin (k x)) . n 2 k k k=1 Z π 2 (f (x) − Sn (x)) d x −π 2 Z π n !! a0 X = f (x) − + (a cos (k x) + b sin (k x)) d x 2 k k −π k=1 Z π Z π n ! def a0 X =∆ + f 2 (x) d x − 2 f (x) + (a cos (k x) + b sin (k x)) d x, 2 k k −π −π k=1

2 Z π n ! a0 X where ∆= + (a cos (k x) + b sin (k x)) d x 2 k k −π k=1 Z π n Z π Z π a0 2 X = d x + a2 cos2 (k x) d x + b2 sin2 (k x) d x 2 k k −π k=1 −π −π n ! a2 X =π 0 + a2 + b2 . 2 k k k=1 n ! Z π a2 X (f (x) − S (x))2 d x = π 0 + a2 + b2 n 2 k k −π k=1 Z π Z π n ! a0 X + f 2 (x) d x − 2 f (x) + (a cos (k x) + b sin (k x)) d x, 2 k k −π −π k=1 Z π 2 Optimal solution to minSn(x) (f (x) − Sn (x)) d x : −π

1 Z π ak = f (x) cos (k x) d x, for k = 0, 1, ··· , n, π −π 1 Z π bk = f (x) sin (k x) d x, for k = 1, ··· , n. π −π Ex: Approximating f (x) = |x| on [−π, π] (I)

1 Z π 2 Z π a0 = |x| d x = x d x = π, π −π π 0 1 Z π 2 Z π ak = |x| cos (k x) d x = x cos (k x) d x π −π π 0 2 = (−1)k − 1 , for k = 1, ··· , n, π k2 1 Z π bk = |x| sin (k x) d x = 0, for k = 1, ··· , n. π −π Trigonometric approximation of degree n

n π 2 X (−1)k − 1 S (x) = + cos (k x) . n 2 π k2 k=1 Ex: Approximating f (x) = |x| on [−π, π] (II) Goal: to determine best trigonometric polynomial 2 m 2 m !!2 X α0 X min y − φ (x ) + α φ (x ) . α0,α1,··· ,α2m j 2 0 j k k j j=0 2 m 2 m k!!=1 2 X α0 X y − φ (x ) + α φ (x ) j 2 0 j j k j j=0 k=1  2 m   2 m 2 m ! X X α0 X = y 2 − 2 y φ (x ) + α φ (x )  j   j 2 0 j k k j  j=0 j=0 k=1 2 m 2 m !2 X α0 X + φ (x ) + α φ (x ) 2 0 j k k j j=0 k=1

Discrete Trigonometric Least Squares j − m Given: Speciﬁc data pairs {(x , y )}2 m−1, with x = π for j j j=0 j m j = 0, 1, ··· , 2 m − 1. With basis functions

φ0 (x) = 1, φk (x) = cos (x) , for k = 1, ··· , n,

φn+k (x) = sin (x) , for k = 1, ··· , n. Discrete Trigonometric Least Squares j − m Given: Speciﬁc data pairs {(x , y )}2 m−1, with x = π for j j j=0 j m j = 0, 1, ··· , 2 m − 1. With basis functions

φ0 (x) = 1, φk (x) = cos (x) , for k = 1, ··· , n,

φn+k (x) = sin (x) , for k = 1, ··· , n. Goal: to determine best trigonometric polynomial 2 m 2 m !!2 X α0 X min y − φ (x ) + α φ (x ) . α0,α1,··· ,α2m j 2 0 j k k j j=0 2 m 2 m k!!=1 2 X α0 X y − φ (x ) + α φ (x ) j 2 0 j j k j j=0 k=1  2 m   2 m 2 m ! X X α0 X = y 2 − 2 y φ (x ) + α φ (x )  j   j 2 0 j k k j  j=0 j=0 k=1 2 m 2 m !2 X α0 X + φ (x ) + α φ (x ) 2 0 j k k j j=0 k=1 P2 m−1 Discrete Orthogonality: j=0 φk (xj ) φr (xj ) = 0 for k 6= r j − m Discrete data points x = π for j = 0, 1, ··· , 2 m − 1. j m 2 m−1 X cos (r xj ) = 0, r = 1, ··· , m, j=0 2 m−1 X sin (r xj ) = 0, r = 1, ··· , m, j=0 2 m−1 X cos (r xj ) cos (k xj ) = 0, r 6= k, r, k = 1, ··· , m, j=0 2 m−1 X sin (r xj ) sin (k xj ) = 0, r 6= k, r, k = 1, ··· , m, j=0 2 m−1 X cos (r xj ) sin (k xj ) = 0, r, k = 1, ··· , m. j=0 2 m−1 2 m−1 X √ √ X √ r π = exp −1 r x = exp − −1 r π exp −1 j j m j=0 j=0 √ √ 1 − exp −1 r π 2 m √ m = exp − −1 r π r π = 0, therefore 1 − exp −1 m 2 m−1 2 m−1 X X cos (r xj ) = sin (r xj ) = 0. 2 m−1 j=0 j=0 X cos (r xj ) cos (k xj ) j=0 2 m−1  2 m−1  1 X X = cos ((r + k) x ) + cos ((r − k) x ) = 0. 2  j   j  j=0 j=0

j − m Proof of Orthogonality: x = π for j = 0, 1, ··· , 2 m − 1. j m 2 m−1  2 m−1  X √ X  cos (r xj ) + −1  sin (r xj ) j=0 j=0 2 m−1 2 m−1 X X cos (r xj ) = sin (r xj ) = 0. 2 m−1 j=0 j=0 X cos (r xj ) cos (k xj ) j=0 2 m−1  2 m−1  1 X X = cos ((r + k) x ) + cos ((r − k) x ) = 0. 2  j   j  j=0 j=0

j − m Proof of Orthogonality: x = π for j = 0, 1, ··· , 2 m − 1. j m 2 m−1  2 m−1  X √ X  cos (r xj ) + −1  sin (r xj ) j=0 j=0 2 m−1 2 m−1 X √ √ X √ r π = exp −1 r x = exp − −1 r π exp −1 j j m j=0 j=0 √ √ 1 − exp −1 r π 2 m √ m = exp − −1 r π r π = 0, therefore 1 − exp −1 m 2 m−1 2 m−1 X X cos (r xj ) = sin (r xj ) = 0. j=0 j=0 j − m Proof of Orthogonality: x = π for j = 0, 1, ··· , 2 m − 1. j m 2 m−1  2 m−1  X √ X  cos (r xj ) + −1  sin (r xj ) j=0 j=0 2 m−1 2 m−1 X √ √ X √ r π = exp −1 r x = exp − −1 r π exp −1 j j m j=0 j=0 √ √ 1 − exp −1 r π 2 m √ m = exp − −1 r π r π = 0, therefore 1 − exp −1 m 2 m−1 2 m−1 X X cos (r xj ) = sin (r xj ) = 0. 2 m−1 j=0 j=0 X cos (r xj ) cos (k xj ) j=0 2 m−1  2 m−1  1 X X = cos ((r + k) x ) + cos ((r − k) x ) = 0. 2  j   j  j=0 j=0 Discrete Trigonometric Least Squares

2 m−1 2 m !!2 X α0 X y − φ (x ) + α φ (x ) j 2 0 j k k j j=0 k=1 2 m−1  2 m−1 2 m ! X X α0 X = y 2 − 2 y φ (x ) + α φ (x ) + ∆,  j   j 2 0 j k k j  j=0 j=0 k=1

2 m−1 2 m !2 X α0 X where ∆ = φ (x ) + α φ (x ) 2 0 j k k j j=0 k=1 2 m−1 2 m 2 m−1  α0 2 X X X = φ2 (x ) + α2 φ (x )2 2 0 j k  k j  j=0 k=1 j=0 2 m ! α2 X = m 0 + α2 2 k k=1 Discrete Trigonometric Least Squares

2 m−1 2 m !!2 X α0 X y − φ (x ) + α φ (x ) j 2 0 j k k j j=0 k=1 2 m−1  2 m−1 2 m ! X X α0 X = y 2 − 2 y φ (x ) + α φ (x )  j   j 2 0 j k k j  j=0 j=0 k=1 2 m ! α2 X +m 0 + α2 . 2 k k=1 Minimize least squares error, 2 m−1 1 X α = y φ (x ) , k = 0, 1, ··· , 2 m, With k m j k j j=0

φ0 (x) = 1, φk (x) = cos (x) , for k = 1, ··· , m,

φm+k (x) = sin (x) , for k = 1, ··· , m. I Fourier analysis by Fourier, 1807

FFT History

I First FFT algorithm by Gauss, 1805 FFT History

I First FFT algorithm by Gauss, 1805

I Fourier analysis by Fourier, 1807 FFT History

I Cooley and Tukey FFT algorithm, 1965