Fourier Series

Introduction

One important application of linear algebra is in the representation and approximation of functions. For example, suppose E(t) is a function that represents the amount of acoustical energy received by a microphone at time t. Since sound energy is in the form of sound waves of varying frequencies, it seems plausible to represent E(t) as a linear combination of simple waves. In mathematics, simple waves are represented using sine and cosine functions of the form sin(kt) and cos(kt), where k is a nonnegative integer, and the greater the k value, the higher the wave frequency. Then

E(t) = c0 + c1 cos(t) + ··· + cn cos(nt) + d1 sin(t) + ··· + dn sin(nt), where c0, . . . , cn, d1, . . . , dn are constant coefficients, and n is a nonnegative integer tht is sufficiently large enough to capture all of the frequencies that comprise E(t). The expression on the right is called a trigonometric polynomial of order n (assuming cn 6= 0 and dn 6= 0). A trigonometric- polynomial representation of E(t) seems quite useful, since it alllows one to determine the dominant acoustical frequencies that comprise the energy function. These frequencies will correspond with values of k for which either ck or dk seems substantial. But how to determine the values of these coefficients?

Recall that the span of the set of wave functions

{1, sin(t), cos(t), sin(2t), cos(2t),..., sin(nt), cos(nt)} forms a subspace Wn of the inner product space C[0, 2π], the set of continuous functions over [0, 2π]. Recall also that C[0, 2π] admits the inner product

Z 2π = f(t)g(t)dt. 0 Moreover, the wave functions are pairwise orthogonal, and can be normalized to form an orthonormal basis.

1 Example 1. Let || cos(kt)|| denote the length of cos(kt) as vector in C[0, 2π]. Show that √ || cos(kt)|| = || sin(kt)|| = π, √ while ||1|| = 2π.

From the lecture on inner product spaces, to obtain the best approximation of E(t) as a linear combination of wave functions (i.e. as an element of Wn), one must project E(t) on to Wn. Furthermore, the coeﬃcient ck can be computed as 1 1 1 Z 2π ck = √ = E(t) cos(kt)dt. π π π 0

√1 √1 Note that the π outside the inner product arises from the fact that the basis vector is ek = π cos(kt). Moreover, when projecting E(t) on to Wn, the inner product must be multiplied with ek, √1 which accounts for the extra π . Similarly, 1 Z 2π dk = E(t) sin(kt)dt. π 0

Each ck and dk is called a Fourier coeﬃcient of E(t), and the series ∞ ∞ X X c0 + ck cos(kt) + dk sin(kt) k=1 k=1 is called the Fourier series of E(t). Notice that the above expression actually represents two series (one of cosines and one of sines), but we will think of it as a single series.

In summary, the projection theorem states that the best approximation of E(t) as an element of Wn is proj(E(t), Wn). But how accurate is this approximation? In other words, what will be the size ⊥ of the vector wn = E(t) − proj(E(t), Wn)? It turns out that we can make some fairly conservative assumptions about E(t) that guarantee that

⊥ limit||wn || = 0, n→∞

2 meaning that, as n increases, proj(E(t), Wn) becomes an increasingly better approximation of E(t). The following theorems are stated without proof.

Theorem 1. If E(t) is continuous on [0, 2π], then

limit||E(t) − proj(E(t), Wn)|| = 0. n→∞

Theorem 2. If E(t) is continuous on [0, 2π], then the Fourier series

∞ ∞ X X c0 + ck cos(kt) + dk sin(kt) k=1 k=1 converges uniformly to E(t). In other words, for any > 0, there exists a suﬃciently large n, independent of t, for which ||E(t) − proj(E(t), Wn)|| < .

In general, whenever a vector v of some inner product space is approximated by its projection on to a subspace W, then ||w⊥||2 = ||v − proj(v, W)||2 is called the sum squared error (sse) of the approximation. In particular, if the inner product space is C[a, b], then the sse is computed as

Z b sse = (f(x) − proj(f, W))2dx. a

Example 2. Find the best approximation of f(x) = x over the subspace spanned by {1, ex} and compute the sum squared error.

3 At this point the reader may wonder why both sines and cosines are used in the Fourier series. After all, sin(t) = cos(t − π/2). The reason is that, when E(t) is measured at time t = 0, the simple waves that comprise E(t) will likely be in diﬀerent phases of their cycles. For example, at t = 0, cos(t) is in the phase of being at its peak, where as cos(t + π/3) is n the phase of being 1/2 the distance to its highest peak. Therefore, a simple wave should have the general form A cos(kt + φ), where π is called the phase angle of the wave, and A is called its amplitude. Now, using the identity, cos(x + y) = cos(x) cos(y) − sin(x) sin(y) yields A cos(kt + φ) = A cos(kt) cos(φ) − A sin(kt) sin(φ). Therefore, A cos(kt+φ) can be written as a linear combination of cos(kt) and sin(kt), which explains why both sines and cosines are needed in the Fourier series. Of course, the advantage in using both sines and cosines lies in the fact that all the simple waves have the same phase angle of φ = 0.

Example 3. Write 4 cos(3t + π/4) as a linear combination of simple waves, each with phase angle φ = 0.

2π Another valid concern is the assumption that the simple waves of E(t) have periods of k , where k is a positive integer. This is a somewhat limited set of periods. Suppose in practice that there are a finite number n of significant (in the sense of having a sufficiently large amplitude) simple waves of E(t), and that their respective periods are T1,...,Tn, where each Ti is a rational number. Let T be the least common multiple of T1,...,Tn, meaning that T is the least rational number for which T/Ti is a positive integer, for every i = 1, . . . , n. Note that T is called the fundamental period. Then measuring E(t) over the interval [0,T ] will completely capture at least one period of each of its simple waves, and 2πt 2πt 2πt 2πt E(t) = c0 + c1 cos( ) + ··· + cn cos( ) + d1 sin( ) + ··· + dn sin( ) T1 Tn T1 Tn over the interval [0,T ].

To analyze E(t) over the preferred period of [0, 2π], we deﬁne a linear function s(t) of time so that 0 maps to 0, and T maps to 2π. Thus s(t) = 2πt/T . For example, if T = 10, and for t = 5 we measure

4 E(5) = 7, then on the s time scale this would correspond to s = (2π5)/10 = π, and we would write E(π) = 7. Thus, when measuring E on the s time scale, the fundamental period changes from T to 2π. Moreover, if 2πt 2πt 2πt 2πt E(t) = c0 + c1 cos( ) + ··· + cn cos( ) + d1 sin( ) + ··· + dn sin( ), T1 Tn T1 Tn

T s then the formula for E on the s time scale can be obtained by using the fact that t = 2π . Thus T s substituting 2π for t in the above formula for E(t) yields

E(s) = c0 + c1 cos(k1s) + ··· + cn cos(kns) + d1 sin(k1s) + ··· + dn sin(kns), where ki = T/Ti is an integer. Therefore, for given E(t), we may ﬁrst derive a Fourier series for E(s) on the s time scale, with fundamental period 2π, and then derive E(t) by substituting s with 2πt/T to yield a Fourier series with fundamental period T .

Example 4. If −3 cos(πt/4) is a term of a Fourier series with fundamental frequency T = 24, then re-write this term if using the s time scale with fundamental period 2π. Conversely, if 1.5 sin(10s) is a term of a Fourier series over an s time scale with fundamental period 2π, re-write this term using the t time scale with a fundamental period 30.

The remainder of the lecture will assume a fundamental period of 2π so that we may exclusively use the orthonormal basis 1 1 1 1 1 {√ , √ cos(t), √ cos(2t),..., √ sin(t), √ sin(2t)}. 2π π π π π

Given function E(t), the n th-order Fourier approximation of E(t) is deﬁned as

F (E, n, t) = c0 + c1 cos(t) + ··· + cn cos(nt) + d1 sin(t) + ··· + dn sin(nt), where 1 Z 2π c0 = E(t)dt, 2π 0

5 and for k = 1, . . . , n 1 Z 2π ck = E(t) cos(kt)dt, π 0 and 1 Z 2π dk = E(t) sin(kt)dt. π 0

Example 5. Find the 2nd-order Fourier approximation of E(t) = t. Provide a general formula for the n th-order Fourier approximation of E(t) = t.

Example 5 Solution. 1 Z 2π 1 4π2 c0 = tdt = ( ) = π. 2π 0 2π 2

Using integration by parts,

Z 2π 2π Z 2π t sin(kt) 1 πck = t cos(kt)dt = − sin(kt)dt = 0. 0 k 0 k 0

Using integration by parts again,

Z 2π 2π Z 2π −t cos(kt) 1 −2π πdk = t sin(kt)dt = + cos(kt)dt = . 0 k 0 k 0 k

Hence, dk = −2/k. Therefore,

n Xsin(kt) F (E(t), n, t) = π − 2 , k k=1 and the 2nd-order Fourier approximation of E(t) = t is

F (E(t), 2, t) = π − 2 sin t − sin(2t).

Exercises

1. Prove that the ﬁrst-order Fourier approximation of E(t) = cos(kt) is equal to cos(kt). In other words, F (cos(kt), 1, t) = cos(kt).

2. Write 10 cos(2t + π/3) as a linear combination of simple waves, each with phase angle φ = 0.

3. Write −2 cos(5t + π/6) as a linear combination of simple waves, each with phase angle φ = 0.

4. Write −2 sin(5t + π/6) as a linear combination of simple waves, each with phase angle φ = 0.

6 5. If 2 cos(2πt) is a term of a Fourier series with fundamental period T = 5, then re-write this term using an s time scale with fundamental period 2π. Conversely, if 4 cos(20s) is a term of the Fourier series over an s time scale with fundamental period 2π, then re-write this term using the t time scale with a fundamental period of 5. 6. If −6 sin(3πt/2) is a term of a Fourier series with fundamental period T = 16, then re-write this term using an s time scale with fundamental period 2π. Conversely, if 10 cos(3s) is a term of the Fourier series over an s time scale with fundamental period 2π, then re-write this term using the t time scale with a fundamental period of 15. 7. Find the 2nd-order Fourier approximation of E(t) = 1 + t. Provide the n th-order Fourier approximation of E(t) = 1 + t. 8. Find the 2nd-order Fourier approximation of E(t) = t2. Provide the n th-order Fourier approximation of E(t) = t2. 9. Find the projection of f(x) = x2 on to the subspace W of C[0, 1] that is spannned by {1, ex}. Provide (but do not evaluate) an expression that gives the sum squared error when approximating f(x) by this projection. 10. Find the projection of f(x) = ex on to the subspace W of C[0, 1] that is spannned by {1, x}. Determine the sum squared error when approximating f(x) by this projection. 11. Given f(x) that has derivatives of all orders, its Maclaurin series is deﬁned as f 0(0) f 00(0) f [n](0) f(0) + x + x2 + ··· + xn + ··· . 1! 2! n! Moreover, its nth-order Maclaurin polynomial is deﬁned as f 0(0) f 00(0) f [n](0) f(0) + x + x2 + ··· + xn. 1! 2! n! Determine the 1st-order Maclaurin polynomial of f(x) = ex. Show that the sse when approximating ex by this polynomial is worse than the sse from Exercise 10. 12. Find the projection of f(x) = sin πx on to the subspace W of C[−1, 1] that is spannned by {1, x, x2}. Determine ||w⊥||2 = ||f − proj(f, W)||2. 13. Determine the 2nd-order Maclaurin polynomial of f(x) = sin πx and determine the sse when approximating f(x) with this polynomial. Compare the sse to the sse of the previous problem. 14. Provide the Fourier series of f(x) = π − x.

Exercise Solutions

1. Since cos(kt) is orthogonal to sin(mt) for all m > 0 and orthogonal to cos(mt) for m 6= k, it follows that the Fourier series of cos(kt) equals 1 1 1 < cos(kt), √ cos(kt)>√ cos(kt) = < cos(kt), cos(kt)> cos(kt). π π π

7 But Z 2π < cos(kt), cos(kt)> = cos2(kt)dt = π. 0 Therefore, the Fourier series of cos(kt) is equal to cos(kt).

2. We have √ 10 cos(2t + π/3) = 10 cos(2t) cos(π/3) − 10 sin(2t) sin(π/3) = 5 cos(2t) − 5 3 sin(2t).

3. We have √ −2 cos(5t + π/6) = −2 cos(5t) cos(π/6) + 2 sin(5t) sin(π/6) = − 3 cos(5t) + sin(5t).

4. We have √ −2 sin(5t + π/6) = −2 sin(5t) cos(π/6) − 2 cos(5t) sin(π/6) = − 3 sin(5t) − cos(5t).

5. Since T = 5 is the fundamental period on the t time scale, we have s = 2πt/5 which implies t = 5s/(2π). Therefore, on the s time scale 2 cos(2πt) is expressed as 2 cos(5s). Conversely, 4 cos(20s) is expressed on the t time scale as 4 cos(8π).

6. Since T = 16 is the fundamental period on the t time scale, we have s = πt/8 which implies t = 8s/π. Therefore, on the s time scale −6 sin(3πt/2) is expressed as −6 sin(12s). Conversely, 10 cos(3s) is expressed on the t time scale as 10 cos(3πt/8).

7. 1 Z 2π 1 4π2 c0 = (1 + t)dt = (2π + ) = 1 + π. 2π 0 2π 2 Using integration by parts,

Z 2π 2π Z 2π (1 + t) sin(kt) 1 πck = (1 + t) cos(kt)dt = − sin(kt)dt = 0. 0 k 0 k 0

Using integration by parts again,

Z 2π 2π Z 2π −(1 + t) cos(kt) 1 −(2π + 1) πdk = (1 + t) sin(kt)dt = + cos(kt)dt = . 0 k 0 k 0 k

Hence, dk = −2/k + 1/(πk). Therefore,

n 1 Xsin(kt) F (1 + t, n, t) = 1 + π − (2 + ) , π k k=1 and the 2nd-order Fourier approximation of E(t) = t is 1 1 F (1 + t, 2, t) = 1 + π − (2 + ) sin t − (1 + ) sin(2t). π 2π

8 8. 1 Z 2π 1 8π3 4π2 c0 = tdt = ( = . 2π 0 2π 3 3 Using integration by parts,

Z 2π 2 2π Z 2π 2 t sin(kt) 2 4π πck = t cos(kt)dt = − t sin(kt)dt = 2 . 0 k 0 k 0 k

4 Hence, ck = k2 . Using integration by parts again,

Z 2π 2 2π Z 2π 2 2 −t cos(kt) 2 −4π πdk = t sin(kt)dt = + t cos(kt)dt = . 0 k 0 k 0 k

−4π Hence, dk = k . Therefore, n n 4π2 Xcos(kt) Xsin(kt) F (t2, n, t) = + 4 − 4π , 3 k2 k k=1 k=1 and the 2nd-order Fourier approximation of E(t) = t2 is 4π2 F (t2, 2, t) = + 4 cos t + cos(2t) − 4 sin t − 2 sin(2t). 3

√1 x −1 2 9. An orthonormal basis for W is {1, α (e − e + 1)}, where α = 2 e + 2e − 3/2. Moreover,

Z 1 3 1 2 x x dx = = 1/3, 0 3 0 and Z 1 Z 1 Z 1 x2(ex − e + 1)dx = x2exdx + (1 − e) x2dx = 0 0 0 Z 1 (1 − e) x2exdx + . 0 3 Using integration by parts, Z 1 Z 1 2 x 2 x 1 x x e dx = x e 0 − 2 xe dx = e − 2. 0 0 Thus, Z 1 (1 − e) 2e x2(ex − e + 1)dx = e − 2 + = − 5/3. 0 3 3 Therefore, (2e/3 − 5/3)(ex − e + 1) proj(x2, W) = 1/3 + . −1 2 2 e + 2e − 3/2 and Z 1 (2e/3 − 5/3)(ex − e + 1) sse = (x2 − 1/3 − )2dx. −1 2 0 2 e + 2e − 3/2

9 √ 10. An orthonormal basis for W is {1, 2 3(x − 1/2)}. Then

Z 1 x x 1 e dx = e |0 = e − 1. 0 Also, Z 1 Z 1 (x − 1/2)exdx = xexdx − e/2 + 1/2. 0 0 But, using integration by parts, Z 1 xexdx = 1. 0 Thus, Z 1 (x − 1/2)exdx = 1 − e/2 + 1/2 = (3 − e)/2. 0 Therefore,

proj(ex, W) = e − 1 + ((12(3 − e)/2)(x − 1/2) = (18 − 6e)x + (4e − 10).

Moreover, Z 1 sse = (ex − (ax + b))2dx, 0 where a = 18−6e, and b = 4e−10. The above integral can be obtained by adding the following integrals Z 1 e2xdx = e2/2 − 1/2, 0

Z 1 −2a xexdx = 12e − 36, 0

Z 1 −2b exdx = (20 − 8e)(e − 1), 0

Z 1 a2 x2dx = (18 − 6e)2/3, 0

Z 1 ab 2xdx = (18 − 6e)(4e − 10), 0 and

Z 1 b2 1dx = (4e − 10)2. 0 Adding the above integrals yields −7 sse = e2 + 20e − 57/2. 2

10 11. Since the derivatives of ex at zero are all one, it follows that the ﬁrst-order Maclaurin series of ex is equal to x + 1. Moreover,

Z 1 1 sse = (ex − (x + 1))2dx = e2 − 2e + 11/6. 0 2 The diﬀerence between the sse from Exercise 10 and the above sse is equal to

4e2 − 22e + 91/3 = 0.087.

Therefore, as expected, the sse is worse than optimal, but not by much!

12. The projection of sin πx on to the subspace spanned by {1, x, x2} is 3 proj(sin πx, W) = x, π

6 with sse 1 − π2 . 13. πx n P 2 14. k sin(kt). k=1