The Fourier Transform Introduction • Orthonormal bases for Rn • – Inner product – Length – Orthogonality – Change of basis – Matrix transpose Complex vectors • Orthonormal bases for Cn • – Inner product – Hermitian transpose Orthonormal bases for 2π periodic functions • – Shah basis – Harmonic signal basis – Fourier series Fourier transform • Orthonormal bases for Rn

T T Let u = [u1,u2] and v = [v1,v2] be vectors in R2. We define the inner product of u and v to be u,v = u1v1 + u2v2. h i We can use the inner product to define notions of length and angle. The length of u is given by the square root of the inner product of u with itself: 1 u = u,u 2 | | h i 2 2 = u1 + u2. The angle between u andq v can also be defined in terms of inner product: u,v = u v cosθ h i | || | where 1 u,v θ = cos− h i . u v | || | Orthogonality An important special case occurs when u,v = u v cosθ = 0. h i | || | When cosθ equals zero, θ = π/2 = 90◦. Orthonormal bases for Rn Any n orthogonal vectors which are of unit length 1 if i = j ui,u j = h i 0 otherwise.  form an orthonormal basis for Rn. Any vector n in R can be expressed as a weighted sum of u1, u2, u3,...,un:

v = w1u1 + w2u2 + w3u3 + ... + wnun.

Question How do we find w1, w2, w3,...,wn? • Answer Using inner product. • Example Consider two orthonormal bases. The first basis 1 is defined by the vectors u = and u = 1 0 2   0 . It is easy to verify that these two vectors 1 form  an orthonormal basis: 1 0 , = 1 0 + 0 1 = 0 0 1 · ·     1 1 , = 1 1 + 0 0 = 1 0 0 · ·     0 0 , = 0 0 + 1 1 = 1. 1 1 · ·     Example (contd.)

cosθ The second, by the vectors u = and 10 sinθ   sinθ u = . It is also easy to verify that 20 −cosθ these two vectors form an orthonormal basis: cosθ sinθ , − = cosθsinθ + cosθsinθ = 0 sinθ cosθ −     cosθ cosθ , = cos2 θ + sin2 θ = 1 sinθ sinθ     sinθ sinθ , = cos2 θ + sin2 θ = 1. −cosθ −cosθ     Example (contd.)

Let the coefficients of v in the first basis be w1 and w2: 1 0 v = w + w . 1 0 2 1     What are the coefficients of v in the second ba- sis? Stated differently, what values of w10 and w20 satisfy: cosθ sinθ v = w + w ? 10 sinθ 20 −cosθ     Figure 1: Change of basis.

Example (contd.)

To find w10 and w20 , we use inner product:

cosθ w1 w10 = , sinθ w2     sinθ w1 w20 = − , . cosθ w2     Example (contd.) The above can be written more economically in matrix notation: w cosθ sinθ w 10 = 1 w0 sinθ cosθ w2  2   −   w 0 = Aw. If the rows of A are orthonormal, then A is an orthonormal matrix. Multiplying by an or- thonormal matrix effects a change of basis. A change of basis between two orthonormal bases is a rotation. Matrix transpose If A rotates w by θ cosθ sinθ A = sinθ cosθ  −  1 T then A− = A rotates w 0 by θ − cosθ sinθ AT = . sinθ −cosθ   In other words, AT undoes the action of A, i.e., they are inverses:

T cos2 θ + sin2 θ cosθsinθ sinθcosθ − AA = cosθsinθ sinθcosθ cos2 θ + sin2 θ − h 1 0 i = . 0 1   For orthonormal matrices, multiplying by the transpose undoes the change of basis. Complex vectors in C2

iθ iθ T 2 v = [a1e 1,a2e 2] is a vector in C . Question Can we define length and angle in • C2 just like in R2? Answer Yes, but we need to redefine inner • product:

u,v = u∗v1 + u∗v2. h i 1 2 Note that this reduces to the inner product for R2 when u and v are real. The norm of a com- plex vector is the square root of the sum of the squares of the amplitudes. For example, for v C2: ∈ 1 u = u,u 2 | | h i = u1∗u1 + u2∗u2. p Orthonormal bases for Cn

Question How about orthonormal bases for • Cn, do they exist?

Answer Yes. If ui,u j = 0 when i = j and • h i 6 ui,u j = 1 when i = j, then the ui form an horthonormali basis for Cn.

Question Do complex orthonormal matrices • exist? Answer Yes, except they are called unitary • T matrices and (A∗) undoes the action of A. That is T A(A∗) = I T H where (A∗) = A is the Hermitian trans- pose of A. The space of 2π periodic functions A function, f , is 2π periodic iff f (t)= f (t + 2π). We can think of two complex 2π periodic functions, e.g., f and g, as infinite dimensional complex vectors. Length, angle, orthogonal- ity, and rotation (i.e., change of basis) still have meaning. All that is required is that we gener- alize the definition of inner product: π f ,g = f ∗(t)g(t)dt. h i Z π − The length (i.e., the norm) of a function is: π 1 2 f = f , f = f ∗(t) f (t)dt. | | h i Z π r − Two functions, f and g, are orthogonal when π f ,g = f ∗(t)g(t)dt = 0. h i Z π − Scaling Property of the Impulse The area of an impulse scales just like the area of a pulse, i.e., contracting an impulse by a fac- 1 tor of a changes its area by a factor of a : | | ∞ 1 ε Π(at)dt = = lim δ(at)dt. Z ∞ a ε 0 Z ε − | | → − It follows that: ε ε lim a δ(at)dt = lim δ(t)dt = 1. ε 0 Z ε | | ε 0 Z ε → − → − Since the impulse is defined by the above inte- gral property, we conclude that: a δ(at)= δ(t). | | Shah basis The Shah function is a train of impulses: ∞ III(t)= ∑ δ(t n). n= ∞ − − We can use the scaling property of the impulse to define a 2π periodic Shah function: ∞ 1 t 1 δ t πIII π = π ∑ π n 2 2 2 n= ∞ 2 −   π ∞−   2 δ π t = π ∑ 2 π n 2 n= ∞ 2 − ∞ −    = ∑ δ(t 2πn). n= ∞ − − Shah basis (contd.) Consider the infinite set of 2π periodic Shah 1 t τ π τ π functions, 2πIII 2−π , for < . Because τ − ≤ 1 III t = δ(t τ) for π t π it follows 2π 2−π
that − − ≤ ≤
1 t τ1 1 t τ2 III − , III − 2π 2π 2π 2π  π    = δ(t τ1)δ(t τ2)dt Z π − − − π equals 0 when τ1 = τ2 and equals π δ(t τ1)dt = 6 − − 1 when τ1 = τ2. It follows that theR infinite set π 1 t τ of 2 periodic Shah functions, 2πIII 2−π , for π τ < π form an orthonormal basis for the space− ≤ of 2π periodic functions.
1

......

......

Figure 2: Making a 2π periodic Shah function. Shah basis (contd.)

Question How do we find the coefficients, • w(τ), representing f (t) in the Shah basis? How do we find w(τ) such that 1 π t τ f (t)= w(τ)III − dτ? 2π Z π 2π −   Answer Take inner products of f with the • infinite set of 2π periodic Shah functions: 1 t τ w(τ)= III − , f (t) . 2π 2π     Shah basis (contd.) 1 t τ δ τ π π Because 2πIII 2−π = (t ) for t it follows that − − ≤ ≤
1 π t τ w(τ) = f (t)III − dt 2π Z π 2π π −   = f (t)δ(t τ)dt Z π − − which by the sifting property of the impulse is just: w(τ)= f (τ). We see that the coefficients of f in the Shah basis are just f itself! Harmonic signal basis

Question How long is a harmonic signal? • Answer The length of a harmonic signal is • ω ω ω 1 e j t = e j t,e j t 2 | | h i 1 π 2 jω t jω t = e− e dt Z π  − 1  π 2 = dt Z π  −  = √2π. Harmonic signal basis (contd.)

Question What is the angle between two har- • monic signals with integer frequencies? Answer The angle between two harmonic • signals with integer frequencies is π jω t jω t jω t jω t e 1 ,e 2 = e− 1 e 2 dt h i Z π − π j(ω ω )t e 2− 1 = ω ω . " j( 2 1)# − π − Since this function is the same at π and π (for − all integers ω1 and ω2), we conclude that ω ω e j 1t,e j 2t = 0 h i when ω1 and ω2 are integers and ω1 = ω2. 6 Fourier Series of 2π Periodic Functions It follows that the infinite set of harmonic sig- nals, 1 e jω t for integer ω and ∞ ω ∞ √2π − ≤ ≤ form an orthonormal basis for the space of 2π periodic functions. Question What are the coefficients of f in • the harmonic signal basis? Answer Take inner products of f with the • infinite set of harmonic signals. This is the analysis formula for Fourier series:

1 ω F(ω) = e j t, f √2π  π  1 jω t = f (t)e− dt √2π Z π − for integer frequency, ω. Fourier Series of 2π Periodic Functions (contd.) The function can be reconstructed using the syn- thesis formula for Fourier series: ∞ 1 ω f (t)= ∑ F(ω)e j t. √2π ω= ∞ − Fourier Series Example The Fourier series for the Shah basis function 1 t f (t)= III 2π 2π is   1 ω 1 t F(ω) = e j t, III √2π 2π 2π  π  1 ω   = δ(t)e j tdt √2π Z π 1 − = . √2π Consequently ∞ 1 ω f (t) = ∑ F(ω)e j t √2π ω= ∞ ∞ − 1 jω t = π ∑ e . 2 ω= ∞ − Deep Thought The analysis formula for Fourier series effects a change of basis. It is a rotation in the space of 2π periodic functions. The synthesis formula undoes the change of basis. It is the opposite rotation. 0.5 0.8 line 1 line 1 line 2 line 2 0.7 0.4 0.6

0.3 0.5

0.4 0.2

0.3

0.1 0.2

0 0.1

0 -0.1 -0.1

-0.2 -0.2 -15 -10 -5 0 5 10 15 -15 -10 -5 0 5 10 15 (a) (b)

1.6 3 line 1 line 1 line 2 line 2 1.4 2.5

1.2 2 1

1.5 0.8

0.6 1

0.4 0.5

0.2 0 0

-0.5 -0.2

-0.4 -1 -15 -10 -5 0 5 10 15 -15 -10 -5 0 5 10 15 (c) (d)

6 12 line 1 line 1 line 2 line 2

5 10

4 8

3 6

2 4

1 2

0 0

-1 -2

-2 -4 -15 -10 -5 0 5 10 15 -15 -10 -5 0 5 10 15 (e) (f)

Figure 3: Re(t) (solid) and Im(t) (dashed) of truncated Fourier series for Shah basis function. (a) 1 ω 1 (b) 2 ω 2 (c) 4 ω 4 (d) 8 ω 8 (e) 16 ω 16 (f) 32 ω 32. − ≤ ≤ − ≤ ≤ − ≤ ≤ − ≤ ≤ − ≤ ≤ − ≤ ≤ 25 45 line 1 line 1 line 2 line 2 40

20 35

30 15

25

10 20

15

5 10

5 0

0

-5 -5 -15 -10 -5 0 5 10 15 -15 -10 -5 0 5 10 15 (a) (b)

90 180 line 1 line 1 line 2 line 2 80 160

70 140

60 120

50 100

40 80

30 60

20 40

10 20

0 0

-10 -20 -15 -10 -5 0 5 10 15 -15 -10 -5 0 5 10 15 (c) (d)

350 700 line 1 line 1 line 2 line 2

300 600

250 500

200 400

150 300

100 200

50 100

0 0

-50 -100 -15 -10 -5 0 5 10 15 -15 -10 -5 0 5 10 15 (e) (f)

Figure 4: Re(t) (solid) and Im(t) (dashed) of truncated Fourier series for Shah basis function. (a) 64 ω 64 (b) 128 ω 128 (c) 256 ω 256 (d) 512 ω 512 (e) 1024 ω −1024≤ (f) ≤2048 ω− 2048.≤ ≤ − ≤ ≤ − ≤ ≤ − ≤ ≤ − ≤ ≤ Fourier Series of T-Periodic Functions A function, f , is T-periodic iff f (t)= f (t +T). Analysis formula • √2π πω F(ω) = e j2 t/T , f * T + π T/2 √2 j2πω t/T = f (t)e− dt T Z T/2 − for integer frequency, ω. Synthesis formula • ∞ √2π πω f (t)= ∑ F(ω)e j2 t/T T ω= ∞ − Observe that if we substitute T = 2π in the above expressions, we get the formulas for 2π peri- odic functions. The Fourier Transform Functions with finite length are termed square integrable. ∞ f = f (t) 2dt | | Z ∞ | | r −∞ = f ∗(t) f (t)dt Z ∞ < ∞r. − For square integrable functions, we can take the limit of the Fourier series for T-periodic func- tions as T ∞, in which case, it is possible to show that...→ The Fourier Transform (contd.)

Analysis formula • π F(s) = e j2 st, f ∞ j2πst = f (t)e − dt Z ∞ − Synthesis formula • ∞ π f (t)= F(s)e j2 stds Z ∞ −