Lecture #10 Multi-rate Signal Processing 2 The DTFT The DTFT Change of Basis . Oftentimes, it is easier to process in a different basis . Hence, we may want to know the diagonalization of a Toeplitz matrix 0 1 2 3 1 ⋱ ⋱1 ⋱0 ⋱1 ⋱2 ⋱ ⋮0 = = = ⋱ ℎ 2 ℎ −1 ℎ −0 ℎ −1 ⋱ −1 ⋱ ℎ ℎ ℎ − ℎ − ⋱ −1 3 2 1 0 2 Λ ⋱ ℎ ℎ ℎ ℎ − ⋱ ⋱ ℎ ℎ ℎ ℎ ⋱ ECE 6534, Chapter 3 ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ ⋮ 4 The DTFT Eigenvalue decomposition . The eigenvalue decomposition of a Toeplitz matrix is 0 1 2 3 1 ⋱ ⋱1 ⋱0 ⋱1 ⋱2 ⋱ ⋮0 = = = ⋱ ℎ 2 ℎ −1 ℎ −0 ℎ −1 ⋱ −1 ⋱ ℎ ℎ ℎ − ℎ − ⋱ −1 3 2 1 0 2 Λ ⋱ ℎ ℎ ℎ ℎ − ⋱ ⋱ ℎ ℎ ℎ ℎ ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ ⋮ = The DTFT Operator = −1 ∗ ECE 6534, Chapter 3 5 The DTFT Eigenvalue decomposition . The eigenvalue decomposition of a Toeplitz matrix is 0 1 2 3 1 ⋱ ⋱1 ⋱0 ⋱1 ⋱2 ⋱ ⋮0 = = = ⋱ ℎ 2 ℎ −1 ℎ −0 ℎ −1 ⋱ −1 ⋱ ℎ ℎ ℎ − ℎ − ⋱ −1 3 2 1 0 2 Λ ⋱ ℎ ℎ ℎ ℎ − ⋱ ⋱ ℎ ℎ ℎ ℎ ⋱ . = ⋱ =⋱ ⋱ ⋱ ⋱ ⋱= ⋮ . −1 −1 −1 −1 So what is ? Λ ΛG ΛV ΛHΛΛ Λ ECE 6534, Chapter 3 6 The DTFT Eigenvalue decomposition . The eigenvalue decomposition of a Toeplitz matrix is 0 1 2 3 1 ⋱ ⋱1 ⋱0 ⋱1 ⋱2 ⋱ ⋮0 = = = ⋱ ℎ 2 ℎ −1 ℎ −0 ℎ −1 ⋱ −1 ⋱ ℎ ℎ ℎ − ℎ − ⋱ −1 3 2 1 0 2 Λ ⋱ ℎ ℎ ℎ ℎ − ⋱ ⋱ ℎ ℎ ℎ ℎ ⋱ . When⋱ does⋱ the inverse⋱ of ⋱filter exist?⋱ ⋱ ⋮ . How do you compute the pseudo-inverse of ? . How do you compute the Weiner deconvolution of ? ECE 6534, Chapter 3 7 The DTFT Eigenvalue decomposition . The eigenvalue decomposition of a Toeplitz matrix is 0 1 2 3 1 ⋱ ⋱1 ⋱0 ⋱1 ⋱2 ⋱ ⋮0 = = = ⋱ ℎ 2 ℎ −1 ℎ −0 ℎ −1 ⋱ −1 ⋱ ℎ ℎ ℎ − ℎ − ⋱ −1 3 2 1 0 2 Λ ⋱ ℎ ℎ ℎ ℎ − ⋱ ⋱ ℎ ℎ ℎ ℎ ⋱ . When⋱ does⋱ the inverse⋱ of ⋱filter exist?⋱ ⋱ ⋮ — When the frequency domain is all non-zero values . How do you compute the pseudo-inverse of ? — Only invert non-zero values in the frequency domain . How do you compute the Weiner deconvolution of ? — Perform a Tikhonov regularized inverse in frequency domain ECE 6534, Chapter 3 8 Exercise Exercise: An allpass filter satisfies = 1 What property must by matrix satisfy to be an allpass filter? ECE 6534, Chapter 3 9 Exercise Exercise: An allpass filter satisfies = 1 What property must by matrix satisfy to be an allpass filter? Answer: The magnitudes of the eigenvalues must be equal to 1. ECE 6534, Chapter 3 10 The DFT The DFT Diagonalization of a shift … 2 1 0 1 2 3 … − − Toeplitz Matrix DTFT Operator 0 0 0 0 1 ⋱ ⋱1 ⋱0 ⋱0 ⋱0 ⋱ ⋮0 = = = ⋱ 0 1 0 0 ⋱ −1 ⋱ ⋱ −1 0 0 1 0 2 Λ ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ ⋮ ECE 6534, Chapter 3 12 The DFT Diagonalization of a circular shift 0 1 2 3 4 5 Circulant Matrix DFT Matrix 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0 1 0 0 0 0 2 = = = 0 0 1 0 0 0 3 −1 0 0 0 1 0 0 4 Λ 0 0 0 0 1 0 5 ECE 6534, Chapter 3 13 The DFT Circular convolution = , ∗ ℎ � ℎ − ∈ℤ DFT Matrix [0] [5] [4] [3] [2] [1] 0 [1] [0] [5] [4] [3] [2] 1 ℎ[2] ℎ[1] ℎ[0] ℎ[5] ℎ[4] ℎ[3] 2 = = = ℎ[3] ℎ[2] ℎ[1] ℎ[0] ℎ[5] ℎ[4] 3 ℎ ℎ ℎ ℎ ℎ ℎ −1 [4] [3] [2] [1] [0] [5] 4 Λ ℎ[5] ℎ[4] ℎ[3] ℎ[2] ℎ[1] ℎ[0] 5 ℎ ℎ ℎ ℎ ℎ ℎ ℎ ℎ ℎ ℎ ℎ ℎ ECE 6534, Chapter 3 14 The DFT The DFT Matrix 1 1 1 1 1 1 1 1 2 3 ⋯ −1 = 1 2 4 6 ⋯ 2 −1 ⋯ 3 6 9 3 −1 1 ⋯ ( ) ⋮ ⋮ −1 2 ⋮−1 3 ⋮−1 ⋱ −1⋮ −1 ⋯ Makes matrix unitary ( = ) ∗ −1 = 2 − ECE 6534, Chapter 3 15 Exercise Question: What property must matrices (filters) satisfy to have a zero group delay (i.e., zero phase)? Show this with matrices. ECE 6534, Chapter 3 16 Exercise Question: What property must matrices (filters) satisfy to have a zero group delay (i.e., zero phase)? Show this with matrices. Answer: The matrix must be symmetric . This is because — = ∗ Λ Real if symmetric ECE 6534, Chapter 3 17 The Graph Fourier Transform Graph Spectrum For a given graph, there exists a shift matrix 1 4 5 Graph Fourier Transform 2 3 0 0 1 0 0 0 1 1 0 0 0 0 1 2 = 0 1 0 0 0 0 3 = 0 1 0 0 0 0 4 −1 0 0 0 0 1 0 5 Λ ECE 6534, Chapter 3 19 Graph Spectrum Question: What are graph frequency components? 1 4 5 Graph Fourier Transform 2 3 0 0 1 0 0 0 1 1 0 0 0 0 1 2 = 0 1 0 0 0 0 3 = 0 1 0 0 0 0 4 −1 0 0 0 0 1 0 5 Λ ECE 6534, Chapter 3 20 Multi-rate Signal Processing Downsampling and Upsampling Multirate signal processing Question: What is multirate signal processing? ECE 6534, Chapter 3 22 Multirate signal processing Periodically Shift-Varying Systems . A discrete-time system T is called periodically shift-varying of order ( , ) when, for any integer and input , = = ′ = ⇒ = 푥 ′ ′ − − . That is, if I shift the input by , I shift the output by ECE 6534, Chapter 3 23 Multirate signal processing Downsampling by 2 . Periodically shift-varying of order (2,1) [if I shift the input by 2, I shift the output by 1] 2 2 1 1 0 0 0 0 0 ⋮ 1 ⋮0 ⋱ ⋱0 ⋱0 ⋱1 ⋱0 ⋱0 ⋱0 ⋱ −0 = −1 ⋱ 0 0 0 0 1 0 ⋱ −1 2 ⋱ 0 0 0 0 0 0 ⋱ 2 ⋱ ⋱ [3] ⋱ ⋱ ⋮ ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ ⋮ ECE 6534, Chapter 3 24 Multirate signal processing Question: When I downsample… . What occurs in time? . What occurs in frequency? ECE 6534, Chapter 3 25 Multirate signal processing Question: When I downsample… . What occurs in time? — Answer: Condense in time (effectively) . What occurs in frequency? — Answer: Expand in frequency (with possible aliasing) ECE 6534, Chapter 3 26 Multirate signal processing Downsampling by 2 . Periodically shift-varying of order (2,1) [if I shift the input by 2, I shift the output by 1] Downsample by 2 Image from Martin Vertelli’s notes ECE 6534, Chapter 3 27 Multirate signal processing Downsampling by N . Periodically shift-varying of order (N,1) [if I shift the input by N, I shift the output by 1] = = 1 / = −1 1 � =0 ECE 6534, Chapter 3 28 Multirate signal processing Upsampling by 2 . Periodically shift-varying of order (1,2) [if I shift the input by 1, I shift the output by 2] 2 2 1 0 0 0 2 ⋮ 1 ⋱ ⋱0 ⋱0 ⋱0 ⋱0 ⋱ ⋮ 1 −0 ⋱ 0 1 0 0 ⋱ −0 = −1 ⋱ 0 0 0 0 ⋱ −1 2 ⋱ 0 0 1 0 ⋱ 2 3 ⋱ 0 0 0 0 ⋱ [3] ⋱ ⋱ ⋱ ⋱ ⋮ ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ ⋮ ECE 6534, Chapter 3 29 Multirate signal processing Question: When I upsampling… . What occurs in time? . What occurs in frequency? ECE 6534, Chapter 3 30 Multirate signal processing Question: When I upsampling… . What occurs in time? — Answer: Expand in time (effectively) . What occurs in frequency? — Answer: Condense in frequency ECE 6534, Chapter 3 31 Multirate signal processing Upsampling by 2 . Periodically shift-varying of order (1,2) [if I shift the input by 1, I shift the output by 2] Upsample by 2 Image from Martin Vertelli’s notes ECE 6534, Chapter 3 32 Multirate signal processing Upsampling by N . Periodically shift-varying of order (1,N) [if I shift the input by 1, I shift the output by N] = , for = / 0 , otherwise ∈ ℤ � = ECE 6534, Chapter 3 33 Multi-rate Signal Processing Upsampling and downsampling Multirate signal processing Question: . What is the adjoint of downsampling? . What is the adjoint of upsampling? ECE 6534, Chapter 3 35 Multirate signal processing Question: . What is the adjoint of downsampling? — Answer: = ∗ . What is the adjoint of upsampling? — Answer: = ∗ ECE 6534, Chapter 3 36 Multirate signal processing Question: . What is the = ? (reminder: matrix operations are right to left) ∗ . What does the result mean? ECE 6534, Chapter 3 37 Multirate signal processing Question: . What is the = ? (reminder: matrix operations are right to left) — Answer: ∗ = = ∗ . What does the result mean? — Answer: — is the right inverse of — is the right inverse of — ∗ is a 1-tight frame ECE 6534, Chapter 3 38 Multirate signal processing Properties of Downsampling and Upsampling . Relationship between upsampling and downsampling = ∗ . Upsampling followed by downsampling = 2 22 ECE 6534, Chapter 3 39 Multirate signal processing Properties of Downsampling and Upsampling . Relationship between upsampling and downsampling = ∗ . Downsampling followed by upsampling = (projection operator) 2 22 ECE 6534, Chapter 3 40 Multirate signal processing Properties of Downsampling and Upsampling . Relationship between upsampling and downsampling = ∗ . Downsampling followed by upsampling = (projection operator) 2 22 ECE 6534, Chapter 3 41 Multirate signal processing Properties of Downsampling and Upsampling . Upsampling by N and downsampling by M commute when N and M have no common factors (i.e., N = 3 and M = 2) 3 23 2 ECE 6534,3 Chapter2 3 42 Multi-rate Signal Processing Filtering with downsampling and upsampling Multirate signal processing Question . Why incorporate filtering? ECE 6534, Chapter 3 44 Multirate signal processing Example (from Martin Veterlli’s notes) Original signal Downsampled by 4 (aliasing) ECE 6534, Chapter 3 45 Multirate signal processing Example (from Martin Veterlli’s notes) Downsampled THEN Filtered THEN filtered (aliasing) downsampled ECE 6534, Chapter 3 46 Multirate signal processing Properties of Downsampling and Upsampling .