Today
ESE 531: Digital Signal Processing ! Review: " Discrete Fourier Transform (DFT)
" Circular Convolution Lec 19: March 29, 2018 ! Fast Convolution Methods Discrete Fourier Transform, Pt 2
Penn ESE 531 Spring 2018 – Khanna Adapted from M. Lustig, EECS Berkeley Penn ESE 531 Spring 2018 - Khanna 2
Discrete Fourier Transform DFT vs DTFT
! The DFT ! Back to example 4
! It is understood that,
Penn ESE 531 Spring 2018 – Khanna Penn ESE 531 Spring 2018 – Khanna Adapted from M. Lustig, EECS Berkeley 3 Adapted from M. Lustig, EECS Berkeley 4
Properties of the DFS/DFT Properties (Continued)
Penn ESE 531 Spring 2018 – Khanna Penn ESE 531 Spring 2018 – Khanna Adapted from M. Lustig, EECS Berkeley Adapted from M. Lustig, EECS Berkeley
1 Duality Duality
Penn ESE 531 Spring 2018 - Khanna 7 Penn ESE 531 Spring 2018 - Khanna 8
Proof of Duality Proof of Duality
Penn ESE 531 Spring 2018 - Khanna 9 Penn ESE 531 Spring 2018 - Khanna 10
Proof of Duality Discrete Fourier Series
! Properties of WN: 0 N 2N " WN = WN = WN = ... = 1 k+r k r k+N k " WN = WN WN and, WN = WN kn ! Example: WN (N=6)
Penn ESE 531 Spring 2018 – Khanna Penn ESE 531 Spring 2018 - Khanna 11 Adapted from M. Lustig, EECS Berkeley 12
2 Proof of Duality Circular Convolution
! Circular Convolution:
For two signals of length N
Penn ESE 531 Spring 2018 – Khanna Penn ESE 531 Spring 2018 - Khanna 13 Adapted from M. Lustig, EECS Berkeley 14
Compute Circular Convolution Sum Result
y[0]=2 y[1]=2 y[2]=3 y[3]=4
Penn ESE 531 Spring 2018 – Khanna Penn ESE 531 Spring 2018 – Khanna Adapted from M. Lustig, EECS Berkeley 15 Adapted from M. Lustig, EECS Berkeley 16
Circular Convolution Multiplication
! For x1[n] and x2[n] with length N ! For x1[n] and x2[n] with length N
" Very useful!! (for linear convolutions with DFT)
Penn ESE 531 Spring 2018 – Khanna Penn ESE 531 Spring 2018 – Khanna Adapted from M. Lustig, EECS Berkeley 17 Adapted from M. Lustig, EECS Berkeley 18
3 Linear Convolution Linear Convolution
! Next.... ! We start with two non-periodic sequences:
" Using DFT, circular convolution is easy
" Matrix multiplication… more later
" But, linear convolution is useful, not circular
" So, show how to perform linear convolution with circular " E.g. x[n] is a signal and h[n] a filter’s impulse response convolution
" Use DFT to do linear convolution (via circular convolution)
Penn ESE 531 Spring 2018 – Khanna Penn ESE 531 Spring 2018 – Khanna Adapted from M. Lustig, EECS Berkeley 19 Adapted from M. Lustig, EECS Berkeley 20
Linear Convolution Linear Convolution via Circular Convolution
! We start with two non-periodic sequences: ! Zero-pad x[n] by P-1 zeros
! Zero-pad h[n] by L-1 zeros " E.g. x[n] is a signal and h[n] a filter’s impulse response
! We want to compute the linear convolution:
! Now, both sequences are length M=L+P-1
" y[n] is nonzero for 0 ≤ n ≤ L+P-2 (ie. length M=L+P-1)
Penn ESE 531 Spring 2018 – Khanna Requires L*P multiplications Penn ESE 531 Spring 2018 – Khanna Adapted from M. Lustig, EECS Berkeley 21 Adapted from M. Lustig, EECS Berkeley 22
Linear Convolution via Circular Convolution Example
! Now, both sequences are length M=L+P-1 ! We can now compute the linear convolution using a circular one with length M=L+P-1
Penn ESE 531 Spring 2018 – Khanna Penn ESE 531 Spring 2018 – Khanna Adapted from M. Lustig, EECS Berkeley 23 Adapted from M. Lustig, EECS Berkeley 24
4 Example Example
Penn ESE 531 Spring 2018 – Khanna Penn ESE 531 Spring 2018 – Khanna Adapted from M. Lustig, EECS Berkeley 25 Adapted from M. Lustig, EECS Berkeley 26
Linear Convolution with DFT Linear Convolution with DFT
! In practice we can implement a circulant ! In practice we can implement a circulant convolution using the DFT property: convolution using the DFT property:
! Advantage: DFT can be computed with Nlog2N complexity (FFT algorithm later!)
! Drawback: Must wait for all the samples -- huge delay -- incompatible with real-time filtering
Penn ESE 531 Spring 2018 – Khanna Penn ESE 531 Spring 2018 – Khanna Adapted from M. Lustig, EECS Berkeley 27 Adapted from M. Lustig, EECS Berkeley 28
Block Convolution Block Convolution
! Problem:
" An input signal x[n], has very long length (could be considered infinite)
" An impulse response h[n] has length P
" We want to take advantage of DFT/FFT and compute convolutions in blocks that are shorter than the signal ! Approach:
" Break the signal into small blocks
" Compute convolutions
" Combine the results
Penn ESE 531 Spring 2018 – Khanna Penn ESE 531 Spring 2018 – Khanna Adapted from M. Lustig, EECS Berkeley 29 Adapted from M. Lustig, EECS Berkeley 30
5 Overlap-Add Method Example
! Decompose into non-overlapping segments L=11
! The input signal is the sum of segments
Penn ESE 531 Spring 2018 – Khanna Penn ESE 531 Spring 2018 – Khanna Adapted from M. Lustig, EECS Berkeley 31 Adapted from M. Lustig, EECS Berkeley 32
Overlap-Add Method Overlap-Add Method
! We can compute xr[n]*h[n] using circular convolution with the DFT
! Using the DFT: ! The output is: " Zero-pad xr[n] to length N
" Zero-pad h[n] to length N and compute DFTN{hzp[n]} " Only need to do once
" Each output segment xr[n]*h[n] is length N=L+P-1 " h[n] has length P
" xr[n] has length L
Penn ESE 531 Spring 2018 – Khanna Penn ESE 531 Spring 2018 – Khanna Adapted from M. Lustig, EECS Berkeley 33 Adapted from M. Lustig, EECS Berkeley 34
Overlap-Add Method Example of Overlap-Add L+P-1=16
! We can compute xr[n]*h[n] using circular convolution with L=11 y the DFT ! Using the DFT: y " Zero-pad xr[n] to length N
" Zero-pad h[n] to length N and compute DFTN{hzp[n]} " Only need to do once y " Compute:
! Results are of length N=L+P-1
" Neighboring results overlap by P-1
" Add overlaps to get final sequence
Penn ESE 531 Spring 2018 – Khanna Penn ESE 531 Spring 2018 – Khanna Adapted from M. Lustig, EECS Berkeley 35 Adapted from M. Lustig, EECS Berkeley 36
6 Example of Overlap-Add L+P-1=16 Example of Overlap-Add L+P-1=16
L=11 y L=11 y
y y
y y
Penn ESE 531 Spring 2018 – Khanna Penn ESE 531 Spring 2018 – Khanna Adapted from M. Lustig, EECS Berkeley 37 Adapted from M. Lustig, EECS Berkeley 38
Overlap-Save Method Example of Overlap-Save
! Basic idea: L+P-1=16 ! Split input into overlapping segments with length L+P-1
" P-1 sample overlap
! Perform circular convolution in each segment, and keep the L sample portion which is a valid linear convolution
Penn ESE 531 Spring 2018 – Khanna Penn ESE 531 Spring 2018 – Khanna Adapted from M. Lustig, EECS Berkeley 39 Adapted from M. Lustig, EECS Berkeley 40
Circular to Linear Convolution Example of Overlap-Save
L+P-1=16 ! An L-point sequence circularly convolved with a P- P-1=5 point sequence Overlap samples " with L - P zeros padded, P < L ! gives an L-point result with
" the first P - 1 values incorrect and
" the next L - P + 1 the correct linear convolution result
Penn ESE 531 Spring 2018 – Khanna Penn ESE 531 Spring 2018 - Khanna 41 Adapted from M. Lustig, EECS Berkeley 42
7 Example of Overlap-Save Discrete Cosine Transform
L+P-1=16 ! Similar to the discrete Fourier transform (DFT), but using only real numbers
! Widely used in lossy compression applications (eg. Mp3, JPEG)
! Why use it?
Penn ESE 531 Spring 2018 – Khanna Adapted from M. Lustig, EECS Berkeley 43 Penn ESE 531 Spring 2018 - Khanna 44
DFT Problems DFT Problems
! For processing 1-D or 2-D signals (especially coding), a ! For processing 1-D or 2-D signals (especially coding), a common method is to divide the signal into “frames” and common method is to divide the signal into “frames” and then apply an invertible transform to each frame that then apply an invertible transform to each frame that compresses the information into few coefficients. compresses the information into few coefficients. ! The DFT has some problems when used for this purpose: ! The DFT has some problems when used for this purpose:
" N real x[n] ↔ N complex X[k] : 2 real, N/2 − 1 conjugate pairs " N real x[n] ↔ N complex X[k] : 2 real, N/2 − 1 conjugate pairs
" DFT is of the periodic signal formed by replicating x[n] " DFT is of the periodic signal formed by replicating x[n] ⇒ Spurious frequency components from boundary discontinuity
The Discrete Cosine Transform (DCT) overcomes these problems.
Penn ESE 531 Spring 2018 - Khanna 45 Penn ESE 531 Spring 2018 - Khanna 46
Discrete Cosine Transform FIR GLP: Type II
! To form the Discrete Cosine Transform (DCT), replicate x[0 : N − 1] but in reverse order and insert a zero between each pair of samples:
Penn ESE 531 Spring 2018 - Khanna 47 Penn ESE 531 Spring 2018 - Khanna 48
8 Discrete Cosine Transform Discrete Cosine Transform
! To form the Discrete Cosine Transform (DCT), replicate ! To form the Discrete Cosine Transform (DCT), replicate x[0 : N − 1] but in reverse order and insert a zero between x[0 : N − 1] but in reverse order and insert a zero between each pair of samples: each pair of samples:
! Take the DFT of length 4N real, symmetric, odd-sample- ! Take the DFT of length 4N real, symmetric, odd-sample- only sequence only sequence ! Result is real, symmetric and anti-periodic: only need first N values
Penn ESE 531 Spring 2018 - Khanna 49 Penn ESE 531 Spring 2018 - Khanna 50
Discrete Cosine Transform Discrete Cosine Transform
! Result is real, symmetric and anti-periodic: only need first N values
Penn ESE 531 Spring 2018 - Khanna 51 Penn ESE 531 Spring 2018 - Khanna 52
Basis Functions DFT of Sine Wave
Penn ESE 531 Spring 2018 - Khanna 53 Penn ESE 531 Spring 2018 - Khanna 54
9 DFT of Sine Wave DCT of Sine Wave
Penn ESE 531 Spring 2018 - Khanna 55 Penn ESE 531 Spring 2018 - Khanna 56
Big Ideas Admin
! Discrete Fourier Transform (DFT) ! HW 8 due tomorrow " For finite signals assumed to be zero outside of defined ! Project length " Work in groups of up to 2 " N-point DFT is sampled DTFT at N points " Can work alone if you want
" DFT properties inherited from DFS, but circular " Use Piazza to find partners operations! ! Fast Convolution Methods
" Use circular convolution (i.e DFT) to perform fast linear convolution
" Overlap-Add, Overlap-Save
! DCT useful for frame rate compression of large signals Penn ESE 531 Spring 2018 – Khanna Penn ESE 531 Spring 2018 – Khanna Adapted from M. Lustig, EECS Berkeley 57 Adapted from M. Lustig, EECS Berkeley 58
10