PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 1
NOTE 1. LINEAR SYSTEM
Finite vs. Infinite Duration Signals
A discrete signal x[n] is finite duration if there exists two integers -∞ < N1 ≤ N2 < ∞, such that x[n] ≠ 0 only for N1 ≤ n ≤ N2. Otherwise, it is of infinite duration.
Right-sided, Left-sided, and Two-sided Signals The terms apply only to infinite duration signals.
If there exists an integer N1 such that x[n] ≠ 0 only for n ≥ N1, then x[n] is right-sided.
If there exists an integer N2 such that x[n] ≠ 0 only for n ≤ N2, then x[n] is left-sided. If x[n] is neither right-sided nor left-sided, then it is two-sided.
Causal and Anti-Causal Signals A signal x[n] is causal if x[n] = 0 for all n < 0. A signal x[n] is anti-causal if x[n] = 0 for all n > 0.
4. Zero phase, linear phase and nonlinear phase signals Zero Phase: Left-right symmetry around sample number zero. Linear Phase: Left-right symmetry around sample number other than zero. Linear phase signal can be changed to zero phase one by simple shifting. Nonlinear Phase: without left-right symmetry.
5. Time Delay
x[n] One Sample Delay x[n-1]
System (Transform, Mapping): A Relationship between Input and Output.
7. Linear System S can be characterized by the following conditions: Homogeneity: For constant k, if system S satisfies S(x[n]) = y[n], then S(kx[n]) = kS(x[n]) = ky[n].
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 2
Additivity:
If S(x1[n]) = y1[n], and S(x2[n]) = y2[n]
Then S(x1[n] + x2[n]) = S(x1[n]) + S(x2[n]) = y1[n] + y2[n]
Commutative: for linear systems A and B A(B(x[n])) = B(A(x[n]))
Any System violates above-mentioned three rules, is a nonlinear system.
8. Shift (Time) Invariance System: if S(x[n]) = y[n], then S(x[n + i]) = y[n + i].
∞ 2 9. Energy of signal x[n]: E = ∑ x[n] . n=−∞
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 3
NOTE 2. Convolution
Convolution – most important technique in DSP, by combining two (or more) signals to form a third output.
Signal Filtering (Estimation), Function Approximation Interpolation Prediction (extrapolation)
1, n = 0 Definition 1: Delta Function (Unit Impulse) δ[n] = 0, others 1, n ≥ 0 Standard Function u(n) = 0, others δ[n] = u[n] - u[n - 1]
∞ u[n] = ∑δ (n − k) k =0
Definition 2: Impulse response h[n] of linear system S. The signal that exits a system when a Delta Function (unit impulse) is the input.
δ[n] Linear System h[n]
h[n] = S(δ[n])
Any sequence x[n] can be represented by weighted Delta function
∞ x[n] = ∑ x(k)δ (n − k) k =−∞
Definition 3: Periodic Sequence x[n] = x[n + N]
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 4
Example: sin[2πf(n+N)] = sin[2πfn], f is normalized frequency.
Definition 4: Convolution y[n] = h[n]∗ x[n]
x[n] Linear System h[n] y[n]
The impulse response of the system expresses the relation between input and output.
∞ y[i] = ∑h[ j]x[i − j] j =−∞ Convolution Sum (Linear System).
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 5
NOTE 3. Convolution Properties
Properties of Delta function: δ[n] is identity for convolution, x[n]∗δ[n] = x[n] x[n]∗ kδ[n] = kx[n] x[n]∗δ[n + i] = x[n + i]
Properties of Convolution (Convolution is a linear system) Commutative: a[n]∗b[n] = b[n]∗ a[n] Associative: {a[n]∗b[n]}∗ c[n] = a[n]∗{b[n]∗ c[n]} Distributive: a[n]∗b[n] + a[n]∗ c[n] = a[n]∗{b[n] + c[n]} Transference between the Input and Output Suppose y[n] = x[n]∗ h[n] ,
If L is a linear system and x1[n] = L{x[n]}, y1[n] = L{y[n]} = ∗ Then y1[n] x1[n] h[n]
M −1 5. (Auto-Regression): y[n] = ∑a[k]x[n − k] k =0 For Example: y[n] = x[n] - x[n-1] (first difference)
M −1 6. (Moving Average): y[n] = b[0]x[n] + ∑b[k]y[n − k] k =1 For Example: y[n] = x[n] + y[n - 1] (running sum)
7. Central Limit Theorem: y[n] = x[n]∗ x[n]∗...∗ x[n] 8. Correlation: Convolution = Cross-Correlation c[n] = a[n]∗b[−n]
Autocorrelation: c[n] = a[n]∗ a[−n] Detector (Not Restoration).
Low-Pass Filter h[n]: ∑h[n] ≠ 0 , ∑(−1)n h[n] = 0 n n Cutoff the high-frequency components (undulation, pitches), smooth the signal
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 6
High-pass Filter h[n]: ∑h[n] = 0 n Only preserve the quick undulation terms
Delta function: All-pass filter x[n] = x[n]∗δ[n]
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 7
NOTE 4. Fourier Representation
Any continuous periodic signal ⇔ sum of properly chosen sinusoidal functions Gibbs Effects: Problem of non-smooth continuous
4 Categories of Fourier Representation ( ω = 2πf )
(1) Aperiodic Continuous Fourier Transform
+∞ X ( f ) = x(t)e− j 2πft dt ∫−∞ Aperiodic Frequency f ∈ R Continuous Time (Shift) t ∈ R Continuous
(2) Periodic Continuous Fourier Series (Frequency Response)
+∞ X ( f ) = ∑ x(n)e− j 2πfn n=−∞ Use to Characterize the Filters and Signals. Response Periodic X(f) = X(f + 1) Frequency f ∈ R Continuous Time (Shift) k ∈ Z Discrete
Aperiodic Discrete Time Fourier Transform
+∞ X (k) = x(t)e− j 2πk∆ft dt ∫−∞ Aperiodic Frequency k ∈ Z Discrete Time (Shift) t ∈ R Continuous
(4) Periodic Discrete Fourier Transform
N −1 X[k] = ∑ x[i]e− j 2πki / N i=0 e− j2πki / N = cos(2πki / N) − jsin(2πki / N)
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 8
N −1 N −1 X[k] = ∑ x[i]cos(2πki / N) − j∑ x[i]sin(2πki / N) i=0 i=0
Periodic X(k + N) = X(k) Frequency k ∈ Z Discrete Time (Shift) n ∈ Z Discrete
Frequency Response Frequency Transform of Sequence h[n]
∞ H (ω) = ∑ h[k]e− jkω k=−∞ Inverse Frequency Transform
π 1 ω h[k] = ∫ H (ω)e jk dω 2π −π
Continuous Periodic H(ω + 2π) = H(ω) ≠ Discrete Fourier Transform (DFT)
Ideal Low-pass Filter π 1, |ω |≤ 2 H (ω) = π π 0, <|ω |≤ 2 2
Inverse Frequency Response kπ π sin π − 1 ω 1 ω = ω jk ω = 2 jk ω = 2 h[k] ∫ H ( )e d ∫π e d π −π π π 2 2 2 k
Not Causal
Technically CAN NOT be Realized
Cut-off cause Gibbs Oscillation
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 9
Frequency Transform of Convolution
∞ y[n] = h[n]* x[n] = ∑ x[k]h[n − k] k =−∞
Y(ω) = X(ω)H(ω)
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 10
NOTE 5. Properties of Frequency Response
Periodic Continuous Fourier Series
+∞ X (ω) = ∑ x(n)e− jωn n=−∞ Frequency Response of Convolution
∞ y[n] = h[n]∗ x[n] = ∑ x[k]h[n − k] k=−∞ ⇓ Y(ω) = X(ω)H(ω)
H(ω) = Y(ω) / X(ω) Transfer Function of the Linear System
|H(ω)| = [Im H(ω)2+Re H(ω)2]1/2 Magnitude of Transfer Function Arg(H(ω)) = arctan(Im H(ω)/Re H(ω)) Phase of Transfer Function
Even Symmetry around Point Zero x[n] = x[-n]
+∞ X (ω) = ∑ x(n)e− jωn n=−∞
∞ − ω ω = x[0] + ∑(x[n]e j n + x[−n]e j n ) n=1
∞ − ω ω = x[0] + ∑ x[n](e j n + e j n ) n=1
∞ = x[0] + 2∑ x[n]cos(ωn) n=1
Real Spectrum Phase Arg(H(ω)) = 0 Zero-phase Signal
Odd Symmetry around Point Zero x[n] = -x[-n], x[0] = 0
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 11
+∞ X (ω) = ∑ x(n)e− jωn n=−∞
∞ − ω ω = x[0] + ∑(x[n]e j n + x[−n]e j n ) n=1
∞ − ω ω = ∑ x[n](e j n − e j n ) n=1
∞ = - 2 j∑ x[n]sin(ωn) n=1
Complex Spectrum Phase Arg(H(ω)) = -π/2
Even Symmetry around sample number N other than zero x[N+n] = x[N-n]
+∞ X (ω) = ∑ x(k)e− jωk k =−∞
N −1 ∞ − ω − ω − ω = x[N]e j N + ∑ x[k]e j k + ∑ x[k]e j k k =−∞ k = N +1
∞ − ω − ω ω − ω = x[N]e j N + e j N ∑ x[n + N](e j n + e j n ) n=1
∞ − ω = e j N {x[N] + 2∑ x[n + N]cos(ωn)} n=1
Complex Spectrum Phase Arg(H(ω)) = -Nω Linear-phase Signal Odd-symmetric signal Generalized linear-pase signal
Even/Odd Decomposition of any Real Signal x[n]
Even Symmetryxe[n] = (x[n] + x[-n]) / 2 xe[n] = xe[-n]
Odd Symmetry xo[n] = (x[n] - x[-n])/2 xo[n] = -xo[-n]
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 12
Even Symmetry Sequence xe[n] generate real spectrum
Odd Symmetry Sequence xo[n] generate complex spectrum
(4) x[n] X(ω)
x[-n] X(ω)
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 13
NOTE 6. Digital Fourier Transform (DFT)
Periodic Continuous Fourier Series of x[n], n = 0, N -1
N −1 X (ω) = ∑ x[n]e− jωn n=0 Let ω = 2πk / N, k = 0, N -1
N −1 = ω = − j2πkn / N X[k] X ( ) |ω =2πk / N ∑ x[n]e n=0
Periodic Discrete Fourier Transform of Real Signal e− j2πki / N = cos(2πki / N) − jsin(2πki / N)
N−1 N −1 Re X[k] = ∑ x[i]cos(2πki / N) Im X[k] = −∑ x[i]sin(2πki / N) i=0 i=0
Periodic Character
X[k + N] = X[k]
Inverse Discrete Fourier Transform
N −1 N −1 N −1 N −1 N −1 π − π π π − ∑ X[k]e j2 km / N = ∑{∑ x[n]e j 2 kn / N }e j 2 km / N = ∑ x[n]{∑e j 2 k (m n) / N } k =0 k =0 n=0 n=0 k =0
N −1 π − N, m = n Because ∑e j2 k (m n) / N = ≠ k =0 0, m n
N −1 π Then ∑ X[k]e j2 km / N = Nx[m] k =0
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 14
N −1 1 π x[n] = ∑ X[k]e j2 kn / N N k =0
Simplified Algorithm
N −1 N −1 − π − π X[N − k] = ∑ x[n]e j2 (N k )n / N = ∑ x[n]e j 2 kn / N n=0 n=0
Re X[N - k] = Re X[k]Im X[N-k] = -Im X[k]
| X[N - k] | = | X[k] |, Arg(X[N - k]) = -ArgX[k]
N −1 N −1 N −1 1 π 1 1 x[n] = ∑ X[k]e j2 kn / N = ∑Re X[k]cos(2πkn / N) − ∑Im X[k]sin(2πkn / N) N k =0 N k =0 N k =0 cos(2π(N - k)n / N) = cos(2πkn / N) sin(2π(N - k)n / N) = -sin(2πkn / N)
x[n] = {Re X[0] + Re X[N / 2] cos(nπ)} / N 2 N / 2−1 2 N / 2−1 + ∑Re X[k]cos(2πkn/ N) − ∑Im X[k]sin(2πkn / N) N k =1 N k =1
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 15
NOTE 7: CHARACTERISTICS OF DFT
= + = + Linear: If x[n] ax1[n] bx2 [n], then X[k] aX 1[k] bX 2 [k].
Circular Shift: − π Time: DFT (x[n − m]) = e j 2 km / N X[k] π Frequency: IDFT(X[k − l]) = e j2 nl / N X[k] Circular Convolution N −1 = ∗ = − = If y[n] x1[n] x2 [n] ∑ x1[m]x2 [n m], then Y[k] X 1[k]X 2 [k]. m=0 N−1 = ∗ = 1 − = If Y[k] X1[k] X 2[k] ∑ X1[m]X 2[k m] , then y[n] x1[n]x2[n]. N m=0 Non-Overlap Condition x1[n] : L non-zero, x2 [n] : M non-zero The zero-extended period duration N ≥ L + M −1
Inverse DFT (can be implemented using the same program of DFT) ∗ N −1 N−1 1 π 1 ∗ − π x[n] = ∑ X[m]e j2 mn / N = ∑ X [m]e 2 mn/ N N m=0 N m=0 N−1 − π DFT: X[k] = ∑ x[m]e j2 mn/ N m=0
Symmetry Theorem If DFT(x[n]) = X[k], then DFT(x[−n]) = X[−k] X[k] DFT = x[−n] N
Initial State N−1 Frequency Domain X[0] = ∑ x[n] n=0 1 N −1 Time Domain x[0] = ∑ X[k] N k=0
Zero Extension Time Domain x[n], 0 ≤ n ≤ N −1 k If g[n] = , then G[k] = DFT(g[n]) = X 0, N ≤ n ≤ rN −1 r
Frequency Domain X[k], 0 ≤ k ≤ N −1 1 n If Y[k] = , then y[n] = IDFT(Y[k]) = x , k = 0, rN −1. 0, N ≤ n ≤ rN −1 r r
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 16
Discrete Cosine Transform Orthogonal Transform Symmetry-Periodic Extension x[−n] = x[n]
JPEG (Joint Photographic Expert Group) MPEG (Moving Picture Expert Group) 2 N−1 (2n +1)kπ DCT X[k] = a[k]∑ x[n]cos N n=0 2N 2 N−1 (2n +1)kπ IDCT x[n] = ∑ a[k]X[k]cos N k=0 2N 2 , k = 0 where a[k] = 2 1, else DCT and IDCT share the same transformation kernel Real spectrum, can be implemented by 2N-dot FFT, smaller boundary effects.
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 17
NOTE 8 ADDITIONAL STATEMENT ON DFT
Parseval Relation N−1 N−1 2 1 2 ∑ x[n] = ∑ X[k] n=0 N k=0 2 N / 2−1 = 1 2 + N + 2 X[0] X 2 ∑ X[k] N 2 k=1 1 1 Let X[0] = X[0] , X[N / 2] = X[N / 2] 2 2 N −1 N / 2 2 2 2 ∑ x[n] = ∑ X[k] n=0 N k=0 New IDFT Equation 1 N /2−1 N /2−1 x[n] = Re X[0] + Re X[N / 2](−1)n + 2 ∑∑Re X[k]cos(2πkn / N) − 2 Im X[k]sin(2πkn /N) N k=1 k=1 1 1 Let X[0] = X[0] , X[N / 2] = X[N / 2] 2 2 2 N / 2 x[n] = ∑{Re X[k]cos(2πkn / N) − Im X[k]sin(2πkn / N)} N k=0 Convolution Equivalence of DFT = j2πkn / N Let hk [n] e N −1 = − = ∗ X[k] ∑ x[i]hk [0 i] x[n] hk [n] |n=0 i=0 Time Sampler of All Filters is 0.
Frequency Response N−1 N −1 − jNω j 2πk − ω − ω π 1 − e e H [ω] = ∑ h [n]e jn = ∑ e jn e j2 nk / N = k k − − jω j 2πk / N n=0 n=0 1 e e Magnitude Response sin(Nω / 2 − πk) | H [ω] |= k sin(ω / 2 − πk / N) π Center Frequency ω = 2 k . k N ω ω ω Frequency Leakage: when signal frequency is between k and k+1 .
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 18
NOTE 9 FOURIER TRANSFORM PAIR
1. Frequency Response (FR) • Delta Function
If x[n] = δ[n], then X(ω) = 1. If x[n] = δ[n - m], then X(ω) = e -jωm .
• Square Function
Example 1: 1− e− jωM sin(ωM / 2) If x[n] = 1, n = 0, M-1, then X (ω) = and | X (ω) |= 1− e− jω sin(ω / 2)
Example 2: sin(ω(2M +1) / 2) If x[n] = 1, |n| = 0, M, then X (ω) = . sin(ω / 2) If y[n] = x[n - M], then Y(ω) = X(ω) e -jωM If y[n] = x[n] ejan,then Y(ω) = X(ω - a)
2. Discrete Fourier Transform (DFT) • Delta Function Time Region If x[n] = δ[n], n=0, N – 1, then X[k] = DFT(δ[n]) = 1, k=0, N – 1.
If x[n] = δ[n - m], n=0, N – 1, then X[k] = DFT(δ[n - m]) = e -j2πkm/N , k=0, N – 1. Magnitude |X[k]| = 1, Phase Arg(X[k]) = arctan[tan(-2πkm/N)]
Frequency Region 1 If X[k] = δ[k], k = 0, N – 1, then x[n] = IDFT(δ[k]) = , n = 0, N – 1. N 1 π If X[k] = δ[k - M], k = 0, N – 1, then x[n] = IDFT(δ[k]) = e j 2 nM / N , n = 0, N – 1. N
• Square Function
Time Region:
Example 1: x[n] = 1, n = 0, M – 1 M −1 − j 2πkM / N − π 1− e π − sin(πkM / N) X[k] = ∑e j 2 nk / N = = e j k (1 M )/ N − − j 2πk / N π n=0 1 e sin( k / N) sin(πkM / N) | X[k]|= sin(πk / N)
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 19
Example 2:
If x[n] = 1, n = 0, M and x[n] = 1, n = N - M, N – 1, sin(πk(2M +1) / N) then X[k] = sin(πk / N)
Frequency Region: If X[k] = 1, when k = 0, M; and X[k] = 1, when k = N - M, N – 1, M N −1 1 π π 1 sin(πn(2M +1) / N) then x[n] = ∑ e j2 kn/ N + ∑ e2 kn/ N = π N k=0 k=N −M N sin( n / N)
DFT(x[n - M]) = X[k]e-j2πkM/N IDFT(X[k - M]) = x[n]ej2πnM/N
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 20
NOTE 10. FAST FOURIER TRANSFORM
FFT: Fast implementation of DFT
Signal Length N = 2L
Let x1[r] = x[2r], x2[r] = x[2r + 1], r = 0, N/2 - 1
X[k] = DFT (x[n]) N / 2−1 N / 2−1 = ∑ x[2r]e− j 2π (2r)k / N + ∑ x[2r +1]e− j2π (2r+1)k / N r=0 r=0 N / 2−1 N / 2−1 = ∑ x[2r]e− j 2πrk /(N / 2) + e− j2πk / N ∑ x[2r +1]e− j 2πrk /( N / 2) r=0 r=0 -j2πk/N = X1[k] + X2[k]e
(N/2 dot FFT !!!) 1. For k = 0, N / 2 – 1,
-j2πk/N X[k] = X1[k] + X2[k]e , k = 0, N / 2 - 1
2. For k = N / 2, N – 1, Because -j2π(k + N/2)/N -j2πk/N X1[k + N / 2] = X1[k], X2[k+N/2]=X2[k], e = -e
-j2πk/N X[k + N / 2] = X1[k] - X2[k]e , k = 0, N / 2 - 1
Butterfly Algorithm
-j2πk/N X1[k] X1[k] + X2[k]e (X[k])
-j2πk/N -j2πk/N X2[k]× e X1[k] - X2[k]e (X[k + N/2])
One Butterfly: generates 2 Dots of DFT Each Layer: N / 2 Butterflies
Each Butterfly: 1 multiply + 2 add Layer Number: L = log2N
Calculation Efficiency: 1. FFT Cost = (N / 2) × L × (1 Multiply + 2 Add) = N log N Multiply + Nlog N Add 2 2 2
2. DFT Cost = N2 Multiply + N(N - 1) Add
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 21
NOTE 11. FFT Application
Fast Convolution For x[n] ∗ h[n], x[n], n = 0, M - 1 and h[n], n = 0, L - 1.
1. To Avoid Overlap, Zero-Extention Choose N = 2J ≥ (M + L - 1)
x[n], n = 0,M −1 x[n] = 0, n = M , N −1
h[n], n = 0,L −1 h[n] = 0, n = L, N −1 2. Do FFT
X[k] = FFT(x[n]), H[k] = FFT(h[n]), k = 0, N - 1
3. Multiplication Y[k] = X[k]H[k]
N −1 1 π 4. IFFT y[n] = ∑Y[k]e j2 kn / N N k =0
N −1 − π = ∑(Y[k]/ N)e j2 kn / N k =0 y[n] = FFT(Y[k]/ N)
Calculation Efficiency Comparison
Traditional Convolution y[n] = ∑i=0, L-1 x[n-i]h[i], n = 0, L+ M – 2.
Multiply: (L + M - 1) × L Add: (L + M - 1) × (L - 1)
FFT Convolution (1) FFT(x[n]) + FFT(h[n]) + FFT(Y[k]/N)
Multiply: (3N / 2) log2N Add: 3Nlog2N
(2) (X[k]H[k]) + (Y[k] / N) Multiply: 2N
Fast Correlation x[n], y[n], n = 0, M - 1 z[n] = x[n] ∗ y[-n] = ∑i = 0, M - 1 x[i]y[i + n]
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 22
Same Routine! (Zero-Extension + FFT)
Just Z[k] = X[k]Y[k]
Calculation Efficiency
Traditional Correlation (Only Count Multiply!!) Cost = M(2M - 1) Multiply
FFT Correlation Cost= (3N / 2) log2N + 2N Multiply
Worst Case: Signal length of x[n], y[n] M = 2L + 1
Then Extended Length: N ≥ 2M – 1 = 1 + 2L + 1 Thus, N = 2L + 2
FFT Cost = (6L + 20) 2L Multiply, while Traditional Cost = (1 + 2L)(1 + 2L + 1) Multiply
When L ≥ 5, M ≥ 33 dots, (N ≥ 128) FFT Method will be more Efficient than Traditional calculation.
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 23
NOTE 12. MULTIRESOLUTION SPACES
Operator: 1 M −1 ω − 2πk Down Sampling: If y[n] = x[Mn], then Y(ω) = ∑ X ( ) M k =0 M x[n], l = Mn Up Sampling: y[l] = , then Y(ω) = X(Mω) 0, others
Scaling Function φ(x) is an Approximating Function .
{}φ−∈ {}∈ If (xn)|nZ is a Rieze basis of V0. Namely, exist unique sequence C0(n)|nZ, for ∈ = φ − any function f (x) V0 , make f (x) ∑C0[n] (x n) . n
Two-Scale Difference Equation φ(x / 2) = 2∑ h[n]φ(x − n) n
Construct the Multiresolution Approximation ⊂ (1) Vm Vm−1 ∈ ⇔ ∈ (2) f (x) Vm f (2x) Vm−1 ∈ ⇒ − m ∈ (3) f (x) Vm f (x 2 n) Vm (4) Exist two constants A and B, 0 < A ≤ B, that 2 A || f || ≤ ∑n |C0[n]| ≤ B || f ||
Multi-Resolution Approximation (From Fine to Coarse)
For any f (x) ∈ L2 (R) , = φ − P0 f (x) ∑C0[n] (x n), for V0 n = φ −m − Pm f (x) ∑Cm[n] (2 x n), for Vm n
Ideal Low-Pass Function is a Scaling function
φ[x] = Sinc()πx πn φ(x / 2) = ∑Sinc φ(x − n) n 2 1 πn h[n] = Sinc 2 2
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 24
NOTE 13. WAVELET TRANSFORM
φ = −m / 2φ −m − ψ = −m / 2ψ −m − m,n (x) 2 (2 x n) , m,n (x) 2 (2 x n) . φ(x / 2) = 2∑h[n]φ(x − n) , ψ (x / 2) = 2∑ g[n]φ(x − n) . n n
∑h[n] = 1, ∑ g[n] = 0 n n
Scaling Transform
∞ C [n] = f (x), φ (x) = f (x)φ (x)dx m m,n ∫−∞ m,n
− Let y = 2 j x − k φ=−m/2φ−m− m,n(x)2(2xn) =2−m/2φ(y)=2−m/22∑h[k]φ(2y−k) k =2∑h[k]{}2−(m−1)/2φ(2−(j−1)x−2n−k) k =φ 2∑h[k]m−1,2n+k(x) k
=φ Cm[n] f(x), m,n(x) =φ 2∑h[k] f(x), m−1,2n+k(x) k =+ 2∑h[k]Cm−1[2nk] k
Wavelet Transform ∞ D [n] = f (x), ψ (x) = f (x)ψ (x)dx m m,n ∫−∞ m,n
Dj[k]=
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 25
ψ=−m/2ψ−m− m,n(x)2(2xn) =2−m/2ψ(y)=2−m/22∑g[k]φ(2y−k) k =2∑g[k]{}2−(m−1)/2φ(2−(j−1)x−2n−k) k =φ 2∑g[k]m−1,2n+k(x) k
=ψ Dm[n] f(x), m,n(x) =φ 2∑g[k] f(x), m−1,2n+k(x) k =+ 2∑g[k]Cm−1[2nk] k
Properties of Orthonormal Bases
φ(x), φ(x − k) = δ[k] φ(x), ψ (x − k) = 0 ψ (x), ψ (x − k) = δ[k]
∞ ∞ φ(x)dx = 1, ψ (x)dx = 0 ∫−∞ ∫−∞
∈ If f (x) V0 , = φ = φ + ψ f (x) ∑C0[i] 0,i (x) ∑C1[n] 1,n (x) ∑ D1[l] 1,l (x) i n l
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 26
NOTE 14. DIGITAL IMPLEMENTATION
DISCRETE WAVELET DECOMPOSITION
= φ = φ Cm[n] f (x), m,n (x) f (x), 2∑h[i] m−1,i+2n (x) i = − 2∑Cm−1[i]h[i 2n] i = ψ = φ Dm[n] f (x), m,n (x) f (x), 2∑ g[i] m−1,i+2n (x) i = − 2∑Cm−1[i]g[i 2n] i
WAVELET RECONSTRUCTION
⊕ = Vm Wm Vm−1 ⇒ ⇒ + = Pm f Vm , Qm f Wm Pm f Qm f Pm−1 f
= φ Pm−1 f ∑Cm−1[n] m−1,n (x) n = + Pm f Qm f = φ + φ ∑Cm[i] m,i (x) ∑ Dm[l] m,l (x) i l
= φ φ Cm−1[n] ∑Cm−1[i] m−1,i(x), m−1,n(x) i = φ φ + ψ φ ∑Cm[i] m,i(x), m−1,n(x) ∑Dm[l] m,l(x), m−1,n(x) i l = φ φ + ψ φ ∑∑Cm[i] 2 h[k] m−1,k+2i(x), m−1,n(x) ∑∑Dm[l] 2 g[k] m−1,k+2l(x), m−1,n(x) ik lk = − + − 2∑Cm[i]h[n2i]∑ Dm[l]g[n2l] i l
= − = − Decomposition Cm[n] 2∑Cm−1[i]h[i 2n] , Dm[n] 2∑Cm−1[i]g[i 2n] . i i
= − + − Reconstruction Cm−1[n] 2 ∑Cm[i]h[n 2i] ∑ Dm[l]g[n 2l] . i l
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 27
For Computer Implementation
= − = − Decomposition Cm[n] ∑Cm−1[i]h[i 2n] , Dm[n] ∑Cm−1[i]g[i 2n]. i i = × − + − Reconstruction Cm−1[n] 2 ∑Cm[i]h[n 2i] ∑ Dm[l]g[n 2l] i l
Perfect Reconstruction Condition
A(ω) = H (ω)[H ∗ (ω)X (ω) + H ∗ (ω −π )X (ω −π )] B(ω) = G(ω)[G∗ (ω)X (ω) + G∗ (ω −π )X (ω −π )]
For Perfect Reconstruction:
X(ω)=A(ω)+B(ω) =X(ω)[][]H(ω)H∗(ω)+G(ω)G∗(ω)+X(ω+π)H(ω)H∗(ω−π)+G(ω)G∗(ω−π)
H(ω)2+G(ω)2=1 H(ω)H∗(ω−π)+G(ω)G∗(ω−π)=0
Conjugate Quadrature Filter (CQF) Let G(ω) = H ∗ (ω −π )[± e− jω (2M +1) ]
G(ω −π ) = −H ∗ (ω)[± e jω (2M +1) ]
∗ ∗ Then H(ω)H (ω −π ) + G(ω)G (ω −π ) = 0
2 2 H(ω) + H(ω −π ) = 1 G(ω) 2 + G(ω −π ) 2 = 1
CQF Example (Let M = 0)
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 28
∗ − ω − G(ω) = H (ω −π )e j , g[k]=()−11kh[1−k]
∗ −ω G(ω)=−H(ω−π)ej , g[k]=()−1kh[1−k]
Operator: Down Sampling y[n] = x[Mn] 1 M −1 ω − 2πk Y (ω) = ∑ X ( ) M k=0 M
Proof:
M−1N−1 N−1 M−1 1 − ω− π 1 −ω π Y(ω) = ∑∑x[l]e j( 2 k)l /M =∑∑x[l]e j l /M e j 2kl /M M k=0 l=0 M l=0 k=0
M −1 π Only when l = Mn , ∑e j2 kl / M = M k =0
M −1 π else ∑e j2 kl / M = 0 (think Why ??) k=0
N / M −1 ω Thus Y (ω) = ∑ x[Mn]e j n n=0 y[n] = x[Mn]
Operator: Up Sampling x[n], l = Mn y[l] = 0, others Y (ω) = x(Mω)
Proof: Y ()ω) = ∑ y[l]e− jωl = ∑ x[n]e− jωMn = X(Mω l l
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 29
LAST REVIEW NOTE
1. Linear System
Finite vs. Infinite Duration Signals Right-sided, Left-sided, and Two-sided Signals The terms apply only to infinite duration signals.
Causal and Anti-Causal Signals Zero phase, Linear phase and Nonlinear phase signals Time Delay.
Linear System: (1) Homogeneity. (2) Additivity. (3) Commutative.
Shift (Time) Invariance System. Energy and Stability.
2. Convolution
Delta Function δ[n]. Standard Function u[n]. δ[n] = u[n] − u[n −1]
∞ u[n] = ∑δ (n − k) k =0 Impulse response h[n]. The signal that exits a system when a Delta Function (unit impulse) is the input. h[n] = S(δ[n])
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 30
Convolution y[n] = h[n]∗ x[n]
∞ y[i] = ∑h[ j]x[i − j] j =−∞ Properties of Delta function: δ[n] is identity for convolution. δ[n]∗ x[n] = x[n] x[n]∗ kδ[n] = kx[n] x[n]∗δ[n + s] = x[n + s]
3. Fourier Representation Fourier Response +∞ − ω X (ω) = ∑ x(n)e j n n=−∞
π 1 ω x[n] = ∫ X (ω)e jn dω 2π −π X (ω) = X (ω + 2π )
Discrete Fourier Transform (DFT) x[n + N] = x[n]
N−1 X[k] = ∑ x[n]e− j 2πkn / N i=0
N −1 1 π x[n] = ∑ X[k]e j2 kn/ N N k=0 X[k + N] = X[k]
Frequency Response of Sequence h[n]
+∞ H (ω) = ∑ h(n)e− jωn n=−∞ Y(ω) H (ω) = X (ω)
H (ω) = Im H (ω)2 + Re H(ω)2 Magnitude of Transfer Function
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 31
ImH (ω) Arg()H (ω) =arctan Phase of Transfer Function Re H (ω)
Ideal Low-pass Filter π ω ≤ 1, H (ω) = 2 π 0, < ω ≤ π 2 kπ sin 2 h[n] = , h[2k] = 0. kπ
Frequency Transform of Convolution y[n] = h[n]∗ x[n], then Y(ω) = X (ω)H(ω)
+∞ − ω X (ω) = ∑ x(n)e j n n=−∞
∗ y[n] = x[−n], Y (ω) = X (ω) or X (−ω)
4. Fourier Pairs Frequency Response Delta Function x[n] = δ[n] , X(ω) = 1
− ω x[n] = δ[n - m], X (ω) = e jm
Square Function
Example 1: x[n] = 1, n = 0, M –1 1− e− jMω X (ω) = 1− e− jω
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 32
Example 2: x[n] = 1, |n| = 0, M ω(2M +1) sin 2 X (ω) = ω sin 2 y[n] = x[n - M], Y(ω) = X(ω)e-jωM y[n] = x[n]ejan, Y(ω) = X(ω - a)
DFT Delta Function X[k] = DFT(δ[n]) = 1 X[k] = DFT(δ[n - m]) = e-j2πkm/N
1 X[k] = δ[k], x[n] = . N e j 2πnM / N X[k] = δ[k - M], x[n] = . N
Square Function x[n] = 1, n = 0, M - 1
− j 2πkM / N 1− e π − sin(πkM / N) X[k] = = e j k (1 M ) / N 1− e− j 2πk / N sin(πk / N ) x[n] = 1, n = 0, M ; and x[n] = 1, n = N - M, N – 1. sin(πk(2M +1)/ N) X[k] = sin(πk / N)
Frequency Region: X[k] = 1, when k = 0, M ; X[k] = 1, when k = N - M, N – 1. 1 sin(πn(2M +1)/ N) Then x[n] = N sin(πn / N)
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 33
DFT( x[n - M] ) = X[k]e-j2πkM/N IDFT( X[k - M] ) = x[n]ej2πnM/N
5. FAST FOURIER TRANSFORM
Signal Length N = 2L
Let x1[r] = x[2r], x2[r] = x[2r + 1], r = 0, N / 2 – 1.
-j2πk/N X[k] = X1[k] + X2[k]e , k = 0, N / 2 – 1.
-j2πk/N X[k + N/2] = X1[k] – X2[k]e , k = 0, N/2 – 1.
Butterfly Algorithm
-j2πk/N X1[k] X1[k] + X2[k]e (X[k])
+
–
-j2πk/N -j2πk/N X2[k] e X1[k] – X2[k]e (X[k + N/2])
One Butterfly: generates 2 Dots of DFT Each Layer: N/2 Butterflies Each Butterfly: 1 multiply + 2 add
Layer Number: L = log2N
Calculation Efficiency: FFT Cost = (N/2) × L × (1 Multiply + 2 Add)
= N log N Multiply + Nlog N Add 2 2 2 DFT Cost = N2 Multiply + N(N –1) Add
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 34
Filter Finite Impulse Response (FIR) filter – Finite Convolution Infinite Impulse Response (IIR) filter – Recursion Low-pass, High-pass, Band-pass, and Band-reject filters
6. Wavelet
Operator: Down Sampling y[n] = x[Mn]
1 M −1 ω − 2πk Y(ω) = ∑ X ( ) M k =0 M
Operator: Up Sampling x[n], l = Mn y[l] = 0, others Y(ω) = X(Mω)
φ = −m / 2φ −m − ψ = −m / 2ψ −m − m,n (x) 2 (2 x n) , m,n (x) 2 (2 x n) . φ(x / 2) = 2∑h[n]φ(x − n) , ψ (x / 2) = 2∑ g[n]φ(x − n) . n n
∑h[n] = 1, ∑ g[n] = 0 n n
= − = − Decomposition Cm[n] 2∑Cm−1[i]h[i 2n] , Dm[n] 2∑Cm−1[i]g[i 2n] . i i = − + − Reconstruction Cm−1[n] 2 ∑Cm[i]h[n 2i] ∑ Dm[l]g[n 2l] . i l
For Computer Implementation
= − = − Decomposition Cm[n] ∑Cm−1[i]h[i 2n] , Dm[n] ∑Cm−1[i]g[i 2n]. i i = × − + − Reconstruction Cm−1[n] 2 ∑Cm[i]h[n 2i] ∑ Dm[l]g[n 2l] i l
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston PHYS 7397 DIGITAL SIGNAL PROCESSING FOR SCIENTISTS 35
Perfect Reconstruction Condition
H (ω) 2 + G(ω) 2 = 1 H (ω)H ∗ (ω −π ) + G(ω)G∗ (ω −π ) = 0
Conjugate Quadrature Filter (CQF) G(ω) = H ∗ (ω −π )[± e− jω(2M +1) ]
∗ ∗ Then H(ω)H (ω −π ) + G(ω)G (ω −π ) = 0
2 2 H(ω) + H(ω −π ) = 1 G(ω) 2 + G(ω −π ) 2 = 1
CQF Example (Let M = 0)
∗ − ω − G(ω) = H (ω −π )e j , g[k]=()−11kh[1−k].
∗ −ω G(ω)=−H(ω−π)ej , g[k]=()−1kh[1−k].
Dr. Zhuoer Shi, [email protected], Department of Physics, University of Houston