Stochastic signals and processes Lec. 8 Samuel Schmidt 25-10-2011
• Introduction to Wiener filters • Linear filters and orthogonality • The time continues Wiener filter • The time discrete Wiener filter • Example
How to filter a signal Y(t) so it became as similar to a desired signal S(t) as possible.
Y(t) S(t) 5 Wiener 2
Y(t) 0 0 ≈ S(t) -5 filter 0 1 2 3 4 5 -2 Time (s) 0 1 2 3 4 5 L[Y(t)] Time (s) • Noise contaminated signal
Y(t)=S(t)+N(t) Wiener filter
• Estimate the linear filter L[·] so Ŝ(t) become as close to S(t) as possible Ŝ(t)=L[Y(t)] Se(t) 5 e(t) 5 0
Se(t) 0 e(t) -5 0 1 2 3 4 5 -5 Time (s) 0 1 2 3 4 5 Time (s)
Y(t) 5
Y(t) 0
-5 0 1 2 3 4 5 Time (s) S(t) 2
0 S(t)
-2 0 1 2 3 4 5 Time (s) • Introduction to Wiener filters • Linear filters and orthogonality • The time continues Wiener filter • The time discrete Wiener filter • Example
• Y(t),S(t) and N(t) are zero mean stationary processes
• L[·] is a linear filter • FIR filters (M-order)
Y(n)=ω0 X(n)+ ω1 X(n-1)+ ω2 X(n-2)….. ωM-1 X(n-M+1)
Or like
푀−1
푌 푛 = 휔푘푋(푛 − 푘) 푘=0
• In linear system the order of multiplication and addition is irrelevant.
• Defined by super position
Ta x [n]b x [n] aTx [n]bTx [n] 1 2 1 2
• Linear algebra: Two vectors is orthogonal if the dot product is zero
• That corresponds zero correlation between two signals
N(t) S(t) 5 2
0
0 N(t) S(t)
-2 -5 0 1 2 3 4 5 0 1 2 3 4 5 Time (s) Time (s) S(t) vs. N(t) Y(t) vs. N(t) Orthogonal (r=0.0) Non-orthogonal (r=0.69)
Scatter between S(t) and N(t) Scatter between Y(t) and N(t) 4 4
3 3
2 2
1 1
N 0 0 N(t)
-1 -1
-2 -2
-3 -3
-4 -4 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 -5 -4 -3 -2 -1 0 1 2 3 4 5 S X(t) • If two process are orthogonal for all delays in a give range the process are also orthogonal after linear transformation.
IF
Then S(t) • 5
Error 0 S(t)
ε(t)= Ŝ(t) - S(t) -5 0 1 2 3 4 5 Time (s) - • Se(t) Mean square error 5
0
Se(t)
2 2 -5 E[|ε(t)| ]= E[| Ŝ(t) - S(t) | ] 0 1 2 3 4 5 Time (s) E[|ε(t)|2]=E[| L[Y(ξ)] - S(t) |2] e(t)= 5
0 2 2 ξ ξ 2 e(t) E[|ε(t)| ]=E[S(t) - 2 S(t)L[Y( )] + L[Y( )] ] -5 0 1 2 3 4 5 Time (s)
4 Y(t) e(t) 3 S(t) Se(t)
2
1
0
-1
-2
-3
1.8 1.9 2 2.1 2.2 2.3 2.4 2.5 2.6 2.7 Time (s) • The Minimum mean-square error is obtained when the error ε(t) is Orthogonal to the unfiltered signal Y(ξ),
where ξ is in the range [ti, tf].
E[Y(ξ) e(t)]=E[(S(t)- Ŝ(t) ) Y(ξ) ]=0
1 0.8
• That means thats the 0.6 filter is optimal when the 0.4
0.2 e(t) Correlation between Y(t+k) 0 -0.2 and e(t) is low for all k -0.4
-0.6
-0.8 -5 -4 -3 -2 -1 0 1 2 3 4 5 Y(t) • The mean-square error is mean when the correlation between the e(t) and the signal is zero, that is the optimal filter Y(t) and e(t) 5 Y(t) 4 e(t)
3
2
1
0
-1
-2
-3
-4
-5 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Time (s) The Minimum mean-square error can be estimated. Since E[(S(t)- Ŝ(t) ) Y(ξ) ]=0 Is E[(S(t)- L[Ŝ(t)] ) L[Y(ξ)] ]=0
Thereby is the Minimum mean-square
e =E[|ε(t)|2]=E[S(t) 2 - 2 S(t)L[Y(ξ)] + L[Y(ξ)]2]= E[(S(t)-L[Y(ξ)]) S(t)] m
em=E[(S(t)-L[Y(ξ)]) S(t)] • Introduction to Wiener filters • Linear filters and orthogonality • The time continues Wiener filter • The time discrete Wiener filter • Example
4 Y(t) e(t) 3 S(t) Se(t)
2
1
0
-1
-2
-3
1.8 1.9 2 2.1 2.2 2.3 2.4 2.5 2.6 2.7 Time (s) • Determine the impulse response h(ξ) of the filter.
• Use the Minimum mean-square error principle which requires the error S(t)-Ŝ(t) to be Orthogonal (correlation=0) to Y (ξ) • If we assume stationarity and time invariance
• Thereby we “just” need to solve for h(ξ) Fourier transform
cross correlation between S(t) and Y(t)
power spectrum of Y(t) Therefor H(f) is the cross correlation between S(t) and Y(t) dived with the power spectrum of Y(t) • If S(t) and N(t) are statistically independent and zero mean
Therefore H(f) can be determined from the power spectrums of noise and the desired signal PSD S(f) 20
0
-20
-40 S(f) dB/Hz
-60
-80 0 2 10 10 Frequency (Hz) PSD S (f)+S (f) ss nn 20
0
-20
(f) (f) dB/Hz nn
-40
(f)+S ss
S -60
-80 0 2 10 10 Frequency (Hz) PSD H(f) 0
-20
-40 H(f) dB/Hz H(f) -60
-80 0 2 10 10 Frequency (Hz) • The minimum mean-square error can be calculated: • The h(t) is not causal and thereby is the filter not realizable in real-time
• There solutions based on spectral factorizations ∞ −훼|휏| −푗2휋푓휏 2훼 푆푠푠 푓 = 푒 푒 푑휏 = 2 2 2 Wolfram −∞ 훼 + 4휋 푓
∞ 푁0 −푗2휋푓휏 푁0 푆푛푛 푓 = 훿 휏 푒 푑휏 = −∞ 2 2
2훼 4훼 훼2+4휋2푓2 푁0 퐻 푓 = 2훼 푁0=4훼 2 2 2 + 푁 +훼 +4휋 푓 훼2+4휋2푓2 2 0 2훼 2 2 2 4훼 푎 퐻 푓 = 훼 +4휋 푓 = 푁0 = 2훼 푁0 4훼 +훼2+4휋2푓2 푏2+4휋2푓2 2 2 2+ 푁0 훼 +4휋 푓 2 • Inver Fourier transform: • Introduction to Wiener filters • Linear filters and orthogonality • The time continues Wiener filter • The time discrete Wiener filter • Example
Ŝ(n)=ω0 Y(n)+ ω1 Y(n-1)+ ω2 Y(n-2)….. ωM-1 Y(n-M+1) • Different methods for estimation of filter coefficients: – Spectrum based • (Estimate impulse response h(n))
Y(n)=h(0) X(n)+ h(1) ω1 X(n-1)+ h(2) X(n-2)….. h(M-1)X(n-M+1)
– Mean square error • (Estimate filter coefficients ω(n) )
ω(n)
Y(n)=ω0 X(n)+ ω1 X(n-1)+ ω2 X(n-2)….. ωM-1 X(n-M+1) • As in the continues domain the transfer function of H(z) is the cross spectrum between S(t) and Y(t) dived by the PSD of Y(t) (Real cases) • The discrete filter
• Determine the coefficients 휔 which minimize the Mean square error
Error: 푒 푛 = 푆 푛 − 푆(푛)
Mean square error: 퐶 휔 = 퐸 푒 푛 푒 푛 = 퐸 푒 푛 2 = 퐸[(푆 푛 − 푆 푛 )2] • Denote the filter coefficients
푇 휔 = 휔0 휔1 휔2 … 휔푀−1
• Denote the signal
푌(푛) = 푌 푛 푌 푛 − 1 푌 푛 − 2 … . 푌(푛 − 푀) 푇 • Thereby filter equations can be described using the dot product of the filter coefficients and the signal.
푌 푛 푌 푛 − 1 푆 푛 = 휔푇푌 푛 = 휔 휔 휔 … 휔 0 1 2 푀−1 푌 푛 − 2 푌(푛 − 푀)
Ŝ(n)=ω0 Y(n)+ ω1 Y(n-1)+ ω2 Y(n-2)….. ωM-1 Y(n-M+1)
푆 푛 = 휔푇푌 푛 =푌 푛 푇휔
(continued) • Since: 푆 푛 = 휔푇푌 푛 =푌 푛 푇휔
• Mean square error is
퐶 휔 = 퐸 푒 푛 푒 푛 = 퐸[ 푆 푛 − 휔푇푌 푛 (푆 푛 − 푌 푛 푇휔)] • Which can be expanded to
퐶 휔 = 퐸 푆 푛 푆 푛 − 휔푇퐸[푌 푛 푆(푛)] − 퐸[푆 푛 푌(푛)푇]휔+ 휔푇E[푌 푛 푌(푛)푇]휔 (continued) • Expanded Mean square error explained
퐶 휔 = 퐸 푆 푛 푆 푛 − 휔푇퐸[푌 푛 푆(푛)] − 퐸[푆 푛 푌(푛)푇]휔+ 휔푇E[푌 푛 푌(푛)푇]휔
Transformed σ 2 Signal variance s Cross correlation vector version of R Autocorrelation matrix of between Y(n) and S(n) ys Y(n) Rys =[rys(0) rys(1) ……..rys(M-1) ] ryy(0) ryy(1) ryy(2) rys(τ)=E[Y(n+τ)S(n)] , τ=0,1,2,…M-1 Ryy= ryy(−1) ryy(0) ryy(1) ryy(−2) ryy(−1) ryy(0)
• Compact form ryy(τ)=E[Y(n+τ)Y(n)] , It an Toeplitz matrix 퐶 휔 = σ 2 − 휔푇R − R 푇 휔+ 휔푇R 휔 s ys ys yy Page 432 (continued) • The Mean square error is a second order equation.
2 푇 푇 푇 퐶 휔 = σs − 휔 Rys − Rys 휔+ 휔 Ryy휔
• Thereby does the equation has a global minimum which can be identified by differentiation according to 휔 and solving for zero Differentiating equation components: 푑 푇 풅푪(흎) 풅 (휔 R )= R = (σ 2 − 휔푇R − R 푇 휔+ 휔푇R 휔) = −2R +ퟐR 휔 = 0 푑휔 ys ys 풅흎 풅흎 s ys ys yy ys yy 푑 (R 푇 휔)= R 푑휔 ys ys
푑 • The filter coefficients is then easily solved. (휔푇R 휔)= ퟐR 휔 푑휔 yy yy
−ퟏ Wiener-Hopf equation: 흎 = Ryy Rys Page 433-35 • Error e(n)=S(n)-YT(n)ω • Minimum mean square error 2 푇 2 푇 −ퟏ 푒푚 = 휎푠 − Rys 휔 = 휎푠 − Rys Ryy Rys • Introduction to Wiener filters • Linear filters and orthogonality • The time continues Wiener filter • The time discrete Wiener filter • Example
• Construct a 4th order wiener filter which estimates S(t)
Y(t) S(t) 5 Wiener 2
Y(t) 0 0 filter ≈ S(t) -5 0 1 2 3 4 5 -2 Time (s) L[Y(t)] 0 1 2 3 4 5 Time (s) Autocorrelation of Y(t) (M=4) xcorr(Y,4)
R ( ) yy 10000 8000
6000
4000 -5 0 5 ryy(τ)=[9970 5089 4993 5046 4984], τ=[0 ….4]
Make Autocorrelation matrix toeplitz(ryy)
9970 5089 4993 5046 4984 5089 9970 5089 4993 5046 Ryy= 4993 5089 9970 5089 4993 5046 4993 5089 9970 5089 4984 5046 4993 5089 9970 Cross correlation of Y(t) and S(t) (M=4) xcorr(S,Y,4)
5120
5115
5110 -5 0 5
Rys(τ)= [5116 5115 5114 5113 5111], τ=[0 ….4] ω
• The filter coefficients is estimated using the Wiener-Hopf equation
−ퟏ 흎 = Ryy Rys
0.1683 − 0.0359 − 0.0323 − 0.0342 − 0.0323 5116 0.1720 −0.0359 0.1697 − 0.0356 − 0.0312 − 0.0342 5115 0.1680 ω = −0.0323 − 0.0356 0.1690 − 0.0356 − 0.0323 5114 = 0.1699 −0.0342 − 0.0312 − 0.0356 0.1697 − 0.0359 5113 0.1677 −0.0323 − 0.0342 − 0.0323 − 0.0359 0.1683 5111 0.1710
Se(n)= 0.172 Y(n)+ 0.1680Y(n-1)+ 0.1699 Y(n-2)+ 0.1677 Y(n-3)+ 0.171Y(n-4) Ŝ
Se(t) 3
Se(t) 2 S(t)
1
0 Se(t)
-1
-2
-3 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Time (s) • When – Both noise and desired signal are stationary processes – You know the desired signal or at least the ideal characteristics of the desired signal. • In these case the wiener filter is the optimal filter • Exercise 7.8 • Matlab Exercise • A blood pressure signal is contaminated by white noise. (Variable name y). The first 8 seconds of the recording is available in a Noise free version (Variable name s) • Make a FIR filter for filtering of the whole signal. • Select an appropriate filter order • http://person.hst.aau.dk/sschmidt/ST/BP.mat