3. The Wiener Filter 3.1 The Wiener-Hopf Equation The Wiener filter theory is characterized by: 1. The assumption that both signal and noise are random processed with known spectral characteristics or, equivalently, known auto- and cross-correlation functions. 2. The criterion for best performance is minimum mean-square error. (This is partially to make the problem mathematically tractable, but it is also a good physical criterion in many applications.) 3. A solution based on scalar methods that leads to the optimal filter weighting function (or transfer function in the stationary case). z(nnn )=+ s( ) v( ) hi() or H( z ) s(ˆ n ) JWSS LTI WSS RRRRsvzszs, , , or SzSzSzS(),(),(), v z sz () z known 2 J=− E{}[] sn() snˆ () Fig. 3.1-1 Wiener Filter Problem 1 We now consider the filter optimization problem that Wiener first solved in the 1940s. Referring to Fig. 3.1-1, we assume the following: 1. The filter input is an additive combination of signal and noise, both of which are jointly wide- sense stationary (JWSS) with known auto- and cross-correlation functions (or corresponding spectral functions). 2. The filter is linear and time-invariant. No further assumption is made as to its form. 3. The output is also wide-sense stationary. 4. The performance criterion is minimum mean-square error. The estimate s(ˆ n ) of a signal s(n) is given by the convolution representation ∞∞ s()()z()()z()()z(ˆ nhnn=∗=∑∑ hnii − = hini −), (3.1-1) ii=−∞ =−∞ where z(i) is the measurement and hn() is the impulse response of the estimator. Let H denote the region of support of hn(), defined by H =≠{nhn:() 0}. Then, Eq. (3.1-1) can be rewritten as 2 s(ˆ nhin )=∑ ( )z(−i ). (3.1-2) i∈H Let the mean-square estimation error (MSE) J be 2 JE=−{[s( n ) s(ˆ n )] } ⎪⎪⎧⎫⎡⎤⎡⎤ =−En⎨⎬⎢⎥s( )∑∑ hinin ( )z( −− )⎢⎥ s( ) hjnj ( )z( − ) (3.1-3) ⎩⎭⎪⎪⎣⎦ij∈∈HH⎣⎦ 2 =−E{}s ( n ) 2∑∑ hiE (){} s()z( n n −+ i )∑ hih () ( jE ) { z( n −− i )z( n j ). } ii∈∈HHj∈H To minimize the MSE, take the partial derivatives of J with respect to hi(), for each hi()≠ 0. Then, set the result equal to zero ∂J =−2Enni{} s( )z( − ) + 2∑ hjEninj ( ) { z(− )z( − ) } ∂hi() j∈H (3.1-4) = 0. Solving Eq. (3.1-4), we find ∑ hjE( ){ z( n− i )z( n−= j )} E{ s( n )z( n − i )} , i ∈H (3.1-5) j∈H 3 which we may express in the form of ∑ hjR()zs ( j− i )= Rz (), i i∈H , (3.1-6) j∈H where Rkz () is the autocorrelation function of z(n) and Rksz () is the crosscorrelation function of s(n) and z(n). Eq. (3.1-6) is the discrete-time Wiener-Hopf equation. It is the basis for the derivation of the Wiener filter. 3.2 The FIR Wiener Filter Suppose the filter has an impulse response hn() with support H, where H is a finite set, e.g., H = {0,1,2,…,N-1}. Then the impulse response hn() has a finite duration, and this type of filter is called a finite impulse response (FIR) filter. For the FIR filter, the Wiener-Hopf equation is written N −1 ∑hjR()zs ( j− i )=≤≤ Rz (),0 i i N− 1. (3.2-1) j=0 4 This may be written in matrix form ⎡⎤RRzz(0) (1) " RN z(− 1) ⎡⎤ h(0) ⎡ Rsz (0) ⎤ ⎢⎥RR(1) (0) " RN(− 2) ⎢⎥ h(1) ⎢ R (1) ⎥ ⎢⎥zz z ⎢⎥= ⎢ sz ⎥, (3.2-2) ⎢⎥##%# ⎢⎥ # ⎢ # ⎥ ⎢⎥⎢⎥⎢ ⎥ ⎣⎦RNzz(−− 1) RN ( 2)" R z(0) ⎣⎦hN(1)− ⎣RNsz (1)− ⎦ where we have used the fact that Rkzz()= R (− k). Denote Eq. (3.2-2) in a convenient form Rhz= rsz (3.2-3) where the autocorrelation function Rz is symmetric and positive-definite. Eq. (3.2-3) is solved for h, −1 hR= zrsz. (3.2-4) This is the FIR Wiener filter of order N −1. Note that Rz is a Toeplitz matrix and Eq. (3.2-4) may be solved using a computationally-efficient method such as the Levinson-Durbin algorithm. 5 Mean-Square Error (MSE) We may write the MSE of the FIR Wiener filter ⎧⎫⎡⎤N −1 MSE=−−En⎨⎬ s( )⎢⎥ s( n )∑ hini ( )z( ) ⎩⎭⎣⎦i=0 (3.2-5) N −1 =−Enn{}s( )s( )∑ hiEnni ( ) { s( )z( − ) } . i=0 To simplify Eq. (3.2-5), rewrite Eq. (3.1-5) in the following form ⎪⎪⎧⎫⎡⎤ Enin⎨⎬z(−− )s()⎢⎥∑ hjnj ()z( −= ) 0 ⎩⎭⎪⎪⎣⎦j∈H Enin{}z(−−= )[] s( ) s(ˆ n ) 0 (3.2-6) Enin{}z(− )s( )=∈ 0, iH. This relation is known as the orthogonality principle for LTI estimators. Applying this relation to Eq. (3.2-5) gives 6 MSE= Enn{ s( )s( )} N−1 2 =−En{}s ( )∑ hiEnni ( ){} s( )z( − ) i=0 N −1 =−RhiRiss(0)∑ ( )z ( ) i=0 (3.2-7) T =−Rhrss(0)z . Observe that if no filtering is performed and we simply use s(ˆ nz )= (n ), the MSE is MSE=−=−+EnznR s() ()2 (0)2 R (0) R (0). (3.2-8) no filter {[]} s sz z We can use Eq. (3.2-8) to measure the effectiveness of the Wiener filter. The reduction in MSE due to Wiener filtering is given by ⎛⎞MSEno filter reduction in MSE= 10log10 ⎜⎟dB . (3.2-9) ⎝⎠MSEfilter We can inspect the MSE to determine whether the filter performance is acceptable. If the MSE is too high, we may wish to use a larger value of N. 7 Example 3.2-1 k Suppose we have a signal s(n ) with autocorrelation function Rks ( )= 0.95 . The signal is 2 observed in the presence of additive white noise with variance σ v = 2 . Hence Rkv ()= 2(δ k). s(n ) and v(n ) are uncorrelated, zero-mean, JWSS random processes. We would like to design the optimum LTI second-order filter h. Since the filter is the second-order, we set N = 3. The matrix equation Eq. (3.2-2) to be solved is ⎡⎤RRRzzz(0) (1) (2) ⎡⎤h(0) ⎡Rsz (0)⎤ ⎢⎥RR(1) (0) R (1)⎢⎥ h (1)= ⎢ R (1) ⎥. (3.2-10) ⎢⎥zz z⎢⎥⎢ sz⎥ ⎢ ⎥ ⎣⎦⎢⎥RRRzzz(2) (1) (0) ⎣⎦⎢⎥h(2) ⎣Rsz (2)⎦ For an additive-noise measurement model z(nn )=+ s( ) v(n ), the autocorrelation functions Rkz () and Rksz () may be given as follows 8 Rkz ()=− E{ z()z( n n k )} =+En{}[][]s( ) v( n ) s( nknk −+− ) v( ) (3.2-11) =−+E{}[]s( n )s( nk ) E{}[] v( n )v( nk− ) =+Rksv( ) Rk ( ), and Rksz ()=−+− E{ s()s( n[ nk ) v( nk )]} =−+E{}[]s( n )s( nk ) E{}[] s( n )v( nk− ) (3.2-12) = Rks ( ). According to Eqs. (3.2-11) and (3.2-12), we have k Rz (kk )=+ 0.95 2δ ( ) k Rksz ( )= 0.95 . Then Eq. (3.2-10) becomes 9 ⎡⎤3 0.95 0.9025⎡h (0)⎤⎡ 1 ⎤ ⎢⎥0.95 3 0.95⎢h (1)⎥= ⎢ 0.95 ⎥. (3.2-13) ⎢⎥⎢⎥⎢⎥ ⎣⎦⎢⎥0.9025 0.95 3⎣⎢h (2)⎦⎥⎣⎢ 0.9025⎦⎥ The solution of Eq. (3.2-13) is T h = [0.2203 0.1919 0.1738] . The MSE of this estimator can be computed via Eq. (3.2-7), T MSE=−=Rhrss (0)z 0.4405. (3.2-13) For comparison, we remark that without any filtering, MSEno filter= RRRR s (0)−+= 2sz (0) z (0)v (0)= 2. (3.2-14) The FIR Wiener filter has reduced the MSE by ⎛⎞MSEno filter reduction in MSE= 10log10 ⎜⎟dB ⎝⎠MSEfilter ⎛⎞2 =10log10 ⎜⎟ (3.2-15) ⎝⎠0.4405 ≈ 6.5dB . 10 3.3 The Noncausal Wiener Filter Rewrite the Wiener-Hopf equation, Eq. (3.1-6), ∑ hjR()zs ( j− i )=∈ Rz (), i i H j∈H and let H = Z, where Z is the set of integers: Z =−−{",2,1,0,1,2,"}. Then, the Wiener-Hopf equation becomes ∞ ∑ hjR( )zs ( j− i )= Rz ( i ), for all i∈Z . (3.3-1) j=−∞ Take the z-transform of Eq. (3.3-1) H ()zSzsz () z= S () z so that Sz() Hz()= sz . (3.3-2) Szz () Eq. (3.3-2) is the solution for the noncausal Wiener filter. 11 Mean-Square Error (MSE) As with the FIR Wiener filter of Section 3.2, we can derive an expression for the MSE. The MSE similar to Eq. (3.2-7) can be written as ∞ MSE=−RhiRs (0)∑ ()sz ()i. (3.3-3) i=−∞ Example 3.3-1 Let us apply a noncausal Wiener filter to the filtering problem of Example 3.2-1. In that example we found that k k Rz (kk )=+ 0.95 2δ ( ), and Rksz ( )= 0.95 . (3.3-4) Take z-transforms of Eq. (3.3-4) 1− (0.95)2 Sz()=+2 z (1−− 0.95zz−1 )(1 0.95 ) 2.3955(1−− 0.7931zz−1 )(1 0.7931 ) 1 =<, 0.7931 z < (1−− 0.95zz−1 )(1 0.95 ) 0.7931 12 and 1− (0.95)2 1 Sz()=<, 0.95 z<. sz (1−− 0.95zz−1 )(1 0.95 ) 0.95 The noncausal Wiener filter, Eq. (3.3-2), is given by Sz() 0.0975 Hz()==sz Sz( ) 2.3955(1−− 0.7931z−1 )(1 0.7931 z ) z 0.1097(1− (0.7931)2 ) 1 =<, 0.95z < . (1−− 0.7931zz−1 )(1 0.7931 ) 0.95 The impulse response of the filter is hn( )= 0.1097(0.7931) n . The mean-square errors associated with the noncausal Wiener filter is according to Eq. (3.3-3), 13 ∞ MSE=−RhnRnss (0)∑ ()z () n=−∞ ∞ =−(0.95)0 ∑ 0.1097(0.7931)nn (0.95) n=−∞ ∞−∞ ⎡ nn −−n n⎤ =−1 0.1097⎢∑∑ (0.7931) (0.95)+ (0.7931) (0.95) ⎥ ⎣ nn==01−⎦ ∞ ⎡⎤nn =−1 0.1097⎢⎥ 2∑ (0.7931) (0.95)− 1 ⎣⎦n=0 ⎡⎤2 =−1 0.1097 ⎢⎥−=1 0.2195.