508 11. Wiener Filtering noisy random walk 8 observations y n 12 signal x 6 n prediction x n/n−1 4 Linear Prediction 2 0 −2 −4 0 50 100 150 200 250 300 time samples, n 12.1 Pure Prediction and Signal Modeling In Sec. 1.17, we discussed the connection between linear prediction and signal modeling. Here, we rederive the same results by considering the linear prediction problem as a special case of the Wiener filtering problem, given by Eq. (11.4.6). Our aim is to cast the results in a form that will suggest a practical way to solve the prediction problem and hence also the modeling problem. Consider a stationary signal yn having a signal model = 2 −1 Syy(z) σ B(z)B(z ) (12.1.1) as guaranteed by the spectral factorization theorem. Let Ryy(k) denote the autocorre- lation of yn : Ryy(k)= E[yn+kyn] The linear prediction problem is to predict the current value yn on the basis of all the past values Yn−1 ={yi , −∞ <i≤ n − 1}. If we define the delayed signal y1(n)= yn−1, then the linear prediction problem is equivalent to the optimal Wiener filtering problem of estimating yn from the related signal y1(n). The optimal estimation filter H(z) is given by Eq. (11.4.6), where we must identify xn and yn with yn and y1(n) of the present −1 notation. Using the filtering equation Y1(z)= z Y(z), we find that yn and y1(n) have the same spectral factor B(z) = −1 = = 2 −1 Sy1y1 (z) (z )(z)Syy(z) Syy(z) σ B(z)B(z ) and also that = = 2 −1 Syy1 (z) Syy(z)z zσ B(z)B(z ) Inserting these into Eq. (11.4.6), we find for the optimal filter H(z) 2 −1 1 Syy (z) 1 zσ B(z)B(z ) H(z)= 1 = , or, 2 −1 2 −1 σ B(z) B(z ) + σ B(z) B(z ) + 1 H(z)= zB(z) (12.1.2) B(z) + 510 12. Linear Prediction 12.1. Pure Prediction and Signal Modeling 511 The causal instruction can be removed as follows: Noting that B(z) is a causal and The prediction error stable filter, we may expand it in the power series en/n−1 = yn − yˆn/n−1 −1 −2 −3 is identical to the whitening sequence driving the signal model (12.1.1) of y , indeed, B(z)= 1 + b1z + b2z + b3z +··· n n = − ˆ = − = − −1 The causal part of zB(z) is then E(z) Y(z) Y(z) Y(z) H(z)Y1(z) Y(z) H(z)z Y(z) −1 −2 −1 −2 −1 1 zB(z) + = [z + b1 + b2z + b3z +··· ]+= b1 + b2z + b3z +··· = 1 − z H(z) Y(z)= Y(z)= (z) B(z) −1 −2 −3 = z b1z + b2z + b3z +··· = z B(z)−1 Thus, in accordance with the results of Sec. 1.13 and Sec. 1.17 The prediction filter H(z) then becomes en/n−1 = yn − yˆn/n−1 = n (12.1.4) 1 1 H(z)= z B(z)−1 = z 1 − (12.1.3) B(z) B(z) The input to this filter is y1(n) and the output is the prediction yˆn/n−1. Example 12.1.1: Suppose that yn is generated by driving the all-pole filter yn = 0.9yn−1 − 0.2yn−2 + n by zero-mean white noise n. Find the best predictor yˆn/n−1. The signal model in this case −1 −2 is B(z)= 1/(1 − 0.9z + 0.2z ) and Eq. (12.1.3) gives Fig. 12.1.1 Linear Predictor. 1 z−1H(z)= 1 − = 1 − (1 − 0.9z−1 + 0.2z−2)= 0.9z−1 − 0.2z−2 B(z) An overall realization of the linear predictor is shown in Fig. 12.1.1. The indicated dividing line separates the linear predictor into the Wiener filtering part and the input The I/O equation for the prediction filter is obtained by part which provides the proper input signals to the Wiener part. The transfer function −1 −1 −2 from yn to en/n−1 is the whitening inverse filter Y(z)ˆ = H(z)Y1(z)= z H(z)Y(z)= 0.9z − 0.2z Y(z) 1 −1 and in the time domain A(z)= = 1 − z H(z) B(z) yˆn/n−1 = 0.9yn−1 − 0.2yn−2 which is stable and causal by the minimum-phase property of the spectral factorization Example 12.1.2: Suppose that (12.1.1). In the z-domain we have (1 − 0.25z−2)(1 − 0.25z2) Syy(z)= (1 − 0.8z−1)(1 − 0.8z) E(z)= (z)= A(z)Y(z) ˆ Determine the best predictor yn/n−1. Here, the minimum phase factor is and in the time domain −2 1 − 0.25z ∞ B(z)= 1 − 0.8z−1 en/n−1 = n = amyn−m = yn + a1yn−1 + a2yn−2 +··· m=0 and therefore the prediction filter is − − − The predicted estimate yˆ − = y − e − is 1 1 − 0.8z 1 0.8z 1 − 0.25z 2 n/n 1 n n/n 1 z−1H(z)= 1 − = 1 − = B(z) 1 − 0.25z−2 1 − .25z−2 yˆn/n−1 =− a1yn−1 + a2yn−2 +··· The I/O equation of this filter is conveniently given recursively by the difference equation These results are identical to Eqs. (1.17.2) and (1.17.3). The relationship noted above yˆn/n−1 = 0.25yˆn−2/n−3 + 0.8yn−1 − 0.25yn−2 between linear prediction and signal modeling can also be understood in terms of the 512 12. Linear Prediction 12.2. Autoregressive Models 513 gapped-function approach of Sec. 11.7. Rewriting Eq. (12.1.1) in terms of the prediction- These equations are known as the normal equations of linear prediction [915–928]. error filter A(z) we have They provide the solution to both the signal modeling and the linear prediction prob- 2 σ lems. They determine the model parameters {a ,a ,... ; σ2} of the signal y directly in S (z)= (12.1.5) 1 2 n yy −1 A(z)A(z ) terms of the experimentally accessible quantities Ryy(k). To render them computation- from which we obtain ally manageable, the infinite matrix equation (12.1.11) must be reduced to a finite one, σ2 A(z)S (z)= (12.1.6) and furthermore, the quantities Ryy(k) must be estimated from actual data samples of yy −1 A(z ) yn. We discuss these matters next. Since we have the filtering equation (z)= A(z)Y(z), it follows that = Sy(z) A(z)Syy(z) 12.2 Autoregressive Models and in the time domain { } ∞ In general, the number of prediction coefficients a1,a2,... is infinite since the pre- Rey(k)= E[nyn−k]= aiRyy(k − i) (12.1.7) dictor is based on the infinite past. However, there is an important exception to this; i=0 namely, when the process yn is autoregressive. In this case, the signal model B(z) is an all-pole filter of the type which is recognized as the gapped function (11.7.1). By construction, n is the orthog- onal complement of y with respect to the entire past subspace Y − ={y − ,k= n n 1 n k 1 1 } = B(z)= = (12.2.1) 1, 2,... , therefore, n will be orthogonal to each yn−k for k 1, 2,.... These are pre- −1 −2 −p A(z) 1 + a1z + a2z +···+apz cisely the gap conditions. Because the prediction is based on the entire past, the gapped function develops an infinite right-hand side gap. Thus, Eq. (12.1.7) implies which implies that the prediction filter is a polynomial ∞ = + −1 + −2 +···+ −p Rey(k)= E[nyn−k]= aiRyy(k − i)= 0 , for all k = 1, 2,... (12.1.8) A(z) 1 a1z a2z apz (12.2.2) i=0 The same result, of course, also follows from the z-domain equation (12.1.6). Both The signal generator for yn is the following difference equation, driven by the un- sides of the equation are stable, but since A(z) is minimum-phase, A(z−1) will be correlated sequence n : maximum phase, and therefore it will have a stable but anticausal inverse 1/A(z−1). y + a y − + a y − +···+a y − = (12.2.3) Thus, the right-hand side of Eq. (12.1.6) has no strictly causal part. Equating to zero all n 1 n 1 2 n 2 p n p n the coefficients of positive powers of z−1 results in Eq. (12.1.8). and the optimal prediction of yn is simply given by: The value of the gapped function at k = 0 is equal to σ2. Indeed, using the gap conditions (12.1.8) we find yˆ − =− a y − + a y − +···+a y − (12.2.4) n/n 1 1 n 1 2 n 2 p n p 2 = 2 = + + +··· σ E[n] E n(yn a1yn−1 a2yn−2 ) In this case, the best prediction of yn depends only on the past p samples {yn−1, = Ry(0)+a1Ry(1)+a2Ry(2)+···=Ry(0)= E[nyn] yn−2,...,yn−p}. The infinite set of equations (12.1.10) or (12.1.11) are still satisfied even though only the first p + 1 coefficients {1,a1,a2,...,ap} are nonzero. Using Eq. (12.1.7) with k = 0 and the symmetry property Ryy(i)= Ryy(−i),wefind The (p + 1)×(p + 1) portion of Eq.