Topic 8: Power spectral density and LTI systems ² The power spectral density of a WSS random process ² Response of an LTI system to random signals ² Linear MSE estimation ES150 { Harvard SEAS 1 The autocorrelation function and the rate of change ² Consider a WSS random process X(t) with the autocorrelation function RX (¿). ² If RX (¿) drops quickly with ¿, then process X(t) changes quickly with time: its time samples become uncorrelated over a short period of time. { Conversely, when RX (¿) drops slowly with ¿, samples are highly correlated over a long time. ² Thus RX (¿) is a measure of the rate of change of X(t) with time and hence is related to the frequency response of X(t). { For example, a sinusoidal waveform sin(2¼ft) will vary rapidly with time if it is at high frequency (large f), and vary slowly at low frequency (small f). ² In fact, the Fourier transform of RX (¿) is the average power density over the frequency domain. ES150 { Harvard SEAS 2 The power spectral density of a WSS process ² The power spectral density (psd) of a WSS random process X(t) is given by the Fourier transform (FT) of its autocorrelation function 1 ¡j2¼f¿ SX (f) = RX (¿)e d¿ ¡1 Z ² For a discrete-time process Xn, the psd is given by the discrete-time FT (DTFT) of its autocorrelation sequence n=1 ¡ 1 1 S (f) = R (n)e j2¼fn ; ¡ · f · x x 2 2 n=¡1 X Since the DTFT is periodic in f with period 1, we only need to 1 consider jfj · 2 . ² RX (¿) (Rx(n)) can be recovered from Sx(f) by taking the inverse FT 1 1=2 j2¼f¿ j2¼fn RX (¿) = SX (f)e df ; RX (n) = SX (f)e df ¡1 ¡1 2 Z Z = ES150 { Harvard SEAS 3 Properties of the power spectral density ² SX (f) is real and even SX (f) = SX (¡f) ² The area under SX (f) is the average power of X(t) 1 2 SX (f)df = RX (0) = E[X(t) ] ¡1 Z ² SX (f) is the average power density, hence the average power of X(t) in the frequency band [f1; f2] is ¡f1 f2 f2 SX (f) df + SX (f) df = 2 SX (f) df ¡ Z f2 Zf1 Zf1 ² SX (f) is nonnegative: SX (f) ¸ 0 for all f. (shown later) ² In general, any function S(f) that is real, even, nonnegative and has ¯nite area can be a psd function. ES150 { Harvard SEAS 4 White noise ² Band-limited white noise: A zero-mean WSS process N(t) which has N0 the psd as a constant 2 within ¡W · f · W and zero elsewhere. { Similar to white light containing all frequencies in equal amounts. { Its average power is W 2 N0 E[X(t) ] = df = N0W ¡ 2 Z W { Its auto-correlation function is N0 sin(2¼W ¿) R (¿) = = N0W sinc(2W ¿) X 2¼¿ { n For any t, the samples X(t § 2W ) for n = 0; 1; 2; : : : are uncorrelated. S (f) R (τ)=NWsinc(2Wτ) x x N/2 1/(2W) −W W f τ 2/(2W) ES150 { Harvard SEAS 5 ² White-noise process: Letting W ! 1, we obtain a white noise process, which has N0 S (f) = for all f X 2 N0 R (¿) = ±(¿) X 2 { For a white noise process, all samples are uncorrelated. { The process has in¯nite power and hence not physically realizable. { It is an idealization of physical noises. Physical systems usually are band-limited and are a®ected by the noise within this band. ² If the white noise N(t) is a Gaussian random process, then we have ES150 { Harvard SEAS 6 Gaussian white noise (GWN) 4 3 2 1 0 Magnitude −1 −2 −3 −4 0 100 200 300 400 500 600 700 800 900 1000 Time { WGN results from taking the derivative of the Brownian motion (or the Wiener process). { All samples of a GWN process are independent and identically Gaussian distributed. { Very useful in modeling broadband noise, thermal noise. ES150 { Harvard SEAS 7 Cross-power spectral density Consider two jointly-WSS random processes X(t) and Y (t): ² Their cross-correlation function RXY (¿) is de¯ned as RXY (¿) = E[X(t + ¿)Y (t)] { Unlike the auto-correlation RX (¿), the cross-correlation RXY (¿) is not necessarily even. However RXY (¿) = RY;X (¡¿) ² The cross-power spectral density SXY (f) is de¯ned as SXY (f) = FfRXY (¿)g In general, SXY (f) is complex even if the two processes are real-valued. ES150 { Harvard SEAS 8 ² Example: Signal plus white noise Let the observation be Z(t) = X(t) + N(t) where X(t) is the wanted signal and N(t) is white noise. X(t) and N(t) are zero-mean uncorrelated WSS processes. { Z(t) is also a WSS process E[Z(t)] = 0 E[Z(t)Z(t + ¿)] = E [fX(t) + N(t)gfX(t + ¿) + N(t + ¿)g] = RX (¿) + RN (¿) { The psd of Z(t) is the sum of the psd of X(t) and N(t) SZ (f) = SX (f) + SN (f) { Z(t) and X(t) are jointly-WSS E[X(t)Z(t + ¿)] = E[X(t + ¿)Z(t)] = RX (¿) Thus SXZ (f) = SZX (f) = SX (f). ES150 { Harvard SEAS 9 Review of LTI systems ² Consider a system that maps y(t) = T [x(t)] { The system is linear if T [®x1(t) + ¯x2(t)] = ®T [x1(t)] + ¯T [x2(t)] { The system is time-invariant if y(t) = T [x(t)] ! y(t ¡ ¿) = T [x(t ¡ ¿)] ² An LTI system can be completely characterized by its impulse response h(t) = T [±(t)] { The input-output relation is obtained through convolution 1 y(t) = h(t) ¤ x(t) = h(¿)x(t ¡ ¿)d¿ ¡1 Z ² In the frequency domain: The system transfer function is the Fourier transform of h(t) 1 ¡ H(f) = F[h(t)] = h(t)e j2¼ftdt ¡1 Z ES150 { Harvard SEAS 10 Response of an LTI system to WSS random signals Consider an LTI system h(t) X(t) Y(t) h(t) ² Apply an input X(t) which is a WSS random process { The output Y (t) then is also WSS 1 E[Y (t)] = mX h(¿)d¿ = mX H(0) ¡1 1Z 1 E [Y (t)Y (t + ¿)] = h(r) h(s) RX (¿ + s ¡ r) dsdr ¡1 ¡1 Z Z { Two processes X(t) and Y (t) are jointly WSS 1 RXY (¿) = h(s)RX (¿ + s)ds = h(¡¿) ¤ RX (¿) ¡1 Z { From these, we also obtain 1 RY (¿) = h(r)RXY (¿ ¡ r)dr = h(¿) ¤ RXY (¿) ¡1 ES150 { Harvard SEAS Z 11 ² The results are similar for discrete-time systems. Let the impulse response be hn { The response of the system to a random input process Xn is Yn = hkXn¡k Xk { The system transfer function is ¡j2¼nf H(f) = hne n X { With a WSS input Xn, the output Yn is also WSS mY = mX H(0) RY [k] = hj hiRX [k + j ¡ i] j i X X { Xn and Yn are jointly WSS RXY [k] = hnRX [k + n] n X ES150 { Harvard SEAS 12 Frequency domain analysis ² Taking the Fourier transforms of the correlation functions, we have ¤ SXY (f) = H (f) SX (f) SY (f) = H(f) SXY (f) where H¤(f) is the complex conjugate of H(f). ² The output-input psd relation 2 SY (f) = jH(f)j SX (f) ² Example: White noise as the input. Let X(t) have the psd as N0 S (f) = for all f X 2 then the psd of the output is N0 S (f) = jH(f)j2 Y 2 Thus the transfer function completely determines the shape of the output psd. This also shows that any psd must be nonnegative. ES150 { Harvard SEAS 13 Power in a WSS random process ² Some signals, such as sin(t), may not have ¯nite energy but can have ¯nite average power. ² The average power of a random process X(t) is de¯ned as T 1 2 PX = E lim X(t) dt T !1 2T ¡ " Z T # ² For a WSS process, this becomes T 1 2 PX = lim E[X(t) ]dt = RX (0) T !1 2T ¡ Z T But RX (0) is related to the FT of the psd S(f). ² Thus we have three ways to express the power of a WSS process 1 2 PX = E[X(t) ] = RX (0) = S(f)df ¡1 Z The area under the psd function is the average power of the process. ES150 { Harvard SEAS 14 Linear estimation ² Let X(t) be a zero-mean WSS random process, which we are interested in estimating. ² Let Y (t) be the observation, which is also a zero-mean random process jointly WSS with X(t) { For example, Y (t) could be a noisy observation of X(t), or the output of a system with X(t) as the input. ² Our goal is to design a linear, time-invariant ¯lter h(t) that processes Y (t) to produce an estimate of X(t), which is denoted as X^(t) ^ Y(t) X(t) h(t) { Assuming that we know the auto- and cross-correlation functions RX (¿), RY (¿), and RXY (¿). ² To estimate each sample X(t), we use an observation window on Y (®) as t ¡ a · ® · t + b. { If a = 1 and b = 0, this is a (causal) ¯ltering problem: estimating ES150 { Harvard SEAS 15 Xt from the past and present observations. { If a = b = 1, this is an in¯nite smoothing problem: recovering Xt from the entire set of noisy observations. ² The linear estimate X^(t) is of the form a X^(t) = h(¿)Y (t ¡ ¿)d¿ ¡ Z b ² Similarly for discrete-time processing, the goal is to design the ¯lter coe±cients hi to estimate Xk as a X^k = hiYk¡i =¡ iXb ² Next we consider the optimum linear ¯lter based on the MMSE criterion.