Chapter 6 Stationary stochastic processes Maria Sandsten Lecture 8 September 24 2019 1 / 32 Chapter 6 Yesterday news: Filtering of stationary processes From the continuous time stationary input process, X (t), t 2 R the stationary output process, Y (t), t 2 R, is given as Z 1 Z 1 Y (t) = h(t − u)X (u)du = h(u)X (t − u)du: −∞ −∞ For a causal filter h(t) = 0, for t < 0. A stationary Gaussian process is filtered to a stationary Gaussian process. h(t) = R H(f )ei2πft df H(f ) = R h(t)e−i2πft dt R mY = mX h(u)du mY = mX H(0) RR 2 rY (τ) = h(u)h(v)rX (τ + u − v)du dv RY (f ) = jH(f )j RX (f ) 2 / 32 Chapter 6 6.2 6.3.2 6.5 9.5.4 Schedule for today I Continuation chapter 6.2: Discrete time filters. Example I Chapter 6.3.2: Differentiation - expected value and covariance of the derivative. Example I Spectral density relations for the derivative. Example. I Chapter 6.5: Cross-covariance I Cross-covariance including derivatives. Example. I Chapters 6.5 and 9.5.4: Cross-spectrum and coherence spectrum. Estimation. 3 / 32 Chapter 6 6.2 6.3.2 6.5 9.5.4 Filtering of discrete time processes Definition 6.1: The output process Yt , t = 0; ±1; ±2 :::, is obtained from the input Xt , t = 0; ±1; ±2 :::, through 1 1 X X Yt = h(t − u)Xu = h(u)Xt−u; u=−∞ u=−∞ where h(t) is the impulse response. Definition 6.2: The frequency function H(f ) of the discrete time impulse response is defined as 1 X H(f ) = h(t)e−i2πft ; t=−∞ with the inverse transform as Z 1=2 h(t) = H(f )ei2πft df : −1=2 4 / 32 Chapter 6 6.2 6.3.2 6.5 9.5.4 Relations Mean value: 1 X mY = mX h(u) = mX H(0): u=−∞ Covariance function: 1 1 X X rY (τ) = h(u)h(v)rX (τ + u − v): u=−∞ v=−∞ Spectral density: 2 RY (f ) = jH(f )j RX (f ): 5 / 32 Chapter 6 6.2 6.3.2 6.5 9.5.4 Example A discrete time filter is defined as h(0) = 1=2; h(2) = 1=2; and zero for all other values. The input process, Xt , t = 0; ±1; ±2 ::: is a zero-mean stationary process with covariance function rX (0) = 1; rX (±1) = −1=2: Calculate the spectral density and the covariance function of the output process. 6 / 32 Chapter 6 6.2 6.3.2 6.5 9.5.4 Solution 1 The spectral density of Xt is X −i2πf τ RX (f ) = rX (τ)e = 1 − cos(2πf ): τ The frequency function is X 1 1 H(f ) = h(u)e−i2πfu = + e−i2πf 2: 2 2 u The spectral density of the output process Yt is 1 1 R (f ) = jH(f )j2R (f ) = ( + cos(4πf )) · (1 − cos(2πf )) = Y X 2 2 1 3 1 1 ::: = − cos(2πf ) + cos(4πf ) − cos(6πf ): 2 4 2 4 7 / 32 Chapter 6 6.2 6.3.2 6.5 9.5.4 Solution 2 The covariance function for Yt is X X rY (τ) = h(u)h(v)rX (τ + u − v) = u v 1 (2r (τ) + r (τ + 2) + r (τ − 2)); 4 X X X 1 3 1 1 with rY (0) = 2 , rY (1) = − 8 , rY (2) = 4 , rY (3) = − 8 and rY (τ) = 0 for τ > 3. Symmetry gives rY (−τ) = rY (τ). Check that rY (0) is positive and has the largest value! The spectral density is given as 3 X −i2πf τ RY (f ) = rY (τ)e = τ=−3 1 3 1 1 = − cos(2πf ) + cos(4πf ) − cos(6πf ): 2 4 2 4 8 / 32 Chapter 6 6.2 6.3.2 6.5 9.5.4 Differentiation To solve a linear stochastic differential equation (SDE), with Y (t) and X (t), t 2 R, e.g., 00 0 Y (t) + a1Y (t) + a0Y (t) = X (t); we need some knowledge of derivatives and integrals of stochastic processes. (For the interested reader, see chapter 8.2.2) 9 / 32 Chapter 6 6.2 6.3.2 6.5 9.5.4 Differentiation A stochastic process X (t), t 2 R, is said to be differentiable (deriverbar) in quadratic mean with the derivative X 0(t) if X (t + h) − X (t) ! X 0(t); h when h ! 0 for all t 2 R, i.e., if " # X (t + h) − X (t) 2 E − X 0(t) ! 0; h when h ! 0. 10 / 32 Chapter 6 6.2 6.3.2 6.5 9.5.4 Differentiation The derivative X 0(t), t 2 R, of a weakly stationary process X (t), t 2 R, is weakly stationary. The derivative X 0(t) of a Gaussian process X (t) is also a Gaussian process. The expected value of the derivative is mX 0 = 0; as X (t + h) − X (t) E[X (t + h)] − E[X (t)] mX 0 = E[ lim ] = lim h!0 h h!0 h m − m = lim X X = 0: h!0 h 11 / 32 Chapter 6 6.2 6.3.2 6.5 9.5.4 Differentiation The covariance function of the derivative relates to the process covariance as 00 rX 0 (τ) = −rX (τ): Proof: X (t + k) − X (t) X (t + τ + h) − X (t + τ) rX 0 (τ) = C lim ; lim k!0 k h!0 h 1 r (τ + h − k) − r (τ + h) r (τ − k) − r (τ) = lim lim X X − X X h!0 h k!0 k k 0 0 −rX (τ + h) + rX (τ) 00 = lim = −rX (τ): h!0 h 12 / 32 Chapter 6 6.2 6.3.2 6.5 9.5.4 Example A Gaussian stationary process X (t), t 2 R, has covariance function 2 rX (τ) = exp(−τ =2): Calculate P(X 0(t) > 1): 13 / 32 Chapter 6 6.2 6.3.2 6.5 9.5.4 Solution The mean value of the differentiated process X 0(t) is always zero, i.e., E[X 0(t)] = 0: The variance is 0 00 −τ 2=2 2 −τ 2=2 V [X (t)] = rX 0 (0) = −rX (0) = e − τ e jτ=0 = 1: Then follows P(X 0(t) > 1) = 1 − Φ(1) = 0:159: 14 / 32 Chapter 6 6.2 6.3.2 6.5 9.5.4 Differentiation, spectral density To find the spectral density of X 0(t) we differentiate the definition R i2πf τ rX (τ) = e RX (f ) df twice. We get Z Z 00 i2πf τ 2 i2πf τ rX 0 (τ) = −rX (τ) = e (2πf ) RX (f ) df = e RX 0 (f ) df ; i.e. the spectral density of X 0(t) is identified 2 RX 0 (f ) = (2πf ) RX (f ): The spectral density RX 0 (f ) is positive and symmetric as RX (f ) is positive and symmetric. It is also integrable if Z 1 Z 1 2 RX 0 (f )df = (2πf ) RX (f )df < 1: −∞ −∞ 15 / 32 Chapter 6 6.2 6.3.2 6.5 9.5.4 Differentiation, summary Let X (t), t 2 R, be a weakly stationary process with covariance function rX (τ). Then the following statements are equivalent: I X (t) is differentiable in quadratic mean I rX (τ) is two times differentiable for every τ R 1 2 I −∞(2πf ) RX (f )df < 1 16 / 32 Chapter 6 6.2 6.3.2 6.5 9.5.4 Example Let X (t), t 2 R, be a stationary Gaussian process with the spectral density, 1 for jf j ≤ 1; R (f ) = X 0 for all other values: Show that the process, Z 1 Y (t) = e−uX (t − u)du; 0 is differentiable in quadratic mean and determine V [Y 0(t)]. 17 / 32 Chapter 6 6.2 6.3.2 6.5 9.5.4 Solution The impulse response of the filter is h(t) = e−t , t > 0, and frequency function, H(f ) = 1=(1 + i2πf ). We find 1 1 R (f ) = jH(f )j2R (f ) = = for jf j ≤ 1; Y X j1 + i2πf j2 1 + (2πf )2 and Z 1 Z 1 2 2 1 (2πf ) RY (f )df = (2πf ) 2 df < 1: −∞ −1 1 + (2πf ) Accordingly, Y (t) is differentiable in quadratic mean. The variance of the derivative is Z 1 Z 1 2 0 2 (2πf ) 1 V [Y (t)] = (2πf ) RY (f )df = 2 df = 2− arctan(2π): −∞ −1 1 + (2πf ) π 18 / 32 Chapter 6 6.2 6.3.2 6.5 9.5.4 Back to correlation 19 / 32 Chapter 6 6.2 6.3.2 6.5 9.5.4 Cross-covariance Definition 6.5: The cross-covariance is defined as rX ;Y (τ) = C[X (t); Y (t + τ)] = E[X (t)Y (t + τ)] − mX mY : It is easily shown that rY ;X (τ) = rX ;Y (−τ); but in general the cross-covariance is not symmetric, i.e. rX ;Y (τ) 6= rX ;Y (−τ); 20 / 32 Chapter 6 6.2 6.3.2 6.5 9.5.4 Cross-covariance of derivatives If X (t), t 2 R, is differentiable in quadratic mean, then X (t) and X 0(t) have a cross-covariance function, 0 X (t + τ + h) − X (t + τ) r 0 (τ) = C[X (t); X (t + τ)] = C[X (t); ]; X ;X h 1 = (C[X (t); X (t + τ + h)] − C[X (t); X (t + τ)]); h 1 = (r (τ + h) − r (τ)) = r 0 (τ); h X X X when h ! 0.