c 1997 by Jay H. Lee, Jin Hoon Choi, and Kwang Soon Lee Chapter 5 STOCHASTIC PROCESSES A sto chastic pro cess refers to a family of random variables indexed by a parameter set. This parameter set can be continuous or discrete. Since we are interested in discrete systems, we will limit our discussion to pro cesses with a discrete parameter set. Hence, a sto chastic pro cess in our context is a time sequence of random variables. 5.1 BASIC PROBABILITY CONCEPTS 5.1.1 DISTRIBUTION FUNCTION Let xk be a sequence. Then, xk ; ;xk form an `-dimensional 1 ` random variable. Then, one can de ne the nite dimensional distribution function and the density function as b efore. For instance, the distribution function F ; ; ; xk ; ;xk , is de ned as: 1 ` 1 ` F ; ; ; xk ; ;xk = Prfxk ; ;xk g 5.1 1 ` 1 ` 1 1 ` ` The density function is also de ned similarly as b efore. We note that the ab ove de nitions also apply to vector time sequences if 102 c 1997 by Jay H. Lee, Jin Hoon Choi, and Kwang Soon Lee xk and 's are taken as vectors and each integral is de ned over the i i space that o ccupies. i 5.1.2 MEAN AND COVARIANCE Mean value of the sto chastic variable xk is Z 1 xk = E fxk g = dF ; xk 5.2 1 Its covariance is de ned as T R k ;k = E f[xk xk ][xk xk ] g x 1 2 1 1 2 2 R R 1 1 T [ xk ][ xk ] dF ; ; xk ;xk = 1 1 2 2 1 2 1 2 1 1 5.3 The cross-covariance of two sto chastic pro cesses xk and y k are de ned as T R k ;k = E f[xk xk ][y k yk ] g xy 1 2 1 1 2 2 R R 1 1 T [ xk ][ yk ] dF ; ; xk ;yk = 1 1 2 2 1 2 1 2 1 1 5.4 Gaussian processes refer to the pro cesses of which any nite-dimensional distribution function is normal. Gaussian pro cesses are completely characterized by the mean and covariance. 5.1.3 STATIONARY STOCHASTIC PROCESSES Throughout this book we will de ne stationary sto chastic pro cesses as those with time-invariant distribution function. Weakly stationary or stationary in a wide sense pro cesses are pro cesses whose rst two moments are 103 c 1997 by Jay H. Lee, Jin Hoon Choi, and Kwang Soon Lee time-invariant. Hence, for a weakly stationary pro cess xk , E fxk g = x 8k 5.5 T E f[xk x][xk x] g = R 8k x In other words, if xk is stationary, it has a constant mean value and its covariance dep ends only on the time di erence . For Gaussian pro cesses, weakly stationary pro cesses are also stationary. For scalar xk , R 0 can be interpreted as the variance of the signal and R R reveals its time correlation. The normalized covariance ranges from R 0 R 0 0 to 1 and indicates the time correlation of the signal. The value of 1 indicates a complete correlation and the value of 0 indicates no correlation. Note that many signals have b oth deterministic and sto chastic comp onents. In some applications, it is very useful to treat these signals in the same framework. One can do this by de ning P 1 N x = lim xk N !1 k =1 N 5.6 P 1 N T R = lim [xk x ][xk x] x N !1 k =1 N Note that in the ab ove, b oth deterministic and sto chastic parts are averaged out. The signals for which the ab ove limits converge are called \quasi-stationary" signals. The ab ove de nitions are consistent with the previous de nitions since,in the purely sto chastic case, a particular realization of a stationary sto chastic pro cess with given mean x and covariance R should satisfy the ab ove relationships. x 5.1.4 SPECTRA OF STATIONARY STOCHASTIC PROCESSES Throughout this chapter, continuous time is rescaled so that each discrete time interval represents one continuous time unit. If the sample interval T s 104 c 1997 by Jay H. Lee, Jin Hoon Choi, and Kwang Soon Lee is not one continuous time unit, the frequency in discrete time needs to be 1 scaled with the factor of . T s Sp ectral density of a stationary pro cess xk is de ned as the Fourier transform of its covariance function: 1 X 1 j ! ! = R e 5.7 x x 2 =1 Area under the curve represents the power of the signal for the particular frequency range. For example, the power of xk in the frequency range ! ;! is calculated by the integral 1 2 Z ! =! 2 2 ! d! x ! =! 1 Peaks in the signal sp ectrum indicate the presence of p erio dic comp onents in the signal at the resp ective frequency. The inverse Fourier transform can be used to calculate R from the x sp ectrum ! as well x Z j ! ! e d! 5.8 R = x x With = 0, the ab ove b ecomes Z T E fxk xk g = R 0 = ! d! 5.9 x x which indicates that the total area under the sp ectral density is equal to the variance of the signal. This is known as the Parseval's relationship. Example: Show plots of various covariances, sp ectra and realizations! **Exercise: Plot the sp ectra of 1 white noise, 2 sinusoids, and 3white noise ltered through a low-pass lter. 105 c 1997 by Jay H. Lee, Jin Hoon Choi, and Kwang Soon Lee 5.1.5 DISCRETE-TIME WHITE NOISE A particular typ e of a sto chastic pro cess called white noise will be used extensively throughout this b o ok. xk is called a white noise or white sequence if P xk jx` = P xk for ` < k 5.10 for all k . In other words, the sequence has no time correlation and hence all the elements are mutually indep endent. In such a situation, knowing the realization of x` in no way helps in estimating xk . A stationary white noise sequence has the following prop erties: E fxk g = x 8k 8 > < 5.11 R if = 0 x T E fxk xxk x g = > : 0 if 6= 0 Hence, the covariance of a white noise is de ned by a single matrix. The sp ectrum of white noise xk is constant for the entire frequency range since from 5.7 1 ! = R 5.12 x x 2 The name \white noise" actually originated from its similarity with white light in sp ectral prop erties. 5.1.6 COLORED NOISE A sto chastic pro cess generated by ltering white noise through a dynamic system is called \colored noise." Imp ortant: 106 c 1997 by Jay H. Lee, Jin Hoon Choi, and Kwang Soon Lee A stationary sto chastic pro cess with any given mean and covariance function can be generated by passing a white noise through an appropriate dynamical system. To see this, consider dk = H q "k + d 5.13 where "k is a white noise of identity covariance and H q is a stable / stably invertible transfer function matrix. Using simple algebra Ljung -REFERENCE, one can show that j! T j! ! = H e H e 5.14 d The sp ectral factorization theorem REFERENCE - Astrom and Wittenmark, 1984 says that one can always nd H q that satis es 5.14 for an arbitrary and has no p ole or zero outside the unit disk. In other d words, the rst and second order moments of any stationary signal can be matched by the ab ove mo del. This result is very useful in mo deling disturbances whose covariance functions are known or xed. Note that a stationary Gaussian pro cess is completely sp eci ed by its mean and covariance. Such a pro cess can be mo delled by ltering a zero-mean Gaussian white sequence through appropriate dynamics determined by its sp ectrum plus adding a bias at the output if the mean is not zero. 107 c 1997 by Jay H. Lee, Jin Hoon Choi, and Kwang Soon Lee 5.1.7 INTEGRATED WHITE NOISE AND NONSTATIONARY PROCESSES Some pro cesses exhibit mean-shifts whose magnitude and o ccurence are random. Consider the following mo del: y k = y k 1 + "k where "k is a white sequence. Such a sequence is called integrated white noise or sometimes random walk. Particular realizations under di erent distribution of "k are shown b elow: P(ζ) y(k) 90% 10% More generally, many interesting signals will exhibit stationary b ehavior combined with randomly o ccuring mean-shifts. Such signals can be mo deled as 108 c 1997 by Jay H. Lee, Jin Hoon Choi, and Kwang Soon Lee ε ( ) 2 k ~ − Hq()1 ε ( ) 1 k 1 ++ 1q− −1 y(k) ε( ) k 1 −1 − −1 Hq() 1q y(k) As shown ab ove, the combined e ects can be expressed as an integrated 1 white noise colored with a lter H q . Note that while y k is nonstationary, the di erenced signal y k is stationary. ∆ ε( ) y(k) ε( ) y(k) k 1 −1 k −1 −1 () () 1q− Hq Hq 5.1.8 STOCHASTIC DIFFERENCE EQUATION Generally, a sto chastic pro cess can be mo deled through the following sto chastic di erence equation. xk +1 = Axk + B"k 5.15 y k = Cxk + D"k where "k is a white vector sequence of zero mean and covariance R .