Lesson 4: Stationary stochastic processes Umberto Triacca Dipartimento di Ingegneria e Scienze dell'Informazione e Matematica Universit`adell'Aquila, [email protected] Umberto Triacca Lesson 4: Stationary stochastic processes Stationary stochastic processes Stationarity is a rather intuitive concept, it means that the statistical properties of the process do not change over time. Umberto Triacca Lesson 4: Stationary stochastic processes Stationary stochastic processes There are two important forms of stationarity: 1 strong stationarity; 2 weak stationarity. Umberto Triacca Lesson 4: Stationary stochastic processes Stationary stochastic processes Strong stationarity concerns the shift-invariance (in time) of its finite-dimensional distributions. Weak stationarity only concerns the shift-invariance (in time) of first and second moments of a process. Umberto Triacca Lesson 4: Stationary stochastic processes Strongly stationary stochastic processes Definition. The process fxt ; t 2 Zg is strongly stationary if Ft1+k;t2+k;··· ;ts +k (b1; b2; ··· ; bs ) = Ft1;t2;··· ;ts (b1; b2; ··· ; bs ) + for any finite set of indices ft1; t2; ··· ; ts g ⊂ Z with s 2 Z , and any k 2 Z. Thus the process fxt ; t 2 Zg is strongly stationary if the joint distibution function of the vector (xt1+k ; xt2+k ; :::; xts +k ) is equal with the one of (xt1 ; xt2 ; :::; xts ) for for any finite set of indices + ft1; t2; ··· ; ts g ⊂ Z with s 2 Z , and any k 2 Z. Umberto Triacca Lesson 4: Stationary stochastic processes Strongly stationary stochastic processes The meaning of the strongly stationarity is that the distribution of a number of random variables of the stochastic process is the same as we shift them along the time index axis. Umberto Triacca Lesson 4: Stationary stochastic processes Strongly stationary stochastic processes If fxt ; t 2 Zg is a strongly stationary process, then x1; x2; x3; ::: have the same distribution function. (x1; x3); (x5; x7); (x9; x11); ::::: have the same joint distribution function and further (x1; x3; x5); (x7; x9; x11); (x13; x15; x17); ::: must have the same joint distribution function, and so on. Umberto Triacca Lesson 4: Stationary stochastic processes Strongly stationary stochastic processes If the process is fxt ; t 2 Zg is strongly stationary, then the joint probability distribution function of (xt1 ; xt2 ; :::; xts ) is invariant under translation. Umberto Triacca Lesson 4: Stationary stochastic processes iid process An iid process is a strongly stationary process. This follows almost immediate from the definition. Since the random variables xt1+k ; xt2+k ; :::; xts +k are iid, we have that Ft1+k;t2+k;··· ;ts +k (b1; b2; ··· ; bs ) = F (b1)F (b2) ··· F (bs ) On the other hand, also the random variables xt1 ; xt2 ; :::; xts are iid and hence Ft1;t2;··· ;ts (b1; b2; ··· ; bs ) = F (b1)F (b2) ··· F (bs ): We can conclude that Ft1+k;t2+k;··· ;ts +k (b1; b2; ··· ; bs ) = Ft1;t2;··· ;ts (b1; b2; ··· ; bs ) Umberto Triacca Lesson 4: Stationary stochastic processes iid process Remark. Let fxt ; t 2 Zg be an iid process. We have that the conditional distribution of xT +h given values of (x1; :::; xT ) is P(xT +h ≤ bjx1; :::; xT ) = P(xT +h ≤ b) So the knowledge of the past has no value for predicting the future. An iid process is unpredictable. Example. Under efficient capital market hypothesis, the stock price change is an iid process. This means that the stock price change is unpredictable from previous stock price changes. Umberto Triacca Lesson 4: Stationary stochastic processes Strongly stationary stochastic processes Consider the discrete stochastic process fxt ; t 2 Ng where xt = A, with A ∼ U (3; 7) (A is uniformly distributed on the interval [3; 7]). This process is of course strongly stationary. Why? Umberto Triacca Lesson 4: Stationary stochastic processes Strongly stationary stochastic processes Consider the discrete stochastic process fxt ; t 2 Ng where xt = tA, with A ∼ U (3; 7) This process is not strongly stationary Why? Umberto Triacca Lesson 4: Stationary stochastic processes Weakly stationary stochastic processes If the second moment of xt is finite for all t, then the mean E(xt ), 2 2 2 the variance var(xt ) = E[(xt − E(xt )) ] = E(xt ) − (E(xt )) and the covariance cov(xt1 ; xt2 ) = E[(xt1 − E(xt1 ))(xt2 − E(xt2 ))] are finite for all t, t1 and t2. Why? Hint: Use the Cauchy-Schwarz inequality jcov(x; y)j2 ≤ var(x)var(y) Umberto Triacca Lesson 4: Stationary stochastic processes Weakly stationary stochastic processes Definition The process fxt ; t 2 Zg is weakly stationary, or covariance-stationary if 2 1 the second moment of xt is finite for all t, that is Ejxt j < 1 for all t 2 the first moment of xt is independent of t, that is E(xt ) = µ 8t 3 the cross moment E(xt1 xt2 ) depends only on t1 − t2, that is cov(xt1 ; xt2 ) = cov(xt1+h; xt2+h) 8t1; t2; h Umberto Triacca Lesson 4: Stationary stochastic processes Weakly stationary stochastic processes Thus a stochastic process is covariance-stationary if 1 it has the same mean value, µ, at all time points; 2 it has the same variance, γ0, at all time points; and 3 the covariance between the values at any two time points, t; t − k, depend only on k, the difference between the two times, and not on the location of the points along the time axis. Umberto Triacca Lesson 4: Stationary stochastic processes Weakly stationary stochastic processes An important example of covariance-stochastic process is the so-called white noise process. Definition . A stochastic process fut ; t 2 Zg in which the random variables ut ; t = 0 ± 1; ±2::: are such that 1 E(ut ) = 0 8t 2 2 Var(ut ) = σu < 1 8t 3 Cov(ut ; ut−k ) = 0 8t; 8k 2 is called white noise with mean 0 and variance σu, written 2 ut ∼ WN(0; σu). First condition establishes that the expectation is always constant and equal to zero. Second condition establishes that variance is constant. Third condition establishes that the variables of the process are uncorrelated for all lags. If the random variables ut are independently and identically 2 distributed with mean 0 and variance σu then we will write 2 ut ∼ IID(0; σu) Umberto Triacca Lesson 4: Stationary stochastic processes White Noise process Figure shows a possible realization of an IID(0,1) process. Figure : A realization of an IID(0,1). Umberto Triacca Lesson 4: Stationary stochastic processes Random Walk process An important example of weakly non-stationary stochastic processes is the following. Let fyt ; t = 0; 1; 2; :::g be a stochastic processs where y0 = δ < 1 and yt = yt−1 + ut for 2 t = 1,2,..., with ut ∼ WN(0; σu). This process is called random walk. Umberto Triacca Lesson 4: Stationary stochastic processes Random Walk process The mean of yt is given by E(yt ) = δ and its variance is 2 Var(yt ) = tσu Thus a random walk is not weakly stationary process. Umberto Triacca Lesson 4: Stationary stochastic processes Random Walk process Figure shows a possible realization of a random walk. Figure : A realization of a random walk. Umberto Triacca Lesson 4: Stationary stochastic processes Relation between strong and weak Stationarity First note that finite second moments are not assumed in the definition of strong stationarity, therefore, strong stationarity does not necessarily imply weak stationarity. For example, an iid process with standard Cauchy distribution is strictly stationary but not weak stationary because the second moment of the process is not finite. Umberto Triacca Lesson 4: Stationary stochastic processes Relation between strong and weak Stationarity If the process fxt ; t 2 Zg is strongly stationary and has finite second moment, then fxt ; t 2 Zg is weakly stationary. PROOF. If the process fxt ; t 2 Zg is strongly stationary, then :::; x−1; x0; x1; ::: have the same distribution function and (xt1 ; xt2 ) and (xt1+h; xt2+h) have the same joint distribution function for all t1, and t2 and h. Because, by hypothesis, the process fxt ; t 2 Zg has finite second moment, this implies that E(xt ) = µ 8t cov(xt1 ; xt2 ) = cov(xt1+h; xt2+h) 8t1; t2; h Umberto Triacca Lesson 4: Stationary stochastic processes Relation between strong and weak Stationarity Of course a weakly stationary process is not necessarily strongly stationary. weak stationarity ; strong stationarity Umberto Triacca Lesson 4: Stationary stochastic processes Relation between strong and weak Stationarity Here we give an example of a weakly stationary stochastic process which is not strictly stationary. Let fxt ; t 2 Zg be a stochastic process defined by ( u if t is even x = t t p1 (u2 − 1) if t is odd 2 t where ut ∼ iidN(0; 1). This process is weakly stationary but it is not strictly stationary. Umberto Triacca Lesson 4: Stationary stochastic processes Relation between strong and weak Stationarity We have ( E(u ) = 0 if t is even E(x ) = t t p1 E(u2 − 1) = 0 if t is odd 2 t and var(ut ) = 1 if t is even var(xt ) = 1 2 2 var(ut−1) = 1 if t is odd Further, because xt and xt−k are independent random variables, we have cov(xt ; xt−k ) = 0 8k Thus, the process xt is weakly stationary. In particular, xt ∼ WN(0; 1). Umberto Triacca Lesson 4: Stationary stochastic processes Relation between strong and weak Stationarity Now, we note that P(xt ≤ 0) = P(ut ≤ 0) = 0:5 for t even and 1 2 P (xt ≤ 0) = P p (u − 1) ≤ 0 2 t 2 = P ut−1 ≤ 1 = P (jut−1j ≤ 1) = P (−1 ≤ ut−1 ≤ 1) = 0:6826 for t odd Hence the random variables of the process are not identically distributed.