sign in sign up
<<
Home , Stationary process

Lesson 4: Stationary stochastic processes

Umberto Triacca

Dipartimento di Ingegneria e Scienze dell’Informazione e Matematica Universit`adell’Aquila, [email protected]

Umberto Triacca Lesson 4: Stationary stochastic processes Stationary stochastic processes

Stationarity is a rather intuitive concept, it that the statistical properties of the process do not change over time.

Umberto Triacca Lesson 4: Stationary stochastic processes Stationary stochastic processes

There are two important forms of stationarity:

1 strong stationarity; 2 weak stationarity.

Umberto Triacca Lesson 4: Stationary stochastic processes Stationary stochastic processes

Strong stationarity concerns the shift-invariance (in time) of its finite-dimensional distributions.

Weak stationarity only concerns the shift-invariance (in time) of first and second moments of a process.

Umberto Triacca Lesson 4: Stationary stochastic processes Strongly stationary stochastic processes

Definition. The process {xt ; t ∈ Z} is strongly stationary if

Ft1+k,t2+k,··· ,ts +k (b1, b2, ··· , bs ) = Ft1,t2,··· ,ts (b1, b2, ··· , bs )

+ for any finite set of indices {t1, t2, ··· , ts } ⊂ Z with s ∈ Z , and any k ∈ Z.

Thus the process {xt ; t ∈ Z} is strongly stationary if the joint

distibution function of the vector (xt1+k , xt2+k , ..., xts +k ) is equal

with the one of (xt1 , xt2 , ..., xts ) for for any finite set of indices + {t1, t2, ··· , ts } ⊂ Z with s ∈ Z , and any k ∈ Z.

Umberto Triacca Lesson 4: Stationary stochastic processes Strongly stationary stochastic processes

The meaning of the strongly stationarity is that the distribution of a number of random variables of the is the same as we shift them along the time index axis.

Umberto Triacca Lesson 4: Stationary stochastic processes Strongly stationary stochastic processes

If {xt ; t ∈ Z} is a strongly stationary process, then

x1, x2, x3, ...

have the same distribution function.

(x1, x3), (x5, x7), (x9, x11), .....

have the same joint distribution function and further

(x1, x3, x5), (x7, x9, x11), (x13, x15, x17), ...

must have the same joint distribution function, and so on.

Umberto Triacca Lesson 4: Stationary stochastic processes Strongly stationary stochastic processes

If the process is {xt ; t ∈ Z} is strongly stationary, then the joint

function of (xt1 , xt2 , ..., xts ) is invariant under translation.

Umberto Triacca Lesson 4: Stationary stochastic processes iid process

An iid process is a strongly stationary process. This follows almost immediate from the definition.

Since the random variables xt1+k , xt2+k , ..., xts +k are iid, we have that

Ft1+k,t2+k,··· ,ts +k (b1, b2, ··· , bs ) = F (b1)F (b2) ··· F (bs )

On the other hand, also the random variables xt1 , xt2 , ..., xts are iid and hence

Ft1,t2,··· ,ts (b1, b2, ··· , bs ) = F (b1)F (b2) ··· F (bs ).

We can conclude that

Ft1+k,t2+k,··· ,ts +k (b1, b2, ··· , bs ) = Ft1,t2,··· ,ts (b1, b2, ··· , bs )

Umberto Triacca Lesson 4: Stationary stochastic processes iid process

Remark. Let {xt ; t ∈ Z} be an iid process. We have that the conditional distribution of xT +h given values of (x1, ..., xT ) is

P(xT +h ≤ b|x1, ..., xT ) = P(xT +h ≤ b)

So the knowledge of the past has no value for predicting the future. An iid process is unpredictable.

Example. Under efficient capital market hypothesis, the stock price change is an iid process. This means that the stock price change is unpredictable from previous stock price changes.

Umberto Triacca Lesson 4: Stationary stochastic processes Strongly stationary stochastic processes

Consider the discrete stochastic process

{xt ; t ∈ N}

where xt = A, with A ∼ U (3, 7) (A is uniformly distributed on the interval [3, 7]).

This process is of course strongly stationary.

Why?

Umberto Triacca Lesson 4: Stationary stochastic processes Strongly stationary stochastic processes

Consider the discrete stochastic process

{xt ; t ∈ N}

where xt = tA, with A ∼ U (3, 7)

This process is not strongly stationary

Why?

Umberto Triacca Lesson 4: Stationary stochastic processes Weakly stationary stochastic processes

If the second of xt is finite for all t, then the E(xt ), 2 2 2 the var(xt ) = E[(xt − E(xt )) ] = E(xt ) − (E(xt )) and

the cov(xt1 , xt2 ) = E[(xt1 − E(xt1 ))(xt2 − E(xt2 ))] are finite for all t, t1 and t2.

Why?

Hint: Use the Cauchy-Schwarz inequality |cov(x, y)|2 ≤ var(x)var(y)

Umberto Triacca Lesson 4: Stationary stochastic processes Weakly stationary stochastic processes

Definition The process {xt ; t ∈ Z} is weakly stationary, or covariance-stationary if 2 1 the second moment of xt is finite for all t, that is E|xt | < ∞ for all t

2 the first moment of xt is independent of t, that is E(xt ) = µ ∀t

3 the cross moment E(xt1 xt2 ) depends only on t1 − t2, that is

cov(xt1 , xt2 ) = cov(xt1+h, xt2+h) ∀t1, t2, h

Umberto Triacca Lesson 4: Stationary stochastic processes Weakly stationary stochastic processes

Thus a stochastic process is covariance-stationary if

1 it has the same mean value, µ, at all time points;

2 it has the same variance, γ0, at all time points; and 3 the covariance between the values at any two time points, t, t − k, depend only on k, the difference between the two times, and not on the location of the points along the time axis.

Umberto Triacca Lesson 4: Stationary stochastic processes Weakly stationary stochastic processes

An important example of covariance-stochastic process is the so-called process. Definition . A stochastic process {ut ; t ∈ Z} in which the random variables ut , t = 0 ± 1, ±2... are such that 1 E(ut ) = 0 ∀t 2 2 Var(ut ) = σu < ∞ ∀t 3 Cov(ut , ut−k ) = 0 ∀t, ∀k 2 is called white noise with mean 0 and variance σu, written 2 ut ∼ WN(0, σu). First condition establishes that the expectation is always constant and equal to zero. Second condition establishes that variance is constant. Third condition establishes that the variables of the process are uncorrelated for all lags. If the random variables ut are independently and identically 2 distributed with mean 0 and variance σu then we will write 2 ut ∼ IID(0, σu)

Umberto Triacca Lesson 4: Stationary stochastic processes White Noise process

Figure shows a possible realization of an IID(0,1) process.

Figure : A realization of an IID(0,1).

Umberto Triacca Lesson 4: Stationary stochastic processes process

An important example of weakly non-stationary stochastic processes is the following. Let {yt ; t = 0, 1, 2, ...}

be a stochastic processs where y0 = δ < ∞ and yt = yt−1 + ut for 2 t = 1,2,..., with ut ∼ WN(0, σu).

This process is called random walk.

Umberto Triacca Lesson 4: Stationary stochastic processes Random Walk process

The mean of yt is given by

E(yt ) = δ

and its variance is 2 Var(yt ) = tσu Thus a random walk is not weakly stationary process.

Umberto Triacca Lesson 4: Stationary stochastic processes Random Walk process

Figure shows a possible realization of a random walk.

Figure : A realization of a random walk.

Umberto Triacca Lesson 4: Stationary stochastic processes Relation between strong and weak Stationarity

First note that finite second moments are not assumed in the definition of strong stationarity, therefore, strong stationarity does not necessarily imply weak stationarity.

For example, an iid process with standard Cauchy distribution is strictly stationary but not weak stationary because the second moment of the process is not finite.

Umberto Triacca Lesson 4: Stationary stochastic processes Relation between strong and weak Stationarity

If the process {xt ; t ∈ Z} is strongly stationary and has finite second moment, then {xt ; t ∈ Z} is weakly stationary.

PROOF. If the process {xt ; t ∈ Z} is strongly stationary, then

..., x−1, x0, x1, ... have the same distribution function and

(xt1 , xt2 ) and (xt1+h, xt2+h)

have the same joint distribution function for all t1, and t2 and h. Because, by hypothesis, the process {xt ; t ∈ Z} has finite second moment, this implies that

E(xt ) = µ ∀t

cov(xt1 , xt2 ) = cov(xt1+h, xt2+h) ∀t1, t2, h

Umberto Triacca Lesson 4: Stationary stochastic processes Relation between strong and weak Stationarity

Of course a weakly stationary process is not necessarily strongly stationary.

weak stationarity ; strong stationarity

Umberto Triacca Lesson 4: Stationary stochastic processes Relation between strong and weak Stationarity

Here we give an example of a weakly stationary stochastic process which is not strictly stationary. Let {xt ; t ∈ Z} be a stochastic process defined by ( u if t is even x = t t √1 (u2 − 1) if t is odd 2 t

where ut ∼ iidN(0, 1). This process is weakly stationary but it is not strictly stationary.

Umberto Triacca Lesson 4: Stationary stochastic processes Relation between strong and weak Stationarity

We have ( E(u ) = 0 if t is even E(x ) = t t √1 E(u2 − 1) = 0 if t is odd 2 t and  var(ut ) = 1 if t is even var(xt ) = 1 2 2 var(ut−1) = 1 if t is odd

Further, because xt and xt−k are independent random variables, we have cov(xt , xt−k ) = 0 ∀k

Thus, the process xt is weakly stationary. In particular, xt ∼ WN(0, 1).

Umberto Triacca Lesson 4: Stationary stochastic processes Relation between strong and weak Stationarity

Now, we note that

P(xt ≤ 0) = P(ut ≤ 0) = 0.5 for t even

and   1 2 P (xt ≤ 0) = P √ (u − 1) ≤ 0 2 t 2  = P ut−1 ≤ 1 = P (|ut−1| ≤ 1)

= P (−1 ≤ ut−1 ≤ 1) = 0.6826 for t odd

Hence the random variables of the process are not identically distributed. This implies that the process is not strongly stationary

Umberto Triacca Lesson 4: Stationary stochastic processes Relation between strong and weak Stationarity

There is one important case however in which weak stationarity implies strong stationarity.

If {xt ; t ∈ Z} is a weakly stationary Gaussian stochastic process, then {xt ; t ∈ Z} is strongly stationary.

Why?

Umberto Triacca Lesson 4: Stationary stochastic processes Relation between strong and weak Stationarity

Let {xt ; t ∈ Z} be a Gaussian stochastic process. Introducing the 0 s vector b = (b1, b2, ..., bs ) ∈ R , the multidimensional density

function of the vector (xt1 , xt2 , ..., xts ) is   1 1 0 −1 ft ,t ,...,t (b) = exp − (b − µ) Σ (b − µ) . 1 2 s p(2π)s det(Σ(Σ(Σ) 2

0   where µ = (E(xt1 ), E(xt2 ), ..., E(ts )) and Σ = Cov(xti , xtj ) . We note that a multivariate Gaussian distribution is fully characterized by its first two moments.

Umberto Triacca Lesson 4: Stationary stochastic processes Relation between strong and weak Stationarity

Let {xt ; t ∈ Z} be a Gaussian stochastic process. Assume that the process is weakly stationary. If the process i weakly stationary, then

1 E(xt ) = µ ∀t

2 Var(xt ) = γ0 < ∞ ∀t

3 Cov(xt1+k , xt2+k ) = Cov(xt1 , xt2 ) ∀t1, t2, ∀k and hence

ft1+k,t2+k,...,ts +k (b) = ft1,t2,...,ts (b) It follows that the joint distibution function of the vector

(xt1+k , xt2+k , ..., xts +k ) is equal with the one of (xt1 , xt2 , ..., xts ) for + for any finite set of indices {t1, t2, ··· , ts } ⊂ Z with s ∈ Z , and any k ∈ Z. This implies that the process {xt ; t ∈ Z} is strongly stationary.

Umberto Triacca Lesson 4: Stationary stochastic processes Relation between strong and weak Stationarity

Conversely, if {xt ; t ∈ Z} is a Gaussian strongly stationary stochastic process, then it is weakly stationary because it has finite variance. Thus we can conclude that in the case of Gaussian stochastic process, the two definitions of stationarity are equivalent.

Umberto Triacca Lesson 4: Stationary stochastic processes White Noise process

We note that a white noise process is not necessarily strongly stationary. Let w be a uniformly distributed in the interval (0; 2π). We consider the process {Zt ; t = 1, 2, ...} defined by Zt = cos(tw) t = 1, 2, ... We have that

1 E(Zt ) = 0 ∀t

2 1 Var(Zt ) = 2 ∀t 3 Cov(Zt , Zt−k ) = 0 ∀t, ∀k

Thus Zt ∼ WN(0,.5). However, it can be shown that is not strongly stationary.

Umberto Triacca Lesson 4: Stationary stochastic processes