Lesson 5: the Autocovariance Function of a Stochastic Process

Lesson 5: The Autocovariance Function of a stochastic process

Umberto Triacca

Dipartimento di Ingegneria e Scienze dell’Informazione e Matematica Universit`adell’Aquila, [email protected]

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Mean and Autocovariance Functions of a stochastic process

A discrete stochastic process {xt ; t ∈ Z} is a family of random variables indexed by a parameter t (usually the time). Thus the moments of the random variables in a stochastic process are function of the parameter t. We will consider two moments functions:

1 The mean function; 2 The autocovariance function.

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Mean Function of a stochastic process

First, we consider the mean function. Definition. Let {xt ; t ∈ Z} be a stochastic process such that Var(xt ) < ∞ ∀t ∈ Z. The function

µx : Z → R

defined by µx (t) = E(xt )

is called Mean Function of the stochastic process {xt ; t ∈ Z}.

The mean function describes the expectation of the random variables in the process.

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Mean Function of a Random Walk with Drift

Let {xt ; t = 0, 1, 2, ...}

be a stochastic processs where x0 = δ and xt = λ + xt−1 + ut for 2 t = 1,2,..., with ut ∼ WN(0, σu).

This process is called random walk with drift. The constant λ is called the drift. The mean function of this process is

µx (t) = δ + λt

which is linear function with intercept δ and slope λ.

Why?

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Autocovariance Function of a stochastic process

Definition. Let {xt ; t ∈ Z} be a stochastic process such that Var(xt ) < ∞ ∀t ∈ Z. The function

γx : Z × Z → R

defined by

γx (t1, t2) = cov(xt1 , xt2 ) is called Autocovariance Function of the stochastic process {xt ; t ∈ Z}.

The autocovariance function describes the strength of the linear

relationship between the random variables xt1 and xt2 . It is clear that autocovariance function evaluated in (t,t) gives the variance, because

h 2i γx (t, t) = E (xt − µt ) = var(xt )

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Autocovariance Function of a Random Walk

Let {xt ; t = 0, 1, 2, ...}

be a random walk processs with initial condition x0 = 0, and where 2 xt = xt−1 + ut for t = 1,2,..., with ut ∼ WN(0, σu).

We have 2 γx (t1, t2) = min {t1, t2} σu

Why?

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Autocovariance Function of a stationary stochastic process

Consider a weakly stationary stochastic process {xt ; t ∈ Z}. We have that

γx (t + k, t) = cov(xt+k , xt ) = cov(xk , x0) = γx (k, 0) ∀t, k ∈ Z.

We observe that γx (t + k, t) does not depend on t. It depends only on the time difference k, therefore is convenient to redefine the autocovariance function of a weakly stationary process as the function of one variable.

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Autocovariance Function of a weakly stationary process

Definition. The function

γx : Z → R

defined by γx (k) = Cov(xt , xt−k ) is called autocovariance function of the weakly stationary stochastic process {xt ; t ∈ Z}.

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Autocovariance Function of a weakly stationary process

Example. Consider a stochastic process {xt ; t ∈ Z} defined by

xt = ut + θut−1 2 with ut ∼ WN(0, σu).

It is possible to show that this process is weakly stationary.

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Autocovariance Function of a weakly stationary process

The autocovariance function of this process is given by

γx (k) = Cov(xt , xt−k )  2
2 1 + θ σu for k = 0  2 = θσu for k = 1  0 for k = 2, 3, ...

We note that autocovariance function of this process ‘cuts off’ after lag 1.

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Autocovariance Function of a weakly stationary process

Some basic properties of the autocovariance function are:

1 γx (0) ≥ 0

2 |γx (k)| ≤ γx (0) ∀k

3 γx (k) = γx (−k) ∀k

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Autocovariance Functions of a weakly stationary process

γx (0) ≥ 0 The first is simply the statement that Var(xt ) ≥ 0

|γx (k)| ≤ γx (0) ∀k The second is an immediate consequence of the fact that the correlation between xt and xt−k is less than or equal to 1 in absolute value

γx (k) = γx (−k) ∀k The third follows straight from the definition

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Autocovariance Function of a stationary process

Another important property of γx (.) is that it is non-negative definite, that is n n X X αi γx (i − j)αj ≥ 0 i=1 j=1 0 n for all positive integers n and vectors α = (α1, ..., αn) ∈ R .

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process Parametric Functions of a covariance-stationary process

In fact, we have

n n n ! X X X αi γx (i − j)αj = Var αi xi i=1 j=1 i=1

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Autocovariance Function of a stationary process

In the class of stationary, zero mean, Gaussian processes there is a one-to-one correspondence between the family of the finite dimensional distributions and autocovariance function

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Autocovariance Function of a stationary process

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Autocovariance Function

Due to this one-to-one correspondence the statistical properties of a stationary, zero mean, Gaussian process are completely determined by its autocovariance function.

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process Autocovariance function

Stationary, zero mean, Gaussian process

'$ DGP γx (k)

7 &%          ? x1, ..., xT 

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process