Lesson 5: The Autocovariance Function of a stochastic process Umberto Triacca Dipartimento di Ingegneria e Scienze dell'Informazione e Matematica Universit`adell'Aquila, [email protected] Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Mean and Autocovariance Functions of a stochastic process A discrete stochastic process fxt ; t 2 Zg is a family of random variables indexed by a parameter t (usually the time). Thus the moments of the random variables in a stochastic process are function of the parameter t. We will consider two moments functions: 1 The mean function; 2 The autocovariance function. Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Mean Function of a stochastic process First, we consider the mean function. Definition. Let fxt ; t 2 Zg be a stochastic process such that Var(xt ) < 1 8t 2 Z. The function µx : Z ! R defined by µx (t) = E(xt ) is called Mean Function of the stochastic process fxt ; t 2 Zg. The mean function describes the expectation of the random variables in the process. Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Mean Function of a Random Walk with Drift Let fxt ; t = 0; 1; 2; :::g be a stochastic processs where x0 = δ and xt = λ + xt−1 + ut for 2 t = 1,2,..., with ut ∼ WN(0; σu). This process is called random walk with drift. The constant λ is called the drift. The mean function of this process is µx (t) = δ + λt which is linear function with intercept δ and slope λ. Why? Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Autocovariance Function of a stochastic process Definition. Let fxt ; t 2 Zg be a stochastic process such that Var(xt ) < 1 8t 2 Z. The function γx : Z × Z ! R defined by γx (t1; t2) = cov(xt1 ; xt2 ) is called Autocovariance Function of the stochastic process fxt ; t 2 Zg. The autocovariance function describes the strength of the linear relationship between the random variables xt1 and xt2 . It is clear that autocovariance function evaluated in (t,t) gives the variance, because h 2i γx (t; t) = E (xt − µt ) = var(xt ) Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Autocovariance Function of a Random Walk Let fxt ; t = 0; 1; 2; :::g be a random walk processs with initial condition x0 = 0, and where 2 xt = xt−1 + ut for t = 1,2,..., with ut ∼ WN(0; σu). We have 2 γx (t1; t2) = min ft1; t2g σu Why? Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Autocovariance Function of a stationary stochastic process Consider a weakly stationary stochastic process fxt ; t 2 Zg. We have that γx (t + k; t) = cov(xt+k ; xt ) = cov(xk ; x0) = γx (k; 0) 8t; k 2 Z: We observe that γx (t + k; t) does not depend on t. It depends only on the time difference k, therefore is convenient to redefine the autocovariance function of a weakly stationary process as the function of one variable. Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Autocovariance Function of a weakly stationary process Definition. The function γx : Z ! R defined by γx (k) = Cov(xt ; xt−k ) is called autocovariance function of the weakly stationary stochastic process fxt ; t 2 Zg. Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Autocovariance Function of a weakly stationary process Example. Consider a stochastic process fxt ; t 2 Zg defined by xt = ut + θut−1 2 with ut ∼ WN(0; σu). It is possible to show that this process is weakly stationary. Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Autocovariance Function of a weakly stationary process The autocovariance function of this process is given by γx (k) = Cov(xt ; xt−k ) 8 2 2 1 + θ σu for k = 0 < 2 = θσu for k = 1 : 0 for k = 2; 3; ::: We note that autocovariance function of this process `cuts off' after lag 1. Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Autocovariance Function of a weakly stationary process Some basic properties of the autocovariance function are: 1 γx (0) ≥ 0 2 jγx (k)j ≤ γx (0) 8k 3 γx (k) = γx (−k) 8k Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Autocovariance Functions of a weakly stationary process γx (0) ≥ 0 The first is simply the statement that Var(xt ) ≥ 0 jγx (k)j ≤ γx (0) 8k The second is an immediate consequence of the fact that the correlation between xt and xt−k is less than or equal to 1 in absolute value γx (k) = γx (−k) 8k The third follows straight from the definition Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Autocovariance Function of a stationary process Another important property of γx (:) is that it is non-negative definite, that is n n X X αi γx (i − j)αj ≥ 0 i=1 j=1 0 n for all positive integers n and vectors α = (α1; :::; αn) 2 R . Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process Parametric Functions of a covariance-stationary process In fact, we have n n n ! X X X αi γx (i − j)αj = Var αi xi i=1 j=1 i=1 Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Autocovariance Function of a stationary process In the class of stationary, zero mean, Gaussian processes there is a one-to-one correspondence between the family of the finite dimensional distributions and autocovariance function Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Autocovariance Function of a stationary process Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process The Autocovariance Function Due to this one-to-one correspondence the statistical properties of a stationary, zero mean, Gaussian process are completely determined by its autocovariance function. Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process Autocovariance function Stationary, zero mean, Gaussian process '$ DGP γx (k) 7 &% ? x1; :::; xT Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process .