238 Chapter 12

Autocorrelation in Time Series Data

In a time series model,

Yi = β0 + β1Xi + εi, i = 1, . . . , n,

1 we allow the εi errors to be correlated with one another; that is, ρ(εi, εj) = 0, i = j. 6 6 12.1 Problems of Autocorrelation

If we treat a time series model as a simple linear regression model; that is, we assume ρ(εi, εj) = 0 when, in fact, ρ(εi, εj) = 0, then the variance in the error terms of the time series model will be underestimated.6 Any confidence intervals or tests for the time series model in this case would then be suspect.

1 By way of comparison, in a simple linear regression model, we assume the error εi have zero covariance, σ(εi, εj ) = 0. Also recall, covariance and correlation are closely related to one another and, in particular, σ(εi, εj ) = 0, i = j implies ρ(εi, εj ) = 0, i = j. 6 6 239 240 Chapter 12. Autocorrelation in Time Series Data (ATTENDANCE 11)

Exercise 12.1 (Problems of Autocorrelation) 1. The errors in (a) below are (circle one) not autocorrelated / autocorrelated because they fluctuate at random above and below zero (0).

error error error 20 20 20

10 10 10

0 0 0

-10 -10 -10

-20 -20 -20 0 2.5 5 7.5 10 0 2.5 5 7.5 10 0 2.5 5 7.5 10 time time time (a) uncorrelated   (b) correlated  =  + u (c) correlated t = t-1 + u t t-1 t t where  = 10 where  = -10 0 0 Figure 12.1 (Positively Autocorrelated Error Terms)

time, t 1 2 3 4 5 6 7 8 9 10 error, εt 1 2 4 -1 -5 0 2 -1 2 2

2. The errors in (b) above are positively autocorrelated because adjacent error terms tend to be of the same magnitude.

time, t 0 1 2 3 4 5 6 7 8 9 10 not autocorrelated error, ut 1 2 4 -1 -5 0 2 -1 2 2 autocorrelated error, εt = εt−1 + ut 10 11 13 17 16 11 11 13 12 14 16

In this case, all of the errors fluctuate at random above the horizontal line at (choose one) −10 / 0 / 10

3. Consider the errors in (c) above.

time, t 0 1 2 3 4 5 6 7 8 9 10 not autocorrelated error, ut 1 2 4 -1 -5 0 2 -1 2 2 autocorrelated error, εt = εt−1 + ut -10 -9 -7 -3 -4 -9 -9 -7 -8 -6 -4

There errors are (circle one) positively / negatively autocorrelated because even though adjacent error terms are all negative, they all tend to be of the same magnitude. Negative correlation occurs when εt = εt−1 + ut, that is, when adjacent error terms are all of the same absolute magnitude,− but opposite in sign. Section 1. Problems of Autocorrelation (ATTENDANCE 11) 241

4. Match residual plots with scatter plots

error error error 20 20 20

10 10 10

0 0 0

-10 -10 -10

-20 -20 -20 0 2.5 5 7.5 10 0 2.5 5 7.5 10 0 2.5 5 7.5 10 time time time (a) residual plot 1 (b) residual plot 2 (c) residual plot 3

Y Y Y 50 50 50

40 40 40

30 30 30

20 20 20

0 0 0 0 2.5 5 7.5 10 0 2.5 5 7.5 10 0 2.5 5 7.5 10 time time time (d) scatter plot 1 (e) scatter plot 2 (f) scatter plot 3 Figure 12.2 (Residual plots and associated scatter plots)

Match the residual plots with the scatter plots.

Residual plot (a) (b) (c) Scatter plot

5. Problem of detecting autocorrelation What if we were unaware that the simple linear regression was, in fact, a time series model? That is, what if we assumed ρ(εi, εj) = 0 when, in fact, we should have assumed ρ(εi, εj) = 0? In this case, we would draw scatterplot (choose one) (d) / (e)6 / (f) in the figure above. In other words, it seems that autocorrelation is difficult to detect. 242 Chapter 12. Autocorrelation in Time Series Data (ATTENDANCE 11)

6. What if Positively Autocorrelated Error Goes Undetected?

Y Y 50 50

40 actual regression fitted error fitted regression 40 actual error 30 30

20 20

0 0 0 2.5 5 7.5 10 0 2.5 5 7.5 10 time time (a) undetected positive autocorrelation (b) detected positive autocorrelation Figure 12.3 (What if Positively Autocorrelated Error Goes Undetected?)

If the positive autocorrelated error remains undetected, then the fitted error terms are (choose one) smaller / larger than the actual error terms. 7. More What if Positively Autocorrelated Error Goes Undetected? If the positive autocorrelated error remains undetected, this means the variance of the error terms in model is often (circle one) underestimated / overesti- mated. Implications of this are (a) the M SE may underestimate the variance of error terms,

(b) s bk may underestimated the standard deviation of the estimated regres- { } sion coefficient, bk, (c) estimated regression coefficients may still be unbiased, but no longer have minimum variance, and this all implies the confidence intervals and tests, using the t and F distri- butions, can not really be used for inference purposes. 8. Property of Positively Autocorrelated Error When errors are positively autocorrelated, initial error, ε0, tends to (circle one) die out quickly (tend to zero) linger (stay at the same magnitude). Consequently, to detect positive autocorrelated error, it makes sense to create a measure that detects error that lingers at the same magnitude. 9. Autocorrelation Versus Cross Correlation True / False The word autocorrelation refers to the analysis of the correlation of error vari- ables in one times series; cross correlation would refer to the analysis of the correlation of error variables in different time series. Section 2. First–Order Autoregressive Error Model (ATTENDANCE 11) 243

12.2 First–Order Autoregressive Error Model

The simple first–order autoregressive error model is

Yt = β0 + β1Xt + εt

εt = ρεt−1 + ut 2 where ρ < 1 and ut are independent N(0, σ ) and ρ is the autocorrelation parameter. In a similar| | way, the multiple first–order autoregressive error model is

Yt = β + β Xt + β Xt + + βp− Xt,p− + εt 0 1 1 2 2 · · · 1 1 εt = ρεt−1 + ut 2 where, again, ρ < 1 and ut are independent N(0, σ ). Notice, in particular, we | | assume in this model that the error εt terms depend on one another according to εt = ρεt−1 + ut.

Exercise 12.2 (Properties of First–Order Autoregressive Error Model) 1. True / False 2 The one and only way the error εt terms can depend on one another is according to εt = ρεt−1 + ut. 2. If ρ = 1, εt = ρεt−1 + ut = εt−1 + ut In this case, the magnitude of the error now, at time t, is (choose one) less than / equal to / greater than the magnitude of the error one time unit in the past, at time t 1. − 3. If ρ = 1, the error terms are (choose one) positively autocorrelated / uncorrelated / negatively au- tocorrelated 4. If ρ = 1, the error terms are (choose−one) positively autocorrelated / uncorrelated / negatively au- tocorrelated 5. If ρ = 0, εt = ρεt−1 + ut = ut

where, recall, ut are assumed to be independent. In this case, the error terms are (choose one) positively autocorrelated / uncorrelated / negatively au- tocorrelated 2 We assume the error εt terms depend on one another in this way because is it reasonable and the mathematics work out this way. 244 Chapter 12. Autocorrelation in Time Series Data (ATTENDANCE 11)

6. Mean E εt = 0. Even though{ } the error terms are autocorrelated, they (circle one) are / are not unbiased.

2 σ2 7. Variance σ εt = 2 . { } 1−ρ The smaller, closer to zero (0), the correlation parameter ρ is, the (circle one) closer / farther away the variance of autocorrelated error terms is to the variance of unautocorrelated error terms, σ2.

σ2 8. Covariance σ εt, εt− = ρ 2 = ρκ { 1} 1−ρ If the error terms are not auto³ correlated´ (ρ = 0), the covariance is (circle one) zero / one / infinity.

9. Correlation ρ εt, εt−1 = ρ If the error terms{ are}not autocorrelated (ρ = 0), the correlation (standardized covariance) is (circle one) zero / one / infinity.

s σ2 s 10. Covariance s time periods apart, σ εt, εt−s = ρ 2 = ρ κ { } 1−ρ The covariance s time periods apart, σ εt, εt−s , is³(circle´one) smaller / larger { } than the covariance one time period apart, σ εt, εt− because 0 ρ 1 { 1} ≤ ≤ s 11. True / False. Correlation s time periods apart, ρ εt, εt−s = ρ { } 12. True / False The variance–covariance matrix is

κ κρ κρn−1 2 . · · · . σ ε = . . ×{ }   n n κρn−1 κρn−2 · · · κ  · · ·    σ2 where κ = 1−ρ2 Section 3. Durbin–Watson Test for Autocorrelation (ATTENDANCE 11) 245

12.3 Durbin–Watson Test for Autocorrelation

SAS program: att11-12-3-read-durbin

If the Durbin–Watson test statistic,

n 2 t=2(et et−1) D = − , et = Yt Yˆt n e2 − P t=1 i is small, then this indicates εt Pεt−1, or, in other words, that the error terms are positively autocorrelated with one≈ another. In fact, if

D > dU ρ = 0, not autocorrelated

dL < D < dU undetermined, or weakly autocorrelated

D < dL ρ > 0, positive autocorrelated where dU and dL are found in Table B.7, pages 1349–1350.

Exercise 12.3 (Durbin–Watson Test for Autocorrelation) illumination, X 1 2 3 4 5 6 7 8 9 10 ability to read, Y 70 70 75 88 91 94 100 92 90 85

1. Statement. The statement of the test is (check one)

(i) H0 : ρ = 0 versus Ha : ρ < 0

(ii) H0 : ρ = 0 versus Ha : ρ > 0

(iii) H : ρ = 0 versus Ha : ρ = 0 0 6 2. Test. From SAS, the Durbin–Watson test statistic,

n 2 (et et− ) D = t=2 − 1 = n e2 P t=1 i (choose one) 0.446 / 0.655 / 0.989P The critical values, dL and dU , (from table B.7, pp 1349–1350) are (circle two) 1.08 / 1.25 / 1.36 (p 1 = 1 and although n = 10, the closest is n = 15) − 3. Conclusion. Since the test statistic is (choose one) smaller / between / larger than the two critical values we 246 Chapter 12. Autocorrelation in Time Series Data (ATTENDANCE 11)

(circle one) accept / cannot either accept or reject / reject the null hypothesis3 that the correlation is zero; in other words, the data indi- cates positive correlation.

4. Residual Plot. True / False The residual plot confirms that there is positive correlation because the residuals tend to be of the same magnitude. First, there is a string of negative residuals, then a string of positive residuals and finally a string of negative residuals.

12.4 Remedial Measures for Autocorrelation

SAS programs: att11-12-4-read-cochrane, att11-12-4-read-hildreth, att11-12-4-read-1stdiff

One way4 to attempt to eliminate or reduce autocorrelation is transform the variables. By transforming the dependent and independent variables in the following way,

0 0 Y = Yt ρYt− , X = Xt ρXt− t − 1 t − 1 we arrive at the simple linear regression

0 0 0 0 Yt = β0 + β1Xt + ut

where the ut are independent, ut = εt ρεt− and where − 1 0 β0 = β0(1 ρ) 0 − β1 = β1

We discuss three methods to estimate ρ, denoted r:

1. Cochrane–Orcutt Procedure

2. Hildreth–Lu Procedure

3. First Difference Procedure

3Notice that the test is more or less a “left–sided” test which rejects the null when the test statistic is less than the critical value. 4Another way to eliminate or reduce autocorrelation is to add one or more predictor variables, such as a linear or exponential trend predictor. Section 4. Remedial Measures for Autocorrelation (ATTENDANCE 11) 247

Exercise 12.4 (Remedial Measures for Autocorrelation) 1. Cochrane–Orcutt Procedure. Use the Cochrane–Orcutt procedure to transform the variables to reduce the autocorrelation of the error terms of the following data.

illumination, X 1 2 3 4 5 6 7 8 9 10 ability to read, Y 70 70 75 88 91 94 100 92 90 85

(a) First Step: Estimation of ρ. From SAS, since

illumination, Xt 1 2 3 4 · · · 9 10 ability to read, Yt 70 70 75 88 · · · 90 85 error, et = Yt − Yˆt -4.618 -7.036 -4.455 6.1277 · · · -3.964 -11.38 et−1 × et −7.036 × (−4.618) = 32.495 31.344 -27.29 · · · -1.802 45.113 an estimate of the slope of this regression, r, is given by

n e − e 254.09455 r = t=2 t 1 t = = n e2 360.52694 P t=2 t−1 (circle one) 0.655 / 0.705P/ 1.845 (b) Fitting the Transformed Model. From SAS, using point estimate, r 0.7047, ≈ Xt 1 2 10 · · · Yt 70 70 85 0 · · · Xt = Xt 0.7047Xt−1 1.295 3.657 0 − · · · Y = Yt 0.7047Yt− 20.665 21.569 t − 1 · · · 0 0 and so the least squares line of the transformed variables, (Xt, Yt ), is (choose one) i. Yˆ 0 = 0.69434 + 20.93322X 0 t − t ii. Yˆ 0 = 1.14984 + 29.770896X 0 t − t iii. Yˆ 0 = 0.69434 + 50.93322X 0 t − t with related standard errors for the regression coefficients of

s b0 = √40.672176 = 6.37747 { 0} s b0 = √6.0596976 = 2.46164 { 1} (c) Durbin–Watson test of the transformed variables. i. Statement. The statement of the test, in this case, is (circle one)

A. H : ρ = 0 versus Ha : ρ = 0 0 6 248 Chapter 12. Autocorrelation in Time Series Data (ATTENDANCE 11)

B. H0 : ρ = 0 versus Ha : ρ > 0

C. H0 : ρ = 0 versus Ha : ρ < 0 ii. Test. From SAS, the Durbin–Watson test statistic is (choose one) 0.665 / 0.857 / 1.5592 The critical values (from table B.7, pp 1349–1350) are, as before, (circle two) 1.08 / 1.25 / 1.36 iii. Conclusion. Since the test statistic, 1.5592, is (choose one) smaller / between / larger than the two critical values, 1.08 and 1.36, we (circle one) accept / cannot either accept or reject / reject the null hypothesis that the correlation is zero. (d) Cochrane–Orcutt Regression Versus Original Linear Regression. Since b0 1.14984 b = 0 = − = 3.89776 0 1 r 1 0.705 − 0 − − b1 = b1 = 29.770896

the fitted regression in the original variables is

i. Yˆt = 0.69434 + 20.93322Xt − ii. Yˆt = 1.14984 + 29.770896Xt − iii. Yˆt = 3.8978 + 29.7709Xt − (e) Cochrane–Orcutt Procedure Effective? True / False The Cochrane–Orcutt procedure has been effective because, according to the Durbin–Watson test, it has reduced the positive autocorrelation.

2. Hildreth–Lu Procedure. Use the Hildreth–Lu procedure to transform the variables to reduce the auto- correlation of the error terms of the following data.

illumination, X 1 2 3 4 5 6 7 8 9 10 ability to read, Y 70 70 75 88 91 94 100 92 90 85

(a) Estimation of ρ. From the SAS output,

ρ 0.76 0.77 · · · 0.91 0.92 0.93 0.94 0.95 SSE 215.60 214.67 · · · 207.74 207.683 207.684 207.75 207.86 Section 4. Remedial Measures for Autocorrelation (ATTENDANCE 11) 249

and so the ρ that minimizes the SSE is given by ρ =. (circle one) 0.76 / 0.91 / 0.92 (b) Fitting the Transformed Model. From SAS, using point estimate, r 0.92, ≈ Xt 1 2 10 · · · Yt 70 70 85 0 · · · Xt = Xt 0.92Xt−1 1.08 1.72 0 − · · · Y = Yt 0.92Yt− 5.6 2.2 t − 1 · · · 0 0 and so the least squares line of the transformed variables, (Xt, Yt ), is (choose one) i. Yˆ 0 = 27.131111 13.3X0 t − t ii. Yˆ 0 = 1.14984 + 29.770896X 0 t − t iii. Yˆ 0 = 0.69434 + 50.93322X 0 t − t with related standard errors for the regression coefficients of s b0 = √154.73217 = 12.4391 { 0} s b0 = √77.263069 = 8.7899 { 1} (c) Durbin–Watson test of the transformed variables. i. Statement. The statement of the test, in this case, is (circle one)

A. H : ρ = 0 versus Ha : ρ = 0 0 6 B. H0 : ρ = 0 versus Ha : ρ > 0

C. H0 : ρ = 0 versus Ha : ρ < 0 ii. Test. From SAS, the Durbin–Watson test statistic is (choose one) 0.665 / 1.5592 / 1.9728322 The critical values (from table B.7, pp 1349–1350) are, as before, (circle two) 1.08 / 1.25 / 1.36 iii. Conclusion. Since the test statistic, 1.9728, is (choose one) smaller / between / larger than the two critical values, 1.08 and 1.36, we (circle one) accept / cannot either accept or reject / reject the null hypothesis that the correlation is zero. (d) Hildreth–Lu Regression Versus Original Linear Regression. Since b0 27.131111 b = 0 = = 339.14 0 1 r 1 0.92 b = b0 −= 13.3 − 1 1 − 250 Chapter 12. Autocorrelation in Time Series Data (ATTENDANCE 11)

the fitted regression in the original variables is

i. Yˆt = 0.69434 + 20.93322Xt − ii. Yˆt = 1.14984 + 29.770896Xt − iii. Yˆt = 339.14 13.3Xt − (e) Hildreth–Lu Procedure Effective? True / False The Hildreth–Lu procedure has been effective because, according to the Durbin–Watson test, it has reduced the positive autocorrelation.

3. First Difference Procedure. Use the first difference procedure to transform the variables to reduce the au- tocorrelation of the error terms of the following data.

illumination, X 1 2 3 4 5 6 7 8 9 10 ability to read, Y 70 70 75 88 91 94 100 92 90 85

(a) First Difference ρ. The first difference procedure specifies that the ρ that minimizes the SSE is given by ρ =. (circle one) 0.76 / 0.91 / 1 (b) Fitting the Transformed Model. From SAS, the least squares line (with zero(0), or no, intercept, using SAS) is (choose one) ˆ 0 0 i. Yt = 27.131111 13.3Xt ˆ 0 − 0 ii. Yt = 1.770896Xt ˆ 0 − 0 iii. Yt = 1.6666Xt with related standard errors for the regression coefficients of s b0 = 2.0949675 { 1} (c) Durbin–Watson test of the transformed variables. i. Statement. The statement of the test, in this case, is (circle one)

A. H : ρ = 0 versus Ha : ρ = 0 0 6 B. H0 : ρ = 0 versus Ha : ρ > 0

C. H0 : ρ = 0 versus Ha : ρ < 0 ii. Test. From SAS, the Durbin–Watson test statistic is (choose one) 1.38 / 1.5592 / 1.6666 The critical values (from table B.7, pp 1349–1350) are, as before, (circle two) 1.08 / 1.25 / 1.36 Section 4. Remedial Measures for Autocorrelation (ATTENDANCE 11) 251

iii. Conclusion. Since the test statistic, 1.6666, is (choose one) smaller / between / larger than the two critical values, 1.08 and 1.36, we (circle one) accept / cannot either accept or reject / reject the null hypothesis that the correlation is zero. (d) First Difference Regression Versus Original Linear Regression. Using the original variables in Y¯ and X¯,

¯ 0 ¯ b0 = Y b1X = 85.5 (1.6666)(5.5) = 76.33 0 − − b1 = b1 = 1.666

the fitted regression in the original variables is

i. Yˆt = 0.69434 + 20.93322Xt − ii. Yˆt = 76.33 + 1.666Xt

iii. Yˆt = 339.14 13.3Xt − (e) First Difference Procedure Effective? True / False The first difference procedure has been effective because, according to the Durbin–Watson test, it has reduced the positive autocorrelation.

4. Understanding Why The Transformations Work.

0 (a) It is (circle one) good / bad that the transformations, Yt = Yt ρYt−1 0 − and Xt = Xt ρXt−1, result in a simple linear regression, as opposed to nonlinear regression.− 0 (b) It is (circle one) good / bad that the transformations, Yt = Yt ρYt−1 0 − and X = Xt ρXt− , result in a simple linear regression with independent t − 1 errors, ut, which replace the previously autocorrelated errors, εt = ρεt−1 + ut. (c) True / False The estimated regression function for

0 0 0 0 Yt = β0 + β1Xt + ut

is 0 0 0 0 Yt = b0 + b1Xt (d) True / False Since

0 β0 = β0(1 ρ) 0 − β1 = β1 252 Chapter 12. Autocorrelation in Time Series Data (ATTENDANCE 11)

the estimated regression function,

0 0 0 0 Yt = b0 + b1Xt

can be transformed back to the fitted regression in the original variables in the following way, b0 b = 0 0 1 r 0 − b1 = b1

(e) True / False The estimated standard deviations of the regression coeffi- cients are s b0 s b = { 0} { 0} 1 r s b = s −b0 { 1} { 1}

12.5 Forecasting with Autocorrelated Error Terms

SAS program: att11-12-5-read-cochranePI

The forecast for the period n + 1 is

Fn+1 = Yˆn+1 + ren

because

Yt = β0 + β1Xt + εt

εt = ρεt−1 + ut and so Yn+1 = β0 + β1Xn+1 + ρεn + un+1 where Yˆn = β + β Xn , r estimates ρ, e estimates ε and E u = 0. Prediction +1 0 1 +1 { } intervals are calculated using the forecast values.

Exercise 12.5 (Forecasting with Autocorrelated Error Terms)

illumination, X 1 2 3 4 5 6 7 8 9 10 ability to read, Y 70 70 75 88 91 94 100 92 90 85 Assume we use the Cochrane–Orcutt procedure throughout the analysis. Section 5. Forecasting with Autocorrelated Error Terms (ATTENDANCE 11) 253

1. Forecasting. Forecast the ability to read when X11 = 11. (a) Last Residual. Since Yˆt = 3.8978 + 29.7709Xt, − e = Y Yˆ = 85 [ 3.8978 + 29.7709(10)] = 10 10 − 10 − − (circle one) −47.25 / −67.25 / −208.8112

(b) Yˆ11 at X11 = 11 Also, since Yˆ = 3.8978 + 29.7709X = 3.8978 + 29.7709(11) = 11 − 11 − (circle one) 91.475 / 91.875 / 323.58

(c) Forecast at X11 = 11 Since, from above, r = 0.705, then

Yˆh = F = Yˆ + re = 323.58 + (0.705)( 208.8112) = 11 11 10 − (circle one) 65.31 / 80.34 / 176.37 2. 95% Prediction Interval. The prediction interval is given by

Yˆh t(1 α/2; n 2)s predmean § − − { } where we (a) Determine t(1 α/2; n 2) at α = 0.05. t(1 α/2, n 3)−= t(0.975;− 9) = 2.262. (circle one) 2.262 / 80.34 / 176.37 (PGRM− INVT− ENTER 9 ENTER 0.975 ENTER) ˆ 0 (b) Determine Yh. ˆ 0 . . . From above, Yh = (circle one) 65 31 / 80 34 / 176 37 (c) Determine s predmean . { } Since, at X11 = 11, from SAS, X0 = X rX = 11 0.705(10) = 3.952 11 11 − 10 − and so, using the transformed data,

1 (X0 X¯0)2 s pred = M SE0 1 + + h − ) 0 0 ¯0 2 { } s n (Xt X ) µ − ¶ 1 P(11 2.476)2 = 31.686512 1 + + − 9 5.229 s µ ¶ = (circle one) 6.9577258 / 80.34 / 176.37 254 Chapter 12. Autocorrelation in Time Series Data (ATTENDANCE 11)

and so a 95% prediction interval is given by

176.37 2.262(6.9577258) = (160.63, 192.26) §