Journal of Modern Applied Statistical Methods Volume 5 | Issue 1 Article 10 5-1-2006 The fficE iency Of OLS In The rP esence Of Auto- Correlated Disturbances In Regression Models Samir Safi James Madison University Alexander White Texas State University Follow this and additional works at: http://digitalcommons.wayne.edu/jmasm Part of the Applied Statistics Commons, Social and Behavioral Sciences Commons, and the Statistical Theory Commons Recommended Citation Safi, Samir and White, Alexander (2006) "The Efficiency Of OLS In The rP esence Of Auto-Correlated Disturbances In Regression Models," Journal of Modern Applied Statistical Methods: Vol. 5 : Iss. 1 , Article 10. DOI: 10.22237/jmasm/1146456540 Available at: http://digitalcommons.wayne.edu/jmasm/vol5/iss1/10 This Regular Article is brought to you for free and open access by the Open Access Journals at DigitalCommons@WayneState. It has been accepted for inclusion in Journal of Modern Applied Statistical Methods by an authorized editor of DigitalCommons@WayneState. Journal of Modern Applied Statistical Methods Copyright © 2006 JMASM, Inc. May, 2006, Vol. 5, No. 1, 107-117 1538 – 9472/06/$95.00 The Efficiency Of OLS In The Presence Of Auto-Correlated Disturbances In Regression Models Samir Safi Alexander White Department of Mathematics and Statistics Department of Mathematics and Statistics James Madison University* Texas State University The ordinary least squares (OLS) estimates in the regression model are efficient when the disturbances have mean zero, constant variance, and are uncorrelated. In problems concerning time series, it is often the case that the disturbances are correlated. Using computer simulations, the robustness of various estimators are considered, including estimated generalized least squares. It was found that if the disturbance structure is autoregressive and the dependent variable is nonstochastic and linear or quadratic, the OLS performs nearly as well as its competitors. For other forms of the dependent variable, rules of thumb are presented to guide practitioners in the choice of estimators. Key words: Autocorrelation, autoregressive, ordinary least squares, generalized least squares, efficiency Introduction X is full column rank k < T and its first column is 1's. The ordinary least squares (OLS) Let the relationship between an observable estimator of β in the regression model (1) is random variable y and k explanatory variables X , X , …, X in a T-finite system be specified −1 1 2 k βˆ = ()X′X X′ y (2) in the following linear regression model: In problems concerning time series, it is y= X β + u (1) often the case that the disturbances are, in fact, correlated. Practitioners are then faced with a where y is a ()T ×1 vector of observations on a decision, use OLS anyway, or try to fit a more response variable, X is a ()T × k design matrix, complicated disturbance structure. The problem β ()× is difficult because the properties of the is a k 1 vector of unknown regression estimators depend highly on the structure of the parameters, and u is a ()T ×1 random vector of independent variables in the model. For more disturbances. For convenience, it is assumed that complicated disturbance structures, many of the properties are not well understood. If the disturbance term has mean zero, i.e. E(u) = 0, *This article was accepted while Samir Safi was but is in fact, autocorrelated, i.e. ()= σ 2 ∑ ∑ × at James Madison University. He is now an Cov u u , where is a T T positive Assistant Professor of Statistics at the Islamic definite matrix and the variance σ 2 is either University of Gaza. His research interests are in u time series analysis, concerning the comparison known or unknown positive and finite scalar, of estimators in regression models with auto- then the OLS parameter estimates will continue correlated disturbances and efficiency of OLS in to be unbiased, i.e. E(βˆ)= β . But it has a the presence of autocorrelated disturbances. different covariance matrix; Alexander White is Associate Professor in Mathematics Education. His research interests −1 −1 (βˆ ) = σ 2 ()′ ′∑ ()′ (3) are in mathematics education, statistics and Cov ∑ u X X X X X X . mathematical finance. 107 108 OLS IN THE PRESENCE OF AUTO-CORRELATED DISTURBANCES The most serious implication of For a linear regression model with first autocorrelated disturbances is not the resulting order autocorrelated disturbances, several inefficiency of OLS, but the misleading alternative estimators for the regression inference when standard tests are used. The coefficients have been discussed in the literature, autocorrelated nature of disturbances is and their efficiency properties have been accounted for in the generalized least squares investigated with respect to the OLS and GLS (GLS) estimator given by: estimators (e.g. Kadiyala, 1968; Maeshiro, 1976; 1979; Ullah et al., 1983). − ~ −1 1 −1 The relative efficiency of GLS to OLS β = ()X′∑ X X′∑ y (4) in the important cases of autoregressive ~ disturbances of order one, AR(1), with which is unbiased, i.e. E(β) = β , with autoregressive coefficient ρ and second order, covariance matrix ()φ φ AR(2), with autoregressive coefficients 1 , 2 for specific choices of the design vector have − ()β~ = σ2 ()′∑ −1 1 Cov u X X . (5) been investigated. Building on work on the economics and The superiority of GLS over OLS is due to the time series literature, the price one must pay for fact that GLS has a smaller variance. According using OLS under suboptimal conditions required to the Generalized Gauss Markov Theorem, the investigation. Different designs are being GLS estimator provides the Best Linear explored, under which relative efficiency of the Unbiased Estimator (BLUE) of β . But the GLS OLS estimator to that of GLS estimator approaches to one or zero, determining ranges of estimator requires prior knowledge of the matrix first-order autoregressive coefficient, ρ , in Σ βˆ correlation structure, . The OLS estimator AR(1) disturbance and second order of is simpler from a computational point of view autoregressive coefficients, ()φ ,φ in AR(2) Σ 1 2 and does not require a prior knowledge of . for which OLS is efficient and quantifying the A common approach for modeling effect of the design on the efficiency of the OLS univariate time series is the autoregressive estimator. Furthermore, a simulation study has autoregressive model. The general finite order been conducted to examine the sensitivity of process of order p or briefly, AR(p), is estimators to model misspecification. In particular, how do estimators perform when an = φ + φ ++ φ + ε ε u t 1u t−1 2 u t−2 p u t−p t , t ~ AR(2) process is appropriate and the process is 2 incorrectly assumed to be an AR(1) or AR(4)? i.i.d. N()0,σε (6) Performance Comparisons There are numerous articles describing In this section, numerical results are the efficiency of the OLS coefficient presented using the formulas in (3) and (5). estimator βˆ , which ignore the correlation of the Focus will be placed on two issues; first, the ~ error, relative to the GLS estimator β , which relative efficiency of GLS estimator as compared with the OLS estimator when the takes this correlation into account. One strand is structure of the design vector, X, is concerned with conditions on regressors and nonstochastic. For example, linear, quadratic, error correlation structure, which guarantee that and exponential design vectors with an intercept OLS is asymptotically as efficient as GLS (e.g. term included in the design vector. Secondly, the Chipman, 1979; Krämer, 1980). The efficiency relative efficiency of the GLS estimator as of the OLS estimators in a linear regression compared with the OLS for a stochastic design containing an autocorrelated error term depends vector. In the example considered here, a on the structure of the matrix of observations on standard Normal stochastic design vector of the independent variables (e.g. Anderson, 1948; length 1000 was generated. The three finite 1971; Grenander & Rosenblatt, 1957). sample sizes used are 50, 100, and 200 for SAFI & WHITE 109 selected values of the autoregressive decreases with increasing values of ρ . For small coefficients. Both AR(1) and AR(2) error processes are considered to discuss the behavior and moderate sample sizes, the efficiency of the of OLS as compared to GLS. OLS estimator appears to be nearly as efficient as the GLS estimator for ρ ≤ .7 . In addition, for Performance Comparisons for AR (1) Process large size sample data, the OLS estimator The relative efficiencies of OLS to GLS performs nearly as efficiently as the GLS are discussed when the disturbance term follows estimator for the additional values of ρ = ±.9 . =ρ +ε = … an AR(1) process, ut ut−1 t , t 1,2, ,T, Further, the efficiency for estimating the slope assuming that the autoregressive coefficient, ρ , mimics the efficiency of the intercept, except for is known priori. The three finite sample sizes large sample size; the efficiency of the OLS used are 50, 100, and 200 for the elected values estimator appears to be nearly as efficient as the ρ ≠ ± of ρ ≤ .9 , evaluated in steps of .2. GLS estimator for .9. The efficiency of GLS estimator to the Table (1) shows the relative efficiencies OLS estimator for the quadratic design agrees of the variances of GLS to OLS for a regression with the behavior for the linear design vector. In coefficient on linear trend with an intercept term contrast, the gain in efficiency of the GLS included in the design. For estimating an estimator for different design vectors