ON THE PARTIAL SUMS OF RESIDUALS IN AUTOREGRESSIVE AND MOVING AVERAGE MODELS BY JUSHAN BAI University of California First version received May 1991 Abstract. The limiting process of partial sums of residuals in stationary and invertible autoregressive moving-average models is studied. It is shown that the partial sums converge to a standard Brownian motion under the assumptions that estimators of unknown parameters are root-n consistent and that innovations are independent and identically distributed random variables with zero mean and finite variance or. more generally, are martingale differences with moment restrictions specified in Theorem 1. Applications for goodness-of-fit and change-point problems are considered. The use of residuals for constructing nonparametric density estimation is discussed. Keywords. Time series models; residual analysis; Brownian motion; change-point problem; nonparametric density estimation. 1. INTRODUCTION Kulperger (1985) investigated the limiting process for partial sums of the residuals in autoregressive AR(p) models showing that the partial sums converge weakly to a standard Brownian motion. This paper extends this and other related results to an important class of time series models, namely autogressive moving-average (ARMA) models. Moreover, results are ob- tained under weaker conditions. More specifically, it is assumed that the innovations which drive the ARMA process have finite variance (for inde- pendent and identically distributed (i.i.d.) case) instead of finite fourth moment. Also, the estimators of unknown parameters are root-n consistent; they are not necessarily from least squares estimation. With the least squares estimators for AR models, Kulperger was able to use the Skorohod represen- tation technique to prove his theorems. In ARMA time series models, however, the residuals depend on the estimated parameters in a more complex way; therefore the Skorohod representation technique seems difficult to apply. Consequently, other techniques have to be used. It turns out that some elementary arguments lead to the desired results. In addition, the weak convergence of the residual process is proved for martingale-difference innovations. In a separate paper, Kulperger (1987) examined the residual process of regression models with autoregressive errors. He also considered a special case of ARMA (1,l) errors with the moving-average parameter being equal 0143-9782193103 247-60 JOURNAL OF TIME SERIES ANALYSIS Vol. 14, No. 3 0 1993 I. Bai. Published by Blackwell Publishers, 108 Cowley Road. Oxford OX4 lJF, UK, and 238 Main Street, Cambridge. MA 02142, USA. 24s J. BAI to one (non-invertible model). Such a model yields an interesting result in that different initial values used in calculating the residuals can lead to different limiting processes. This is in contrast with the results in this paper. As will be seen, the limiting process is not affected by the initial values. Residual partial sum processes and their applications have been studied for a variety of models. MacNeill (1978a,b) studied the residual processes of polynomial regression models and of general linear regression models, and proposed some testing statistics for parameter changes at unknown times. The residual partial sums for nonlinear regression models were studied by Mac- Neill and Jandhyala (1985). Recently, Jandhyala and MacNeill (1989) ob- tained the limit processes for partial sums of linear functions of regression residuals. They also derived asymptotic forms of some testing statistics for parameter changes. An important topic in time series analysis is to determine the adequacy of a fitted model. Many tests have been developed to perform such tasks. Most are based on residuals, such as the Box-Pierce portmanteau test, the turning-point and the rank test (for a brief summary see Brockwell and Davis, 1986, pp. 296-304). The results of this paper allow one to use some goodness-of-fit test statistics based on the partial sums. This paper is organized as follows. Section 2 introduces notations and assumptions and states the main results. Section 3 proves the weak converg- ence result for the ARMA (1,l) model and then generalizes the proof to ARMA (p,q) models. Some analytical properties of the power series expansion of the reciprocal of the associated polynomials are crucial to the generalization. These properties are derived in Lemma 1 and may be of independent interest. Statistical applications to the change-point problem and density estimation are considered in the final section. 2. NOTATIONS, ASSUMPTIONS AND RESULTS Consider the following ARMA (p, q) time series model: x, = p,x,-, + . + ppx,-p+ E, + el€,-,+. + eqE,-, (t = . ., -l,O, 1,. .), where p,, . ., pp and el,.. ., 8, are unknown parameters, {E,} is a sequence of i.i.d. random variables with zero mean and finite variance or, more generally, {E,} is a weakly stationary martingale-difference sequence (m.d.s.) with uniformly bounded 2 + S moment (S > 0). We also assume that the roots of 1- plz - fizz - . - pPzp = 0 and the roots of 1 + 8,z + dZzz+ . + 8,zq = 0 all lie outside the unit circle, and that the two polynomials (referred to as the associated polynomials) have no common roots, so that the process is stationary (weakly stationary unless i.i.d. innovations) and invertible. In the ARMA (1,l) case this requires lpll < 1 and 1011 < 1. ON THE PARTIAL SUMS OF RESIDUALS IN ARMA MODELS 249 Given n + p observations, X-,+l, X-,+Z,. ., Xo,X1,. .,X,,the n residuals can be calculated via the recursion 8, = x,- p'xf-l -. - p,x,-, - 8^'8,-1 -. - i3q8,-q, (t = 1, 2,.. ., n) (1) h A where PI,. ., P, and el,.. ., Oq are the estimators of pl,. .,pp and el, , . ., Oq respectively. The initial value of (8-q, . ., Z0) is set to the null vector as is commonly done in practice. The choice of initial values is inconsequential to our results. It could be estimated from the data or generated according to a random law (see the remarks at the end of Section 3) * In what follows op(l) (0,(1)) represents a sequence of random variables which converges to zero (remains bounded) in probability, and [x] denotes the greatest integer function of x. Moreover, Rk will denote the k-dimen- sional Euclidean space with k 3 1; R = R'; and (IuIII = cf=lluilfor u E Rk. Let so that B(")(x)and 8(")(x)are the partial sums based on the innovations and on the residuals respectively. THEOREM1. Assume that the following conditions hold. (a.1) The E, are i.i.d. with zero mean and variance uz, or (a.1') the E, are martingale differences satisfying n E(E~I~,-~) = o EE: = a2 n-' cE(E:lV,-l)-. a2, I=' and, for some 6 > 0, sup E(&,(2+6< m, t where Vfis the a-field generated by E,, s Q t. (a4 n'b(pi - pi) = 0,(1) i = I,. ., p n'P(Gi - ei)= 0,(1) i = I,. ., q. Then sup lB(")(x)- B'"'(x)l = op(l). 04x41 The proof of Theorem 1 is given in the next section. Note that condition (a.1') allows the conditional variance of E, to be non-constant. It is well known that under (a.1) or (a.1') B(")(x)converges weakly to 250 J. BAI B(x), where B(x) is a standard Brownian motion on [0,1] and the converg- ence is in the sense of D[O, 11, the space of right continuous real-valued functions on [0,1] endowed with the Skorohod J1 topology. See, for example, Theorem 16.1 of Billingsley (1968) for the i.i.d. case and Theorem 3.2 of McLeish (1974) for the m.d.s. case (with weaker assumptions). Therefore, under the conditions of Theorem 1, fi(")(x) also converges weakly to a standard Brownian motion. Consider as an application the problem of goodness-of-fit. A test for this problem could be based on where 6 is a consistent estimator of u, such as .n If the conditions of Theorem 1 are satisfied, then T, converges in distribution to supOsxsllB(x)I. High values of T,, may indicate inadequacy of the fitted model and suggest modifications. Other applications will be considered in the final section. 3. PROOF OF THEOREM 1 First consider the case with p = 1 and q = 1. The general case will be considered later. Omit the subscripts on the parameters; then (1) becomes h hE, = x,- ,ax,-,- eEt-l. Rewrite the ARMA (1, 1) model correspondingly, E, = x, - px,-,- eEf-l, and then subtract the second equation from the first to obtain h h h E, - E, = - e(2f-l- E,J - (p- p)x,-,- (e - e)E,-,. (2) Notice the recursiveness of 2, - E,. By repeated substitution and use of go = 0, we have A E, - Et = f- 1 f-1 (-1)t-l8t&o - (p - p) c (-l)j8jxf-l-j - (8 - 0) C (-l)j8j&,-l-j, j=O j=O (3) There are alternative expressions for relating i?, and E,, but (3) is easier to work with. From (3) we have ON THE PARTIAL SUMS OF RESIDUALS IN ARMA MODELS 25 1 lnxl f-1 1 [nx] 1-1 - n1I2 (8 - e) - 2 C (-i)j8j~~+~. (4) on f=l j=o Since 101 < 1, there is a 8 > 0 such that 101 < 8 < 1. By assumption (a.2), P(6 $ [-8, 61) + 0. Consequently, to prove Theorem 1, it suffices to show that 1 [nxl sup sup - 2 JU'&oI = o,(l), (5) lUlSB OSXCl n'i2 We shall prove (6) only, since (5) is obvious and (7) is simpler than (6). The proof of (6) is completed in (22). The main arguments are (8) and (20). Let us first show that for each u E [ - 8, 81, (8) Notice that The first term on the right-hand side consists of partial sums of a stationary process which are more convenient to work with, and the second is o,(l) uniformly in x since Consequently, we need to be concerned with the first term on the right-hand side of (9).