Gaza ـــ Al- Azhar University Deanship of Postgraduate Studies & Scientific Research Faculty of Economics and Administrative sciences Department of Applied Statistics

Comparative Study of Portmanteau Tests for ARMA Models

دراﺳﺔ ﻣﻘﺎرﻧﺔ ﻻﺧﺗﺑﺎرات اﻻ رﺗﺑﺎط اﻟذاﺗﻲ ﻟﻸﺧطﺎء اﻟﻌﺷواﺋﯾﺔ

ﻟﻧﻣﺎذج اﻵرﻣﺎ

Presented By Alaa Ahmad Salman Al-Reqep

Supervised By Samir Khaled Safi, Ph.D. Associate Professor of Statistics

SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF APPLIED STATISTICS

2013-1435

To My Family

Acknowledgment

Foremost, I would to thank God for the wisdom and perseverance that he has been bestowed upon me during this thesis, and indeed, throughout my life. I am grateful to a number of people who have guided and supported me. My first sincere gratitude and appreciation goes to my advisor Professor Samir Safi, for his patience, continuous help and support in all stages of this thesis and for encouraging and helping me to shape my interest and ideas. His recommendations and instructions have enabled me to assemble and finish my thesis effectively. Besides my advisor, I would like to thank Dr. Mahmoud Okasha and Dr. Abdalla Al Habil, I would not have been able to achieve my learning in the same manner without their immense knowledge. I would like to thank Dr. Bisher Iqelan for accepting to be my external examiner. Also I would like to express the deepest appreciation to Professor Thomas J. Fisher from Department of Mathematics at University of Missouri, Kansas City, USA for his comments and guidance for the simulation study. Also I would like to express my thanks to Mr. Eyad El Shamy from Department of Information Technology at the Islamic University of Gaza for his help in programming software in order to accelerate simulation's time. Especial thanks to Energy Authority, Khan Younis for providing me by Electricity Consumption data and to my teacher in the first class Ms. Halimah Al Tartory. My sincere gratitude to my parents, brothers and sisters especially Dr. Bara'ah for believing in my abilities and for their unconditional support, spiritually throughout my life. And special thanks to my husband Eyad, my sons and two persons I consider them as long as my parents; my husband's parents for providing the moral and emotional support I needed to complete my thesis, without them I would not have been success in my life. My friends have supported and helped me along the course of this degree by giving encouragement; to them, I am eternally grateful. Finally, I would like to thank all the people who contributed in some way to the work described in this thesis.

Table of Contents

List of Tables iii List of Figures iv List of Outputs v Abstract vi Abstract in Arabic vii Chapter 1 Introduction and Literature Review 1 1.1 Introduction 1 1.2 Problem of The Study 1 1.3 Objectives of The Study 2 1.4 Literature Review 2 1.5 Fundamental Concepts 3 1.5.1 Mean, Variance and Covariance 3 1.5.2 Time Series 6 1.5.3 Stationarity 7 1.5.4 Nonstationarity 8 1.5.5 Differencing 9 1.5.6 Unit Root Tests 9 1.6 Box-Jenkins ARIMA Models 10 1.6.1 Auto-Regressive AR(p) Model 10 1.6.2 Moving Average MA(q) Model 11 1.6.3 Auto-Regressive – Moving Average ARMA(p, q) Model 12 1.7 Autocorrelation, Partial Autocorrelation, and Extended Partial Autocorrelation Functions 13 1.7.1 Autocorrelation Function (ACF) 13 1.7.2 Partial Autocorrelation Function (PACF) 13 1.7.3 Extended autocorrelation function (EACF) 14 1.8 Summary 17

Chapter 2 Model Diagnostic 18 2.1 Introduction 18 2.2 Residual Analysis 18 2.2.1 Homogeneity of Variance and Zero Mean 18 2.2.2 Normality of Residuals Distribution 19 2.2.3 Autocorrelation of Residuals 20 2.3 Summary 26

Chapter 3 Portmanteau Tests For ARMA Models 27 3.1 Introduction 27 3.2 Portmanteau Tests 27 3.2.1 Box and Pierce Portmanteau Test 27 3.2.2 Ljung-Box Portmanteau Test 29 3.2.3 Monti Portmanteau Test 31

i 3.2.4 Peña and Rodríguez Portmanteau Test (2002) 33 3.2.5 Peña and Rodríguez Portmanteau Test (2006) 34 3.2.6 Fisher Portmanteau Tests 37 3.3 Summary 39

Chapter 4 Simulation and Case Study 40 4.1 Introduction 40 4.2 Simulation Study 40 4.2.1 Simulation Study on Short Term Data 40 4.2.2 Simulation Study on moderate Term Data 42 4.2.3 Simulation Study on Long Term Data 43 4.3 Numerical Example 45 4.3.1 ARMA Model Building Process 45 4.3.2 Data Exploration 46 4.3.3 Fitting an Inappropriate ARIMA(1,1,0) Model 47 4.3.4 Fitting an Appropriate Model 47 4.4 Summary 48

Chapter 5 Conclusion and Recommendations 49 5.1 Conclusion 49 5.2 Recommendations and Future Research 50

References 51 Appendix A 54

ii

List of Tables

Table 1.1 Theoretical EACF Table for an ARMA(1,1) Model 15 Table 1.2 Behavior of the ACF and PACF for ARMA Models 15 Table 1.3 ACFs and PACFs Plots 16 Table 4.1 Powers of Portmanteau Tests for N = 50,   0.05 42 Table 4.2 Powers of Portmanteau Tests for N = 200,   0.05 43 Table 4.3 Powers of Portmanteau Tests for N = 500,   0.05 44 Table 4.4 P-values of the Portmanteau Tests of the Residuals for ARIMA(1,1,0) Model 47 Table 4.5 EACF for Difference of Electricity Consumption Series 48 Table 4.6 P-value of the Portmanteau Tests of the Residuals for ARIMA(0,1,1) Model 48 Table A.1 Powers of Portmanteau Tests for N = 50, m=10 and 15, and   0.05 54 Table A.2 Powers of Portmanteau Tests for N = 200, m=10 and 20, and   0.05 54 Table A.3 Powers of Portmanteau Tests for N = 500, m=15 and 20, and   0.05 55

iii

List of Figures

Figure 1.1 Stationary Time Series 7 Figure 1.2 Nonstationary Time Series 8 Figure 1.3 Random Walk Model 8 Figure 2.1 Standardized Residuals over the Time 19 Figure 2.2 Histogram Plot for Residuals 19 Figure 2.3 Normal QQ plot for Residuals 20 Figure 2.4 Sample ACF of the Residuals 21 Figure 2.5 Standardized Residuals from MA(1) Model 21 Figure 2.6 Histogram for the Residuals from MA(1) Model 22 Figure 2.7 QQ-plot for the Residuals from MA(1) Model 22 Figure 2.8 Sample Autocorrelation of Residuals from MA(1) Model 23 Figure 2.9 Standardized Residuals for AR(2) Model from NHtemp Data 24 Figure 2.10 Histogram for the Residuals for AR(2) Model from NHtemp Data 24 Figure 2.11 QQ-plot for the Residuals for AR(2) Model from NHtemp Data 25 Figure 2.12 Sample ACF of Residuals for AR(2) Model from NHtemp Data 25 Figure 3.1 Diagnostic Display for the ARMA(0,1,1) Model of Kings Series 31 Figure 4.1 ARMA Model Building Process 45 Figure 4.2 Monthly Amount of Electricity Consumption 46 Figure 4.3 Difference of Amount of Electricity Consumption 46

iv

List of Outputs

Output 2.1 Shapiro-Wilk Test for Residuals of MA(1) Model 23 Output 2.2 Shapiro-Wilk Test for Residuals of AR(2) Model from (NHtemp) data 25 Output 3.1 Box-Pierce Test 29 Output 3.2 Ljung-Box Test 30 Output 3.3 Monti Test for m = 5 , 15 32 Output 3.4 Gvtest 36 Output 3.5 Weighted Box Test for m = 10, 15 38 Output 3.6 Weighted Monti Test for m = 7, 13 39

v

Abstract

The portmanteau statistic for testing the adequacy of an autoregressive moving average (ARMA) model is based on the first m autocorrelations of the residuals from the fitted model. We consider some of portmanteau tests for univariate linear time series such as Box and Pierce (1970), Ljung and Box (1978), Monti (1994), Peña and Rodríguez (2002, 2006), Generalized Variance Test (Gvtest) by Mahdi and McLeod (2012) and Fisher (2011). We conduct an extensive computer simulation time series data, to make comparison among these tests. We consider different model parameters for short, moderate and long terms data to examine the effect of lag m on the power of the selected tests and determine the most powerful test for ARMA models. An example of real data is also given. The similar portmanteau tests models was evaluated for the real data set on electricity consumption in Khan Younis, Palestine (April 2009 - May 2013). We found that, the long term data (N = 500) has the highest values of power comparing to short and moderate terms data (N = 50 and 200). We found that the portmanteau tests are sensitive to the chosen for m value, Indeed there are loss of the power for lags m ranging from m = 5 to 20, where Box-Pierce, Ljung-Box and Monti tests have the more power loss than the other selected tests. The power loss reaches its smallest values for long term data comparing to small and moderate terms. In addition, the results of the simulation study and real data analysis showed that the highest powerful tests varies between Gvtest (2012) test and Fisher tests (2011).

vi

Abstract in Arabic

إﺣﺻـــــﺎء اﻻرﺗﺑـــــﺎط اﻟـــــذاﺗﻲ ﻟﻸﺧطـــــﺎء اﻟﻌﺷـــــواﺋﯾﺔ ﻻﺧﺗﺑـــــﺎر ﻛﻔـــــﺎءة ﻧﻣـــــوذج اﻻﻧﺣـــــدار اﻟـــــذاﺗﻲ ﻟﻠوﺳـــــط اﻟﻣﺗﺣـــــرك

(ARMA) ﯾﻌﺗﻣــد ﻋﻠــﻰ أول ﻋــدد m ﻣــن اﻻرﺗﺑﺎطــﺎت اﻟذاﺗﯾــﺔ ﻟﻸﺧطــﺎء اﻟﻌﺷــواﺋﯾﺔ ﻟﻠﻧﻣــوذج اﻟﻣﻧﺎﺳــب اﻟــذي ﺗــم

ﻣﻼﺋﻣﺗﻪ ﻟﻠﺑﯾﺎﻧﺎت.

ﻗـدﻣت ﻫــذﻩ اﻟدراﺳــﺔ اﺧﺗﺑـﺎرات اﻻرﺗﺑــﺎط اﻟــذاﺗﻲ ﻟﻸﺧطــﺎء اﻟﻌﺷـواﺋﯾﺔ ﻣﻧﻬــﺎ (Ljung- ،Box-Pierce (1970 (Fisher ، Gvtest(2012)، Peña and Rodríguez (2002, 2006)،Monti(1994،Box (1978

(2011) ﻟﻠﺳﻼﺳـــــل اﻟزﻣﻧﯾـــــﺔ اﻟﺧطﯾـــــﺔ ذات اﻟﻣﺗﻐﯾـــــرات اﻷﺣﺎدﯾـــــﺔ، وﻗـــــد ﻗﺎﻣـــــت ﺑﺎﻟﻣﻘﺎرﻧـــــﺔ ﺑـــــﯾن ﻫـــــذﻩ اﻻﺧﺗﺑـــــﺎرات

ﻣﺳـﺗﺧدﻣﺔ ﺑﯾﺎﻧــﺎت ﻟﺳﻼﺳــل زﻣﻧﯾــﺔ ﻣوﻟــدة ذات ﻣﻌﻠﻣــﺎت ﻣﺧﺗﻠﻔــﺔ ﻟﺑﯾﺎﻧــﺎت ﻗﺻــﯾرة وﻣﺗوﺳــطﺔ وطوﯾﻠــﺔ اﻟﻣــدى ﻟﻣﻌرﻓــﺔ

ﻣدى ﺗﺄﺛر ﻗوة اﻻﺧﺗﺑﺎر ﺑـﻔﺎرق اﻟزﻣن (m) وﻣن ﺛم ﺗﺣدﯾد اﻻﺧﺗﺑﺎر اﻷﻛﺛر ﻗـوة ﻣـن ﺑﯾﻧﻬـﺎ ﻟﻧﻣـﺎذج اﻻﻧﺣـدار اﻟـذاﺗﻲ

ﻟﻠوﺳــط اﻟﻣﺗﺣــرك (ARMA). وﻗﺪ ﺗﻢ ﺗﻄﺒﯿﻖ ھﺬه اﻻﺧﺘﺒﺎرات ﻋﻠﻰ ﺑﯿﺎﻧﺎت ﺣﻘﯿﻘﯿﺔ وھﻲ ﻛﻤﯿﺎت اﺳﺘﮭﻼك اﻟﻜﮭﺮﺑﺎء ﻟﻤﺪﯾﻨﺔ ﺧﺎن ﯾﻮﻧﺲ، ﻓﻠﺴﻄﯿﻦ ﻣﻦ اﺑﺮﯾﻞ 2004 إﻟﻰ ﻣﺎﯾﻮ 2013.

وﻗــد وﺟـــدﻧﺎ ﻣـــن ﺧـــﻼل ﻧﺗـــﺎﺋﺞ اﻟﻣﺣﺎﻛـــﺎة أن اﻻﺧﺗﺑـــﺎرات ﺗﻛــون أﻗـــوى ﻓـــﻲ ﺣﺎﻟـــﺔ اﻟﻌﯾﻧـــﺎت اﻟﻛﺑﯾـــرة (N=500) ﻣﻧﻬـــﺎ

ﻟﻠﻌﯾﻧــﺎت اﻟﺻــﻐﯾرة(N=50, 200). وﺧﻠﺻــت اﻟدراﺳــﺔ إﻟــﻰ أن اﻻﺧﺗﺑــﺎرات دﻗﯾﻘــﺔ ﺑﺎﻟﻧﺳــﺑﺔ ﻟـــﻘﯾﻣﺔ (m) ﺣﯾــث أﻧــﻪ

ﯾوﺟد ﺧﺳﺎرة ﻓﻲ ﻗوة اﻻﺧﺗﺑﺎر ﻋﻧد اﻻﻧﺗﻘـﺎل ﻣـن m=5 إﻟـﻰ m=20 ، ﻓﺎﺧﺗﺑـﺎري Ljung-Box & Monti ﻟﻬﻣـﺎ

أﻛﺛر ﻣﻌدل ﺧﺳﺎرة ﻣن ﺑﯾن اﻻﺧﺗﺑﺎرات اﻷﺧرى، و ﺗﻛـون ﻫـذﻩ اﻟﺧﺳـﺎرة ﺿـﺋﯾﻠﺔ ﺟـداً ﺑﺎﻟﻧﺳـﺑﺔ ﻟﻠﻌﯾﻧـﺎت اﻟﻛﺑﯾـرة ﻋﻠـﻰ

ﻋﻛس اﻟﻌﯾﻧﺎت اﻟﺻﻐﯾرة و اﻟﻣﺗوﺳطﺔ.

و ﻗــد أظﻬــرت ﻧﺗــﺎﺋﺞ اﻟﻣﺣﺎﻛــﺎة و ﻧﺗــﺎﺋﺞ ﺗﺣﻠﯾــل ﺑﯾﺎﻧــﺎت ﻛﻣﯾــﺔ اﺳــﺗﻬﻼك اﻟﻛﻬرﺑــﺎء أن أﻛﺛــر اﻻﺧﺗﺑــﺎرات ﻗــوة ﻫــﻲ

(Gvtest (2012 و (Fisher (2011.

vii

Chapter 1

Introduction and Literature Review

1.1 Introduction

Time series model diagnostic checking is the most important stage of time series model building. In examining the adequacy of a statistical model, an analysis of the residuals is often performed. The study of the distribution of residual autocorrelations in linear time-series models started with the seminal work of Box and Pierce (1970). If the appropriate model has been chosen, there will be zero autocorrelation in the errors and we use one of the portmanteau tests in time series analysis for testing the adequacy of a fitted linear time series model, also known as autoregressive moving average (ARMA) or Box–Jenkins model. This study proceeds as follows: Chapter 1 describes several major literature review with some important time series terminology. Chapter 2 focuses on the model diagnostic phase in building time series ARMA models. Chapter 3, presents properties of portmanteau tests with some illustrative examples. Chapter 4 discusses the power of the portmanteau tests based on both Monte Carlo study and real case study. Finally, Chapter 5 summarizes the results of the study and proposed some recommendations based in our study.

1.2 Problem of The Study

A portmanteau test is proposed to test the goodness of fit of ARMA models in time series. This test firstly has been studied by Box and Pierce (1970), then it has been improved by Ljung and Box (1978), this test is known as the Ljung–Box test. Over decades, this test was improved by many statisticians, as a result, we have many test statistics for the same purpose for univariate time series models. So our problem of the study is that we don’t know which is the most effective and powerful test for univariate and linear ARMA models.

1 1.3 Objectives of The Study

In this study we compare the performance of portmanteau tests through an extensive numerical simulation for different model parameters and sample sizes. These simulations examine the sensitivity of choosing model parameters to different sample sizes. In particular, how do these tests perform for different model parameter specifications and for small, moderate and large sample sizes? In addition, determine the most powerful portmanteau test and study the effect of the lags m on the power of these tests based on both simulation study and real data set.

1.4 Literature Review

Portmanteau tests have been studied by many authors, we present some previous papers in this field:

The properties of the portmanteau test statistic was examined by Ljung (1986) for various choices of m autocorrelations of the residuals from the fitted ARMA model. A modification which allows the use of small values of m is shown to result in a more powerful test. Test of goodness of fit for time series models was proposed by Monti (1994) based on the sum of squared residuals partial autocorrelations. The test statistic is asymptotically  2 . Its small sample performance is studied through a Monte Carlo experiment. It appears sensitive to erroneous specifications especially when the fitted model understates the order of the moving average component. Finite-sample performance of Monti's test was investigated by Kwan and Wu (1997), paying special attention to its estimated sizes and empirical powers. Their simulation results indicate that (i) the test size can be affected by the choice of the number of residual partial autocorrelations, m, and (ii) the empirical powers of the Monti and the Ljung-Box tests are similar in the cases of both seasonal and nonseasonal data if m is properly chosen. Chand, S. and Kamal, S. (2006) compared the performances of Box-Ljung test and Monti’s test under the different alternative hypothesis using Monte Carlo experiment. They show that Monti’s test shows better approximation to chi-square distribution and is at least as good as that of the Ljung-Box statistic. Monti’s test provides stable results over different values of lag m. Lin and McLeod (2006) noted several problems with the diagnostic test that has been suggested by Peña and Rodríguez in (2002) and an improved Monte Carlo version of this test is suggested. It is shown that quite often the test statistic recommended by Peña and Rodríguez (2002) may not exist and their asymptotic distribution of the test does not agree with the suggested gamma approximation very well if the number of lags used by the test is small. It is shown that the

2 convergence of this test statistic to its asymptotic distribution may be quite slow when the series length is less than 1000 and so a Monte Carlo test is recommended. Simulation experiment suggest the Monte-Carlo test is usually more powerful than the test given by Peña and Rodríguez (2002) and often much more powerful than the Ljung Box portmanteau test. In this study Peña and Rodríguez (2006) proposed a finite sample modification of their

1 ˆ m  previous test which is D n1  R m  this statistic based on the determinant of the mth residual   autocorrelation matrix Rm . The new modified test is asymptotically equivalent to the previous one but it has a more intuitive explanation and it can be 25% more powerful for small sample size. The test statistic is the logarithm of the determinant of the mth autocorrelation matrix. Two new statistics were introduced by Fisher (2011) that are weighted variations of the common Ljung-Box and, the less-common, Monti statistics for checking the adequacy of a fitted stationary ARMA process. The new test statistics put more emphasis (weight) on the first few autocorrelations those most likely to deviate from zero and hence is more likely to detect the fitted ARMA model.

1.5 Fundamental Concepts

In this section we will provide a cursory look for the basic terms used in time series analysis.

1.5.1 Mean, Variance and Covariance

We will define expectation, variance and covariance which we will use through some important time series definitions. Firstly, we must introduce some basic concepts such that discrete and continuous random variable, probability density function and probability mass function. Definition 1.1: (Casella & Berger, 2002) A random variable is a function from a sample space S into the real numbers. Example1.1: When we toss two dice, then X = sum of the numbers is a random variable. Definition 1.2: (Hogg & etl., 2005) Let X be a random variable. Then its cumulative distribution function (cdf), is defined by, F()((,])() x P  x  P X  x XX (1.1)

Theorem 1.1: (Casella & Berger, 2002) The function F() x is a cdf if and only if the following three conditions hold: a. limF( x ) 0 and lim F ( x )  1 x x 

3 b. F() x is a nondecreasing function of x . c. F() x is right-continuous; that is, for every number x 0 , F( x ) F ( x 0 ). limx x 0 Definition 1.3: (Hogg & etl., 2005) A random variable is a continuous random variable if its cumulative distribution function FX () x is a continuous function for all x R , where R is the real number. Definition 1.4: (Hogg & etl., 2005) We say a random variable is a discrete random variable if its space is either finite or countable. The following example discusses the cdf of a continuous random variable. 1 Example1.2: An example of a continuous cdf is the function F() x  , so F() x X 1e x X which satisfies the conditions of Theorem 1.1, limF( x ) 0 sin ce lim e x   x x  and limF( x ) 1 sin ce lim e x  0 x x 

Differentiating FX () x gives

d e x FX ( x )2  0, dx 1e x 

Showing that FX () x is increasing. FX is not only right-continuous, but also continuous. This is a special case of the logistic distribution. Definition 1.5: (Hogg & etl., 2005) Let X be discrete random variable with space S . The probability mass function (pmf) of X is given by p(), x P X  x for x  S X   (1.2)

Note that pmf satisfy the following two properties: 0p ( x )  1, x  S and p ( x )  1. XXx S Example 1.3: For the geometric distribution we have the pmf

p1 px 1 , x  1,2,... pX () x P X  x    0elsewhere .

Definition 1.6: (Casella & Berger, 2002) The probability density function (pdf), fX () x ,of a continuous random variable X is the function that satisfies

4 x F()(). x f t dt for all x XX (1.3)  Note that pdf satisfy the following two properties:

 f( x ) 0 and f ( t ) dt  1 XX  Example 1.4: For logistic distribution of Example 1.2 we have cdf 1 F() x  X 1e x and hence, d e x fXX()(). x F x  2 dx 1e x  Definition 1.7: (Hogg & etl., 2005) Let X be a random variable: If X is a continuous random variable with probability density function (pdf) f() x and

  x f(), x dx   (1.4)  then the expected value of X is

 E()() X  xf x dx (1.5)  If X is a discrete random variable with probability mass function (pmf) p() x and

 x p(), x   (1.6) x then the expected value of X is

E()() X  xp x (1.7) x and it is also called the expectation of X or the mean of X and is often denoted  or X . Example 1.5: (Discrete random variable) Let the random variable X of the discrete type have the pmf given by X 1 2 3 4 p(x) 0.4 0.1 0.3 0.2 Here p(x) = 0 if x is not equal to one of the first four positive integers. We have EX( ) 1(0.4)  2(0.1)  3(0.3)  4(0.2)  2.3 Example 1.6: (Continuous random variable) Let X have the pdf 4x2 0 x  1 f() x    0elsewhere .

5 Then,

1 1 5 1 3 4 4x  4 E( X ) x (4 x ) dx   4 x dx    . 0 0 5 0 5 Definition 1.8: (Cryer & Chan, 2008) Let X be a random variable. The variance of X is defined by

2 Var()() X E X  E X  (1.8)

The variance is often denoted by  or X .In general, it may be shown that

2 2 Var()()() x E X  E X  (1.9)

Definition 1.9: (Cryer & Chan, 2008) Let X and Y be two random variables. The covariance of X and Y is defined by

Cov(,) X Y E X XY Y     (1.10)

Definition 1.10: A measure of linear association between two random variables X and Y is called correlation and defined by Cov(,) X Y  Corr(,) X Y  (1.11) Var()() X Var Y

It is always true that 1   1.The larger the absolute value of  , the stronger the linear association between X and Y.

1.5.2 Time Series

Definition 1.11: A time series is a sequence of data points, measured typically at successive points in time spaced at uniform time intervals.

For example, the daily closing stock prices, monthly CO2 concentration, the annual flow volume of the Nile River, and so on. Time series are used in statistics, signal processing, pattern recognition, econometrics, mathematical finance, weather forecasting, earth quake prediction, astronomy, and communications engineering. The usage of time series analysis is twofold: (i) To obtain and understanding of the underlying forces and structure that produced the observed data, (ii) To fit a model and proceed to forecasting, monitoring or even feedback and feed forward control. There are many methods used to fit and forecast time series data, we will deal with Box-Jenkins ARIMA models for univariate time series model in our study.

6 Definition 1.12: (NIST, 2010) A univariate time series is a time series that consists of single (scalar) observations recorded sequentially over equal time increments.

1.5.3 Stationarity

Definition 1.13: A model {}Y t is covariance stationary if:

EY()t  Y a constant for all t

2 Var() Y t  Y a constant for all t

Corr(,) Yt Y t k is a function of k a lone for all t and integer k . That is the mean, the variance and the correlation structure of stationary series do not change over the time. So time series with trends, or with seasonality, are not stationary. On the other hand, a white noise series is stationary, it does not matter when you observe it, it should look much the same at any period of time.

Definition 1.14: A stationary time series{}t is said to be white noise if :

E ()t  constant ,t

Var(t ) , for k  0 Cov (,)t  t k   0 ,for k  0

Thus, t is a sequence of uncorrelated random variables with constant variance and constant mean. White noise is usually assumed to has zero mean, (Cryer & Chan, 2008). Some cases can be confusing; a time series with cyclic behavior (but not trend or seasonality) is stationary. That is because the cycles are not of fixed length, so before we observe the series we cannot be sure where the peaks and troughs of the cycles will be. In general, a stationary time series will have no predictable patterns in the long-term. Time plots will show the series to be roughly horizontal (although some cyclic behavior is possible) with constant variance as in Figure 1.1.

Figure 1.1 Stationary Time Series

7 1.5.4 Nonstationarity

Definition 1.15: (Cryer & Chan, 2008) A nonstationary time series is any time series without a constant mean over time as in Figure 1.2. That mean it have systematic trends, such as linear, quadratic, and so on. A random walk model is very widely used for non-stationary data, particularly finance and economic data.

Figure 1.2 Nonstationary Time Series

Definition 1.16: A random walk is defined as a process where the current value of a variable is composed of the past value plus an error term(t ) defined as a white noise. Algebraically a random walk is represented as follows:

YYt t1  t (1.12) The forecasts from a random walk model are equal to the last observation, as future movements are unpredictable, and are equally likely to be up as in Figure 1.3 or down. Random walks typically have: (i) Long periods of apparent trends up or down, (ii) Sudden and unpredictable changes in direction. A nonstationary series can be made stationary by differencing.

Figure 1.3 Random Walk Model

8 1.5.5 Differencing

Definition 1.17: (Hyndman & Athanasopoulos, 2013) Differencing of a time series is the transformation of the series Y t to a new time series d t where the values d t are the differences between consecutive values of Y t .

This procedure may be applied consecutively more than once, giving rise to the "first differences", "second differences", etc. The first difference removes linear trend, the second difference removes quadratic trend, and so on.

The first differences d t (1) of a time series Y t are described by the following expression: d(1)  Y  Y  Y t t t t 1 (1.13)

The differenced series will have only n-1 observations since it is not possible to calculate a difference Y t  for the first observation. When the differenced series is white noise, the model for the original series can be written as

Y Y  or Y  Y   . t t1 t t t  1 t (1.14)

Occasionally the differenced data will not appear stationary and it may be necessary to difference the data a second time to obtain a stationary series, so the second differences d t (2) may be computed from the first differences d t (1) according to the expression:  dt(2) d t (1)  d t 1 (1)  YYYt t   t 1  (1.15)  YYYY     t t1  t  1 t  2 

In this case, Y t  will have n−2 observations. In practice, it is almost never necessary to go beyond second-order differences. The general expression for the differences of order j is given by the recursive formula d( j ) d ( j  1)  d ( j  1) , j  1,2,... t t t 1 (1.16)

1.5.6 Unit Root Tests

One way to determine more objectively if differencing is required is to use a unit root test. These are statistical hypothesis tests of stationarity that are designed for determining whether differencing is required.

9 A number of unit root tests are available, and they are based on different assumptions. One of the most popular tests is the Augmented Dickey-Fuller (ADF) test. Implementation in R: The test can be carried out using tseries package with R command: adf.test(x, alternative = "stationary") The null-hypothesis for an ADF test is that the data are non-stationary. So large p-values are indicative of nonstationarity, and small p-values suggest stationarity. Using the usual 5% threshold, differencing is required if the p-value is greater than 0.05. Another popular unit root test is the Kwiatkowski-Phillips-Schmidt-Shin (KPSS) test. This reverses the hypotheses, so the null-hypothesis is that the data are stationary. In this case, small p-values (e.g., less than 0.05) suggest that differencing is required. Implementation in R: The test can be carried out using tseries package with the following R command: kpss.test(x)

1.6 Box-Jenkins ARIMA Models

ARIMA is the abbreviation of an auto-regressive integrated moving average model. The three terms to be estimated in the model are auto-regressive (p), integrated (d) , and moving average (q).

Remark 1.1: Terms (d) is the terms needed to make a nonstationary time series stationary.

ARIMA model is a generalization of an autoregressive moving average ARMA model. Given a time series of data Y t , the ARMA model is a tool for understanding and predicting future values in this series. The model consists of two parts, an autoregressive (AR) part and a moving average (MA) part. The model is usually then referred to as the ARMA(p, q) model where p is the order of the autoregressive part and q is the order of the moving average part, so there are three common processes:

1.6.1 Auto-Regressive AR(p) Model

Definition 1.18: (Chan & Cryer, 2008) The notation AR(p) refers to the autoregressive model of order p. The AR(p) model is written

p

YYt i t i   t (1.17) i 1

where1,, p are the parameters of the model and t is white noise.

10 With AR characteristic polynomial

(x ) 1   x   x2    x p 1 2 p (1.18)

Each observation is made up of a random error component (random shock,  ) and a linear combination of prior observations.

Stationarity requirement: An autoregressive process will only be stable if the parameters are within a certain range, for example; if there is only one autoregressive parameter then is must fall within the interval of -1 <  < 1. Otherwise, past effects would accumulate and the values of successive Y t s would move towards infinity, that is, the series would not be stationary. If there are two such parameters (p = 2) they must also meet the following requirements:

  1,    1 and   1 1 2 2 1 2 (1.19)

For general AR(p) the process is stationary if the roots of (x ) 0exceed 1 in modulus, (Chan & Cryer, 2008).

It is possible to write any stationary AR(p) model as an MA( ) model. For example, using repeated substitution, we can demonstrate this for an AR(1) model :

YYt t1   t    Y t2   t  1    t  2  Y t2    t  1   t  (1.20) 3 2  Y         t3 t  2 t  1 t     Provided -1 <  < 1, the value of  k will get smaller as k gets larger. So eventually we obtain:

Y      2  , t t t1 t  2 (1.21) an MA(  ) process, (Hyndman & Athanasopoulos, 2013).

1.6.2 Moving Average MA(q) Model

Definition 1.19: (Chan & Cryer, 2008) The notation MA(q) refers to the moving average model of order q

q

Y t t    i  t i (1.22) i 1

11 where the 1,, q are the parameters of the model and the t,,  t 1  are again, white noise error terms. With MA characteristic polynomial

2 q (x ) 1  1 x   2 x   q x (1.23)

Each observation is made up of a random error component (random shock,  ) and a linear combination of prior random shocks.

Invertibility requirement: The MA(q) process is invertible if the roots of  (x ) 0 exceed 1 in modulus, so if   1 in MA(1) then the model is invertible. For an MA(2) model:

1  2  1, 1  2  1 and 2 1 , (Hyndman & Athanasopoulos, 2013). That is, that we can write any invertible MA(q) process as an AR(  ) process. we can demonstrate this for an MA(1) model :

Y t t    t 1 (1.24)

First rewriting this as tY t    t 1 , then

tY t    t 1   YYt  t1    t  2   2  YYt  t1    t  2  (1.25) 2 3  YYY       t t1 t  2 t  3    

Since -1 <  < 1, the value of  k will get smaller as k gets larger. So eventually we obtain

2 tYYY t   t1   t  2 , (1.26) or

2 YYYt t (  t1   t  2  ), (1.27) an AR(  ) process. Hence we say that the MA(1) model is invertible if and only if   1, (Chan & Cryer, 2008).

1.6.3 Auto-Regressive – Moving Average ARMA(p, q) Model

Definition 1.20: (Chan & Cryer, 2008) This model contains the AR(p) and MA(q) models,

p q

YYt t   i t i    i  t  i (1.28) i1 i  1 where i,  i and t as a above . For general ARMA(p,q) model, we require both stationarity and invertibility.

12 1.7 Autocorrelation, Partial Autocorrelation, and Extended Partial Autocorrelation Functions

1.7.1 Autocorrelation Function (ACF)

The concept of correlation plays a key role in time series analysis. The stationarity

assumption implies that Corr(,) Yt Y t k depends only on the time separation k , and not on the time location t. Remark 1.2: Lag k is the time periods between two observations. For example, lag 1 is

between Y t and Y t 1 , lag 2 is between Y t and Y t 2 .Time series can also be lagged forward, Y t

and Y t 1 .

The Corr(,) Yt Y t k (denoted by k ) is called an autocorrelation since it is a correlation between the time series with its own past and future values. ACF is the pattern of autocorrelations in a time series at numerous lags.

Definition 1.21: For a covariance stationary time series {}Y t the autocorrelation function k is given by :

k Corr(,) Yt Y t k for k 1, 2, 3, (1.29)

For the observed series YYY1,,, 2  n we have the sample or estimated autocorrelation

function rk at lag k is:

n  ()()YYYYt t k  t k 1 rk  n for k 1,2, (1.30) 2 ()YYt  t 1

ACF is a good indicator of the order of the MA(q) model since it cuts off after lag q (i.e., k = 0 for k  q ). On the other hand the ACF tails off for AR(p) model, (Cryer & Chan, 2008). Implementation in R: The function can be carried out using TSA or stats packages with R command: acf(x)

1.7.2 Partial Autocorrelation Function (PACF)

Since for AR(p) model the ACFs do not become zero after a certain number of lags, they die off rather than cut off; the different function is needed to help determine the order of AR(p) models. That function is called partial autocorrelation function. The partial autocorrelations are self-correlations with intermediate autocorrelations partialed out. There are several ways to make this definition precise. 13 Definition 1.22: (Cryer & Chan, 2008) If { Y t } is normally distributed time series, then the partial autocorrelation at lag k is given by

kk Corr( Yt , Y t k | Yt1 , Y t  2 , , Y t  k 1 ) (1.31) Also the partial autocorrelation function is given by

k 1 k   k1, j  k  j j 1 kk  k 1 (1.32) 1 k1, j  j j 1 where,

k, j  k 1, j   kk  k  1, k  j for j 1,2, , k  1

Example 1.7 : Using 11  1 to get started, we have

2 2  11  1  2   1 22   2 111  1 1   1

As before with 21  11  22  11 , which is needed for the next step. Then

3  21  2   22  1 33  121  1   22  2

We may thus calculate numerically as many values for kk as desired. By replacing  's with r' s , we can obtain the estimated or sample partial autocorrelations instead of theatrical partial autocorrelations. Implementation in R: The function can be carried out using TSA or stats packages with R command: pacf(x)

1.7.3 Extended autocorrelation function (EACF)

The ACF and PACF are not informative in determining the order of an ARMA model. The extended autocorrelation function (EACF) is used to specify the order of an ARMA process. The basic idea of EACF is relatively simple. If we can obtain a consistent estimate of the AR component of an ARMA model, then we can derive the MA component. From the derived MA series, we can use the ACF to identify the order of the MA component. Yet the function is easy to use. The output of the EACF is a two-way table, where the rows correspond to AR order p and the columns to MA order q. The theoretical version of the EACF for an ARMA(1,1) model is given in Table 1.1. The key feature of the table is that it contains a triangle of O’s with the upper left vertex located at the order (1,1). This is the characteristic we use to identify the order

14 of an ARMA process. In general, for an ARMA(p, q) model, the triangle of O’s will have its upper left vertex at the (p, q) position. Table 1.1 is constructed by using the following notation: 1. X denotes that the absolute value of the corresponding EACF is greater than or equal to 2 / n , which is twice the asymptotic standard error of the EACF. 2. O denotes that the corresponding EACF is less than2 / n in modulus.(Tsay, 2010)

Remark 1.3: If Y t is a white noise, then for large n, the sample autocorrelations ACFs are 1 approximately normally distributed with zero mean and var(  ) = . k n Implementation in R: The function can be carried out using TSA package with R command: eacf(x)

Table 1.1 Theoretical EACF Table for an ARMA(1,1) Model MA 0 1 2 3 4 5 6 7 8 9 10 11 12 13 0 X X X X X X X X X X X X X X 1 X 0 0 0 0 0 0 0 0 0 0 0 0 0 2 * X 0 0 0 0 0 0 0 0 0 0 0 0 AR 3 * * X 0 0 0 0 0 0 0 0 0 0 0 4 * * * X 0 0 0 0 0 0 0 0 0 0 5 * * * * X 0 0 0 0 0 0 0 0 0 6 * * * * * X 0 0 0 0 0 0 0 0 7 * * * * * * X 0 0 0 0 0 0 0

Table 1.2 indicates the general behavior of the ACF and PACF for ARMA models and some plots of ACFs and PACFs are illustrated in Table 1.3.

Table 1.2 Behavior of the ACF and PACF for ARMA Models AR(p) MA(q) ARMA(p, q)

k  0 . k  0 for k  q. That is, the spikes in the Autocorrelations die out ACF There are spikes until lag correlogram decay smoothly after q lags. q. slowly

kk  0 for k  p. kk  0 . PACFs die out smoothly There are spikes in until The spikes in the PACF after p lags. lag p. correlogram decay slowly

15 Table 1.3 ACFs and PACFs Plots ACF PACF

AR(1) 0.8 0.8 0.6 0.6 0.4 0.4 ACF 0.2 PartialACF 0.2 0.0 0.0 -0.2 -0.2

2 4 6 8 10 12 14 16 2 4 6 8 10 12 14 16 Lag Lag   0 for k  1, (i.e., PACF "cuts off" k  0 , (i.e., ACF "tails off"), that is, kk the spikes in the correlogram decay at lag 1), so the model is AR(1) slowly. AR(3) 0.6 0.6 0.4 0.4 0.2 0.2 ACF 0.0 PartialACF 0.0 -0.2 -0.2 -0.4 -0.4 -0.6

2 4 6 8 10 12 14 2 4 6 8 10 12 14 Lag Lag

k  0 , ACF does look somewhat like kk  0 for k  3, (i.e., PACF "cuts off" at lag 3), so the model is AR(3) the damped wave, that is, the spikes in

the correlogram decay slowly. MA(1) 0.4 0.4 0.3 0.2 0.2 0.1 ACF 0.0 PartialACF 0.0 -0.2 -0.1 -0.2

2 4 6 8 10 12 14 16 18 20 2 4 6 8 10 12 14 16 18 20 Lag Lag

k  0 for k  1, (i.e., ACF "cuts off" at kk  0 .(i.e., PACF "tails off"), The lag 3), so the model is MA(1) spikes die out exponentially.

16 ARMA 0.6 (p,q) 0.6 0.4

0.4 ACF 0.2 0.2 Partial ACF 0.0 0.0 -0.2 -0.2

2 4 6 8 10 12 14 16 18 20 2 4 6 8 10 12 14 16 18 20 Lag Lag ACFs die out smoothly after q lags. PACFs die out smoothly after p lags.

The autocorrelation function and partial autocorrelation function play major role in time series model diagnostics, many tests are proposed basing on these functions; which are called portmanteau tests. Definition 1.23: (Katayama, 2009) Portmanteau test is known as a goodness-of-fit test statistic and is defined by the sum of squares of the first m residual autocorrelations. Among portmanteau tests are both the Ljung–Box test and the Box–Pierce test. The portmanteau test is useful in working with ARMA models.

1.8 Summary

In this chapter we have introduced the problems and the objectives of the study and several major literature reviews in the field of model diagnostics in ARMA model. Then we illustrated basic time series terminologies that serve this study.

17

Chapter 2

Model Diagnostic

2.1 Introduction

After identification and estimation stages in time series model building, the proposed model must be tested to know how does it fit the data? Are the values of observations predicted from the model close to actual ones? This is called model diagnostics so we shall perform analysis for the residuals of the fitted model with some illustrative examples on both simulated data and real data.

2.2 Residual Analysis

Residual analysis is a key part of all statistical modeling. If the model is good, the residuals (differences between actual and predicted values) of the model are a series of random errors. We are interested in finding whether the residuals of our model are nearly white noise, they expected to be roughly normal and uncorrelated with common variance and zero mean. We will test our hypotheses at the usual significance levels (95%). Now we will discuss the homogeneity of variance with zero mean, normality of residuals distribution and the autocorrelation of residuals.

2.2.1 Homogeneity of Variance and Zero Mean

After the model is proposed, our first diagnostic check is to examine plots of standardized residuals to assess homogeneity of variance over time. If the model is an adequate one, then the plot seems to be a rectangular scatter around a zero horizontal level with out any trends as in Figure 2.1.

18 2 1 0 -1 Standardized Residuals Standardized -2 -3

0 50 100 150 200 250 300 350 Time

Figure 2.1 Standardized Residuals over the Time

2.2.2 Normality of Residuals Distribution

For inspecting the normality of residuals distribution, we test the null hypothesis of residuals normality. We can use graphical methods such as the histogram plot and quantile- quantile (QQ) plot. One can use such quantitative test as Shapiro–Wilk test. We described the two mentioned graphical methods and Shapiro–Wilk test with an illustrative examples. Histogram plot: An informal method to testing normality is the histogram plot of the residuals or standardized residuals. The plot of the histogram should be bell-shaped as in Figure 2.2 and resemble the normal distribution so as not to reject the null hypothesis. This might be difficult to see if the sample is small so the histogram is not be the best choice for judging the distribution of residuals. The most sensitive graph is the QQ plot.

100

80

60

Frequency 40

20

0

-3 -2 -1 0 1 2 3

Residuals

Figure 2.2 Histogram Plot for Residuals

Implementation in R: The function can be carried out using graghics packages with R command: hist(x)

19 Quantile-Quantile plot (QQ plot): A common graphical tool for assessing the normality is the QQ plot of the quantiles of the data against the theoretical quantiles of a normal distribution. For normal residuals the points plotted in the QQ plot should fall approximately on a straight line. Figure 2.3 shows a largely straight line pattern.

3

2

1

0

SampleQuantiles -1

-2

-3

-3 -2 -1 0 1 2 3

Theoretical Quantiles

Figure 2.3 Normal QQ plot for Residuals

Shapiro–Wilk test: Finally, we want a formal test of agreement with normality or not; such that test is Shapiro – Wilk test. Recalling the null hypothesis; if the p-value is less than the chosen alpha level, then the null hypothesis is rejected. But if the p-value is greater than the chosen alpha level, then one does not reject the null hypothesis that the residuals are normally distributed. Implementation in R: The function can be carried out using stats packages with R command: Shapiro.test(x)

2.2.3 Autocorrelation of Residuals

During the diagnostic phase we test the null hypothesis which is the residuals autocorrelations are uncorrelated ( k  0 ). Once the model is developed and residuals are computed for each lags individually, there should be no remaining autocorrelations or partial autocorrelations at various lags in the ACFs to have no evidence of autocorrelation in the residuals of that model (not rejecting the null hypothesis). In Figure 2.4 all the values are within the horizontal dashed lines which are placed at ± approximately two standard errors of the sample autocorrelations (  2 / n ).

20 0.3 0.2 0.1 0.0 ACF -0.1 -0.2 -0.3

2 4 6 8 10 12 14

Lag

Figure 2.4 Sample ACF of the Residuals

Now we will perform a residual analysis for the following examples to inspect the assumptions, Example 2.1 for a simulation time series and Example 2.2 for a real time series.

Example 2.1 An MA(1) model with coefficient equal to 1  0.9 and of length n=120 is a data which is available in R software, in TSA package, (Chan & Ripley, 2012). For performing residual analysis for this model we will achieve the three assumptions.

1. Checking a homogeneity of variance and zero mean: Figure 2.5 shows a plot of standardized residuals over time, this plot suggests a rectangular scatter with no trends whatsoever. So it supports the model.

3 2 1 0 -1 Standardized Residuals Standardized -2

0 20 40 60 80 100 120 Time

Figure 2.5 Standardized Residuals from MA(1) Model

21 2. Checking a normality of residuals distribution: Histogram plot: Figure 2.6 displays a frequency histogram of the residuals from MA(1) model. The plot is somewhat bell shaped but certainly not ideal. Perhaps a QQ plot will tell us more.

Histogram of res1 25 20 15 Frequency 10 5 0

-4 -2 0 2 4 Reseduals

Figure 2.6 Histogram for the Residuals from MA(1) Model

QQ-plot: Figure 2.7 form a reasonably linear pattern in the center of the data. However, the tails show a departure from the fitted line, so we have some suspicions about the normal distribution. So we will remove our suspicions about that by Shapiro-Wilk test.

Normal Q-Q Plot 3 2 1 0 Sample Quantiles Sample -1 -2

-2 -1 0 1 2

Theoretical Quantiles

Figure 2.7 QQ-plot for the Residuals from MA(1) Model

22 Shapiro-Wilk test: Output 2.1 shows that the p-value is greater than 0.05, hence we cannot reject the null hypothesis that is the residuals are normally distributed. Shapiro-Wilk normality test data: res1 W = 0.9888, p-value = 0.4362

Output 2.1 Shapiro-Wilk Test for Residuals of MA(1) Model

3. Checking the autocorrelation of the residuals: Figure 2.8 displays ACF of the residuals from MA(1) model and according to that non of the hypotheses k  0 can be rejected, and it is a reasonable to confirm the white noise for the residuals of this series.

Series res1 0.15 0.10 0.05 0.00 ACF -0.05 -0.10 -0.15

5 10 15 20 Lag

Figure 2.8 Sample Autocorrelation of Residuals from MA(1) Model

Conclusion: The last residual analysis provides an adequate fit for this model.

Example 2.2: Data (NHtemp) is a time series contains the mean annual temperature in degrees Fahrenheit in New Haven, from 1912 to 1971. This series is AR(2) model, the same steps will performed as in Example 2.1.

1. Homogeneity of variance and zero mean: Figure 2.9 indicates sort of rectangular shape, since there are some variation at the beginning of the series and around 1950.

23 2 1 0 -1 Standardized Residuals Standardized -2

1910 1920 1930 1940 1950 1960 1970

Time

Figure 2.9 Standardized Residuals for AR(2) Model from NHtemp Data

2. Normality of residuals distribution: Histogram plot: Figure 2.10 indicates a symmetric, moderate tailed distribution, that doesn’t dying off out in the tails. So the recommended next step is to do a QQ plot.

Histogram of res2 20 15 10 Frequency 5 0

-6 -4 -2 0 2 4 6 Reseduals

Figure 2.10 Histogram for the Residuals for AR(2) Model from NHtemp Data

QQ-plot: The points on Figure 2.11 appear in the form of a straight line sort of and we can consider the residuals are approximately normally distributed, we can confirm that by Shapiro- Wilk test.

24 Normal Q-Q Plot 3 2 1 0 SampleQuantiles -1 -2 -3

-2 -1 0 1 2

Theoretical Quantiles

Figure 2.11 QQ-plot for the Residuals for AR(2) Model from NHtemp Data

Shapiro-Wilk test : Output 2.2 shows that the p-value is greater than 0.05, so the test verifies that an assumption of normality is in fact reasonable.

Shapiro-Wilk normality test data: res2 W = 0.9907, p-value = 0.928

Output 2.2 Shapiro-Wilk Test for Residuals of AR(2) Model from (NHtemp) data

3. Autocorrelation of the residuals: From the figure 2.12 there is no evidence of autocorrelation in the residuals of the AR(2) model.

Series res2 0.2 0.1 0.0 ACF -0.1 -0.2

5 10 15 Lag

Figure 2.12 Sample ACF of Residuals for AR(2) Model from NHtemp Data

25 Conclusion: From the last residual analysis we can conclude that the AR(2) model is a good model for this time series data.

2.3 Summary

This chapter is concerned with model diagnostic phase in ARMA model building. We looked at various plots of residuals, checking the error terms for constant variance with zero mean, normality and the uncorrelatedness, in the next chapter we will gather these diagnostic tools in one test statistic. So we consider this chapter as a beginning of our study of residual analysis to investigate the quality of the fitted model.

26

Chapter 3

Portmanteau Tests For ARMA Models

3.1 Introduction

Residual analysis discussed in Chapter 2, they looked at residual correlations at individual lags, so it is useful to have a tests that take into account their magnitudes as a group. In this chapter we will present Box-Pierce (1970), Ljung-Box (1978), Monti(1994), Peña and Rodríguez (2002, 2006), Mahdi and McLeod "Gvtest'' (2012) and Fisher (2011) portmanteau tests for univariate ARMA time series model extensively for this purpose. The null hypothesis assuming that the fitted model is an adequate model and the residuals behave like white noise series. Most of portmanteau tests are based on the residual autocorrelation coefficients which is provided by:

n ˆ ˆ  t  t k t k 1 rˆk n , k  1,2,... (3.1) ˆ2 t t 1 where ˆ1,,  ˆt are the residuals obtained after estimating the model in a sample of size n. (Peña & Rodríguez, 2006)

3.2 Portmanteau Tests

3.2.1 Box and Pierce Portmanteau Test

The classical portmanteau test statistic is the one proposed by Box and Pierce (1970).

 Definition 3.1: (Box & Pierce, 1970) Box-Pierce(Q BP ) test statistic up to lag m is

m  ˆ ˆ2 QBP() r n r k (3.2) k 1

27 where rˆk is the sample autocorrelation of order k of the residual, m is the number of lags being tested and n is the sample size.

Properties: Asymptotic distribution: This statistic is used to test for significant correlation up to lag m . It is well known that for independent and identically distributed data, as n   the autocorrelations behave as independent normally distributed random variables, and therefore  under the null hypothesis (correctly fitted model) Q BP is shown to be asymptotically distributed  2 random variable with m() p  q degrees of freedom, where p and q are the order of autoregressive and moving average terms estimated in the fitted model, respectively.

Critical region: For significance level  , the critical region for rejection the null hypothesis is

 2 QBP 1 , m  p  q .

Limitation: This classical statistic has been widely studied. Kwan et al., (2005) indicated that the normalization procedure used in Box-Pierce test is inappropriate for an independent and identically distributed (iid) normal series with an unknown mean. Consequently, the poor empirical performance of the test is not entirely unexpected. On other hand, Arranz, (2005) showed that in finite samples its distribution falls apart from the asymptotic one. This fact has been the starting point of some modifications

Implementation in R: This test statistic is implemented in the R using portes package with function BoxPierce(),the usage of this function is extremely simple: BoxPierce(obj,order =p+q).

Remark 3.1: The argument 'order' is used for degrees of freedom of asymptotic chi-square distribution, so it is important to enter the value of order equals to p + q, in order to calculate the value of  2 with (m ( p  q )) degrees of freedom, otherwise the function will calculate the value of  2 with ()m degrees of freedom when order =0 or when we use the function without the argument 'order'(equivalently using BoxPierce(obj)).

Example 3.1: In this example, we fit an ARIMA(1,1,1) model to measurements of the annual flow of the river Nile at Aswan from the year 1871 to1970. The Ljung-Pierce portmanteau test is applied on the residuals of the values of m= 5, 10, 15, 20, 25, and 30. Output 3.1 yields that the assumption of the adequacy in the model is not rejected, since p-values are greater than the chosen alpha level 0.05.

28

Box-Pierce test Lags Statistic df p-value 5 1.187848 3 0.7559203 10 8.731657 8 0.3654364 15 10.196602 13 0.6777839 20 11.372622 18 0.8778806 25 12.552514 23 0.9610361 30 14.624301 28 0.9821334

Output 3.1 Box-Pierce Test

3.2.2 Ljung-Box Portmanteau Test

After some discussions about the finite sample distribution of the test statistic proposed by Box and Pierce (1970); Ljung and Box (1978) proposed a modified version of that test.

 Definition 3.2: (Ljung & Box, 1978) Ljung-Box Q LB portmanteau test is

m rˆ2  ˆ k QLB ( r ) n ( n  2) (3.3) k 1 n k where n,, m rˆk as in Definition 3.1.

Properties: 1. Ljung & Box, (1978) showed that their test provides a substantially improved

2 approximation to m p  q distribution that should be adequate for most practical purposes with  the same critical region as Q BP . 2. In many applications the value of m has been as high as 20 or 30 even when a simple low- order model has been believed to be appropriate.(Ljung, 1986)

 2 3. The variance of Q LB increases relative to var ( m p  q ) for larger m. The increase in the

2 variance results from a positive correlation among the rˆk ' s at higher lags for finite n. This can  be seen by examining var (Q LB ) for a white noise process and by noting that the residual autocorrelations at higher lags behave like white noise autocorrelations, (Ljung, 1986).

Implementation in R: The Ljung-Box portmanteau test is implemented in R using portes package with function LjungBox(),the usage of this function is LjungBox(obj, order = p+q).

29 Remarks 3.2: 1. We must deal with argument 'order' as mentioned in Box-Pierce test in Remark 3.1.  2. In R, the function Box.test()using stats package was built to compute the Q BP and  Q LB tests only in the univariate case where we can not use more than one single lag value at the same time. The functions BoxPierce()and LjungBox()are more general than Box.test()and can be used in the univariate or multivariate time series at vector of different lag values. 3. One can uses tsdiag function using stats package. This is a generic function. It will generally plot the residuals, often standardized, the autocorrelation function of the residuals, and the p-values of Ljung-Box test for the maximum number of lags for the test, but we must be careful when performing this test because the Chi square will be computed for m degree of freedom not for m-(p+q).

Example 3.2: A data of the age of death of successive Kings of England, started with William the Conqueror is used in this example, (Hyndman, 2008). ARMA((0,1,1) model is fitted for the data, the Ljung-Box portmanteau test is applied on the residuals on the fitted model. Output3.2 yields that portmanteau test suggests the ARMA(0,1,1) as an adequate model since p-values are grater than 0.05 for all lags.

Ljung-Box test Lags Statistic df p-value 5 2.469197 4 0.6501599 10 4.819358 9 0.8497623 15 10.804383 14 0.7013304 20 13.584395 19 0.8073577 25 17.876582 24 0.8089540 30 19.769290 29 0.8999539

Output 3.2 Ljung-Box Test

Figure 3.1 shows the three diagnostic tools in one display, a sequence plot of the standardized residuals, the sample ACF of the residuals, and p-values for the Ljung-Box test for a whole range of the values of m, the estimated model seems to be adequate fit.

30 Standardized Residuals 2 1 0 -2

0 10 20 30 40

Time

ACF of Residuals 1.0 0.4 ACF -0.2

0 5 10 15

Lag

p values for Ljung-Box statistic 0.8 0.4 p p value 0.0

0 5 10 15 20 25 30 lag

Figure 3.1 Diagnostic Display for the ARMA(0,1,1) Model of Kings Series

3.2.3 Monti Portmanteau Test

The tests in Definitions 3.1 and 3.2 are based on residual autocorrelation; alternatively Monti (1994) proposed a test statistic based on residual partial autocorrelation.

Definition 3.3: (Monti, 1994) The k th residual partial autocorrelation is the correlation between the residuals of the regressions of ˆt and ˆt k on the intervening ˆ's :  ˆt1 ,...,  ˆ t  k  1 , and is denoted ˆk . We can compute it from Equation (1.32) by replacing k with rˆk can be computed from Equation (3.1), (see Example 1.7).

th Definition 3.4: (Monti, 1994) Let ˆk be the k residual partial autocorrelation, then Monti portmanteau test up to lag m is provided by: m ˆ 2  ˆ k QM ( ) n ( n  2) (3.4) k 1 n k where n is the length of the time series.

31 Properties:  1. Monti(1994) proved that if the model is correctly identified, QM is asymptotically

2 distributed as a m p  q random variable.

2. If the model is correctly specified, ˆk is approximately distributed as normal with zero n k mean and variance . (Kwan & Wu,1997) n n  2 3. One can alternatively uses the statistic

m *ˆ ˆ 2 QM() n  k (3.5) k 1

*  *  where QM is asymptotically equivalent toQM . The difference between QM and QM is the   same as between the Box-Pierce test Q BP and Ljng-Box test Q LB . In both cases, the approximation to the small-sample distribution by a  2 is more accurate for the latter versions, which is therefore recommended.(Monti, 1994)  4. Monti (1994) showed by simulations that the performance of QM is comparable to that of  Q LB and better if the order of the moving average is underestimated. On the other hand,  Q LB performs better if the order of the autoregressive part is underestimated.

Implementation in R: The Monti test is implemented in R using WeightedPortTest package with function Weighted.Bx.test(),the usage of this function is Weighted.Box.test(x, lag = value of m , fitdf = p+q ,type = "Monti",weighted = FALSE).

Example 3.3 (Continued to example 2.1) Monti test is applied on the residual of MA(1) model of lags 5 and 15 according to the results in Output 3.3, the Monti test clearly suggests that the MA(1) model at  0.05 since p-values greater than 0.05. Monti test, m = 5 data: res.ma1 X-squared on Residuals for fitted ARMA process = 1.9419, df = 4, p-value = 0.7465 Monti test, m = 15 X-squared on Residuals for fitted ARMA process = 11.5738, df = 14, p-value = 0.640

Output 3.3 Monti Test for m = 5 , 15 32 3.2.4 Peña and Rodríguez Portmanteau Test (2002)

The estimated residuals can be considered as a sample of multivariate data from some distribution, Peña & Rodríguez (2002) interested in testing the adequacy for the ARMA models on a statistic based on the determinant of the residual autocorrelation matrix:

1 rˆ1  r ˆm    rˆ1  r ˆ Rˆ  1m  1  (3.6) m        rˆm  r ˆ1 1  where rˆk is the sample autocorrelation of order k of the residual. So under the null hypothesis; ˆ testing for model adequacy is equivalent to testing if Rm is approximately the identity matrix. Thus, it is sensible to explore a test based on this statistic.

Definition 3.5: (Peña & Rodríguez, 2002) For stationary time series data a portmanteau diagnostic test statistic up to the lag m is

1 m D n1  Rˆ  m  (3.7) where n is the length of the time series.

Properties: 1. Peña and Rodríguez (2002) showed that if the model is correctly identified D is asymptotically distributed as a linear combination of chi-squared random variables and is approximately a Gamma distributed random variable for large values of m with parameters  and  , where 1 f( x ; ,  ) x 1 exp(  x  ) (3.8) ()   where,

2 3m m 1  2 p  q       (3.9) 2 2m 1 2 m  1  12 m p  q   and

3m m 1  2 p  q       (3.10) 2m 1 2 m  1  12 m p  q 

33 ˆ 2. In practice, they recommend the matrix R m be constructed using the standardized

residuals as this improves the Gamma distribution approximation by replacing rˆk with rk ,

1 2  ˆ ˆ  where rk n 2  n  k  r k .Hence R m will replaced with R m ,(Lin & McLeod, 2006).

Limitations: 1. As pointed out in Lin and McLeod (2006), the statistic D constructed using the  standardized residuals is frequently does not exist in practice since the matrix Rm is not always positive definite, so they recommended to concentrate on the original D statistic. 2. Lin and McLeod (2006) showed that the test based on the gamma approximation is not conservative, despite the fact that as shown by Peña and Rodríguez (2002) the small sample performance is acceptable in some cases, the more general use of tests based on the gamma approximation cannot be recommended. 3. D test statistic may be difficult to implement since it involves calculating the determinant of a matrix (Fisher, 2011).  4. For finite sample size, Monti test QM is always better than D test, (Peña & Rodríguez, 2006). In order to improve the properties of D test, Peña and Rodríguez proposed a new test statistic for diagnostics.

3.2.5 Peña and Rodríguez Portmanteau Test (2006)

A finite sample modification of a test by Peña and Rodríguez is proposed. The new modified test has a more intuitive explanation than the previous one.

Definition 3.6: (Peña & Rodríguez, 2006) For stationary time series data a new portmanteau diagnostic test statistic is n DR*   log ˆ (3.11) m 1 m the notation as outlined in Definition 3.4.

Properties: 1. Peña and Rodríguez (2006), showed that the test statistic D * is asymptotically distributed as a linear combination of Chi-squared random variables and proposed two different approximations to the asymptotic distribution of that test statistic:

34 - The first one is based on the Gamma distribution. The test statistic follows asymptotically a Gamma distribution with parameters and  , where

2 3m 1  m  2 p  q       (3.12) 2 2m 2 m 1  12 m  1 p  q   and

3m 1  m  2 p  q     (3.13) 2m 2 m 1  12 m  1 p  q  m and the distribution has mean  1   p  q  and variance 2 m2 m  1  2  2 p  q  .They denoted this first approximation by GD  which is 3m  1 distributed as G (,)  . - The second approximation is based on Normal distribution. Peña and Rodríguez, (2006) suggested a power transformation which reduces the skewness in order to improve the normal approximation. The test statistic is

1  1  1 1  1  1    NDD 1 2      1 1        2    (3.14) 2     and

1 2m p  q m2 4 m  1  p  q   2          1  2 (3.15) 3m 2 m 1 6 m  1  p  q           where for m moderately large we get   4 and and  are the values as shown in

Equation (3.12) and Equation (3.13) respectively. The statistic ND  is the second approximation which is distributed as a N (0,1) . 2. Peña and Rodríguez, (2006) found that the performance of both approximations, GD  or ND  for checking for goodness of fit in linear models is similar. 3. By Monte Carlo study Peña and Rodríguez (2006), showed that the new test can be up to 50% more powerful than the Ljung-Box and Monti tests, and for finite sample size is always better than previous one, D . So there will no be an implementation for Peña and Rodríguez (2002) test in R as long as the same statistician developed it to D * . 4. The new tests do not seem to be affected by the value of m . The statistics D and Ljung-  Box, Q LB , have a good size performance but they are much more sensitive to the value of m (Peña & Rodríguez, 2006).

35 5. Their simulations showed an improvement in small sample time series, but the Type I error rates appear to be poor. So Mahdi and McLeod (2012) improved D * test such that the degrees of freedom for the  2 approximation allow the improved one to have conservative Type I errors in practice, ( Fisher & Gallagher, 2012). Mahdi and McLeod (2012) generalized the results of Peña and Rodríguez (2002, 2006) to the multivariate setting. In the univariate case, they recommended the statistic 3n DR*   log ˆ (3.16) m2m  1 m

The null distribution is approximately  2 with (1.5m ( m 1)(2 m  1)1  p  q ) degrees of freedom and it is implemented in the R function gvtest(), so this modified statistic is called as Gvtest, since it is a generalized variance portmanteau test based on the determinant matrix, Hence we will not include Peña and Rodríguez tests in our comparison because of their generalization by Mahdi and McLeod (2012).

* Implementation in R: Gvtest Dm statistic is implemented in R using portes package with gvtest()function with output which are the generalized variance portmanteau test statistic and its associated p-values for different lags based on asymptotic Chi square distribution as given in Mahdi and McLeod (2012). In the univariate time series, the usage of this function is gvtest(obj, order=(p+q)).

Example 3.4: The portmanteau test is applied on the same data in Example 3.1 using

* gvtest()function. Output 3.4 displays results of Gvtest Dm which yields the adequacy of the fitted model since p-values are greater than 0.05.

Gvtest Lags Statistic df p-value 5 0.7785927 2.090909 0.6984832 10 3.5238228 5.857143 0.7252698 15 7.3204983 9.612903 0.6610836 20 9.9992640 13.36585 0.7201647 25 12.148675 17.11764 0.7972129 30 14.122429 20.86885 0.8594566

Output 3.4 Gvtest

36 3.2.6 Fisher Portmanteau Tests

Fisher, (2011) introduced two new statistics that are weighted variations of the common Ljung-Box and, the less-common, Monti statistics and these two new statistics are easy to implemented.

Definition 3.7: (Fisher, 2011) Weighted portmanteau tests are provided by m m k 1 rˆ2  ˆ k QWL ( r ) n ( n  2) (3.17) k 1 m n k and m m k 1 ˆ 2  ˆ k QWM ( ) n ( n  2) (3.18) k 1 m n k where rˆk andˆk as in Definitions 3.1 and 3.3 respectively.

Properties: (Fisher, 2011) 1. The two statistics look similar to the Ljung-Box and Monti statistics with the exception a m k 1 weight, on each autocorrelation or partial autocorrelation. The weights are derived m using multivariate analysis techniques on the matrix of autocorrelations or matrix of partial autocorrelations. m 2. The sample autocorrelation at lag 1, rˆ , is given weight 1. The sample 1 m m 1 autocorrelation at lag 2, rˆ , is given weight 1 . We can interpret the weights as putting 2 m more emphasis on the first autocorrelation, and the least emphasis on the autocorrelation at lag 1 m (corresponding weight ). This matches the intuition about statistical estimators. The m first autocorrelation rˆ1 is calculated using information from all n observations. The second

th autocorrelation rˆ2 is based on n 1 observations, and the m autocorrelation is based on n m observations. Intuitively, it makes sense to put more emphasis on the first autocorrelation as it should be the most accurate. This idea also holds true for the partial autocorrelations. 3. The two statistics are asymptotically distributed as a linear combination of Chi-squared random variables. This is the same asymptotic distribution as the statistics in Peña and   Rodríguez (2002, 2006). The weighted Ljung-Box QWL and weighted Monti, QWM , statistics are asymptotically equivalent to D but have the added benefit of easy calculation and

37 computational stability. When a small number of parameters have been fit under the null   hypothesis of an adequate model, the statistics QWL and QWM are approximately distributed as Gamma random variables with shape parameter

2 2 3m m  2 m  1 p  q     (3.19) 3 2 2  4 2m 3 m  m  6 m  2 m  1 p  q   and scale parameter

3 2 2  2 2m 3 m  m  6 m  2 m  1 p  q     (3.20) 2  3m m m  2 m  1 p  q   The Gamma approximation is constructed to have the same theoretical mean and variance as the true asymptotic distribution.

Implementation in R: Two new weighted portmanteau tests are implemented in R using WeightedPortTest package with function Weighted.Bx.test(),the usage of this function is Weighted.Box.test(x, lag = value of m, fitdf = p+q, type = c("Ljung-Box", "Monti"), weighted = TRUE).

 Example 3.5: The weighted Box QWL portmanteau test is applied on Annual measurements of the level, in feet, of Lake Huron series from 1875 to1972, where the ARMA(1,1) model is fitted, the approximation asymptotic distribution of the statistics in Output 3.5 suggests that the ARMA(1,1) model as an adequate model since p-values are grater than 0.05.

Weighted Ljung-Box test (Gamma Approximation), m = 5

data: resLH Weighted X-squared on Residuals for fitted ARMA process = 1.3347, Shape = 8.176, Scale = 0.673, p-value = 0.9992 ------Weighted Ljung-Box test (Gamma Approximation), m = 10

data: resLH Weighted X-squared on Residuals for fitted ARMA process = 3.9811, Shape =10.652 , Scale = 0.986, p-value = 0.9956

Output 3.5 Weighted Box Test for m = 10, 15

38  Example 3.6: We will apply the weighted Monti portmanteau test QWM on simulated time series data with AR(2) model, and it is clear from Output 3.6 that we cannot reject the adequate of the AR(2) model since p-values are greater than 0.05. Weighted Monti test (Gamma Approximation), m = 7

data: res.ar2 Weighted X-squared on Residuals for fitted ARMA process = 2.1677, Shape = 9.333, Scale = 0.429, p-value = 0.9434

Weighted Monti test (Gamma Approximation), m = 13

data: resLH Weighted X-squared on Residuals for fitted ARMA process = 4.0566, Shape = 8.608, Scale = 0.813, p-value = 0.9113

Output 3.6 Weighted Monti Test for m = 7, 13

3.3 Summary

Autocorrelation of the residuals play important role in model diagnostics using portmanteau tests. In this chapter we introduced several portmanteau tests with there properties and there implementations in R statistical program. Some examples were illustrated.

39

Chapter 4

Simulation and Case Study

4.1 Introduction

In this section, we consider the robustness of various portmanteau tests. We will compare the    power among some of the portmanteau tests such as Box-Pierce QBP , Ljung-Box QLB , Monti QM ,

*   Gvtest D m , Weighted Ljung-Box QWL , and Weighted Monti QWM . These simulations examine the sensitivity of the selected portmanteau tests to model diagnostics. In particular, which is the most appropriate test for examining the adequacy of linear and nonseasonal ARMA models. Firstly we will present the results of simulation study, then the results of comparisons on electricity consumption time series data in Khan Younis city.

4.2 Simulation Study Three finite sample sizes (50, 200, and 500) are generated from different ARMA(p,q) models with different values of the model parameters, where p , q ≤ 2 and AR(1) is fitted for each model. In each case 1000 of Monte-Carlo simulations with 1000 replications were generated by R statistical software package and the powers of the tests were computed for selected lags m for 0.05 nominal level.

4.2.1 Simulation Study on Short Term Data

We generate small sample size (N=50) by R statistical software and the power of the tests are computed for m = 5, 10, 15, 20. The Results are presented in Table 4.1 for m = 5 and 20. The other choices are in ,Appendix A, Table A.1.

The simulation results reveal that all portmanteau tests are sensitive to the choice of lag m and reach its maximum at lag m = 5. For an example, in model 3, MA(1), there are deficiencies of  the power of the portmanteau test QWM when we go from lags 5 to lag 10 which equals to 40 0.293 0.245 100  16.3% (see Table 4.1 and Appendix A, Table A.1). The other models 0.293 have the same calculation. Specifically, the averages of the power lacks with respect to m from

   * lag 5 to lag 10 are 21.2%, 21.1% ,19.8%, 13.3%, 18.6% and 15.1% for QBP , QLB , QM , Dm ,   QWL and QWM , respectively with exception in model 5.

 For AR(p) and MA(q), the simulation results show that the performance of QM is better than  QLB if the order of the moving average component is underestimated (see models 1, 2, 3, 6 and   7 for all selected lags). On the other hand, QLB performs better than QM if the order of the autoregressive component is underestimated (see models 4 and 5 for all lags).

To further investigate the power of the portmanteau tests, the simulation results indicate that  the power of the test is increased -on average- by 23.6% when using QWL , which is proposed  by Fisher (2011), instead of QLB . In addition, the power of the test is increased -on average-   by 25.0% when using QWM instead of QM . Based on the simulation results, the difference of

*  the power test between Dm and QWM is very small (in the average about 0.009 only) and is

*  smaller than the difference of the power test between Dm and QWL tests (about 0.05), with exception in model 5.

* Furthermore, the simulation results reveal that Gvtest ( D m ), is the most powerful test for the  most selected models for large lags (m = 15 and 20) , whereas Weighted Ljung-Box (QWL ) and

 * Weighted Monti QWM outperform and more powerful than Dm for moderate lags (m =5 and 10).

* The test Dm seems to be sensitive for model 5, AR (2) with parameters 1.2 and -0.73 for all

* lags in small data (N=50). For large samples, we will notice later that Dm will behave similar as the other tests. In Table 4.1 the test statistics with the highest power for any particular model are shown in bold font.

41 Table 4.1 Powers of Portmanteau Tests for N = 50,   0.05 m=5 Model ϕ ϕ θ θ    D *   1 2 1 2 QBP QLB QM m QWL QWM 1 0.7 0.428 0.417 0.548 0.685 0.571 0.682 2 0.4 0.140 0.136 0.138 0.196 0.172 0.193 3 -0.5 0.200 0.197 0.209 0.294 0.260 0.293 4 0.6 0.3 0.252 0.244 0.231 0.300 0.325 0.300 5 1.2 -0.73 0.997 0.997 0.995 0.143 0.999 0.999 6 1 -0.6 0.726 0.712 0.755 0.816 0.817 0.816 7 0.24 0.1 0.117 0.115 0.127 0.177 0.154 0.163 8 0.8 0.4 0.095 0.088 0.093 0.116 0.114 0.115 9 0.5 -0.7 0.700 0.676 0.879 0.944 0.864 0.943 10 -0.2 -0.6 0.186 0.183 0.193 0.293 0.233 0.296 11 0.7 0.2 -0.5 0.234 0.210 0.272 0.406 0.336 0.402 12 1.3 -0.35 0.1 0.287 0.295 0.276 0.414 0.406 0.404 13 0.4 -0.6 0.3 0.580 0.530 0.821 0.881 0.766 0.882 14 0.9 -0.3 1.3 -0.5 0.172 0.155 0.201 0.281 0.237 0.271 m=20 1 0.7 0.274 0.241 0.327 0.525 0.319 0.490 2 0.4 0.112 0.105 0.106 0.137 0.125 0.122 3 -0.5 0.157 0.149 0.117 0.205 0.179 0.187 4 0.6 0.3 0.156 0.142 0.117 0.204 0.190 0.185 5 1.2 -0.73 0.974 0.943 0.930 0.116 0.986 0.988 6 1 -0.6 0.496 0.441 0.484 0.716 0.622 0.696 7 0.24 0.1 0.096 0.085 0.086 0.120 0.106 0.108 8 0.8 0.4 0.071 0.074 0.067 0.076 0.070 0.071 9 0.5 -0.7 0.444 0.382 0.596 0.859 0.655 0.829 10 -0.2 -0.6 0.133 0.116 0.108 0.197 0.146 0.154 11 0.7 0.2 -0.5 0.167 0.142 0.139 0.272 0.186 0.238 12 1.3 -0.35 0.1 0.212 0.202 0.168 0.276 0.250 0.250 13 0.4 -0.6 0.3 0.353 0.310 0.529 0.792 0.442 0.769 14 0.9 -0.3 1.3 -0.5 0.133 0.102 0.100 0.175 0.124 0.149

4.2.2 Simulation Study on moderate Term Data

We consider a moderate sample size N=200 is generated by R statistical software. Table 4.2 presents the results for m = 5, 15 for the portmanteau tests. The other choices of m shown in, Appendix A, Table A.2. The simulation study shows that the power of the portmanteau tests is more powerful for N=200 than N = 50, on the contrary the decrease in power of the portmanteau tests when we move from m = 5 to m = 10 are as in the average of 13.3%, 14% ,11.2%, 4.9%, 6.7% and 5.3%

   *   for QBP , QLB , QM , Dm , QWL and QWM respectively with some exception in some models (see models 5, 6, 9 and 13), where all tests have the same high power for most of lags which about 100%.   It can be seen in Tables 4.2 QLB tends to perform better than QM when the fitted model underestimates the order of the autoregressive component (see model 4 for all lags in Table 4.2 & Table A.2). Conversely if the fitted model underestimates the order of the moving average   component then QM does better than QLB (see models 1, 2, 3, 6 and 7 for all lags in Table A.2 42  additional to the Table 4.2). The results reveals that Weighted Ljung-Box QWL test raises the   power of the classical one QLB test by 25.0% and Weighted Monti QWM test raises the power  of Monti QM test by 25.1% with some exceptions in models 5,6,9 and 13 as mentioned before all tests have the same high power. We notice that the highest power of the tests for other

*   models varies between Dm , QWL and QWM for lags 5 and 10 such that the differences in power between these tests are small and in the average of 0.008 only, but in general for later lags

* Dm is the most powerful test. The test statistic with the highest power for any particular model is in bold font.

Table 4.2 Powers of Portmanteau Tests for N = 200,   0.05 m=5 Model ϕ ϕ θ θ    *   1 2 1 2 QBP QLB QM Dm QWL QWM 1 0.7 0.997 0.997 0.999 1 1 0.999 2 0.4 0.356 0.349 0.381 0.497 0.477 0.497 3 -0.5 0.666 0.665 0.721 0.840 0.812 0.839 4 0.6 0.3 0.920 0.919 0.914 0.956 0.962 0.956 5 1.2 -0.73 1 1 1 1 1 1 6 1 -0.6 1 1 1 1 1 1 7 0.24 0.1 0.373 0.372 0.385 0.503 0.492 0.509 8 0.8 0.4 0.570 0.568 0.602 0.660 0.643 0.660 9 0.5 -0.7 1 1 1 1 1 1 10 -0.2 -0.6 0.663 0.661 0.707 0.834 0.788 0.814 11 0.7 0.2 -0.5 0.836 0.851 0.891 0.942 0.927 0.948 12 1.3 -0.35 0.1 0.824 0.819 0.823 0.930 0.922 0.928 13 0.4 -0.6 0.3 1 1 1 1 1 1 14 0.9 -0.3 1.3 -0.5 0.830 0.822 0.830 0.932 0.920 0.931 m=15 1 0.7 0.897 0.885 0.989 1 0.993 0.999 2 0.4 0.212 0.204 0.277 0.363 0.332 0.335 3 -0.5 0.422 0.407 0.471 0.684 0.619 0.679 4 0.6 0.3 0.761 0.751 0.734 0.916 0.907 0.911 5 1.2 -0.73 1 1 1 1 1 1 6 1 -0.6 0.998 0.997 1 1 1 1 7 0.24 0.1 0.220 0.212 0.233 0.366 0.335 0.356 8 0.8 0.4 0.346 0.335 0.348 0.562 0.529 0.553 9 0.5 -0.7 1 1 1 1 1 1 10 -0.2 -0.6 0.399 0.387 0.457 0.689 0.604 0.679 11 0.7 0.2 -0.5 0.595 0.573 0.701 0.867 0.796 0.886 12 1.3 -0.35 0.1 0.667 0.656 0.627 0.854 0.83 0.820 13 0.4 -0.6 0.3 0.997 0.995 1 1 1 1 14 0.9 -0.3 1.3 -0.5 0.609 0.578 0.621 0.844 0.795 0.839

4.2.3 Simulation Study on Long Term Data

We consider a large long sample size N=500 is generated by R and the power of the tests are analyzed for m = 5, 10, 15, 20. Table 4.3 presents the results for m = 5, 10 for the portmanteau tests. The other choices of m shown in Table A.3, Appendix A.

43 Table 4.3 shows that all tests appear sensitivity to the choice of the value m and reach its maximum at lag m = 5, where the average lacks in the power of the tests from lag 5 to lag 10

   *   are 3.6%, 3.6%, 2.8%, 1.1%, 1% and 1% for QBP , QLB , QM , Dm , QWL and QWM respectively.   In Table 4.3, for AR(p) and MA(q), the performance of QM is better than QLB if the order of  the moving average component is underestimated. Contrariwise, QLB performs better if the order of the autoregressive component is underestimated but it seems there's no difference   between QLB and QM for models 1, 4, 5 and 6 from lag 5 to lag 15 .  We can see that QWL test increases the power only by the average 7.9 % when we replace     QLB by QWL , and if we replace QM by the other proposed test QWM the average increase in power over the models considered is 6.9 %, with exceptions in models 2 and 7 such that the increase is slightly greater than that. According to Monti Carlo study the differences in power

*   between Dm ,QWL and QWM tests are very small around 0.003 only unlike previous samples. It is clear that there is no general rule for choosing the best portmanteau test, as most of the results have high power which reaches to 100% or close to it. But for models in which we can

*   distinguish the power results, Dm ,QWL and QWM statistics tend to be the most powerful tests. In Table 4.3 the test statistic with the highest power for any particular model is in bold font.

Table 4.3 Powers of Portmanteau Tests for N = 500,   0.05

m=5 Model ϕ1 ϕ2 θ1 θ2    *   QBP QLB QM Dm QWL QWM 1 0.7 1 1 1 1 1 1 2 0.4 0.747 0.747 0.782 0.873 0.841 0.864 3 -0.5 0.989 0.989 0.992 0.998 0.995 0.996 4 0.6 0.3 1 1 1 1 1 1 5 1.2 -0.73 1 1 1 1 1 1 6 1 -0.6 1 1 1 1 1 1 7 0.24 0.1 0.791 0.790 0.803 0.895 0.885 0.895 8 0.8 0.4 0.977 0.967 0.978 0.979 0.991 0.979 9 0.5 -0.7 1 1 1 1 1 1 10 -0.2 -0.6 0.988 0.988 0.991 0.999 0.996 0.998 11 0.7 0.2 -0.5 1 1 1 1 1 1 12 1.3 -0.35 0.1 0.998 0.997 0.998 0.999 1 1 13 0.4 -0.6 0.3 1 1 1 1 1 1 14 0.9 -0.3 1.3 -0.5 0.999 0.999 1 1 1 1

44 m=10 1 0.7 1 1 1 1 1 1 2 0.4 0.598 0.590 0.682 0.814 0.800 0.810 3 -0.5 0.957 0.957 0.969 0.993 0.988 0.992 4 0.6 0.3 1 1 1 1 1 1 5 1.2 -0.73 1 1 1 1 1 1 6 1 -0.6 1 1 1 1 1 1 7 0.24 0.1 0.659 0.655 0.664 0.832 0.821 0.831 8 0.8 0.4 0.928 0.927 0.942 0.975 0.987 0.977 9 0.5 -0.7 1 1 1 1 1 1 10 -0.2 -0.6 0.947 0.947 0.957 0.994 0.99 0.994 11 0.7 0.2 -0.5 0.996 0.997 0.999 1 1 1 12 1.3 -0.35 0.1 0.995 0.993 0.991 0.998 1 1 13 0.4 -0.6 0.3 1 1 1 1 1 1 14 0.9 -0.3 1.3 -0.5 0.992 0.994 0.998 0.999 0.999 0.999

4.3 Numerical Example To validate the simulation results, we apply the portmanteau tests for testing the adequacy of the fitted model on real data set. We consider the monthly consumption of electricity (in kilowatt- hours, KWH) in Khan Younis city, Palestine, from April 2009 through May 2013. R-statistical software is used for fitting ARMA model for the time series.

4.3.1 ARMA Model Building Process

Firstly we will introduce an illustrative sketch to show ARMA model building process:

yes Plot Is it Model series specification stationary?

No

Suitable Model transformation estimation

No Is model Modify adequate? model

yes

Forecasts

Figure 4.1 ARMA Model Building Process

45 4.3.2 Data Exploration

Figure 4.2 displays the time series plot. The series displays considerable fluctuations over time, and a stationary model does not seem to be reasonable. The higher values display considerably more variation than the lower values. In addition, software implementation of the ADF test for stationarity is applied to the original consumption leads to a test statistic of -2.4231 and a p-value of 0.4046. With nonstationarity as the null hypothesis, this provides strong evidence supporting the nonstationarity and the appropriateness of taking a difference of the original series. 1.3e+07 1.1e+07 Monthly Electricity Consumption Electricity Monthly

9.0e+06 2010 2011 2012 2013 Years

Figure 4.2 Monthly Amount of Electricity Consumption

The differences of the electricity values are displayed in Figure 4.3. The differenced series looks much more stationary when compared with the original time series shown in Figure 4.2. On the basis of this plot, we might well consider a stationary model as appropriate. ADF test is applied to the differenced series leads to a test statistic of -5.0478 and a p-value of 0.01. That is, we reject the null hypothesis of nonstationarity. 2e+06 0e+00 -2e+06 Diff. of (Monthly Electricity Consumption) Electricity (Monthly of Diff. 10 20 30 40 50

Years

Figure 4.3 Difference of Amount of Electricity Consumption

46 4.3.3 Fitting an Inappropriate ARIMA(1,1,0) Model

Now, suppose the difference of the electricity consumption time series data is fitted wrongly by ARIMA(1,1,0) model. We perform the aforementioned tests to the ARIMA(1,1,0) fitted model. Table 4.4 shows the p-values of the portmanteau tests for selected lags, m=5, 10, 15, and 20 with   0.05. Clearly the result indicates that for lag 5, all portmanteau tests reject the null hypothesis of adequacy model, i.e. all portmanteau have the same result for autocorrelation diagnostics.  In addition, for more clarification if one uses Box-Pierce (QBP ) test for model adequacy  diagnostics, although for m = 5, QBP test shows that the fitted ARIMA(1,1,0) model appears to  be inadequate, however for m =10, 15 and 20 the QBP test indicates significant evidence to  support the null hypothesis of model adequacy. While the classic Ljung-Box (QLB )and Monti    (QM )tests give unstable p-values across m, since for m =5 and 15 , QLB and QM show that the fitted ARIMA(1,1,0) model appears to be inadequate model, whereas for m =10 and 20, these  tests support the model adequacy. Likewise, Weighted Ljung-Box test (QWL )suggests an inadequate model for m =5 and 10 and an adequate model for m = 15 and 20.

*  Furthermore, the analysis shows that Gvtest (Dm ) and Weighted Monti (QWM ) portmanteau tests tend to be the most powerful statistics in detecting inadequacy fitted model, the corresponding p-values for all selected lags are small enough to reject the null hypothesis of

*  adequacy model. Therefore, Gvtest ( Dm ) and Weighted Monti (QWM ) portmanteau tests are recommended for autocorrelation diagnostics. So we conclude to modify the fitted model according to Figure 4.1.

Table 4.4 P-values of the Portmanteau Tests of the Residuals for ARIMA(1,1,0) Model Model m    D *   QBP QLB QM m QWL QWM 5 0.0363 0.0248 0.0126 0.0044 0.0051 0.0013 ARIMA 10 0.2002 0.1330 0.0738 0.0189 0.0430 0.0165 (1,1,0) 15 0.1519 0.0486 0.0148 0.0241 0.0582 0.0154 20 0.3675 0.1421 0.0556 0.0357 0.0692 0.0176

4.3.4 Fitting an Appropriate Model

The sample EACF computed on the first differences of the electricity consumption series is shown in Table 4.5. In this table, an ARMA(p,q) process will have a theoretical pattern of a triangle of zeroes, with the upper left-hand vertex corresponding to the ARMA orders. 47 Table 4.5 EACF for Difference of Electricity Consumption Series MA 0 1 2 3 4 5 6 7 8 9 10 11 12 13 0 X 0* 0 0 0 0 0 0 0 0 0 0 0 0 1 X 0 0 0 0 0 0 0 0 0 0 0 0 0 2 X 0 0 0 0 0 0 0 0 0 0 0 0 0 AR 3 0 X 0 0 0 0 0 0 0 0 0 0 0 0 4 X X 0 0 0 0 0 0 0 0 0 0 0 0 5 0 X 0 0 0 0 0 0 0 0 0 0 0 0 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 X 0 0 0 0 0 0 0 0 0 0 0 0 0

Table 4.5 displays the schematic pattern for an MA(1) model. The upper left-hand vertex of the triangle of zeros is marked with the symbol 0* and is located in the p = 0 row and q = 1 column, an indication of an MA(1) model. The model for the original electricity consumption series would then be a nonstationary IMA(1,1) model. We perform the aforementioned Portmanteau tests to the the IMA(1,1) model. Table 4.6 shows the p-values of the portmanteau tests for selected lags, m=5, 10, 15, and 20 with   0.05. Clearly the result indicates that for all lags, all portmanteau tests have insufficient evidence to reject the null hypothesis of adequacy model, i.e. all portmanteau have the same result for autocorrelation diagnostics and detect the fitted model correctly.

Table 4.6 P-value of the Portmanteau Tests of the Residuals for ARIMA(0,1,1) Model Model m    D *   QBP QLB QM m QWL QWM 5 0.5851 0.5369 0.5999 0.5099 0.7073 0.7418

ARIMA (0,1,1) 10 0.6392 0.4886 0.4291 0.7531 0.8245 0.8358 15 0.6216 0.3595 0.2226 0.6589 0.6839 0.6119 20 0.6329 0.5617 0.4097 0.6615 0.6515 0.5221

Overall, in case of choosing an inappropriate and appropriate fitted models, Gvtest and Weighted Monti portmanteau tests detect the model correctly in both cases for all selected lags. Therefore, for most of the circumstances Gvtest and Weighted Monti portmanteau tests are recommended to use for model diagnostic phase.

4.4 Summary

In this Chapter we performed portmanteau test statistics on simulated data for different ARMA models with small, moderate and large samples, then we verified our simulation results on real case study which is electricity consumption data.

48

Chapter 5

Conclusion and Recommendations

5.1 Conclusion

We compared among six portmanteau tests for goodness-of- fit for univariate ARMA time series models. Using Monti Carlo simulation, we found that these tests have the highest values of power for long term data (N=500) comparing to short and moderate terms data (N = 50, and 200).

The study concluded that these tests are sensitive to the m values, such that there are loss in the power with respect to m ranging from m =5 to 20. The power loss reaches its smallest values for long term data comparing to small and moderate terms. For N = 50 and 200, Gvtest and Weighted Monti test were not affected by the m values as large as the other tests and the same behavior for Weighted Monti and Weighted Box for N = 500.   For AR(p) and MA(q), it seems that the performance of QM is better than QLB if the order of the  moving average component is underestimated, contrariwise, QLB performs better if the order of the   autoregressive component is underestimated. Similar behavior of QM and QLB tests was also noticed by Monti (1994).

* Fisher and Gallagher (2012) compared Fisher tests (2011) and Gvtest ( D m ) for N=100 at lag m = 20 and showed that one of the two Weighted statistics is always produce the most powerful test, or tied for most powerful test.

* For N=50, 200, we showed that the highest power of the tests varies between Gvtest ( D m ) and Fisher (2011) tests for lags m=5 and 10.

* In general Gvtest ( D m ) statistic outperforms and preferable for lags m=15 and 20. On other hand,

* when N=500 all tests are almost identical, however, Gvtest ( D m ) and Fisher (2011) tests outperform for some time series models. 49 * It is interesting to note from real data analysis that Gvtest (D m ) and Weighted Monti tests tend to have more stable p-values across m and to be the most powerful tests in detecting the goodness of fit for ARMA time series models, hence these tests are strongly recommended to be used.

5.2 Recommendations and Future Research

Many opportunities of future research are available. The plan of the future research on portmanteau tests for the residuals autocorrelation diagnostics in ARMA models can be split into four main areas: 1. Extension of the portmanteau tests for seasonal, Generalized Autoregressive Conditional Heteroskedasticity (GARCH) and Threshold Autoregressive (TAR) models.

2. Extend the research to examine the relationship between ARMA(p,q) coefficients and the power of the tests for testing the goodness-of- fit tests in time series.

  3. One may study a portmanteau test that combines Monti (QM )and Ljung –Box (QLB ) tests for   detecting adequacy fitting for ARMA(p,q) model since QLB tends to perform better than QM when the fitted model underestimates the order of the autoregressive component and vice versa.

4. Finally, determine the appropriate range for lag m to each portmanteau test to increase the ability for model misspecification detection.

50

References

[1] Arranz, M.A. (2005). “Portmanteau test statistics in Time Series”, Tol-Project.

[2] Box, G.E.P. and Pierce, D.A. (1970). “Distribution of Residual Autocorrelations in Autoregressive-Integrated Moving Average Time Series Models”, Journal of the American Statistical Association, Vol. 65, No. 332, pp. 1509—1526.

[3] Casella, G. and Berger, R.L. (2002). “Statistical Inference”, 2nd Ed., Duxbury Press, CA.

[4] Chand, S. and Kamal, S. (2006). “A Comparative Study of Portmanteau Tests for Univariate Time Series Models ”, Pak. j. stat. oper. res., Vol.II , No.2, 111-114

[5] Cryer, J. D. and Chan, K. S. (2008). “ Time Series Analysis with Application in R”, 2nd Ed., Springer, New York.

[6] Fisher, T. J. (2011). “Testing the Adequacy of ARMA Models using a Weighted Portmanteau Test on Residual Autocorrelations”, Contributed Paper 327-2011 to 2011 SAS Global Forum, Las Vegas, NV.

[7] Fisher, T. J. and Gallagher, C. M. (2012). “New Weighted Portmanteau Statistics for Time Series Goodness of Fit Testing”, Journal of the American Statistical Association, 107:498, 777- 787.

[8] Hogg, R.V., McKean, J.W. and Criag, A.T. (2005). “Introduction to Mathematical Statistics”, 6th Ed., Prentic Hall.

[9] Katayama, N. (2009 ). “On Multiple Portmanteau Tests”, Journal of Time Series Analysis, Vol. 30, Issue 5, pp. 487-504.

51 [10] Kwan, A.C.C. and Wu, Y. (1997). “Further results on the finite sample distribution of Monti’s portmanteau test for the adequacy of an ARMA (p, q) model”, Biometrika, Vol. 84, No. 3, 733-736.

[11] Kwan, A.C.C., Sim, A. and Wu, Y. (2005). “A comparative study of finite-sample performance of some portmanteau tests for randomness of a time series", Computational Statistics and Data Analysis, 48, 391-413.

[12] Ljung, G.M. (1986). “Diagnostic testing of univariate time series models”, Biometrika, Vol. 73, No. 3, 725-30.

[13] Ljung, G.M. and Box, G.E.P. (1978). “On a measure of lack of fit in time series models”, Biometrika, Vol. 65, No. 2, pp. 297—303.

[14] Lin, J.W. and McLeod, A.I. (2006). “Improved Peña-Rodríguez portmanteau test” Computational Statistics and Data Analysis, Vol. 51, No. 3, pp. 1731—1738.

[15] Mahdi, E. and McLeod, A.I. (2012). “Improved Multivariate Portmanteau Test”, Journal of Time Series Analysis, 33(2), 211-222.

[16] Monti, A.C. (1994). “A Proposal for a Residual Autocorrelation Test in Linear Models”, Biometrika, Vol. 81, No. 4, pp. 776—790.

[17] Peña, D. and Rodríguez, J. (2002). “A Powerful Portmanteau Test of Lack of Fit for Time Series”, Journal of the American Statistical Association, Vol. 97, No. 458, pp. 601—610.

[18] Peña, D. and Rodríguez, J. (2006). “The log of the determinant of the autocorrelation matrix for testing goodness of fit in time series”, Journal of Statistical Planning and Inference, Vol. 136, No. 8, pp. 2706—2718.

[19] Tsay, R. S. (2010). “Analysis of Financial Time Series”, 3rd Ed., Wiley, New Jersey.

52

Webpages References

[20] Chan, K. and Ripley, B. (2012). TSA: Time Series Analysis. R package version 1.01, http://cran.r-project.org/web/packages/TSA/

[21] Hyndman, R.J. and Athanasopoulos, G. (2013). “ Forecasting: principles and practice”, Chapter 8, https://www.otexts.org/book/fpp/8 , Accessed on .

[22] Hyndman, R.J., (2008), Kings Data, http://robjhyndman.com/tsdldata/misc/kings.dat , Accessed on .

[23] NIST/SEMATECH, (2012). e-Handbook of Statistical Methods, Chapter 6, http://www.itl.nist.gov/div898/handbook/ , Accessed on .

53

Appendix A

Table A.1 Powers of Portmanteau Tests for N = 50, m=10 and 15, and  0.05

m=10    *   Model ϕ1 ϕ2 θ1 θ2 QBP QLB QM Dm QWL QWM 1 0.7 0.298 0.279 0.423 0.606 0.444 0.597 2 0.4 0.117 0.110 0.118 0.156 0.148 0.155 3 -0.5 0.162 0.146 0.166 0.248 0.213 0.245 4 0.6 0.3 0.110 0.183 0.145 0.255 0.256 0.234 5 1.2 -0.73 0.987 0.983 0.981 0.134 0.997 0.996 6 1 -0.6 0.572 0.537 0.651 0.786 0.744 0.784 7 0.24 0.1 0.105 0.104 0.107 0.147 0.122 0.136 8 0.8 0.4 0.072 0.069 0.074 0.092 0.092 0.091 9 0.5 -0.7 0.558 0.510 0.753 0.919 0.735 0.910 10 -0.2 -0.6 0.148 0.147 0.173 0.244 0.175 0.211 11 0.7 0.2 -0.5 0.192 0.168 0.192 0.335 0.234 0.320 12 1.3 -0.35 0.1 0.232 0.235 0.199 0.340 0.322 0.317 13 0.4 -0.6 0.3 0.428 0.386 0.71 0.866 0.614 0.864 14 0.9 -0.3 1.3 -0.5 0.146 0.117 0.128 0.226 0.177 0.206 m=15 1 0.7 0.282 0.257 0.346 0.560 0.358 0.535 2 0.4 0.116 0.107 0.110 0.144 0.139 0.133 3 -0.5 0.149 0.136 0.140 0.221 0.199 0.208 4 0.6 0.3 0.159 0.145 0.125 0.222 0.221 0.206 5 1.2 -0.73 0.975 0.964 0.956 0.123 0.992 0.992 6 1 -0.6 0.519 0.478 0.531 0.749 0.670 0.738 7 0.24 0.1 0.102 0.094 0.096 0.129 0.109 0.108 8 0.8 0.4 0.069 0.069 0.066 0.083 0.073 0.078 9 0.5 -0.7 0.475 0.421 0.647 0.880 0.643 0.867 10 -0.2 -0.6 0.133 0.122 0.113 0.209 0.158 0.171 11 0.7 0.2 -0.5 0.177 0.150 0.150 0.301 0.201 0.265 12 1.3 -0.35 0.1 0.211 0.208 0.177 0.301 0.283 0.272 13 0.4 -0.6 0.3 0.363 0.325 0.607 0.822 0.506 0.805 14 0.9 -0.3 1.3 -0.5 0.133 0.100 0.115 0.200 0.144 0.170

Table A.2 Powers of Portmanteau Tests for N = 200, m=10 and 20, and  0.05

m=10    *   Model ϕ1 ϕ2 θ1 θ2 QBP QLB QM Dm QWL QWM 1 0.7 0.964 0.962 0.994 1 0.997 0.999 2 0.4 0.262 0.255 0.329 0.412 0.393 0.411 3 -0.5 0.501 0.493 0.561 0.751 0.693 0.751 4 0.6 0.3 0.841 0.839 0.837 0.944 0.943 0.942 5 1.2 -0.73 1 1 1 1 1 1 6 1 -0.6 0.999 0.999 1 1 1 1 7 0.24 0.1 0.269 0.261 0.292 0.437 0.412 0.434 8 0.8 0.4 0.432 0.425 0.446 0.619 0.590 0.622 9 0.5 -0.7 1 1 1 1 1 1 10 -0.2 -0.6 0.4999 0.482 0.562 0.768 0.678 0.735 11 0.7 0.2 -0.5 0.678 0.684 0.763 0.911 0.867 0.913 12 1.3 -0.35 0.1 0.726 0.721 0.702 0.883 0.860 0.860 13 0.4 -0.6 0.3 1 1 1 1 1 1 14 0.9 -0.3 1.3 -0.5 0.701 0.680 0.724 0.891 0.848 0.889 54

Table A.2 Powers of Portmanteau Tests for N = 200, m=10 and 20, and  0.05- (Continued) m=20    D *   Model ϕ1 ϕ2 θ1 θ2 QBP QLB QM m QWL QWM 1 0.7 0.849 0.820 0.996 0.997 0.985 0.998 2 0.4 0.190 0.185 0.220 0.324 0.297 0.314 3 -0.5 0.381 0.371 0.407 0.639 0.557 0.626 4 0.6 0.3 0.733 0.696 0.674 0.886 0.866 0.878 5 1.2 -0.73 1 1 1 1 1 1 6 1 -0.6 0.996 0.996 0.998 1 1 1 7 0.24 0.1 0.179 0.170 0.188 0.319 0.289 0.307 8 0.8 0.4 0.320 0.304 0.302 0.507 0.472 0.500 9 0.5 -0.7 0.995 0.991 0.997 1 1 1 10 -0.2 -0.6 0.355 0.334 0.378 0.650 0.531 0.618 11 0.7 0.2 -0.5 0.524 0.504 0.605 0.828 0.742 0.857 12 1.3 -0.35 0.1 0.607 0.595 0.565 0.808 0.78 0.784 13 0.4 -0.6 0.3 0.981 0.976 1 1 1 1 14 0.9 -0.3 1.3 -0.5 0.544 0.512 0.561 0.808 0.739 0.797

Table A.3 Powers of Portmanteau Tests for N = 500, m=15 and 20, and  0.05

m=15    *   Model ϕ1 ϕ2 θ1 θ2 QBP QLB QM Dm QWL QWM 1 0.7 1 1 1 1 1 1 2 0.4 0.493 0.484 0.574 0.742 0.712 0.729 3 -0.5 0.895 0.890 0.736 0.990 0.980 0.985 4 0.6 0.3 1 0.999 1 1 1 1 5 1.2 -0.73 1 1 1 1 1 1 6 1 -0.6 1 1 1 1 1 1 7 0.24 0.1 0.551 0.548 0.573 0.788 0.765 0.784 8 0.8 0.4 0.867 0.863 0.905 0.960 0.974 0.967 9 0.5 -0.7 1 1 1 1 1 1 10 -0.2 -0.6 0.906 0.904 0.919 0.981 0.971 0.984 11 0.7 0.2 -0.5 0.985 0.985 0.996 1 1 1 12 1.3 -0.35 0.1 0.992 0.992 0.986 0.998 1 1 13 0.4 -0.6 0.3 1 1 1 1 1 1 14 0.9 -0.3 1.3 -0.5 0.986 0.989 0.992 0.999 0.999 0.999 m=20 1 0.7 1 1 1 1 1 1 2 0.4 0.444 0.431 0.521 0.691 0.662 0.683 3 -0.5 0.840 0.832 0.891 0.981 0.969 0.978 4 0.6 0.3 0.996 0.998 0.997 1 1 1 5 1.2 -0.73 1 1 1 1 1 1 6 1 -0.6 1 1 1 1 1 1 7 0.24 0.1 0.468 0.468 0.492 0.744 0.722 0.737 8 0.8 0.4 0.813 0.808 0.862 0.951 0.966 0.957 9 0.5 -0.7 1 1 1 1 1 1 10 -0.2 -0.6 0.858 0.842 0.878 0.973 0.955 0.977 11 0.7 0.2 -0.5 0.974 0.973 0.989 1 0.998 1 12 1.3 -0.35 0.1 0.980 0.981 0.973 0.998 0.997 0.998 13 0.4 -0.6 0.3 1 1 1 1 1 1 14 0.9 -0.3 1.3 -0.5 0.974 0.975 0.984 0.999 0.996 0.999

55 اﺳﻢ اﻟﻤﻠﻒ: Alaa -Thesis.Feb.11. اﻟﺪﻟﯿﻞ: I: اﻟﻘﺎﻟﺐ:

C:\Users\user\AppData\Roaming\Microsoft\Templates\Normal.dotm اﻟﻌﻨﻮان: Table of Contents اﻟﻤﻮﺿﻮع: اﻟﻜﺎﺗﺐ: Al-Qema اﻟﻜﻠﻤﺎت اﻷﺳﺎﺳﯿﺔ: ﺗﻌﻠﯿﻘﺎت: ﺗﺎرﯾﺦ اﻹﻧﺸﺎء: 01/ 20/ 2014 01:50:00 م رﻗﻢ اﻟﺘﻐﯿﯿﺮ: 13 اﻟﺤﻔﻆ اﻷﺧﯿﺮ ﺑﺘﺎرﯾﺦ: 02/ 11/ 2014 08:03:00 م اﻟﺤﻔﻆ اﻷﺧﯿﺮ ﺑﻘﻠﻢ: user زﻣﻦ اﻟﺘﺤﺮﯾﺮ اﻹﺟﻤﺎﻟﻲ: 103 دﻗﺎﺋﻖ اﻟﻄﺒﺎﻋﺔ اﻷﺧﯿﺮة: 02/ 11/ 2014 08:05:00 م ﻣﻨﺬ آﺧﺮ طﺒﺎﻋﺔ ﻛﺎﻣﻠﺔ ﻋﺪد اﻟﺼﻔﺤﺎت: 65 ﻋﺪد اﻟﻜﻠﻤﺎت: 19,257 ( ﺗﻘﺮﯾﺒﺎً) ﻋﺪد اﻷﺣﺮف: 109,769 ( ﺗﻘﺮﯾﺒﺎً)