Gaza ـــ Al- Azhar University Deanship of Postgraduate Studies & Scientific Research Faculty of Economics and Administrative sciences Department of Applied Statistics Comparative Study of Portmanteau Tests for ARMA Models دراﺳﺔ ﻣﻘﺎرﻧﺔ ﻻﺧﺗﺑﺎرات اﻻ رﺗﺑﺎط اﻟذاﺗﻲ ﻟﻸﺧطﺎء اﻟﻌﺷواﺋﯾﺔ ﻟﻧﻣﺎذج اﻵرﻣﺎ Presented By Alaa Ahmad Salman Al-Reqep Supervised By Samir Khaled Safi, Ph.D. Associate Professor of Statistics SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF APPLIED STATISTICS 2013-1435 To My Family Acknowledgment Foremost, I would to thank God for the wisdom and perseverance that he has been bestowed upon me during this thesis, and indeed, throughout my life. I am grateful to a number of people who have guided and supported me. My first sincere gratitude and appreciation goes to my advisor Professor Samir Safi, for his patience, continuous help and support in all stages of this thesis and for encouraging and helping me to shape my interest and ideas. His recommendations and instructions have enabled me to assemble and finish my thesis effectively. Besides my advisor, I would like to thank Dr. Mahmoud Okasha and Dr. Abdalla Al Habil, I would not have been able to achieve my learning in the same manner without their immense knowledge. I would like to thank Dr. Bisher Iqelan for accepting to be my external examiner. Also I would like to express the deepest appreciation to Professor Thomas J. Fisher from Department of Mathematics at University of Missouri, Kansas City, USA for his comments and guidance for the simulation study. Also I would like to express my thanks to Mr. Eyad El Shamy from Department of Information Technology at the Islamic University of Gaza for his help in programming software in order to accelerate simulation's time. Especial thanks to Energy Authority, Khan Younis for providing me by Electricity Consumption data and to my teacher in the first class Ms. Halimah Al Tartory. My sincere gratitude to my parents, brothers and sisters especially Dr. Bara'ah for believing in my abilities and for their unconditional support, spiritually throughout my life. And special thanks to my husband Eyad, my sons and two persons I consider them as long as my parents; my husband's parents for providing the moral and emotional support I needed to complete my thesis, without them I would not have been success in my life. My friends have supported and helped me along the course of this degree by giving encouragement; to them, I am eternally grateful. Finally, I would like to thank all the people who contributed in some way to the work described in this thesis. Table of Contents List of Tables iii List of Figures iv List of Outputs v Abstract vi Abstract in Arabic vii Chapter 1 Introduction and Literature Review 1 1.1 Introduction 1 1.2 Problem of The Study 1 1.3 Objectives of The Study 2 1.4 Literature Review 2 1.5 Fundamental Concepts 3 1.5.1 Mean, Variance and Covariance 3 1.5.2 Time Series 6 1.5.3 Stationarity 7 1.5.4 Nonstationarity 8 1.5.5 Differencing 9 1.5.6 Unit Root Tests 9 1.6 Box-Jenkins ARIMA Models 10 1.6.1 Auto-Regressive AR(p) Model 10 1.6.2 Moving Average MA(q) Model 11 1.6.3 Auto-Regressive – Moving Average ARMA(p, q) Model 12 1.7 Autocorrelation, Partial Autocorrelation, and Extended Partial Autocorrelation Functions 13 1.7.1 Autocorrelation Function (ACF) 13 1.7.2 Partial Autocorrelation Function (PACF) 13 1.7.3 Extended autocorrelation function (EACF) 14 1.8 Summary 17 Chapter 2 Model Diagnostic 18 2.1 Introduction 18 2.2 Residual Analysis 18 2.2.1 Homogeneity of Variance and Zero Mean 18 2.2.2 Normality of Residuals Distribution 19 2.2.3 Autocorrelation of Residuals 20 2.3 Summary 26 Chapter 3 Portmanteau Tests For ARMA Models 27 3.1 Introduction 27 3.2 Portmanteau Tests 27 3.2.1 Box and Pierce Portmanteau Test 27 3.2.2 Ljung-Box Portmanteau Test 29 3.2.3 Monti Portmanteau Test 31 i 3.2.4 Peña and Rodríguez Portmanteau Test (2002) 33 3.2.5 Peña and Rodríguez Portmanteau Test (2006) 34 3.2.6 Fisher Portmanteau Tests 37 3.3 Summary 39 Chapter 4 Simulation and Case Study 40 4.1 Introduction 40 4.2 Simulation Study 40 4.2.1 Simulation Study on Short Term Data 40 4.2.2 Simulation Study on moderate Term Data 42 4.2.3 Simulation Study on Long Term Data 43 4.3 Numerical Example 45 4.3.1 ARMA Model Building Process 45 4.3.2 Data Exploration 46 4.3.3 Fitting an Inappropriate ARIMA(1,1,0) Model 47 4.3.4 Fitting an Appropriate Model 47 4.4 Summary 48 Chapter 5 Conclusion and Recommendations 49 5.1 Conclusion 49 5.2 Recommendations and Future Research 50 References 51 Appendix A 54 ii List of Tables Table 1.1 Theoretical EACF Table for an ARMA(1,1) Model 15 Table 1.2 Behavior of the ACF and PACF for ARMA Models 15 Table 1.3 ACFs and PACFs Plots 16 Table 4.1 Powers of Portmanteau Tests for N = 50, 0.05 42 Table 4.2 Powers of Portmanteau Tests for N = 200, 0.05 43 Table 4.3 Powers of Portmanteau Tests for N = 500, 0.05 44 Table 4.4 P-values of the Portmanteau Tests of the Residuals for ARIMA(1,1,0) Model 47 Table 4.5 EACF for Difference of Electricity Consumption Series 48 Table 4.6 P-value of the Portmanteau Tests of the Residuals for ARIMA(0,1,1) Model 48 Table A.1 Powers of Portmanteau Tests for N = 50, m=10 and 15, and 0.05 54 Table A.2 Powers of Portmanteau Tests for N = 200, m=10 and 20, and 0.05 54 Table A.3 Powers of Portmanteau Tests for N = 500, m=15 and 20, and 0.05 55 iii List of Figures Figure 1.1 Stationary Time Series 7 Figure 1.2 Nonstationary Time Series 8 Figure 1.3 Random Walk Model 8 Figure 2.1 Standardized Residuals over the Time 19 Figure 2.2 Histogram Plot for Residuals 19 Figure 2.3 Normal QQ plot for Residuals 20 Figure 2.4 Sample ACF of the Residuals 21 Figure 2.5 Standardized Residuals from MA(1) Model 21 Figure 2.6 Histogram for the Residuals from MA(1) Model 22 Figure 2.7 QQ-plot for the Residuals from MA(1) Model 22 Figure 2.8 Sample Autocorrelation of Residuals from MA(1) Model 23 Figure 2.9 Standardized Residuals for AR(2) Model from NHtemp Data 24 Figure 2.10 Histogram for the Residuals for AR(2) Model from NHtemp Data 24 Figure 2.11 QQ-plot for the Residuals for AR(2) Model from NHtemp Data 25 Figure 2.12 Sample ACF of Residuals for AR(2) Model from NHtemp Data 25 Figure 3.1 Diagnostic Display for the ARMA(0,1,1) Model of Kings Series 31 Figure 4.1 ARMA Model Building Process 45 Figure 4.2 Monthly Amount of Electricity Consumption 46 Figure 4.3 Difference of Amount of Electricity Consumption 46 iv List of Outputs Output 2.1 Shapiro-Wilk Test for Residuals of MA(1) Model 23 Output 2.2 Shapiro-Wilk Test for Residuals of AR(2) Model from (NHtemp) data 25 Output 3.1 Box-Pierce Test 29 Output 3.2 Ljung-Box Test 30 Output 3.3 Monti Test for m = 5 , 15 32 Output 3.4 Gvtest 36 Output 3.5 Weighted Box Test for m = 10, 15 38 Output 3.6 Weighted Monti Test for m = 7, 13 39 v Abstract The portmanteau statistic for testing the adequacy of an autoregressive moving average (ARMA) model is based on the first m autocorrelations of the residuals from the fitted model. We consider some of portmanteau tests for univariate linear time series such as Box and Pierce (1970), Ljung and Box (1978), Monti (1994), Peña and Rodríguez (2002, 2006), Generalized Variance Test (Gvtest) by Mahdi and McLeod (2012) and Fisher (2011). We conduct an extensive computer simulation time series data, to make comparison among these tests. We consider different model parameters for short, moderate and long terms data to examine the effect of lag m on the power of the selected tests and determine the most powerful test for ARMA models. An example of real data is also given. The similar portmanteau tests models was evaluated for the real data set on electricity consumption in Khan Younis, Palestine (April 2009 - May 2013). We found that, the long term data (N = 500) has the highest values of power comparing to short and moderate terms data (N = 50 and 200). We found that the portmanteau tests are sensitive to the chosen for m value, Indeed there are loss of the power for lags m ranging from m = 5 to 20, where Box-Pierce, Ljung-Box and Monti tests have the more power loss than the other selected tests. The power loss reaches its smallest values for long term data comparing to small and moderate terms. In addition, the results of the simulation study and real data analysis showed that the highest powerful tests varies between Gvtest (2012) test and Fisher tests (2011). vi Abstract in Arabic إﺣﺻـــــﺎء اﻻرﺗﺑـــــﺎط اﻟـــــذاﺗﻲ ﻟﻸﺧطـــــﺎء اﻟﻌﺷـــــواﺋﯾﺔ ﻻﺧﺗﺑـــــﺎر ﻛﻔـــــﺎءة ﻧﻣـــــوذج اﻻﻧﺣـــــدار اﻟـــــذاﺗﻲ ﻟﻠوﺳـــــط اﻟﻣﺗﺣـــــرك (ARMA) ﯾﻌﺗﻣــد ﻋﻠــﻰ أول ﻋــدد m ﻣــن اﻻرﺗﺑﺎطــﺎت اﻟذاﺗﯾــﺔ ﻟﻸﺧطــﺎء اﻟﻌﺷــواﺋﯾﺔ ﻟﻠﻧﻣــوذج اﻟﻣﻧﺎﺳــب اﻟــذي ﺗــم ﻣﻼﺋﻣﺗﻪ ﻟﻠﺑﯾﺎﻧﺎت.