DOCSLIB.ORG

A Portmanteau Test for Self-Exciting Threshold Autoregressive-Type Nonlinearity in Time Series Author(S): Joseph D

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

Biometrika Trust A Portmanteau Test for Self-Exciting Threshold Autoregressive-Type Nonlinearity in Time Series Author(s): Joseph D. Petruccelli and Neville Davies Source: Biometrika, Vol. 73, No. 3 (Dec., 1986), pp. 687-694 Published by: Oxford University Press on behalf of Biometrika Trust Stable URL: https://www.jstor.org/stable/2336533 Accessed: 31-07-2019 20:19 UTC REFERENCES Linked references are available on JSTOR for this article: https://www.jstor.org/stable/2336533?seq=1&cid=pdf-reference#references_tab_contents You may need to log in to JSTOR to access the linked references. JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected]. Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at https://about.jstor.org/terms Biometrika Trust, Oxford University Press are collaborating with JSTOR to digitize, preserve and extend access to Biometrika This content downloaded from 130.215.176.72 on Wed, 31 Jul 2019 20:19:11 UTC All use subject to https://about.jstor.org/terms Biometrika (1986), 73, 3, pp. 687-94 Printed in Great Britain A portmanteau test for self-exciting threshold autoregressive-type nonlinearity in time series BY JOSEPH D. PETRUCCELLI Department of Mathematical Sciences, Worcester Polytechnic Institute, Massachusetts 01609, U.S.A. AND NEVILLE DAVIES Department of Mathematics, Statistics and Operational Research, Trent Polytechnic, Nottingham NG1 4BU, U.K. SUMMARY A portmanteau test to detect self-exciting threshold autoregressive-type nonlinearity in time series data is proposed. The test is based on cumulative sums of standardized residuals from autoregressive fits to the data. Significance levels for the test under the hypothesis of linearity are obtain from the asymptotic distribution of the cumulative sums as Brownian motion. The performance of the test is evaluated for simulated data from linear, bilinear and self-exciting threshold autoregressive models. It is also compared with another test which has been suggested for detecting general nonlinearity. Features of the proposed test, which make it useful in identifying the autoregressive order and the lag in threshold models, are discussed. Some key words: Nonlinear time series; Portmanteau test; Self-exciting threshold autoregressive model. 1. INTRODUCTION Recently authors such as Tong (1978, 1983), Tong & Lim (1980), Granger & Anderson (1978), Priestley (1980) and Haggan & Ozaki (1981) have argued convincingly for the need for nonlinear models in time series analysis. The incidence of nonlinearity in a large sample of sets of real data is discussed by Davies & Petruccelli (1986). If, as appears to be the case, there is a need for nonlinear modelling in time series analysis, it certainly follows that there is a need for methods to ascertain when a series is nonlinear. One such method relies on the fact, inherent in the work of Granger & Newbold (1976), that, for a series Y, which is normal, Pk( Yt) = {Pk( Yt)}2, where Pk is the lag k autocorrela- tion. Granger & Anderson (1978) point out that departures from this relation could indicate nonlinearity. Maravall (1983) considers the possibility of examining.the sample autocorrelation functions of either the series itself and its squared values or the residuals from a fitted model and their squared values in order to detect nonlinearity. Davies & Petruccelli (1986) discuss this approach and some possible problems with it. McLeod & Li (1983) propose a portmanteau test, based on the autocorrelations of squared residuals from a linear fit, to detect nonlinearity in time series data. Their test statistic is analogous to the well-known statistics of Box & Pierce (1970) and Ljung & Box (1978). Significance levels are based on the asymptotic x2 distribution of the test statistic when the process is linear. This content downloaded from 130.215.176.72 on Wed, 31 Jul 2019 20:19:11 UTC All use subject to https://about.jstor.org/terms 688 JOSEPH D. PETRUCCELLI AND NEVILLE DAVIES Keenan (1985) takes a novel approach in devising a test for nonlinearity. He first assumes the series can be adequately approximated by a second-order Volterra expansion (Wiener, 1958). 00 00 00 Yt = O+ E Ouat-u + E E 0UVat_uat_v5 (1 1 U=-00 v=-00 u=-00 where {at} is a strictly stationary process. The second term on the right-hand side of (1-1) is analogous to an interaction term in regression analysis, and linearity of (1-1) is equivalent to this term being zero. To test whether this is the case, Keenan obtains an order M autoregressive approximation to the process (1.1) and calculates an F statistic which emulates Tukey's one degree of freedom for nonadditivity test in regression analysis. Spectral tests for nonlinearity (Subba Rao & Gabr, 1980; H-inich, 1982) are not considered here. The state dependent models described by Priestley (1980) contain many standard nonlinear models, such as threshold autoregressive, bilinear and amplitude dependent, as subclasses. The fitting procedure for state dependent models proposed by Haggan, Heravi & Priestley (1984) appear to indicate at the same time, using graphical methods, to which subclass the model belongs. In the present paper we introduce a portmanteau test designed to detect the specific class of self-exciting threshold autoregressive models. In ? 2 we describe the theory and implementation of the test, while in ? 3 we summarize results obtained in applying the test to simulated data from linear, bilinear and threshold autoregressive models. 2. A PORTMEANTEAU TEST FOR NONLINEARITY The proposed test is based on cumulative sums of standardized one-step forecast errors from autoregressive fits to the data. The idea for such a statistic in the piecewise linear regression context was developed by Ertel & Fowlkes (1976). We assume that, rather than following a piecewise linear regression model, the data follow the self-exciting threshold autoregressive model of order p, p Yt = +o)+5 I ( Yt-i + at (2.1) i=l if rj-1 < Yt-d d rj (=j =1, . ..., l), t=p+1, ..., n and d is a positive integer. Here the thresholds are -oo = ro < r, < ... < r, = 0o, and {at} is a sequence of identically indepen- dently distributed zero mean random noise terms with variance a2. In the discussion that follows we assume = 2, so that there is one nontrivial threshold. Let Y(i) (i = 1, ... n -p) be the ith smallest observation among { Yp+l-d, ..., Yn-d}. The expression in (2-1) can then be formulated as a finite autoregression in the Y(1). If the threshold lies between the m and (m + 1)th ordered Yt values, the complete pth order autoregression implied by (2.1) can be written as P 0o) ( 1)Y(i)+d-1+a(i)+d (i= 1, ... ., m), Y(i)+d = I- (2-2) This content downloaded from 130.215.176.72 on Wed, 31 Jul 2019 20:19:11 UTC All use subject to https://about.jstor.org/terms Test for nonlinearity in time series 689 If we proceed with successive autoregressions, of fixed order p, that use the first r values of the Y(i), for 1 < rmin - r - m, then the standardized one-step-ahead forecast errors should be roughly identically independently distributed with zero mean and unit variance. However, from (2.2), as r begins to exceed m, the nonlinearity of the process should cause systematic deviations in the forecast errors. Let the ordered autoregression implied by (2. 1) be represented by Y = X4 + a, where Y is a column vector that contain the Y(i)+d (i = 1,..., n -p), X is an (n -p) x (p + 1) matrix with first column containing unity and the remaining columns appropriate lagged Y(i)+d values, 4 is a column vector of the Oyi) (i = 0, 1, ... , p; j= 1, ..., 1) and a is a column vector of noise terms. In general our procedure is as follows. Step 1. Choose the order of autoregression, p, the lag d and rmin. Step 2. For rmin < r - n -p, find the multiple regression of the first r rows of Y on the first r rows of X and compute the successive one-step-ahead standardized forecast errors, Zr+i, say. Step 3. Form the cumulative sums r Zr= E zi (r=rmin+1 ..., n-p). i =rmin+ 1 The zi's may be calculated very efficiently using regression updating methods (Ertel & Fowlkes, 1976). The cumulative sums may be plotted sequentially to give a graphical method for detecting not only nonlinearity, but also, for threshold autoregressive models, the approximate location of thresholds. Standard tests may be employed to detect when the cumulative sums go out of control. The scheme of Ertel & Fowlkes (1976), for example requires a run in the sign of the zi's of sufficient length and a minimum increase in the cumulative sum over this span in order to conclude the process is no longer linear. To develop a portmanteau test for nonlinearity, we use an invariance principle for random walks (Feller, 1970, pp. 342-3). Let Tn= max lZrl. rmin+l1rGn-p Then as n -> oo, 00 pr{Tn/(n-p-rmin)1<t}->4,-1 Z (-)k(2k+ 1)-1 exp {-(2k+1) r2/(8t2)}. (2 3) k=O We consider the hypothesis that model (2.1) is linear Ho: +'l '-'(i = 0x 19 . .. 9 p) versus the alternative hypothesis that Ho does not hold. Under H0, (2.3) should hold approximately for moderate samples sizes. For a given series let 1 -p* denote the value computed from the right-hand side of (2 3) with t given by max IZrl/(n-p-rmin)2.
Recommended publications
  • 6. the CHI-SQUARE TEST a Chi-Squared Test, Also

    6. the CHI-SQUARE TEST a Chi-Squared Test, Also

    SEC 4 Page 1 of 7 6. THE CHI-SQUARE TEST A chi-squared test, also referred to as chi-square test or test, is any statistical hypothesis test in which the sampling distribution of the test statistic is a chi-squared distribution when the null hypothesis is true. Also considered a chi-squared test is a test in which this is asymptotically true, meaning that the sampling distribution (if the null hypothesis is true) can be made to approximate a chi-squared distribution as closely as desired by making the sample size large enough. Some examples of chi-squared tests where the chi-squared distribution is only approximately valid: Pearson's chi-squared test, also known as the chi-squared goodness-of-fit test or chi-squared test for independence. When the chi-squared test is mentioned without any modifiers or without other precluding context, this test is usually meant (for an exact test used in place of , see Fisher's exact test). Yates's correction for continuity, also known as Yates' chi-squared test. Cochran–Mantel–Haenszel chi-squared test. McNemar's test, used in certain 2 × 2 tables with pairing Tukey's test of additivity The portmanteau test in time-series analysis, testing for the presence of autocorrelation Likelihood-ratio tests in general statistical modelling, for testing whether there is evidence of the need to move from a simple model to a more complicated one (where the simple model is nested within the complicated one). One case where the distribution of the test statistic is an exact chi-squared distribution is the test that the variance of a normally distributed population has a given value based on a sample variance.
  • An Asymptotic F Test for Uncorrelatedness in the Presence of Time Series Dependence∗

    An Asymptotic F Test for Uncorrelatedness in the Presence of Time Series Dependence∗

    An Asymptotic F Test for Uncorrelatedness in the Presence of Time Series Dependence∗ Xuexin Wang Yixiao Sun School of Economics and WISE Department of Economics Xiamen University, China UC San Diego, USA Abstract We propose a simple asymptotic F-distributed Portmanteau test for zero autocor- relations in an otherwise dependent time series. By employing the orthonormal series variance estimator of the variance matrix of sample autocovariances, our test statis- tic follows an F distribution asymptotically under fixed-smoothing asymptotics. The asymptotic F theory accounts for the estimation error in the underlying variance estima- tor, which the asymptotic chi-squared theory ignores. Monte Carlo simulations reveal that the F approximation is much more accurate than the corresponding chi-squared approximation in finite samples. Compared with the nonstandard test proposed by Lobato (2001), the asymptotic F test is as easy to use as the chi-squared test: There is no need to obtain critical values by simulations. Further, Monte Carlo simulations indicate that Lobato’s (2001) nonstandard test tends to be heavily undersized under the null and suffers from substantial power loss under the alternatives. ∗Email: [email protected]; [email protected]. Wang gratefully acknowledges the Fundamental Re- search Funds for the Central Universities [Grant No. 20720171043]. Sun gratefully acknowledges partial research support from the NSF [Grant No. SES-1530592]. 1 Keywords: Lack of autocorrelations; Portmanteau test; Fixed-smoothing asymptotics; F distribution; Orthonormal series variance estimator 1 INTRODUCTION In this study, we test the null hypothesis that a time series process is uncorrelated up to a certain order. This is a basic problem in time series analysis.
  • Testing Adequacy of ARMA Models Using a Weighted Portmanteau Test on the Residual Autocorrelations Thomas J

    Testing Adequacy of ARMA Models Using a Weighted Portmanteau Test on the Residual Autocorrelations Thomas J

    SAS Global Forum 2011 Statistics and Data Analysis Paper 327-2011 Testing Adequacy of ARMA Models using a Weighted Portmanteau Test on the Residual Autocorrelations Thomas J. Fisher, Department of Mathematics & Statistics, University of Missouri- Kansas City, Kansas City, MO, 64110, USA ABSTRACT In examining the adequacy of a statistical model, an analysis of the residuals is often performed. This includes anything from performing a residual analysis in a simple linear regression to utilizing one of the portmanteau tests in time-series analysis. When modeling an autoregressive-moving average time series we typically use the Ljung-Box statistic on the residuals to see if our fitted model is adequate. In this paper we introduce two new statistics that are weighted variations of the common Ljung-Box and, the less-common, Monti statistics. A brief simulation study demonstrates that the new statistics are more powerful than the commonly used Ljung-Box statistic. The new statistics are easy to implement in SAS® and source code is provided. INTRODUCTION A plethora of situations are known to occur in which the errors in a regression model are not independent. This violates the underlying assumptions of regression and can lead to a multitude of problems. Modeling the errors via a time series analysis does not address all of the issues but allows us to have better predictive models. However, much like checking the adequacy of the regression through an F-test, checking the adequacy of the fitted time-series model is of the utmost importance. Let be a time series for where is the number of observations.

  • An Improved Portmanteau Test for Autocorrelated Errors in Interrupted Time-Series Regression Models

    Behavior Research Methods 2007, 39 (3), 343-349 An improved portmanteau test for autocorrelated errors in interrupted time-series regression models BRADLEY E. HUITEMA AND JOSEPH W. M CKEAN Western Michigan University, Kalamazoo, Michigan A new portmanteau test for autocorrelation among the errors of interrupted time-series regression models is proposed. Simulation results demonstrate that the inferential properties of the proposed QH–M test statistic are considerably more satisfactory than those of the well known Ljung–Box test and moderately better than those of the Box–Pierce test. These conclusions generally hold for a wide variety of autoregressive (AR), moving averages (MA), and ARMA error processes that are associated with time-series regression models of the form described in Huitema and McKean (2000a, 2000b). Several approaches are available for the analysis of in- on intervention-effect coefficients (Huitema, McKean, & terrupted time-series (or time-series intervention) experi- McKnight, 1994). Interestingly, the problem of distorted mental designs. The simplest version of this design has Type I error rates is also the main reason that these alter- two phases. The first phase (sometimes called the base- native estimation methods are recommended in the first line or preintervention phase) has n1 observations, and place. Hence, there are two reasons why it is important to the second phase (postintervention) has n2 observations. know whether the errors are autocorrelated. First, if the The purpose of the statistical analysis is to quantitatively errors are not autocorrelated, OLS is the preferred method describe and evaluate possible intervention effects. of estimation, and corrective methods should be avoided. Autoregressive integrated moving average (ARIMA) Second, if the errors are autocorrelated, corrective meth- models and time-series regression models are frequently ods are called for in order to maintain error rates at the satisfactory for this purpose.
  • A Comparative Study of Portmanteau Tests for Univariate Time Series Models

    A Comparative Study of Portmanteau Tests for Univariate Time Series Models

    A Comparative Study of Portmanteau Tests for Univariate Time Series Models Sohail Chand Institute of Statistics University of the Punjab Lahore-Pakistan E-mail: [email protected] Shahid Kamal Institute of Statistics University of the Punjab Lahore-Pakistan E-mail: [email protected] Abstract Time series model diagnostic checking is the most important stage of time series model building. In this paper the comparison among several suggested diagnostic tests has been made using the simulation time series data. 1. Introduction Until the advent of powerful and accessible computing methods, the experimenter was often confronted with a difficult choice. Either describes an accurate model of a phenomenon, which would usually preclude the computation of explicit answers, or choose a standard model which would allow this computation, but may not be a close representation of a realistic model. This dilemma is present in many branches of statistical applications, for example, in electrical engineering, aeronautics, biology, networks, economics, and astronomy. To use realistic models, the researches in these disciplines have often developed original approaches for model fitting that are customized for their own problems. Model building is an important part of time series analysis (Chatfield; 2004). The task facing the modern time series is to develop reasonably simple models capable of forecasting, interpreting, and testing hypothesis concerning its field of application (Enders; 2004). Time series model building is considered as a three-stage iterative procedure based on – Identification – Estimation – Diagnostic checking. Diagnostic checking is to check the fitted model in its relation to the data with intent to reveal inadequacies and so to achieve the model improvement (Box et.
  • Advances in Portmanteau Diagnostic Tests

    Advances in Portmanteau Diagnostic Tests

    Western University Scholarship@Western Electronic Thesis and Dissertation Repository 9-2-2016 12:00 AM Advances in Portmanteau Diagnostic Tests Jinkun Xiao The University of Western Ontario Supervisor Dr. A. Ian McLeod The University of Western Ontario Graduate Program in Statistics and Actuarial Sciences A thesis submitted in partial fulfillment of the equirr ements for the degree in Doctor of Philosophy © Jinkun Xiao 2016 Follow this and additional works at: https://ir.lib.uwo.ca/etd Part of the Applied Statistics Commons, Longitudinal Data Analysis and Time Series Commons, and the Statistical Methodology Commons Recommended Citation Xiao, Jinkun, "Advances in Portmanteau Diagnostic Tests" (2016). Electronic Thesis and Dissertation Repository. 4100. https://ir.lib.uwo.ca/etd/4100 This Dissertation/Thesis is brought to you for free and open access by Scholarship@Western. It has been accepted for inclusion in Electronic Thesis and Dissertation Repository by an authorized administrator of Scholarship@Western. For more information, please contact [email protected]. Abstract Portmanteau test serves an important role in model diagnostics for Box-Jenkins Modelling pro- cedures. A large number of portmanteau test based on the autocorrelation function are proposed for a general purpose goodness-of-fit test. Since the asymptotic distribution for the statistics has a complicated form which makes it hard to obtain the p-value directly, the gamma approx- imation is introduced to obtain the p-value. But the approximation will inevitably introduce approximation errors and needs a large number of observation to yield a good approximation. To avoid some pitfalls in the approximation, the Lin-Mcleod Test is further proposed to obtain a numeric solution to this problems based on Monte Carlo Simulation.
  • An Asymptotic F Test for Uncorrelatedness in the Presence

    An Asymptotic F Test for Uncorrelatedness in the Presence

    An Asymptotic F Test for Uncorrelatedness in the Presence of Time Series Dependence∗ Xuexin Wang Yixiao Sun School of Economics and WISE Department of Economics Xiamen University, China UC San Diego, USA Abstract We propose a simple asymptotically F-distributed Portmanteau test for zero autocorrela- tions in an otherwise dependent time series. By employing the orthonormal series variance estimator of the variance matrix of sample autocovariances, our test statistic follows an F distribution asymptotically under fixed-smoothing asymptotics. The asymptotic F theory accounts for the estimation error in the underlying variance estimator, which the asymptotic chi-squared theory ignores. Monte Carlo simulations reveal that the F approximation is much more accurate than the corresponding chi-squared approximation in finite samples. Compared with the nonstandard test proposed by Lobato (2001), the asymptotic F test is as easy to use as the chi-squared test: There is no need to obtain critical values by simulations. Further, Monte Carlo simulations indicate that Lobato's (2001) nonstandard test tends to be heavily undersized under the null and suffers from substantial power loss under the alternatives. Keywords: Lack of autocorrelations; Portmanteau test; Fixed-smoothing asymptotics; F dis- tribution; Orthonormal series variance estimator ∗Email: [email protected]; [email protected]. Wang gratefully acknowledges the Fundamental Research Funds for the Central Universities [Grant No. 20720171043]. Sun gratefully acknowledges partial research support from the NSF [Grant No. SES-1530592]. We thank the anonymous Co-Editor and reviewers for constructive comments. 1 1 INTRODUCTION In this study, we test the null hypothesis that a time series process is uncorrelated up to a certain order.
  • Arxiv:1903.00776V2 [Stat.ME] 26 May 2021 an Empirical Bayes Method

    Arxiv:1903.00776V2 [Stat.ME] 26 May 2021 an Empirical Bayes Method

    An Empirical Bayes Method for Chi-Squared Data Lilun Du and Inchi Hu∗ Hong Kong University of Science and Technology, Hong Kong May 27, 2021 Abstract In a thought-provoking paper, Efron (2011) investigated the merit and limitation of an empirical Bayes method to correct selection bias based on Tweedie’s formula first reported in Robbins (1956). The exceptional virtue of Tweedie’s formula for the normal distribution lies in its representation of selection bias as a simple function of the derivative of log marginal likelihood. Since the marginal likelihood and its derivative can be estimated from the data directly without invoking prior information, bias correction can be carried out conveniently. We propose a Bayesian hierarchical model for chi-squared data such that the resulting Tweedie’s formula has the same virtue as that of the normal distribution. Because the family of noncentral chi- squared distributions, the common alternative distributions for chi-squared tests, does not constitute an exponential family, our results cannot be obtained by extending arXiv:1903.00776v2 [stat.ME] 26 May 2021 existing results. Furthermore, the corresponding Tweedie’s formula manifests new phenomena quite different from those of the normal distribution and suggests new ways of analyzing chi-squared data. Keywords: False discovery rate; High dimensional data analysis; Large-scale inference; Post-selection inference; Selection bias; Tweedie’s formula ∗Address for correspondence: Inchi Hu, Department of Information Systems, Business Statistics and Operations Management, Hong Kong University of Science and Technology, Hong Kong. E-mail: [email protected] 1 1 Introduction In this paper, we take the chi-squared test to be any statistical test such that the test statistic under the null hypothesis is approximately chi-squared distributed.
  • Corrected Portmanteau Tests for Var Models with Time-Varying Variance

    Corrected Portmanteau Tests for Var Models with Time-Varying Variance

    CORRECTED PORTMANTEAU TESTS FOR VAR MODELS WITH TIME-VARYING VARIANCE Valentin Patileaa and Hamdi Raïssib∗ a CREST (Ensai) & IRMAR (UEB) b IRMAR-INSA May 2011 Abstract The problem of test of fit for Vector AutoRegressive (VAR) processes with unconditionally heteroscedastic errors is studied. The volatility structure is deterministic but time-varying and allows for changes that are commonly observed in economic or financial multivariate series such as breaks or smooth transitions. Our analysis is based on the residual autocovariances and autocorrelations obtained from Ordinary Least Squares (OLS), Generalized Least Squares (GLS) and Adaptive Least Squares (ALS) estimation of the autoregressive parameters. The OLS residuals are the standards estimates of the VAR model errors. To build the GLS residuals we use the GLS estimate of the VAR coefficients to estimate the model errors that we further standardize by the time-varying volatility. Hence, the GLS estimates require the knowledge of the variance structure. The ALS approach is the GLS approach adapted to the unknown time-varying volatility that is then estimated by kernel smoothing. The properties of the three types of residual autocovariances and autocorrelations are derived. In particular it is shown that the ALS and GLS residual autocorrelations are asymptotically equivalent. It is also found that the asymptotic distribution of the OLS residual autocorrelations can be quite different from the standard chi-square asymptotic distribution obtained in a correctly specified VAR model with iid innovations. As a consequence the standard portmanteau tests are unreliable in our framework. The correct critical values of the standard portmanteau tests based on the OLS residuals are derived.
  • Comparative Study of Portmanteau Tests for ARMA Models اﻟذاﺗﻲ ﻟﻸﺧطﺎء

    Comparative Study of Portmanteau Tests for ARMA Models اﻟذاﺗﻲ ﻟﻸﺧطﺎء

    Gaza ـــ Al- Azhar University Deanship of Postgraduate Studies & Scientific Research Faculty of Economics and Administrative sciences Department of Applied Statistics Comparative Study of Portmanteau Tests for ARMA Models دراﺳﺔ ﻣﻘﺎرﻧﺔ ﻻﺧﺗﺑﺎرات اﻻ رﺗﺑﺎط اﻟذاﺗﻲ ﻟﻸﺧطﺎء اﻟﻌﺷواﺋﯾﺔ ﻟﻧﻣﺎذج اﻵرﻣﺎ Presented By Alaa Ahmad Salman Al-Reqep Supervised By Samir Khaled Safi, Ph.D. Associate Professor of Statistics SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF APPLIED STATISTICS 2013-1435 To My Family Acknowledgment Foremost, I would to thank God for the wisdom and perseverance that he has been bestowed upon me during this thesis, and indeed, throughout my life. I am grateful to a number of people who have guided and supported me. My first sincere gratitude and appreciation goes to my advisor Professor Samir Safi, for his patience, continuous help and support in all stages of this thesis and for encouraging and helping me to shape my interest and ideas. His recommendations and instructions have enabled me to assemble and finish my thesis effectively. Besides my advisor, I would like to thank Dr. Mahmoud Okasha and Dr. Abdalla Al Habil, I would not have been able to achieve my learning in the same manner without their immense knowledge. I would like to thank Dr. Bisher Iqelan for accepting to be my external examiner. Also I would like to express the deepest appreciation to Professor Thomas J. Fisher from Department of Mathematics at University of Missouri, Kansas City, USA for his comments and guidance for the simulation study. Also I would like to express my thanks to Mr.