Lecture 2: ARMA(P,Q) Models (Part 2)

Home , Moving average

Lecture 2: ARMA(p,q) models (part 2)

Florian Pelgrin

University of Lausanne, Ecole´ des HEC Department of mathematics (IMEA-Nice)

Sept. 2011 - Jan. 2012

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 1 / 40 Introduction Motivation

Characterize the main properties of MA(q) models.

Estimation of MA(q) models

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 2 / 40 Introduction Road map

1 Introduction

2 MA(1) model

3 Application of a ”counterfactual” MA(1)

4 Moving average model of order q, MA(q)

5 Application of a MA(q) model

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 3 / 40 MA(1) model Moving average models 2.1. Moving average model of order 1, MA(1)

Deﬁnition

A stochastic process (Xt )t∈Z is said to be a moving average model of order 1 if it satisﬁes the following equation :

Xt = µ + t − θt−1 ∀t

where θ 6= 0, µ is a constant term, (t )t∈Z is a weak white noise process 2 2 with expectation zero and variance σ (t ∼ WN(0, σ )).

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 4 / 40 MA(1) model

Remarks : 1. In lag notation, one has :

Xt = µ + Θ(L)t ≡ µ + (1 − θL)t

2. The previous process can be written in mean-deviation as follows :

X˜t = t − θt−1

where

X˜t = Xt − µ.

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 5 / 40 MA(1) model

Remarks (cont’d) :

3. The properties of (Xt ) only depend on those of the weak white noise process (t ). To some extent, the behavior of (Xt ) is more noisy relative to an AR(1) process... 4. Iterating on the past inﬁnite (and with some regularity conditions), the inﬁnite autoregressive representation writes :

∞ µ X X = + θk X + . t 1 − θ t−k t k=1 5. The inﬁnite autoregressive representation illustrates the fact that a certain form of persistence is captured by a moving average model, especially when θ is close to one.

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 6 / 40 MA(1) model

Simulation of a moving average process of order 1 (θ = 0.9)

4 3 2 1 0 -1 -2 -3 -4 -5 50 100 150 200 250 300 350 400 450 500

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 7 / 40 MA(1) model

Scatter plots of a moving average process of order 1: Left panel (X t-1 versus X t) and right panel (X t-2 versus X t)

4 4

3 3

2 2

1 1

0 0

-1 -1 X _(t-2) X _(t-1)

-2 -2

-3 -3

-4 -4

-5 -5 -5 -4 -3 -2 -1 0 1 2 3 4 -5 -4 -3 -2 -1 0 1 2 3 4

X_t X_t

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 8 / 40 Figure: Scatter plots MA(1) model Stationarity and invertibility conditions

Since (t ) is a weak noise process, (Xt ) is weakly stationary (by deﬁnition).

The invertibility condition is the counterpart of the stability (stationary) condition of an AR(1) process :

1 If |θ| < 1, then (Xt ) is invertible.

2 If |θ| = 1, then (Xt ) is non invertible.

3 If |θ| > 1, there exists a non-causal invertible representation of (Xt ) that we rule out.

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 9 / 40 MA(1) model

Alternatively, if |θ| < 1, then :

∞ X (1 − θL)−1 = θk Lk k=0 and

−1 t = (1 − θL) (Xt − µ) i.e. ∞ µ X X = + θk X + t 1 − θ t−k t k=1

1 This is the inﬁnite autoregressive representation of a MA(1) process. 2 The MA(1) representation is then called the fundamental or causal representation.

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 10 / 40 MA(1) model

More formally... Deﬁnition The representation of the moving average process of order one deﬁned by :

Xt = µ + t − θt−1, is said to be causal or fundamental—(t ) is the innovation process—if the root of the characteristic equation zΘ(z−1) = 0 ≡ z − θ = 0 lies outside the unit circle :

|z| < 1 ⇔ |θ| < 1.

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 11 / 40 MA(1) model

Remark : One can also use the inverse characteristic equation to ﬁnd the invertibility condition :

Θ(z) = 0 ⇔ 1 − θz = 0.

The condition writes (for a MA(1) process) :

|z∗| > 1 ⇔ |θ| < 1.

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 12 / 40 MA(1) model Moments of a MA(1)

Deﬁnition

Let (Xt ) be a stationary stochastic process that satisﬁes a (fundamental) MA(1) representation, Xt = µ + t − θt−1. Then :

E [Xt ] = µ 2 2 V [Xt ] = (1 + θ )σ 2 γX (1) = −θσ γX (h) = 0 for all |h| > 1

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 13 / 40 MA(1) model

Deﬁnition

Let (Xt ) be a stationary stochastic process that satisﬁes a (fundamental) MA(1) representation, Xt = µ + t − θt−1. Then, the autocorrelation function is given by :

 1 if h = 0  θ ρX (h) = − 1+θ2 if h = ±1  0 otherwise.

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 14 / 40 MA(1) model

The autocorrelation function of a moving average process of order 1, MA(1), is always zero for orders higher than 1 (|h| > 1) : MA(1) process has no memory beyond 1 period (see Scatter plots and autocorrelograms). This property generalizes to MA(p) processes.

Partial autocorrelations : Nothing special, with the exception that it should decrease (possibly, with damped oscillations) ! The partial autocorrelation function cannot help for characterizing a MA(1).

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 15 / 40 MA(1) model

Correlograms of a moving average process of order one ( θ = 0.9, 0.5, and 0.2)

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 16 / 40 Figure: (Partial) Autocorrelation function MA(1) model

Correlogram of an AR(1) with phi=0.8 Correlogram of an AR(1) with phi=-0.8 0.8 1

0.6 0.5

0.4 0 0.2

-0.5 0

-0.2 -1 0 5 10 15 20 0 5 10 15 20

Correlogram of an MA(1) with theta=0.8 Correlogram of an MA(2) with theta=(0.4;0.3) 0.2 0.2

0.1 0

0 -0.2 -0.1

-0.4 -0.2

-0.6 -0.3 0 5 10 15 20 0 5 10 15 20

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 17 / 40 Figure: AR(1) versus MA(1) and MA(2) MA(1) model Estimation

Estimation is ”more diﬃcult” since the t terms are not observed !

Diﬀerent techniques : 1 Conditional nonlinear least squares estimator 2 Maximum likelihood estimator 3 Generalized method of moments estimator.

Without loss of generality, the constant term is omitted and the model is written as :

Xt = t + θt−1.

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 18 / 40 MA(1) model Nonlinear conditional least squares estimator

The ”objective function” of the ordinary least squares estimator is :

T X 2 θˆmco = argmin (xt − θt−1) θ t=2

Conditionally on 0, one has (backcasting procedure) :

t−2 X j t−1 t−1 = (−θ) xt−1−j + (−θ) 0 j=0

Suppose that 0 = 0, the nonlinear objective function (with respect to θ) writes :

2 T  t−2  X X j xt − θ (−θ) xt−1−j  t=2 j=0

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 19 / 40 MA(1) model

The conditional nonlinear least squares estimator of θ is deﬁned by :

2 T  t−2  ˆ X X j θcnls = argmin xt − θ (−θ) xt−1−j  θ t=2 j=0

The asymptotic distribution is given by : √ a.d. 2 T θˆcnls − θ → N (0, 1 − θ )

The eﬀect of 0 = 0 dies out if T is suﬃciently large. An alternative is to consider 0 as an unknown parameter. 2 An estimator of σ is :

T 1 X σˆ2 = (x − θˆ )2. T − 1 t cnls t−1 t=2

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 20 / 40 MA(1) model Maximum likelihood estimator

Two estimators : the conditional maximum likelihood estimator and the exact maximum likelihood estimator The conditional maximum likelihood estimator proceeds in the same way as the conditional nonlinear least squares estimator (backcasting procedure) :

Suppose that t is a Gaussian White noise process For t = 1 :

1 = x1 − θ0

For t > 1 :

t−1 X j t t = (−θ) xt−j + (−θ) 0 j=0

Write the conditional likelihood function (with 0 = 0) and maximize 2 with respect to θ and σ .

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 21 / 40 MA(1) model

The exact maximum likelihood estimator can be calculated by two convenient algorithms : 1 The Kalman ﬁlter

2 The triangular factorization of the variance-covariance matrix of a MA(1) process

In contrast to the conditional maximum likelihood estimator, the exact maximum likelihood estimator does not require that the moving average representation is invertible.

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 22 / 40 MA(1) model The (Generalized) method of moments estimator

A simple method of moments estimator... Consider the ﬁrst two moments of a MA(1) :

2 2 2 2 E Xt = (1 + θ )σ and E [Xt Xt−1] = θσ Using the empirical counterpart of these two moments conditions yields :

T 2 2 2 2 −1 X xt − σ (1 + θ ) gT (x; θ, σ ) = T 2 xt xt−1 − σ θ t=1 −1 PT 2 2 2 ! T t=1 xt − σ (1 + θ ) = −1 PT 2 T t=2 xt xt−1 − σ θ

ˆ 2 Solving the exactly (just-) identiﬁed equation gT (x; θ, σˆ ) = 02×1 for ˆ 2 θ andσ ˆ (with sone regularity conditions...) gives the method of moment estimator. Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 23 / 40 MA(1) model

Estimation of moving average processes of order 1

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 24 / 40 Figure: Estimation using Monte Carlo simulations Application of a ”counterfactual” MA(1) 3. Application of a ”counterfactual” MA(1)

Eﬀective Fed fund rate : 1970 :01-2010 :01 (monthly observations)

As to be expected from the (partial) autocorrelogram function (and thus theory !), a moving average model of order 1 is not probably the most appropriate model...

However, it is interesting to compare it with the AR(1) speciﬁcation of the eﬀective Fed fund rate.

The estimate of the constant term (respectively, θ) is 6.284 (respectively, 0.941). Both estimates are statistically signiﬁcant.

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 25 / 40 Application of a ”counterfactual” MA(1)

Effective Fed fund rate---MA(1) model

15 12 10 8 5 4 0 0

-4 1970 1975 1980 1985 1990 1995 2000 2005 Residual Actual Fitted

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 26 / 40 Application of a ”counterfactual” MA(1)

Effective Fed fund rate---diagnostics of a MA(1) model

1.0

0.8

0.6

0.4

0.2 Autocorrelation

0.0 2 4 6 8 10 12 14 16 18 20 22 24

Actual Theoretical

1.2

0.8

0.4

0.0

Partial autocorrelation Partial -0.4 2 4 6 8 10 12 14 16 18 20 22 24

Actual Theoretical

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 27 / 40 Application of a ”counterfactual” MA(1)

Effective Fed fund rate---impulse response function of the estimated MA(1) model

Impulse Response ± 2 S.E. 2.5 2.0 1.5 1.0 0.5 0.0 2 4 6 8 10 12 14 16 18 20 22 24

Accumulated Response ± 2 S.E. 4.0 3.5 3.0 2.5 2.0 1.5 2 4 6 8 10 12 14 16 18 20 22 24

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 28 / 40 Moving average model of order q, MA(q) Moving average models 4. Moving average model of order q, MA(q)

Deﬁnition

A stochastic process (Xt )t∈Z is said to be a moving average model of order q if it satisﬁes the following equation :

Xt = µ + t + θ1t−1 + ··· + θqt−q ∀t

= µ + Θ(L)t

where θq 6= 0, µ is a constant term, (t )t∈Z is a weak white noise process 2 2 with expectation zero and variance σ (t ∼ WN(0, σ )), and q Θ(L) = 1 + θ1L + ··· + θqL .

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 29 / 40 Moving average model of order q, MA(q)

Remark : One can also use the notation

∗ ∗ Xt = µ + t − θ1t−1 − · · · − θqt−q ∀t ∗ = µ + Θ (L)t where

∗ θj = −θj for j = 1, ··· , q ∗ θq 6= 0 ∗ ∗ ∗ q Θ (L) = 1 − θ1L − · · · − θqL 2 t ∼ WN(0, σ ).

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 30 / 40 Moving average model of order q, MA(q) Stationarity and invertibility conditions

A MA(q) process is always weakly stationary irrespective of the moving average part

A MA(q) process is invertible if all the roots of the characteristic equation zqΘ(z−1) = 0 are of modulus less than one :

q q−1 2 z + θ1z + θ2λ + ··· + θq = 0 ⇔ |zi | < 1

for i = 1, ··· , q.

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 31 / 40 Moving average model of order q, MA(q)

Deﬁnition The representation of the moving average process of order q deﬁned by :

Xt = µ + t + θ1t−1 + ··· + φqt−q, is said to be causal or fundamental—(t ) is the innovation process–all the roots of the characteristic equation zqΘ(z−1) = 0 are of modulus less than one :

q q−1 2 z + θ1z + θ2λ + ··· + θq = 0 ⇔ |zi | < 1 for i = 1, ··· , q.

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 32 / 40 Moving average model of order q, MA(q) Moments of stationary MA(q)

Mean and autocovariances Using the insights of a MA(1) model, one gets :

E(Xt ) = µ  q  σ2 1 + Pθ2 if h = 0  i  i=1  q !  2 P γX (h) = σ θh + θi θi−h if 1 ≤ |h| < q  i=h+1  2  θqσ if |h| = q  0 if |h| > q

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 33 / 40 Moving average model of order q, MA(q)

Autocorrelations  1 if h = 0  q  P  θh+ θi θi−h  i=h+1  q if 1 ≤ |h| < q  P 2 1+ θi ρX (h) = i=1  θq  q if |h| = q  1+Pθ2  i  i=1  0 if |h| > q

The autocorrelation function of a moving average process of order q, MA(q), is always zero for orders high than q (|h| > q) : MA(q) process has no memory beyond q periods.

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 34 / 40 Moving average model of order q, MA(q)

Partial autocorrelations : Nothing special ! The theoretical partial autocorrelation of an MA(q) dies out as h → ∞ but is not exactly zero as an AR(p) process for h > p.

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 35 / 40 Application of a MA(q) model 5. Application

Eﬀective Fed fund rate : 1970 :01-2010 :01 (monthly observations)

As to be expected from the (partial) autocorrelogram function (and thus theory !), a moving average model of order q is probably not the most appropriate model...

However, it is interesting for comparison purposes.

In particular, increasing the number of lags augments the overall persistence at the expense of over-parameterizing the model...

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 36 / 40 Application of a MA(q) model

ML estimation of the eﬀective Fed fund rate : MA(4) and MA(8) Coeﬃcients Estimates Std. Error P-value µ 6.232 0.237 0.000 θ1 1.812 0.034 0.000 ...... θ4 0.607 0.035 0.000 µ 6.229 0.357 0.000 θ1 1.719 0.046 0.000 θ2 2.167 0.087 0.000 ...... θ8 0.078 0.045 0.087

All parameters are statistically signiﬁcant at 1%, with the exception of θ8...All the roots are outside the unit circle.

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 37 / 40 Application of a MA(q) model

Effective Fed fund rate---MA(4) model

20 15 10 6 5 4 0 2 0 -2 -4 1970 1975 1980 1985 1990 1995 2000 2005 Residual Actual Fitted

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 38 / 40 Application of a MA(q) model

Effective Fed fund rate: diagnostics MA(4) MA(8)

1.0 1.0

0.8 0.8

0.6 0.6

0.4 0.4

0.2

0.2 Autocorrelation Autocorrelation

0.0 0.0 2 4 6 8 10 12 14 16 18 20 22 24 2 4 6 8 10 12 14 16 18 20 22 24

Actual Theoretical Actual Theoretical

1.0 1.0

0.5 0.5

0.0 0.0

-0.5 -0.5 Partial autocorrelation Partial-1.0 autocorrelation -1.0 2 4 6 8 10 12 14 16 18 20 22 24 2 4 6 8 10 12 14 16 18 20 22 24

Actual Theoretical Actual Theoretical

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 39 / 40 Application of a MA(q) model

Effective Fed fund rate---impulse response functions

MA(4) MA(8)

Impulse Response ± 2 S.E. Impulse Response ± 2 S.E.

2.0 1.6

1.2 1.5

0.8 1.0

0.4

0.5 0.0

0.0 -0.4 2 4 6 8 10 12 14 16 18 20 22 24 2 4 6 8 10 12 14 16 18 20 22 24

Accumulated Response ± 2 S.E. Accumulated Response ± 2 S.E.

6 10

5 8

4 6 3 4 2

2 1

0 0 2 4 6 8 10 12 14 16 18 20 22 24 2 4 6 8 10 12 14 16 18 20 22 24

Florian Pelgrin (HEC) Univariate time series Sept. 2011 - Jan. 2012 40 / 40