Lecture 4

Estimating the Asymptotic Covariance Matrices of Maximum Likelihood

The asymptotic -covariance matrix of MLEs can be estimated by

−1   2  −1  ∂ ln L  [I (θ )] = − E  . ~ |θ =θˆ   ∂θ ∂θ′  ~ ~ ML   ~ ~ |θ =θˆ ~ ~ ML

However, this may not be available because of the difficulty in calculating the necessary expectations to get the information matrix. There do, however, exist two alternative consistent estimators.

Alternative 1 (Direct use of Hessian)

−1 2 −1 −1  ∂ ln L(θ ) [Iˆ(θˆ)] = [− H (θˆ)] = − ~  ~ ~  ∂θ ∂θ′   ~ ~ |θ =θˆ ~ ~

where plimθˆ = θ . ~ ~

Of course, θˆ could be chosen as the maximum likelihood estimator θˆ . ~ ~ ML

Alternative 2 (BHHH or outer product of gradients)

− −1 N 1  ˆ ˆ    ˆ ˆ −1 I (θ ) = gˆi gˆi′ = [GG′]  ~  ∑     i=1 ~ ~ 

  ∂ln L X i ;θ    ~ ˆ = ˆ ˆ  ˆ ˆ =  ~  where G g1 g2 gN  and gi  ~ ~ ~  ~ ∂θ ~ |θ =θ ~ ~ ML

The first alternative requires the derivation of the second derivations of the log likelihood function (i.e., the Hessian matrix) while the second alternative only requires

1 the derivation of the first derivatives of the log likelihood function (i.e., the gradient vector).

Alternative 3 (QMLE/Sandwich Variance-Covariance Matrix)

For the Type 2 ML estimator (see Davidson and McKinnon (2004, p. 406)) it can be shown that 1/ 2 ˆ −1 −1 Var( p lim n (θ −θ 0 )) = H (θ 0 ) I(θ 0 )H (θ 0 ) . n→∞ ~ ~ ~ ~ ~ Therefore, a consistent estimate of the variance-covariance matrix of the Type 2 ML estimator is ˆ ˆ −1 ˆ ˆ ˆ −1 VaˆrQMLE (θ ) = [H (θ )] GG'[H (θ )] . ~ ~ ~ (A Type 2 ML estimator is defined as a solution to the first order conditions for the

likelihood function, namely, g(x,θˆ) = 0, where g(x,θˆ) is the gradient (score) vector ~ ~ ~ ~ ~ ~ ~

of the likelihood function evaluated at θ = θˆ .) In the case that the likelihood ~ ~

function has been properly specified, this estimator has little to recommend it since it is more tedious to calculate than the BHHH estimate. However, unlike the information matrix estimate or the Hessian and BHHH estimates, it is still valid even when the information matrix equality does not hold. Since this equality does not tend to hold when the likelihood function is misspecified, the QMLE estimate of the covariance matrix can still be consistent under general circumstances. See White (1982) and Gourieroux, Monfort, and Trognon (1984) for further discussion. Thus, when the proper specification of the likelihood function is in doubt, one would be well served to compare the inferences drawn for the conventional ML covariance estimates with the inferences drawn when using the QMLE covariance matrix.

Asymptotic Inference for Nonlinear Functions of Parameters - The Delta Method

A) The Univariate Delta method

Let θˆ be a consistent, asymptotically normal estimate of θ . Then,

2 asy Theorem: If N (θˆ −θ )→ N(0, σ 2 ) and if c(θ ) is a of θ

not dependent on N , then the distribution of c(θˆ) is given by

asy 2 dc(θ ) N (c(θˆ)− c(θ ))→ N(0, [c′(θ )] σ 2 ) where c′(θ ) = . dθ

2 Obviously, a consistent estimate of Var(c(θˆ)) is [c′(θˆ)] σˆ 2 .

B) The Multivariate Delta method

Let θ be a K ×1 vector of parameters. Furthermore, let θˆ be a consistent, ~ ~ asymptotically normal estimate of θ . Then, ~

asy Theorem: If N (θˆ−θ )→ N(0, Σ) and if c(θ ) is a J ×1, J ≤ K , continuous ~ ~ ~ ~ ~

function of θ not dependent on N , then ~

asy ′ N (c(θˆ)− c(θ ))→ N0, C(θ ) ΣC(θ )  ~ ~ ~ ~  ~ ~ ~ 

dc1 (θ ) dc2 (θ ) dcJ (θ ) where C(θ )=  ~ ~  ~  . ~  d θ d θ d θ   ~ ~ ~  ′ Obviously, a consistent estimate of Var(c(θˆ)) is given by C(θˆ) ΣˆC(θˆ) , where Σˆ ~ ~ ~ ~ is a of Σ and C(θˆ) is a consistent estimator of C(θ ) . ~ ~

3