Robust and Clustered Standard Errors

Molly Roberts

March 6, 2013

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 1 / 35 An Introduction to Robust and Clustered Standard Errors Outline

1 An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance GLM’s and Non-constant Variance Cluster-Robust Standard Errors

2 Replicating in R

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 2 / 35 An Introduction to Robust and Clustered Standard Errors Outline

1 An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance GLM’s and Non-constant Variance Cluster-Robust Standard Errors

2 Replicating in R

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 3 / 35 Errors are the vertical distances between observations and the unknown Conditional Expectation Function. Therefore, they are unknown.

Residuals are the vertical distances between observations and the estimated regression function. Therefore, they are known.

An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Review: Errors and Residuals

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 4 / 35 Residuals are the vertical distances between observations and the estimated regression function. Therefore, they are known.

An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Review: Errors and Residuals

Errors are the vertical distances between observations and the unknown Conditional Expectation Function. Therefore, they are unknown.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 4 / 35 An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Review: Errors and Residuals

Errors are the vertical distances between observations and the unknown Conditional Expectation Function. Therefore, they are unknown.

Residuals are the vertical distances between observations and the estimated regression function. Therefore, they are known.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 4 / 35 y = Xβ + u u = y − Xβ

Residuals represent the difference between the outcome and the estimated mean.

y = Xβˆ + uˆ uˆ = y − Xβˆ

An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Notation

Errors represent the difference between the outcome and the true mean.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 5 / 35 An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Notation

Errors represent the difference between the outcome and the true mean.

y = Xβ + u u = y − Xβ

Residuals represent the difference between the outcome and the estimated mean.

y = Xβˆ + uˆ uˆ = y − Xβˆ

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 5 / 35 Residuals represent the difference between the outcome and the estimated mean.

y = Xβˆ + uˆ uˆ = y − Xβˆ

An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Notation

Errors represent the difference between the outcome and the true mean.

y = Xβ + u u = y − Xβ

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 5 / 35 y = Xβˆ + uˆ uˆ = y − Xβˆ

An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Notation

Errors represent the difference between the outcome and the true mean.

y = Xβ + u u = y − Xβ

Residuals represent the difference between the outcome and the estimated mean.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 5 / 35 An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Notation

Errors represent the difference between the outcome and the true mean.

y = Xβ + u u = y − Xβ

Residuals represent the difference between the outcome and the estimated mean.

y = Xβˆ + uˆ uˆ = y − Xβˆ

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 5 / 35 An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Variance of βˆ depends on the errors

− βˆ = X0X 1 X0y − = X0X 1 X0(Xβ + u) − = β + X0X 1 X0u

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 6 / 35 An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Variance of βˆ depends on the errors

− V [βˆ] = V [β] + V [X0X 1 X0u] − = 0 + V [X0X 1 X0u] − − − − = E[X0X 1 X0uu0X X0X 1] − E[X0X 1 X0u]E[X0X 1 X0u]0 − − = E[X0X 1 X0uu0X X0X 1] − 0

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 7 / 35 An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Variance of βˆ depends on the errors (continued)

− V [βˆ] = V [β] + V [X0X 1 X0u] − = 0 + V [X0X 1 X0u] − − − − = E[X0X 1 X0uu0X X0X 1] − E[X0X 1 X0u]E[X0X 1 X0u]0 − − = E[X0X 1 X0uu0X X0X 1] − 0 − − = X0X 1 X0E[uu0]X X0X 1 − − = X0X 1 X0ΣX X0X 1

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 8 / 35 u ∼ Nn(0, Σ)

 σ2 0 0 ... 0   0 σ2 0 ... 0  Σ = Var(u) = E[uu0] =    .   .  0 0 0 . . . σ2 What does this mean graphically for a CEF with one explanatory variable?

An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Constant Error Variance and Dependence

Under standard OLS assumptions,

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 9 / 35  σ2 0 0 ... 0   0 σ2 0 ... 0  Σ = Var(u) = E[uu0] =    .   .  0 0 0 . . . σ2 What does this mean graphically for a CEF with one explanatory variable?

An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Constant Error Variance and Dependence

Under standard OLS assumptions,

u ∼ Nn(0, Σ)

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 9 / 35 What does this mean graphically for a CEF with one explanatory variable?

An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Constant Error Variance and Dependence

Under standard OLS assumptions,

u ∼ Nn(0, Σ)

 σ2 0 0 ... 0   0 σ2 0 ... 0  Σ = Var(u) = E[uu0] =    .   .  0 0 0 . . . σ2

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 9 / 35 An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Constant Error Variance and Dependence

Under standard OLS assumptions,

u ∼ Nn(0, Σ)

 σ2 0 0 ... 0   0 σ2 0 ... 0  Σ = Var(u) = E[uu0] =    .   .  0 0 0 . . . σ2 What does this mean graphically for a CEF with one explanatory variable?

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 9 / 35 An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Evidence of Non-constant Error Variance (4 examples)

● ●

● 1.5 ● ● ● ● ● 2.0 ●● ● ● ● ● ● ● ● ● ● ●● ●●● ● ● ● ●● ●●●● ●● ●● ● ●● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0.5 ● ● ● ● ● ● ●● ●● 1.0 ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● y ● ● ● y ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ●●●● ●● ● ● ●● ● ● ● ● ●●● ●●● ● ● ● ● ● ● ● ● ● −0.5 ● ● 0.0 ● ● ● ● ● ● ● ● ● ● ● ● ● −1.5 −1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 x x

● ● ● ● ● ● ● ●●● 1.2 1.0 ● ● ●●● ● ● ●● ●● ● ● ● ●● ● ● ● ●●● ● ● ● ● ● ● ● 0.8 ● ●●● ●● ● ● ● ● ●●● ● ● ● ● ● ● ●● ● 0.8 ● ●● ●

0.6 ● ●●● ● ●● ●●●● ● ● ● ● ●●●●● ● y ● y ● ● ●●● ● ● ●●● ● ● ●●

● 0.4 ● ● ● ●●●● ● ●●● ● ● ● ● ●● ● ●●●● ● ● ● 0.4 ● ●● ● ● ● ● ●● 0.2 ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ●●●● ● ● ● ● ● ● ● ● ● 0.0 ● ● ●● ● ● 0.0

0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 x x

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 10 / 35  σ2 0 0 ... 0   0 σ2 0 ... 0  Var(u) = E[uu0] =    .   .  0 0 0 . . . σ2

In this section we will allow violations of this assumption in the following heteroskedastic form.

 2  σ1 0 0 ... 0  0 σ2 0 ... 0  Var(u) = E[uu0] =  2   .   .  2 0 0 0 . . . σn

An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Notation

The constant error variance assumption sometimes called homoskedasiticity states that

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 11 / 35 In this section we will allow violations of this assumption in the following heteroskedastic form.

 2  σ1 0 0 ... 0  0 σ2 0 ... 0  Var(u) = E[uu0] =  2   .   .  2 0 0 0 . . . σn

An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Notation

The constant error variance assumption sometimes called homoskedasiticity states that

 σ2 0 0 ... 0   0 σ2 0 ... 0  Var(u) = E[uu0] =    .   .  0 0 0 . . . σ2

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 11 / 35  2  σ1 0 0 ... 0  0 σ2 0 ... 0  Var(u) = E[uu0] =  2   .   .  2 0 0 0 . . . σn

An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Notation

The constant error variance assumption sometimes called homoskedasiticity states that

 σ2 0 0 ... 0   0 σ2 0 ... 0  Var(u) = E[uu0] =    .   .  0 0 0 . . . σ2

In this section we will allow violations of this assumption in the following heteroskedastic form.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 11 / 35 An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Notation

The constant error variance assumption sometimes called homoskedasiticity states that

 σ2 0 0 ... 0   0 σ2 0 ... 0  Var(u) = E[uu0] =    .   .  0 0 0 . . . σ2

In this section we will allow violations of this assumption in the following heteroskedastic form.

 2  σ1 0 0 ... 0  0 σ2 0 ... 0  Var(u) = E[uu0] =  2   .   .  2 0 0 0 . . . σn

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 11 / 35 I For “α” level tests, probability of Type I error will not be α.

I “1 − α” conﬁdence intervals will not have 1 − α coverage probability.

I The LS estimator is no longer BLUE.

However,

I The degree of the problem depends on the amount of heteroskedasticity.

I βˆ is still unbiased for β

An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Consequences of non-constant error variance

2 2 I The σˆ will not be unbiased for σ .

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 12 / 35 I “1 − α” conﬁdence intervals will not have 1 − α coverage probability.

I The LS estimator is no longer BLUE.

However,

I The degree of the problem depends on the amount of heteroskedasticity.

I βˆ is still unbiased for β

An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Consequences of non-constant error variance

2 2 I The σˆ will not be unbiased for σ .

I For “α” level tests, probability of Type I error will not be α.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 12 / 35 I The LS estimator is no longer BLUE.

However,

I The degree of the problem depends on the amount of heteroskedasticity.

I βˆ is still unbiased for β

An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Consequences of non-constant error variance

2 2 I The σˆ will not be unbiased for σ .

I For “α” level tests, probability of Type I error will not be α.

I “1 − α” conﬁdence intervals will not have 1 − α coverage probability.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 12 / 35 I The degree of the problem depends on the amount of heteroskedasticity.

I βˆ is still unbiased for β

An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Consequences of non-constant error variance

2 2 I The σˆ will not be unbiased for σ .

I For “α” level tests, probability of Type I error will not be α.

I “1 − α” conﬁdence intervals will not have 1 − α coverage probability.

I The LS estimator is no longer BLUE.

However,

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 12 / 35 I βˆ is still unbiased for β

An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Consequences of non-constant error variance

2 2 I The σˆ will not be unbiased for σ .

I For “α” level tests, probability of Type I error will not be α.

I “1 − α” conﬁdence intervals will not have 1 − α coverage probability.

I The LS estimator is no longer BLUE.

However,

I The degree of the problem depends on the amount of heteroskedasticity.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 12 / 35 An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Consequences of non-constant error variance

2 2 I The σˆ will not be unbiased for σ .

I For “α” level tests, probability of Type I error will not be α.

I “1 − α” conﬁdence intervals will not have 1 − α coverage probability.

I The LS estimator is no longer BLUE.

However,

I The degree of the problem depends on the amount of heteroskedasticity.

I βˆ is still unbiased for β

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 12 / 35  2  σ1 0 0 ... 0  0 σ2 0 ... 0  V [u] = Σ =  2   .   .  2 0 0 0 . . . σn then Var(βˆ) = (X0X)−1 X0ΣX (X0X)−1 and Huber (and then White) showed that  ˆ2  u1 0 0 ... 0 2 −  0 uˆ 0 ... 0  − X0X 1 X0  2  X X0X 1  .   .  2 0 0 0 ... uˆn

is a consistent estimator of V [βˆ].

An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Heteroskedasticity Consistent Estimator

Suppose

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 13 / 35 then Var(βˆ) = (X0X)−1 X0ΣX (X0X)−1 and Huber (and then White) showed that  ˆ2  u1 0 0 ... 0 2 −  0 uˆ 0 ... 0  − X0X 1 X0  2  X X0X 1  .   .  2 0 0 0 ... uˆn

is a consistent estimator of V [βˆ].

An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Heteroskedasticity Consistent Estimator

Suppose  2  σ1 0 0 ... 0  0 σ2 0 ... 0  V [u] = Σ =  2   .   .  2 0 0 0 . . . σn

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 13 / 35  ˆ2  u1 0 0 ... 0 2 −  0 uˆ 0 ... 0  − X0X 1 X0  2  X X0X 1  .   .  2 0 0 0 ... uˆn

is a consistent estimator of V [βˆ].

An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Heteroskedasticity Consistent Estimator

Suppose  2  σ1 0 0 ... 0  0 σ2 0 ... 0  V [u] = Σ =  2   .   .  2 0 0 0 . . . σn then Var(βˆ) = (X0X)−1 X0ΣX (X0X)−1 and Huber (and then White) showed that

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 13 / 35 is a consistent estimator of V [βˆ].

An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Heteroskedasticity Consistent Estimator

Suppose  2  σ1 0 0 ... 0  0 σ2 0 ... 0  V [u] = Σ =  2   .   .  2 0 0 0 . . . σn then Var(βˆ) = (X0X)−1 X0ΣX (X0X)−1 and Huber (and then White) showed that  ˆ2  u1 0 0 ... 0 2 −  0 uˆ 0 ... 0  − X0X 1 X0  2  X X0X 1  .   .  2 0 0 0 ... uˆn

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 13 / 35 An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Heteroskedasticity Consistent Estimator

is a consistent estimator of V [βˆ].

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 13 / 35 I large enough for each estimate (e.g., large enough in both treatment and baseline groups or large enough in both runoff and non-runoff groups) I large enough for consistent estimates (e.g., need n ≥ 250 for Stata default when highly heteroskedastic (Long and Ervin 2000)). 2. Doesn’t make βˆ BLUE 3. What are you going to do with predicted probabilities?

An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Things to note about this approach

1. Requires larger sample size

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 14 / 35 I large enough for consistent estimates (e.g., need n ≥ 250 for Stata default when highly heteroskedastic (Long and Ervin 2000)). 2. Doesn’t make βˆ BLUE 3. What are you going to do with predicted probabilities?

An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Things to note about this approach

1. Requires larger sample size

I large enough for each estimate (e.g., large enough in both treatment and baseline groups or large enough in both runoff and non-runoff groups)

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 14 / 35 2. Doesn’t make βˆ BLUE 3. What are you going to do with predicted probabilities?

An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Things to note about this approach

1. Requires larger sample size

I large enough for each estimate (e.g., large enough in both treatment and baseline groups or large enough in both runoff and non-runoff groups) I large enough for consistent estimates (e.g., need n ≥ 250 for Stata default when highly heteroskedastic (Long and Ervin 2000)).

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 14 / 35 An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance Things to note about this approach

1. Requires larger sample size

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 14 / 35 An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance Outline

1 An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance GLM’s and Non-constant Variance Cluster-Robust Standard Errors

2 Replicating in R

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 15 / 35 Under certain conditions, you can get the standard errors, even if your model is misspeciﬁed.

These are the robust standard errors that scholars now use for other glm’s, and that happen to coincide with the linear case.

An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance What happens when the model is not linear?

Huber (1967) developed a general way to ﬁnd the standard errors for models that are speciﬁed in the wrong way.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 16 / 35 These are the robust standard errors that scholars now use for other glm’s, and that happen to coincide with the linear case.

An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance What happens when the model is not linear?

Huber (1967) developed a general way to ﬁnd the standard errors for models that are speciﬁed in the wrong way.

Under certain conditions, you can get the standard errors, even if your model is misspeciﬁed.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 16 / 35 An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance What happens when the model is not linear?

Huber (1967) developed a general way to ﬁnd the standard errors for models that are speciﬁed in the wrong way.

Under certain conditions, you can get the standard errors, even if your model is misspeciﬁed.

These are the robust standard errors that scholars now use for other glm’s, and that happen to coincide with the linear case.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 16 / 35 ∗ I Pretend that there is heteroskedasticity on the linear model for y .

I Heteroskedasticity in the latent variable formulation will completely change the functional form of P(y = 1|x).

I What does this mean? The P(y = 1|x) 6= Φ(xβ). Your model is wrong.

An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance What does it mean for a non-linear model to have heteroskedasticity?

I Think about the probit model in the latent variable formulation.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 17 / 35 I Heteroskedasticity in the latent variable formulation will completely change the functional form of P(y = 1|x).

I What does this mean? The P(y = 1|x) 6= Φ(xβ). Your model is wrong.

An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance What does it mean for a non-linear model to have heteroskedasticity?

I Think about the probit model in the latent variable formulation. ∗ I Pretend that there is heteroskedasticity on the linear model for y .

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 17 / 35 I What does this mean? The P(y = 1|x) 6= Φ(xβ). Your model is wrong.

An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance What does it mean for a non-linear model to have heteroskedasticity?

I Think about the probit model in the latent variable formulation. ∗ I Pretend that there is heteroskedasticity on the linear model for y .

I Heteroskedasticity in the latent variable formulation will completely change the functional form of P(y = 1|x).

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 17 / 35 An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance What does it mean for a non-linear model to have heteroskedasticity?

I Think about the probit model in the latent variable formulation. ∗ I Pretend that there is heteroskedasticity on the linear model for y .

I Heteroskedasticity in the latent variable formulation will completely change the functional form of P(y = 1|x).

I What does this mean? The P(y = 1|x) 6= Φ(xβ). Your model is wrong.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 17 / 35 y ∼ fi (y|θ)

Then our likelihood function is given by

n Y fi (Yi |θ) i=1 and thus the log-likelihood is

n X L(θ) = log fi (Yi |θ) i=1

An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance RSEs for GLMs

To derive robust standard errors in the general case, we assume that

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 18 / 35 Then our likelihood function is given by

n Y fi (Yi |θ) i=1 and thus the log-likelihood is

n X L(θ) = log fi (Yi |θ) i=1

An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance RSEs for GLMs

To derive robust standard errors in the general case, we assume that

y ∼ fi (y|θ)

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 18 / 35 n Y fi (Yi |θ) i=1 and thus the log-likelihood is

n X L(θ) = log fi (Yi |θ) i=1

An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance RSEs for GLMs

To derive robust standard errors in the general case, we assume that

y ∼ fi (y|θ)

Then our likelihood function is given by

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 18 / 35 and thus the log-likelihood is

n X L(θ) = log fi (Yi |θ) i=1

An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance RSEs for GLMs

To derive robust standard errors in the general case, we assume that

y ∼ fi (y|θ)

Then our likelihood function is given by

n Y fi (Yi |θ) i=1

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 18 / 35 n X L(θ) = log fi (Yi |θ) i=1

An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance RSEs for GLMs

To derive robust standard errors in the general case, we assume that

y ∼ fi (y|θ)

Then our likelihood function is given by

n Y fi (Yi |θ) i=1 and thus the log-likelihood is

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 18 / 35 An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance RSEs for GLMs

To derive robust standard errors in the general case, we assume that

y ∼ fi (y|θ)

Then our likelihood function is given by

n Y fi (Yi |θ) i=1 and thus the log-likelihood is

n X L(θ) = log fi (Yi |θ) i=1

An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance RSEs for GLMs

We will denote the ﬁrst and second partial derivatives of L to be:

An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance RSEs for GLMs

We will denote the ﬁrst and second partial derivatives of L to be:

n n 0 X 00 X L (θ) = gi (Yi |θ), L (θ) = hi (Yi |θ) i=1 i=1

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 19 / 35 and δ2 h (Y |θ) = [logf (y|θ)]00 = log f (y|θ) i i i δθ2 i

An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance RSEs for GLMs

We will denote the ﬁrst and second partial derivatives of L to be:

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 19 / 35 and δ2 h (Y |θ) = [logf (y|θ)]00 = log f (y|θ) i i i δθ2 i

An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance RSEs for GLMs

We will denote the ﬁrst and second partial derivatives of L to be:

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 19 / 35 δ2 h (Y |θ) = [logf (y|θ)]00 = log f (y|θ) i i i δθ2 i

An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance RSEs for GLMs

We will denote the ﬁrst and second partial derivatives of L to be:

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 19 / 35 An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance RSEs for GLMs

We will denote the ﬁrst and second partial derivatives of L to be:

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 19 / 35 Remember, the Fisher information matrix is −Eθ[hi (Yi |θ)].

An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance RSEs for GLMs

This shouldn’t be too unfamiliar.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 20 / 35 An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance RSEs for GLMs

This shouldn’t be too unfamiliar.

Remember, the Fisher information matrix is −Eθ[hi (Yi |θ)].

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 20 / 35 I Then we can use the Taylor approximation for the log-likelihood function to estimate what the likelihood function looks like around θ0: 1 L(θ) = L(θ ) + L0(θ )(θ − θ ) + (θ − θ )T L00(θ )(θ − θ ) 0 0 0 2 0 0 0

An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance RSEs for GLMs

I Let’s assume the model is correct – there is a true value θ0 for θ.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 21 / 35 1 L(θ) = L(θ ) + L0(θ )(θ − θ ) + (θ − θ )T L00(θ )(θ − θ ) 0 0 0 2 0 0 0

An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance RSEs for GLMs

I Let’s assume the model is correct – there is a true value θ0 for θ. I Then we can use the Taylor approximation for the log-likelihood function to estimate what the likelihood function looks like around θ0:

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 21 / 35 An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance RSEs for GLMs

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 21 / 35 I We want to ﬁnd the maximum of the log-likelihood function, so we set L0(θ) = 0: 0 T 00 L (θ0) + (θ − θ0) L (θ0) = 0

ˆ 00 −1 0 T θ − θ0 = [−L (θ0)] L (θ0)

ˆ 00 −1 0 00 −1 Avar(θ) = [−L (θ0)] [Cov(L (θ0))][−L (θ0)]

An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance RSEs for GLMs

ˆ I We can use the Taylor approximation to approximate θ − θ0 and therefore the variance covariance matrix.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 22 / 35 0 T 00 L (θ0) + (θ − θ0) L (θ0) = 0

ˆ 00 −1 0 T θ − θ0 = [−L (θ0)] L (θ0)

ˆ 00 −1 0 00 −1 Avar(θ) = [−L (θ0)] [Cov(L (θ0))][−L (θ0)]

An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance RSEs for GLMs

ˆ I We can use the Taylor approximation to approximate θ − θ0 and therefore the variance covariance matrix.

I We want to ﬁnd the maximum of the log-likelihood function, so we set L0(θ) = 0:

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 22 / 35 ˆ 00 −1 0 T θ − θ0 = [−L (θ0)] L (θ0)

ˆ 00 −1 0 00 −1 Avar(θ) = [−L (θ0)] [Cov(L (θ0))][−L (θ0)]

An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance RSEs for GLMs

ˆ I We can use the Taylor approximation to approximate θ − θ0 and therefore the variance covariance matrix.

I We want to ﬁnd the maximum of the log-likelihood function, so we set L0(θ) = 0: 0 T 00 L (θ0) + (θ − θ0) L (θ0) = 0

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 22 / 35 ˆ 00 −1 0 00 −1 Avar(θ) = [−L (θ0)] [Cov(L (θ0))][−L (θ0)]

An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance RSEs for GLMs

ˆ I We can use the Taylor approximation to approximate θ − θ0 and therefore the variance covariance matrix.

I We want to ﬁnd the maximum of the log-likelihood function, so we set L0(θ) = 0: 0 T 00 L (θ0) + (θ − θ0) L (θ0) = 0

ˆ 00 −1 0 T θ − θ0 = [−L (θ0)] L (θ0)

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 22 / 35 An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance RSEs for GLMs

ˆ I We can use the Taylor approximation to approximate θ − θ0 and therefore the variance covariance matrix.

I We want to ﬁnd the maximum of the log-likelihood function, so we set L0(θ) = 0: 0 T 00 L (θ0) + (θ − θ0) L (θ0) = 0

ˆ 00 −1 0 T θ − θ0 = [−L (θ0)] L (θ0)

ˆ 00 −1 0 00 −1 Avar(θ) = [−L (θ0)] [Cov(L (θ0))][−L (θ0)]

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 22 / 35 An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance RSEs for GLMs

It’s the sandwich estimator.

ˆ 00 −1 0 00 −1 Avar(θ) = [−L (θ0)] [Cov(L (θ0))][−L (θ0)]

" n #−1 " n #" n #−1 X ˆ X ˆ T ˆ X ˆ = − hi (Yi |θ) gi (Yi |θ) gi (Yi |θ) − hi (Yi |θ) i=1 i=1 i=1

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 23 / 35 An Introduction to Robust and Clustered Standard Errors GLM’s and Non-constant Variance RSEs for GLMs

It’s the sandwich estimator.

" n #−1 " n #" n #−1 X ˆ X ˆ T ˆ X ˆ = − hi (Yi |θ) gi (Yi |θ) gi (Yi |θ) − hi (Yi |θ) i=1 i=1 i=1

h i−1 h i Pn ˆ Pn ˆ T ˆ Bread: − i=1 hi (Yi |θ) Meat: i=1 gi (Yi |θ) gi (Yi |θ)

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 24 / 35 Instead of summing over each individual, we ﬁrst sum over the groups.

 n  X X ˆ T ˆ  gi (Yi |θ) gi (Yi |θ)

j=1 i∈cj

An Introduction to Robust and Clustered Standard Errors Cluster-Robust Standard Errors Cluster-Robust Standard Errors

Using clustered-robust standard errors, the meat changes.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 25 / 35  n  X X ˆ T ˆ  gi (Yi |θ) gi (Yi |θ)

j=1 i∈cj

An Introduction to Robust and Clustered Standard Errors Cluster-Robust Standard Errors Cluster-Robust Standard Errors

Using clustered-robust standard errors, the meat changes.

Instead of summing over each individual, we ﬁrst sum over the groups.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 25 / 35 An Introduction to Robust and Clustered Standard Errors Cluster-Robust Standard Errors Cluster-Robust Standard Errors

Using clustered-robust standard errors, the meat changes.

Instead of summing over each individual, we ﬁrst sum over the groups.

 n  X X ˆ T ˆ  gi (Yi |θ) gi (Yi |θ)

j=1 i∈cj

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 25 / 35 An Introduction to Robust and Clustered Standard Errors Cluster-Robust Standard Errors Cluster-Robust Standard Errors

Using clustered-robust standard errors, the meat changes.

Instead of summing over each individual, we ﬁrst sum over the groups.

 n  X X ˆ T ˆ  gi (Yi |θ) gi (Yi |θ)

j=1 i∈cj

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 25 / 35 An Introduction to Robust and Clustered Standard Errors Cluster-Robust Standard Errors Cluster-Robust Standard Errors

Using clustered-robust standard errors, the meat changes.

Instead of summing over each individual, we ﬁrst sum over the groups.   X X ˆ T ˆ  gi (Yi |θ) gi (Yi |θ)

j i∈cj

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 26 / 35 Replicating in R Outline

1 An Introduction to Robust and Clustered Standard Errors Linear Regression with Non-constant Variance GLM’s and Non-constant Variance Cluster-Robust Standard Errors

2 Replicating in R

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 27 / 35 I Sometimes it’s difﬁcult to ﬁgure out what is going on in Stata.

I But by really understanding what is going on in R, you will be able to replicate once you know the equation for Stata.

Replicating in R Replicating in R

I There are lots of different ways to replicate these standard errors in R.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 28 / 35 I But by really understanding what is going on in R, you will be able to replicate once you know the equation for Stata.

Replicating in R Replicating in R

I There are lots of different ways to replicate these standard errors in R.

I Sometimes it’s difﬁcult to ﬁgure out what is going on in Stata.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 28 / 35 Replicating in R Replicating in R

I There are lots of different ways to replicate these standard errors in R.

I Sometimes it’s difﬁcult to ﬁgure out what is going on in Stata.

I But by really understanding what is going on in R, you will be able to replicate once you know the equation for Stata.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 28 / 35 load("Gov2001CodeLibrary.RData")

This particular dataset is from the paper "When preferences and commitments collide: The effect of relative partisan shifts on International treaty compliance." in International Organization by Joseph Grieco, Christopher Gelpi, and Camber Warren.

Thanks to Michele Margolis and Dan Altman for their contributions to the library!

Replicating in R Some Data

I’m going to use data from the Gov 2001 Code library.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 29 / 35 This particular dataset is from the paper "When preferences and commitments collide: The effect of relative partisan shifts on International treaty compliance." in International Organization by Joseph Grieco, Christopher Gelpi, and Camber Warren.

Thanks to Michele Margolis and Dan Altman for their contributions to the library!

Replicating in R Some Data

I’m going to use data from the Gov 2001 Code library. load("Gov2001CodeLibrary.RData")

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 29 / 35 Thanks to Michele Margolis and Dan Altman for their contributions to the library!

Replicating in R Some Data

I’m going to use data from the Gov 2001 Code library. load("Gov2001CodeLibrary.RData")

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 29 / 35 Replicating in R Some Data

I’m going to use data from the Gov 2001 Code library. load("Gov2001CodeLibrary.RData")

Thanks to Michele Margolis and Dan Altman for their contributions to the library!

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 29 / 35 fmla <- as.formula(restrict ~ art8 + shift_left + flexible + gnpcap + regnorm + gdpgrow + resgdp + bopgdp + useimfcr + surveil + univers + resvol + totvol + tradedep + military + termlim + parli + lastrest + lastrest2 + lastrest3)

fit <-glm(fmla, data=treaty1, family=binomial(link="logit"))

Replicating in R The Model

First, let’s run their model:

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 30 / 35 fit <-glm(fmla, data=treaty1, family=binomial(link="logit"))

Replicating in R The Model

First, let’s run their model:

fmla <- as.formula(restrict ~ art8 + shift_left + flexible + gnpcap + regnorm + gdpgrow + resgdp + bopgdp + useimfcr + surveil + univers + resvol + totvol + tradedep + military + termlim + parli + lastrest + lastrest2 + lastrest3)

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 30 / 35 Replicating in R The Model

First, let’s run their model:

fit <-glm(fmla, data=treaty1, family=binomial(link="logit"))

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 30 / 35 library(sandwich) bread <-vcov(fit)

For the meat, we are going to use the estimating function to create the matrices ﬁrst derivative: est.fun <- estfun(fit)

Note: if estfun doesn’t work for your glm, there is a way to do it using numericGradient().

Replicating in R The Meat and Bread

First recognize that the bread of the sandwich estimator is just the variance covariance matrix.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 31 / 35 For the meat, we are going to use the estimating function to create the matrices ﬁrst derivative: est.fun <- estfun(fit)

Note: if estfun doesn’t work for your glm, there is a way to do it using numericGradient().

Replicating in R The Meat and Bread

First recognize that the bread of the sandwich estimator is just the variance covariance matrix. library(sandwich) bread <-vcov(fit)

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 31 / 35 Replicating in R The Meat and Bread

First recognize that the bread of the sandwich estimator is just the variance covariance matrix. library(sandwich) bread <-vcov(fit)

For the meat, we are going to use the estimating function to create the matrices ﬁrst derivative: est.fun <- estfun(fit)

Note: if estfun doesn’t work for your glm, there is a way to do it using numericGradient().

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 31 / 35 meat <- t(est.fun)%*%est.fun sandwich <- bread%*%meat%*%bread And put them back in our table library(lm.test) coeftest(fit, sandwich)

Note: For the linear case, estfun() is doing something a bit different than in the logit, so use: robust <- sandwich(lm.1, meat=crossprod(est.fun)/N)

Replicating in R The Sandwich

So we can create the sandwich

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 32 / 35 And put them back in our table library(lm.test) coeftest(fit, sandwich)

Note: For the linear case, estfun() is doing something a bit different than in the logit, so use: robust <- sandwich(lm.1, meat=crossprod(est.fun)/N)

Replicating in R The Sandwich

So we can create the sandwich

meat <- t(est.fun)%*%est.fun sandwich <- bread%*%meat%*%bread

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 32 / 35 library(lm.test) coeftest(fit, sandwich)

Note: For the linear case, estfun() is doing something a bit different than in the logit, so use: robust <- sandwich(lm.1, meat=crossprod(est.fun)/N)

Replicating in R The Sandwich

So we can create the sandwich

meat <- t(est.fun)%*%est.fun sandwich <- bread%*%meat%*%bread And put them back in our table

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 32 / 35 Replicating in R The Sandwich

So we can create the sandwich

meat <- t(est.fun)%*%est.fun sandwich <- bread%*%meat%*%bread And put them back in our table library(lm.test) coeftest(fit, sandwich)

Note: For the linear case, estfun() is doing something a bit different than in the logit, so use: robust <- sandwich(lm.1, meat=crossprod(est.fun)/N)

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 32 / 35 First, we have to identify our clusters: fc <- treaty1$imf_ccode m <- length(unique(fc)) k <- length(coef(fit))

Then, we sum the u’s by cluster u <- estfun(fit) u.clust <- matrix(NA, nrow=m, ncol=k) for(j in 1:k){ u.clust[,j] <- tapply(u[,j], fc, sum) }

Replicating in R Clustered Standard Errors

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 33 / 35 fc <- treaty1$imf_ccode m <- length(unique(fc)) k <- length(coef(fit))

Then, we sum the u’s by cluster u <- estfun(fit) u.clust <- matrix(NA, nrow=m, ncol=k) for(j in 1:k){ u.clust[,j] <- tapply(u[,j], fc, sum) }

Replicating in R Clustered Standard Errors

First, we have to identify our clusters:

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 33 / 35 Then, we sum the u’s by cluster u <- estfun(fit) u.clust <- matrix(NA, nrow=m, ncol=k) for(j in 1:k){ u.clust[,j] <- tapply(u[,j], fc, sum) }

Replicating in R Clustered Standard Errors

First, we have to identify our clusters: fc <- treaty1$imf_ccode m <- length(unique(fc)) k <- length(coef(fit))

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 33 / 35 u <- estfun(fit) u.clust <- matrix(NA, nrow=m, ncol=k) for(j in 1:k){ u.clust[,j] <- tapply(u[,j], fc, sum) }

Replicating in R Clustered Standard Errors

First, we have to identify our clusters: fc <- treaty1$imf_ccode m <- length(unique(fc)) k <- length(coef(fit))

Then, we sum the u’s by cluster

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 33 / 35 Replicating in R Clustered Standard Errors

First, we have to identify our clusters: fc <- treaty1$imf_ccode m <- length(unique(fc)) k <- length(coef(fit))

Then, we sum the u’s by cluster u <- estfun(fit) u.clust <- matrix(NA, nrow=m, ncol=k) for(j in 1:k){ u.clust[,j] <- tapply(u[,j], fc, sum) }

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 33 / 35 cl.vcov <- vcov %*% ((m / (m-1)) * t(u.clust) %*% (u.clust)) %*% + vcov And test our coefﬁcients coeftest(fit, cl.vcov)

Replicating in R

Last, we can make our cluster robust matrix:

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 34 / 35 And test our coefﬁcients coeftest(fit, cl.vcov)

Replicating in R

Last, we can make our cluster robust matrix: cl.vcov <- vcov %*% ((m / (m-1)) * t(u.clust) %*% (u.clust)) %*% + vcov

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 34 / 35 coeftest(fit, cl.vcov)

Replicating in R

Last, we can make our cluster robust matrix: cl.vcov <- vcov %*% ((m / (m-1)) * t(u.clust) %*% (u.clust)) %*% + vcov And test our coefﬁcients

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 34 / 35 Replicating in R

Last, we can make our cluster robust matrix: cl.vcov <- vcov %*% ((m / (m-1)) * t(u.clust) %*% (u.clust)) %*% + vcov And test our coefﬁcients coeftest(fit, cl.vcov)

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 34 / 35 I But it’s good to know what is going on, especially when you are replicating.

I Beware: degrees of freedom corrections.

Replicating in R A couple notes

I There are easier ways to do this in R (see for example hccm).

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 35 / 35 I Beware: degrees of freedom corrections.

Replicating in R A couple notes

I There are easier ways to do this in R (see for example hccm).

I But it’s good to know what is going on, especially when you are replicating.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 35 / 35 Replicating in R A couple notes

I There are easier ways to do this in R (see for example hccm).

I But it’s good to know what is going on, especially when you are replicating.

I Beware: degrees of freedom corrections.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 35 / 35 Replicating in R A couple notes

I There are easier ways to do this in R (see for example hccm).

I But it’s good to know what is going on, especially when you are replicating.

I Beware: degrees of freedom corrections.

Molly Roberts Robust and Clustered Standard Errors March 6, 2013 35 / 35