Hypothesis Testing

Home , Wald test

Economics 583: Econometric Theory I A Primer on Asymptotics: Hypothesis Testing

Eric Zivot

October 12, 2011 Hypothesis Testing

1. Specify hypothesis to be tested

H0 : null hypothesis versus. H1 : alternative hypothesis

2. Specify signiﬁcance level α (size) of test

α =Pr(Reject H0 H0 is true) |

3. Construct test statistic, T, from observed data

4. Use test statistic T to evaluate data evidence regarding H0

T is big evidence against H0 ⇒ T is small evidence in favor of H0 ⇒ 5. Decide to reject H0 at speciﬁed signiﬁcance level if value of T falls in the rejection region

T rejection region reject H0 ∈ ⇒ Remark: Usually the rejection region of T is determined by a critical value, cvα, such that

T>cvα reject H0 ⇒ T cvα do not reject H0 ≤ ⇒ Typically, cvα is the (1 α) 100% quantile of the probability distribution − × of T under H0 such that

α =Pr(Reject H0 H0 is true)=Pr(T>cvα) | Decision Making and Hypothesis Tests

Reality Decision H0 is true H0 is false Reject H0 Type I error No error Do not reject H0 No error Type II error

Signiﬁcance Level (size) of Test

level =Pr(Type I error)

Pr(Reject H0 H0 is true) | Goal: Constuct test to have a speciﬁed small signiﬁcance level

level =5%or level =1% Power of Test

power =1 Pr(Type II error) − =Pr(Reject H0 H0 is false) | or

Pr(Reject H0 H1 is true) | Goal: Construct test to have high power

Problem: Impossible to simultaneously have level 0 and power 1. As level ≈ ≈ 0 power also 0. → → Example (Exact Tests in Finite Samples): Let X1,...,Xn be iid random variables with X N(μ, σ2). Consider testing the hypotheses i ∼ H0 : μ = μ0 vs. H1 : μ>μ0 One potential test statistic is the t-statistic

μˆ μ0 tμ=μ = − 0 SE(ˆμ) 1 n σ μˆ = Xi, SE(ˆμ)= n √n iX=1 Intuition: We should reject H0 if tμ=μ0 >> 0.

Result: For any ﬁxed n, under H0 : μ = μ0 n μˆ μ0 1 Xi μ0 tμ=μ = − = − N(0, 1) 0 SE(ˆμ) √n σ ∼ iX=1 µ ¶ Remarks

1. The ﬁnite sample distribution of tμ=μ0 is N(0, 1), which is independent of the population parameters μ0 and σ (sometimes called nuisance parameters). When a test statistic does not depend on nuisance parameters, it is called a pivotal statistic or pivot.

2. Let α (0, 1) denote the signiﬁcance level (size). Because we have a ∈ one-sided test, the rejection region is determined by the critical value cvα such that

Pr(Reject H0 H0 is true) =Pr(Z>cvα)=α | where Z N(0, 1). Hence, cvα is the (1 α) 100% quantile of the ∼ − × standard normal distribution. For example, if α =0.05 then cv.05 = 1.645. 3. The t-test that rejects when tμ=μ0 >cvα is an exact size α ﬁnite sample test.

4. We can calculate the ﬁnite sample distribution of tμ=μ0 becausewemade a number of very strong assumptions: X1,...,Xn be iid random variables with X N(μ, σ2). In particular, if X is not normally distributed then i ∼ i tμ=μ0 will not be normally distributed. Example Continued: Computing Finite Sample Power

Recall,

power =Pr(Reject H0 H1 is true) | Tocomputepower,onehastospecifyH1. Suppose

H1 : μ = μ1 >μ0 Then μˆ μ0 power =Pr − >cvα H1 : μ = μ1 Ã SE(ˆμ) | ! Under H1 : μ = μ1, μˆ μ μˆ μ + μ μ μˆ μ μ μ − 0 = − 1 1 − 0 = − 1 + 1 − 0 SE(ˆμ) SE(ˆμ) SE(ˆμ) SE(ˆμ) = Z + δ N(δ, 1) ∼ where μˆ μ Z = − 1 N(0, 1) SE(ˆμ) ∼ μ μ μ μ δ = 1 − 0 = √n 1 − 0 SE(ˆμ) µ σ ¶ Therefore, under H1 : μ = μ1

μˆ μ0 power =Pr − >cvα H1 : μ = μ1 Ã SE(ˆμ) | ! =Pr(Z + δ>cvα)=Pr(Z>cvα δ) − Remark:

1. Power is monotonic in δ.

2. For any ﬁxed alternative μ >μ , as n 1 0 →∞ μ μ δ = √n 1 − 0 µ σ ¶ →∞ and power 1. → Hypothesis Testing Based on Asymptotic Distributions

Statistical inference in large-sample theory (asymptotic theory) is based on • test statistics whose asymptotic distributions are known under the truth of the null hypothesis

The derivation of the distribution of test statistics in large-sample theory • is much easier than in ﬁnite-sample theory because we only care about the large-sample approximation to the unknown ﬁnite sample distribution

— the LLN, CTL, Slutsky’s Theorem and the Continuous Mapping The- orem (CMT) allow us to derive asymptotic distributions fairly easily Example (Asymptotic tests). Let X1,...,Xn be a sequence of covariance 2 stationary and ergodic random variables with E[Xi]=μ and var(Xi)=σ . We can write

Xi = μ + εi 2 where εi is a covariance stationary MDS with variance σ .

Consider testing the hypotheses

H0 : μ = μ vs. H1 : μ = μ 0 6 0 Two asymptotic test statistics are the asymptotic t-statistic and Wald statistic 2 μˆ μ0 2 (ˆμ μ0) tμ=μ = − , Wald = tμ=μ = − 0 ase(ˆμ) 0 avar(ˆμ) n ³ ´ n 1 σˆ 2 1 2 μˆ = Xi, ase(ˆμ)= , σˆ = (Xi μˆ) nd √n nd − iX=1 iX=1 d The CLT for stationary and ergodic MDS can be used to show that under H0 : μ = μ0 2 A σˆ μˆ N (μ , avar(ˆμ)) , avar(ˆμ)= ∼ 0 n The Ergodic theorem and Slutsky’sd theoremd can be used to show that A tμ=μ N(0, 1) = Z 0 ∼ Additionally, the CMT can be used to show that

2 A 2 Wald = tμ=μ χ (1) = χ 0 ∼ ³ ´ If the signiﬁcance level of each test is 5%, then the asymptotic critical values are determined by

t-test : Pr( Z >cv.05)=0.05 cv.05 =1.96 | | ⇒ Wald test : Pr(χ>cv.05)=0.05 cv.05 =3.84 ⇒ Remarks:

1.Theasymptotict-testisdiﬀerent from the ﬁnite sample t-test in two re- spects:

(a) The way the standard error is computed (ase(ˆμ) vs. se(ˆμ))

(b) The actual size (significance level) of thed asymptotic test is 5% (the nominal significance level) only as n . In finite samples, the →∞ actual size of the asymptotic test may be smaller or larger than 5%. The difference between the actual size and the nominal size is called the size distortion. Remarks continued:

2. For ﬁxed alternatives (e.g. H1 : μ = μ = μ ) The power of the asymp- 1 6 0 totic tests converge to 1 as n . That is, the asymptotic tests are →∞ consistent tests. To see this, consider the t-test μˆ μ μˆ μ − 0 = √n − 0 ase(ˆμ) µ σˆ ¶ μˆ μ1 + μ1 μ0 μˆ μ1 μ1 μ0 = √n − d− = √n − + √n − µ σˆ ¶ µ σˆ ¶ µ σˆ ¶ d N(0, 1) + ( )= → ±∞ ±∞ Therefore, as n →∞ power =Pr(reject H0 H1 is true) | =Pr tμ=μ > 1.96 =Pr( > 1.96) = 1 0 ∞ ³¯ ¯ ´ ¯ ¯ ¯ ¯ Remarks continued:

3. If there are several asymptotic tests for the same hypotheses, we can com- pare the asymptotic power of these tests by considering asymptotic power under so-called local alternatives of the form δ H1 : μ = μ + ,δﬁxed 1 0 √n Under this local alternative we can calculate non-trivial asymptotic power. To see this consider the t-test: μˆ μ μˆ μ − 0 = √n − 0 ase(ˆμ) µ σˆ ¶ μˆ μ1 + μ1 μ0 μˆ μ1 μ1 μ0 = √n − d− = √n − + √n − µ σˆ ¶ µ σˆ ¶ µ σˆ ¶ μ + δ μ μˆ μ1 0 √n − 0 = √n − + √n ⎛ ⎞ µ σˆ ¶ σˆ ⎜ ⎟ μˆ μ δ d ⎝ δ ⎠δ = √n − 1 + N(0, 1) + = N , 1 µ σˆ ¶ σˆ → σ µσ ¶ So that

power =Pr(reject H0 H1 is true) δ | =Pr N , 1 > 1.96 µ¯ µσ ¶¯ ¶ ¯ ¯ which depends on δ. ¯ ¯ ¯ ¯