Hypothesis testing and likelihood ratio tests

• i t ti li li ti t t
• t

• Y
• We will adopt the following model for observed data. The distribution of Y = (Y , ..., Y ) is

• 1
• n

parameter t θ, which may be a vector θ = (θ , ..., θ ); θ∈Θ, considered known except for some p

parameter space
. The parameter space will usually be an open set. If Y is a continuous

• 1
• k

Y

• the
• t

• probability density function
• y
• Y
• random variable, its p

ilit it ti (pdf) will de denoted f(y ;θ) . If Y is

• y
• probability mass function
• y
• Y y

discrete then f(y ;θ) represents the p ilit ti (pmf); f(y ;θ) = P ( = ).

θ

statistical hypothesis
A t ti ti l i is a statement about the value of θ. We are interested in testing the null t

hypothesis H : θ∈Θ versus the alternative hypothesis H : θ∈Θ . Where Θ and Θ ⊂ Θ.

• 0
• 0
• 1
• 1
• 0
• 1

hypothesis test
Naturally Θ ∩ Θ = ∅, but we need not have Θ ∪ Θ = Θ. A h t tis a procedure

• t
• i

• 0
• 1
• 0
• 1

• critical region
• for deciding between H and H based on the sample data. It is equivalent to a c iti l i :

• 0
• 1

n

ya critical region is a set C ⊂ R such that if y = (y , ..., y ) ∈ C, H is rejected. Typically C is

• 1
• n
• 0

test statistic

• expressed in terms of the value of some t
• t t ti ti , a function of the sample data. For

y – µ₀ s / n

• example, we might have C = {(y , ..., y ):
• ≥ 3.324}. The number 3.324 here is called a

• 1
• n

Y – µ₀ S / n

• critical value
• iti l l of the test statistic
• .

y
If ∈C but θ∈Θ , we have committed a Type I error. If y∉C but θ∈Θ , we have committed a

• 0
• 1

Type II error. The ideal hypothesis test would simultaneously minimize the probabilities of both types of error, but this turns out to be impossible in principle. So what we do is to select a

Max

• significance level
• i
• i i
• l

lα =

P (Y∈C) to be a small number; α = 0.05 and α = 0.01 are

θ ∈ Θ₀

θ

traditional. Then a "good" test is one with a small probability of Type II error, among all possible ytests with significance level α. When y ∈C for a test of level α (that is, H is rejected), we say the

0

statistically significant results are s t ti ti i i tat level α.

• l
• i

For a particular set of data, the smallest significance level that leads to the rejection of H is called

0

p−value l . H is rejected if and only if p≤α. the

0

power
P (Y∈C) can be viewed as a function of θ. If θ∈Θ , we refer to this quantity as the p of

θ

1

Likelihood Ratio Tests: Page 1 of 3 the test C. For any good test, we will have P (Y∈C)

↑

1 as n → ∞ for each θ∈Θ . This

θ

1

provides a way to choose sample size. For a fixed value θ∈Θ that is of scientific interest, choose

1

n large enough so that the probability of rejecting H is acceptably high.

0

• likelihood ratio
• i li ti
• How do we construct good hypothesis tests? Usually it is hard to beat the l

Max

L(θ;y)

θ ∈ Θ₀

• tests
• y
• t t , which are defined as follows. C = {y : λ =
• ≤ k}. The value of k (0<k<1)

Max

_{θ ∈ Θ}L(θ;y)

varies from problem to problem. It is chosen so that the test will have significance level ("size") α. Notice that the denominator of λ is just the likelihood function evaluated at the MLE. The numerator is the likelihood function evaluated at a sort of restricted MLE, in which θ is forced to stay in the set Θ . Also notice that when λ=0, H is always rejected, and when λ=1, H is never

• 0
• 0
• 0

rejected. There are two main versions of the likelihood ratio approach. One version leads to exact tests, and exact likelihood ratio tests the other leads to large−sample approximations. The e t li li ti t t are obtained

by working on the critical region C, and re−expressing it in terms of the value of some test statistic whose distribution (given that H is true) is known exactly. This is where we get most of the

0

standard statistical tests, including t−tests and F−tests in regression and the analysis of variance. Sometimes, after the critical region C has been re−expressed in terms of some seemingly convenient test statistic, nobody can figure out its distribution under H . This means we are unable to choose

0

k ∈ (0,1) so that the test has size α. And if the MLE has to be approximated numerically, there is large−sample likelihood ratio tests

• li li ti t t ;
• little hope of an exact test. In such cases we resort to l

we will use the following result. l
Likelihood Ratio Tests: Page 2 of 3
Let θ = (θ , ..., θ ); we want to test H : θ∈Θ versus H : θ∈Θ , where Θ ∪ Θ = Θ. Let

1

p

• 0
• 0
• 1
• 1
• 0
• 1

r≤p be the number of parameters θ whose values are restricted by H . Then under some

• j
• 0

2

smoothness conditions, G = −2 log(λ) has (for large n) an approximate χ distribution with r

2

degrees of freedom. We reject H when G is greater than the critical value of the χ distribution.

0

If g is the observed value of the statistic G calculated from the sample data, the p−value is 1−γ(g),

2

where γ is the cumulative distribution function of a χ distribution with r degrees of freedom.
Likelihood Ratio Tests: Page 3 of 3