Hypothesis Testing and Likelihood Ratio Tests

Hypothesis testing and likelihood ratio tests <ul style="display: flex;"><li style="flex:1">i t ti li li ti t t </li><li style="flex:1">t</li></ul><ul style="display: flex;"><li style="flex:1">Y</li><li style="flex:1">We will adopt the following model for observed data. The distribution of Y = (Y , ..., Y ) is </li></ul><ul style="display: flex;"><li style="flex:1">1</li><li style="flex:1">n</li></ul>parameter t θ, which may be a vector θ = (θ , ..., θ ); θ∈Θ, considered known except for some p parameter space . The parameter space will usually be an open set. If Y is a continuous <ul style="display: flex;"><li style="flex:1">1</li><li style="flex:1">k</li></ul>Y<ul style="display: flex;"><li style="flex:1">the </li><li style="flex:1">t</li></ul><ul style="display: flex;"><li style="flex:1">probability density function </li><li style="flex:1">y</li><li style="flex:1">Y</li><li style="flex:1">random variable, its p </li></ul>ilit it ti (pdf) will de denoted f(y ;θ) . If Y is <ul style="display: flex;"><li style="flex:1">y</li><li style="flex:1">probability mass function </li><li style="flex:1">y</li><li style="flex:1">Y y </li></ul>discrete then f(y ;θ) represents the p ilit ti (pmf); f(y ;θ) = P ( = ). θstatistical hypothesis A t ti ti l i is a statement about the value of θ. We are interested in testing the null thypothesis H : θ∈Θ versus the alternative hypothesis H : θ∈Θ . Where Θ and Θ ⊂ Θ. <ul style="display: flex;"><li style="flex:1">0</li><li style="flex:1">0</li><li style="flex:1">1</li><li style="flex:1">1</li><li style="flex:1">0</li><li style="flex:1">1</li></ul>hypothesis test Naturally Θ ∩ Θ = ∅, but we need not have Θ ∪ Θ = Θ. A h t tis a procedure <ul style="display: flex;"><li style="flex:1">t</li><li style="flex:1">i</li></ul><ul style="display: flex;"><li style="flex:1">0</li><li style="flex:1">1</li><li style="flex:1">0</li><li style="flex:1">1</li></ul><ul style="display: flex;"><li style="flex:1">critical region </li><li style="flex:1">for deciding between H and H based on the sample data. It is equivalent to a c iti l i : </li></ul><ul style="display: flex;"><li style="flex:1">0</li><li style="flex:1">1</li></ul>nya critical region is a set C ⊂ R such that if y = (y , ..., y ) ∈ C, H is rejected. Typically C is <ul style="display: flex;"><li style="flex:1">1</li><li style="flex:1">n</li><li style="flex:1">0</li></ul>test statistic <ul style="display: flex;"><li style="flex:1">expressed in terms of the value of some t </li><li style="flex:1">t t ti ti , a function of the sample data. For </li></ul>y – µ0 s / n <ul style="display: flex;"><li style="flex:1">example, we might have C = {(y , ..., y ): </li><li style="flex:1">≥ 3.324}. The number 3.324 here is called a </li></ul><ul style="display: flex;"><li style="flex:1">1</li><li style="flex:1">n</li></ul>Y – µ0 S / n <ul style="display: flex;"><li style="flex:1">critical value </li><li style="flex:1">iti l l of the test statistic </li><li style="flex:1">.</li></ul>y If ∈C but θ∈Θ , we have committed a Type I error. If y∉C but θ∈Θ , we have committed a <ul style="display: flex;"><li style="flex:1">0</li><li style="flex:1">1</li></ul>Type II error. The ideal hypothesis test would simultaneously minimize the probabilities of both types of error, but this turns out to be impossible in principle. So what we do is to select a Max <ul style="display: flex;"><li style="flex:1">significance level </li><li style="flex:1">i</li><li style="flex:1">i i </li><li style="flex:1">l</li></ul>lα = P (Y∈C) to be a small number; α = 0.05 and α = 0.01 are θ ∈ Θ0 θtraditional. Then a "good" test is one with a small probability of Type II error, among all possible ytests with significance level α. When y ∈C for a test of level α (that is, H is rejected), we say the 0statistically significant results are s t ti ti i i tat level α. <ul style="display: flex;"><li style="flex:1">l</li><li style="flex:1">i</li></ul>For a particular set of data, the smallest significance level that leads to the rejection of H is called 0p−value l . H is rejected if and only if p≤α. the 0power P (Y∈C) can be viewed as a function of θ. If θ∈Θ , we refer to this quantity as the p of θ1Likelihood Ratio Tests: Page 1 of 3 the test C. For any good test, we will have P (Y∈C) ↑1 as n → ∞ for each θ∈Θ . This θ1provides a way to choose sample size. For a fixed value θ∈Θ that is of scientific interest, choose 1n large enough so that the probability of rejecting H is acceptably high. 0<ul style="display: flex;"><li style="flex:1">likelihood ratio </li><li style="flex:1">i li ti </li><li style="flex:1">How do we construct good hypothesis tests? Usually it is hard to beat the l </li></ul>Max L(θ;y) θ ∈ Θ0 <ul style="display: flex;"><li style="flex:1">tests </li><li style="flex:1">y</li><li style="flex:1">t t , which are defined as follows. C = {y : λ = </li><li style="flex:1">≤ k}. The value of k (0<k<1) </li></ul>Max θ ∈ ΘL(θ;y) varies from problem to problem. It is chosen so that the test will have significance level ("size") α. Notice that the denominator of λ is just the likelihood function evaluated at the MLE. The numerator is the likelihood function evaluated at a sort of restricted MLE, in which θ is forced to stay in the set Θ . Also notice that when λ=0, H is always rejected, and when λ=1, H is never <ul style="display: flex;"><li style="flex:1">0</li><li style="flex:1">0</li><li style="flex:1">0</li></ul>rejected. There are two main versions of the likelihood ratio approach. One version leads to exact tests, and exact likelihood ratio tests the other leads to large−sample approximations. The e t li li ti t t are obtained by working on the critical region C, and re−expressing it in terms of the value of some test statistic whose distribution (given that H is true) is known exactly. This is where we get most of the 0standard statistical tests, including t−tests and F−tests in regression and the analysis of variance. Sometimes, after the critical region C has been re−expressed in terms of some seemingly convenient test statistic, nobody can figure out its distribution under H . This means we are unable to choose 0k ∈ (0,1) so that the test has size α. And if the MLE has to be approximated numerically, there is large−sample likelihood ratio tests <ul style="display: flex;"><li style="flex:1">li li ti t t ; </li><li style="flex:1">little hope of an exact test. In such cases we resort to l </li></ul>we will use the following result. l Likelihood Ratio Tests: Page 2 of 3 Let θ = (θ , ..., θ ); we want to test H : θ∈Θ versus H : θ∈Θ , where Θ ∪ Θ = Θ. Let 1p<ul style="display: flex;"><li style="flex:1">0</li><li style="flex:1">0</li><li style="flex:1">1</li><li style="flex:1">1</li><li style="flex:1">0</li><li style="flex:1">1</li></ul>r≤p be the number of parameters θ whose values are restricted by H . Then under some <ul style="display: flex;"><li style="flex:1">j</li><li style="flex:1">0</li></ul>2smoothness conditions, G = −2 log(λ) has (for large n) an approximate χ distribution with r 2degrees of freedom. We reject H when G is greater than the critical value of the χ distribution. 0If g is the observed value of the statistic G calculated from the sample data, the p−value is 1−γ(g), 2where γ is the cumulative distribution function of a χ distribution with r degrees of freedom. Likelihood Ratio Tests: Page 3 of 3

Hypothesis Testing and Likelihood Ratio Tests

Details

Download

Copyright

We respect the copyrights and intellectual property rights of all users. All uploaded documents are either original works of the uploader or authorized works of the rightful owners.

Support