Recap p-Values Problems Summary Recap p-Values Problems Summary . Last Lecture . Biostatistics 602 - Statistical Inference Lecture 22 p-Values • Is the exact distribution of LRT statistic typically easy to obtain? . • How about its asymptotic distribution? For testing which null/alternative hypotheses is the asymptotic distribution valid? Hyun Min Kang • What is a Wald Test? • Describe a typical way to construct a Wald Test. April 9th, 2013 Hyun Min Kang Biostatistics 602 - Lecture 22 April 9th, 2013 1 / 31 Hyun Min Kang Biostatistics 602 - Lecture 22 April 9th, 2013 2 / 31 Recap p-Values Problems Summary Recap p-Values Problems Summary . Asymptotics of LRT Wald Test . Wald test relates point estimator of θ to hypothesis testing about θ. Theorem 10.3.1 . Definition Consider testing H0 : θ = θ0 vs H1 : θ = θ0. Suppose X1, , Xn are iid . ̸ ··· 2 samples from f(x θ), and θˆ is the MLE of θ, and f(x θ) satisfies certain Suppose Wn is an estimator of θ and Wn (θ, σ ). Then Wald test | | ∼ AN W ”regularity conditions” (e.g. see misc 10.6.2), then under H0: statistic is defined as d 2 Wn θ0 2 log λ(x) χ1 Zn = − − → Sn .as n . → ∞ .where θ0 is the value of θ under H0 and Sn is a consistent estimator of σW Hyun Min Kang Biostatistics 602 - Lecture 22 April 9th, 2013 3 / 31 Hyun Min Kang Biostatistics 602 - Lecture 22 April 9th, 2013 4 / 31 Recap p-Values Problems Summary Recap p-Values Problems Summary . p-Values Advantage to reporting a test result via a p-value . Conclusions from Hypothesis Testing . • • Reject H0 or accept H0. The size α does not need to be predefined • Each reader can choose the α he or she considers appropriate • If size of the test is (α) small, the decision to reject H0 is convincing. • And then can compare the reported p(x) to α • If α is large, the decision may not be very convincing. • So that each reader can individually determine whether these data lead . to acceptance or rejection to H0. Definition: p-Value • The p-value quantifies the evidence against H0. • The smaller the p-value, the stronger, the evidence for rejecting H . A p-value p(X) is a test statistic satisfying 0 p(x) 1 for every sample 0 ≤ ≤ • A p-value reports the results of a test on a more continuous scale point x. Small values of p(X) given evidence that H is true. A p-value is 1 • Rather than just the dichotomous decision ”Accept H0” or ”Reject H0”. valid if, for every θ Ω and every 0 α 1, ∈ 0 ≤ ≤ Pr(p(X) α θ) α . ≤ | ≤ Hyun Min Kang Biostatistics 602 - Lecture 22 April 9th, 2013 5 / 31 Hyun Min Kang Biostatistics 602 - Lecture 22 April 9th, 2013 6 / 31 Recap p-Values Problems Summary Recap p-Values Problems Summary . Constructing a valid p-value Example : Two-sided normal p-value . Theorem 8.3.27. Problem . 2 Let W(X) be a test statistic such that large values of W give evidence Let X = (X , , Xn) be a random sample from a (θ, σ ) population. 1 ··· N that H1 is true. For each sample point x, define Consider testing H0 : θ = θ0 versus H1 : θ = θ0. p(x) = sup Pr(W(X) W(x) θ) ̸ θ Ω0 ≥ | 1. Construct a size α LRT test. ∈ 2. Find a valid p-value, as a function of x. .Then p(X) is a valid p-value. Hyun Min Kang Biostatistics 602 - Lecture 22 April 9th, 2013 7 / 31 Hyun Min Kang Biostatistics 602 - Lecture 22 April 9th, 2013 8 / 31 Recap p-Values Problems Summary Recap p-Values Problems Summary . Solution - Constructing LRT Solution - Constructing and Simplifying the Test Combining the results together 2 2 n Ω = (θ, σ ): θ R, σ > 0 σˆ2 /2 { 2 ∈ 2 } λ(x) = Ω0 = (θ, σ ): θ = θ0, σ > 0 2 σˆ0 { } 2 ( ) sup 2 2 L x (θ,σ ):θ=θ0,σ >0 (θ, σ ) x { } | λ( ) = 2 LRT test rejects H0 if and only if sup (θ,σ2):θ R,σ2>0 L(θ, σ x) { ∈ } | n/2 2 σˆ2 For the denominator, the MLE of θ and σ are c σˆ2 ≤ ˆ X ( 0 ) θ = n/2 2 2 2 (Xi X) n 1 2 (xi x) /n σˆ = − = − s c { n n X − 2 ∑ (xi θ0) /n ≤ 2 ( ∑ − ) For the numerator, the MLE of θ and σ are (x x)2 ∑ i c − 2 ∗ ˆ (xi θ ) ≤ θ0 = θ0 ∑ 0 2 − 2 (Xi θ0) { σˆ0 = n− ∑ ∑ Hyun Min Kang Biostatistics 602 - Lecture 22 April 9th, 2013 9 / 31 Hyun Min Kang Biostatistics 602 - Lecture 22 April 9th, 2013 10 / 31 Recap p-Values Problems Summary Recap p-Values Problems Summary . Solution - Simplifying the LRT Solution - Obtaining size α test Under H (x x)2 0 i c 2 − 2 ∗ (xi s) + n(x θ0) ≤ X θ −∑ − 0 1 − Tn 1 sX/√n ∼ − ∑ 2 c∗ n(x θ0) ≤ 1 + − 2 (xi x) X θ0 − Pr − c∗∗∗ = α 2 n(x∑ θ ) sX/√n ≥ 0 c ( ) − 2 ∗∗ (xi x) ≥ Pr ( Tn 1 c∗∗∗) = α − | − | ≥ x θ0 c∗∗∗ = tn 1,α/2 ∑ − c∗∗∗ − sX/√n ≥ x θ0 Therefore, size α LRT test rejects H0 if and only if − tn 1,α/2 sX/√n x θ0 ≥ − LRT test rejects H0 if − c∗∗∗ . The next step is specify c to get sX/√n ≥ size α test. Hyun Min Kang Biostatistics 602 - Lecture 22 April 9th, 2013 11 / 31 Hyun Min Kang Biostatistics 602 - Lecture 22 April 9th, 2013 12 / 31 Recap p-Values Problems Summary Recap p-Values Problems Summary . Solution - p-value from two-sided test Example : One-sided normal p-value X θ0 For a test statistic W(X) = − , under H0, regardless of the value of sX/√n σ2, W(X) T . Then, a valid p-value can be defined by n 1 . ∼ − Problem p(x) = sup Pr(W(X) W(x) θ, σ2) . Let X X X be a random sample from a 2 population. θ Ω0 ≥ | = ( 1, , n) (θ, σ ) ∈ ··· N 2 Consider testing H0 : θ θ0 versus H1 : θ > θ0. = Pr(W(X) W(x) θ0, σ ) ≤ ≥ | 1. Construct a size α LRT test. = 2 Pr(Tn 1 W(x)) − ≥ 2. Find a valid p-value, as a function of x. = 2 1 F 1 W(x) . T−n 1 − − { } [ ] where F 1 ( ) is the inverse CDF of t-distribution with n 1 degrees of T−n 1 freedom. − · − Hyun Min Kang Biostatistics 602 - Lecture 22 April 9th, 2013 13 / 31 Hyun Min Kang Biostatistics 602 - Lecture 22 April 9th, 2013 14 / 31 Recap p-Values Problems Summary Recap p-Values Problems Summary . Constructing LRT test Obtaining one-sided p-value Consider any θ θ0 and any σ. As shown in previous lectures, the LRT size α test rejects H0 if ≤ x θ 2 X θ0 2 W x 0 t Pr(W(X) W(x) θ, σ ) = Pr − W(x) θ, σ ( ) = − n 1,α ≥ | sX/√n ≥ | sX/√n ≥ − ( ) X θ θ θ Pr W x 0 2 Because the null hypothesis contains multiple possible θ θ0, we first = − ( ) + − θ, σ sX/√n ≥ sX/√n| want to show that the supreme in the definition of p-value≤ ( ) θ0 θ 2 = Pr Tn 1 W(x) + − θ, σ p(x) = sup Pr(W(X) W(x) θ, σ2) − ≥ sX/√n| ≥ | ( ) θ Ω0 Pr (Tn 1 W(x)) ∈ ≤ − ≥ = Pr W(X) W(x) θ , σ2 always occurs at when θ = θ0, and the value of σ does not matter. ≥ | 0 ( ) Hyun Min Kang Biostatistics 602 - Lecture 22 April 9th, 2013 15 / 31 Hyun Min Kang Biostatistics 602 - Lecture 22 April 9th, 2013 16 / 31 Recap p-Values Problems Summary Recap p-Values Problems Summary . Obtaining one-sided p-value (cont’d) p-Values by conditioning on on sufficient statistic Suppose S(X) is a sufficient statistic for the model f(x θ): θ Ω0 . (not necessarily including alternative hypothesis). If{ the| null hypothesis∈ } is Thus, the p-value for this one-side test is true, the conditional distribution of X given S = s does not depend on θ. Again, let W(X) denote a test statistic where large value give evidence 2 p(x) = sup Pr(W(X) W(x) θ, σ ) that H1 is true. Define θ Ω0 ≥ | ∈ = Pr(W(X) W(x) θ , σ2) p(x) = Pr(W(X) W(x) S = S(x)) ≥ | 0 ≥ | = Pr(T W(x)) = 1 F 1 [W(x)] n 1 T−n 1 − ≥ − − If we consider only the conditional distribution, by Theorem 8.3.27, this is a valid p-value, meaning that Pr(p(X) α S = s) α ≤ | ≤ Hyun Min Kang Biostatistics 602 - Lecture 22 April 9th, 2013 17 / 31 Hyun Min Kang Biostatistics 602 - Lecture 22 April 9th, 2013 18 / 31 Recap p-Values Problems Summary Recap p-Values Problems Summary . p-Values by conditioning on sufficient statistic (cont’d) Example - Fisher’s Exact Test . Problem . Let X and X be independent observations with X Binomial(n , p ), 1 2 1 ∼ 1 1 and X2 Binomial(n2, p2). Consider testing H0 : p1 = p2 versus Then for any θ Ω0, unconditionally we have ∼ ∈ .H1 : p1 > p2. Find a valid p-value function. Pr(p(X) α θ) = Pr(p(X) α S = s) Pr(S = s θ) . ≤ | s ≤ | | Solution ∑ . α Pr(S = s θ) = α Under H0, if we let p denote the common value of p1 = p2. Then the join ≤ s | pmf of (X1, X2) is ∑ n n 1 x1 n1 x1 2 x2 n2 x2 f(x , x p) = p (1 p) − p (1 p) − Thus, p(X) is a valid p-value.