Testing Statistical Hypotheses
TESTING STATISTICAL HYPOTHESES
David M. Lane. et al. Introduction to Statistics : pp. 370-447
ioc.pdf [email protected] ICY0006: Lecture 9 1 / 28 Contents
1 Basic Concepts in Testing
ioc.pdf
[email protected] ICY0006: Lecture 9 2 / 28 Next section
1 Basic Concepts in Testing
ioc.pdf
[email protected] ICY0006: Lecture 9 3 / 28 Statistical Hypotheses
Any claim made about one or more populations of interest constitutes a statistical hypothesis. These hypotheses usually involve population parameters, the nature of the population, the relation between the populations, and so on.
For example, we may hypothesize that: The mean of a population, µ, is 2, or Two populations have the same variance, or A population is normally distributed, or The means in four populations are the same, or Two populations are independent, and so on.
Procedures leading to either the acceptance or rejection of statistical hypotheses are called statistical tests. There are two approaches to accept or reject hypothesis:
I Bayesian approach, which assigns probabilities to hypotheses directly (see our lecture Probability) I the frequentist (classical) approach (see below) ioc.pdf
[email protected] ICY0006: Lecture 9 4 / 28 Statistical Hypotheses
Any claim made about one or more populations of interest constitutes a statistical hypothesis. These hypotheses usually involve population parameters, the nature of the population, the relation between the populations, and so on.
For example, we may hypothesize that: The mean of a population, µ, is 2, or Two populations have the same variance, or A population is normally distributed, or The means in four populations are the same, or Two populations are independent, and so on.
Procedures leading to either the acceptance or rejection of statistical hypotheses are called statistical tests. There are two approaches to accept or reject hypothesis:
I Bayesian approach, which assigns probabilities to hypotheses directly (see our lecture Probability) I the frequentist (classical) approach (see below) ioc.pdf
[email protected] ICY0006: Lecture 9 4 / 28 Statistical Hypotheses
Any claim made about one or more populations of interest constitutes a statistical hypothesis. These hypotheses usually involve population parameters, the nature of the population, the relation between the populations, and so on.
For example, we may hypothesize that: The mean of a population, µ, is 2, or Two populations have the same variance, or A population is normally distributed, or The means in four populations are the same, or Two populations are independent, and so on.
Procedures leading to either the acceptance or rejection of statistical hypotheses are called statistical tests. There are two approaches to accept or reject hypothesis:
I Bayesian approach, which assigns probabilities to hypotheses directly (see our lecture Probability) I the frequentist (classical) approach (see below) ioc.pdf
[email protected] ICY0006: Lecture 9 4 / 28 Hypothesis Testing Steps
1 State the hypotheses (the null hypothesis and an alternative hypothesis) 2 Formulate an analysis plan (e.g. the signi cance level is 0.05, the test method one-sample z-test) 3 Analyse sample data 4 Interpret result
ioc.pdf
[email protected] ICY0006: Lecture 9 5 / 28 Hypothesis
Null Hypothesis (denoted H0): is the statement being tested in a test of hypothesis. Alternative Hypothesis (H1): is what is believe to be true if the null hypothesis is false.
Null Hypothesis: H0 Must contain condition of equality =, >,or 6 Test the Null Hypothesis directly: reject H0 or fail to reject H0
Alternative Hypothesis: H1 Must be true if H0 is false, correspondingly 6=, <,or > `opposite' of Null Hypothesis
ioc.pdf
[email protected] ICY0006: Lecture 9 6 / 28 Hypothesis
Null Hypothesis (denoted H0): is the statement being tested in a test of hypothesis. Alternative Hypothesis (H1): is what is believe to be true if the null hypothesis is false.
Null Hypothesis: H0 Must contain condition of equality =, >,or 6 Test the Null Hypothesis directly: reject H0 or fail to reject H0
Alternative Hypothesis: H1 Must be true if H0 is false, correspondingly 6=, <,or > `opposite' of Null Hypothesis
ioc.pdf
[email protected] ICY0006: Lecture 9 6 / 28 Hypothesis
Null Hypothesis (denoted H0): is the statement being tested in a test of hypothesis. Alternative Hypothesis (H1): is what is believe to be true if the null hypothesis is false.
Null Hypothesis: H0 Must contain condition of equality =, >,or 6 Test the Null Hypothesis directly: reject H0 or fail to reject H0
Alternative Hypothesis: H1 Must be true if H0 is false, correspondingly 6=, <,or > `opposite' of Null Hypothesis
ioc.pdf
[email protected] ICY0006: Lecture 9 6 / 28 Stating the hypothesis
If you wish to support your claim, the claim must be stated so that it becomes the alternative hypothesis.
H0 must always contain equality; however some claims are not stated using equality. Therefore sometimes the claim and H0 will not be the same. Ideally all claims should be stated that they are Null Hypothesis
Example The problem: In the 1970s, 2029 year old men in the U.S. had a mean µ body weight of 77 kg. Standard deviation σ was 19 kg. We test whether mean body weight in the population now diers.
Null hypothesis H0 : µ = 77 (no dierence) Alternative hypothesis can be either
H1 : µ > 77 (one-sided test) or H1 : µ 6= 77 (two-sided test)
ioc.pdf
[email protected] ICY0006: Lecture 9 7 / 28 Formulation of an analysis plan.
The analysis plan describes how to use sample data to evaluate the null hypothesis. The evaluation often focuses around a single test statistic.
The analysis plan should specify the following elements. Signi cance level. Often, researchers choose signi cance levels equal to 0.01, 0.05, or 0.10; but any value between 0 and 1 can be used. Test method. Use the one-sample t-test to determine whether the hypothesized mean diers signi cantly from the observed sample mean. Alternative options are:
I the one-sample z-test I the two-sample t-test I the two-sample z-test I etc.
ioc.pdf
[email protected] ICY0006: Lecture 9 8 / 28 Analyse sample data: Test Statistic
Find the value of the test statistic (mean score, proportion, t statistic, z-score, etc.) described in the analysis plan.
One-sample test of a mean when σ is known √ Take a sample of the size with the mean and a standard error x n µ0 sx = σ/ n Compute z-statistic for the µ0 ≡ population mean assuming H0 is true:
µ − µ0 z = x sx
Example: Body Weight
µ0 = 77 and σ = 19. Let's take a SRS of size n = 64.
ioc.pdf
[email protected] ICY0006: Lecture 9 9 / 28 Analyse sample data: Test Statistic Find the value of the test statistic (mean score, proportion, t statistic, z-score, etc.) described in the analysis plan.
One-sample test of a mean when σ is known √ Take a sample of the size with the mean and a standard error x n µ0 sx = σ/ n Compute z-statistic for the µ0 ≡ population mean assuming H0 is true:
µ − µ0 z = x sx
Example: Body Weight
µ0 = 77 and σ = 19. Let's take a SRS of size n = 64.
If we nd a sample mean 79, then we have µx =
σ 19 s = √ = √ = 2.375 x n 64
µ − µ0 79 − 77 z = x = = 0.842 2 375 ioc.pdf sx .
[email protected] ICY0006: Lecture 9 9 / 28 Analyse sample data: Test Statistic
Find the value of the test statistic (mean score, proportion, t statistic, z-score, etc.) described in the analysis plan.
One-sample test of a mean when σ is known √ Take a sample of the size with the mean and a standard error x n µ0 sx = σ/ n Compute z-statistic for the µ0 ≡ population mean assuming H0 is true:
µ − µ0 z = x sx
Example: Body Weight
µ0 = 77 and σ = 19. Let's take a SRS of size n = 64.
If we nd a sample mean 83, then we have µx =
µ − µ0 83 − 77 x 2 652 z = = 2 375 = . sx . ioc.pdf
[email protected] ICY0006: Lecture 9 9 / 28 Sampling distribution
Distribution of all the x's
x1 = 79 x2 = 83 x3 = 75 . . . . . Kilograms
Population mean µ0 = 77 kg
Sampling distribution of x under H0 : µ = 77 for n = 64 ⇒ x ∼ N (77,2.375)
ioc.pdf
[email protected] ICY0006: Lecture 9 10 / 28 Decision Errors
Two types of errors can result from a hypothesis test. Type I error occurs when the researcher rejects a null hypothesis when it is true. The probability of committing a Type I error is called the signi cance level. This probability is also called alpha, and is often denoted by α. Type II error occurs when the researcher fails to reject a null hypothesis that is false. The probability of committing a Type II error is called Beta, and is often denoted by β. The probability of not committing a Type II error is called the Power of the test.
ioc.pdf
[email protected] ICY0006: Lecture 9 11 / 28 The Table Summarizing Type I and Type II Errors
α = P(H1|H0) β = P(H0|H1)
ioc.pdf
[email protected] ICY0006: Lecture 9 12 / 28 Decision Rules
The analysis plan includes decision rules for rejecting the null hypothesis. In practice, statisticians describe these decision rules in two ways with reference to a P-value or with reference to a region of acceptance.
A. P-value. The strength of evidence in support of a null hypothesis is measured by the P-value. Suppose the test statistic is equal to S. The P-value is the probability of observing a test statistic as extreme as S, assuming the null hypothesis is true. If the P-value is less than the signi cance level, we reject the null hypothesis. B. The region of acceptance is a range of values. If the test statistic falls within the region of acceptance, the null hypothesis is not rejected. The region of acceptance is de ned so that the chance of making a Type I error is equal to the signi cance level. The set of values outside the region of acceptance is called the region of rejection. If the test statistic falls within the region of rejection, the null hypothesis is rejected. In such cases, we say that the hypothesis has been rejected at the α level of signi cance. ioc.pdf
[email protected] ICY0006: Lecture 9 13 / 28 P-value
The P-value corresponds to the answer the question: What is the probability of the observed test statistic or one more extreme when H0 is true? This is the area under the curve of the Standard Normal distribution beyond the z. Convert z statistic to P-value:
I For H1 : µ > µ0 ⇒ p = P(Z > z) = area under right-tail beyond z I For H1 : µ < µ0 ⇒ p = P(Z < z) = area under left-tail beyond z I For H1 : µ 6= µ0 ⇒ p = 2 × one-tailed P-value
ioc.pdf
[email protected] ICY0006: Lecture 9 14 / 28 One-sided P-value for z of 0.842
ioc.pdf
[email protected] ICY0006: Lecture 9 15 / 28 One-sided P-value for z of 2.652
ioc.pdf
[email protected] ICY0006: Lecture 9 16 / 28 Two-sided P-value
One-sided H1 is either µ > µ0 or µ < µ0; Two-sided H1 is µ 6= µ0; For Two-sided H1 we consider potential deviations in both directions, hence double the one- sided P-value
Examples:
If one-sided P = 0.0010, then two-sided P = 2 × 0.0010 = 0.0020. If one-sided P = 0.2743, then two-sided P = 2 × 0.2743 = 0.5486.
ioc.pdf
[email protected] ICY0006: Lecture 9 17 / 28 Interpretation
P-value answers the question: What is the probability of the observed test statistic ... when H0 is true? Thus, smaller and smaller P-values provide stronger and stronger evidence against H0 Small P-value means strong evidence for H1.
Conventions
P > 0.10 ⇒ non-signi cant evidence against H0 0.05 < P 6 0.10 ⇒ marginally signi cant evidence against H0 0.01 < P 6 0.05 ⇒signi cant evidence against H0 P 6 0.01 ⇒ highly signi cant evidence against H0
Examples
P = 0.27 ⇒ non-signi cant evidence against H0 P = 0.01 ⇒ highly signi cant evidence against H0
ioc.pdf
[email protected] ICY0006: Lecture 9 18 / 28 Method of Critical Region
Critical Region : is the set of all values of the test statistic that would cause a rejection of the null hypothesis
ioc.pdf
[email protected] ICY0006: Lecture 9 19 / 28 Method of Critical Region
Critical Region : is the set of all values of the test statistic that would cause a rejection of the null hypothesis
ioc.pdf
[email protected] ICY0006: Lecture 9 19 / 28 Method of Critical Region
Critical Region : is the set of all values of the test statistic that would cause a rejection of the null hypothesis
ioc.pdf
[email protected] ICY0006: Lecture 9 19 / 28 Method of Critical Region (2)
Critical Value : is the value (s) that separates the critical region from the values that would not lead to a rejection of H0.
ioc.pdf
[email protected] ICY0006: Lecture 9 20 / 28 Method of Critical Region (2)
Critical Value : is the value (s) that separates the critical region from the values that would not lead to a rejection of H0.
ioc.pdf
[email protected] ICY0006: Lecture 9 20 / 28 Method of Critical Region (2)
Critical Value : is the value (s) that separates the critical region from the values that would not lead to a rejection of H0.
ioc.pdf
[email protected] ICY0006: Lecture 9 20 / 28 Controlling Type I and Type II Errors
α, β, and n are related when two of the three are chosen, the third is determined α and n are usually chosen try to use the largest α you can tolerate if Type I error is serious, select a smaller α value and a larger n value
Recall:
α = P(H1|H0) β = P(H0|H1) = Pr(retain H0|H0 is false) ≡ probability of a Type II error 1 − β is called power of hypothesis test; 1 − β = P(reject H0|H0 is false) ≡ probability of avoiding a Type II error
ioc.pdf
[email protected] ICY0006: Lecture 9 21 / 28 Controlling Type I and Type II Errors
α, β, and n are related when two of the three are chosen, the third is determined α and n are usually chosen try to use the largest α you can tolerate if Type I error is serious, select a smaller α value and a larger n value
Recall:
α = P(H1|H0) β = P(H0|H1) = Pr(retain H0|H0 is false) ≡ probability of a Type II error 1 − β is called power of hypothesis test; 1 − β = P(reject H0|H0 is false) ≡ probability of avoiding a Type II error
ioc.pdf
[email protected] ICY0006: Lecture 9 21 / 28 Power of a z test
√ |µ0 − µ1| n 1 − β = Φ −z1 α + − 2 σ where Φ(z) represent the cumulative probability of Standard Normal Z µ0 represent the population mean under the null hypothesis µ1 represents the population mean under the alternative hypothesis
Example: Calculating Power
A study of n = 16 retains H0 : µ = 170 at α = 0.05 (two-sided); σ = 40. What was the power of test's conditions to identify a population mean of 190? √ |µ0 − µ1| n 1 − β = Φ −z1 α + − 2 σ √ 170 190 16 ! 1 96 | − | = Φ − . + 40
= Φ(0.04) = 0.5160
ioc.pdf
[email protected] ICY0006: Lecture 9 22 / 28 Reasoning Behind Power
Competing sampling distributions Top curve (next page) assumes H0 is true Bottom curve assumes H1 is true α is set to 0.05 (two-sided) We will reject H0 when a sample mean exceeds 189.6 (right tail, top curve) The probability of getting a value greater than 189.6 on the bottom curve is 0.5160, corresponding to the power of the test
ioc.pdf
[email protected] ICY0006: Lecture 9 23 / 28 ioc.pdf [email protected] ICY0006: Lecture 9 24 / 28 Sample Size Required
2 2 z + z α σ 1−β 1− 2 n = ∆2 where 1 − β ≡ desired power α ≡ desired signi cance level (two-sided) σ ≡ population standard deviation ∆ = µ0˘µ1 ≡ the dierence worth de
Example: Sample Size How large a sample is needed for a one-sample z test with 90% power and α = 0.05 (two-tailed) when σ = 40? Let H0 : µ = 170 and H1 : µ = 190 (thus, ∆ = µ0 − µ1 = 170 − /190 = −20)
2 2 z1 + z α 2 2 σ −β 1− 2 40 (1.28 + 1.96) n = = = 41.99 2 (−20)2 Round up to 42 to ensure adequate power. ioc.pdf
[email protected] ICY0006: Lecture 9 25 / 28 ioc.pdf [email protected] ICY0006: Lecture 9 26 / 28 Conclusions in Hypothesis Testing
Always test the null hypothesis
1 Fail to reject the H0 2 Reject the H0 need to formulate correct wording of nal conclusion (see next slide)
ioc.pdf
[email protected] ICY0006: Lecture 9 27 / 28 Wording of Conclusions in Hypothesis Tests
ioc.pdf
[email protected] ICY0006: Lecture 9 28 / 28