Chapter 13 Contrasts and Custom Hypotheses
Total Page:16
File Type:pdf, Size:1020Kb
Chapter 13 Contrasts and Custom Hypotheses Contrasts ask specific questions as opposed to the general ANOVA null vs. alter- native hypotheses. In a one-way ANOVA with a k level factor, the null hypothesis is µ1 = ··· = µk, and the alternative is that at least one group (treatment) population mean of the outcome differs from the others. If k = 2, and the null hypothesis is rejected we need only look at the sample means to see which treatment is \better". But if k > 2, rejection of the null hypothesis does not give the full information of interest. For some specific group population means we would like to know if we have sufficient evidence that they differ from certain other group population means. E.g., in a test of the effects of control and two active treatments to increase vocabulary, we might find that based on a the high value for the F-statistic we are justified in rejecting the null hypothesis µ1 = µ2 = µ3. If the sample means of the outcome are 50, 75 and 80 respectively, we need additional testing to answer specific questions like \Is the control population mean lower than the average of the two active treatment population means?" and \Are the two active treatment population means different?" To answer questions like these we frame \custom" hypotheses, which are formally expressed as contrast hypothesis. Comparison and analytic comparison are other synonyms for contrast. 319 320 CHAPTER 13. CONTRASTS AND CUSTOM HYPOTHESES 13.1 Contrasts, in general A contrast null hypothesis compares two population means or combinations of pop- ulation means. A simple contrast hypothesis compares two population means, e.g. H0 : µ1 = µ5. The corresponding inequality is the alternative hypothesis: H1 : µ1 6= µ5. A contrast null hypotheses that has multiple population means on either or both sides of the equal sign is called a complex contrast hypothesis. In the vast majority of practical cases, the multiple population means are combined as µ1+µ2 µ3+µ4+µ5 their mean, e.g., the custom null hypothesis H0 : 2 = 3 represents a test of the equality of the average of the first two treatment population means to the average of the next three. An example where this would be useful and interesting is when we are studying five ways to improve vocabulary, the first two of which are different written methods and the last three of which are different verbal methods. It is customary to rewrite the null hypothesis with all of the population means on one side of the equal sign and a zero on the other side. E.g., H0 : µ1 − µ5 = 0 µ1+µ2 µ3+µ4+µ5 or H0 : 2 − 3 = 0. This mathematical form, whose left side is checked for equality to zero is the standard form for a contrast. In addition to hypothesis testing, it is also often of interest to place a confidence interval around a contrast of population means, e.g., we might calculate that the 95% CI for µ3 − µ4 is [-5.0, +3.5]. As in the rest of classical statistics, we proceed by finding the null sampling distribution of the contrast statistic. A little bit of formalism is needed so that we can enter the correct custom information into a computer program, which will then calculate the contrast statistic (estimate of the population contrast), the standard error of the statistic, a corresponding t-statistic, and the appropriate p- value. As shown later, this process only works under the special circumstances called \planned comparisons"; otherwise it requires some modifications. Let γ (gamma) represent the population contrast. In this section, will use an example from a single six level one-way ANOVA, and use subscripts 1 and 2 to distinguish two specific contrasts. As an example of a simple (population) contrast, define γ1 to be µ3 − µ4, a contrast of the population means of the outcomes for the third vs. the fourth treatments. As an example of a complex contrast let γ2 µ1+µ2 µ3+µ4+µ5 be 2 − 3 , a contrast of the population mean of the outcome for the first two treatments to the population mean of the outcome for the third through fifth treatments. We can write the corresponding hypothesis as H01 : γ1 = 0;HA1 : 13.1. CONTRASTS, IN GENERAL 321 γ1 6= 0 and H02 : γ2 = 0;HA2 : γ2 6= 0. If we call the corresponding estimates, g1 and g2 then the appropriate estimates y¯1+¯y2 y¯3+¯y4+¯y5 are g1 =y ¯3 − y¯4 and g2 = 2 − 3 . In the hypothesis testing situation, we are testing whether or not these estimates are consistent with the corresponding null hypothesis. For a confidence interval on a particular population contrast (γ), these estimates will be at the center of the confidence interval. In the chapter on probability theory, we saw that the sampling distribution of any of the sample means from a (one treatment) sample of size n using the assump- 2 tions of Normality, equal variance, and independent errors isy ¯i ∼ N(µi; σ =n), i.e., across repeated experiments, a sample mean is Normally distributed with the \cor- rect" mean and the variance equal to the common group variance reduced by a factor of n. Now we need to find the sampling distribution for some particular combination of sample means. To do this, we need to write the contrast in \standard form". The standard form involves writing a sum with one term for each population mean (µ), whether or not it is in the particular contrast, and with a single number, called a contrast coefficient in front of each population mean. For our examples we get: γ1 = (0)µ1 + (0)µ2 + (0)µ3 + (1)µ4 + (−1)µ5 + (0)µ6 and γ2 = (1=2)µ1 + (1=2)µ2 + (−1=3)µ3 + (−1=3)µ4 + (−1=3)µ5 + (0)µ6: In a more general framing of the contrast we would write γ = C1µ1 + ··· + Ckµk: In other words, each contrast can be summarized by specifying its k coefficients (C values). And it turns out that the k coefficients are what most computer programs want as input when you specify the contrast of a custom null hypothesis. In our examples, the coefficients (and computer input) for null hypothesis H01 are [0, 0, 1, -1, 0, 0], and for H02 they are [1/2, 1/2, -1/3, -1/3, -1/3, 0]. Note that the zeros are necessary. For example, if you just entered [1, -1], the computer would not understand which pair of treatment population means you want it to compare. Also, note that any valid set of contrast coefficients must add to zero. 322 CHAPTER 13. CONTRASTS AND CUSTOM HYPOTHESES It is OK to multiply the set of coefficients by any (non-zero) number. E.g., we could also specify H02 as [3, 3, -2, -2, -2, 0] and [-3, -3, 2, 2, 2, 0]. These alternate contrast coefficients give the same p-value, but they do give different estimates of γ, and that must be taken in to account when you interpret confidence intervals. If you really want to get a confidence interval on the difference in average group population outcome means for the first two vs. the next three treatments, it will be directly interpretable only in the fraction form. A positive estimate for γ indicates higher means for the groups with positive coefficients compared to those with negative coefficients, while a negative estimate for γ indicates higher means for the groups with negative coefficients compared to those with positive coefficients To get a computer program to test a custom hypothesis, you must enter the k coefficients that specify that hypothesis. If you can handle a bit more math, read the theory behind contrast estimates provided here. The simplest case is for two independent random variables Y1 and Y2 for which the population means are µ1 and µ2 and the variances are 2 2 σ1 and σ2. (We allow unequal variance, because even under the equal variance assumption, the sampling distribution of two means, depends on their sample sizes, which might not be equal.) In this case it is true that 2 2 2 2 E(C1Y1 + C2Y2) = C1µ1 + C2µ2 and Var(C1Y1 + C2Y2) = C1 σ1 + C2 σ2. If in addition, the distributions of the random variables are Normal, we can conclude that the distribution of the linear combination of the random 2 2 variables is also Normal. Therefore Y1 ∼ N(µ1; σ1);Y2 ∼ N(µ2; σ2); ) 2 2 2 2 C1Y1 + C2Y2 ∼ N(C1µ1 + C2µ2;C1 σ1 + C2 σ2): 13.1. CONTRASTS, IN GENERAL 323 We will also use the fact that if each of several independent random variables has variance σ2, then the variance of a sample mean of n of these has variance σ2=n. From these ideas (and some algebra) we find that in a one-way ANOVA with k treatments, where the group sample means are independent, if 2 we let σ be the common population variance, and ni be the number of ¯ ¯ subjects sampled for treatment i, then Var(g) = Var(C1Y1 + ··· + CkYk) = 2 Pk 2 σ [ i=1(Ci =ni)]. In a real data analysis, we don't know σ2 so we substitute its estimate, the within-group mean square. Then the square root of the estimated variance is the standard error of the contrast estimate, SE(g). For any normally distributed quantity, g, which is an estimate of a parameter, γ, we can construct a t-statistic, (g − γ)=SE(g). Then the sampling distribution of that t-statistic will be that of the t-distribution with df equal to the number of degrees of freedom in the standard error (dfwithin).