Bennett, PJ PSY710 Chapter 3 Y¯u is simply the mean of the group means; di↵erences in the size of the groups (if they exist) are ignored, and so Y¯u is said to be the unweighted mean of the group means.↵ ˆj is simply the di↵erence between the mean of group j and Y¯u.Forthe reduced model, setting the one free parameter, µ,tothegrandaverage,Y¯ ,minimizes the sum of squared residuals. 3.3.1 F formula Next, we need to derive a quantitative measure of the relative goodness-of-fit of the two models. We denote the sum of squared residuals for the best-fitting full and reduced models as EF and ER, respectively. Associated with EF and ER are degrees-of-freedom df = N a and df = N 1, respectively, where N is the total F − R − number of observations and a is the number of groups. Note that dfR dfF = a 1 is the di↵erence between the number of parameters estimated in the full− model (3−↵’s and 1 intercept) and the reduced model (1 intercept). The formula for computing the di↵erence between the two models is (E E )/(df df ) F = R − F R − F (13) EF /dfF Equation 13 can be used to compare all nested linear models. All tests in ANOVA, analysis of covariance, and multiple regression can be computed using this formula. Bennett, PJ PSYCH 710 Chapter 4 3.3.2 Null Hypothesis Testing Finally, we are in a position to evaluate the hypothesis of no di↵erence between the goodness-of-fit of the full and reduced models. Note that this comparison is equivalent to evaluating the hypothesis that all of the groups have the same mean; or Notes on Maxwell & Delaney (equivalently) that all ↵j’s areOmnibus zero. More formally, vs. Focussed we are comparing theF hypothesestests PSYCHPSYCH 710 710 H0: ↵1 = ↵2 = = ↵a =0 H1: ↵ ···=0 j 6 Linear Contrasts 4 Chapter 4 - Individual Comparisons of Means The null hypothesis is that all of the e↵ects are zero, and therefore that all group means are equal. The alternative• A significant hypothesis omnibus is that at F least test one supports e↵ect is nota very zero, general and hypothesis 4.1 Omnibus vs. Focused Tests Week 4 therefore that not all group means- not are all equal. means When are the equal residuals, eij,aredistributed as independent, normal random variables, with mean of zero and a constant variance, For the one-way ANOVAs that we have beenProf. discussing Patrick so Bennett far, a significant F test means that we reject the null hypothesis H0: ↵1 = ↵2 = = ↵a = 0 in favor of the alternative hypothesisthen H1:F in Equation 13 follows- annotF distributionall group effects with (df areR dfzeroF )anddf F degrees of ↵ = 0 (for at least one group, j). So, when··· a 3, a significant omnibus F test does not tell − j 6 ≥ freedom in the numerator and• Significant denominator, F doesn’t repsectively tell (Figureus how 1).group Under means the null differ us precisely how the group means di↵er from each other. To address this question, and makehypothesis, our therefore, large values of F should be relatively rare (Figure 2). Using inferences more precise, we need to use more focused comparisons of the group means. There is another reason for using focused comparisons. In a sense, the omnibus F test examines • Generality of omnibus F often comes at cost of reduced power all possible patterns of di↵erences among the group means. The advantage of this test is obvious: 4 we don’t need to specify the pattern of e↵ects in advance. However, there is a penalty to this general approach, namely reduced power. As we shall see, asking a focused question can result in a statistical test with much greater power than the omnibus F test. The following example shows that a focused comparison among means can be significant even when the omnibus F test is not significant. >set.seed(seed=934) >y<-c(rnorm(n=60,mean=100,sd=20),rnorm(n=10,110,sd=20)) Bennett, PJ PSYCH 710 Chapter 4 >g<-factor(rep(seq(1,7,1),each=10),labels="g",ordered=FALSE) If you examine the line y<-c(rnorm(n=60,...,sd=20));carefully, you will see that it consists of 60 random numbers drawn from a gaussian distribution with a mean of 100, and 10 random numbers drawn from a gaussian distribution with a mean of 110. Our grouping variable, g, will be used to group the y’s into 7 sets of 10 numbers each. Boxplots of the data are shown in Figure 1. Omnibus vs. Focussed F tests >boxplot(y~g) Linear Comparisons OmnibusNow F we test conduct is not significant: a standard ANOVA. Bennett, PJ PSYCH 710 Chapter 4 >lm01<-lm(y~g); 160 Bennett,>anova(lm01); PJ PSYCH 710 Chapter 4 140 >source(url("http://psycserv.mcmaster.ca/bennett/psy710/Rscripts/linear_contrast_v2.R"))Analysis of Variance Table Bennett, PJ PSYCH 710 Chapter 4 • focussed comparison among group means [1] "loading function linear.comparison" >source(url("http://psycserv.mcmaster.ca/bennett/psy710/Rscripts/linear_contrast_v2.R"))Response: y 120 The sourceDfcommand Sum Sq loaded Mean Sq several F value commands Pr(>F) that can be used to perform a linear comparison among means in a • can be more direct test of hypothesis of interest [1] "loading function linear.comparison" one-wayg63431571.81.7260.13>source(url("http://psycserv.mcmaster.ca/bennett/psy710/Rscripts/linear_contrast_v2.R")) design. Next, I have to specify my contrast weights : 100 TheResidualssource 63command 20873 loaded 331.3 several commands that can be used to perform a linear comparison among means in a >my.weights<-c(-1,-1,-1,-1,-1,-1,6)[1] "loading function linear.comparison" • generally more powerful, but less general, than omnibus F one-way design. Next, I have to specify my contrast weights : 80 We willNotice discuss that the the meaning omnibus of theF contrasttest fails weights to find in a the significant following section. di↵erence Finally, among I use the the groups.linear.comparison Next, we The source command loaded several commands that can be used to perform a linear comparison among means in a Yet>my.weights<-c(-1,-1,-1,-1,-1,-1,6)command,compare a linear group which comparison was 7 to loaded the of other intog7 to R’s groups.g1-g6 workspace is Specifically,significant: by the previous we asksource whethercommand, the mean to perform of group the linear 7 di↵ comparison:ers from theone-way mean design. of the Next, 6 other I have groups. to specify To my docontrast this analysis, weights I: first need to download60 some R commands: We>my.contrast<-linear.comparison(y,g,c.weights=my.weights) will discuss the meaning of the contrast weights in the following section. Finally, I use the linear.comparison >my.weights<-c(-1,-1,-1,-1,-1,-1,6) command, which was loaded into R’s workspace by the previous source command, to perform the● linear comparison: [1] "computing linear comparisons assuming equal variances among groups" We will discuss the meaning of the contrast weights in the following section. Finally,40 I use the linear.comparison >my.contrast<-linear.comparison(y,g,c.weights=my.weights)[1] "C 1: F=7.218, t=2.687, p=0.009, psi=100.218,1 CI=(-2.897,203.332), adj.CI=g1 g2 (25.673,174.762)"g3 g4 g5 g6 g7 command, which was loaded into R’s workspace by the previous source command, to perform the linear comparison: [1]>my.contrast[[1]]$F "computing linear comparisons assuming equal variances among groups" [1]>my.contrast<-linear.comparison(y,g,c.weights=my.weights) "C 1: F=7.218, t=2.687, p=0.009, psi=100.218, CI=(-2.897,203.332), adj.CI= (25.673,174.762)" [1] 7.21771 >my.contrast[[1]]$F[1] "computing linear comparisons assuming equal variances among groups"Figure 1: Boxplots of y data for di↵erent groups, g. >my.contrast[[1]]$p.2tailed[1] "C 1: F=7.218, t=2.687, p=0.009, psi=100.218, CI=(-2.897,203.332), adj.CI= (25.673,174.762)" [1] 7.21771 [1]>my.contrast[[1]]$F 0.00921941 >my.contrast[[1]]$p.2tailed [1]The 7.21771F test for this linear contrast, or linear comparison, is significant, even though the omnibus [1] 0.00921941 F>my.contrast[[1]]$p.2tailedtest was not. The F test for this linear contrast, or linear comparison, is significant, even though the omnibus [1] 0.00921941 F4.1.1test was using not. standard R commands 2 In thisThe sectionF test I for will this showlinear you contrast how to, performor linear thecomparison linear comparison, is significant, with even R’s built-in though commands the omnibus 4.1.1(i.e.,F test without was using not. using standard the linear.comparison R commands command). The first step is to inform R that I want to perform a particular comparison among the various groups represented by the factor g: In this section I will show you how to perform the linear comparison with R’s built-in commands (i.e.,>contrasts(g)<-my.weights4.1.1 without using using standard the linear.comparison R commands command). The first step is to inform R that I want to performNext,In this Iperform section a particular I an will ANOVA. comparison show you Note how among that to I’m performthe using various the groupsaov linearcommand, represented comparison rather by with thethan R’s factor the built-inlmg:command: commands >contrasts(g)<-my.weights>my.aov<-aov(y~g)(i.e., without using the linear.comparison command).