Random Effects ANOVA
Total Page:16
File Type:pdf, Size:1020Kb
Random Effects ANOVA James H. Steiger Department of Psychology and Human Development Vanderbilt University James H. Steiger (Vanderbilt University) 1 / 29 Random Effects ANOVA 1 Introduction 2 A One-Way Random Effects ANOVA The Basic Model Calculations Expected Mean Squares and F Test 3 Two-Way Model with Both Effects Random 4 A Typical Two-Way Model with One Random Effect Factor B Factor A The Basic Model Computations and Expected Mean Squares 5 Two-Way Mixed Model ANOVA: An Example James H. Steiger (Vanderbilt University) 2 / 29 Introduction Introduction So far, in our coverage of ANOVA, we have dealt only with the case of fixed-effects models. With fixed-effects factors, the effects of the different levels of the independent variable are treated as fixed constants to be estimated. In this module, we introduce the idea of a random-effect factor. The treatment effects for random effects factors are treated as random variables, or, equivalently, random samples from a population of possible treatment effects. When models include random effects, the Expected Mean Squares will often differ from the same model with fixed effects. In most cases, this will affect how F -tests are performed, and the distribution of the F -statistic and its degrees of freedom. James H. Steiger (Vanderbilt University) 3 / 29 A One-Way Random Effects ANOVA A One-Way Random Effects ANOVA Suppose you are interested in the natural degree of variation in sodium content across brands of beer. There are hundreds of brands of beer being sold in the U.S., and you only have time to test 8. You select your 8 brands randomly from a list of all brands available, and you test 6 bottles of beer for each brand. James H. Steiger (Vanderbilt University) 4 / 29 A One-Way Random Effects ANOVA A One-Way Random Effects ANOVA Although we can visualize an “effect” for each brand, we recognize that this effect need not be conceptualized as a fixed value | in an important sense, the effect of the first brand is a random variable, since that brand was sampled from a larger set. If we were only interested in generalizing to the 8 brands in the study, we could choose to regard the effect of each beer as a fixed constant. But because we randomly sampled the beer brands from a larger population, we actually can generalize back to the entire population. Let's see how. James H. Steiger (Vanderbilt University) 5 / 29 A One-Way Random Effects ANOVA The Basic Model The first thing we have to realize is that the basic ANOVA model has changed. We now have Yij = µj + ij = µ + αj + ij 1 ≤ i ≤ n; 1 ≤ j ≤ a (1) with 2 ij ∼ i.i.d. N(0; σe ) (2) 2 αij ∼ i.i.d N(0; σA) (3) I should note immediately that many books distinguish between fixed and random effects by using Greek letters for the former and standard Arabic letters for the latter. We are not using that convention here, as neither textbook used recently in this course employs it. However, there are advantages to a notation that explicitly \types" its effects as fixed or random. James H. Steiger (Vanderbilt University) 6 / 29 A One-Way Random Effects ANOVA The Basic Model A One-Way Random Effects ANOVA The Basic Model This shift in models means that now there are two sources of random variation on the right side of the model equation, while before there was only one. One immediate consequence of this fact is that now observations are correlated within group! Let's use standard linear combination theory to derive the variances and covariances of scores. Since the effects αj and the errors ij are independent, it follows that 2 2 Var(Yij ) = σA + σe (4) James H. Steiger (Vanderbilt University) 6 / 29 A One-Way Random Effects ANOVA The Basic Model A One-Way Random Effects ANOVA The Basic Model On the other hand, consider two observations Y11 and Y21, both in group 1. What is their covariance? We can solve directly for this, using results from Psychology 310. Since Y11 = µ + α1 + 11, Y21 = µ + α1 + 21, and covariances are unaffected by additive constants, we can see that the covariance between Y11 and Y21 is the covariance between α1 + 11 and α1 + 21. Remember the heuristic rule? We simply multiply the two expressions and apply a conversion rule. Here is the multiplication. 2 (α1 + 11)(α1 + 21) = α1 + 1121 + α111 + α121 (5) Next we apply the conversion rule Cov(Y11; Y21) = Var(α1) + Cov(11; 21) + Cov(α1; 11) + Cov(α1; 21) 2 = σA + 0 + 0 + 0 2 = σA (6) James H. Steiger (Vanderbilt University) 7 / 29 A One-Way Random Effects ANOVA The Basic Model A One-Way Random Effects ANOVA The Basic Model So while the observations within any group are independent in the fixed-effects model, they are correlated in the random effects model. Since the correlation coefficient is the ratio of the covariance to the product of 2 2 standard deviations, and each observation has a variance of σA + σe , it follows that, within any group, pairs of observations observations have a correlation of σ2 ρ = A (7) 2 2 σA + σe This correlation is sometimes called the population intraclass correlation. James H. Steiger (Vanderbilt University) 8 / 29 A One-Way Random Effects ANOVA Calculations A One-Way Random Effects ANOVA Calculations We are very fortunate, in that the sums of squares for the random effects model are calculated in exactly the same way they are calculated for the fixed effects model. James H. Steiger (Vanderbilt University) 9 / 29 A One-Way Random Effects ANOVA Expected Mean Squares and F Test A One-Way Random Effects ANOVA Expected Mean Squares and F Test Remember that earlier in the course, I mentioned that ultimately Expected Mean Squares would play an important role in deciding how to perform an F test on a particular model. We have almost reached that point. Notice in the table below that the expected mean squares for the random effects model are almost identical to those for the fixed effects model. James H. Steiger (Vanderbilt University) 10 / 29 A One-Way Random Effects ANOVA Expected Mean Squares and F Test A One-Way Random Effects ANOVA Expected Mean Squares and F Test The F test construction principle says that, to construct a test for an effect of interest: 1 Take the E(MS) for the effect. Examine which component(s) of the E(MS) involve the effect of interest. Under the null hypothesis, these will be zero. 2 Imagine that the null hypothesis is true. This will cause the terms identified in the previous step to drop out. This revised formula is your \Null Effect Formula." It will be the numerator of your F statistic for this effect. 3 Scan the list of E(MS) formulas, and find a formula that is identical to the \Null Effect Formula." This will be the denominator (error) term for your F test. James H. Steiger (Vanderbilt University) 11 / 29 A One-Way Random Effects ANOVA Expected Mean Squares and F Test A One-Way Random Effects ANOVA Expected Mean Squares and F Test Below is a table with Expected Mean Squares for 1-Way ANOVA with fixed effects, and with random effects. In this case, how do we compute the F statistic for the A effect? Let's do it step by step for the random effects model: Expected MS Factor Fixed Effects Model Random Effects Model A 22 22 σθe + n A σσe + n A S/A 2 2 σ e σ e 1 The MS for your effect (MSA) will be the numerator of the F 2 2 statistic. Take the E(MS) for the effect. (E(MSA) = σe + nσA) Examine which component(s) of the E(MS) involve the effect of 2 interest. (nσA) Under the null hypothesis, these will be zero. 2 2 Imagine that the null hypothesis (H0 : σA = 0), is true. This will cause the term identified in the previous step to drop out. This revised 2 2 2 formula is your \Null Effect Formula." (E(MSA) = σe + nσA = σe ) 3 Scan the list of E(MS) formulas, and find a formula that is identical to the \Null Effect Formula." This will be the denominator (error) 2 term for your F test. Since E(MSS=A) = σe , MSS=A will be our denominator (error) term. James H. Steiger (Vanderbilt University) 12 / 29 A One-Way Random Effects ANOVA Expected Mean Squares and F Test A One-Way Random Effects ANOVA Expected Mean Squares and F Test You can quickly see that the F test for the A effect in the fixed effects model also uses MSS=A as the error term. Thus, by our F -test construction principle, the F statistic is computed as MSA=MSS=A in both models. With the F test computed the same way, one might be tempted to think that it has the same distribution. It does when the null hypothesis is true, but it does not when the null hypothesis is false. James H. Steiger (Vanderbilt University) 13 / 29 A One-Way Random Effects ANOVA Expected Mean Squares and F Test A One-Way Random Effects ANOVA Expected Mean Squares and F Test The general distribution of the F -statistic in the one-way random effects model is 2 nσA (1 + 2 )Fa−1;a(n−1) σe Note that when H0 is true, the distribution has a central F distribution identical to the fixed effects model.