Completely Randomized Designs (CRD) One-Way ANOVA Lukas Meier, Seminar für Statistik Example: Meat Storage Study (Kuehl, 2000, Example 2.1) . Researcher wants to investigate the effect of packaging on bacterial growth of stored meat. Some studies suggested controlled gas atmospheres as alternatives to existing packaging. Different treatments (= packaging types) . Commercial plastic wrap (ambient air) Current techniques (control groups) . Vacuum package . 1% CO, 40% O2, 59% N New techniques . 100% CO2 . Experimental units: 12 beef steaks (ca. 75g). Measure effectiveness of packaging by measuring how successful they are in suppressing bacterial growth. 1 Example: Meat Storage Study . Three beef steaks were randomly assigned to each of the packaging conditions. Each steak was packaged separately in its assigned condition. Response: (logarithm of the) number of bacteria per square centimeter. The number of bacteria was measured after nine days of storage at 4 degrees Celsius in a standard meat storage facility. 2 First Step (Always): Exploratory Data Analysis . If very few observations: Plot all data points. With more observations: Use boxplots (side-by-side) . Alternatively: Violin-plots, histogram side-by-side, … . See examples in R: 02_meat_storage.R Such plots typically give you the same (or even more) information as a formal analysis (see later). 3 Side Remark: Factors . Categorical variables are also called factors. The different values of a factor are called levels. Factors can be nominal or ordinal (ordered) . Hair color: {black, blond, …} nominal . Gender: {male, female} nominal . Treatment: {commercial, vacuum, mixed, CO2} nominal . Income: {<50k, 50-100k, >100k} ordinal . Useful functions in R: . factor . as.factor . levels 4 Completely Randomized Design: Formal Setup . Compare 푔 treatments . Available resources: 푁 experimental units . Need to assign the 푁 experimental units to 푔 different treatments (groups) having 푛푖 observations each, 푖 = 1, … , 푔. Of course: 푛1 + 푛2 + … + 푛푔 = 푁. Use randomization: . Choose 푛1 units at random to get treatment 1, . 푛2 units at random to get treatment 2, . ... This randomization produces a so called completely randomized design (CRD). 5 Setting up the Model . Need to set up a model in order to do statistical inference. Good message: problem looks rather easy. Bad message: Some complications ahead regarding parametrization. 6 Remember: Two Sample 푡-Test for Unpaired Data . Model 2 . 푋푖 i. i. d. ∼ 푁 휇푋, 휎 , 푖 = 1, … , 푛 2 . 푌푗 i. i. d. ∼ 푁 휇푌, 휎 , 푗 = 1, … , 푚 . 푋푖, 푌푗 independent . 풕-Test . 퐻0: 휇푋 = 휇푌 . 퐻퐴: 휇푋 ≠ 휇푌 (or one-sided) (푋 −푌 ) . 푇 = 푛 푚 ∼ 푡 under 퐻 1 1 푛+푚−2 0 푆 + 푝표표푙 푛 푚 . Allows us to test or construct confidence intervals for the true (unknown) difference 휇푋 − 휇푌. Note: Both groups have their “individual” mean but they share a common variance (can be extended to other situations). 7 From Two to More Groups . In the meat storage example we had 4 groups. Hence, the 푡-test is not directly applicable. Could try to construct something using only pairs of groups (e.g., doing all pairwise comparisons). Will do so later. Now we want to expand the model that we used for the two sample 푡-test to the more general situation of 푔 > 2 groups. As we might run out of letters, we use a common letter (say 푌) for all groups and put the grouping and replication information in the index. 8 Cell Means Model . We need two indices to distinguish between the different treatments (groups) and the different observations. Let 푌푖푗 be the 푗th observation in the 푖th treatment group, 푖 = 1, … , 푔; 푗 = 1, … , 푛푖. Cell means model: Every group (treatment) has its own mean value, i.e. 2 푌푖푗 ∼ 푁 휇푖, 휎 , independent group 푖 observation 푗 . Also called separate means model. Note: Variance constant across groups (as for standard two-sample 푡-test!) 9 Illustration of Cell Means Model . See R-Code: 02_model_illustration.R . Or visit https://gallery.shinyapps.io/anova_shiny_rstudio/ . Why cell means? Have a look at meat storage data: Commercial Vacuum Mixed CO2 7.66 5.26 7.41 3.51 6.98 5.44 7.33 2.91 7.80 5.80 7.04 3.66 cell 10 Cell Means Model: Alternative Representation 2 . We can “extract” the deterministic part in 푌푖푗 ∼ 푁(휇푖, 휎 ). Leads to 푌푖푗 = 휇푖 + 휖푖푗 2 with 휖푖푗 i. i. d. ∼ 푁 0, 휎 . The 휖푖푗’s are random “errors” that fluctuate around zero. In the regression context: . 푌 is the response. Treatment is a categorical predictor (a factor). Hence, this is nothing else than a regression model with a categorical predictor. 11 Yet Another Representation (!) . We can also write 휇푖 = 휇 + 훼푖, 푖 = 1, … , 푔. E.g., think of 휇 as a “global mean” and 훼푖 as the corresponding deviation from the global mean. 훼푖 is also called the 푖th treatment effect. This looks like a needless complication now, but will be very useful later (with factorial treatment structure). Unfortunately this model is not identifiable anymore. Reason: 푔 + 1 parameters (휇, 훼1, … , 훼푔) for 푔 different means… 12 Ensuring Identifiability . Need side constraint: many options available. Sum of the treatment effects is zero, i.e. 훼푔 = − 훼1 + ⋯ 훼푔−1 (R: contr.sum) . Sum of weighted treatment effects is zero: … (R: do manually) . Set 휇 = 휇1, hence 훼1 = 0, 훼2 = 휇2 − 휇1, 훼3 = 휇3 − 휇1, … i.e. a comparison with group 1 as reference level. (R: contr.treatment) . Only 푔 − 1 elements of the treatments effect are allowed to vary freely. We also say that the treatment effect has 푔 − 1 degrees of freedom (df). 13 Encoding Scheme of Factors . The encoding scheme (i.e., the side constraint being used) of a factor is called contrast in R. To summarize: we have a total of 푔 parameters: 휇, 훼1, … , 훼푔−1 to parametrize the 푔 group means 휇1, … , 휇푔. The interpretation of the parameters 휇, 훼1, … , 훼푔−1 strongly depends on the parametrization that is being used. We will re-discover the word “contrast” in a different way later… 14 Parameter Estimation . Choose parameter estimates 휇 , 훼 1, … , 훼 푔−1 such that model fits the data “well”. Criterion: Choose parameter estimates such that 푔 푛푖 2 푖=1 푗=1 푦푖푗 − 휇 − 훼 푖 is minimal (so called least squares criterion, exactly as in regression). The estimated cell means are simply 휇 푖 = 휇 + 훼 푖 15 Illustration of Goodness of Fit . See blackboard (incl. definition of residual) 16 Some Notation Symbol Meaning Formula 푛푖 푦푖⋅ Sum of all values in group 풊 푦푖⋅ = 푦푖푗 푗=1 푛 1 푖 1 푦 푖⋅ Sample average in group 풊 푦 푖⋅ = 푦푖푗 = 푦푖⋅ 푛푖 푛푖 푗=1 푔 푛푖 푦⋅⋅ Sum of all observations 푦⋅⋅ = 푦푖푗 푖=1 푗=1 푦 푦 Grand mean 푦 = ⋅⋅ ⋅⋅ ⋅⋅ 푁 Rule: If we replace an index with a dot (“⋅”) it means that we are summing up values over that index. 17 Parameter Estimates, the Other Way Round . “Obviously”, the 휇 푖’s that minimize the least squares criterion are 휇 푖 = 푦 푖⋅. Means: Expectation of group 푖 is estimated with sample mean of group 푖. ′ . The 훼푖푠 are then simply estimated by applying the corresponding parametrization, i.e. 훼 푖 = 휇 푖 − 휇 = 푦 푖⋅ − 푦 ⋅⋅ The fitted values 휇 푖 (and the residuals) are independent of the parametrization, but the 훼 푖’s (heavily) depend on it! 18 Parameter Estimation . We denote residual (or error) sum of squares by 푔 푛푖 2 푆푆퐸 = 푖=1 푗=1 푦푖푗 − 푦 푖⋅ empirical variance in group 푖 2 . Estimator for 휎 is 푀푆퐸, mean squared error, i.e. 2 1 1 푔 2 휎 = 푀푆 = 푆푆 = 푛 − 1 푠 퐸 푁−푔 퐸 푁−푔 푖=1 푖 푖 . This is an unbiased estimator for 휎2 (reason for 푁 − 푔 instead of 푁 in the denominator). We also say that the error estimate has 푁 − 푔 degrees of freedom (푁 observations, 푔 parameters) or 푔 푁 − 푔 = 푖=1(푛푖 − 1 ). 19 Estimation Accuracy . Standard errors for the parameters (using the sum of weighted treatment effects constraint) Parameter Estimator Standard Error 휇 푦 ⋅⋅ 휎/ 푁 휇푖 푦 푖⋅ 휎/ 푛푖 1 1 훼푖 푦 푖⋅ − 푦 ⋅⋅ 휎 − 푛푖 푁 1 1 휇푖 − 휇푗 = 훼푖 − 훼푗 푦 푖⋅ − 푦 푗⋅ 휎 + 푛푖 푛푗 . Therefore, a 95% confidence interval for 훼푖 is given by 0.975 1 1 훼 푖 ± 푡푁−푔 ⋅ 휎 − 푛푖 푁 푁 − 푔 degrees of freedom because of 20 97.5% quantile of 푡푁−푔 distribution degrees of freedom of 푀푆퐸 Single Mean Model . Extending the null-hypothesis of the 푡-test to the situation where 푔 > 2, we can (for example) use the (very strong) null-hypothesis that treatment has no effect on the response. In such a setting, all values (also across different treatments) fluctuate around the same “global” mean 휇. 2 . Model reduces to: 푌푖푗 i. i. d. ∼ 푁(휇, 휎 ) 2 . Or equivalently: 푌푖푗 = 휇 + 휖푖푗, 휖푖푗 i. i. d. ∼ 푁 0, 휎 . This is the single mean model. 21 Comparison of models . Note: Models are “nested”, single mean model is a special case of cell means model. Or: Cell means model is more flexible than single mean model. Which one to choose? Let a statistical test decide. Cell means model Single mean model 22 Analysis of Variance (ANOVA) . Classical approach: decompose “variability” of response into different “sources” and compare them. More modern view: Compare (nested) models. In both approaches: Use statistical test with global null hypothesis 퐻0: 휇1 = 휇2 = ⋯ = 휇푔 versus the alternative 퐻퐴: 휇푘 ≠ 휇푙 for at least one pair 푘 ≠ 푙 . 퐻0 says that the single mean model is ok. 퐻0 is equivalent to 훼1 = 훼2 = … = 훼푔 = 0.