5.2 Principles of Experimental Design Cell Phone Use and Driving We want to see if talking on a hands-free cell phone distracts drivers. We have a simulator and 40 randomly chosen undergraduate students. We need to design an experiment. We will use braking time to determine how distracted they are. Randomized Comparative Experiments This is the type of experiments we will be dealing with in this class. Randomization produces 2 groups of subjects that we expect to be similar in all respects before the treatments are applied. Comparative design helps ensure that influences by variables, other than the one we want to measure, are equal on both groups. This lets us assume that the difference in our variable of interest is due to our treatment. Goal of experiments: To Find Statistical Significance This means that the observed effect is so large that it is very unlikely to occur by random chance. Tells us that the treatment we are testing is having an actual effect on the subjects. Experimental Design Basic Principles of statistical design of experiments: Control the effects of lurking variables on the response (compare 2 or more treatments) Replicate each treatment on many units to reduce chance of variation in the results Randomize—use impersonal chance to assign experimental units to treatments Experimental Design Completely randomized design—all units are assigned at random. Each unit has the same probability of being chosen. Our cell phone experiment is an example of this. Experimental Design Block design —group of units that are known to be similar before the treatments. This would be used if we know that the drug we are testing effects men and women differently. Form blocks based on most important, unavoidable sources of variability. We use this to mitigate variables that we cannot control and do not want to measure. This is only used when the blocking variable has an obvious effect on the response variable. It will not always be gender. Soy-Bean Yields We have 2 farms that we can use for testing the effect of tillage type (two types) and pesticide application (three types) on soy-bean yields. We need to use both farms in order to have enough samples. We know that soil type and fertility vary greatly by location. Experimental Design Matched Pairs design —block 2 units together (one control; one treatment) based on variables (that you’re not interested in testing) that you believe will effect the response variable. Match units with another unit similar to it. You can be matched with yourself. Note: The two units are not independent of each other, since they were paired specifically. Cell Phone Experiment - Revisited The driving-cell phone experiment was actually done with a matched pairs design. Each subject was paired with themselves. Double blind experiment —neither subjects nor those who measure the response variable know which treatment a subject received Helps treat every subject the same, regardless of treatment Lack of realism —cannot realistically duplicate conditions we want to study Keep in Mind Statistical analysis of an experiment cannot tell us how far the results will generalize to other settings. This is why being sure to select our sample appropriately, from our population of interest is very important. Example: Ultra-marathon runners tend to develop respiratory infections after the race. Will 600 mg of vitamin C daily reduce these infections? Researchers randomly assigned runners to receive either vitamin C or a placebo. Separately, they also randomly assigned these treatments to a group of non-runners. McDonalds Example Do consumers prefer the taste of a cheeseburger from McDonald’s or from Wendy’s in a blind test in which neither burger is identified? Describe the design of a matched pairs experiment..