AP Statistics – 5.2 Designing Experiments – Notes
1. Observational Study – we observe individuals and measure variables of interest but do not attempt to influence the responses. 2. Experiment – we deliberately impose some treatment on (that is, do something to) individuals in order to observe their responses. a. Experimental Units – individuals in which the experiment is done b. Subjects – humans in an experiment c. Treatment – specific experimental condition applied to the units d. Double-Blind Experiment –neither the subjects nor those who measure the response variable know which treatment a subject received.
3. Purpose of Experiments – to reveal the response of one variable to changes in other variables. Experiments can give good evidence for causation. a. Factors – explanatory variables in experiments b. Level – When experiments study joint effects of several factors, each treatment is formed by combining a specific value (level) of each factor.
4. Forms of Control a. Comparison – determine differences/results from two or more groups (only valid when the treatments are given to similar groups of experimental units). Comparison helps ensure that all outside influences affect both groups the same. i. Control Group – the group which receives a placebo (fake treatment) in order to control lurking/confounding variables. ii. Experimental Group – the group that receives the actual treatment b. Replication – use enough subjects to reduce chance variation. c. Randomization – use chance to divide experimental units into groups (without judgment of the experimenter or any characteristic of the experimental units). The two groups are similar before treatments are applied. 5. Random Selection & Random Assignment a. Random Selection is where the subjects are randomly selected from the population. The advantage of this is that the sample is representative of the population which allows us to generalize our findings to the population. b. Random Assignment (Randomization) is when we randomly assign subjects to a treatment group. The advantage of randomization is that both groups in theory are identical except for the treatment. This allows us to establish a cause-and-effect relationship.
6. Principles of Experimental Design a. Control the effects of lurking variables on the response, most simply by comparing two or more treatments. b. Replicate each treatment on many units to reduce chance variation in the results. c. Randomize use impersonal chance to assign experimental units to treatments.
7. Randomized Comparative Experiments a. Statistically Significant – an observed effect so large that it would rarely occur by chance b. Completely Randomized Design – all experimental units are allocated at random among all treatments
8. Blocking a. Block is a group of experimental units or subjects that are known before the experiment to be similar in some way that is expected to systematically affect the response to the treatments. b. Block Design – the random assignment of units to treatments is carried out separately within each block (any size). There is no randomization when dividing the blocks. We can draw conclusions about each block. c. Randomization vs Blocking i. You block to control for the variables you know about that might affect the response. ii. You randomize to control for the variables you do not know about d. Matched Pairs Design – common form of blocking for comparing just two treatments. i. In some matched pairs designs, each subject receives both treatments in a random order. ii. In others, the subjects are matched in pairs as closely as possible, and one subject in each pair receives one treatment, and the other subject receives the other treatment.
9. Cautions about Experimentation a. Lack of Realism – make sure the experiment is as similar to the actual situation as possible. b. Statistical analysis of an experiment cannot tell us how far the results will generalize to other settings.