Experiments and Observational Studies Chapter 12 Objectives: • Observational study • Principles of experimental • Retrospective study design • Prospective study • Statistically significant • Experiment • Control group • Experimental units • Blinding • treatment • Placebo • response • Blocking • Factor • Matching • Level • Confounding Observational Study • Observes individuals and records variables of interest but does not attempt to influence the response (does not impose a treatment). – Allows the researcher to directly observe the behavior of interest rather than rely on the subject’s self-descriptions (survey). – Allows the researcher to study the subject in its natural environment, thus removing the potentially biased effect of the unnatural laboratory setting on the subject’s performance (animal behavior). Observational Study • Example: – Researchers compared the scholastic performance of music students with that of non-music students. The music students had a much higher overall grade point average than the non-music students, 3.59 to 2.91. Also, 16% of the music students had all A’s compared with only 5% of the non-music students. Observational Study • In an observational study, researchers don’t assign choices; they simply observe them. – The example looked at the relationship between music education and grades. – Since the researchers did not assign students to get music education and simply observed students “in the wild,” it was an observational study. – Because researchers in the example first identified subjects who studied music and then collected data on their past grades, this was a retrospective study. Retrospective Study • Observational studies that try to discover variables related to rare outcomes, such as specific diseases, are often retrospective. They first identify people with the disease and then look into their history and heritage in search of things that may be related to their condition. • Retrospective studies have a restricted view of the world because they are usually restricted to a small part of the entire population. • Because retrospective studies are based on historical data, they can have errors. – Do you recall exactly what you ate yesterday? How about last Monday? Prospective Study • A somewhat better approach to a observational study, then using historical data such as in a retrospective study, is to identify subjects in advance and collect data as events unfold. This called a prospective study. • In our example studying the relationship between music education and grades, had the researchers identified subjects in advance and collected data over an entire school year or years, the study would have been a prospective study. Observational Study • Observational studies are valuable for discovering trends and possible relationships. • However, it is not possible for observational studies, whether prospective or retrospective, to demonstrate a cause and effect relationship. There are too many lurking variables that may affect the relationship. Lurking Variables • A third unforeseen variable that affects observational studies • EX. A study shows that there’s is a positive association between ice cream sales and drowning. – People eat too much ice cream and therefore drown? – People hear others drowning and go out to buy ice cream because they are depressed? – Lurking Variable: the summer heat temp. Experiment • Definition: Experiment – deliberately imposes some treatment on individuals in order to observe their responses. • Basic Experimental Design – Subject Treatment Observation • The purpose of an experiment is to reveal the response of one variable to changes in other variables, the distinction between explanatory and response variables is essential. Experiment • An experiment is a study design that allows us to prove a cause-and-effect relationship. • In an experiment, the experimenter must identify at least one explanatory variable, called a factor, to manipulate and at least one response variable to measure. • An experiment: – Manipulates factor levels to create treatments. – Randomly assigns subjects to these treatment levels. – Compares the responses of the subject groups across treatment levels. Experiment • In an experiment, the experimenter actively and deliberately manipulates the factors to control the details of the possible treatments, and assigns the subjects to those treatments at random. • The experimenter then observes the response variable and compares responses for different groups of subjects who have been treated differently. Experiment • In general, the individuals on whom or which we experiment are called experimental units. – When humans are involved, they are commonly called subjects or participants. • The specific values that the experimenter chooses for a factor are called the levels of the factor. • A treatment is a combination of specific levels from all the factors that an experimental unit receives. Review - Experimental Terminology • Experimental Units – The individuals or items on which the experiment is performed. – When the experimental units are human beings, the term subject is often used in place of experimental unit. • Response variable – The characteristic of the experimental outcome that is being measured or observed. Review - Experimental Terminology • Factor – The explanatory variables in an experiment. – A variable whose effect on the response variable is of interest in the experiment. • Levels – The different possible values of a factor. Review - Experimental Terminology • Treatment – A specific experimental condition applied to the units of an experiment. – For one-factor experiments, the treatments are the levels of the single factor. – For multifactor experiments, each treatment is a combination of the levels of the factors. Example: • Researchers studying the absorption of a drug into the bloodstream inject the drug into 25 people. 30 minutes after the injection they measure the concentration of the drug in each person’s blood. • Identify the; a) Experimental units. b) Response variable. c) Factors. d) Levels of each factor. e) Treatments. Answer: Researchers studying the absorption of a drug into the bloodstream inject the drug into 25 people. 30 minutes after the injection they measure the concentration of the drug in each person’s blood. a) Experimental units – Subjects, the 25 people injected b) Response variable – Concentration of the drug in the blood c) Factors – Single factor – the drug d) Levels – One level – the dose e) Treatment – Injecting the drug Your Turn: • Weight gain of Golden Torch Cacti. Researchers examined the effects of a hydrophilic polymer and irrigation regime on weight gain. For this study the researchers chose the hydrophilic polymer P4. P4 was either used or not used, and five irrigation regimes were employed: none, light, medium, heavy, and very heavy. • Identify the; a) Experimental units. b) Response variable. c) Factors. d) Levels of each factor. e) Treatments. Answer: Weight gain of Golden Torch Cacti. Researchers examined the effects of a hydrophilic polymer and irrigation regime on weight gain. For this study the researchers chose the hydrophilic polymer P4. P4 was either used or not used, and five irrigation regimes were employed: none, light, medium, heavy, and very heavy. a) Experimental units – The cacti used in the study b) Response variable – The weight gain of the cacti c) Factors – Two factors – the hydrophilic polymer P4 and the irrigation regime d) Levels – P4 has two levels; with and without. – Irrigation regime has five levels; none, light, medium, heavy, and very heavy. e) Treatment – There are 10 different treatments, each a combination of a level of P4 and a level of irrigation regime. See next slide for treatments. Schematic for the 10 Treatments in the Cactus Study Factors Levels Treatments Randomized, Comparative Experiment 1. Manipulates the factor levels to create treatments. 2. Randomly assigns subjects to these treatments. 3. Compares the responses of the subject groups across treatment levels. The Four Principles of Experimental Design 1. Control 2. Randomize 3. Replicate 4. Block The Four Principles of Experimental Design 1. Control: – Good experimental design reduces variability by controlling the sources of variation. – We control sources of variation other than the factors we are testing by making conditions as similar as possible for all treatment groups. – Comparison is an important form of control. Every experiment must have at least two groups so the effect of a treatment can be compared with either the effect of a traditional treatment or the effect of no treatment at all. The Four Principles of Experimental Design 2. Randomize: – Subjects should be randomly divided into groups to avoid unintentional selection bias in constituting the groups, that is, to make the groups as similar as possible. – Randomization allows us to equalize the effects of unknown or uncontrollable sources of variation. • It does not eliminate the effects of these sources, but it spreads them out across the treatment levels so that we can see past them. – Without randomization, you do not have a valid experiment and will not be able to use the powerful methods of Statistics to draw conclusions from your study. The Four Principles of Experimental Design 2. Randomize: – One source of variation is confounding variables (will discuss later), variables that we did not think to measure but which can affect the response variable. – Randomization to treatment groups reduces bias by equalizing the effects