II. Principles of Experimental Design A. Experimental Units The experimental unit is that object to which a treatment or condition is independently applied. EXAMPLE Carcinogenic substances 20 rats are randomly assigned to each of 4 doses of a potential carcinogen: none, low, medium, and high. The rats are kept in individual cages under the same environmental conditions in the same room. Each rat has its assigned dose stirred into its daily meal for 4 weeks. The number of tumors found in each rat is recorded at the end of the 4 week period. What is the experimental unit? 13 The rats are also called the measurement units or observational units because the outcome of interest is measured on each rat. EXAMPLE Carcinogenic substances 20 rats are randomly assigned to 4 cages (5 rats in each). Each cage is then randomly assigned to one dose of a potential carcinogen: none, low, medium, and high. The rats are kept in their assigned cages under the same environmental conditions in the same room. Rats are not fed individually; food is placed in each cage in a common dish out of which all rats eat. Each cage has its assigned dose stirred into the food for 4 weeks. The number of tumors found in each rat is recorded at the end of the 4 week period. What is the experimental unit? What is the measurement unit? 14 . Experimental Factors If a study is an experimental design, then there are one or more experimental factors : covariates whose levels or values are assigned by the investigator to each experimental unit. Observational studies, and some experimental designs, have classification factors : covariates whose levels or values are observed for each experimental unit by the investigator Ex . Diabetes study Treatment - Gender œ Ex . Midwest Heart Study Education - Race/ethnicity - 15 .. Experimental factors which correspond to treatments A placebo is an inactive treatment. It is constructed to look, feel, or taste as much like the active treatment under study as possible. Ex . Surgery for knee pain Active treatment: incision in the knee followed by a cleaning of the knee joint using sterile saline solution Placebo: incision in the knee followed by sound effects meant to mimic those made during a cleaning Ex . HIV/AIDS medications Active treatment: 2 drugs in pill form œ ZDV and ddI Placebo: 2 drugs in pill form œ ZDV and an inactive pill that looks and tastes like ddI. 16 - A placebo is used so that any one study participant is not likely to know which treatment they are receiving. If every subject is not informed of which treatment they are receiving, the study is called single-blinded . The subject is —blind“ to treatment assignment. - If the investigator who is measuring the outcome of interest is also —blind“ to treatment assignment for every participant, the study is called double-blinded . - If there is a statistician who is running interim analyses (analyses carried out to compare groups before the study is completed), and that person is also —blind“ to treatment assignment for every participant, the study is called triple-blinded . 17 - A placebo is a type of control treatment, a baseline or benchmark to which the active treatment will be compared. No treatment at all (null treatment or null control ) is sometimes used in addition to or instead of a placebo. Ex . Study of gum disease treatment Treatment: oral rinse with an active drug Active control (placebo): oral rinse with no active drug Null control: no oral rinse 18 D. Study types A cross-sectional study collects data on all experimental units at only one point in time. A longitudinal study or follow-up study collects data for at least two points in time. A prospective study determines the experimental units at the current time and then collects data on them forward in time. A retrospective study determines the experimental units at the current time and then collects data on them from past records. 19 A clinical trial collects health outcomes (clinical data) on humans and follows them forward in time for changes in that health outcome or for occurrence of new health outcomes. When clinical trials recruit and follow participants at multiple locations, then it is called a multi-center clinical trial . See PubH 7420 Clinical Trials PubH 7400 (section 003) Statistics for Translational and Clinical Research 20 E. Types of outcomes - Many experiments measure multiple outcomes or responses. One of them is usually selected as the primary outcome , the outcome that is of most scientific interest, and information on this outcome may be used to calculate how many experimental units are needed to run the research study. - Other outcomes, those needed to answer other questions of scientific interest, are sometimes called the secondary outcomes . 21 - Sometimes it is impractical to measure the outcome of primary interest (e.g., death) because the outcome is rare, or difficult or impossible to measure. Researchers may then use a surrogate outcome , a response that is supposed to be related to (predictive of) the outcome of interest. Ex . HIV/AIDS studies often use: 22 F. Sources of Errors When statisticians talk about errors , we are referring to observed variability in the recorded value of an outcome across experimental units. One source of error is assumed to always exist: experimental error . This describes the variability in the outcome among identically and independently treated experimental units. Experimental error can arise from: • natural or inherent differences between experimental units • variability in the measurement process 23 • inability to exactly replicate the treatment or condition from one experimental unit to the next • a treatment that acts differently across units • extraneous factors that influence the measurements How can we minimize these problems? • Select experimental units which are as homogenous as possible. When this is difficult to do, or when doing so would constrain the generalizability of the research results, consider blocking designs. We will see these in Chapter 8. 24 • Take measurements in as uniform a manner as possible. Be aware of how the following can introduce variability that is irrelevant to the purpose of the study: o person-to-person differences in how measurements are taken (people with more vs. less experience, people with better vs. poorer training in measuring, people with more vs. less patience, etc.) o machine-to-machine differences in how measurements are recorded (calibration, age or condition of the machine, etc.) o differences in equipment (pipettes, calipers, microtitre plates, rulers, etc.) 25 o differences in techniques (assays, dilutions, solution, survey questions, etc.) o within person differences (how tired was the student collecting the measurements?) o differences in supplies (salines, soils, chemicals, drugs, growth media, etc.) • Construct the treatments(s) in as uniform a manner as possible. • Control the environment to be as uniform as possible (temperature, humidity, light, accessibility of clinics, financial incentives, other incentives, etc.) 26 Other variability can be due to specific sources: • Lab studies using mice should track which animals are related to each other, as should human studies (genetic variability). • Crop studies should determine the physical attributes of the soil. Is one half of the field always wetter than the other half because the field has a downward slope? (physical variability) • Studies that take time to complete (due to e.g. lab or recruitment constraints) should keep track of the days/weeks/months on which measurements are collected. (temporal, seasonal variability) (Those recruited in summer, for example, may somehow be different from those recruited in winter.) 27 • Studies involving repeated measurements on each experimental unit need to keep track of which measurements came from which unit (within unit variability). Such sources of variability can be controlled and then measured using blocking designs as well (Chapter 8). Alternatively, statistical control of such factors may be achieved by adjusting for covariates in the statistical model (e.g., season). We will see this in Chapter 17. 28 G. Replication Use of replication in an experiment means that we assign several experimental units to each treatment group. Why is replication important? • It allows the researcher to demonstrate reproducibility: the treatment shows similar results in each of several experimental units. • It guards against an experiment failing merely because one or a few experimental units become unusable (e.g., mice died, plants died, people dropped out, etc.). 29 • It allows us to measure experimental error. • It improves the precision with which we can measure the treatment effect. A critical question in experimental design is, how many replications should be used? We will come back to this question in several lectures during this course. 30 Pseudo-replications are different. These are repeated measurements take from the same experimental unit. This is sometimes called sub- sampling . Such measurements are not independent of each other, and statistical analysis of such data must account for the correlation. We‘ll see this in several chapters of the book. Also consider PubH 7430 Methods for Correlated Data. 31 H. Randomization: Why do we do it? Randomization is a critical component in the design of experiments. It involves assigning subjects to treatment groups so that all possible assignments are, in theory, equally likely . Experimental units should have an equal chance of being assigned to any of the treatment groups. Why do we randomize? 32 • To protect against confounding . Confounding occurs when the effect of one factor cannot be distinguished from the effect of another factor. Obvious Ex . Assign all of the fat rats to diet A and all of the skinny rats to diet B. Measure weight gain in all rats one week later. If we see a difference between rats on diet A and rats on diet B, what do we conclude about the two diets? 33 Better Ex .