Chapter 9, Quantitative Research Design BOX 9.1 Guidelines for Critiquing Research Design in a Quantitative Study

1. Was the design experimental, quasi-experimental, or nonexperimental? What specific design was used? Was this a cause-probing study? Given the type of question (therapy, prognosis, etc.), was the most rigorous possible design used? 2. What type of comparison was called for in the research design? Was the comparison strategy effective in illuminating key relationships? 3. If the study involved an intervention, were the intervention and control conditions adequately described? Was blinding used, and if so, who was blinded? If not, is there a good rationale for failure to use blinding? Was attention paid to intervention fidelity? 4. If the study was nonexperimental, why did the researcher opt not to intervene? If the study was cause probing, which criteria for inferring causality were potentially compromised? Was a retrospective or prospective design used, and was such a design appropriate? 5. Was the study longitudinal or cross-sectional? Was the number and timing of data collection points appropriate? 6. What steps did the researcher take in designing the study to enhance statistical conclusion validity? Were these steps adequate? 7. What did the researcher do to control confounding external factors and intrinsic participant characteristics, and were the procedures effective? What are the threats to the study’s internal validity? Does the design enable the researcher to draw causal inferences about the relationship between the independent variable and the outcome? 8. What are the major limitations of the design used? Are these limitations acknowledged by the researcher and taken into account in interpreting results? What can be said about the study’s external validity?