Study Designs and Methodologic Sources of Error
Total Page：16
File Type：pdf, Size：1020Kb
General Articles E REVIEW ARTICLE CME Clinical Research Methodology 1: Study Designs and Methodologic Sources of Error Daniel I. Sessler, MD, and Peter B. Imrey, PhD * † Clinical research can be categorized by the timing of data collection: retrospective or pro- spective. Clinical research also can be categorized by study design. In case-control studies, investigators compare previous exposures (including genetic and other personal factors, envi- ronmental infuences, and medical treatments) among groups distinguished by later disease status (broadly defned to include the development of disease or response to treatment). In cohort studies, investigators compare subsequent incidences of disease among groups dis- tinguished by one or more exposures. Comparative clinical trials are prospective cohort stud- ies that compare treatments assigned to patients by the researchers. Most errors in clinical research fndings arise from 5 largely distinguishable classes of methodologic problems: selec- tion bias, confounding, measurement bias, reverse causation, and excessive chance variation. (Anesth Analg 2015;121:1034–42) or most of medical history, practice has been largely evidence-based medicine, midwifed by Sackett et al.,14 have ad hoc, based on suboptimal evidence from personal contributed greatly to revolutionary advances in clinical Fexperience and anecdotes shared among physicians. understanding and practice. The application of formal meta- Although individual cases and experiences often yield invalu- analytic techniques from education and social science to med- able insights to perceptive physicians, multiple anecdotes icine and epidemiology, through evidence-based medicine do not replace systematically collected data and organized centers and the Cochrane Collaboration,15,16 has systematized comparisons, and practice mostly based on personal experi- aggregation of research evidence for the medical community. ence often has subsequently been shown to be suboptimal. We thus now know much about how humans work and (The pleural of anecdote is not data!) Voltaire nicely summa- how they respond to disease and drugs. And we know rized early medical practice in 1760, writing that physicians even more about how cells and rodents respond. Physicians “poured drugs of which they knew little, for diseases of are thus encouraged to practice evidence-based medicine, which they knew less, into humans of which they knew noth- which means that clinical decisions should be based on ing.” One hundred years after Voltaire’s comment but con- good evidence, preferably from relevant, high-quality, and siderably before the microbial origin of disease was accepted, reproducible studies in humans rather than on the physi- the British anesthesiologist John Snow, informed by work of cian’s personal clinical experience alone. John Graunt and formal vital statistics systems established However, any physician who tries restricting practice in mid-16th century London,1 used systematic, dispassionate to methods established on the basis of strong evidence in data collection to infer that a contaminated water supply was humans quickly discovers that there is distressingly little the proximal cause of mid-18th century London cholera out- basis for current medical care, although cardiology and breaks.2 During the next century, frameworks for systematic oncology, with histories of strong National Institutes of collection and evaluation of data, such as Koch’s postulates Health funding and methodologic innovation, seem ahead for assessing microbial causation3 and Hill’s guidelines for of most specialties in this regard.17,18 The purpose of clinical inferring causality more generally,4 fostered greater scientifc research is to bridge the remaining wide gap from the under- rigor in conjunction with clinical observation. standing of basic and animal science to the care of patients Clinical research methods have since matured immensely. in an effort to improve medical outcomes. Systematic clini- The meticulous work of Feinstein,5–7 Sackett,8 and others on cal research is necessary because humans have proven to be the intellectual foundations of clinical epidemiology; the fos- a poor model for rodents! tering of randomized clinical trials (RCTs) particularly by Even the best clinical studies have potential limitations, Hill,9,10 Chalmers,11,12 and Cochrane13; and the emergence of and it is helpful to understand the strengths and weak- nesses of various approaches. By this, we do not mean that From the Departments of Outcomes Research and Quantitative Health clinical research is generally problematic. However, it helps Sciences, Cleveland Clinic, Cleveland, Ohio. * † to understand how study design can infuence results, and Accepted for publication April 7, 2015. which types of studies are most reliable and thus the best Funding: Internal departmental funds. basis for clinical decisions. In this and 2 subsequent articles, The authors declare no conficts of interest. we will discuss aspects of clinical research methodology as Address correspondence to Daniel I. Sessler, MD, Department of Outcomes Research, Cleveland Clinic, 9500 Euclid Ave./P77, Cleveland, OH 44195. a guide to understanding and interpreting reported results. Address e-mail to [email protected]. Even a series of articles can only have limited scope. Copyright © 2015 International Anesthesia Research Society Our focus will be on design choices in clinical research and DOI: 10.1213/ANE.0000000000000815 the advantages and disadvantages of various approaches Number 4 ڇ Volume 121 ڇ www.anesthesia-analgesia.org October 2015 1034 Copyright © 2015 International Anesthesia Research Society. Unauthorized reproduction of this article is prohibited. Study Designs and Sources of Error to address clinical research questions. We focus on design In cohort studies, investigators look forward in time from because, although error can occur in design, data collection, exposure to outcome, by comparing frequencies and severi- and/or analysis and reporting of a study, poor study design ties of outcomes in groups defned by different exposures. generally cannot be remedied by subsequent steps. For example, do people with low vitamin D serum concen- A consequence of our focus on design is that we will trations more often develop serious postoperative compli- not discuss operational issues, including ways to maintain cations? The distinction can be nonobvious. For example, blinding, electronic data acquisition, how to minimize and consider a study in which investigators compared anesthetic deal with missing data, or strategies for the prevention of requirement in patients with and without fbromyalgia. The fraud.19 Nor will we discuss result reporting, which also is design might appear to be case-control because patients with subject to various types of error and potential infuence of and without disease are being selected. But, in fact, fbromy- competing interests. Furthermore, the 3 articles in this series algia is the exposure in this study and anesthetic require- will include only the most basic statistical approaches. ment is the outcome. It is thus a cohort design. An advantage of prospective studies is that they allow researchers to plan and manage data collection, which usu- RESEARCH APPROACHES ally improves data quality. Prospective studies also allow Clinical research studies can be broadly categorized as ret- investigators to answer specifc research questions more rospective or prospective. Retrospective studies use exist- directly than is usually possible in retrospective research. ing data on current and past patients to answer questions. The trade-off for these benefts is higher research costs and These studies are conducted by assembling and organizing longer waiting times for answers. contemporaneously recorded information on past events, Comparative studies on treatments can be observational, analyzing previously stored biosamples, and/or by return- when the researcher simply observes and describes current ing to patients and physicians for further information about clinical practice. However, experimental studies generally are the past. Prospective studies answer questions by collect- less prone to spurious fndings than observational studies, ing new data on current and future patients over a future and prospective cohort studies become experiments when period during which medically relevant events occur, gen- researchers manage, rather than simply observe, the choice erally using methods specifc to the intended research. and use of treatments. When treatments are allocated ran- Prospective studies can be either observational (nonin- domly (i.e., “randomized”) to patients, the result is a special terventional) or experimental (interventional), in the sense type of cohort study called a randomized clinical trial (RCT). of manipulating study-related treatments. For example, an Randomization commonly is used, often in conjunction observational study might involve determining the con- with concealment of treatment assignments from study centration of a blood biomarker and evaluating relevant participants and some investigators, a process known as outcomes. Conversely, experimental interventions might “masking” or “blinding” allocation. RCTs that are blinded include the use of a novel anesthesia regimen, intraopera- to the extent practical provide the clearest evidence of a tive monitoring device, or temperature management pro- therapy’s effects but are not always practical, and there are tocol. Clinical studies also can