E REVIEW ARTICLE Clinical Research Methodology 2: Observational Clinical Research Daniel I. Sessler, MD, and Peter B. Imrey, PhD * † Case-control and cohort studies are invaluable research tools and provide the strongest fea- sible research designs for addressing some questions. Case-control studies usually involve retrospective data collection. Cohort studies can involve retrospective, ambidirectional, or prospective data collection. Observational studies are subject to errors attributable to selec- tion bias, confounding, measurement bias, and reverse causation—in addition to errors of chance. Confounding can be statistically controlled to the extent that potential factors are known and accurately measured, but, in practice, bias and unknown confounders usually remain additional potential sources of error, often of unknown magnitude and clinical impact. Causality—the most clinically useful relation between exposure and outcome—can rarely be defnitively determined from observational studies because intentional, controlled manipu- lations of exposures are not involved. In this article, we review several types of observa- tional clinical research: case series, comparative case-control and cohort studies, and hybrid designs in which case-control analyses are performed on selected members of cohorts. We also discuss the analytic issues that arise when groups to be compared in an observational study, such as patients receiving different therapies, are not comparable in other respects. (Anesth Analg 2015;121:1043–51) bservational clinical studies are attractive because Group, and the American Society of Anesthesiologists they are relatively inexpensive and, perhaps more Anesthesia Quality Institute. importantly, can be performed quickly if the required Recent retrospective perioperative studies include data O 1,2 data are already available. Epidemiologic and health ser- from tens-of-thousands to tens-of-millions of patients. vices investigators have used such approaches for decades. Large numbers per se limit research errors attributable But until recently, surgical and perioperative retrospective to chance, but do not prevent bias and confounding, and studies were too often “100-patient chart reviews,” which can exacerbate their effects by inducing overconfdence in rarely produced valid conclusions. Increasingly, though, biased results. But large samples do support more in depth the availability of electronic data and sophisticated statisti- and effective application of statistical techniques to com- cal techniques makes retrospective observational studies of pensate for known confounding factors than is possible surgical and perioperative practices a valuable tool. with small studies. Especially in anesthesia, there has been a revolution in Although analyses of observational data should gener- data quality and availability because of electronic anesthe- ally be considered exploratory rather than defnitive, they sia and hospital records. Consequently, some institutions are nonetheless often a relatively inexpensive and quick have large and dense (i.e., minute-to-minute) registries of way to evaluate the plausibility of hypotheses and build anesthesia care. Often they are linked to related hospital support for subsequent experimental studies. Done well, databases, such as those from clinical laboratories and retrospective analyses can provide good estimates of treat- blood banks, and to outcomes such as duration-of-hos- ment effect3–7 although they tend to underestimate harms pitalization and date-of-death. Billing codes also provide associated with interventions.8 Excellent guidelines have valuable information about diagnoses and procedures been published to encourage uniform and complete report- although the intricacies and requirements of reimburse- ing of observational studies, including full acknowledge- ment mechanisms, and of administrative record systems ment of limitations.9,10 more generally, can sometimes distort clinical realities. To glean the most information from such registries, there CASE SERIES are now several national registries that pool data from A case series—a description of what happened to a series of various institutions, notably the National Surgical Quality patients with a particular diagnosis, perhaps treated with a Improvement Project, Multicenter Perioperative Outcomes particular strategy—is certainly an improvement over anec- dotal experience and case reports because compiled data From the Department of Outcomes Research, Anesthesiology Institute, from a group are less likely to be idiosyncratic than results Cleveland Clinic, Cleveland, Ohio; and Department of Quantitative Health from 1 or a few individuals. There are many examples in Sciences, Lerner* Research Institute, Cleveland Clinic, Cleveland, Ohio. which case series, and even case reports, have provided crit- † Accepted for publication May 1, 2015. ical advances.9 Malignant hyperthermia, for example, was Funding: Supported by internal sources. initially reported as a case series and, because it is so rare, The authors declare no conficts of interest. has never been subject to randomized trial.10,11 Address correspondence to Daniel I. Sessler, MD, Department of Outcomes In a typical case series, physicians might report that 79 of Research, Anesthesiology Institute, Cleveland Clinic, 9500 Euclid Ave./P77, Cleveland, OH 44195. Address e-mail to [email protected]; website: www.OR.org. their patients given a particular treatment for heart failure Copyright © 2015 International Anesthesia Research Society had a median survival of 37 months. If you are comparable DOI: 10.1213/ANE.0000000000000861 to their patients and get exactly the same treatment, it is Number 4 www.anesthesia-analgesia.org 1043 ڇ Volume 121 ڇ October 2015 Copyright © 2015 International Anesthesia Research Society. Unauthorized reproduction of this article is prohibited. E REVIEW ARTICLE reasonable to expect to have about a 50-50 chance of living reintubations are also rare—too rare, in fact—to practically more or less than 37 months. evaluate in prospective trials. The trouble is that this result, although useful for con- An alternative approach is to fnd patients who required sidering the prognosis of an individual already committed reintubation in the postanesthesia care unit and compare to the particular treatment, offers no comparative context. them to a similar group of patients who did not. Investigators Median survival for comparable patients is not all that we can then look backward in time and determine, for exam- want to know; what we really need to understand is whether ple, which patients in each group had their neuromuscular this survival (or any other outcome) is better or worse with block reversed. If patients who required reintubation were this treatment than with alternatives, and that is where the signifcantly less likely to have been given reversal agents, logic gets tricky. The danger is that in assessing alternative one can conclude that there is an association between inad- treatment plans, the results of a case series of a new treat- equate block reversal and emergent reintubation. The basic ment are almost always implicitly or explicitly compared approach to such case-control studies is shown in Figure 1. with previous results, that is, to a “historical control.” For case-control studies to be valid, the case and control Historically controlled studies tend to falsely generate groups must be chosen or statistically adjusted to be compa- a conclusion that new treatment or local management is rable with respect to potential confounding factors, and the superior because the comparative effects of the treatment assessment of exposures and potential confounding factors tend to get mixed up with, or confounded by, other time- must be equally complete and accurate for both. A danger dependent changes. For instance, recent patients may have with clinical records is that important potential confound- been diagnosed earlier in their disease courses than his- ers may not have been evaluated or may have been inaccu- torical patients because of the improvement in diagnostic rately recorded. They may also be nonrandomly erroneous: imaging or other technology. Patients diagnosed earlier— for example, patients who have complications may have a the recent ones—would thus be expected to survive lon- more complete list of preexisting conditions. A further dan- ger from the time of diagnoses whether or not the new ger is that important confounding factors may be unknown treatment they receive is an improvement. This problem and thus not even considered by investigators. is known as “lead time” bias in the context of studies of But because investigators must look back in time— medical screening. Because there is no way to recover when sometimes quite far back—it is diffcult to determine the diagnoses would have occurred had the historical reference extent to which the groups might meaningfully differ. subjects been seen under present conditions, it is diffcult Equivalent exposure assessment for both groups may also or impossible to retroactively correct for this problem. be hard to ensure. For example, data about cases often must Alternatively, therapy for some disease-related complica- be obtained directly from patients or their families, whose tions may have improved over time. Improved treatment memories may be stimulated by the symptoms, diagnostic of these complications, rather than the new therapy, may processes, and concerns about the disease. In contrast, the have improved