The Hierarchy of Evidence Pyramid
Total Page:16
File Type:pdf, Size:1020Kb
The Hierarchy of Evidence Pyramid Available from: https://www.researchgate.net/figure/Hierarchy-of-evidence-pyramid-The-pyramidal-shape-qualitatively-integrates-the- amount-of_fig1_311504831 [accessed 12 Dec, 2019] Available from: https://journals.lww.com/clinorthop/Fulltext/2003/08000/Hierarchy_of_Evidence__From_Case_Reports_to.4.aspx [accessed 14 March 2020] CLINICAL ORTHOPAEDICS AND RELATED RESEARCH Number 413, pp. 19–24 © 2003 Lippincott Williams & Wilkins, Inc. Hierarchy of Evidence: From Case Reports to Randomized Controlled Trials Brian Brighton, MD*; Mohit Bhandari, MD, MSc**; Downloaded from https://journals.lww.com/clinorthop by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AWnYQp/IlQrHD3oaxD/v Paul Tornetta, III, MD†; and David T. Felson, MD* In the hierarchy of research designs, the results This hierarchy has not been supported in two re- of randomized controlled trials are considered cent publications in the New England Journal of the highest level of evidence. Randomization is Medicine which identified nonsignificant differ- the only method for controlling for known and ences in results between randomized, controlled unknown prognostic factors between two com- trials, and observational studies. The current au- parison groups. Lack of randomization predis- thors provide an approach to organizing pub- poses a study to potentially important imbal- lished research on the basis of study design, a hi- ances in baseline characteristics between two erarchy of evidence, a set of principles and tools study groups. There is a hierarchy of evidence, that help clinicians distinguish ignorance of evi- with randomized controlled trials at the top, con- dence from real scientific uncertainty, distin- trolled observational studies in the middle, and guish evidence from unsubstantiated opinions, uncontrolled studies and opinion at the bottom. and ultimately provide better patient care. In the hierarchy of research designs, the results of randomized controlled trials are considered From the *Boston University School of Medicine and 26 School of Public Health, Boston University, Boston, MA; the highest level of evidence. Randomization the **Department of Clinical Epidemiology and Biosta- is the only method for controlling for known H2r74b7+NAL/T0XV2hVa0sPFhqAJ/HU+hxuW0= on 03/19/2020 tistics, McMaster University, Hamilton, Ontario, Canada; and unknown prognostic factors between two and the †Department of Orthopaedic Surgery, Boston Uni- comparison groups.1, 26 Lack of randomization versity, Boston, MA. predisposes a study to potentially important Reprint requests to Mohit Bhandari, MD, McMaster Uni- imbalances in baseline characteristics between versity Health Sciences Center, Department of Clinical Epidemiology and Biostatistics, Health Science Centre, two study groups. The role of nonrandomized Room 2C3, 1200 Main Street West, Hamilton, Ontario, (observational) studies in evaluating treat- L8N 3Z5 Canada. Phone: 905–525-9140 ext. 22825; Fax: ments is an area of continued debate: deliber- 905–524-3841; E-mail: [email protected]. ate choice of the treatment for each patient DOI: 10.1097/01.blo.0000079323.41006.12 19 Clinical Orthopaedics 20 Brighton et al and Related Research implies that observed outcomes may be caused mate goal of judging whether a particular ex- by differences among people being given the posure causes or prevents disease. Analytic two treatments, rather than the treatments design strategies include observational stud- alone. Unrecognized confounding factors can ies, such as case-control and cohort studies, interfere with attempts to correct for identified and clinical trials. The difference between the differences between groups. There has been two types of analytic studies is the role that the considerable debate about whether the results investigator plays in each of the studies.20 In of nonrandomized studies are consistent with the observational study, the investigator sim- the results of randomized controlled trials.3, 10, ply observes the natural course of events. In 12–14, 21, 22, 28 Nonrandomized studies, or obser- the randomized controlled trial, the investiga- vational studies, have been reported to overes- tor assigns the intervention or treatment. Al- timate or underestimate treatment effects.21, 22 though an in-depth discussion on assessing These considerations have supported a hi- the methodologic quality of each particular erarchy of evidence, with randomized con- study is beyond the scope of the current study, trolled trials at the top, controlled observa- the basic strengths and limitations of each re- tional studies in the middle, and uncontrolled search strategy will be addressed. studies and opinion at the bottom. However, Levels of Evidence these findings have not been supported in two recent publications in the New England Jour- Investigators have attempted to minimize po- nal of Medicine that identified nonsignificant tentially harming patients by basing clinical differences in results between randomized, decisions of the sorts of evidence that are controlled trials and observational studies.3, 13 least likely to be wrong. Two studies defined The current authors provide an approach to what was thought to be the evidence provid- organizing published research on the basis of ing the least biased estimate of the effect of an study design, a hierarchy of evidence. The key intervention: a systematic review document- features and the advantages and disadvantages ing homogeneity in results of a large number of specific study designs will be addressed. of high-quality randomized controlled trials The concepts presented hopefully will enable (randomized with concealment, blinded, com- clinicians and healthcare personnel to practice plete followup, and intention-to-treat analy- 8,25 in the context of evidence-based orthopaedics. sis). This was termed Level 1 Evidence. These investigators additionally categorized Types of Study Design studies of an intervention based on an increas- The types of study designs used in clinical re- ing degree of potential bias: systematic re- search can be classified broadly according to views with randomized controlled trials that whether the study focuses on describing the reveal differences in treatment effect (hetero- distributions or characteristics of a disease or geneity); individual high-quality randomized elucidating its determinants. controlled trials (Level IB evidence); less rig- Descriptive studies are concerned with de- orous randomized controlled trials; cohort or scribing the general characteristics of the dis- observational studies (Level 2 evidence); tribution of a disease, particularly in relation case-control studies (Level 3 evidence); case to person, place, and time. Cross-sectional series (Level 4 evidence), and expert opinion studies, case reports, and case series represent (Level 5 evidence).23 Based on the various types of descriptive studies. Information on levels of evidence of a particular treatment, each can provide clues leading to the genera- grades of recommendation can be deter- tion of a hypothesis that is consistent with ex- mined.25,26 For example, the following grades isting knowledge of disease occurrence. of recommendations have been proposed: (1) Analytic studies focus on determinants of a Grade A, consistent Level 1 studies; (2) Grade disease by testing a hypothesis with the ulti- B, consistent Level 2 or Level 3 studies; (3) Number 413 August, 2003 Hierarchy of Evidence 21 Grade C, Level 4 studies; and (4) Grade D, with those without the outcome. Although the Level 5 studies.25,26 odds ratio can be used to estimate the relative Case Report and Case Series risk, it cannot provide information about ex- cess risk. The case-control study can be useful Case reports are an uncontrolled, descriptive in studying rare outcomes, outcomes with study design involving an intervention and out- multiple potential etiologic factors, or looking come with a detailed profile of one patient. An at outcomes that take a considerable length of early example from the orthopaedic literature time to develop. Additionally, they can be done of a case report is Birkett’s 1869 description of in a short time with small sample sizes, and for 7 a fracture-dislocation of the hip. Expansion of less money than other types of studies. How- the individual case report to include multiple ever, because the information usually is col- patients with an outcome of interest is a case lected from patients or their hospital records, series. In 1981, a famous case series involving data may be inaccurate because of the effect of five homosexual men in Los Angeles, CA with recall bias and measurement bias. Pneumocystis carinii between 1980 and 1981 marked the beginning of the AIDS epidemic in Cohort Study the United States.9 Although descriptive stud- The term cohort comes from the Roman word ies are limited in their design to make causal in- for a group of soldiers that marched into battle ferences about the relationship between risk together.15 In the cohort study design, the cohort factors and an outcome of interest, they are represents a group of people followed up with helpful in developing a hypothesis that can be time to see whether an outcome of interest de- tested using an analytic study design. velops. Ideally this group meets a level of cer- Case-Control Study tain predetermined criteria representative of a population of interest and is followed with well- One type of observational study is the case- defined outcome variables. The Framingham control study that