Euro Jnl Phil Sci (2011) 1:451–466 DOI 10.1007/s13194-011-0034-6 ORIGINAL PAPER IN PHILOSOPHY OF MEDICINE

Just a paradigm: evidence-based medicine in epistemological context

Miriam Solomon

Received: 8 October 2010 /Accepted: 28 July 2011 /Published online: 6 September 2011 # Springer Science+Business Media B.V. 2011

Abstract Evidence-Based Medicine (EBM) developed from the work of clinical epidemiologists at McMaster University and Oxford University in the 1970s and 1980s and self-consciously presented itself as a "new paradigm" called "evidence- based medicine" in the early 1990s. The techniques of the randomized controlled trial, systematic review and meta-analysis have produced an extensive and powerful body of research. They have also generated a critical literature that raises general concerns about its methods. This paper is a systematic review of the critical literature. It finds the description of EBM as a Kuhnian paradigm helpful and worth taking further. Three kinds of criticism are evaluated in detail: criticisms of procedural aspects of EBM (especially from Cartwright, Worrall and Howick), data showing the greater than expected fallibility of EBM (Ioaanidis and others), and concerns that EBM is incomplete as a philosophy of science (Ashcroft and others). The paper recommends a more instrumental or pragmatic approach to EBM, in which any ranking of evidence is done by reference to the actual, rather than the theoretically expected, reliability of results. Emphasis on EBM has eclipsed other necessary research methods in medicine. With the recent emphasis on translational medicine, we are seeing a restoration of the recognition that clinical research requires an engagement with basic theory (e.g. physiological, genetic, biochemical) and a range of empirical techniques such as bedside observation, laboratory and animal studies. EBM works best when used in this context.

Keywords Evidence-based medicine . Philosophy of medicine . Kuhnian paradigm . Nancy Cartwright . Randomized controlled trial . Translational medicine . Evidence hierachy. Meta-analysis . John Ioannidis . John Worrall

I am grateful to two anonymous reviewers for the European Journal for Philosophy of Science for helpful corrections and comments on this paper. M. Solomon (*) Department of Philosophy, Temple University, Philadelphia, PA 19122, USA e-mail: [email protected] 452 Euro Jnl Phil Sci (2011) 1:451–466

1 Introduction

Evidence-Based Medicine (EBM)1 is the application of methods of clinical epidemiology to the practice of medicine more generally. It was inspired by the post-World War II work of Archibald Cochrane, developed from the work of clinical epidemiologists at McMaster University and Oxford University in the 1970s and 1980s, and self-consciously presented as a “new paradigm” called “evidence-based medicine” in the early 1990s (Evidence-Based Medicine Working Group 1992). EBM was embraced in Canada and the UK in the 1990s, received with some ambivalence in the United States, and adopted in many other countries, both developed and developing (Daly 2005). Its techniques of population based studies and systematic review have produced an extensive and powerful body of knowledge about medical diagnosis and treatment. A canonical and helpful definition of EBM2 is that of Davidoff et al. (Davidoff et al. 1995)inaneditorialin the British Medical Journal: “In essence, evidence based medicine is rooted in five linked ideas: firstly, clinical decisions should be based on the best available scientific evidence; secondly, the clinical problem - rather than habits or protocols - should determine the type of evidence to be sought; thirdly, identifying the best evidence means using epidemiological and biostatistical ways of thinking; fourthly, conclusions derived from identifying and critically appraising evidence are useful only if put into action in managing patients or making health care decisions; and, finally, performance should be constantly evaluated.” EBM regards its own epistemic techniques as superior to other more traditional methods such as clinical experience, expert opinion, and physiological reasoning. This is because the more traditional techniques are viewed as more fallible. There is not one new technique, but several. The following are typically regarded as part of EBM: 1. Rigorous design of clinical trials, especially the randomized controlled trial (RCT). The RCT is to be used wherever physically and ethically feasible. The trial should be double-masked (traditionally, “double-blinded”3)wherever possible. 2. Systematic evidence review and meta-analysis, including grading of the evidence in “evidence hierarchies.” 3. Outcome measures (leading to suggestions for improvement)

1 The term “evidence-based practice” may be replacing EBM, acknowledging the fact that the practice of medicine requires not only physicians but other health care professionals. 2 Some canonical definitions are unhelpful for a general understanding of EBM, for example, that given in (Sackett et al. 1996): “Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.” This particular definition is probably used widely because it is brief, and because it has a rhetorical purpose—to address the frequent criticism of EBM that it applies to populations, not individuals. 3 “Double-masked” has largely replaced “double-blinded,” in order to avoid inappropriate use of terms relating to disability. Euro Jnl Phil Sci (2011) 1:451–466 453

The RCT has often been described as the “gold standard” of evidence for effectiveness of medical interventions. It is a powerful technique, originally developed by the geneticist R.A. Fisher and applied for the first time in a medical context by A. Bradford Hill’s 1948 evaluation of streptomycin for tuberculosis (Doll et al. 1999). The double-masked RCT is designed to control for the placebo effect, for selection and other confounding biases, and for confirmation biases. EBM also includes systematic and formal techniques for combining the results of different clinical trials. A systematic review does a thorough search of the literature and an evaluation and grading of clinical trials. An evidence hierarchy is typically used to structure the judgments of quality and strength of evidence. Meta-analysis integrates the actual data from different but similar high-quality trials to give an overall single statistical result. Often, EBM is supplemented with formal techniques from Medical Decision Making (MDM) such as risk/benefit calculations. The risk/benefit calculations can be made for individual patients, making use of patient judgments of utility, or they can be made in the context of health care economics, for populations. MDM seeks to avoid common errors of judgment, such as availability and salience biases, in medical decision making.4 The overall project is to use the techniques of EBM (and sometimes also MDM) to construct practice guidelines and to take care of individual patients. Each technique—the RCT, other high quality clinical trials, meta-analysis and systematic review–is based on its own core technical successes. The techniques fit together, and share a reliance on statistics, probability theory and utility theory. Journals, centers, clearinghouses, collaborations, educational programs, textbooks, committees and governments all produce and disseminate EBM. EBM rose to dominance right after the prominence of consensus conferences for assessment of complex and sometimes conflicting evidence and may have been partly responsible for the decline of traditional consensus conferences. As late as 1990, an Institute of Medicine report evaluating the international uses of medical consensus conferences said “Group judgment methods are perhaps the most widely used means of assessment of medical technologies in many countries” (Baratz et al. 1990). Just a few years later, expert consensus is viewed in the same medical circles as the lowest level of evidence, when it is included in the evidence at all. For example, the Canadian Task force on Preventive Health Care which began in 1979 as a consensus conference program now explicitly declares “Evidence takes precedence over consensus” (Canadian Task Force on Preventive Health Care). EBM has not completely replaced group judgment, however. Consensus conferences (or something similar) are often still used for producing evidence- based guidelines or policy, that is, for translating a systematic review into a practical recommendation. And group judgment may be needed to set the standards to be used in systematic review. I will comment on this continued reliance on consensus methods later in the paper. There is some indication that EBM is now past its peak, and being overshadowed in part by a new approach, that of “translational medicine” (Woolf 2008). Considerable resources from the NIH, from the European Commission and from

4 A useful essay discussing the differences and interactions between EBM and MDM is (Elstein 2004) 454 Euro Jnl Phil Sci (2011) 1:451–466 the National Institute for Health Research in the UK have been redirected to “bench to bedside and back” research, which is typically the research that takes place before the clinical trials that are core to EBM. Donald Berwick, the founder of the Institute for Healthcare Improvement (which is the leading organization for quality improvement in healthcare), now claims that “we have overshot the mark” with EBM and created an “intellectual hegemony” that excludes important research methods from recognition (Berwick 2005). Berwick calls the overlooked methods “pragmatic science” and sees them as crucial for scientific discovery. He mentions the same sorts of approaches (use of local knowledge, exploration of hypotheses) that “translational medicine” advocates describe. After the discussions in this paper, some reasons for the recent turn to translational medicine will become clearer. There is a vast literature on evidence-based medicine, most consisting of systematic evidence reviews for particular health care questions. A substantial portion of the literature, however, is a critical engagement with EBM as a whole, pointing out both difficulties and limitations. These discussions come from outsiders as well as insiders to the field of EBM. My goal in this paper is to do something like a systematic review of this literature, discerning the kinds of criticisms that seem cogent and presenting them in a structured manner. EBM, like all methodologies in medicine, has both core strengths and limitations. I will begin with an overview of some general social and philosophical characteristics of EBM, and then turn to the criticisms.

2 EBM as a “Kuhnian paradigm”

When the Evidence-Based Medicine Working Group described themselves as having a “new paradigm” of medical knowledge (Evidence-Based Medicine Working Group 1992), they particularly had in mind Kuhn’s(1962) characterization of a paradigm as setting the standards for what is to count as admissible evidence.5 EBM assessments make use of an “evidence hierarchy” (often called “levels of evidence”) in which higher levels of evidence are regarded as of higher quality than lower levels of evidence. A typical evidence hierarchy6 puts double-masked (or “double-blinded”) RCTs at the top, or perhaps right after meta-analyses or systematic reviews of RCTs. Unmasked RCTs come next, followed by well designed case controlled or cohort studies and then observational studies and case reports. Expert opinion, expert consensus, clinical experience and physiological rationale are at the bottom. The rationale for the evidence hierarchy is that higher levels of evidence are thought to avoid biases that are present in the lower levels of evidence. Specifically, randomization avoids selection and other confounding biases (but see (Worrall 2007b)) and masking helps to distinguish real from placebo effects (but see (Howick 2008; Howick 2011)) and avoid confirmation bias. Powering the trial with sufficient numbers of participants and using statistical tools avoids the salience and availability

5 They were not, however, claiming along with many Kuhnians that there is subjectivity or relativity involved in what gets to count as evidence. 6 There are many such hierarchies in use, but all put double-masked RCTs at the top, or right after meta- analyses of RCTs, and clinical experience, expert consensus and physiological rationale at the bottom. Euro Jnl Phil Sci (2011) 1:451–466 455 biases that can skew informal assessments and unsystematic clinical experience. Ultimately, trial results are graded for both quality and strength of evidence. The language of Kuhnian paradigms has been overused and become somewhat clichéd, meaning something like “transformational new theory” in the typical quote from the EBM Working Group cited in the previous paragraph. In fact, EBM has many characteristics of a traditional Kuhnian paradigm,7 having all three of the characteristics of a Kuhnian paradigm discerned by Margaret Masterman (Lakatos and Musgrave 1970) and agreed to by Kuhn (Kuhn et al. 2000). These characteristics8 are helpful for understanding the import of EBM. First, EBM is a social movement with associated institutions such as Evidence-Based Practice Centers, official collaborations, textbooks, courses and journals. It is also, secondly, a general philosophy of medicine, defining both the questions of interest and the appropriate evidence. It is seen as the central methodology of medicine by its practitioners and as an unwelcome politically dominant movement by its detractors (e.g. (Charlton and Miles 1998; Denny 1999)). And third (sometimes overlooked by those who use Kuhn’sterm“paradigm”), it is characterized by a core of technical results and successful exemplars that have been extended over time. Kuhn referred to such exemplars as “concrete puzzle solutions…employed as models or examples” (Kuhn 1970) and later as a “disciplinary matrix” including “symbolic generalization, models and exemplars” (Kuhn 1977). He regards this third meaning as the original and fundamental meaning of the term “paradigm” (Kuhn 1977). Contrary to appearances and self-presentation, this core of technical results is not produced by a general algorithm or set of precise methodological rules. One of the things that Kuhn emphasized about paradigms is that they are driven primarily by exemplars, and not by rules. He writes (Kuhn 1970) that exemplars are “one sort of element…the concrete puzzle-solutions employed as exemplars which can replace explicit rules as a basis for the solution of the remaining puzzles of normal science. Kuhn argued that this is significant because the rules are not the basis for the development of the science. Rather, Kuhn argues, less precise judgments about similarity of examples are used (Kuhn 1977). The medical RCT traces its beginning to A. Bradford Hill’s 1948 evaluation of streptomycin for tuberculosis (Doll et al. 1999). It was initially resisted by many physicians used to treating each patient individually, therapeutically and with confidence in treatment choice (Marks 1997). Nevertheless, important trials such as the polio vaccine field trial of 1954 and the 1955 evaluation of treatments for rheumatic fever helped bring the RCT into routine use (Meldrum 1998; Meldrum 2000). In 1970 the RCT achieved official status in the USA with inclusion in the new FDA requirements for pharmaceutical testing (Meldrum 2000). One of the most well-known early successful uses of RCTs was the 1980s international study of aspirin and streptokinase for the prevention of myocardial infarction. However, not

7 Interestingly, Sehon and Stanley (2003) argue that EBM should not be thought of as a Kuhnian paradigm, but, instead, as part of a Quinean holistic network of beliefs. My focus here is on the historic, rather than the popular meaning of “paradigm” and on what EBM is, not on what it should be. 8 There are many other characteristics of Kuhnian paradigms, such as the narrative of paradigm change, the emphasis on high-level theory, and the idea that paradigms replace one another, that do not apply to this case. I am not trying to apply Kuhn exhaustively, just to use some of his concepts where they may be explanatorily useful. 456 Euro Jnl Phil Sci (2011) 1:451–466 all early use of the RCT was straightforward or successful: the 1960s-1970s NCI randomized controlled trial of lumpectomy versus mastectomy for early stage breast cancer was not masked,9 yet it was highly regarded, and the attempt to conduct a Diet-Heart study in the 1960s was hampered and finally frustrated by the difficulties in implementing major changes in diet in one arm of the study (Marks 1997). This is an example of the finding that the methodology of the RCT does not readily apply to all the situations in which we might wish to use it. As Kuhn might put it, normal science is not a matter of simple repetition of the paradigm case; it requires minor or major tinkering, and sometimes ends in frustration (or what Kuhn would call an “anomaly”). There are also variations in the design and analyses of RCTs. For example, some trials do an “intention to treat” analysis, dropping no experimental subjects from the trial, even if they fail to go through the course of treatment, and some trials do a “per-protocol” analysis in which only patients who complete the trial are included in the final results. Some trails have a placebo in the control arm and some trials have an established treatment in the control arm. It is often said that design and evaluation of an RCT requires “judgment” (see for example (Rawlins 2008)); by this what is meant is that trials cannot be designed by a universal set of rules and that the design and evaluation of trials requires domain expertise, not only statistical expertise. For example, domain expertise is needed in order to design both the dosage and the intervention in the control arm, and domain expertise is needed in order to specify appropriate trial selection criteria. The same insights apply to systematic reviews and meta-analyses. The first systematic review is often identified as the Oxford Database of Perinatal Trials 1989 study of corticosteroids for fetal lung development; this study was the basis for the development of the Cochrane Collaboration in 1993 which has since then done over 3,000 systematic reviews. Other organizations producing systematic reviews include the Agency for Health Care Research and Quality (AHRQ) and its fourteen Evidence-Based Practice Centers and the American College of Physicians (ACP) Journal Club. All systematic reviews use evidence hierarchies, but there is some variation in the hierarchies in use. The RCT is always at the top or just below meta- analyses of RCTs, but there are variations in where other kinds of studies are ranked, and in whether or not animal trials, basic science and expert opinion are included. Hierarchical rank is just one measure of the quality of a trial, which needs to be considered together with other measures of quality such as how well the trial handles withdrawals and how well it is randomized and masked. In 2002 the AHRQ reported forty systems of rating in use, six of them within its own network of evidence-based practice centers (AHRQ 2002). The GRADE Working Group, established in 2000, is attempting to reach consensus on one system of rating the quality and strength of evidence (Guyatt et al. 2008). This is an ironic development, given that EBM intends to replace group judgment methods! Meta-analysis combines the results of several high-quality trials to get an overall measure of strength of evidence. It requires judgments about the similarity of trials for combination and the quality of evidence in each trial, as well as about the possibility of systematic bias in the evidence overall, for example due to publication

9 Double-masking is the standard for high quality RCTs. Euro Jnl Phil Sci (2011) 1:451–466 457 bias and pharmaceutical company support. Meta-analysis is a formal technique, but not an algorithmic one: judgments need to be made about trial quality (as with systematic reviews, use of an evidence hierarchy is part of the process) and similarity of trial endpoints or other aspects of studies. Different meta-analyses of the same data have produced different conclusions (Juni et al. 1999; Yank et al. 2007). Steven Goodman (2002) is concerned that the disagreement between meta-analyses, specifically in the case of mammography screening, represents a “crisis for EBM.” I think it is not so much a crisis as a reminder of the limits of EBM. The identification of EBM with a Kuhnian paradigm, useful though it is, should not be taken too scrupulously. Exemplars and judgments of similarity are important, but rules also play a role. Kuhnian claims about incommensurability between paradigms and the social constitution of objectivity are controversial here and would certainly be denied by practitioners of EBM.10 We have moved on from Kuhn’s ideas, revolutionary in the 1960s, but now built upon and transformed in more sophisticated ways.

3 Critical discussions of EBM

Critical discussions of EBM tend to focus on questioning the procedural necessity and sufficiency of the technical requirements (especially for the RCT), the reliability of EBM in practice, or on EBM’s explicit or implicit claims to be a general philosophy of medicine. I’ll examine these three areas in turn.

3.1 Criticisms of procedural aspects of EBM

Many of these criticisms of EBM procedures have come from British philosophers of science associated with the London School of Economics. Their main approach is to argue that the “gold standard” (the double-masked RCT) is neither necessary (Howick 2011; Worrall 2007b) nor sufficient (Cartwright 2010) for clinical research. They argue that RCTs do not always control for the biases they are intended to control, they do not produce reliably generalizable knowledge, or they can be unnecessary constraints on clinical testing. These arguments are theoretical and abstract in character, although they are sometimes illustrated by examples. I distinguish them from arguments that RCTs have difficulties in practice, that is, from evaluation of RCTs based on the actual outcomes of such studies, which will be discussed in the next subsection (3B). John Worrall (2002, 2007a, 2007b) argues that randomization is just one way, and an imperfect way, of controlling for confounding factors that might produce bias. The problem is that randomization can control only for most but not for all confounding factors. When there are indefinitely many factors, both known and unknown, which may lead to bias, chances are that any one randomization will not randomize with respect to all these factors. Under these circumstances, Worrall concludes, chances are that any particular clinical trial will have at least one kind of

10 Critics of EBM have occasionally presented EBM as a political and rhetorical movement, e.g. (Charlton and Miles 1998), emphasizing the ways in which it appears to lack rationality. 458 Euro Jnl Phil Sci (2011) 1:451–466 bias, making the experimental group relevantly different from the control group, just by accident. The only way to avoid this is to re-randomize and do another clinical trial, which may, again by chance, eliminate the first trial’s confounding bias but introduce another. Worrall concludes that the RCT does not yield reliable results unless it is repeated time and again, re-randomizing each time, and the results are aggregated and analyzed overall. This is practically speaking impossible. In context, Worrall is less worried about the reliability of RCTs than he is about the assumption that they are much more reliable—in a different epistemic class—than e.g. well- designed observational (“historically controlled”) studies in which there is no randomization. He is arguing that the RCT should be taken off its pedestal and that all trials can have inadvertent bias due to differences between the control and the experimental group. In a series of articles, Nancy Cartwright (2007a,b, 2009, 2010) points out that RCTs may have internal validity, but their external validity and hence their applicability to real world questions is dependent on the similarity of the test population and context to the population and context targeted by the intervention. For example, she cites the failure of the California class-size reduction program, which was based on the success of a RCT in Tennessee, as due to failure of external validity (Cartwright 2009). She does not give a medical example of actual failure of external validity (hence my classification of her criticisms of EBM as theoretical in character), although she gives one of possible failure: prophylactic antibiotic treatment of children with HIV in developing countries. UNAIDS and UNICEF 2005 treatment recommendations were based on the results of a 2004 RCT in Zaire. Cartwright is concerned that the Zaire results will not generalize to resource-poor settings across other countries in sub-Saharan Africa (Cartwright 2007b) Concern about external validity is reasonable and there are classic medical examples of lack of external validity. For example, some recommendations for the treatment of heart disease, developed in trials of men only, do not apply to women. There is a history of challenges to RCTs on the grounds that they have excluded certain groups from participation (e.g. women, the elderly, children) yet are used for general health recommendations. The exclusions are made on epistemic and/or ethical grounds. These days, women are less likely to be excluded because the NIH and other granting organizations require their participation in almost all clinical trials, but other exclusions, such as those based on age, remain. Cartwright expresses the concern about external validity in its most general form. In her most recent work (Cartwright 2010) she describes four conditions that need to be met for external validity.11 These four conditions demand considerable domain knowledge i.e. knowledge of the particular causal interactions that the intervention relies upon. Jeremy Howick (2008, 2011) argues that masking is not useful outside of contexts in which outcomes are measured subjectively and that masking is both impossible and unnecessary when dealing with large effect size. It is impossible when dealing with large effect size because the effects of the drug unmask the assignment. He also argues that masking is in practice inadequate for placebo controlled trials, since

11 Cartwright’s(2010) four conditions are knowledge of “Roman laws“(laws that are general enough), “the right support team” (all necessary conditions), “straight sturdy ladders” (for climbing up and down levels of abstraction) and “unbroken bridges” (no interfering conditions). Euro Jnl Phil Sci (2011) 1:451–466 459 participants can usually tell through the presence of side effects whether or not they are receiving an active intervention. These three sets of criticisms by Worrall, Cartwright and Howick are significant, and I evaluate them next, beginning with Worrall’s papers. Randomization is unlikely to control for confounding factors only in the event that there are many unrelated population variables that influence outcome, because only in that complex case is one of those variables likely to be accidentally unbalanced by the randomization. Worrall considers only the abstract possibility of multiple unknown variables; he does not consider the likely relationship (correlation) of those variables with one another and he does not give us reason to think that, in practice, randomization generally (rather than rarely) leaves some causally relevant population variables accidentally selected for and thereby able to bias the outcome.12 In addition, successful replication adds evidence that any confounder inadvertently introduced is not causally responsible for the outcome. Cartwright uses examples from education, economics and international development to show lack of external validity. In general, her examples show failures of interventions to generalize, often because of cultural differences between populations. External validity in medical trials is more explored territory. We typically already know, from some of the trial selection criteria, where the controversies about generalization lie (see also Table 2 in Rawlins (2008), which sets out the problems with generalization). This does not mean that we can figure out the domain of application of trial results in a simple or formulaic manner. Domain expertise is essential for projection, as of course Nelson Goodman argued long ago (1955). Cartwright’sfourconditions(2010)havea role here as non-formal criteria for assessing external validity. It should also be noted that Cartwright’s discussion applies broadly to experimental and evidential reasoning and not specifically to trial methodology. It is not a specific criticism of RCTs, although because of what she calls the “vanity of rigor” of EBM (Cartwright 2007a), the criticism is especially pertinent to RCTs. Howick is correct that masking is important only for detecting small effects with subjectively measured outcomes, but this is (unfortunately) true of many recent advances in medicine and therefore widely applicable. It is not often that we have a new intervention with the dramatic success of e.g. insulin for diabetes or surgery for acute appendicitis. Howick is right to see that the methodology of the RCT is suited for some interventions and not suited for others, but the methodology is, in fact, suited for many if not most of the health care interventions currently in development. The approaches used by Worrall, Cartwright and Howick argue that the RCT is neither a necessary nor a sufficient method for getting knowledge from clinical trials. They argue that other methods can be equally or more effective in specific circumstances and that knowledge from trials always involves projection to untested domains. EBM enthusiasts are beginning to acknowledge this sensible moderation of their views, as the recent Harveian Oration by Sir Michael Rawlins13 shows (Rawlins 2008). In this paper, Rawlins argues that “the notion that evidence can be

12 Worrall might respond that he is only criticizing those methodologists who make abstract and general claims about freedom from “all possible” biases. This is fine, so long as no conclusions are drawn for RCTs in practice. 13 Michael Rawlins is the head of NICE (National Institute of Health and Clinical Excellence) in the UK, which bases its policies and guidelines on the results of EBM. 460 Euro Jnl Phil Sci (2011) 1:451–466 reliably placed in hierarchies is illusory,” that “striking effects can be discerned without the need for RCTs” and that the findings of RCTs should be extrapolated with caution. Granting these qualifications puts EBM in the same category as other successful scientific methodologies. They are useful tools in the domains in which they work, but they do not work everywhere or always.

3.2 Effectiveness of EBM methods in practice

How reliable is EBM in practice? RCT and meta-analyses generate claims with stated confidence levels. Typically, RCTs give 95% confidence levels and meta- analyses much higher confidence levels. It follows that each RCT has a 5% chance of producing a false positive (and each meta-analysis much less). Yet, in practice, RCTs and meta-analyses are much more fallible. Ioannidis (2005) did a study of 59 highly cited original research studies. Less than half (44%) were replicated; 16% were contradicted by subsequent studies and 16% found the effect to be smaller than in the original study; the rest were not repeated or challenged. Another, more well known statistic is that studies funded by pharmaceutical companies—even when properly masked and of highest quality— have an astonishingly higher chance (three or four times the probability of studies not funded by pharmaceutical companies) of showing effectiveness of an intervention than studies not funded by pharmaceutical companies (Als-Nielsen et al. 2003; Bekelman et al. 2003; Bero et al. 2007; Lexchin et al. 2003). And LeLorier et al. (LeLorier et al. 1997) found that 35% of the time, the outcome of RCTs is not predicted accurately by previous meta-analysis. This is a large and partly unexplained failure rate. Some suggest that factors such as publication bias, time to publication bias and pharmaceutical funding bias (which subtly affects trial design and evaluation) are responsible for the worse-than- expected track record of RCTs, systematic reviews and meta-analyses. Publication bias occurs when studies with null or negative results14 are not written up or not accepted for publication because they are wrongly thought to be of less scientific significance. Steps to address this bias have been taken in many areas of medical research by creating trial registries and making the results of all trials public. Time to publication bias is a more recently discovered phenomenon: trials with null or negative results, even when they are published, take much longer than trials with positive results (6–8 years for null or negative results compared with 4–5 years for positive results) (Hopewell et al. 2007). It is possible that steps taken to correct for publication bias will also help correct for time to publication bias. The additional bias created by pharmaceutical funding is not fully understood, especially since many of these trials are properly randomized and double-masked and satisfy rigorous methodological criteria. Some suggest that pharmaceutical companies deliberately select a weak control arm, for example by selecting a low dose of the standard treatment, giving the new drug a greater chance of relative

14 In this context, a positive trial is one in which the experimental arm of the trial is more effective, a null result is one in which both arms are equally effective, and a negative trial is one in which the control arm is more effective. Euro Jnl Phil Sci (2011) 1:451–466 461 success. There can also be biases that enter into the analysis of data, particularly when endpoints are not specified in advance. It is hoped that a clinical trials registry will help correct for publication bias and ex post facto manipulation of endpoints. However, at present we are a long way from correcting for bias created by pharmaceutical funding.15 Disclosure of funding source is helpful for evaluation, but often this information is lost in systematic review and meta-analysis. Since the performance of RCTs is so flawed, it is worth asking the question whether other kinds of clinical trials, further down the evidence hierarchy, are even less reliable. This would be expected in the abstract, since the further down the evidence hierarchy, the more possible sources of bias. Studies by Benson and Hartz (2000) and Concato et al. (2000) find that many well-designed observational studies produce the same results as RCTs.16 The matter is controversial, but a recent article by Ian Shrier et al. strongly argues for the inclusion of observational studies in systematic reviews and meta-analyses (Shrier et al. 2007) This result corroborates the intervention of early AIDS activists, who argued against the imposition of RCTs for AZT on both ethical and epistemic grounds (Epstein 1996). They argued that such trials are morally objectionable in that they deprive the individuals in the placebo arm of the only hope for a cure (at that time). And they argued that such trials are epistemically unnecessary because an RCT is not the only way to discern the effectiveness of anti-retroviral drugs—a claim that, in hindsight, has proved correct as a combination of historical controlled trials and laboratory studies have provided the knowledge of dramatically effective anti- retrovirals in clinical use today. These days, of course, no-one needs to get a placebo, and RCTs can continue to detect small improvements of protocol without such strenuous moral objections. Finally, EBM has been asked to evaluate itself using its own standards of evaluation This would involve showing not merely that a specific EBM intervention improves outcomes, but that more general use of systematic evidence reviews and so forth in clinical decision making results in improved outcomes for patients. In theory, we would of course expect improved outcomes. But what matters here is not theory but practice, and no-one has yet designed or carried out a study to test this (Charlton and Miles 1998; Cohen et al. 2004; Straus and McAlister 2000).

3.3 Criticisms of EBM as a general philosophy of medicine

Like most paradigms (new ways of knowing) the light shone on the paradigm flatters it and puts everything else into the shadows. Critics have protested, variously, that EBM overlooks the role of clinical experience, expert judgment, intuition, medical authority, patient goals and values, local health care constraints and the basic medical sciences including the structure of theory and the relations of causation. This is a long and complex list of intertwined scientific, hermeneutic, political and ethical considerations. Perhaps the most common criticism of EBM is that it deals with

15 I recommend that in the meantime we correct for funding bias by asking for a higher level of significance from the results of trials funded by pharmaceutical companies. 16 An editorial in the same issue of NEJM (Pocock and Elbourne 2000) strongly protests these conclusions, partly in the name of EBM orthodoxy, but partly also on the basis of some well known RCTs which contradicted the results of observational trials 462 Euro Jnl Phil Sci (2011) 1:451–466 statistical results, and application of those results to particular cases is said to require a different set of skills (e.g. (Cohen et al. 2004; Feinstein and Horwitz 1997; Hampton 2002; Straus and McAlister 2000; Tonelli 1998)). EBM advocates dispute this, and this is the reason for the early redefinition of the enterprise: “Evidence- based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients” (Sackett et al. 1996). Sorting out this complicated dispute is beyond the scope of this paper; I do so elsewhere (Solomon, in progress). Another common complaint, which covers several specific complaints, is that EBM is a scientific approach that overlooks the “art” of medicine (e.g. (Montgomery 2006; Tonelli 1998). Elsewhere I have argued that the traditional dichotomy between the “art” and the “science” of medicine is no longer helpful (Solomon 2008). This paper focuses on what EBM leaves out, rather than on whether to count that as art or as science. In this subsection I will focus on the persistent criticism that EBM ignores the basic sciences that guide both research and clinical practice (Ashcroft 2004; Bluhm 2005; Charlton and Miles 1998; Cohen et al. 2004; Harari 2001; Tonelli 1998). Basic sciences guide research in suggesting hypotheses about disease processes and mechanisms for action of interventions. Basic sciences guide clinical practice in helping physicians tailor the results of epidemiological studies to the needs of particular patients, who may have unique physiological and pathological conditions. EBM is scientifically superficial: it measures correlations. EBM does not model or theorize about the complete organism, still less the complete organism in its social and environmental context. In terms of scientific theory, it is thin; what some have called “empiricistic” (Harari 2001).17 Charlton and Miles (1998) claim that it is “statistical rather than scientific.” Ashcroft writes that EBM is “autonomous of the basic sciences” and “blind to mechanisms of explanation and causation” (2004, p. 134). Ashcroft regards this as an advantage, rather than a disadvantage, because it means that EBM does not have to worry that our basic theories may be incorrect. Ashcroft allies himself with Nancy Cartwright’s realism about phenomenal laws and antirealism about deeper laws18 at this point. Others, however, see the eschewing of scientific theorizing in favor of discovery of robust statistical correlations as problematic (Harari 2001; Tonelli 2006). They consider theorizing as important to medicine as it is to the pure sciences. Whatever one’s views about scientific realism, EBM typically depends upon a background of basic science research that develops the interventions and suggests the appropriate protocols. It is rare for an intervention without physiological rationale to be tested (although this does happen, especially in the areas of complementary and alternative medicine, but in these cases there is typically an alternative rationale, perhaps in the frameworks of Asian metaphysics). Moreover, as discussed above, scientific judgment enters into the design of appropriate randomized controlled trials (choice of control, test population etc.) and into the interpretation of the applicability of results (external validity). Of course, many interventions with excellent physiological rationales and good in vitro and in vivo performance fail when tested in human beings or fail when tested for external

17 In fact, EBM is the successor to ancient “empiric” approaches to medicine. 18 Cartwright is, of course, a realist about underlying causal processes. Euro Jnl Phil Sci (2011) 1:451–466 463 validity. That does not mean that there is anything wrong with physiological reasoning or that we can use a more reliable method. The basic science work is fallible, but it is not dispensable. Even Nancy Cartwright (1989) would agree that we cannot replace physical theory with phenomenal laws alone. In the past 5 years a new approach to medical research has risen to prominence internationally: what is called “translational medicine,” to be achieved by creating research centers, as well as journals, conferences, training programs and so forth. The NIH has made it a priority in its “Roadmap” in 2004 and started offering Clinical and Translational Science Awards in 2006. 55 Institutes have been created (as of May 2011), mostly in universities and medical centers, and the NIH hopes to fund 60, at a total cost of $500 million annually. The European Commission plans to use most of its billion Euro a year budget for the next few years for translational research. In the UK, the National Institute for Health Research has established 11 centers at a total cost of about 100 million pounds annually. The idea behind translational medicine is to facilitate greater interaction between basic science research and research in clinical medicine.19 The buzzwords are “synergize,” “catalyze” and “interdisciplinary.” The idea is to bring the different researchers and their laboratories into greater physical proximity. This is an interesting retro- intervention in these days of global electronic communication and global travel. From the perspective of the discussion in this section, the development of translational medicine or something like it was only a matter of time. EBM has such high claims to scientific objectivity that it attracted much talent and effort from clinical researchers. Perhaps the increased focus on formal epidemiological work eventually made apparent what was left out, namely engagement with substantial physiological and biomolecular theories. The model of basic science doing the research and clinical researchers testing the products is now perceived as limited; actually, it leaves all the fun and the creativity to the basic researchers, and deprives them of the input of clinical knowledge and observations from the clinical researchers.

4 Conclusions

EBM gives a set of formal techniques for evaluating the effectiveness of clinical interventions. The techniques are powerful, especially when evaluating interventions that offer incremental advantages to current standards of care, and especially when the determination of success has subjective elements. EBM techniques do not deliver the reliability that is theoretically and statistically expected from them. Results are compromised by publication bias, time to publication bias, interests of funding organizations and other unknown factors. Maintaining a strict evidence hierarchy makes little sense when the actual reliability of “gold standard” evidence is so much less than the expected reliability. I recommend a more instrumental or pragmatic approach to EBM, in which any ranking of evidence is done by reference to the

19 Technically, “translational research” includes both the bench-to-bedside-and-back (T1) and the clinical research to everyday practice (T2) “translational blocks.” See (Woolf 2008). But most of the resources and rhetoric favor the former. 464 Euro Jnl Phil Sci (2011) 1:451–466 actual, rather than the theoretically expected, reliability of results. So, for example, RCTs and observational trials might be at the same level in the hierarchy (based on their comparable reliability in practice) and trials designed and funded by pharmaceutical companies a level below independent trials (irrespective of apparent trial rigor, based on their track record of biased outcomes). Emphasis on EBM has eclipsed other necessary research methods in medicine, even those methods necessary for its own development and application. With the recent emphasis on translational medicine, we are seeing a restoration of the recognition that clinical research requires an engagement with basic theory (e.g. physiological, genetic, biochemical) and a range of empirical techniques such as bedside observation, laboratory and animal studies. EBM works best when used in this pluralistic methodological context.

References

AHRQ. (2002). Systems to rate the strength of scientific evidence. Evidence Report/Technology Assessment 47, Als-Nielsen, B., Chen, W., Gluud, C., & Kjaergard, L. L. (2003). Association of funding and conclusions in randomized drug trials: A reflection of treatment effect or adverse events? JAMA: The Journal of the American Medical Association, 290(7), 921–928. Ashcroft, R. E. (2004). Current epistemological problems in evidence based medicine. Journal of Medical Ethics, 30(2), 131–135. Baratz, S. R., Goodman, C., & Council on Health Care Technology. (1990). Improving consensus development for health technology assessment: An international perspective. Washington: National Academy Press. Bekelman, J. E., Li, Y., & Gross, C. P. (2003). Scope and impact of financial conflicts of interest in biomedical research: A systematic review. JAMA: The Journal of the American Medical Association, 289(4), 454–465. Benson, K., & Hartz, A. J. (2000). A comparison of observational studies and randomized, controlled trials. The New England Journal of Medicine, 342(25), 1878–1886. Bero, L., Oostvogel, F., Bacchetti, P., & Lee, K. (2007). Factors associated with findings of published trials of drug- drug comparisons: Why some statins appear more efficacious than others. PLoS Medicine, 4(6), e184. Berwick, D. M. (2005). Broadening the view of evidence-based medicine. Qual. Saf. Health Care, 14(5), 315–316. Bluhm, R. (2005). From hierachy to network: a richer view of evidence for evidence-based medicine. Perspectives in Biology and Medicine, 48(4), 535–547. Canadian Task Force on Preventive Health Care. CTFPHC History/Methodology. Retrieved 07/07, 2009, from http://www.ctfphc.org Cartwright, N. (1989). Nature's capacities and their measurement. Oxford; New York: Clarendon Press; Oxford University Press. Cartwright, N. (2007a). Are RCTs the gold standard? Biosocieties, 2(2), 11–20. Cartwright, N. (2007b). Evidence-based policy: Where is our theory of evidence? Center for Philosophy of Natural and Social Science, London School of Economics, Technical Report 07/07, Cartwright, N. (2009). Evidence-based policy: What's to be done about relevance. Philosophical Studies, 143(1), 127–136. Cartwright, N. (2010). Will this policy work for you? predicting effectiveness better: How philosophy helps. Unpublished manuscript. Charlton, B. G., & Miles, A. (1998). The rise and fall of EBM. Quarterly Journal of Medicine, 91(5), 371- 371-374. Cohen, A. M., Stavri, P. S., & Hersh, W. R. (2004). A categorization and analysis of the criticisms of evidence-based medicine. International Journal of Medical Informatics, 73(1), 35-35-43. Concato, J., Shah, N., & Horwitz, R. I. (2000). Randomized, controlled trials, observational studies, and the hierarchy of research designs. The New England Journal of Medicine, 342(25), 1887–1892. Euro Jnl Phil Sci (2011) 1:451–466 465

Daly, J. (2005). Evidence-based medicine and the search for a science of clinical care. Berkeley; New York: University of California Press; Milbank Memorial Fund. Davidoff, F., Haynes, B., Sackett, D., & Smith, R. (1995). Evidence based medicine. BMJ (Clinical Research Ed.), 310(6987), 1085–1086. Denny, K. (1999). Evidence-based medicine and medical authority. Journal of Medical Humanities, 20(4), 247–263. Doll, R., Peto, R., & Clarke, M. (1999). First publication of an individually randomized trial. Controlled Clinical Trials, 20(4), 367–368. Elstein, A. S. (2004). On the origins and development of evidence-based medicine and medical decision making. Inflammation Research, 53(Supplement 2), S184-S184-S189. Epstein, S. (1996). Impure science: AIDS, activism, and the politics of knowledge. Berkeley: University of California Press. Evidence-Based Medicine Working Group. (1992). Evidence-Based Medicine. A new approach to teaching the practice of medicine. JAMA: The Journal of the American Medical Association, 268(17), 2420–2425. Feinstein, A. R., & Horwitz, R. I. (1997). Problems in the “evidence” of “evidence-based medicine”. The American Journal of Medicine, 103(6), 529–535. Goodman, N. (1955). Fact, fiction and forecast. Cambridge: Harvard University Press. Goodman, S. N. (2002). The mammography dilemma: A crisis for evidence-based medicine? Annals of Internal Medicine, 137(5 Part 1), 363–365. Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., et al. (2008). GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ (Clinical Research Ed.), 336(7650), 924–926. Hampton, J. R. (2002). Evidence-based medicine, opinion-based medicine, and real-world medicine. Perspectives in Biology and Medicine, 45(4), 549–568. Harari, E. (2001). Whose evidence? lessons from the philosophy of science and the epistemology of medicine. Australian and New Zealand Journal of Psychiatry, 35(6), 724-724-730. Hopewell, S., Clarke, M. J., Stewart, L., & Tierney, J. (2007). Time to publication for results of clinical trials. Cochrane Database of Systematic Reviews, (1) doi:10.1002/14651858.MR000011.pub2 Howick, J. (2008). Against a priori judgments of bad methodology: Questioning double-blinding as a universal methodological virtue of clinical trials. PhilSci Archive Howick, J. (2011). The philosophy of evidence-based medicine Wiley-Backwell. Ioannidis, J. P. (2005). Contradicted and initially stronger effects in highly cited clinical research. JAMA: The Journal of the American Medical Association, 294(2), 218–228. Juni, P., Witschi, A., Bloch, R., & Egger, M. (1999). The hazards of scoring the quality of clinical trials for meta-analysis. JAMA: The Journal of the American Medical Association, 282(11), 1054–1060. Kuhn, T. S. (1962). The structure of scientific revolutions. Chicago: University of Chicago Press. Kuhn, T. S. (1970). The structure of scientific revolutions (2nd ed.). Chicago: University of Chicago Press. Kuhn, T. S. (1977). The essential tension: Selected studies in scientific tradition and change. Chicago: University of Chicago Press. Kuhn, T. S., Conant, J., & Haugeland, J. (2000). The road since structure: Philosophical essays, 1970– 1993, with an autobiographical interview. Chicago: University of Chicago Press. Lakatos, I., & Musgrave, A. (1970). Criticism and the growth of knowledge. Cambridge: University Press. LeLorier, J., Gregoire, G., Benhaddad, A., Lapierre, J., & Derderian, F. (1997). Discrepancies between meta-analyses and subsequent large randomized, controlled trials. The New England Journal of Medicine, 337(8), 536–542. Lexchin, J., Bero, L. A., Djulbegovic, B., & Clark, O. (2003). Pharmaceutical industry sponsorship and research outcome and quality: Systematic review. BMJ (Clinical Research Ed.), 326(7400), 1167– 1170. Marks, H. M. (1997). The progress of experiment: Science and therapeutic reform in the united states, 1900–1990. Cambridge: Cambridge University Press. Meldrum, M. L. (1998). A calculated risk: The salk polio vaccine field trials of 1954. British Medical Journal, 317, 1233-1233-6. Meldrum, M. L. (2000). A brief history of the randomized controlled trial: From oranges and lemons to the gold standard. Hematology/Oncology Clinics of North America, 14(4), 745-745-760. Montgomery, K. (2006). How doctors think: Clinical judgment and the practice of medicine. Oxford: Oxford University Press. Pocock, S. J., & Elbourne, D. R. (2000). Randomized trials or observational tribulations? The New England Journal of Medicine, 342(25), 1907–1909. 466 Euro Jnl Phil Sci (2011) 1:451–466

Rawlins, M. (2008). De testimonio: On the evidence for decisions about the use of therapeutic interventions. Lancet, 372(9656), 2152–2161. Sackett, D. L., Rosenberg, W. M., Gray, J. A., Haynes, R. B., & Richardson, W. S. (1996). Evidence based medicine: What it is and what it isn't. BMJ (Clinical Research Ed.), 312(7023), 71–72. Sehon, S. R., & Stanley, D. E. (2003). A philosophical analysis of the evidence-based medicine debate. BMC Health Services Research, 3(14). Shrier, I., Boivin, J. F., Steele, R. J., Platt, R. W., Furlan, A., Kakuma, R., et al. (2007). Should meta- analyses of interventions include observational studies in addition to randomized controlled trials? A critical examination of underlying principles. American Journal of Epidemiology, 166(10), 1203– 1209. Solomon, M. (2008). Epistemological reflections on the art of medicine and narrative medicine. Perspectives in Biology and Medicine, 51(3), 406–417. Solomon, M. (in progress). Beyond the art and science of medicine. Unpublished manuscript. Straus, S. E., & McAlister, F. A. (2000). Evidence-based medicine: A commentary on common criticisms. Canadian Medical Association Journal, 163(7), 837–841. Tonelli, M. R. (1998). The philosophical limits of evidence-based medicine. Academic Medicine, 73(12), 1234–1240. Tonelli, M. R. (2006). Integrating evidence into clinical practice: An alternative to evidence-based approaches. Journal of Evaluation in Clinical Practice, 12(3), 248–256. Woolf, S. H. (2008). The meaning of translational research and why it matters. JAMA: The Journal of the American Medical Association, 299(2), 211–213. Worrall, J. (2002). What evidence in evidence-based medicine? Philosophy of Science, 69(S3), 316–330. Worrall, J. (2007a). Evidence in medicine and evidence-based medicine. Philosophy Compass, 2(6), 981– 1022. Worrall, J. (2007b). Why there's no cause to randomize. The British Journal for the Philosophy of Science, 58(3), 451–488. Yank, V., Rennie, D., & Bero, L. A. (2007). Financial ties and concordance between results and conclusions in meta-analyses: Retrospective cohort study. BMJ (Clinical Research Ed.), 335(7631), 1202–1205.

Evidence-Based Medicine A New Approach to Teaching the Practice of Medicine Evidence-Based Medicine Working Group

A NEW paradigm for medical practice dose of phenytoin intravenously and the year is between 43% and 51%, and at 3 is emerging. Evidence-based medicine drug is continued orally. A computed years the risk is between 51% and 60%. de-emphasizes intuition, unsystematic tomographic head scan is completely nor¬ After a seizure-free period of 18 months clinical experience, and pathophysiolog- mal, and an electroencephalogram shows his risk of recurrence would likely be ic rationale as sufficient grounds for clin- only nonspecific findings. The patient is less than 20%. She conveys this infor¬ ical decision making and stresses the very concerned about his risk of seizure mation to the patient, along with a rec¬ examination of evidence from clinical re- recurrence. How might the resident ommendation that he take his medica¬ search. Evidence-based medicine re- proceed? tion, see his family doctor regularly, and quires new skills of the physician, in- have a review of his need for medication cluding efficient literature searching and The Way of the Past ifhe remains seizure-free for 18 months. the application of formal rules of evi- Faced with this situation as a clinical The patient leaves with a clear idea of dence evaluating the clinical literature. clerk, the resident was told by her se¬ his likely prognosis. An important goal of our medical res- nior resident (who was supported in his idency program is to educate physicians view by the attending physician) that A PARADIGM SHIFT in the practice of evidence-based med- the risk of seizure recurrence is high Thomas Kuhn has described scientific icine. Strategies include a weekly, for- (though he could not put an exact num¬ paradigms as ways of looking at the mal academic half-day for residents, de- ber on it) and that was the information world that define both the problems that voted to learning the necessary skills; that should be conveyed to the patient. can legitimately be addressed and the recruitment into teaching roles of phy- She now follows this path, emphasizing range of admissible evidence that may sicians who practice evidence-based to the patient not to drive, to continue bear on their solution.4 When defects in medicine; sharing among faculty of ap- his medication, and to see his family an existing paradigm accumulate to the proaches to teaching evidence-based physician in follow-up. The patient leaves extent that the paradigm is no longer medicine; and providing faculty with in a state of vague trepidation about his tenable, the paradigm is challenged and feedback on their performance as role risk of subsequent seizure. replaced by a new way of looking at the models and teachers of evidence-based world. Medical practice is changing, and medicine. The influence of evidence- The Way of the Future the change, which involves using the based medicine on clinical practice and The resident asks herselfwhether she medical literature more effectively in medical education is increasing. knows the prognosis of a first seizure guiding medical practice, is profound and realizes she does not. She that it can be called CLINICAL SCENARIO proceeds enough appropriately to the library and, using the Grateful a paradigm shift. A junior medical resident working in Med program,1 conducts a computerized The foundations of the paradigm shift a teaching hospital admits a 43-year-old literature search. She enters the Med¬ lie in developments in clinical research previously well man who experienced a ical Subject Headings terms epilepsy, over the last 30 years. In 1960, the ran¬ witnessed grand mal seizure. He had prognosis, and recurrence, and the pro¬ domized clinical trial was an oddity. It is never had a seizure before and had not gram retrieves 25 relevant articles. Sur¬ now accepted that virtually no drug can had any recent head trauma. He drank veying the titles, one2 appears directly enter clinical practice without a demon¬ alcohol once or twice a week and had not relevant. She reviews the paper, finds stration of its efficacy in clinical trials. had alcohol on the day of the seizure. that it meets criteria she has previously Moreover, the same randomized trial Findings on physical examination are learned for a valid investigation ofprog¬ method increasingly is being applied to normal. The patient is given a loading nosis,3 and determines that the results surgical therapies6 and diagnostic tests.6 are applicable to her patient. The search Meta-analysis is gaining increasing ac¬ costs the resident $2.68, and the entire ceptance as a method of summarizing the A complete list of members of the Evidenced-Based process (including the trip to the library results of a number of randomized trials, Medicine Working Group appears at the end of this ar- and the time to make a photocopy of the and ultimately may have as profound an ticle. article) took half an hour. effect on treatment as have Reprint requests to McMaster University Health Sci- setting policy ences Centre, Room 3W10,1200 Main St W, Hamilton, The results ofthe relevant study show randomized trials themselves.7 While Ontario, Canada L8N 3Z5 (Gordon Guyatt, MD). that the patient risk of recurrence at 1 less dramatic, crucial methodological ad-

Downloaded From: http://jama.jamanetwork.com/ by a University of Ottawa User on 10/02/2014 vanees have also been made in other ar¬ sense is sufficient to allow one to eval¬ uncertainty and to acknowledge that eas, such as the assessment of diagnostic uate new tests and treatments. management decisions are often made tests8·9 and prognosis.2 4. Content expertise and clinical ex¬ in the face of relative ignorance of their A new philosophy of medical practice perience are a sufficient base from which true impact. and teaching has followed these meth¬ to generate valid guidelines for clinical The new paradigm puts a much lower odological advances. This paradigm shift practice. value on authority.20 The underlying be¬ is manifested in a number of ways. A According to this paradigm clinicians lief is that physicians can gain the skills profusion of articles has been published have a number of options for sorting out to make independent assessments of ev¬ instructing clinicians on how to access,10 clinical problems they face. They can idence and thus evaluate the credibility evaluate,11 and interpret12 the medical reflect on their own clinical experience, of opinions being offered by experts. literature. Proposals to apply the prin¬ reflect on the underlying biology, go to The decreased emphasis on authority ciples of clinical epidemiology to day- a textbook, or ask a local expert. Read¬ does not imply a rejection of what one to-day clinical practice have been put ing the introduction and discussion sec¬ can learn from colleagues and teachers, forward.3 A number of major medical tions of a paper could be considered an whose years of experience have provid¬ journals have adopted a more informa¬ appropriate way of gaining the relevant ed them with insight into methods of tive structured abstract format, which information from a current journal. history taking, physical examination, and incorporates issues of methods and de¬ This paradigm puts a high value on diagnostic strategies. This knowledge sign into the portion of an article the traditional scientific authority and ad¬ can never be gained from formal scien¬ reader sees first.13 The American Col¬ herence to standard approaches, and an¬ tific investigation. A final assumption of lege of Physicians has launched a jour¬ swers are frequently sought from direct the new paradigm is that physicians nal, ACP Journal Club, that summa¬ contact with local experts or reference whose practice is based on an under¬ rizes new publications of high relevance to the writings ofinternational experts.19 standing ofthe underlying evidence will and methodological rigor.14 Textbooks care. The New provide superior patient that provide a rigorous review of avail¬ Paradigm REQUIREMENTS FOR THE able evidence, including a methods sec¬ The assumptions ofthe new paradigm PRACTICE OF EVIDENCE-BASED tion describing both the methodological are as follows: criteria used to systematically evaluate 1. Clinical experience and the devel¬ MEDICINE the validity of the clinical evidence and opment of clinical instincts (particularly The role modeling, practice, and teach¬ the quantitative techniques used for with respect to diagnosis) are a crucial ing of evidence-based medicine requires summarizing the evidence, have begun and necessary part of becoming a com¬ skills that are not traditionally part of to appear.1516 Practice guidelines based petent physician. Many aspects of clin¬ medical training. These include precise¬ on rigorous methodological review ofthe ical practice cannot, or will not, ever be ly defining a patient problem, and what available evidence are increasingly com¬ adequately tested. Clinical experience information is required to resolve the mon.17 A final manifestation is the grow¬ and its lessons are particularly impor¬ problem; conducting an efficient search ing demand for courses and seminars tant in these situations. At the same of the literature; selecting the best of that instruct physicians on how to make time, systematic attempts to record ob¬ the relevant studies and applying rules more effective use of the medical liter¬ servations in a reproducible and unbi¬ of evidence to determine their validity3; ature in their day-to-day patient care.3 ased fashion markedly increase the con¬ being able to present to colleagues in a We call the new paradigm "evidence- fidence one can have in knowledge about succinct fashion the content of the ar¬ based medicine."18 In this article, we de¬ patient prognosis, the value of diagnos¬ ticle and its strengths and weaknesses; scribe how this approach differs from tic tests, and the efficacy of treatment. and extracting the clinical message and prior practice and briefly outline how In the absence of systematic observa¬ applying it to the patient problem. We we are building a residency program in tion one must be cautious in the inter¬ will refer to this process as the critical which a key goal is to practice, act as a pretation of information derived from appraisal exercise. role model, teach, and help residents clinical experience and intuition, for it Evidence-based medicine also involves become highly adept in evidence-based may at times be misleading. applying traditional skills of medical medicine. We also describe some of the 2. The study and understanding of training. A sound understanding of problems educators and medical prac¬ basic mechanisms of disease are neces¬ pathophysiology is necessary to inter¬ titioners face in implementing the new sary but insufficient guides for clinical pret and apply the results of clinical re¬ paradigm. practice. The rationales for diagnosis and search. For instance, most patients to treatment, which follow from basic whom we would like to the The Former generalize Paradigm pathophysiologic principles, may in fact results of randomized trials would, for The former paradigm was based on be incorrect, leading to inaccurate pre¬ one reason or another, not have been the following assumptions about the dictions about the performance of diag¬ enrolled in the most relevant study. The knowledge required to guide clinical nostic tests and the efficacy of treat¬ patient may be too old, be too sick, have practice. ments. other underlying illnesses, or be unco¬ 1. Unsystematic observations from 3. Understanding certain rules of operative. Understanding the underly¬ clinical experience are a valid way of evidence is necessary to correctly in¬ ing pathophysiology allows the clinician building and maintaining one's knowl¬ terpret literature on causation, progno¬ to better judge whether the results are edge about patient prognosis, the value sis, diagnostic tests, and treatment applicable to the patient at hand and of diagnostic tests, and the efficacy of strategy. also has a crucial role as a conceptual treatment. It follows that clinicians should reg¬ and memory aid. 2. The study and understanding of ularly consult the original literature (and Another traditional skill required of basic mechanisms of disease and patho- be able to critically appraise the meth¬ the evidence-based physician is a sen¬ physiologic principles are a sufficient ods and results sections) in solving clin¬ sitivity to patients' emotional needs. Un¬ guide for clinical practice. ical problems and providing optimal pa¬ derstanding patients' suffering21 and how 3. A combination of thorough tradi¬ tient care. It also follows that clinicians that suffering can be ameliorated by the tional medical training and common must be ready to accept and live with caring and compassionate physician are

Downloaded From: http://jama.jamanetwork.com/ by a University of Ottawa User on 10/02/2014 Evaluation Form for Clinical Teaching Unit Attending Physicians Rating Domain Unsatisfactory Needs Improvement Satisfactory Good Excellent Role model of practice Seldom cites evidence Often fails to substantiate Usually substantiates Substantiates decisions; Always substantiates of evidence-based to support clinical decisions with evidence decisions with is aware of decisions or medicine decisions evidence methodological issues acknowledges limitations of evidence Leads practice of Never assigns problems Produces suboptimal volume Assigns problems and Discusses literature Same as "Good" rating, evidence-based to be resolved or follow-through of problem follows through with retrieval, methodology and makes it exciting medicine through literature resolution through literature discussion, including of papers, application and fun methodology to individual patient

fundamental requirements for medical articles and arriving at bottom lines re¬ ical teaching roles available to them. practice. These skills can be acquired garding the strength of evidence and Second, a program of more rigorous through careful observation of patients how it bears on the clinical problem. evaluation of attending physicians has and of physician role models. Here too, They learn to present the methods and been instituted. One of the areas eval¬ though, the need for systematic study results in a succinct fashion, emphasiz¬ uated is the extent to which attending and the limitations of the present evi¬ ing only the key points. A wide-ranging physicians are effective in teaching ev¬ dence must be considered. The new par¬ discussion, including issues of underly¬ idence-based medicine. The relevant adigm would call for using the techniques ing pathophysiology and related ques¬ items from the evaluation form are re¬ of behavioral science to determine what tions of diagnosis and management, fol¬ produced in the Table. patients are really looking for from their lows presentation of the articles. Third, because it is new to both teach¬ physicians22 and how physician and pa¬ The second part of the half-day is de¬ ers and learners, and because most clin¬ tient behavior affects the outcome of voted to the physical examination. Clin¬ ical teachers have observed few role care.23 Ultimately, randomized trials ical teachers present optimal techniques models and have not received formal using different strategies for interact¬ of examination with attention to what is training, teaching evidence-based med¬ ing with patients (such as the random¬ known about their reproducibility and icine is not easy. To help attending phy¬ ized trial conducted by Greenfield and accuracy. sicians improve their skills in this area, colleagues24 that demonstrated the pos¬ 2. Facilities for computerized litera¬ we have encouraged them to form part¬ itive effects of increasing patients' in¬ ture searching are available on the teach¬ nerships, which involve attending the volvement with their care) may be ing medical ward in each of the four partner's clinical rounds, making obser¬ appropriate. teaching hospitals. Costs of searching vations, and providing formal feedback. Since evidence-based medicine in¬ are absorbed by the residency program. One learns through observation and volves skills ofproblem defining, search¬ Residents not familiar with computer through criticisms of one's performance. ing, evaluating, and applying original searching, or the Grateful Med program A number of faculty members have par¬ medical literature, it is incumbent on we use, are instructed at the beginning ticipated in this program. residency programs to teach these skills. of the rotation. Research in our insti¬ To further facilitate attending physi¬ Understanding the barriers to educat¬ tution has shown that MEDLINE cians' improving their skills, the De¬ ing physicians-in-training in evidence- searching from clinical settings is fea¬ partment of Medicine held a retreat de¬ based medicine can lead to more effec¬ sible with brief training.26 A subsequent voted to sharing strategies for effective tive teaching strategies. investigation demonstrated that inter¬ clinical teaching. Part of the workshop, nal medicine house staff who have com¬ attended more than 30 faculty mem¬ EVIDENCE-BASED MEDICINE IN A by puter access on the ward and feedback was devoted to evidence- MEDICAL RESIDENCY bers, teaching concerning their searching do an aver¬ based medicine. Some of the strategies The Internal Medicine Residency Pro¬ age of more than 3.6 searches per that were adduced are briefly summa¬ gram at McMaster University has an month.26 House staff believe that more rized in the next section. explicit commitment to producing prac¬ than 90% oftheir searches that are stim¬ titioners of evidence-based medicine. ulated by a patient problem lead to some EFFECTIVE TEACHING OF While other clinical departments at improvement in patient care.25 EVIDENCE-BASED MEDICINE McMaster have devoted themselves to 3. Assessment of searching and crit¬ teaching evidence-based medicine, the ical appraisal skills is being incorporat¬ Role Modeling commitment is strongest in the Depart¬ ed into the evaluation of residents. Attending physicians must be enthu¬ ment of Medicine. We will therefore fo¬ 4. We believe that the new paradigm siastic, effective role models for the prac¬ cus on the Internal Medicine Residency will remain an academic mirage with tice of evidence-based medicine (even in in our discussion and briefly outline some little relation to the world of day-to-day high-pressure clinical settings, such as of the strategies we are using in imple¬ clinical practice unless physicians-in- intensive care units). Providing a model menting the paradigm shift. training are exposed to role models who goes a long way toward inculcating at¬ 1. The residents spend each Wednes¬ practice evidence-based medicine. As a titudes that lead learners to develop day afternoon at an academic half-day. result, the residency program has skills in critical appraisal. Acting as a At the beginning of each new academic placed major emphasis on ensuring this role model involves specifying the year, the rules of evidence that relate to exposure. strength of evidence that supports clin¬ articles concerning therapy, diagnosis, First, a focus of recruitment for our ical decisions. In one case, the teacher prognosis, and overviews are reviewed. Department of Medicine faculty has can point to a number of large random¬ In subsequent sessions, the discussion been internists with training in clinical ized trials, rigorously reviewed and in¬ is built around a clinical case, and two epidemiology. These individuals have the cluded in a meta-analysis, which allows original articles that bear on the prob¬ skills and commitment to practice evi¬ one to say how many patients one must lem are presented. The residents are dence-based medicine. The residency treat to prevent a death. In other cases, responsible for critically appraising the program works to ensure they have clin- the best evidence may come from ac-

Downloaded From: http://jama.jamanetwork.com/ by a University of Ottawa User on 10/02/2014 cepted practice or one's clinical experi¬ Diagnosis.—Has the diagnostic test Institutional experience can also pro¬ ence and instincts. The clinical teacher been evaluated in a patient sample that vide important insights. Diagnostic tests should make it clear to learners on what included an appropriate spectrum ofmild may differ in their accuracy depending basis decisions are being made. This can and severe, treated and untreated dis¬ on the skill of the practitioner. A local be done efficiently. For instance: ease, plus individuals with different but expert in, for instance, diagnostic ultra¬ confused disorders?28 sound Prospective studies suggest that Mr Jones' commonly Was may produce far better results there an blind than the risk of a major vascular event in the first independent, comparison average from the published lit¬ year after his infarct is 4%; a meta-analysis with a "gold standard" of diagnosis?28 erature. The effectiveness and compli¬ of randomized trials of aspirin in this situa¬ Treatment.—Was the assignment of cations associated with therapeutic in¬ tion suggests a risk reduction of 25%; we patients to treatments randomized?29 terventions, particularly surgical pro¬ such would have to treat 100 patients to pre¬ Were all patients who entered the study cedures, may also differ among institu¬ vent an event21; given the minimal expense accounted for at its conclusion?29 tions. When optimal care is taken to and toxicity of low-dose, enteric-coated as¬ Review Articles.—Were both record observations pirin, treating Mr Jones is clearly warranted. explicit reproducibly methods used to determine which arti¬ and avoid bias, clinical and institutional Or: cles to include in the review?30 experience evolves into the systematic How longto treat a patient with antibiotics fol¬ As learners become more sophisticat¬ search for knowledge that forms the core lowing pneumonia has not been systematically ed, additional criteria can be introduced. of evidence-based medicine.32 studied; so, my recommendation that we give The criteria should not be presented in Misinterpretation 2.—Understand¬ Mrs Smith 3 days of intravenous antibiotics such a way that fosters nihilism (if the ing ofbasic investigation and pathophys- and treat her for a total of 10 is days arbitrary; is not it is useless no in evidence-based somewhat shorter or courses of treat¬ study randomized, iology plays part longer and no valuable medicine. ment would be equally reasonable. provides information), but as a way of arrive at the Correction.—The dearth of In the latter of helping adequate type situation, dog¬ strength of inference associated with a evidence demands that clinical problem matic or insistence on a rigid following clinical decision. Teachers can point out solving must rely on an understanding course of action would not be particular instances in which criteria can be vio¬ of underlying pathophysiology. More¬ appropriate. lated without reducing the strength of over, a good understanding of patho¬ inference. is for Critical physiology necessary interpreting Appraisal clinical observations and for appropri¬ It is crucial that critical is¬ appraisal METHODS FOR SCALING THE ate interpretation of evidence (especial¬ sues arise from patient problems that ly in deciding on its generalizability). the learner is BARRIERS TO THE DISSEMINATION currently confronting, OF EVIDENCE-BASED MEDICINE Misinterpretation 3.—Evidence- demonstrating that critical appraisal is based medicine standard About ignores aspects a pragmatic and central aspect, not an Misapprehensions of clinical training, such as the physical academic or tangential element of op¬ Evidence-Based Medicine examination. timal patient care. The problem select¬ In developing the practice and teach¬ Correction.—Careful history taking ed for critical appraisal must be one that ing of evidence-based medicine at our and physical examination provide much, the learners recognize as important, feel institution, we have found that the na¬ and often the best, evidence for diag¬ uncertain, and do not fully trust expert ture of the new paradigm is sometimes nosis and direct treatment decisions. The opinion; in other words, they must feel misinterpreted. Recognizing the limita¬ clinical teacher of evidence-based med¬ it is worth the effort to find out what the tions of intuition, experience, and un¬ icine must give considerable attention literature says on a topic. The likeliest derstanding of pathophysiology in per¬ to teaching the methods of history tak¬ candidate topics are common problems mitting strong inferences may be mis¬ ing and clinical examination, with par¬ where learners have been exposed to interpreted as rejecting these routes to ticular attention to which items have divergent opinions (and thus there is knowledge. Specific misinterpretations demonstrated validity and to strategies disagreement and/or uncertainty among ofevidence-based medicine and their cor¬ that enhance observer agreement. the learners). The clinical teacher should rections follow: keep these requirements in mind when Misinterpretation 1.—Evidence- Barriers to Teaching considering questions to encourage the based medicine ignores clinical experi¬ Evidence-Based Medicine learners to address. It can be useful to ence and clinical intuition. Difficulties we have encountered in ask all members of the group their opin¬ Correction.—On the contrary, it is teaching evidence-based medicine in¬ ion about the clinical problem at hand. important to expose learners to excep¬ clude the following: One can then ensure that the problem is tional clinicians who have a gift for in¬ 1. Many house staff start with rudi¬ appropriate for a critical appraisal ex¬ tuitive diagnosis, a talent for precise mentary critical appraisal skills and the ercise by asking the group the following observation, and excellent judgment in topic may be threatening for them. questions: making difficult management decisions. 2. People like quick and easy answers. 1. It seems the group is uncertain Untested signs and symptoms should Cookbook medicine has its appeal. Crit¬ about the optimal approach. Is that not be rejected out of hand. They may ical appraisal involves additional time right? prove extremely useful and ultimately and effort and may be perceived as in¬ 2. Do you feel it is important for us to be proved valid through rigorous test¬ efficient and distracting from the real sort out this question by going to the ing. The more the experienced clinicians goal (to provide optimal care for pa¬ original literature? can dissect the process they use in di¬ tients). and it to learn¬ 3. For clinical Criteria agnosis,31 clearly present many questions, high- Methodological ers, the greater the benefit. Similarly, quality evidence is lacking. If such ques¬ Criteria for methodological rigor must the gain for students will be greatest tions predominate in attempts to intro¬ be few and simple. Most published crite¬ when clues to optimal diagnosis and duce critical appraisal, a sense offutility ria can be overwhelming for the novice. treatment are culled from the barrage can result. Suggested criteria for studies of diagno¬ of clinical information in a systematic 4. The concepts of evidence-based sis, treatment, and review articles follow: and reproducible fashion. medicine are met with skepticism by

Downloaded From: http://jama.jamanetwork.com/ by a University of Ottawa User on 10/02/2014 many faculty members who are there¬ views meeting scientific principles30,33 tient outcomes may appear to be an in¬ fore unenthusiastic about modifying and collections ofmethodologically sound ternal contradiction. As has been point¬ their teaching and practice in accordance and highly relevant articles14 can mark¬ ed out, however, evidence-based with its dictates. edly increase efficiency. Other solutions medicine does not advocate a rejection These problems can be ameliorated will emerge over time. Health educa¬ of all innovations in the absence of de¬ by use of the strategies described in the tors will continue to find better ways of finitive evidence. When definitive evi¬ previous section on effective teaching of role modeling and teaching evidence- dence is not available, one must fall back evidence-based medicine. Threat can be based medicine. Standards in writing on weaker evidence (such as the com¬ reduced by making a contract with the reviews and texts are likely to change, parison of graduates of two medical residents, which sets out modest and with a greater focus on methodological schools that use different approaches achievable goals, and further reduced rigor.15·16 Evidence-based summaries will cited above) and on biologic rationale. by the attending physician role model¬ therefore become increasingly available. The rationale in this case is that physi¬ ing the practice of evidence-based med¬ Practical approaches to making evi¬ cians who are up-to-date as a function of icine. Inefficiency can be reduced by dence-based summaries easier to apply their ability to read the current litera¬ teaching effective searching skills and in clinical practice, many based on com¬ ture critically, and are able to distin¬ simple guidelines for assessing the va¬ puter technology, will be developed and guish strong from weaker evidence, are lidity of the papers. In addition, one can expanded. As described earlier, we are likely to be more judicious in the ther¬ emphasize that critical appraisal as a already using computer searching on the apy they recommend. Physicians who strategy for solving clinical problems is ward. In the future, the results of di¬ understand the properties of diagnostic most appropriate when the problems are agnostic tests may be provided with the tests and are able to use a quantitative common in one's own practice. Futility associated sensitivity, specificity, and approach to those tests are likely to make can be reduced by, particularly initially, likelihood ratios. Health policymakers more accurate diagnoses. While this ra¬ targeting critical appraisal exercises to may find that the structure of medical tionale appears compelling to us, com¬ areas in which there is likely to be high- practice must be shifted in basic ways to pelling rationale has often proved mis¬ quality evidence that will affect clinical facilitate the practice of evidence-based leading. Until more definitive evidence decisions. Skepticism of faculty mem¬ medicine. Increasingly, scientific over¬ is adduced, adoption of evidence-based bers can be reduced by the availability views will be systematically integrated medicine should appropriately be re¬ of "quick and dirty" (as well as more with information regarding toxicity and stricted to two groups. One group com¬ sophisticated) courses on critical apprais¬ side effects, cost, and the consequences prises those who find the rationale com¬ al of evidence and by the teaching part¬ of alternative courses of action to de¬ pelling, and thus believe that use of the nerships and teaching workshops de¬ velop clinical policy guidelines.34 The evidence-based medicine approach is scribed earlier. prospects for these developments are likely to improve clinical care. A second Many problems in the practice and both bright and exciting. group comprises those who, while skep¬ teaching of evidence-based medicine re¬ tical of improvements in patient out¬ main. both DOES TEACHING AND LEARNING believe it is that Many physicians, including EVIDENCE-BASED MEDICINE come, very unlikely residents and members, are still deterioration in care results from the faculty IMPROVE PATIENT OUTCOMES? skeptical about the tenets of the new evidence-based approach and who find paradigm. A medical residency is full of The proof of the pudding of evidence- that the practice of medicine in the new competing demands, and the appropri¬ based medicine lies in whether patients paradigm is more exciting and fun. ate balance between is not cared for in this fashion better goals always enjoy CONCLUSION evident. At the same time, we are buoyed health. This proof is no more achievable by the number of residents and faculty for the new paradigm than it is for the Based on an awareness of the limita¬ who have enthusiastically adopted the old, for no long-term randomized trials tions of traditional determinants of clin¬ new approach and found ways to inte¬ of traditional and evidence-based med¬ ical decisions, a new paradigm for med¬ grate it into their learning and practice. ical education are likely to be carried ical practice has arisen. Evidence-based out. What we do have are a number of medicine deals directly with the uncer¬ Barriers to Practicing short-term studies which confirm that tainties of clinical medicine and has the Evidence-Based Medicine the skills ofevidence-based medicine can potential for transforming the educa¬ Even if our residency program is suc¬ be taught to medical students35 and med¬ tion and practice of the next generation cessful in producing graduates who en¬ ical residents.36 In addition, a study com¬ of physicians. These physicians will con¬ ter the world of clinical practice enthu¬ pared the graduates of a medical school tinue to face an exploding volume of siastic to apply what they have learned that operates under the new paradigm literature, rapid introduction of new about evidence-based medicine, they will (McMaster) with the graduates of a tra¬ technologies, deepening concern about face difficult challenges. Economic con¬ ditional school. A random sample of burgeoning medical costs, and increas¬ straints and counterproductive incen¬ McMaster graduates who had chosen ing attention to the quality and outcomes tives may compete with the dictates of careers in family medicine were more of medical care. The likelihood that ev¬ evidence as determinants of clinical de¬ knowledgeable with respect to current idence-based medicine can help amelio¬ cisions; the relevant literature may not therapeutic guidelines in the treatment rate these problems should encourage be readily accessible; and the time avail¬ of hypertension than were the gradu¬ its dissemination. able may be insufficient to carefully re¬ ates of the traditional school.37 These Evidence-based medicine will require view the evidence (which may be volu¬ results suggest that the teaching of ev¬ new skills for the physician, skills that minous) relevant to a pressing clinical idence-based medicine may help grad¬ residency programs should be equipped problem. uates stay up-to-date. Further evalua¬ to teach. While strategies for inculcat¬ Some solutions to these problems are tion of the evidence-based medicine ap¬ ing the principles ofevidence-based med¬ already available. Optimal integration proach is necessary. icine remain to be refined, initial expe¬ ofcomputer technology into clinical prac¬ Our advocating evidence-based med¬ rience has revealed a number of effec¬ tice facilitates finding and accessing ev¬ icine in the absence of definitive evi¬ tive approaches. Incorporating these idence. Reference to literature over- dence of its superiority in improving pa- practices into postgraduate medical ed-

Downloaded From: http://jama.jamanetwork.com/ by a University of Ottawa User on 10/02/2014 ucation and continuing to work on their partment of Medicine, McMaster University; Allan tize Cardio-Vasculaires, Paris, France; Virginia further will result in more Detsky, MD, PhD, Department of Clinical Epide¬ Moyer, MD, Department of Pediatrics, University development miology and Biostatistics, McMaster University of Texas, Houston; Cynthia Mulrow, MD, Depart¬ rapid dissemination and integration of and Departments of Health Administration and ment of Medicine, University of Texas, San Anto¬ the new paradigm into medical practice. Medicine, University of Toronto (Ontario); Murray nio; Paul Links, MD, MSc, Department of Psychi¬ Enkin, MD, Departments of Clinical Epidemiology atry, McMaster University; Andrew Oxman, MD, The Evidence-Based Medicine Working Group and Biostatistics and Obstetrics and Gynaecology, MSc, Departments of Clinical Epidemiology and comprised the following: Gordon Guyatt (chair), McMaster University; Pamela Frid, MD, Depart¬ Biostatistics and Family Medicine, McMaster Uni¬ MD, MSc, John Cairns, MD, David Churchill, MD, ment of Pediatrics, Queen's University, Kingston, versity; Jack Sinclair, MD, Departments of Clinical MSc, Deborah Cook, MD, MSc, Brian Haynes, MD, Ontario; Martha Gerrity, MD, Department of Med¬ Epidemiology and Biostatistics and Pediatrics, MSc, PhD, Jack Hirsh, MD, Jan Irvine, MD, MSc, icine, University of North Carolina, Chapel Hill; McMaster University; and Peter Tugwell, MD, Mark Levine, MD, MSc, Mitchell Levine, MD, MSc, Andreas Laupacis, MD, MSc, Department of Clin¬ MSc, Department of Medicine, University of Ot¬ Jim Nishikawa, MD, and David Sackett, MD, MSc, ical Epidemiology and Biostatistics, McMaster tawa (Ontario). Departments of Medicine and Clinical Epidemiol¬ University and Department of Medicine, Univer¬ Drs Cook and Guyatt are Career Scientists of ogy and Biostatistics, McMaster University, sity of Ottawa (Ontario); Valerie Lawrence, MD, the Ontario Ministry of Health. Dr Haynes is a Na¬ Hamilton, Ontario; Patrick Brill-Edwards, MD, Department of Medicine, University of Texas tional Health Scientist, National Health Research Hertzel Gerstein, MD, MSc, Jim Gibson, MD, Ro¬ Health Science Center at San Antonio and Audie L. and Development Program, Canada. Drs Jaeschke man Jaeschke, MD, MSc, Anthony Kerigan, MD, Murphy Memorial Veterans Hospital, San Antonio, and Cook are Scholars of the St Joseph's Hospital MSc, Alan Neville, MD, and Akbar Panju, MD, De- Tex; Joel Menard, MD, Centre de Medicine Trezen- Foundation, Hamilton, Ontario.

References 1. Lindberg DA. Information systems to support 13. Haynes RB, Mulrow CD, Huth EJ, Altman DG, ings. In: Clayton PD, ed. Proceedings of the Fifth medical practice and scientific discovery. Methods Gardner MJ. More informative abstracts revisited. Annual Symposium on Computer Applications in Inform Med. 1989;28:202-206. Ann Intern Med. 1990;113:69-76. Medical Care, November 1991; Washington, DC. 2. Hart YM, Sander JW, Johnson AL, Shorvon SD. 14. Haynes RB. The origins and aspirations of ACP New York, NY: McGraw-Hill International Book National general practice study of epilepsy: recur- Journal Club. Ann Intern Med. 1991;114(ACP J Co; 1992:73-77. rence after a first seizure. Lancet. 1990;336:1271\x=req-\ Club. suppl 1):A18. 27. Laupacis A, Sackett DL, Roberts RS. An as- 1274. 15. Chalmers I, Enkin M, Keirse MJNC, eds. Ef- sessment of clinically useful measures of the con- 3. Sackett DL, Haynes RB, Guyatt GH, Tugwell P. fective Care in Pregnancy and Childbirth. New sequences of treatment. N Engl J Med. 1988;318: Clinical Epidemiology: A Basic Science for Clini- York, NY: Oxford University Press Inc; 1989. 1728-1733. cial Medicine. 2nd ed. Boston, Mass: Little Brown 16. Sinclair JC, Braken MB, eds. Effective Care of 28. Department of Clinical Epidemiology and Bio- & Co Inc; 1991:173-186. the Newborn Infant. New York, NY: Oxford Uni- statistics, McMaster University. How to read clin- 4. Kuhn TS. The Structure of Scientific Revolu- versity Press Inc; 1992. ical journals, II: to learn about a diagnostic test. tions. Chicago, Ill: University of Chicago Press; 17. Audet AM, Greenfield S, Field M. Medical prac- Can Med Assoc J. 1981;124:703-710. 1970. tice guidelines: current activities and future direc- 29. Department of Clinical Epidemiology and Bio- 5. European Carotid Surgery Trialists' Collabora- tions. Ann Intern Med. 1990;113:709-714. statistics, McMaster University. How to read clin- tive Group. MRC European Carotid Surgery Trial: 18. Guyatt GH. Evidence-based medicine. Ann In- ical journals, V: to distinguish useful from useless interim results for symptomatic patients with se- tern Med. 1991;114(ACP J Club. suppl 2):A-16. or even harmful therapy. Can Med Assoc J. 1981; vere (70-99%) or with mild (0-29%) carotid stenosis. 19. Light DW. Uncertainty and control in profes- 124:1156-1162. Lancet. 1991;337:1235-1243. sional training. J Health Soc Behav. 1979;20:310\x=req-\ 30. Oxman AD, Guyatt GH. Guidelines for reading 6. Larsen ML, Horder M, Mogensen EF. Effect of 322. literature reviews. Can Med Assoc J. 1988;138:697\x=req-\ long-term monitoring of glycosylated hemoglobin 20. Chalmers I. Scientific inquiry and authoritar- 703. levels in insulin-dependent diabetes mellitus. NEngl ianism in perinatal care and education. Birth. 1983; 31. Campbell EJM. The diagnosing mind. Lancet. J Med. 1990;323:1021-1025. 10:151-164. 1987;1:849-851. 7. L'Abbe KA, Detsky AS, O'Rourke K. Meta-anal- 21. Cassell EJ. The nature of suffering and the 32. Feinstein AR. What kind of basic science for ysis in clinical research. Ann Intern Med. 1987;107: goals of medicine. N Engl J Med. 1982;306:639-645. clinical medicine? N Engl J Med. 1970;283:847-853. 224-233. 22. Ende J, Kazis L, Ash A, Moskowitz MA. Mea- 33. Mulrow CD. The medical review article: state 8. Ransohoff DF, Feinstein AR. Problems of spec- suring patients' desire for autonomy: decision-mak- of the science. Ann Intern Med. 1987;106:485-488. trum and bias in evaluating the efficacy of diag- ing and information-seeking preferences among 34. Eddy DM. Guidelines for policy statements: the nostic tests. N Engl J Med. 1978;299:926-930. medical patients. J Gen Intern Med. 1989;4:23-30. explicit approach. JAMA. 1990;263:2239-2243. 9. Nierenberg AA, Feinstein AR. How to evaluate 23. Carter WB, Inui TS, Kukull WA, Haigh VH. 35. Bennett KJ, Sackett DL, Haynes RB, et al. A a diagnostic market test: lessons from the rise and Outcome-based doctor-patient interaction analysis. controlled trial of teaching critical appraisal of the fall ofdexamethasone suppression test. JAMA. 1988; Med Care. 1982;20:550-556. clinical literature to medical students. JAMA. 1987; 259:1699-1702. 24. Greenfield S, Kaplan S, Ware JE Jr. Expanding 257:2451-2454. 10. Haynes RB, McKibbon KA, Fitzgerald D, patient involvement in care: effects on patient out- 36. Kitchens JM, Pfeifer MP. Teaching residents to Guyatt GH, Walker CJ, Sackett DL. How to keep comes. Ann Intern Med. 1985;102:520-528. read the medical literature: a controlled trial of a up with the medical literature, V: access by per- 25. Haynes RB, McKibbon A, Walker CJ, Ryan N, curriculum in critical appraisal/clinical epidemiol- sonal computer to the medical literature. Ann In- Fitzgerald D, Ramsden ME. Online access to MED- ogy. J Gen Intern Med. 1989;4:384-387. tern Med. 1986;105:810-816. LINE in clinical settings: a study of use and use- 37. Shin J, Haynes RB. Does a problem-based, self- 11. Bulpitt CJ. Confidence intervals. Lancet. 1987; fulness. Ann Intern Med. 1990;112:78-84. directed undergraduate medical curriculum pro- 1:494-497. 26. McKibbon KA, Haynes RB, Johnston ME, mote continuing clinical competence? Clin Res. 1991; 12. Godfrey K. Simple linear regression in medical Walker CJ. A study to enhance clinical end-user 39:143A. research. N Engl J Med. 1985;313:1629-1636. MEDLINE search skills: design and baseline find-

Downloaded From: http://jama.jamanetwork.com/ by a University of Ottawa User on 10/02/2014