Euro Jnl Phil Sci (2011) 1:451–466 DOI 10.1007/s13194-011-0034-6 ORIGINAL PAPER IN PHILOSOPHY OF MEDICINE

Just a paradigm: evidence-based medicine in epistemological context

Miriam Solomon

Received: 8 October 2010 /Accepted: 28 July 2011 /Published online: 6 September 2011 # Springer Science+Business Media B.V. 2011

Abstract Evidence-Based Medicine (EBM) developed from the work of clinical epidemiologists at McMaster University and Oxford University in the 1970s and 1980s and self-consciously presented itself as a "new paradigm" called "evidence- based medicine" in the early 1990s. The techniques of the randomized controlled trial, systematic review and meta-analysis have produced an extensive and powerful body of research. They have also generated a critical literature that raises general concerns about its methods. This paper is a systematic review of the critical literature. It finds the description of EBM as a Kuhnian paradigm helpful and worth taking further. Three kinds of criticism are evaluated in detail: criticisms of procedural aspects of EBM (especially from Cartwright, Worrall and Howick), data showing the greater than expected fallibility of EBM (Ioaanidis and others), and concerns that EBM is incomplete as a philosophy of science (Ashcroft and others). The paper recommends a more instrumental or pragmatic approach to EBM, in which any ranking of evidence is done by reference to the actual, rather than the theoretically expected, reliability of results. Emphasis on EBM has eclipsed other necessary research methods in medicine. With the recent emphasis on translational medicine, we are seeing a restoration of the recognition that clinical research requires an engagement with basic theory (e.g. physiological, genetic, biochemical) and a range of empirical techniques such as bedside observation, laboratory and animal studies. EBM works best when used in this context.

Keywords Evidence-based medicine . Philosophy of medicine . Kuhnian paradigm . Nancy Cartwright . Randomized controlled trial . Translational medicine . Evidence hierachy. Meta-analysis . John Ioannidis . John Worrall

I am grateful to two anonymous reviewers for the European Journal for Philosophy of Science for helpful corrections and comments on this paper. M. Solomon (*) Department of Philosophy, Temple University, Philadelphia, PA 19122, USA e-mail: [email protected] 452 Euro Jnl Phil Sci (2011) 1:451–466

1 Introduction

Evidence-Based Medicine (EBM)1 is the application of methods of clinical epidemiology to the practice of medicine more generally. It was inspired by the post-World War II work of Archibald Cochrane, developed from the work of clinical epidemiologists at McMaster University and Oxford University in the 1970s and 1980s, and self-consciously presented as a “new paradigm” called “evidence-based medicine” in the early 1990s (Evidence-Based Medicine Working Group 1992). EBM was embraced in Canada and the UK in the 1990s, received with some ambivalence in the United States, and adopted in many other countries, both developed and developing (Daly 2005). Its techniques of population based studies and systematic review have produced an extensive and powerful body of knowledge about medical diagnosis and treatment. A canonical and helpful definition of EBM2 is that of Davidoff et al. (Davidoff et al. 1995)inaneditorialin the British Medical Journal: “In essence, evidence based medicine is rooted in five linked ideas: firstly, clinical decisions should be based on the best available scientific evidence; secondly, the clinical problem - rather than habits or protocols - should determine the type of evidence to be sought; thirdly, identifying the best evidence means using epidemiological and biostatistical ways of thinking; fourthly, conclusions derived from identifying and critically appraising evidence are useful only if put into action in managing patients or making health care decisions; and, finally, performance should be constantly evaluated.” EBM regards its own epistemic techniques as superior to other more traditional methods such as clinical experience, expert opinion, and physiological reasoning. This is because the more traditional techniques are viewed as more fallible. There is not one new technique, but several. The following are typically regarded as part of EBM: 1. Rigorous design of clinical trials, especially the randomized controlled trial (RCT). The RCT is to be used wherever physically and ethically feasible. The trial should be double-masked (traditionally, “double-blinded”3)wherever possible. 2. Systematic evidence review and meta-analysis, including grading of the evidence in “evidence hierarchies.” 3. Outcome measures (leading to suggestions for improvement)

1 The term “evidence-based practice” may be replacing EBM, acknowledging the fact that the practice of medicine requires not only physicians but other health care professionals. 2 Some canonical definitions are unhelpful for a general understanding of EBM, for example, that given in (Sackett et al. 1996): “Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.” This particular definition is probably used widely because it is brief, and because it has a rhetorical purpose—to address the frequent criticism of EBM that it applies to populations, not individuals. 3 “Double-masked” has largely replaced “double-blinded,” in order to avoid inappropriate use of terms relating to disability. Euro Jnl Phil Sci (2011) 1:451–466 453

The RCT has often been described as the “gold standard” of evidence for effectiveness of medical interventions. It is a powerful technique, originally developed by the geneticist R.A. Fisher and applied for the first time in a medical context by A. Bradford Hill’s 1948 evaluation of streptomycin for tuberculosis (Doll et al. 1999). The double-masked RCT is designed to control for the placebo effect, for selection and other confounding biases, and for confirmation biases. EBM also includes systematic and formal techniques for combining the results of different clinical trials. A systematic review does a thorough search of the literature and an evaluation and grading of clinical trials. An evidence hierarchy is typically used to structure the judgments of quality and strength of evidence. Meta-analysis integrates the actual data from different but similar high-quality trials to give an overall single statistical result. Often, EBM is supplemented with formal techniques from Medical Decision Making (MDM) such as risk/benefit calculations. The risk/benefit calculations can be made for individual patients, making use of patient judgments of utility, or they can be made in the context of health care economics, for populations. MDM seeks to avoid common errors of judgment, such as availability and salience biases, in medical decision making.4 The overall project is to use the techniques of EBM (and sometimes also MDM) to construct practice guidelines and to take care of individual patients. Each technique—the RCT, other high quality clinical trials, meta-analysis and systematic review–is based on its own core technical successes. The techniques fit together, and share a reliance on statistics, probability theory and utility theory. Journals, centers, clearinghouses, collaborations, educational programs, textbooks, committees and governments all produce and disseminate EBM. EBM rose to dominance right after the prominence of consensus conferences for assessment of complex and sometimes conflicting evidence and may have been partly responsible for the decline of traditional consensus conferences. As late as 1990, an Institute of Medicine report evaluating the international uses of medical consensus conferences said “Group judgment methods are perhaps the most widely used means of assessment of medical technologies in many countries” (Baratz et al. 1990). Just a few years later, expert consensus is viewed in the same medical circles as the lowest level of evidence, when it is included in the evidence at all. For example, the Canadian Task force on Preventive Health Care which began in 1979 as a consensus conference program now explicitly declares “Evidence takes precedence over consensus” (Canadian Task Force on Preventive Health Care). EBM has not completely replaced group judgment, however. Consensus conferences (or something similar) are often still used for producing evidence- based guidelines or policy, that is, for translating a systematic review into a practical recommendation. And group judgment may be needed to set the standards to be used in systematic review. I will comment on this continued reliance on consensus methods later in the paper. There is some indication that EBM is now past its peak, and being overshadowed in part by a new approach, that of “translational medicine” (Woolf 2008). Considerable resources from the NIH, from the European Commission and from

4 A useful essay discussing the differences and interactions between EBM and MDM is (Elstein 2004) 454 Euro Jnl Phil Sci (2011) 1:451–466 the National Institute for Health Research in the UK have been redirected to “bench to bedside and back” research, which is typically the research that takes place before the clinical trials that are core to EBM. Donald Berwick, the founder of the Institute for Healthcare Improvement (which is the leading organization for quality improvement in healthcare), now claims that “we have overshot the mark” with EBM and created an “intellectual hegemony” that excludes important research methods from recognition (Berwick 2005). Berwick calls the overlooked methods “pragmatic science” and sees them as crucial for scientific discovery. He mentions the same sorts of approaches (use of local knowledge, exploration of hypotheses) that “translational medicine” advocates describe. After the discussions in this paper, some reasons for the recent turn to translational medicine will become clearer. There is a vast literature on evidence-based medicine, most consisting of systematic evidence reviews for particular health care questions. A substantial portion of the literature, however, is a critical engagement with EBM as a whole, pointing out both difficulties and limitations. These discussions come from outsiders as well as insiders to the field of EBM. My goal in this paper is to do something like a systematic review of this literature, discerning the kinds of criticisms that seem cogent and presenting them in a structured manner. EBM, like all methodologies in medicine, has both core strengths and limitations. I will begin with an overview of some general social and philosophical characteristics of EBM, and then turn to the criticisms.

2 EBM as a “Kuhnian paradigm”

When the Evidence-Based Medicine Working Group described themselves as having a “new paradigm” of medical knowledge (Evidence-Based Medicine Working Group 1992), they particularly had in mind Kuhn’s(1962) characterization of a paradigm as setting the standards for what is to count as admissible evidence.5 EBM assessments make use of an “evidence hierarchy” (often called “levels of evidence”) in which higher levels of evidence are regarded as of higher quality than lower levels of evidence. A typical evidence hierarchy6 puts double-masked (or “double-blinded”) RCTs at the top, or perhaps right after meta-analyses or systematic reviews of RCTs. Unmasked RCTs come next, followed by well designed case controlled or cohort studies and then observational studies and case reports. Expert opinion, expert consensus, clinical experience and physiological rationale are at the bottom. The rationale for the evidence hierarchy is that higher levels of evidence are thought to avoid biases that are present in the lower levels of evidence. Specifically, randomization avoids selection and other confounding biases (but see (Worrall 2007b)) and masking helps to distinguish real from placebo effects (but see (Howick 2008; Howick 2011)) and avoid confirmation bias. Powering the trial with sufficient numbers of participants and using statistical tools avoids the salience and availability

5 They were not, however, claiming along with many Kuhnians that there is subjectivity or relativity involved in what gets to count as evidence. 6 There are many such hierarchies in use, but all put double-masked RCTs at the top, or right after meta- analyses of RCTs, and clinical experience, expert consensus and physiological rationale at the bottom. Euro Jnl Phil Sci (2011) 1:451–466 455 biases that can skew informal assessments and unsystematic clinical experience. Ultimately, trial results are graded for both quality and strength of evidence. The language of Kuhnian paradigms has been overused and become somewhat clichéd, meaning something like “transformational new theory” in the typical quote from the EBM Working Group cited in the previous paragraph. In fact, EBM has many characteristics of a traditional Kuhnian paradigm,7 having all three of the characteristics of a Kuhnian paradigm discerned by Margaret Masterman (Lakatos and Musgrave 1970) and agreed to by Kuhn (Kuhn et al. 2000). These characteristics8 are helpful for understanding the import of EBM. First, EBM is a social movement with associated institutions such as Evidence-Based Practice Centers, official collaborations, textbooks, courses and journals. It is also, secondly, a general philosophy of medicine, defining both the questions of interest and the appropriate evidence. It is seen as the central methodology of medicine by its practitioners and as an unwelcome politically dominant movement by its detractors (e.g. (Charlton and Miles 1998; Denny 1999)). And third (sometimes overlooked by those who use Kuhn’sterm“paradigm”), it is characterized by a core of technical results and successful exemplars that have been extended over time. Kuhn referred to such exemplars as “concrete puzzle solutions…employed as models or examples” (Kuhn 1970) and later as a “disciplinary matrix” including “symbolic generalization, models and exemplars” (Kuhn 1977). He regards this third meaning as the original and fundamental meaning of the term “paradigm” (Kuhn 1977). Contrary to appearances and self-presentation, this core of technical results is not produced by a general algorithm or set of precise methodological rules. One of the things that Kuhn emphasized about paradigms is that they are driven primarily by exemplars, and not by rules. He writes (Kuhn 1970) that exemplars are “one sort of element…the concrete puzzle-solutions employed as exemplars which can replace explicit rules as a basis for the solution of the remaining puzzles of normal science. Kuhn argued that this is significant because the rules are not the basis for the development of the science. Rather, Kuhn argues, less precise judgments about similarity of examples are used (Kuhn 1977). The medical RCT traces its beginning to A. Bradford Hill’s 1948 evaluation of streptomycin for tuberculosis (Doll et al. 1999). It was initially resisted by many physicians used to treating each patient individually, therapeutically and with confidence in treatment choice (Marks 1997). Nevertheless, important trials such as the polio vaccine field trial of 1954 and the 1955 evaluation of treatments for rheumatic fever helped bring the RCT into routine use (Meldrum 1998; Meldrum 2000). In 1970 the RCT achieved official status in the USA with inclusion in the new FDA requirements for pharmaceutical testing (Meldrum 2000). One of the most well-known early successful uses of RCTs was the 1980s international study of aspirin and streptokinase for the prevention of myocardial infarction. However, not

7 Interestingly, Sehon and Stanley (2003) argue that EBM should not be thought of as a Kuhnian paradigm, but, instead, as part of a Quinean holistic network of beliefs. My focus here is on the historic, rather than the popular meaning of “paradigm” and on what EBM is, not on what it should be. 8 There are many other characteristics of Kuhnian paradigms, such as the narrative of paradigm change, the emphasis on high-level theory, and the idea that paradigms replace one another, that do not apply to this case. I am not trying to apply Kuhn exhaustively, just to use some of his concepts where they may be explanatorily useful. 456 Euro Jnl Phil Sci (2011) 1:451–466 all early use of the RCT was straightforward or successful: the 1960s-1970s NCI randomized controlled trial of lumpectomy versus mastectomy for early stage breast cancer was not masked,9 yet it was highly regarded, and the attempt to conduct a Diet-Heart study in the 1960s was hampered and finally frustrated by the difficulties in implementing major changes in diet in one arm of the study (Marks 1997). This is an example of the finding that the methodology of the RCT does not readily apply to all the situations in which we might wish to use it. As Kuhn might put it, normal science is not a matter of simple repetition of the paradigm case; it requires minor or major tinkering, and sometimes ends in frustration (or what Kuhn would call an “anomaly”). There are also variations in the design and analyses of RCTs. For example, some trials do an “intention to treat” analysis, dropping no experimental subjects from the trial, even if they fail to go through the course of treatment, and some trials do a “per-protocol” analysis in which only patients who complete the trial are included in the final results. Some trails have a placebo in the control arm and some trials have an established treatment in the control arm. It is often said that design and evaluation of an RCT requires “judgment” (see for example (Rawlins 2008)); by this what is meant is that trials cannot be designed by a universal set of rules and that the design and evaluation of trials requires domain expertise, not only statistical expertise. For example, domain expertise is needed in order to design both the dosage and the intervention in the control arm, and domain expertise is needed in order to specify appropriate trial selection criteria. The same insights apply to systematic reviews and meta-analyses. The first systematic review is often identified as the Oxford Database of Perinatal Trials 1989 study of corticosteroids for fetal lung development; this study was the basis for the development of the Cochrane Collaboration in 1993 which has since then done over 3,000 systematic reviews. Other organizations producing systematic reviews include the Agency for Health Care Research and Quality (AHRQ) and its fourteen Evidence-Based Practice Centers and the American College of Physicians (ACP) Journal Club. All systematic reviews use evidence hierarchies, but there is some variation in the hierarchies in use. The RCT is always at the top or just below meta- analyses of RCTs, but there are variations in where other kinds of studies are ranked, and in whether or not animal trials, basic science and expert opinion are included. Hierarchical rank is just one measure of the quality of a trial, which needs to be considered together with other measures of quality such as how well the trial handles withdrawals and how well it is randomized and masked. In 2002 the AHRQ reported forty systems of rating in use, six of them within its own network of evidence-based practice centers (AHRQ 2002). The GRADE Working Group, established in 2000, is attempting to reach consensus on one system of rating the quality and strength of evidence (Guyatt et al. 2008). This is an ironic development, given that EBM intends to replace group judgment methods! Meta-analysis combines the results of several high-quality trials to get an overall measure of strength of evidence. It requires judgments about the similarity of trials for combination and the quality of evidence in each trial, as well as about the possibility of systematic bias in the evidence overall, for example due to publication

9 Double-masking is the standard for high quality RCTs. Euro Jnl Phil Sci (2011) 1:451–466 457 bias and pharmaceutical company support. Meta-analysis is a formal technique, but not an algorithmic one: judgments need to be made about trial quality (as with systematic reviews, use of an evidence hierarchy is part of the process) and similarity of trial endpoints or other aspects of studies. Different meta-analyses of the same data have produced different conclusions (Juni et al. 1999; Yank et al. 2007). Steven Goodman (2002) is concerned that the disagreement between meta-analyses, specifically in the case of mammography screening, represents a “crisis for EBM.” I think it is not so much a crisis as a reminder of the limits of EBM. The identification of EBM with a Kuhnian paradigm, useful though it is, should not be taken too scrupulously. Exemplars and judgments of similarity are important, but rules also play a role. Kuhnian claims about incommensurability between paradigms and the social constitution of objectivity are controversial here and would certainly be denied by practitioners of EBM.10 We have moved on from Kuhn’s ideas, revolutionary in the 1960s, but now built upon and transformed in more sophisticated ways.

3 Critical discussions of EBM

Critical discussions of EBM tend to focus on questioning the procedural necessity and sufficiency of the technical requirements (especially for the RCT), the reliability of EBM in practice, or on EBM’s explicit or implicit claims to be a general philosophy of medicine. I’ll examine these three areas in turn.

3.1 Criticisms of procedural aspects of EBM

Many of these criticisms of EBM procedures have come from British philosophers of science associated with the London School of Economics. Their main approach is to argue that the “gold standard” (the double-masked RCT) is neither necessary (Howick 2011; Worrall 2007b) nor sufficient (Cartwright 2010) for clinical research. They argue that RCTs do not always control for the biases they are intended to control, they do not produce reliably generalizable knowledge, or they can be unnecessary constraints on clinical testing. These arguments are theoretical and abstract in character, although they are sometimes illustrated by examples. I distinguish them from arguments that RCTs have difficulties in practice, that is, from evaluation of RCTs based on the actual outcomes of such studies, which will be discussed in the next subsection (3B). John Worrall (2002, 2007a, 2007b) argues that randomization is just one way, and an imperfect way, of controlling for confounding factors that might produce bias. The problem is that randomization can control only for most but not for all confounding factors. When there are indefinitely many factors, both known and unknown, which may lead to bias, chances are that any one randomization will not randomize with respect to all these factors. Under these circumstances, Worrall concludes, chances are that any particular clinical trial will have at least one kind of

10 Critics of EBM have occasionally presented EBM as a political and rhetorical movement, e.g. (Charlton and Miles 1998), emphasizing the ways in which it appears to lack rationality. 458 Euro Jnl Phil Sci (2011) 1:451–466 bias, making the experimental group relevantly different from the control group, just by accident. The only way to avoid this is to re-randomize and do another clinical trial, which may, again by chance, eliminate the first trial’s confounding bias but introduce another. Worrall concludes that the RCT does not yield reliable results unless it is repeated time and again, re-randomizing each time, and the results are aggregated and analyzed overall. This is practically speaking impossible. In context, Worrall is less worried about the reliability of RCTs than he is about the assumption that they are much more reliable—in a different epistemic class—than e.g. well- designed observational (“historically controlled”) studies in which there is no randomization. He is arguing that the RCT should be taken off its pedestal and that all trials can have inadvertent bias due to differences between the control and the experimental group. In a series of articles, Nancy Cartwright (2007a,b, 2009, 2010) points out that RCTs may have internal validity, but their external validity and hence their applicability to real world questions is dependent on the similarity of the test population and context to the population and context targeted by the intervention. For example, she cites the failure of the California class-size reduction program, which was based on the success of a RCT in Tennessee, as due to failure of external validity (Cartwright 2009). She does not give a medical example of actual failure of external validity (hence my classification of her criticisms of EBM as theoretical in character), although she gives one of possible failure: prophylactic antibiotic treatment of children with HIV in developing countries. UNAIDS and UNICEF 2005 treatment recommendations were based on the results of a 2004 RCT in Zaire. Cartwright is concerned that the Zaire results will not generalize to resource-poor settings across other countries in sub-Saharan Africa (Cartwright 2007b) Concern about external validity is reasonable and there are classic medical examples of lack of external validity. For example, some recommendations for the treatment of heart disease, developed in trials of men only, do not apply to women. There is a history of challenges to RCTs on the grounds that they have excluded certain groups from participation (e.g. women, the elderly, children) yet are used for general health recommendations. The exclusions are made on epistemic and/or ethical grounds. These days, women are less likely to be excluded because the NIH and other granting organizations require their participation in almost all clinical trials, but other exclusions, such as those based on age, remain. Cartwright expresses the concern about external validity in its most general form. In her most recent work (Cartwright 2010) she describes four conditions that need to be met for external validity.11 These four conditions demand considerable domain knowledge i.e. knowledge of the particular causal interactions that the intervention relies upon. Jeremy Howick (2008, 2011) argues that masking is not useful outside of contexts in which outcomes are measured subjectively and that masking is both impossible and unnecessary when dealing with large effect size. It is impossible when dealing with large effect size because the effects of the drug unmask the assignment. He also argues that masking is in practice inadequate for placebo controlled trials, since

11 Cartwright’s(2010) four conditions are knowledge of “Roman laws“(laws that are general enough), “the right support team” (all necessary conditions), “straight sturdy ladders” (for climbing up and down levels of abstraction) and “unbroken bridges” (no interfering conditions). Euro Jnl Phil Sci (2011) 1:451–466 459 participants can usually tell through the presence of side effects whether or not they are receiving an active intervention. These three sets of criticisms by Worrall, Cartwright and Howick are significant, and I evaluate them next, beginning with Worrall’s papers. Randomization is unlikely to control for confounding factors only in the event that there are many unrelated population variables that influence outcome, because only in that complex case is one of those variables likely to be accidentally unbalanced by the randomization. Worrall considers only the abstract possibility of multiple unknown variables; he does not consider the likely relationship (correlation) of those variables with one another and he does not give us reason to think that, in practice, randomization generally (rather than rarely) leaves some causally relevant population variables accidentally selected for and thereby able to bias the outcome.12 In addition, successful replication adds evidence that any confounder inadvertently introduced is not causally responsible for the outcome. Cartwright uses examples from education, economics and international development to show lack of external validity. In general, her examples show failures of interventions to generalize, often because of cultural differences between populations. External validity in medical trials is more explored territory. We typically already know, from some of the trial selection criteria, where the controversies about generalization lie (see also Table 2 in Rawlins (2008), which sets out the problems with generalization). This does not mean that we can figure out the domain of application of trial results in a simple or formulaic manner. Domain expertise is essential for projection, as of course Nelson Goodman argued long ago (1955). Cartwright’sfourconditions(2010)havea role here as non-formal criteria for assessing external validity. It should also be noted that Cartwright’s discussion applies broadly to experimental and evidential reasoning and not specifically to trial methodology. It is not a specific criticism of RCTs, although because of what she calls the “vanity of rigor” of EBM (Cartwright 2007a), the criticism is especially pertinent to RCTs. Howick is correct that masking is important only for detecting small effects with subjectively measured outcomes, but this is (unfortunately) true of many recent advances in medicine and therefore widely applicable. It is not often that we have a new intervention with the dramatic success of e.g. insulin for diabetes or surgery for acute appendicitis. Howick is right to see that the methodology of the RCT is suited for some interventions and not suited for others, but the methodology is, in fact, suited for many if not most of the health care interventions currently in development. The approaches used by Worrall, Cartwright and Howick argue that the RCT is neither a necessary nor a sufficient method for getting knowledge from clinical trials. They argue that other methods can be equally or more effective in specific circumstances and that knowledge from trials always involves projection to untested domains. EBM enthusiasts are beginning to acknowledge this sensible moderation of their views, as the recent Harveian Oration by Sir Michael Rawlins13 shows (Rawlins 2008). In this paper, Rawlins argues that “the notion that evidence can be

12 Worrall might respond that he is only criticizing those methodologists who make abstract and general claims about freedom from “all possible” biases. This is fine, so long as no conclusions are drawn for RCTs in practice. 13 Michael Rawlins is the head of NICE (National Institute of Health and Clinical Excellence) in the UK, which bases its policies and guidelines on the results of EBM. 460 Euro Jnl Phil Sci (2011) 1:451–466 reliably placed in hierarchies is illusory,” that “striking effects can be discerned without the need for RCTs” and that the findings of RCTs should be extrapolated with caution. Granting these qualifications puts EBM in the same category as other successful scientific methodologies. They are useful tools in the domains in which they work, but they do not work everywhere or always.

3.2 Effectiveness of EBM methods in practice

How reliable is EBM in practice? RCT and meta-analyses generate claims with stated confidence levels. Typically, RCTs give 95% confidence levels and meta- analyses much higher confidence levels. It follows that each RCT has a 5% chance of producing a false positive (and each meta-analysis much less). Yet, in practice, RCTs and meta-analyses are much more fallible. Ioannidis (2005) did a study of 59 highly cited original research studies. Less than half (44%) were replicated; 16% were contradicted by subsequent studies and 16% found the effect to be smaller than in the original study; the rest were not repeated or challenged. Another, more well known statistic is that studies funded by pharmaceutical companies—even when properly masked and of highest quality— have an astonishingly higher chance (three or four times the probability of studies not funded by pharmaceutical companies) of showing effectiveness of an intervention than studies not funded by pharmaceutical companies (Als-Nielsen et al. 2003; Bekelman et al. 2003; Bero et al. 2007; Lexchin et al. 2003). And LeLorier et al. (LeLorier et al. 1997) found that 35% of the time, the outcome of RCTs is not predicted accurately by previous meta-analysis. This is a large and partly unexplained failure rate. Some suggest that factors such as publication bias, time to publication bias and pharmaceutical funding bias (which subtly affects trial design and evaluation) are responsible for the worse-than- expected track record of RCTs, systematic reviews and meta-analyses. Publication bias occurs when studies with null or negative results14 are not written up or not accepted for publication because they are wrongly thought to be of less scientific significance. Steps to address this bias have been taken in many areas of medical research by creating trial registries and making the results of all trials public. Time to publication bias is a more recently discovered phenomenon: trials with null or negative results, even when they are published, take much longer than trials with positive results (6–8 years for null or negative results compared with 4–5 years for positive results) (Hopewell et al. 2007). It is possible that steps taken to correct for publication bias will also help correct for time to publication bias. The additional bias created by pharmaceutical funding is not fully understood, especially since many of these trials are properly randomized and double-masked and satisfy rigorous methodological criteria. Some suggest that pharmaceutical companies deliberately select a weak control arm, for example by selecting a low dose of the standard treatment, giving the new drug a greater chance of relative

14 In this context, a positive trial is one in which the experimental arm of the trial is more effective, a null result is one in which both arms are equally effective, and a negative trial is one in which the control arm is more effective. Euro Jnl Phil Sci (2011) 1:451–466 461 success. There can also be biases that enter into the analysis of data, particularly when endpoints are not specified in advance. It is hoped that a clinical trials registry will help correct for publication bias and ex post facto manipulation of endpoints. However, at present we are a long way from correcting for bias created by pharmaceutical funding.15 Disclosure of funding source is helpful for evaluation, but often this information is lost in systematic review and meta-analysis. Since the performance of RCTs is so flawed, it is worth asking the question whether other kinds of clinical trials, further down the evidence hierarchy, are even less reliable. This would be expected in the abstract, since the further down the evidence hierarchy, the more possible sources of bias. Studies by Benson and Hartz (2000) and Concato et al. (2000) find that many well-designed observational studies produce the same results as RCTs.16 The matter is controversial, but a recent article by Ian Shrier et al. strongly argues for the inclusion of observational studies in systematic reviews and meta-analyses (Shrier et al. 2007) This result corroborates the intervention of early AIDS activists, who argued against the imposition of RCTs for AZT on both ethical and epistemic grounds (Epstein 1996). They argued that such trials are morally objectionable in that they deprive the individuals in the placebo arm of the only hope for a cure (at that time). And they argued that such trials are epistemically unnecessary because an RCT is not the only way to discern the effectiveness of anti-retroviral drugs—a claim that, in hindsight, has proved correct as a combination of historical controlled trials and laboratory studies have provided the knowledge of dramatically effective anti- retrovirals in clinical use today. These days, of course, no-one needs to get a placebo, and RCTs can continue to detect small improvements of protocol without such strenuous moral objections. Finally, EBM has been asked to evaluate itself using its own standards of evaluation This would involve showing not merely that a specific EBM intervention improves outcomes, but that more general use of systematic evidence reviews and so forth in clinical decision making results in improved outcomes for patients. In theory, we would of course expect improved outcomes. But what matters here is not theory but practice, and no-one has yet designed or carried out a study to test this (Charlton and Miles 1998; Cohen et al. 2004; Straus and McAlister 2000).

3.3 Criticisms of EBM as a general philosophy of medicine

Like most paradigms (new ways of knowing) the light shone on the paradigm flatters it and puts everything else into the shadows. Critics have protested, variously, that EBM overlooks the role of clinical experience, expert judgment, intuition, medical authority, patient goals and values, local health care constraints and the basic medical sciences including the structure of theory and the relations of causation. This is a long and complex list of intertwined scientific, hermeneutic, political and ethical considerations. Perhaps the most common criticism of EBM is that it deals with

15 I recommend that in the meantime we correct for funding bias by asking for a higher level of significance from the results of trials funded by pharmaceutical companies. 16 An editorial in the same issue of NEJM (Pocock and Elbourne 2000) strongly protests these conclusions, partly in the name of EBM orthodoxy, but partly also on the basis of some well known RCTs which contradicted the results of observational trials 462 Euro Jnl Phil Sci (2011) 1:451–466 statistical results, and application of those results to particular cases is said to require a different set of skills (e.g. (Cohen et al. 2004; Feinstein and Horwitz 1997; Hampton 2002; Straus and McAlister 2000; Tonelli 1998)). EBM advocates dispute this, and this is the reason for the early redefinition of the enterprise: “Evidence- based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients” (Sackett et al. 1996). Sorting out this complicated dispute is beyond the scope of this paper; I do so elsewhere (Solomon, in progress). Another common complaint, which covers several specific complaints, is that EBM is a scientific approach that overlooks the “art” of medicine (e.g. (Montgomery 2006; Tonelli 1998). Elsewhere I have argued that the traditional dichotomy between the “art” and the “science” of medicine is no longer helpful (Solomon 2008). This paper focuses on what EBM leaves out, rather than on whether to count that as art or as science. In this subsection I will focus on the persistent criticism that EBM ignores the basic sciences that guide both research and clinical practice (Ashcroft 2004; Bluhm 2005; Charlton and Miles 1998; Cohen et al. 2004; Harari 2001; Tonelli 1998). Basic sciences guide research in suggesting hypotheses about disease processes and mechanisms for action of interventions. Basic sciences guide clinical practice in helping physicians tailor the results of epidemiological studies to the needs of particular patients, who may have unique physiological and pathological conditions. EBM is scientifically superficial: it measures correlations. EBM does not model or theorize about the complete organism, still less the complete organism in its social and environmental context. In terms of scientific theory, it is thin; what some have called “empiricistic” (Harari 2001).17 Charlton and Miles (1998) claim that it is “statistical rather than scientific.” Ashcroft writes that EBM is “autonomous of the basic sciences” and “blind to mechanisms of explanation and causation” (2004, p. 134). Ashcroft regards this as an advantage, rather than a disadvantage, because it means that EBM does not have to worry that our basic theories may be incorrect. Ashcroft allies himself with Nancy Cartwright’s realism about phenomenal laws and antirealism about deeper laws18 at this point. Others, however, see the eschewing of scientific theorizing in favor of discovery of robust statistical correlations as problematic (Harari 2001; Tonelli 2006). They consider theorizing as important to medicine as it is to the pure sciences. Whatever one’s views about scientific realism, EBM typically depends upon a background of basic science research that develops the interventions and suggests the appropriate protocols. It is rare for an intervention without physiological rationale to be tested (although this does happen, especially in the areas of complementary and alternative medicine, but in these cases there is typically an alternative rationale, perhaps in the frameworks of Asian metaphysics). Moreover, as discussed above, scientific judgment enters into the design of appropriate randomized controlled trials (choice of control, test population etc.) and into the interpretation of the applicability of results (external validity). Of course, many interventions with excellent physiological rationales and good in vitro and in vivo performance fail when tested in human beings or fail when tested for external

17 In fact, EBM is the successor to ancient “empiric” approaches to medicine. 18 Cartwright is, of course, a realist about underlying causal processes. Euro Jnl Phil Sci (2011) 1:451–466 463 validity. That does not mean that there is anything wrong with physiological reasoning or that we can use a more reliable method. The basic science work is fallible, but it is not dispensable. Even Nancy Cartwright (1989) would agree that we cannot replace physical theory with phenomenal laws alone. In the past 5 years a new approach to medical research has risen to prominence internationally: what is called “translational medicine,” to be achieved by creating research centers, as well as journals, conferences, training programs and so forth. The NIH has made it a priority in its “Roadmap” in 2004 and started offering Clinical and Translational Science Awards in 2006. 55 Institutes have been created (as of May 2011), mostly in universities and medical centers, and the NIH hopes to fund 60, at a total cost of $500 million annually. The European Commission plans to use most of its billion Euro a year budget for the next few years for translational research. In the UK, the National Institute for Health Research has established 11 centers at a total cost of about 100 million pounds annually. The idea behind translational medicine is to facilitate greater interaction between basic science research and research in clinical medicine.19 The buzzwords are “synergize,” “catalyze” and “interdisciplinary.” The idea is to bring the different researchers and their laboratories into greater physical proximity. This is an interesting retro- intervention in these days of global electronic communication and global travel. From the perspective of the discussion in this section, the development of translational medicine or something like it was only a matter of time. EBM has such high claims to scientific objectivity that it attracted much talent and effort from clinical researchers. Perhaps the increased focus on formal epidemiological work eventually made apparent what was left out, namely engagement with substantial physiological and biomolecular theories. The model of basic science doing the research and clinical researchers testing the products is now perceived as limited; actually, it leaves all the fun and the creativity to the basic researchers, and deprives them of the input of clinical knowledge and observations from the clinical researchers.

4 Conclusions

EBM gives a set of formal techniques for evaluating the effectiveness of clinical interventions. The techniques are powerful, especially when evaluating interventions that offer incremental advantages to current standards of care, and especially when the determination of success has subjective elements. EBM techniques do not deliver the reliability that is theoretically and statistically expected from them. Results are compromised by publication bias, time to publication bias, interests of funding organizations and other unknown factors. Maintaining a strict evidence hierarchy makes little sense when the actual reliability of “gold standard” evidence is so much less than the expected reliability. I recommend a more instrumental or pragmatic approach to EBM, in which any ranking of evidence is done by reference to the

19 Technically, “translational research” includes both the bench-to-bedside-and-back (T1) and the clinical research to everyday practice (T2) “translational blocks.” See (Woolf 2008). But most of the resources and rhetoric favor the former. 464 Euro Jnl Phil Sci (2011) 1:451–466 actual, rather than the theoretically expected, reliability of results. So, for example, RCTs and observational trials might be at the same level in the hierarchy (based on their comparable reliability in practice) and trials designed and funded by pharmaceutical companies a level below independent trials (irrespective of apparent trial rigor, based on their track record of biased outcomes). Emphasis on EBM has eclipsed other necessary research methods in medicine, even those methods necessary for its own development and application. With the recent emphasis on translational medicine, we are seeing a restoration of the recognition that clinical research requires an engagement with basic theory (e.g. physiological, genetic, biochemical) and a range of empirical techniques such as bedside observation, laboratory and animal studies. EBM works best when used in this pluralistic methodological context.

References

AHRQ. (2002). Systems to rate the strength of scientific evidence. Evidence Report/Technology Assessment 47, Als-Nielsen, B., Chen, W., Gluud, C., & Kjaergard, L. L. (2003). Association of funding and conclusions in randomized drug trials: A reflection of treatment effect or adverse events? JAMA: The Journal of the American Medical Association, 290(7), 921–928. Ashcroft, R. E. (2004). Current epistemological problems in evidence based medicine. Journal of Medical Ethics, 30(2), 131–135. Baratz, S. R., Goodman, C., & Council on Health Care Technology. (1990). Improving consensus development for health technology assessment: An international perspective. Washington: National Academy Press. Bekelman, J. E., Li, Y., & Gross, C. P. (2003). Scope and impact of financial conflicts of interest in biomedical research: A systematic review. JAMA: The Journal of the American Medical Association, 289(4), 454–465. Benson, K., & Hartz, A. J. (2000). A comparison of observational studies and randomized, controlled trials. The New England Journal of Medicine, 342(25), 1878–1886. Bero, L., Oostvogel, F., Bacchetti, P., & Lee, K. (2007). Factors associated with findings of published trials of drug- drug comparisons: Why some statins appear more efficacious than others. PLoS Medicine, 4(6), e184. Berwick, D. M. (2005). Broadening the view of evidence-based medicine. Qual. Saf. Health Care, 14(5), 315–316. Bluhm, R. (2005). From hierachy to network: a richer view of evidence for evidence-based medicine. Perspectives in Biology and Medicine, 48(4), 535–547. Canadian Task Force on Preventive Health Care. CTFPHC History/Methodology. Retrieved 07/07, 2009, from http://www.ctfphc.org Cartwright, N. (1989). Nature's capacities and their measurement. Oxford; New York: Clarendon Press; Oxford University Press. Cartwright, N. (2007a). Are RCTs the gold standard? Biosocieties, 2(2), 11–20. Cartwright, N. (2007b). Evidence-based policy: Where is our theory of evidence? Center for Philosophy of Natural and Social Science, London School of Economics, Technical Report 07/07, Cartwright, N. (2009). Evidence-based policy: What's to be done about relevance. Philosophical Studies, 143(1), 127–136. Cartwright, N. (2010). Will this policy work for you? predicting effectiveness better: How philosophy helps. Unpublished manuscript. Charlton, B. G., & Miles, A. (1998). The rise and fall of EBM. Quarterly Journal of Medicine, 91(5), 371- 371-374. Cohen, A. M., Stavri, P. S., & Hersh, W. R. (2004). A categorization and analysis of the criticisms of evidence-based medicine. International Journal of Medical Informatics, 73(1), 35-35-43. Concato, J., Shah, N., & Horwitz, R. I. (2000). Randomized, controlled trials, observational studies, and the hierarchy of research designs. The New England Journal of Medicine, 342(25), 1887–1892. Euro Jnl Phil Sci (2011) 1:451–466 465

Daly, J. (2005). Evidence-based medicine and the search for a science of clinical care. Berkeley; New York: University of California Press; Milbank Memorial Fund. Davidoff, F., Haynes, B., Sackett, D., & Smith, R. (1995). Evidence based medicine. BMJ (Clinical Research Ed.), 310(6987), 1085–1086. Denny, K. (1999). Evidence-based medicine and medical authority. Journal of Medical Humanities, 20(4), 247–263. Doll, R., Peto, R., & Clarke, M. (1999). First publication of an individually randomized trial. Controlled Clinical Trials, 20(4), 367–368. Elstein, A. S. (2004). On the origins and development of evidence-based medicine and medical decision making. Inflammation Research, 53(Supplement 2), S184-S184-S189. Epstein, S. (1996). Impure science: AIDS, activism, and the politics of knowledge. Berkeley: University of California Press. Evidence-Based Medicine Working Group. (1992). Evidence-Based Medicine. A new approach to teaching the practice of medicine. JAMA: The Journal of the American Medical Association, 268(17), 2420–2425. Feinstein, A. R., & Horwitz, R. I. (1997). Problems in the “evidence” of “evidence-based medicine”. The American Journal of Medicine, 103(6), 529–535. Goodman, N. (1955). Fact, fiction and forecast. Cambridge: Harvard University Press. Goodman, S. N. (2002). The mammography dilemma: A crisis for evidence-based medicine? Annals of Internal Medicine, 137(5 Part 1), 363–365. Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., et al. (2008). GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ (Clinical Research Ed.), 336(7650), 924–926. Hampton, J. R. (2002). Evidence-based medicine, opinion-based medicine, and real-world medicine. Perspectives in Biology and Medicine, 45(4), 549–568. Harari, E. (2001). Whose evidence? lessons from the philosophy of science and the epistemology of medicine. Australian and New Zealand Journal of Psychiatry, 35(6), 724-724-730. Hopewell, S., Clarke, M. J., Stewart, L., & Tierney, J. (2007). Time to publication for results of clinical trials. Cochrane Database of Systematic Reviews, (1) doi:10.1002/14651858.MR000011.pub2 Howick, J. (2008). Against a priori judgments of bad methodology: Questioning double-blinding as a universal methodological virtue of clinical trials. PhilSci Archive Howick, J. (2011). The philosophy of evidence-based medicine Wiley-Backwell. Ioannidis, J. P. (2005). Contradicted and initially stronger effects in highly cited clinical research. JAMA: The Journal of the American Medical Association, 294(2), 218–228. Juni, P., Witschi, A., Bloch, R., & Egger, M. (1999). The hazards of scoring the quality of clinical trials for meta-analysis. JAMA: The Journal of the American Medical Association, 282(11), 1054–1060. Kuhn, T. S. (1962). The structure of scientific revolutions. Chicago: University of Chicago Press. Kuhn, T. S. (1970). The structure of scientific revolutions (2nd ed.). Chicago: University of Chicago Press. Kuhn, T. S. (1977). The essential tension: Selected studies in scientific tradition and change. Chicago: University of Chicago Press. Kuhn, T. S., Conant, J., & Haugeland, J. (2000). The road since structure: Philosophical essays, 1970– 1993, with an autobiographical interview. Chicago: University of Chicago Press. Lakatos, I., & Musgrave, A. (1970). Criticism and the growth of knowledge. Cambridge: University Press. LeLorier, J., Gregoire, G., Benhaddad, A., Lapierre, J., & Derderian, F. (1997). Discrepancies between meta-analyses and subsequent large randomized, controlled trials. The New England Journal of Medicine, 337(8), 536–542. Lexchin, J., Bero, L. A., Djulbegovic, B., & Clark, O. (2003). Pharmaceutical industry sponsorship and research outcome and quality: Systematic review. BMJ (Clinical Research Ed.), 326(7400), 1167– 1170. Marks, H. M. (1997). The progress of experiment: Science and therapeutic reform in the united states, 1900–1990. Cambridge: Cambridge University Press. Meldrum, M. L. (1998). A calculated risk: The salk polio vaccine field trials of 1954. British Medical Journal, 317, 1233-1233-6. Meldrum, M. L. (2000). A brief history of the randomized controlled trial: From oranges and lemons to the gold standard. Hematology/Oncology Clinics of North America, 14(4), 745-745-760. Montgomery, K. (2006). How doctors think: Clinical judgment and the practice of medicine. Oxford: Oxford University Press. Pocock, S. J., & Elbourne, D. R. (2000). Randomized trials or observational tribulations? The New England Journal of Medicine, 342(25), 1907–1909. 466 Euro Jnl Phil Sci (2011) 1:451–466

Rawlins, M. (2008). De testimonio: On the evidence for decisions about the use of therapeutic interventions. Lancet, 372(9656), 2152–2161. Sackett, D. L., Rosenberg, W. M., Gray, J. A., Haynes, R. B., & Richardson, W. S. (1996). Evidence based medicine: What it is and what it isn't. BMJ (Clinical Research Ed.), 312(7023), 71–72. Sehon, S. R., & Stanley, D. E. (2003). A philosophical analysis of the evidence-based medicine debate. BMC Health Services Research, 3(14). Shrier, I., Boivin, J. F., Steele, R. J., Platt, R. W., Furlan, A., Kakuma, R., et al. (2007). Should meta- analyses of interventions include observational studies in addition to randomized controlled trials? A critical examination of underlying principles. American Journal of Epidemiology, 166(10), 1203– 1209. Solomon, M. (2008). Epistemological reflections on the art of medicine and narrative medicine. Perspectives in Biology and Medicine, 51(3), 406–417. Solomon, M. (in progress). Beyond the art and science of medicine. Unpublished manuscript. Straus, S. E., & McAlister, F. A. (2000). Evidence-based medicine: A commentary on common criticisms. Canadian Medical Association Journal, 163(7), 837–841. Tonelli, M. R. (1998). The philosophical limits of evidence-based medicine. Academic Medicine, 73(12), 1234–1240. Tonelli, M. R. (2006). Integrating evidence into clinical practice: An alternative to evidence-based approaches. Journal of Evaluation in Clinical Practice, 12(3), 248–256. Woolf, S. H. (2008). The meaning of translational research and why it matters. JAMA: The Journal of the American Medical Association, 299(2), 211–213. Worrall, J. (2002). What evidence in evidence-based medicine? Philosophy of Science, 69(S3), 316–330. Worrall, J. (2007a). Evidence in medicine and evidence-based medicine. Philosophy Compass, 2(6), 981– 1022. Worrall, J. (2007b). Why there's no cause to randomize. The British Journal for the Philosophy of Science, 58(3), 451–488. Yank, V., Rennie, D., & Bero, L. A. (2007). Financial ties and concordance between results and conclusions in meta-analyses: Retrospective cohort study. BMJ (Clinical Research Ed.), 335(7631), 1202–1205.

Evidence-Based Medicine A New Approach to Teaching the Practice of Medicine Evidence-Based Medicine Working Group

A NEW paradigm for medical practice dose of phenytoin intravenously and the year is between 43% and 51%, and at 3 is emerging. Evidence-based medicine drug is continued orally. A computed years the risk is between 51% and 60%. de-emphasizes intuition, unsystematic tomographic head scan is completely nor¬ After a seizure-free period of 18 months clinical experience, and pathophysiolog- mal, and an electroencephalogram shows his risk of recurrence would likely be ic rationale as sufficient grounds for clin- only nonspecific findings. The patient is less than 20%. She conveys this infor¬ ical decision making and stresses the very concerned about his risk of seizure mation to the patient, along with a rec¬ examination of evidence from clinical re- recurrence. How might the resident ommendation that he take his medica¬ search. Evidence-based medicine re- proceed? tion, see his family doctor regularly, and quires new skills of the physician, in- have a review of his need for medication cluding efficient literature searching and The Way of the Past ifhe remains seizure-free for 18 months. the application of formal rules of evi- Faced with this situation as a clinical The patient leaves with a clear idea of dence evaluating the clinical literature. clerk, the resident was told by her se¬ his likely prognosis. An important goal of our medical res- nior resident (who was supported in his idency program is to educate physicians view by the attending physician) that A PARADIGM SHIFT in the practice of evidence-based med- the risk of seizure recurrence is high Thomas Kuhn has described scientific icine. Strategies include a weekly, for- (though he could not put an exact num¬ paradigms as ways of looking at the mal academic half-day for residents, de- ber on it) and that was the information world that define both the problems that voted to learning the necessary skills; that should be conveyed to the patient. can legitimately be addressed and the recruitment into teaching roles of phy- She now follows this path, emphasizing range of admissible evidence that may sicians who practice evidence-based to the patient not to drive, to continue bear on their solution.4 When defects in medicine; sharing among faculty of ap- his medication, and to see his family an existing paradigm accumulate to the proaches to teaching evidence-based physician in follow-up. The patient leaves extent that the paradigm is no longer medicine; and providing faculty with in a state of vague trepidation about his tenable, the paradigm is challenged and feedback on their performance as role risk of subsequent seizure. replaced by a new way of looking at the models and teachers of evidence-based world. Medical practice is changing, and medicine. The influence of evidence- The Way of the Future the change, which involves using the based medicine on clinical practice and The resident asks herselfwhether she medical literature more effectively in medical education is increasing. knows the prognosis of a first seizure guiding medical practice, is profound and realizes she does not. She that it can be called CLINICAL SCENARIO proceeds enough appropriately to the library and, using the Grateful a paradigm shift. A junior medical resident working in Med program,1 conducts a computerized The foundations of the paradigm shift a teaching hospital admits a 43-year-old literature search. She enters the Med¬ lie in developments in clinical research previously well man who experienced a ical Subject Headings terms epilepsy, over the last 30 years. In 1960, the ran¬ witnessed grand mal seizure. He had prognosis, and recurrence, and the pro¬ domized clinical trial was an oddity. It is never had a seizure before and had not gram retrieves 25 relevant articles. Sur¬ now accepted that virtually no drug can had any recent head trauma. He drank veying the titles, one2 appears directly enter clinical practice without a demon¬ alcohol once or twice a week and had not relevant. She reviews the paper, finds stration of its efficacy in clinical trials. had alcohol on the day of the seizure. that it meets criteria she has previously Moreover, the same randomized trial Findings on physical examination are learned for a valid investigation ofprog¬ method increasingly is being applied to normal. The patient is given a loading nosis,3 and determines that the results surgical therapies6 and diagnostic tests.6 are applicable to her patient. The search Meta-analysis is gaining increasing ac¬ costs the resident $2.68, and the entire ceptance as a method of summarizing the A complete list of members of the Evidenced-Based process (including the trip to the library results of a number of randomized trials, Medicine Working Group appears at the end of this ar- and the time to make a photocopy of the and ultimately may have as profound an ticle. article) took half an hour. effect on treatment as have Reprint requests to McMaster University Health Sci- setting policy ences Centre, Room 3W10,1200 Main St W, Hamilton, The results ofthe relevant study show randomized trials themselves.7 While Ontario, Canada L8N 3Z5 (Gordon Guyatt, MD). that the patient risk of recurrence at 1 less dramatic, crucial methodological ad-

Downloaded From: http://jama.jamanetwork.com/ by a University of Ottawa User on 10/02/2014 vanees have also been made in other ar¬ sense is sufficient to allow one to eval¬ uncertainty and to acknowledge that eas, such as the assessment of diagnostic uate new tests and treatments. management decisions are often made tests8·9 and prognosis.2 4. Content expertise and clinical ex¬ in the face of relative ignorance of their A new philosophy of medical practice perience are a sufficient base from which true impact. and teaching has followed these meth¬ to generate valid guidelines for clinical The new paradigm puts a much lower odological advances. This paradigm shift practice. value on authority.20 The underlying be¬ is manifested in a number of ways. A According to this paradigm clinicians lief is that physicians can gain the skills profusion of articles has been published have a number of options for sorting out to make independent assessments of ev¬ instructing clinicians on how to access,10 clinical problems they face. They can idence and thus evaluate the credibility evaluate,11 and interpret12 the medical reflect on their own clinical experience, of opinions being offered by experts. literature. Proposals to apply the prin¬ reflect on the underlying biology, go to The decreased emphasis on authority ciples of clinical epidemiology to day- a textbook, or ask a local expert. Read¬ does not imply a rejection of what one to-day clinical practice have been put ing the introduction and discussion sec¬ can learn from colleagues and teachers, forward.3 A number of major medical tions of a paper could be considered an whose years of experience have provid¬ journals have adopted a more informa¬ appropriate way of gaining the relevant ed them with insight into methods of tive structured abstract format, which information from a current journal. history taking, physical examination, and incorporates issues of methods and de¬ This paradigm puts a high value on diagnostic strategies. This knowledge sign into the portion of an article the traditional scientific authority and ad¬ can never be gained from formal scien¬ reader sees first.13 The American Col¬ herence to standard approaches, and an¬ tific investigation. A final assumption of lege of Physicians has launched a jour¬ swers are frequently sought from direct the new paradigm is that physicians nal, ACP Journal Club, that summa¬ contact with local experts or reference whose practice is based on an under¬ rizes new publications of high relevance to the writings ofinternational experts.19 standing ofthe underlying evidence will and methodological rigor.14 Textbooks care. The New provide superior patient that provide a rigorous review of avail¬ Paradigm REQUIREMENTS FOR THE able evidence, including a methods sec¬ The assumptions ofthe new paradigm PRACTICE OF EVIDENCE-BASED tion describing both the methodological are as follows: criteria used to systematically evaluate 1. Clinical experience and the devel¬ MEDICINE the validity of the clinical evidence and opment of clinical instincts (particularly The role modeling, practice, and teach¬ the quantitative techniques used for with respect to diagnosis) are a crucial ing of evidence-based medicine requires summarizing the evidence, have begun and necessary part of becoming a com¬ skills that are not traditionally part of to appear.1516 Practice guidelines based petent physician. Many aspects of clin¬ medical training. These include precise¬ on rigorous methodological review ofthe ical practice cannot, or will not, ever be ly defining a patient problem, and what available evidence are increasingly com¬ adequately tested. Clinical experience information is required to resolve the mon.17 A final manifestation is the grow¬ and its lessons are particularly impor¬ problem; conducting an efficient search ing demand for courses and seminars tant in these situations. At the same of the literature; selecting the best of that instruct physicians on how to make time, systematic attempts to record ob¬ the relevant studies and applying rules more effective use of the medical liter¬ servations in a reproducible and unbi¬ of evidence to determine their validity3; ature in their day-to-day patient care.3 ased fashion markedly increase the con¬ being able to present to colleagues in a We call the new paradigm "evidence- fidence one can have in knowledge about succinct fashion the content of the ar¬ based medicine."18 In this article, we de¬ patient prognosis, the value of diagnos¬ ticle and its strengths and weaknesses; scribe how this approach differs from tic tests, and the efficacy of treatment. and extracting the clinical message and prior practice and briefly outline how In the absence of systematic observa¬ applying it to the patient problem. We we are building a residency program in tion one must be cautious in the inter¬ will refer to this process as the critical which a key goal is to practice, act as a pretation of information derived from appraisal exercise. role model, teach, and help residents clinical experience and intuition, for it Evidence-based medicine also involves become highly adept in evidence-based may at times be misleading. applying traditional skills of medical medicine. We also describe some of the 2. The study and understanding of training. A sound understanding of problems educators and medical prac¬ basic mechanisms of disease are neces¬ pathophysiology is necessary to inter¬ titioners face in implementing the new sary but insufficient guides for clinical pret and apply the results of clinical re¬ paradigm. practice. The rationales for diagnosis and search. For instance, most patients to treatment, which follow from basic whom we would like to the The Former generalize Paradigm pathophysiologic principles, may in fact results of randomized trials would, for The former paradigm was based on be incorrect, leading to inaccurate pre¬ one reason or another, not have been the following assumptions about the dictions about the performance of diag¬ enrolled in the most relevant study. The knowledge required to guide clinical nostic tests and the efficacy of treat¬ patient may be too old, be too sick, have practice. ments. other underlying illnesses, or be unco¬ 1. Unsystematic observations from 3. Understanding certain rules of operative. Understanding the underly¬ clinical experience are a valid way of evidence is necessary to correctly in¬ ing pathophysiology allows the clinician building and maintaining one's knowl¬ terpret literature on causation, progno¬ to better judge whether the results are edge about patient prognosis, the value sis, diagnostic tests, and treatment applicable to the patient at hand and of diagnostic tests, and the efficacy of strategy. also has a crucial role as a conceptual treatment. It follows that clinicians should reg¬ and memory aid. 2. The study and understanding of ularly consult the original literature (and Another traditional skill required of basic mechanisms of disease and patho- be able to critically appraise the meth¬ the evidence-based physician is a sen¬ physiologic principles are a sufficient ods and results sections) in solving clin¬ sitivity to patients' emotional needs. Un¬ guide for clinical practice. ical problems and providing optimal pa¬ derstanding patients' suffering21 and how 3. A combination of thorough tradi¬ tient care. It also follows that clinicians that suffering can be ameliorated by the tional medical training and common must be ready to accept and live with caring and compassionate physician are

Downloaded From: http://jama.jamanetwork.com/ by a University of Ottawa User on 10/02/2014 Evaluation Form for Clinical Teaching Unit Attending Physicians Rating Domain Unsatisfactory Needs Improvement Satisfactory Good Excellent Role model of practice Seldom cites evidence Often fails to substantiate Usually substantiates Substantiates decisions; Always substantiates of evidence-based to support clinical decisions with evidence decisions with is aware of decisions or medicine decisions evidence methodological issues acknowledges limitations of evidence Leads practice of Never assigns problems Produces suboptimal volume Assigns problems and Discusses literature Same as "Good" rating, evidence-based to be resolved or follow-through of problem follows through with retrieval, methodology and makes it exciting medicine through literature resolution through literature discussion, including of papers, application and fun methodology to individual patient

fundamental requirements for medical articles and arriving at bottom lines re¬ ical teaching roles available to them. practice. These skills can be acquired garding the strength of evidence and Second, a program of more rigorous through careful observation of patients how it bears on the clinical problem. evaluation of attending physicians has and of physician role models. Here too, They learn to present the methods and been instituted. One of the areas eval¬ though, the need for systematic study results in a succinct fashion, emphasiz¬ uated is the extent to which attending and the limitations of the present evi¬ ing only the key points. A wide-ranging physicians are effective in teaching ev¬ dence must be considered. The new par¬ discussion, including issues of underly¬ idence-based medicine. The relevant adigm would call for using the techniques ing pathophysiology and related ques¬ items from the evaluation form are re¬ of behavioral science to determine what tions of diagnosis and management, fol¬ produced in the Table. patients are really looking for from their lows presentation of the articles. Third, because it is new to both teach¬ physicians22 and how physician and pa¬ The second part of the half-day is de¬ ers and learners, and because most clin¬ tient behavior affects the outcome of voted to the physical examination. Clin¬ ical teachers have observed few role care.23 Ultimately, randomized trials ical teachers present optimal techniques models and have not received formal using different strategies for interact¬ of examination with attention to what is training, teaching evidence-based med¬ ing with patients (such as the random¬ known about their reproducibility and icine is not easy. To help attending phy¬ ized trial conducted by Greenfield and accuracy. sicians improve their skills in this area, colleagues24 that demonstrated the pos¬ 2. Facilities for computerized litera¬ we have encouraged them to form part¬ itive effects of increasing patients' in¬ ture searching are available on the teach¬ nerships, which involve attending the volvement with their care) may be ing medical ward in each of the four partner's clinical rounds, making obser¬ appropriate. teaching hospitals. Costs of searching vations, and providing formal feedback. Since evidence-based medicine in¬ are absorbed by the residency program. One learns through observation and volves skills ofproblem defining, search¬ Residents not familiar with computer through criticisms of one's performance. ing, evaluating, and applying original searching, or the Grateful Med program A number of faculty members have par¬ medical literature, it is incumbent on we use, are instructed at the beginning ticipated in this program. residency programs to teach these skills. of the rotation. Research in our insti¬ To further facilitate attending physi¬ Understanding the barriers to educat¬ tution has shown that MEDLINE cians' improving their skills, the De¬ ing physicians-in-training in evidence- searching from clinical settings is fea¬ partment of Medicine held a retreat de¬ based medicine can lead to more effec¬ sible with brief training.26 A subsequent voted to sharing strategies for effective tive teaching strategies. investigation demonstrated that inter¬ clinical teaching. Part of the workshop, nal medicine house staff who have com¬ attended more than 30 faculty mem¬ EVIDENCE-BASED MEDICINE IN A by puter access on the ward and feedback was devoted to evidence- MEDICAL RESIDENCY bers, teaching concerning their searching do an aver¬ based medicine. Some of the strategies The Internal Medicine Residency Pro¬ age of more than 3.6 searches per that were adduced are briefly summa¬ gram at McMaster University has an month.26 House staff believe that more rized in the next section. explicit commitment to producing prac¬ than 90% oftheir searches that are stim¬ titioners of evidence-based medicine. ulated by a patient problem lead to some EFFECTIVE TEACHING OF While other clinical departments at improvement in patient care.25 EVIDENCE-BASED MEDICINE McMaster have devoted themselves to 3. Assessment of searching and crit¬ teaching evidence-based medicine, the ical appraisal skills is being incorporat¬ Role Modeling commitment is strongest in the Depart¬ ed into the evaluation of residents. Attending physicians must be enthu¬ ment of Medicine. We will therefore fo¬ 4. We believe that the new paradigm siastic, effective role models for the prac¬ cus on the Internal Medicine Residency will remain an academic mirage with tice of evidence-based medicine (even in in our discussion and briefly outline some little relation to the world of day-to-day high-pressure clinical settings, such as of the strategies we are using in imple¬ clinical practice unless physicians-in- intensive care units). Providing a model menting the paradigm shift. training are exposed to role models who goes a long way toward inculcating at¬ 1. The residents spend each Wednes¬ practice evidence-based medicine. As a titudes that lead learners to develop day afternoon at an academic half-day. result, the residency program has skills in critical appraisal. Acting as a At the beginning of each new academic placed major emphasis on ensuring this role model involves specifying the year, the rules of evidence that relate to exposure. strength of evidence that supports clin¬ articles concerning therapy, diagnosis, First, a focus of recruitment for our ical decisions. In one case, the teacher prognosis, and overviews are reviewed. Department of Medicine faculty has can point to a number of large random¬ In subsequent sessions, the discussion been internists with training in clinical ized trials, rigorously reviewed and in¬ is built around a clinical case, and two epidemiology. These individuals have the cluded in a meta-analysis, which allows original articles that bear on the prob¬ skills and commitment to practice evi¬ one to say how many patients one must lem are presented. The residents are dence-based medicine. The residency treat to prevent a death. In other cases, responsible for critically appraising the program works to ensure they have clin- the best evidence may come from ac-

Downloaded From: http://jama.jamanetwork.com/ by a University of Ottawa User on 10/02/2014 cepted practice or one's clinical experi¬ Diagnosis.—Has the diagnostic test Institutional experience can also pro¬ ence and instincts. The clinical teacher been evaluated in a patient sample that vide important insights. Diagnostic tests should make it clear to learners on what included an appropriate spectrum ofmild may differ in their accuracy depending basis decisions are being made. This can and severe, treated and untreated dis¬ on the skill of the practitioner. A local be done efficiently. For instance: ease, plus individuals with different but expert in, for instance, diagnostic ultra¬ confused disorders?28 sound Prospective studies suggest that Mr Jones' commonly Was may produce far better results there an blind than the risk of a major vascular event in the first independent, comparison average from the published lit¬ year after his infarct is 4%; a meta-analysis with a "gold standard" of diagnosis?28 erature. The effectiveness and compli¬ of randomized trials of aspirin in this situa¬ Treatment.—Was the assignment of cations associated with therapeutic in¬ tion suggests a risk reduction of 25%; we patients to treatments randomized?29 terventions, particularly surgical pro¬ such would have to treat 100 patients to pre¬ Were all patients who entered the study cedures, may also differ among institu¬ vent an event21; given the minimal expense accounted for at its conclusion?29 tions. When optimal care is taken to and toxicity of low-dose, enteric-coated as¬ Review Articles.—Were both record observations pirin, treating Mr Jones is clearly warranted. explicit reproducibly methods used to determine which arti¬ and avoid bias, clinical and institutional Or: cles to include in the review?30 experience evolves into the systematic How longto treat a patient with antibiotics fol¬ As learners become more sophisticat¬ search for knowledge that forms the core lowing pneumonia has not been systematically ed, additional criteria can be introduced. of evidence-based medicine.32 studied; so, my recommendation that we give The criteria should not be presented in Misinterpretation 2.—Understand¬ Mrs Smith 3 days of intravenous antibiotics such a way that fosters nihilism (if the ing ofbasic investigation and pathophys- and treat her for a total of 10 is days arbitrary; is not it is useless no in evidence-based somewhat shorter or courses of treat¬ study randomized, iology plays part longer and no valuable medicine. ment would be equally reasonable. provides information), but as a way of arrive at the Correction.—The dearth of In the latter of helping adequate type situation, dog¬ strength of inference associated with a evidence demands that clinical problem matic or insistence on a rigid following clinical decision. Teachers can point out solving must rely on an understanding course of action would not be particular instances in which criteria can be vio¬ of underlying pathophysiology. More¬ appropriate. lated without reducing the strength of over, a good understanding of patho¬ inference. is for Critical physiology necessary interpreting Appraisal clinical observations and for appropri¬ It is crucial that critical is¬ appraisal METHODS FOR SCALING THE ate interpretation of evidence (especial¬ sues arise from patient problems that ly in deciding on its generalizability). the learner is BARRIERS TO THE DISSEMINATION currently confronting, OF EVIDENCE-BASED MEDICINE Misinterpretation 3.—Evidence- demonstrating that critical appraisal is based medicine standard About ignores aspects a pragmatic and central aspect, not an Misapprehensions of clinical training, such as the physical academic or tangential element of op¬ Evidence-Based Medicine examination. timal patient care. The problem select¬ In developing the practice and teach¬ Correction.—Careful history taking ed for critical appraisal must be one that ing of evidence-based medicine at our and physical examination provide much, the learners recognize as important, feel institution, we have found that the na¬ and often the best, evidence for diag¬ uncertain, and do not fully trust expert ture of the new paradigm is sometimes nosis and direct treatment decisions. The opinion; in other words, they must feel misinterpreted. Recognizing the limita¬ clinical teacher of evidence-based med¬ it is worth the effort to find out what the tions of intuition, experience, and un¬ icine must give considerable attention literature says on a topic. The likeliest derstanding of pathophysiology in per¬ to teaching the methods of history tak¬ candidate topics are common problems mitting strong inferences may be mis¬ ing and clinical examination, with par¬ where learners have been exposed to interpreted as rejecting these routes to ticular attention to which items have divergent opinions (and thus there is knowledge. Specific misinterpretations demonstrated validity and to strategies disagreement and/or uncertainty among ofevidence-based medicine and their cor¬ that enhance observer agreement. the learners). The clinical teacher should rections follow: keep these requirements in mind when Misinterpretation 1.—Evidence- Barriers to Teaching considering questions to encourage the based medicine ignores clinical experi¬ Evidence-Based Medicine learners to address. It can be useful to ence and clinical intuition. Difficulties we have encountered in ask all members of the group their opin¬ Correction.—On the contrary, it is teaching evidence-based medicine in¬ ion about the clinical problem at hand. important to expose learners to excep¬ clude the following: One can then ensure that the problem is tional clinicians who have a gift for in¬ 1. Many house staff start with rudi¬ appropriate for a critical appraisal ex¬ tuitive diagnosis, a talent for precise mentary critical appraisal skills and the ercise by asking the group the following observation, and excellent judgment in topic may be threatening for them. questions: making difficult management decisions. 2. People like quick and easy answers. 1. It seems the group is uncertain Untested signs and symptoms should Cookbook medicine has its appeal. Crit¬ about the optimal approach. Is that not be rejected out of hand. They may ical appraisal involves additional time right? prove extremely useful and ultimately and effort and may be perceived as in¬ 2. Do you feel it is important for us to be proved valid through rigorous test¬ efficient and distracting from the real sort out this question by going to the ing. The more the experienced clinicians goal (to provide optimal care for pa¬ original literature? can dissect the process they use in di¬ tients). and it to learn¬ 3. For clinical Criteria agnosis,31 clearly present many questions, high- Methodological ers, the greater the benefit. Similarly, quality evidence is lacking. If such ques¬ Criteria for methodological rigor must the gain for students will be greatest tions predominate in attempts to intro¬ be few and simple. Most published crite¬ when clues to optimal diagnosis and duce critical appraisal, a sense offutility ria can be overwhelming for the novice. treatment are culled from the barrage can result. Suggested criteria for studies of diagno¬ of clinical information in a systematic 4. The concepts of evidence-based sis, treatment, and review articles follow: and reproducible fashion. medicine are met with skepticism by

Downloaded From: http://jama.jamanetwork.com/ by a University of Ottawa User on 10/02/2014 many faculty members who are there¬ views meeting scientific principles30,33 tient outcomes may appear to be an in¬ fore unenthusiastic about modifying and collections ofmethodologically sound ternal contradiction. As has been point¬ their teaching and practice in accordance and highly relevant articles14 can mark¬ ed out, however, evidence-based with its dictates. edly increase efficiency. Other solutions medicine does not advocate a rejection These problems can be ameliorated will emerge over time. Health educa¬ of all innovations in the absence of de¬ by use of the strategies described in the tors will continue to find better ways of finitive evidence. When definitive evi¬ previous section on effective teaching of role modeling and teaching evidence- dence is not available, one must fall back evidence-based medicine. Threat can be based medicine. Standards in writing on weaker evidence (such as the com¬ reduced by making a contract with the reviews and texts are likely to change, parison of graduates of two medical residents, which sets out modest and with a greater focus on methodological schools that use different approaches achievable goals, and further reduced rigor.15·16 Evidence-based summaries will cited above) and on biologic rationale. by the attending physician role model¬ therefore become increasingly available. The rationale in this case is that physi¬ ing the practice of evidence-based med¬ Practical approaches to making evi¬ cians who are up-to-date as a function of icine. Inefficiency can be reduced by dence-based summaries easier to apply their ability to read the current litera¬ teaching effective searching skills and in clinical practice, many based on com¬ ture critically, and are able to distin¬ simple guidelines for assessing the va¬ puter technology, will be developed and guish strong from weaker evidence, are lidity of the papers. In addition, one can expanded. As described earlier, we are likely to be more judicious in the ther¬ emphasize that critical appraisal as a already using computer searching on the apy they recommend. Physicians who strategy for solving clinical problems is ward. In the future, the results of di¬ understand the properties of diagnostic most appropriate when the problems are agnostic tests may be provided with the tests and are able to use a quantitative common in one's own practice. Futility associated sensitivity, specificity, and approach to those tests are likely to make can be reduced by, particularly initially, likelihood ratios. Health policymakers more accurate diagnoses. While this ra¬ targeting critical appraisal exercises to may find that the structure of medical tionale appears compelling to us, com¬ areas in which there is likely to be high- practice must be shifted in basic ways to pelling rationale has often proved mis¬ quality evidence that will affect clinical facilitate the practice of evidence-based leading. Until more definitive evidence decisions. Skepticism of faculty mem¬ medicine. Increasingly, scientific over¬ is adduced, adoption of evidence-based bers can be reduced by the availability views will be systematically integrated medicine should appropriately be re¬ of "quick and dirty" (as well as more with information regarding toxicity and stricted to two groups. One group com¬ sophisticated) courses on critical apprais¬ side effects, cost, and the consequences prises those who find the rationale com¬ al of evidence and by the teaching part¬ of alternative courses of action to de¬ pelling, and thus believe that use of the nerships and teaching workshops de¬ velop clinical policy guidelines.34 The evidence-based medicine approach is scribed earlier. prospects for these developments are likely to improve clinical care. A second Many problems in the practice and both bright and exciting. group comprises those who, while skep¬ teaching of evidence-based medicine re¬ tical of improvements in patient out¬ main. both DOES TEACHING AND LEARNING believe it is that Many physicians, including EVIDENCE-BASED MEDICINE come, very unlikely residents and members, are still deterioration in care results from the faculty IMPROVE PATIENT OUTCOMES? skeptical about the tenets of the new evidence-based approach and who find paradigm. A medical residency is full of The proof of the pudding of evidence- that the practice of medicine in the new competing demands, and the appropri¬ based medicine lies in whether patients paradigm is more exciting and fun. ate balance between is not cared for in this fashion better goals always enjoy CONCLUSION evident. At the same time, we are buoyed health. This proof is no more achievable by the number of residents and faculty for the new paradigm than it is for the Based on an awareness of the limita¬ who have enthusiastically adopted the old, for no long-term randomized trials tions of traditional determinants of clin¬ new approach and found ways to inte¬ of traditional and evidence-based med¬ ical decisions, a new paradigm for med¬ grate it into their learning and practice. ical education are likely to be carried ical practice has arisen. Evidence-based out. What we do have are a number of medicine deals directly with the uncer¬ Barriers to Practicing short-term studies which confirm that tainties of clinical medicine and has the Evidence-Based Medicine the skills ofevidence-based medicine can potential for transforming the educa¬ Even if our residency program is suc¬ be taught to medical students35 and med¬ tion and practice of the next generation cessful in producing graduates who en¬ ical residents.36 In addition, a study com¬ of physicians. These physicians will con¬ ter the world of clinical practice enthu¬ pared the graduates of a medical school tinue to face an exploding volume of siastic to apply what they have learned that operates under the new paradigm literature, rapid introduction of new about evidence-based medicine, they will (McMaster) with the graduates of a tra¬ technologies, deepening concern about face difficult challenges. Economic con¬ ditional school. A random sample of burgeoning medical costs, and increas¬ straints and counterproductive incen¬ McMaster graduates who had chosen ing attention to the quality and outcomes tives may compete with the dictates of careers in family medicine were more of medical care. The likelihood that ev¬ evidence as determinants of clinical de¬ knowledgeable with respect to current idence-based medicine can help amelio¬ cisions; the relevant literature may not therapeutic guidelines in the treatment rate these problems should encourage be readily accessible; and the time avail¬ of hypertension than were the gradu¬ its dissemination. able may be insufficient to carefully re¬ ates of the traditional school.37 These Evidence-based medicine will require view the evidence (which may be volu¬ results suggest that the teaching of ev¬ new skills for the physician, skills that minous) relevant to a pressing clinical idence-based medicine may help grad¬ residency programs should be equipped problem. uates stay up-to-date. Further evalua¬ to teach. While strategies for inculcat¬ Some solutions to these problems are tion of the evidence-based medicine ap¬ ing the principles ofevidence-based med¬ already available. Optimal integration proach is necessary. icine remain to be refined, initial expe¬ ofcomputer technology into clinical prac¬ Our advocating evidence-based med¬ rience has revealed a number of effec¬ tice facilitates finding and accessing ev¬ icine in the absence of definitive evi¬ tive approaches. Incorporating these idence. Reference to literature over- dence of its superiority in improving pa- practices into postgraduate medical ed-

Downloaded From: http://jama.jamanetwork.com/ by a University of Ottawa User on 10/02/2014 ucation and continuing to work on their partment of Medicine, McMaster University; Allan tize Cardio-Vasculaires, Paris, France; Virginia further will result in more Detsky, MD, PhD, Department of Clinical Epide¬ Moyer, MD, Department of Pediatrics, University development miology and Biostatistics, McMaster University of Texas, Houston; Cynthia Mulrow, MD, Depart¬ rapid dissemination and integration of and Departments of Health Administration and ment of Medicine, University of Texas, San Anto¬ the new paradigm into medical practice. Medicine, University of Toronto (Ontario); Murray nio; Paul Links, MD, MSc, Department of Psychi¬ Enkin, MD, Departments of Clinical Epidemiology atry, McMaster University; Andrew Oxman, MD, The Evidence-Based Medicine Working Group and Biostatistics and Obstetrics and Gynaecology, MSc, Departments of Clinical Epidemiology and comprised the following: Gordon Guyatt (chair), McMaster University; Pamela Frid, MD, Depart¬ Biostatistics and Family Medicine, McMaster Uni¬ MD, MSc, John Cairns, MD, David Churchill, MD, ment of Pediatrics, Queen's University, Kingston, versity; Jack Sinclair, MD, Departments of Clinical MSc, Deborah Cook, MD, MSc, Brian Haynes, MD, Ontario; Martha Gerrity, MD, Department of Med¬ Epidemiology and Biostatistics and Pediatrics, MSc, PhD, Jack Hirsh, MD, Jan Irvine, MD, MSc, icine, University of North Carolina, Chapel Hill; McMaster University; and Peter Tugwell, MD, Mark Levine, MD, MSc, Mitchell Levine, MD, MSc, Andreas Laupacis, MD, MSc, Department of Clin¬ MSc, Department of Medicine, University of Ot¬ Jim Nishikawa, MD, and David Sackett, MD, MSc, ical Epidemiology and Biostatistics, McMaster tawa (Ontario). Departments of Medicine and Clinical Epidemiol¬ University and Department of Medicine, Univer¬ Drs Cook and Guyatt are Career Scientists of ogy and Biostatistics, McMaster University, sity of Ottawa (Ontario); Valerie Lawrence, MD, the Ontario Ministry of Health. Dr Haynes is a Na¬ Hamilton, Ontario; Patrick Brill-Edwards, MD, Department of Medicine, University of Texas tional Health Scientist, National Health Research Hertzel Gerstein, MD, MSc, Jim Gibson, MD, Ro¬ Health Science Center at San Antonio and Audie L. and Development Program, Canada. Drs Jaeschke man Jaeschke, MD, MSc, Anthony Kerigan, MD, Murphy Memorial Veterans Hospital, San Antonio, and Cook are Scholars of the St Joseph's Hospital MSc, Alan Neville, MD, and Akbar Panju, MD, De- Tex; Joel Menard, MD, Centre de Medicine Trezen- Foundation, Hamilton, Ontario.

References 1. Lindberg DA. Information systems to support 13. Haynes RB, Mulrow CD, Huth EJ, Altman DG, ings. In: Clayton PD, ed. Proceedings of the Fifth medical practice and scientific discovery. Methods Gardner MJ. More informative abstracts revisited. Annual Symposium on Computer Applications in Inform Med. 1989;28:202-206. Ann Intern Med. 1990;113:69-76. Medical Care, November 1991; Washington, DC. 2. Hart YM, Sander JW, Johnson AL, Shorvon SD. 14. Haynes RB. The origins and aspirations of ACP New York, NY: McGraw-Hill International Book National general practice study of epilepsy: recur- Journal Club. Ann Intern Med. 1991;114(ACP J Co; 1992:73-77. rence after a first seizure. Lancet. 1990;336:1271\x=req-\ Club. suppl 1):A18. 27. Laupacis A, Sackett DL, Roberts RS. An as- 1274. 15. Chalmers I, Enkin M, Keirse MJNC, eds. Ef- sessment of clinically useful measures of the con- 3. Sackett DL, Haynes RB, Guyatt GH, Tugwell P. fective Care in Pregnancy and Childbirth. New sequences of treatment. N Engl J Med. 1988;318: Clinical Epidemiology: A Basic Science for Clini- York, NY: Oxford University Press Inc; 1989. 1728-1733. cial Medicine. 2nd ed. Boston, Mass: Little Brown 16. Sinclair JC, Braken MB, eds. Effective Care of 28. Department of Clinical Epidemiology and Bio- & Co Inc; 1991:173-186. the Newborn Infant. New York, NY: Oxford Uni- statistics, McMaster University. How to read clin- 4. Kuhn TS. The Structure of Scientific Revolu- versity Press Inc; 1992. ical journals, II: to learn about a diagnostic test. tions. Chicago, Ill: University of Chicago Press; 17. Audet AM, Greenfield S, Field M. Medical prac- Can Med Assoc J. 1981;124:703-710. 1970. tice guidelines: current activities and future direc- 29. Department of Clinical Epidemiology and Bio- 5. European Carotid Surgery Trialists' Collabora- tions. Ann Intern Med. 1990;113:709-714. statistics, McMaster University. How to read clin- tive Group. MRC European Carotid Surgery Trial: 18. Guyatt GH. Evidence-based medicine. Ann In- ical journals, V: to distinguish useful from useless interim results for symptomatic patients with se- tern Med. 1991;114(ACP J Club. suppl 2):A-16. or even harmful therapy. Can Med Assoc J. 1981; vere (70-99%) or with mild (0-29%) carotid stenosis. 19. Light DW. Uncertainty and control in profes- 124:1156-1162. Lancet. 1991;337:1235-1243. sional training. J Health Soc Behav. 1979;20:310\x=req-\ 30. Oxman AD, Guyatt GH. Guidelines for reading 6. Larsen ML, Horder M, Mogensen EF. Effect of 322. literature reviews. Can Med Assoc J. 1988;138:697\x=req-\ long-term monitoring of glycosylated hemoglobin 20. Chalmers I. Scientific inquiry and authoritar- 703. levels in insulin-dependent diabetes mellitus. NEngl ianism in perinatal care and education. Birth. 1983; 31. Campbell EJM. The diagnosing mind. Lancet. J Med. 1990;323:1021-1025. 10:151-164. 1987;1:849-851. 7. L'Abbe KA, Detsky AS, O'Rourke K. Meta-anal- 21. Cassell EJ. The nature of suffering and the 32. Feinstein AR. What kind of basic science for ysis in clinical research. Ann Intern Med. 1987;107: goals of medicine. N Engl J Med. 1982;306:639-645. clinical medicine? N Engl J Med. 1970;283:847-853. 224-233. 22. Ende J, Kazis L, Ash A, Moskowitz MA. Mea- 33. Mulrow CD. The medical review article: state 8. Ransohoff DF, Feinstein AR. Problems of spec- suring patients' desire for autonomy: decision-mak- of the science. Ann Intern Med. 1987;106:485-488. trum and bias in evaluating the efficacy of diag- ing and information-seeking preferences among 34. Eddy DM. Guidelines for policy statements: the nostic tests. N Engl J Med. 1978;299:926-930. medical patients. J Gen Intern Med. 1989;4:23-30. explicit approach. JAMA. 1990;263:2239-2243. 9. Nierenberg AA, Feinstein AR. How to evaluate 23. Carter WB, Inui TS, Kukull WA, Haigh VH. 35. Bennett KJ, Sackett DL, Haynes RB, et al. A a diagnostic market test: lessons from the rise and Outcome-based doctor-patient interaction analysis. controlled trial of teaching critical appraisal of the fall ofdexamethasone suppression test. JAMA. 1988; Med Care. 1982;20:550-556. clinical literature to medical students. JAMA. 1987; 259:1699-1702. 24. Greenfield S, Kaplan S, Ware JE Jr. Expanding 257:2451-2454. 10. Haynes RB, McKibbon KA, Fitzgerald D, patient involvement in care: effects on patient out- 36. Kitchens JM, Pfeifer MP. Teaching residents to Guyatt GH, Walker CJ, Sackett DL. How to keep comes. Ann Intern Med. 1985;102:520-528. read the medical literature: a controlled trial of a up with the medical literature, V: access by per- 25. Haynes RB, McKibbon A, Walker CJ, Ryan N, curriculum in critical appraisal/clinical epidemiol- sonal computer to the medical literature. Ann In- Fitzgerald D, Ramsden ME. Online access to MED- ogy. J Gen Intern Med. 1989;4:384-387. tern Med. 1986;105:810-816. LINE in clinical settings: a study of use and use- 37. Shin J, Haynes RB. Does a problem-based, self- 11. Bulpitt CJ. Confidence intervals. Lancet. 1987; fulness. Ann Intern Med. 1990;112:78-84. directed undergraduate medical curriculum pro- 1:494-497. 26. McKibbon KA, Haynes RB, Johnston ME, mote continuing clinical competence? Clin Res. 1991; 12. Godfrey K. Simple linear regression in medical Walker CJ. A study to enhance clinical end-user 39:143A. research. N Engl J Med. 1985;313:1629-1636. MEDLINE search skills: design and baseline find-

Downloaded From: http://jama.jamanetwork.com/ by a University of Ottawa User on 10/02/2014

Blackwell Science, LtdOxford, UKJEPJournal of Evaluation in Clinical Practice1356-1294Blackwell Publishing Ltd 2004123248256Original ArticleIntegrating evidence into clinical practiceM.R. Tonelli

Journal of Evaluation in Clinical Practice, 12, 3, 248–256

Integrating evidence into clinical practice: an alternative to evidence-based approaches

Mark R. Tonelli MD MA Associate Professor of Medicine, Division of Pulmonary and Critical Care Medicine, Department of Medicine and, Department of Medical History and Ethics, University of Washington, Seattle, WA, USA

Correspondence Abstract Dr Mark R. Tonelli Evidence-based medicine (EBM) has thus far failed to adequately account University of Washington Box 356522 for the appropriate incorporation of other potential warrants for medical 1959 NE Pacific Street decision making into clinical practice. In particular, EBM has struggled with Seattle, WA 98195-6522 the value and integration of other kinds of medical knowledge, such as USA those derived from clinical experience or based on pathophysiologic ratio- E-mail: [email protected] nale. The general priority given to empirical evidence derived from clinical Keywords: casuistry, clinical judgement, research in all EBM approaches is not epistemically tenable. A casuistic epistemology, evidence-based medicine, alternative to EBM approaches recognizes that five distinct topics, 1) medical decision making empirical evidence, 2) experiential evidence, 3) pathophysiologic rationale, 4) patient goals and values, and 5) system features are potentially relevant Accepted for publication: 12 August 2004 to any clinical decision. No single topic has a general priority over any other and the relative importance of a topic will depend upon the circumstances of the particular case. The skilled clinician must weigh these potentially con- flicting evidentiary and non-evidentiary warrants for action, employing both practical and theoretical reasoning, in order to arrive at the best choice for an individual patient.

ula. (Green 1999). Such a widespread adoption of Introduction EBM into graduate medical education promises to Evidence-based medicine (EBM) has been champi- alter the way that the next generation of doctors oned as a ‘new paradigm’ for medical education and practise clinical medicine, yet it is not clear whether practice (The Evidence-Based Medicine Working such a change will ultimately benefit patients. Group 1992). Nowhere has the impact of EBM been Although the bulk of the literature of EBM as prominent as that in the academic medical centre. focuses on the practical issues related to the devel- Advocates of EBM have consistently noted the opment, acquisition, interpretation and incorpora- importance of learning EBM during the formative tion of the results of clinical research into clinical period of residency training (Rosenberg & Sackett practice, EBM rests on certain philosophical assump- 1996). Despite the lack of good evidence that teach- tions and arguments about the nature of medical ing EBM improves the quality of medical education knowledge that have not been as fully elucidated or the subsequent care of patients (Parkes et al. 2001; (Miles et al. 2004). Though now acknowledging the Hatala & Guyatt 2002), the vast majority of internal need to integrate various kinds of medical and non- medicine programmes in the United States now medical knowledge, proponents of EBM have said incorporate some aspect of EBM into residency very little about how such integration should actually training and a third have a free standing curriculum take place. Interestingly, this application of evidence in EBM (Green 2000). Other residency training pro- to clinical decision making is also the component of grammes have also embraced EBM into their curric- EBM that is least likely to be dealt with in EBM edu-

248 © 2006 Blackwell Publishing Ltd

Integrating evidence into clinical practice

cational curricula (Green 2000). It remains unclear The notion of a general priority of empirical evi- by what process clinicians are to balance what may be dence was challenged by doctors and other commen- conflicting warrants for action coming from not only tators, many of whom pointed out the inherent published clinical research but also personal experi- limitations of attempting to apply knowledge gained ence, pathophysiologic understanding of disease, the from population studies to the care of individual preferences of individual patients or other ‘non- patients (Tanenbaum 1993; Tanenbaum 1994; Hor- evidentiary’ sources. witz 1996; Feinstein & Horwitz 1997). The response Here I will argue that, to date, the few attempts by to this critique was a subsequent semantic switch proponents of EBM to describe a method for inte- on the part of EBM proponents. Rather than ‘de- grating various kinds of medical knowledge and emphasizing’ non-evidentiary kinds of medical reasoning into clinical decision making have been knowledge, EBM began to call for the ‘integration’ of unsatisfactory. The failure of the EBM approach cen- some alternate kinds of medical knowledge as well as tres on its attempt to treat different potential war- patient goals and values into clinical decision making rants for medical decision making, such as empiric (Sackett et al. 1996). The approach has evolved over evidence, clinical experience and pathophysiologic the decade since the introduction of EBM. Initial rationale, as different in degree, rather than different calls for integration of alternate kinds of medical in kind. Under such an understanding, one form of knowledge focused on personal clinical experience medical knowledge, specifically that derived from (Sackett 1997), the devaluing of which was not coin- clinical research, can be said to be superior to the cidentally the primary objection of many clinicians to others. I will argue that this approach is philosophi- EBM. Still, little was said about how to actually go cally untenable and that various potential warrants about integrating evidence with clinical experience, for medical decision making differ in kind from one other than that it should be performed in a ‘con- another. Such an understanding of the nature of scientious, explicit and judicious’ fashion (Sackett medical knowledge requires an alternative method 1997). Subsequently, pathophysiologic rationale was for integrating various warrants into a particular added by some to the list of non-evidentiary kinds of medical decision, a method that closely resembles medical knowledge that needed to be integrated with the casuistic, or case-based, approach to medical the evidence (Ellrodt et al. 1997). Of note, expert ethics advanced by Jonsen and others (Jonsen et al. opinion as a grounds for clinical decision making 1992). Ultimately, this understanding of medical has not experienced a similar rehabilitation (Tonelli epistemology can provide for an explicit defence of a 1999). Throughout this initial broadening of per- more ‘common-sense’ practice of medicine desired spective, EBM recognized clinical experience and by many clinicians (Porta 2004). pathophysiologic rationale as different in kind from empirical evidence (Canadian Task Force on the Periodic Health Examination 1979; Hadorn et al. 1996). Integrating the evidence: EBM approaches This recognition of the value of alternate kinds of EBM began by stating a clear and explicit preference medical knowledge, however, presents a problem for for empirical evidence, defined as evidence derived EBM. If non-evidentiary forms of medical knowl- from formal and systematic clinical research, over edge are given equal status to empirical evidence, alternate kinds of medical knowledge, specifically then the primary axiom of EBM, which identifies individual clinical experience, expert opinion and research-derived evidence as the ‘best evidence’ to pathophysiologic rationale, as grounds for clinical guide clinical decision making, is undermined. That decision making (The Evidence-Based Medicine is, if non-evidentiary knowledge is allowed to trump Working Group 1992). ‘Non-evidentiary’ kinds of empirical evidence, then the clinician who knows and knowledge were not completely dismissed, but were understands the clinical research relevant to a partic- deemed useful only attributed to ‘the dearth of ular condition but chooses to treat patients with that adequate evidence’ (The Evidence-Based Medicine condition in a manner not supported by the empirical Working Group 1992). evidence can still claim to be practising ‘EBM’, as she

© 2006 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice, 12, 3, 248–256 249

M.R. Tonelli

is simply integrating clinical experience in a consci- dence, proponents of EBM can structure hierarchies entious, explicit and judicious fashion that happens of evidence that place evidence derived from clinical to give priority to that experience. EBM, to avoid research higher than experiential or physiologic ‘evi- becoming simply a meaningless appellation to which dence’. In this way, EBM maintains a general priority anyone could claim allegiance despite radically dif- of empirical evidence as ‘best evidence’ while recog- ferent approaches to clinical medicine, needs to be nizing some value of previously non-evidentiary able to preserve at least a general preference for the kinds of knowledge. The hierarchy implies that use of empirical evidence while simultaneously rec- empirical evidence, especially when of high quality, ognizing some value of at least particular kinds of should be viewed as so compelling as to obviate the non-evidentiary forms of medical knowledge to need to consider clinical experience or physiologic clinical practice. understanding in a clinical decision. The most recent attempt to resolve this problem Both the decision to include clinical experience involves the co-opting of kinds of medical knowledge and physiologic understanding under the umbrella of previously viewed as non-evidentiary in nature under ‘evidence’ and the subsequent assignment of these the umbrella of ‘evidence’. Specifically, personal kinds of knowledge to the lower rungs of the hierar- clinical experience and physiologic understanding chy of evidence are, of course, not themselves are currently defined by some proponents of EBM as evidence-based, but rather represent philosophical forms of medical ‘evidence’ that differ in degree, but (more precisely, epistemological) assertions. Both not in kind from empirical evidence derived from assertions, I will argue, are incorrect. clinical research (Guyatt & Rennie 2002). This First, the recent attempt to classify clinical experi- epistemic shift is clear in the change in the hierar- ence and physiologic understanding as ‘evidence’ chies of evidence that have been advanced since the errs in asserting that these kinds of knowledge are formal introduction of EBM (Table 1). Classification similar enough to evidence derived from clinical of clinical experience and physiologic understanding research to be encompassed under a single epistemic as forms of ‘evidence’ allows EBM to make two heading. What the various kinds of knowledge do claims. First, a doctor relying on clinical experience share is the potential to serve as warrants for clinical or physiologic reasoning for making a clinical deci- decisions, but this does not suffice to claim each is sion in the absence of empirical evidence derived a kind of ‘evidence’ unless one makes the circular from clinical research can still be said to be practising argument that all warrants for medical decisions EBM. More importantly, by classifying clinical expe- constitute evidence. But proponents of EBM cannot rience and physiologic understanding as forms of evi- defend that argument, as they clearly differentiate

Table 1 Evolution of hierarchies or levels of evidence in evidence-based medicine

Early hierarchy Current hierarchy

1. Well-conducted randomized controlled trials 1. N-of-1 randomized controlled trial 2. Well-conducted cohort studies or case–control study (observational studies) 2. Systematic review of randomized trials 3. Poorly controlled or uncontrolled studies 3. Single randomized trial 4. Expert opinion 4. Systematic review of observational studies 5. Single observational study 6. Physiologic studies 7. Unsystematic clinical observation Adapted from Canadian Task Force (1979) and Hadorn et al. (1996) Adapted from Guyatt & Rennie (2002) Note the absence of personal clinical experience and pathophysiologic Note the addition of physiology and clinical reasoning from the category of ‘evidence’. experience to the category of ‘evidence’. Expert opinion is no longer considered ‘evidence’.

250 © 2006 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice, 12, 3, 248–256

Integrating evidence into clinical practice

between clinical experience and expert opinion as ‘evidence’, the assertion that empirical evidence well as between physiologic understanding and intu- derived from clinical research, when present, always ition, with explicit exclusion of expert opinion and provides ‘better evidence’ to guide clinical decision intuition from the sphere of ‘evidence’ despite the making is erroneous. Each potential source of medi- fact that each may serve as a warrant for particular cal knowledge carries with it a different complement medical decisions. Proponents of EBM, then, must of strengths and weaknesses. Proponents of EBM assert that clinical experience, physiologic under- have clearly outlined the limitations of relying on standing and empirical evidence share some impor- clinical experience and physiologic reasoning for tant feature not present in excluded potential medical decision making (The Evidence-Based Med- warrants, such as expert opinion. Yet clinical experi- icine Working Group 1992; Sackett & Rosenberg ence seems to be quite different from evidence 1995), while critics have pointed to the inherent lim- derived from clinical research. The former is direct itations of applying empirical evidence to the care and personal, the latter indirect and general. In fact, of particular patients (Feinstein & Horwitz 1997; a classic philosophical distinction exists between Tonelli 1998; Tanenbaum 1999). Clinical experience first-hand, primary experience and empirical evi- and/or pathophysiologic rationale may at times seem dence derived from systematic observation and to contradict conclusions about optimal care drawn experimentation, what Reed terms ‘processed infor- from published empirical evidence. For instance, low mation’(Reed 1996). This distinction was, in fact, cen- tidal volume ventilation has been demonstrated to be tral to the initial claims of the superiority of EBM safe and to improve survival in respiratory failure (The Evidence-Based Medicine Working Group from acute respiratory distress syndrome (ARDS 1992). Only of late has the concept of ‘evidence’ Network 2000). Yet an individual patient may become so inclusive. respond to the acidosis associated with this therapy Likewise, the inclusion of physiologic understand- with some negative physiologic response, such as ing as a kind of ‘evidence’ is philosophically problem- cardiac arrhythmia, not otherwise described in the atic. Proponents of EBM would be consistent and study. A clinician may choose to ignore this individ- convincing if they limited the inclusion of physiologic ual anomaly and continue to provide the treatment understanding to evidence derived from specific that has been demonstrated to reduce mortality, but physiologic experiments in humans or animals, does so here at significant risk to the individual. experiments that provide no clinical outcome mea- In addition to clinical experience and pathophysi- sures. However, doctors do not generally cite specific ologic rationale, other features of a particular case physiologic studies, as they do specific clinical trials, may conspire to support a decision that would other- when defending particular clinical decisions. Instead, wise be contrary to that made in accordance with the they argue from physiologic principles, from theories empirical evidence only. The most obvious examples derived from experiments rather than from the involve the refusals of patients to undergo interven- particular experiments themselves. Reasoning from tions that are medically indicated and supported by physiologic principles represents a form of rational- excellent clinical evidence. For instance, a man post ism in medicine, the very notion of which has been myocardial infarction may refuse beta-blocker ther- more and more vehemently rejected over the last two apy because of the side effect of impotence while centuries. The new empiricism of EBM, again, was fully aware of the statistical likelihood of prolonged built on the explicit rejection of rationalism, recog- survival provided by the drug. Recently, EBM has nizing the kinds of errors that have followed from paid more attention to the goals and values of indi- this method of reasoning. Yet reliance on theory vidual patients, though trying to quantify and process remains important in clinical medicine, rightly affect- these into the form of ‘patient utilities’ more amena- ing how we interpret and value empirical evidence ble to algorithms and decision analysis (Naglie et al. (Vandenbroucke & de Craen 2001). 1997; Man-Son-Hing et al. 2000). Other simple exam- Even if we allow that clinical experience, physio- ples where high quality evidence cannot determine logic understanding and empirical evidence are care centre around the availability of resources to enough alike to allow each to be considered a kind of provide care supported by empirical evidence. For

© 2006 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice, 12, 3, 248–256 251

M.R. Tonelli

instance, the benefit of emergent angioplasty in myo- experiential evidence and pathophysiologic under- cardial infarction will certainly not be realized in a standing differ in kind from one another, not in centre that does not have access to an advanced pro- degree. Hence, these three kinds of medical knowl- cedure lab and an interventional cardiologist. edge serve as the first three topics of a casuistic The clinician, then, must contend with not only model of clinical decision making. A hierarchy of empirical evidence, but also her own experiential value can be developed within each topic, but the knowledge, an understanding of pathophysiology, importance of one topic relative to another is not the goals and values of the patient and the barriers fixed; clinical experience is not necessarily of less and facilitators of care of the medical system in which importance than empirical evidence nor more impor- she works. The relative weight to be assigned to each tant than pathophysiologic understanding. Rather, of these potential warrants for action depends upon the relative weight assigned to each topic will depend the particulars of the case at hand. Recognition of on the case at hand. A brief exposition of each of this dependence on the particular case suggests that a these initial three topics follows. casuistic understanding of clinical decision making Empirical evidence derives from clinical research may provide a more satisfactory description of opti- and becomes knowable to the practising doctor only mal medical practice than that provided by EBM through published reports. A hierarchy of empirical thus far. evidence, which ranks examples of such evidence by study design and statistical robustness, is generally agreed upon [though there is still some debate about Integrating the evidence: a casuistic approach relative strengths of particular study designs (Con- In ‘The Abuse of Casuistry’, Jonsen and Toulmin’s cato et al. 2000)]. Teaching how to access, interpret, examination of the history of practical moral reason- and critically appraise the published reports of ing, the authors use clinical medicine as their primary empirical evidence has become a major focus of example of a ‘practical’ enterprise (Jonsen & Toul- Western medical education (Green 2000). The value min 1988). They claim that clinical decision making of relying on this kind of medical knowledge is well demands ‘prudence’ or phronesis, in addition to the- catalogued by proponents of EBM. The major limi- oretical understanding of medicine and specific tech- tations of empirical evidence relate to the fact that it nical skills, in order to deal with particularities of cannot be directly applied to any particular patient individual illness. Hence, the authors argue, clinical (Feinstein & Horwitz 1997; Tonelli 1998). medicine is a casuistic undertaking. Hunter also Experiential evidence encompasses the knowledge made a compelling argument for clinical medicine as gained from the direct care of patients. The practising casuistry 3 years before the first mention of the term doctor may rely on personal experience or attempt to ‘evidence-based medicine’ appeared in the medical learn from the personal experience of others. With literature (Hunter 1989). Jonsen and others have experiential knowledge, more is generally considered subsequently argued for and codified a casuistic better than less. Hence, expert opinion, when based approach to medical ethics by delineating the topics on extensive experience with large numbers of common to clinical situations and a method for con- patients with a particular disease, may be viewed as sidering each (Jonsen et al. 1992). Significantly less the highest form of experiential evidence (Tonelli attention has been paid to delineating the relevant 1999). The major problem with experiential knowl- topics for clinical decision making itself. The topics edge is that it is prone to multiple kinds of cognitive necessary for a casuistic understanding of clinical bias (Elstein & Schwarz 2002), with potentially false medicine will clearly overlap with those topics conclusions about causality or treatment effect being central to assessing the ethical issues involved in a drawn. particular clinical decision. Pathophysiologic understanding follows from the As noted above, the current attempt of EBM to general Western understanding of illness as a pertur- place multiple potential warrants for medical deci- bation of the physical self. Understanding basic sion making under the umbrella of ‘evidence’ fails biologic and physiologic principles allows doctors to primarily because of the fact that empirical evidence, both relate presenting features to a diagnosis and to

252 © 2006 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice, 12, 3, 248–256

Integrating evidence into clinical practice

anticipate and measure response to therapy. The casuistic model of clinical medicine, can be viewed as strength of pathophysiologic reasoning primarily the ability to process and weigh particular warrants depends on the strength of the underlying biologic or for action from each topic area and arrive at a pre- physiologic theory. The major limitation of reasoning sumptive conclusion regarding the best course of from scientific principles centres on the uncertain action. In many cases, the clinical decision is rela- relationship between physiologic measures and tively straightforward, as when the empirical evi- meaningful outcomes such as mortality and quality dence, experiential evidence and pathophysiologic of life. understanding all suggest a particular treatment plan, In addition to the three epistemic topics noted one that is available and that the patient wishes to above, two other topics are relevant to any medical pursue. But in other instances, empirical evidence decision. Perhaps most obviously, the goals, values and experiential evidence or pathophysiologic and preferences of the individual patient must be understanding may suggest different approaches. The considered. Medical knowledge alone only has the approach of EBM has been to give general priority to power to produce hypothetical imperatives of the empirical evidence in such circumstances. But such form: ‘If you want to maximise the chances of A, then an approach does not make philosophical sense. do B’. The incorporation of patient values is required Rather, the ‘conscientious and judicious’ use of to determine whether or not maximizing ‘the chances empirical evidence may require the clinician to set of A’ is an appropriate and meaningful goal (Tonelli that evidence aside when individual circumstances 1998). appear to require it. The final topic relevant to any clinical decision falls under a broad heading of ‘System Features’, to Empirical evidence in isolation include the economic, logistic, legal and cultural bar- riers or facilitators of care. The cost, availability or Thus far, I have not specifically addressed how legality of specific interventions may preclude use empirical evidence is incorporated into clinical deci- even in the setting of strong empirical and experien- sion making other than to argue that it does not have tial evidence in support. More subtly, the very system general priority over other kinds of medical knowl- of health care delivery, as well as professional and edge. Another important but generally overlooked societal values, may influence decisions regarding the aspect of EBM deals with how doctors are to use care of individuals. Not always are these consider- empirical evidence in itself, rather than how that kind ations made explicit, nor even recognized. of medical knowledge relates to other potential The five topics relevant for any medical decision warrants for medical decision making. I assert that are listed in Table 2. Again, it is important to note reasoning from empirical evidence specifically also that no topic takes general priority over any other. represents a form of practical or casuistic reasoning. Any topic may prove to be determinative in a partic- Clinical research, and particularly the randomized ular clinical decision. Clinical judgment, under this controlled trial, does not generate universal and gen- eralizable knowledge, but rather reports observa- tions about treatment effects in populations of Table 2 The five topics of clinical decision making individuals, who share some qualities but not others, under certain specific circumstances. Thus, empirical Empirical evidence: derived from clinical research. evidence derived from clinical research does not pro- Experiential evidence: derived from personal clinical vide us with universal truths from which we can draw experience or the clinical experience of others (i.e. expert deductive conclusions about the patient-at-hand, but opinion). Pathophysiologic rationale: based on underlying theories of rather with an example of a composite, ‘average physiology, disease and healing. patient’, (Feinstein & Horwitz 1997) to which we can Patient values and preferences: derived from personal compare the patient-at-hand. interaction with individual patients. Using Jonsen and Toulmin’s description of casu- System features: including resource availability, societal and istry, clinical research, when well done, provides cli- professional values, legal and cultural concerns. nicians with a ‘paradigmatic’ case. The clinician must

© 2006 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice, 12, 3, 248–256 253

M.R. Tonelli

decide whether the patient-at-hand resembles the education, medical training must directly address the ‘average’ patient provided by the clinical research limitations of relying primarily on empirical evidence closely enough to warrant incorporating the conclu- for clinical decision making. sions of that research into that patient’s care. If the Second, a casuistic understanding of clinical med- patient differs from the ‘average’ study subject, the icine recognizes that clinical decision making is a per- clinician must decide whether that difference is sonal and prudential undertaking and therefore important enough to mean the conclusions drawn clinicians may arrive at different conclusions in sim- from the study are not relevant to the current deci- ilar cases. This acceptance of variability runs counter sion. How closely the patient-at-hand resembles the to one of the stated benefits of EBM, the limiting of ‘average’ patient in a study will determine how much practice variability. By failing to explicitly acknowl- weight to give the results. At times, clinicians must edge that empirical research is not prescriptive and ask whether the patient-at-hand more closely resem- that reasoning from empirical evidence is not deduc- bles those enrolled in one study or in another that tive, EBM presumes that there is a right course of yielded a different result. This task, acknowledged as action in particular cases: the one ‘supported by the vital by proponents of EBM, is inherently casuistic. If evidence’. Only when the body of empirical evidence performed correctly, this process involves not only is viewed as insufficient does EBM allow for signifi- comparisons with other potential ‘paradigmatic’ cant variability in practice, given the perceived un- cases from the medical literature, but also with real reliability of other kinds of medical knowledge. cases from the clinician’s personal experience and But clinical decisions remain presumptive not only perhaps patients, individual or in aggregate, from the because of a paucity of clinical research, but also experience of expert colleagues. Reasoning will be in because exceptions to any rule derived from such the form of analogy, with a search for cases, actual or research will always exist. Excellent and copious clin- ‘average’, that are most similar to the case-at-hand. ical research can become compelling enough to guide Differences must be considered to see if they repre- care in a large number of straightforward cases, but sent important enough distinctions to reject the the doctor must always remain open to the possibility provisional conclusions about the present case. that the patient-at-hand represents an exception. Eventually, physiologic and system considerations, as And EBM must not only allow but also encourage well as patient goals and values, will have to be con- doctors to consider all potentially relevant kinds of sidered in order to arrive at a presumptive conclusion medical knowledge in each case. To do less is to risk regarding the best course of action. creating a new dogmatic kind of medical authority, one based on ‘the evidence’ (Feinstein & Horwitz 1997). Implications of the casuistic approach for EBM The casuistic approach does, however, embrace A casuistic understanding of clinical medicine has at some of the core values of EBM. In particular, this least two important implications for the current approach demands that medical reasoning be rigor- understanding of EBM. First, the current focus of ous and explicit. Doctors may differ in their final EBM on teaching students and doctors how to assessments based on the weight they give to differ- develop, acquire and critically appraise clinical ent potential warrants, but they must be aware of research must be acknowledged to be insufficient to and fully consider the empirical evidence available encourage the provision of optimal medical care. By in reaching conclusions. The casuistic method out- not acknowledging the difficulties in negotiating lined above offers an easily teachable and formal between evidentiary and non-evidentiary warrants, approach to clinical decision making that likely will EBM runs the risk of training a generation of doctors facilitate the education of doctors-in-training. What who, while skilled in study interpretation and statis- might otherwise appear to be simply differences in tical methods, fail to recognize and develop the com- style or opinion on the part of clinicians can be plex reasoning skills necessary for sound clinical made explicit and traced back to differences in the judgment. While skills in the critical appraisal of weight given to conflicting warrants. As such, the reports of clinical research will remain vital to doctor root causes of practice variability can be made clear

254 © 2006 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice, 12, 3, 248–256

Integrating evidence into clinical practice

and debated openly, improving the educational Force on the Periodic Health Examination. Canadian process. Medical Association Journal 121, 1193–1254. Concato J., Shah N. & Horwitz R. (2000) Randomized, controlled trials, observational studies, and the hierarchy Conclusion of research design. New England Journal of Medicine 342, 1887–1892. The philosophical underpinnings of EBM have never Ellrodt G., Cook D., Lee J., Cho M., Hunt D. & been well elucidated and have shifted significantly Weingarten S. (1997) Evidence-based disease manage- over the last decade. A careful analysis of the kind ment. Journal of the American Medical Association 278, of knowledge generated by clinical research reveals 1687–1692. that attempts to view such knowledge as prescriptive Elstein A. & Schwarz A. (2002) Clinical problem solving in individual clinical decision making is in error. and diagnostic decision making: selective review of the Rather, the clinician uses empirical research in a cognitive literature. British Medical Journal 324, 729– casuistic fashion, comparing the patient-at-hand with 732. those paradigmatic cases provided in the medical Feinstein A.R. & Horwitz R.I. (1997) Problems in the ‘evi- literature. Subsequently, the clinician must then con- dence’ of ‘evidence-based medicine’. American Journal sider other potential warrants for action in order to of Medicine 103, 529–535. arrive at a presumptive conclusion about the case. Green M. (1999) Graduate medical education training in clinical epidemiology, critical appraisal, and evidence- The five topics relevant to any particular clinical deci- based medicine: a critical review of curricula. Academic sion are (1) empirical evidence (2) experiential evi- Medicine 74, 686–694. dence (3) pathophysiologic understanding (4) patient Green M. (2000) Evidence-based medicine training in goals and values and (5) system features. The skilled internal medicine residency programs: a national survey. clinician, then, must weigh these potentially conflict- Journal of General Internal Medicine 15, 129–133. ing warrants for action when dealing with the Guyatt G. & Rennie D. (2002) Users’ Guides to the Medical patient-at-hand, employing both practical and theo- Literature. AMA Press, Chicago, IL. retical reasoning and comparing the patient with Hadorn D.C., Baker D., Hodges J.S. & Hicks N. (1996) paradigmatic cases from both the literature and Rating the quality of evidence for clinical practice experience, before coming to a presumptive conclu- guidelines. Journal of Clinical Epidemiology 49, 749– sion regarding the appropriate course of action. This 754. casuistic understanding of clinical medicine has Hatala R. & Guyatt G. (2002) Evaulating the teaching of evidence-based medicine. Journal of the American Med- important implications for medical education as well ical Association 288, 1110–1111. as the provision of health care. The personal and pru- Horwitz R.I. (1996) The dark side of evidence-based med- dential nature of clinical decision making means that icine. Cleveland Clinic Journal of Medicine 63, 320–323. clinicians may reasonably differ in assessments, con- Hunter K. (1989) A science of individuals: medicine and clusions and recommendations regarding the care of casuistry. Journal of Medicine and Philosophy 14, 193– individual patients. The optimal practice of clinical 212. medicine, though requiring the knowledge of the Jonsen A.R., Siegler M. & Winslade W.J. (1992) Clinical results of clinical research, demands that doctors Ethics. McGraw-Hill, New York, NY. thoughtfully consider both evidentiary and non- Jonsen A.R. & Toulmin S. (1988) The Abuse of Casuistry. evidentiary warrants for action in each attempt to University of California Press, Berkeley, VA. deliver the best care to a particular individual. Man-Son-Hing M., Laupacis A.O., Connor A., Coyle D., Berquist R. & McAlister F. (2000) Patient preference- based treatment thresholds and recommendations: a References comparison of decision-analytic modeling with the prob- ability-tradeoff technique. Medical Decision Making 20, ARDS Network. (2000) Ventilation with lower tidal vol- 394–403. umes as compared with traditional tidal volumes. New Miles A., Grey J., Polychronis A., Price N. & Melchiorri C. England Journal of Medicine 342, 1301–1308. (2004) Developments in the evidence-based health care Canadian Task Force on the Periodic Health Examination. debate-2004. Journal of Evaluation in Clinical Practice (1979) The periodic health examination. Canadian Task 10, 129–142.

© 2006 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice, 12, 3, 248–256 255

M.R. Tonelli

Naglie G., Krahn M., Naimark D., Redelmeier D. & Tanenbaum S.J. (1993) What physicians know. New Detsky A. (1997) Primer on medical decision analysis: England Journal of Medicine 329, 1268–1271. part 3 – estimating probabilities and utilities. Medical Tanenbaum S.J. (1994) Knowing and acting in medical Decision Making 17, 136–141. practice: the epistemologic politics of outcomes Parkes J., Hyde C., Deeks J. & Milne R. (2001) Teaching research. Journal of Health Politics, Policy and Law 19, critical appraisal skills in health care settings. Cochrane 27–44. Database of Systematic Reviews 3. Tanenbaum S. (1999) Evidence and expertise: the chal- Porta M. (2004) Is there life after evidence-based medicine? lenge of the outcomes movement to medical profession- Journal of Evaluation in Clinical Practice 10, 147–152. alism. Academic Medicine 74, 757–763. Reed E. (1996) The Necessity of Experience. Yale Univer- The Evidence-Based Medicine Working Group. (1992) sity Press, New Haven, CT. Evidence-based medicine: a new approach to teaching Rosenberg W.M. & Sackett D.L. (1996) On the need for the practice of medicine. Journal of the American Medi- evidence-based medicine. Therapie 51, 212–217. cal Association 268, 2420–2425. Sackett D. (1997) Evidence-based medicine. Seminars in Tonelli M.R. (1998) The philosophical limits of evidence- Perinatology 21, 3–5. based medicine. Academic Medicine 73, 234–240. Sackett D. & Rosenberg W. (1995) On the need for Tonelli M.R. (1999) In defense of expert opinion. Aca- evidence-based medicine. Health Economics 4, 249–254. demic Medicine 74, 1187–1192. Sackett D.L., Rosenberg W.M., Gray J.A., Haynes R.B. & Vandenbroucke J. & de Craen A. (2001) Alternative Richardson W.S. (1996) Evidence based medicine: what medicine: a ‘mirror image’ for scientific reasoning in con- it is and what it isn’t [editorial]. British Medical Journal ventional medicine. Annals of Internal Medicine 135, 312, 71–72. 507–513.

256 © 2006 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice, 12, 3, 248–256

Blackwell Science, LtdOxford, UKJEPJournal of Evaluation in Clinical Practice1356-1294Blackwell Publishing Ltd 2005123296298CommentaryBeyond ‘evidence’M. Gupta

Journal of Evaluation in Clinical Practice, 12, 3, 296–298

COMMENTARY Beyond ‘evidence’. Commentary on Tonelli (2006), Integrating evidence into clinical practice: an alternative to evidence-based approaches. Journal of Evaluation in Clinical Practice 12, 248–256

M. Gupta MD CM FRCPC MA Assistant Clinical Professor, Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, ON, Canada

In his excellent article, ‘Integrating evidence into piece of knowledge must be evaluated in the context clinical practice: an alternative to evidence-based of an entire clinical scenario. This process is the approaches’, Tonelli tackles the very important ques- essence of casuistic reasoning. Tonelli puts forward tion of what constitutes ‘integration’, a term found in casuistry as a model of clinical decision making in the definition of evidence-based medicine (EBM), response to EBM’s deficits as a decision-making that is, ‘EBM is the integration of best research evi- model, particularly its failure to explain what it dence with clinical expertise and patient values’. means by ‘integration.’ EBM really only addresses Although this definition serves as the foundation for the component of medical decision making con- the entire EBM project, EBM’s proponents have cerned with the effectiveness of interventions even paid very little scholarly attention to analysing its though many other factors are required to make an various component terms. Furthermore, as Tonelli actual decision. Through his discussion of casuistry, rightly points out, EBM assumes a great deal, par- Tonelli argues cogently for a richer and more tex- ticularly about the nature of knowledge. These tured notion of clinical decision making than what assumptions create certain conceptual problems. EBM currently provides. One such problem according to Tonelli, is that EBM EBM has lacked clarity concerning whether it is a treats all sources of information as similar in kind, model of clinical decision making, and if so, how it is rendering them directly comparable. This allows supposed to work. Tonelli’s contribution is therefore EBM to rank order various sources of information. apposite and timely. While one of the authoritative Tonelli argues that sources of information are not all texts on EBM, the Users’ Guide to the Medical Lit- similar in kind and therefore not comparable. He erature (Guyatt & Rennie 2002), states that EBM is proposes five different ‘topics’ which comprise the ‘about clinical decision making’, it alternately states factors relevant to clinical decision making: empirical that EBM is ‘about solving clinical problems’. In a evidence, experiential evidence, pathophysiologic longer statement of purpose, the Users’ Guide states rationale, patient values and preferences, and system that ‘. . . the goal is to be aware of the evidence on features. In his view one cannot, for example, con- which one’s practice is based, the soundness of the sider research data to be a superior form of knowl- evidence, and the strength of inference that evidence edge to patient values. These kinds of knowledge are permits.’ Another authoritative source, Evidence- entirely different and must be considered on their Based Medicine: How to Practice and Teach EBM own terms. Their weight will vary from case to case (Sackett et al. 2000), reveals a different view. Here depending on the particular details of any given EBM is described as a ‘philosophy of medical prac- clinical situation. tice based on knowledge and understanding of the The implication of this view for clinical practice is medical literature supporting each clinical decision’ that different types of knowledge cannot be priori- and later as a practice whose goal is forming ‘a diag- tized in terms of their importance in all circum- nostic and therapeutic alliance’ between doctor and stances but rather, the relative contribution of each patient as a means of ‘optimizing clinical outcomes

296 © 2006 Blackwell Publishing Ltd

Beyond ‘evidence’

and quality of life.’ While these descriptions of the store she was in when a symptom started, but this purpose of EBM vary considerably, they all seem to may not increase the doctor’s knowledge about her. imply, at the very least, that EBM is supposed to have What we mean by evidence has both general and something to do with making decisions in a clinical context-specific features. For a piece of information context. If EBM is a model of clinical decision mak- or data to be considered evidence, it must increase ing, it is insufficiently detailed in its discussion of the likelihood of an inference being true. In other basic issues such as what ‘integration’ actually entails. words, evidence must be evidence for something. Where I differ with Tonelli’s analysis is in his clas- Other general features of the concept of evidence are sification of empirical data and experience as forms that it must be based on information emerging from a of ‘evidence’ or evidentiary warrants for action while credible source and be relevant to the inference at pathophysiologic rationale, patient values and pref- hand (Schum 1994, pp. 17–20). erences, and systems issues are non-evidentiary war- Can we state definitively, as Tonelli does, that there rants for action. This division of the five topics into are some factors that will always, or never, be eviden- evidentiary and non-evidentiary sources can have the tiary in the context of decision making about individ- effect of replicating the very hierarchical ordering ual patients’ care? Clinical research data about a that Tonelli is trying to avoid. Because EBM is treatment for a certain condition may always be con- focused on finding and applying ‘evidence’, however, sidered to be potentially evidentiary when making defined, identifying something as ‘non-evidentiary’ decisions about whether to recommend that treat- suggests from the outset that it is less valuable ment. However, any particular data set may be suffi- than those bits of information considered ‘eviden- ciently flawed to be lacking in credibility or unlikely tiary.’ This tendency reflects EBM’s under analysis of to increase the probability of some inference being the concept of evidence itself. EBM supplies only a true. In this case, we could not call those research basic discussion of evidence which reveals that evi- data ‘evidence’, because they fail a basic test of what dence is the same thing as data from clinical research constitutes evidence: they do not have the capability studies, and that this form of information is superior of supporting an inference. to other forms, which are implied to be ‘non-evi- On the other hand, are there pieces of information dence.’ A conceptual analysis of evidence is a com- that are non-evidentiary under any circumstances? plex topic to which many excellent scholars have What about patient values? How can someone’s devoted extensive discussion. My aim is neither to belief that something is morally right constitute evi- summarize nor do justice to that body of work here. dence? When the definition of EBM mentions inte- Indeed, some authors have concluded that the con- grating ‘patient values’, it is writing from the point of cept of ‘evidence’ may be resistant to a single analysis view of practitioners. That is, the notion of ‘patient applicable to all circumstances in which we use the values’ refers to the fact of people having values, not term (Schum 1994, p. 16). Nevertheless, there are the actual values themselves. The value itself may be some general features of evidence that are worth non-evidentiary, but the fact that someone holds a recalling when considering the different types of certain value may very well be evidentiary in certain medical knowledge and when thinking about what cases. For example, a patient might hold a moral the label ‘evidence-based’ actually means in clinical value that no one should have treatment forcibly decision making. administered, including for severe mental illness, We use several different terms to describe the basis because bodily integrity is a basic good. A practitio- of our clinical decisions, for example, data, informa- ner may know that this particular person holds this tion, intuition, judgement, knowledge and evidence. value so strongly that she or he will fight any form of There are differences in what these terms mean, and forcible medication in hospital and will refuse the these can revealed by the context in which they are treatment when out of hospital. This patient’s values used. For example, a numerical result from an ran- about forcible treatment are certainly not evidence domized control trial may be a piece of data, but it for an inference about the effectiveness of the treat- may not be informative. Or, a patient may reveal a ment one may wish to offer. But, the fact of him or piece of information about herself, such as which her having this value is evidence about an inference

© 2006 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice, 12, 3, 296–298 297

M. Gupta

about whether she or he might want the treatment in borrowing from the perceived credibility of ‘evi- question. Let us further imagine there is a law in dence’ and EBM. Beginning with our various sources place that allows forcible treatment under certain cir- of information on an equal footing also reminds us of cumstances. Again, this is not evidence for an infer- the fallibilism of our knowledge, including what we ence that the treatment works, but it is evidence for ultimately deem to be ‘evidentiary’ (Upshur 2000, p. an inference about what society’s values are in such 96). Forcing all information in clinical decision mak- cases. ing to be scrutinized more closely meets the legiti- These and many other inferences will have to be mate challenge EBM has offered: to be explicit in considered in order to make a clinical recommenda- justifying our medical recommendations. We may tion in this scenario. Here is where Tonelli’s use of never be able to justify every decision, empirically or casuistry as a description of clinical decision making otherwise, but we should try to make explicit the pro- is apt. Casuistry allows us to weigh up different infer- cess through which clinical decision making occurs. ences and facts, and consider the weight of each in light of the particular circumstances of each case. We Acknowledgements are not required to prejudge the value of any piece of knowledge until the context of the scenario is under- The author wishes to gratefully acknowledge the sup- stood. Research data about effectiveness of treat- port of the citizens of Canada, through the Canadian ments can be understood in a more balanced way – Institutes of Health Research, in the preparation of not as a definitive or singular guide to action, but one this manuscript. of many factors that must be considered when mak- ing a clinical recommendation. References If casuistry is a good model for clinical decision making, does it matter if we classify certain inputs as Guyatt G. & Rennie D. (2002) Users’ Guide to the Medi- evidentiary and others as non-evidentiary? Inasmuch cal Literature: A Manual for Evidence-Based Clinical as EBM states that we ought to base our practice Practice. American Medical Association Press, Chi- upon ‘evidence’, it is important to understand what cago, IL, USA. evidence is. EBM has provided little analysis of this Sackett D.L., Straus S.E., Richardson W.S., Rosenberg W. concept, nor has it sufficiently defended its claim & Haynes R.B. (2000) Evidence-Based Medicine: How to Practice and Teach EBM. 2nd edn. Churchill Living- about the inherent superiority of certain kinds of stone, Edinburgh. empirical research data in clinical decision making. Schum D.A. (1994) The study of evidence. In The Eviden- As such, it risks becoming as authoritarian as the tial Foundations of Probabilistic Reasoning, pp. 11–65. practices it eschews. Stripping certain types of infor- John Wiley & Sons Inc, New York. mation of the privileged label of ‘evidence’ requires Upshur R.E.G. (2000) Seven characteristics of medical us to consider the relevance and correctness of each evidence. Journal of Evaluation in Clinical Practice 6 (2), piece of knowledge on its own terms rather than 93–97.

298 © 2006 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice, 12, 3, 296–298

Blackwell Science, LtdOxford, UKJEPJournal of Evaluation in Clinical Practice1356-1294Blackwell Publishing Ltd 2005123281288CommentaryEBM and casuistryR.E.G. Upshur

Journal of Evaluation in Clinical Practice, 12, 3, 281–288

COMMENTARY The complex, the exhausted and the personal: reflections on the relationship between evidence-based medicine and casuistry. Commentary on Tonelli (2006), Integrating evidence into clinical practice: an alternative to evidence-based approaches. Journal of Evaluation in Clinical Practice 12, 248–256

Ross E. G. Upshur BA (Hons) MA MD MSc FRCPC Director, Primary Care Research Unit, Sunnybrook and Women’s College Health Sciences Centre, University of Toronto, Toronto, ON, Canada; Associate Professor, Departments of Family and Community Medicine and Public Health Sciences, University of Toronto, Toronto, ON, Canada

I will argue that this approach is philosophically Introduction untenable and that various potential warrants It is a great honour to be asked to comment on for medical decision-making differ in kind from Dr Tonelli’s paper ‘Integrating evidence into one another. Such an understanding of the clinical practice, an alternative to evidence-based nature of medical knowledge requires an alter- approaches’. Dr Tonelli has consistently been one of native method for integrating various warrants the more thoughtful commentators on the evidence- into particular medical decision, a method that based medicine (EBM) debate and, in fact, published closely resembles the casuistic or case-based one of the first critiques of EBM from a philosophical approach to medical ethics advanced by Johnson point of view (Tonelli 1998). In this paper, I propose and others. to explicate and critique some of the ideas found in I am in agreement with Dr Tonelli that proponents Dr Tonelli’s paper. In addition, I hope to further the of EBM have not fully articulated the philosophical debate by questioning whether EBM and casuistry dimensions of EBM. There is no shortage of other as outlined by Dr Tonelli are oriented more towards thoughtful people variously tied or related to EBM explaining or providing a basis for clinical decision who have provided such a philosophical analysis. making or towards the basis of a philosophy of Notable examples include the work of Tonelli (1998, medicine or the provision of care. I think this is an 2001) himself as well as recent, quite thought- important distinction that has not yet been fully provoking papers both supportive and critical of appreciated, and difficulties arise when one tries to EBM by Sehon & Stanley (2003), and Ashcroft treat them as identical pursuits. (2004) looking at the epistemological nature of evi- dence. There is no doubt that considerable thought has gone into the nature of the critique of EBM as Limitations of EBM: a reprise witnessed by the burgeoning set of thematic issues Dr Tonelli’s central thesis consists of a challenge to devoted towards EBM.1 The Journal of Evaluation of the philosophical assumptions and arguments about Clinical Practice can take pride in that it was the first the nature of medical knowledge as embraced by journal to devote dedicated issues to the concept of EBM. He correctly points out important shortcom- EBM. ings of EBM, particularly in his claim that EBM However, it may be too much for the proponents centres on an attempt to treat differential potential of EBM to provide such a philosophical justification. warrants for medical decision making such as empir- As pointed out by commentators such as Buetow ical evidence, clinical experience and pathophysio- logic rationale as different in degree rather than kind. 1 See for example thematic issues by such journals as Journal of As he states: Medical Ethics, and Brief Psychotherapy and Crisis Intervention.

© 2006 Blackwell Publishing Ltd 281

R.E.G. Upshur

and others (Buetow 2002; Cohen et al. 2004), EBM patients, for instance – attempts to involve other proponents have been unwilling or unable to family members might violate strong cultural embark upon a philosophical examination of the norms (Guyatt et al. 2000). epistemological issues raised by EBM or a debate It is striking that EBM is completely silent on tech- on the relative strengths and weaknesses within niques and tips on how to achieve these important the doctrine. A recent paper by Haynes (2002), for goals, and seems relatively perplexed about how to example, points out the orientation of EBM as less proceed. As Haynes (2002) wrote recently: philosophical and more pragmatic. Apologies for Furthermore, the issue of when a research find- EBM, such as the one by Straus & McAlister (2000), ing is ready for clinical application remains simply gloss over any apparent difficulties. When a mired in the lack of a satisfactory resolution of movement has been successful in such trendy prac- how findings from groups can be applied to tices as ‘branding’ and ‘knowledge translation’ or individuals. For one thing, our understanding of outright marketing, it is scarcely surprising that few how to determine what patients want is primi- among the converted have embarked on the difficult tive. Also problematic, the circumstances in task of thinking through the root assumptions of the which patients are treated can vary widely from most cherished tenets of the creed. That EBM has location to location (including locations that transformed its self-image in an unacknowledged are right across the street from one another): manner, perhaps in response to criticisms, is appar- the resources, expertise and patients are often ent, and several aspects of Dr Tonelli’s paper have quite different and the same research evidence pointed to these changes, as have others (Upshur & cannot be applied in the same way, or not at Tracy 2004). all. Dr Tonelli argues that EBM has failed to provide Tonelli notes the tenuous position of EBM being explicit instructions on how to integrate evidence on the verge of becoming, as he says, ‘simply a into clinical practice. Indeed, the very popular and meaningless appellation to which anyone can claim influential User’s Guide to the Medical Literature allegiance’. ‘Evidence-based’ is something most ends where, some would argue, the really interest- reasonably rational individuals would like to have ing thinking should begin. As stated in the final associated with their professional practices. The dif- instalment: ficulties associated with the restrictive definition of Thus, knowing the tools of evidence-based prac- evidence employed by EBM have been discussed tice is necessary but not sufficient for delivering elsewhere (Feinstein & Horwitz 1997; Malterud the highest quality patient care. In addition to 2002; Upshur & Colak 2003). Whereas Dr Tonelli clinical expertise, the clinician requires com- believes that the concept of an evidence hierarchy is passion, sensitive listening skills, and broad relatively well agreed upon, in my estimation, this is perspectives from the humanities and social not the case. There has been a proliferation of evi- sciences. These attributes allow understanding dence hierarchies that are disparate in their con- of patients’ illnesses in the context of their expe- structions and rating of evidence, some giving higher rience, personalities, and cultures. The sensitive priority to some forms of study, and some to others understanding of the patient links to evidence- (Upshur 2003). It becomes more complex when the based practice in a number of ways. For some concept of a recommendation is integrated into this patients, incorporation of patient values for concept of evidence. Which hierarchy is correct? major decisions will mean a full enumeration of What animates the desire for such a hierarchy? The the possible benefits, risks, and inconvenience idea of an evidence hierarchy has been challenged, associated with alternative management strate- and some have argued that a hierarchy cannot do gies that are relevant to the particular patient. the work it is meant to do, and that the concept of For some of these patients and problems, this evidence as a hierarchically ordered set of warrants discussion should involve the patients’ family. based on the robustness of study design is likely to For other problems – the discussion of screening be a faulted strategy, particularly when one regards with prostate-specific antigen with older male clinical research evidence as but one of many war-

282 © 2006 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice, 12, 3, 281–288

EBM and casuistry

rants available to justify decisions or provide care the analogy between a casuistic approach in ethical (Upshur et al. 2001; Upshur & Colak 2003). thinking and casuistic understanding in clinical deci- Dr Tonelli also correctly points out how EBM is sion making. I think this is a fruitful application, and founded on a form of empiricism and was built on his highlighting the need for prudence or phronesis the explicit rejection of rationalism. The problem is an important insight.2 The refined judgement of a with a purely research-derived conception of evi- clinician entails far more than the application of the dence that privileges randomized designs is that it rules of critical appraisal to studies, and then apply- leads to contradictions that serve as its own ultimate ing those studies to patients. The casuistic model of reductio ad absurdum. For example, one cannot use clinical decision making consists of five elements theory or physical law to reject an evidence derived as outlined by Dr Tonelli. These include empirical from clinical research, no matter how absurd the evidence, experiential evidence, pathophysiologic conclusions. A notable example of this is the prob- rationale, patient values and preferences, and system lematic meta-analyses of homeopathy, published in features. the Lancet, which concluded that homeopathy is Tonelli argues that clinical research can provide a effective. (Linde et al. 1997). Two commentaries paradigmatic case, but does not tell us how to bridge (Langman 1997; Vandenbroucke 1997) that accom- the gap between the studies and the patient at hand. panied the study rejected the findings of the study: As he writes: ‘the clinician must decide whether the one on the ground that homeopathy cannot be true patient at hand resembles the average patient pro- because it violates established physical laws; the vided by clinical research closely enough to warrant other on the ground that even if it were true, it incorporating in the conclusions of that research into would not be worth pursuing on ground of resource the patient’s care. If the patient differs from the aver- allocation. Neither objection is available to support- age study subject, the clinician must decide whether ers of EBM. That homeopathy flies in the face of that difference is important enough to mean the physical laws is not an objection that an EBM advo- conclusions drawn from the study are not relevant to cate can employ with security, as arguments from the current decision’, and he further adds ‘how theory come from lower on the hierarchy than a closely the patient at hand resembles the average meta-analyis, which rests at the pinnacle, regardless patient in the study will determine how much weight of the outcome. Similarly, arguments from a moral to give to the results. perspective have no weight at all in the EBM hierar- But does the casuistic approach provide guidance chy, no matter how cogent or compelling. The only to clinicians on how to integrate research evidence objections that can be raised are methodological, into practice? The question is precisely how to go so one is left with the usual quibbles about the about determining what is best for a patient. Is it nec- adequacy and quality of the individual study designs essary then to look at all possible studies, especially within the meta-analysis, publication bias, and so on, the inclusion and exclusion criteria, to determine if a which is no different from any other meta-analysis, study applies to a patient? EBM itself has largely Cochrane or otherwise. One must be mindful that abandoned this strategy as a desirable way to prac- from the perspective of probability theory, a meta- tice medicine3 And is such an integration of clinical analysis can be ‘positive’ by chance alone, and the research evidence into decision making the overarch- entire edifice of EBM provides no tools to diagnose ing imperative of clinical medicine? If casuistry is a this possibility that are consistent with its own tenets. 2 It has been pointed out that the concept of being a good clinician involves the development of good judgement or clinical wisdom, Is the casuistic approach a departure an area that was at the centre of the work of Alvin Feinstein, who from EBM? devoted considerable energy to the study of what constitutes good clinical judgement. His book Clinical Judgement stands as a coun- The heart of Dr Tonelli’s approach is to integrate terpoint to EBM. 3 Instead, it has enshrined the authority of EBM behind the veil of evidence through a casuistic approach, drawing on pre-appraised evidence sources and the creation of a legion of evi- Jonsen & Toulmin’s (1988) landmark book. He draws dence users as opposed to evidence-based practitioners.

© 2006 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice, 12, 3, 281–288 283

R.E.G. Upshur

guide to the integration of research evidence into the application of clinical study trial evidence to an practice, it may be argued that casuistry serves as an individual patient (Naylor 1995). Even proponents of extension of, rather than alternative to, EBM. Even if EBM, as noted above, admit that this is a problem for we accept that warrants are different in kind, we are which they have no solution.4 still left with a host of difficulties on how to proceed This thorny issue has been nicely examined by both when there is and, when there is not research Richard Horton (2000) In his paper ‘Common sense evidence upon which to guide care. Doctors rarely and figures: the rhetoric of validity’, he points out the have an intimate understanding of such inclusion and problems with various strategies of applying research exclusion criteria. They are poorly represented in evidence in clinical care. Whereas Dr Tonelli does not meta-analyses unless one reads an entire Cochrane provide a worked example of casuistry which would Review, and are not at hand in terms of approaches have strengthened his arguments for its utility such as Clinical Evidence, or other forms of pre- considerably, Horton provides an analysis of the digested evidence where one simply is given the ambiguities of commonly used terms such as gener- bottom line, which is an increasingly advocated alizability, external validity and applicability. Sug- method by EBM. gesting that what doctors are most interested in is How then does one make this leap from under- applicability, he examines reasons why evidence- standing what is paradigmatic to what applies in a based approaches are bound to fall short of being patient’s case? The casuistic model is one attempt to compelling to clinicians. Using the example of coro- traverse this gap, and other such approaches have nary artery stents, he quickly shows that approaches been suggested. The casuistic approach, however, relying solely upon randomized control trials (RCT) does point to the importance of the particular case at will founder when faced by markets flooded with a hand. The importance of context in clinical decision wide range of permutations and combinations of dif- making and clinical care is nicely highlighted in the ferent stents, materials, clinical practice styles and casuistic model. Other approaches such as narrative- techniques to apply them. Instead, he argues for con- based, context-based and patient-centred care all sideration of a strategy of inference derived from the take the point of departure for clinical reasoning as philosophy of Charles Saunders Peirce’s abductive the clinical situation rather than the search for reasoning. One uses the widest range of possible con- evidence. siderations as having standing in deliberation and rig- Evidence-based medicine falters partly because orously tests them in practice. Horton then proposes of its problematic relationship to the patient. It a strategy, based on an interpretive framework for exemplifies what could be termed an extract and decision making. Horton’s strategy is to pose five apply model where the patient is seen as a vehicle questions. These questions are, in the context of an or object from which to extract information. The intervention: clinician searches databanks for methodologically 1 what evidence is available for us to favour sound studies, critically appraises them and then intervention; brings them back for the application to the patient, 2 what evidence is available to us to refute the use and evaluates the outcome. Patients are almost of intervention; complete bystanders to the process of EBM as 3 what evidence is lacking; originally described. As Dr Tonelli rightly points 4 what is an acceptable range of interpretations of out, and is definitely true, personal preference can the available evidence; and trump any form of evidence no matter of what qual- 5 what circumstances and according to what rules is ity, and this does not necessarily result in a bad clin- this evidence being presented and described. ical decision or inferior care. Quite the contrary in Significantly, Horton also argues for the need for a some circumstances. more time-rich environment to make decisions and The crux of the matter is how to traverse the grey provide care. Interestingly, it is time pressure that has zones of practice. As David Naylor has pointed out in his seminal essay ‘Grey zones in evidence-based 4 Except of course to insist that all therapy be given in the context practice’, there are always going to be difficulties of of an n-of-1 trial.

284 © 2006 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice, 12, 3, 281–288

EBM and casuistry

partially scuttled the utility of EBM as classically (though related) issues, and because of this the short- defined. One wonders the same for casuistry. coming of EBM are most evident when one reflects One other dimension that requires highlighting upon what it means to give care, as opposed to what here is the nature of the warrant provided by clinical it means to make a clinical decision. evidence. As Dr Tonelli argues, medical knowledge Consider three illustrative examples. I will refer to has the power to produce hypothetical imperatives of them as the complex, the exhausted and the personal. the form: If you want to maximize the chances of A, The complex refers to the burgeoning population of then do B. The incorporation of patient values is older patients. In my own clinical practice, the major- required to determine whether or not maximizing the ity of patients I see regularly are 75 years and older, chances of A is an appropriate or meaningful goal. many of whom have five or six ongoing chronic ill- Those who hold that there is a moral imperative to nesses such as coronary artery disease, osteoarthritis, practice EBM have failed to successfully reflect upon type 2 diabetes and complex mental health issues the nature of the type of warrants provided by such as depression, early and advanced dementia, clinical research. At best, clinical research evidence insomnia and grief. As we all know, the vast majority gives a plausible direction or provisional justification of pharmaceutical prescribing occurs in this age for action but lacks compelling moral or epistemic group. However, most of these individuals would be force. Even if evidence provided certainty, which it excluded from any form of clinical trial, so it is hard does not, it can be over-ridden in light of other to see how clinical research provides a paradigm for considerations. the care of these individuals. What it requires is some The casuistic approach lends itself to a reflection form of reasonable extrapolation into the hopeful in upon the meaning of clinical enterprise. It is not acci- order to determine whether intervening will provide dental that Stephen Toulmin has a background in more benefit than harm. If one endorses the strategy logic and philosophy of science. His work on argu- of following guidelines as the means of securing mentation theory (Toulmin 1958) has influenced excellent care for this population, problems also writers to provide a model by which the nature of evi- emerge. Guidelines privilege one physiological sys- dence is put into an argumentation framework stress- tem/disease entity at the expense of others. For ing what type of a warrant clinical research evidence example, following the guidelines for the manage- can be. (Dickinson 1998; Upshur & Colak 2003). As ment of osteoarthritis would lead one to the con- with any type of warrant for action, clinical research sideration of a non-steroidal inflammatory treatment evidence is subject to rebuttals and counter-claims, for pain in osteoarthritis. In the elderly, this puts one and hence the concept of providing care or making a hard up against gastrointestinal complications, com- decision is not envisioned as a one-way stream from promise of renal function, and increases the risk in the well-honed cognitive domain of the doctor to the some cases of overt heart failure or myocardial inf- application to the patient, but is, in fact, situated arction. Diabetes guidelines seek to preserve renal within a dialogic context of shared concern. In some function and keep sugar levels optimal in situations writing, it appears that EBM is, in essence, an where less aggressive management may be desired. I approach to enhancing and sharpening the cognitive have yet to sit back and document the many instances skills of doctors. The approach of EBM has been very where guidelines are contradictory within the same one-sided, and its notions of truth and rationality patient, or where optimal compliance with all appro- focus more on that of the clinicians rather than seeing priate guidelines would result in a regimen more tax- the fate of the doctor and the patient as intimately ing than the most rigorous regimen of ascetic monks. intertwined in some form of the unfolding of illness For the most part, these recommendations will apply in the life course of a human (Upshur & Colak 2003). to a patient population excluded from the research This tension is also present in Dr Tonelli’s paper studies that are to form the solid scientific basis for where he writes about EBM and causistry providing judgement. Simply put, neither is there a set of clin- a framework for clinical decision making as well as ical research evidence available to provide direction inconsistently using the term ‘care’. It is quite likely on how to care for these individuals, nor will there that care and decision making are two distinct ever be (at least in my lifetime!). It does require a

© 2006 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice, 12, 3, 281–288 285

R.E.G. Upshur

certain vigilance, trial and error, and a mindfulness of mutually the struggles and challenges that a patient what patients wish and desire as they enter the late faces. If there is something inherently flawed in this years of their life. I think the casuistic approach belief, I welcome the discourse to correct the flaws in offers some help here, but then the question I would my reasoning. pose is: to which paradigms do we look for the care of Evidence-based medicine has been almost patho- these complex elderly people? It has been argued logical in its avoidance of the nature of relationship recently that neither internal medicine nor primary in the provision of care. A focus on decision making care has appropriate models for the provision of care places undue weight on the cognitive structure of a for these complex individuals, and this remains doctor at the expense of looking at how to optimize a challenge to the essence of modern medicine the determinants of good care and understand the (Whitcomb & Cohen 2004). needs of those seeking our care. Issues like trust, The exhausted refers to those individuals who care, therapeutic alliance, being with, and if one have tried every form of care, evidence based or wants to even expand into the role of healing, are otherwise and still suffer from the initial complaint. avoided in this kind of empiricist mode of conceptu- In this regard, I think of one unfortunate patient alizing medicine. I think casuistry has a chance to from my experience as paradigmatic. A gentleman move this forward by realizing that the clinical presented with acute herpes zoster in the T7 nerve encounter between a patient and doctor is embedded distribution. This older man had coronary artery into distinct life courses in an evolving temporal disease and severe chronic obstructive pulmonary horizon. The expectation that an answer is there, or disease. Being good academic doctors, my residents that a solution exists to all problems, is likely an and I tried in sequence every treatment that had unfounded one. Hence, those of us in primary care been shown in meta-analysis to be successful in the are often in the situation of providing care, support treatment of the post-herpetic neuralgia he devel- and lending a helpful ear to those who face oped. Every treatment failed, and we sent this poor intractable and often fatal problems. The provision gentleman to a pain clinic for a nerve block. This of research evidence and the existence of RCTs failed, too. The gentleman laboured in great pain and meta-analysis are irrelevant in some of these with only marginal relief from large doses of narcotic circumstances. analgesic before unfortunately succumbing to a myo- Finally, I am led to a speculation. I wonder what cardial infarction. the generation of evidence-based trained doctors The point here is that having research evidence will look for when it comes to the provision of their does not, in any way, equate with clinical success. own and their family’s medical care. As the cohort Although on average it increases the chances of suc- of doctors ages, will they look for a doctor who is cess in many cases, this does not always translate skilled in the searching and appraising of literature, into success for individual patients. When clinical or will they look for someone who has exemplary success fails to occur, a specialist always ends the diagnostic clinical skills and a humane, compas- consultation note, ‘I return this gentleman to your sionate presence? good care’. When even evidence-based approaches There is a small literature relating to the applica- or any form of specialist care is exhausted, the need tion of evidence-based care to doctors themselves. I to be with the patient remains, even if the only think notably of the article by Chris del Mar in the ‘decision’ left is to endure together. It is not tenable British Medical Journal. This is an amusing but to simply dismiss patients with the solace that there cautionary tale: while sitting in for an objective is nothing that we can do, and increasingly I find structured clinical examination (OSCE), he real- there are more and more conditions that prove ized that there was no urine to be dipped, so he intractable to even the most exemplary medical care. provided his own urine at the OSCE station. He Thus, there are many contexts in which the clinical was found to have trace haemoglobin in his urine. encounter does not result in making any clinical This led him on an interesting odyssey where he decision but are explorations into the understanding tried to determine what the best strategy to follow of a patient and their fate, and trying to understand up for his asymptomatic haematuria would be. He

286 © 2006 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice, 12, 3, 281–288

EBM and casuistry

consulted his clinical colleagues, he looked at the family and consulting trusted colleagues. I eventually literature, and again was faced with what is the most opted for surgical decompression and, to my great pressing issue not addressed by EBM, and this is pleasure, all signs and symptoms resolved, for now. uncertainty. The reality was that the evidence was unclear and did These steps are not easy. Searching the pub- not direct me to an unequivocal answer, and many lished reports is still awkward and time consum- uncertainties remain and will become clear only in ing. Some answers are difficult to find. How the fullness of time. There is, to borrow from long, for example, should we carry on looking Kierkegaard, a certain leap of faith required in mak- before concluding that there seems to be no ing some decisions. published work to guide us? We also need a In conclusion, the casuistic model proposed by forum of peers and those skilled at evidence Tonelli is a fruitful one, and he is to be commended based medicine in which test out our ideas for his ongoing thoughtful and stimulating critiques so that we can reassure ourselves that we are of EBM. The challenges he raises are as much not completely off course. If health authorities epistemic as they are existential and moral. I think are serious about promoting evidence based that Tonelli, following Toulmin and Jonsen, has medicine in clinical practice, they may have opened up a fruitful avenue for future discussion. to consider providing a service (perhaps like Evidence-based medicine arose in the 1990s as an pathology, radiology, or referred specialist iconoclastic movement. One could see them pushing opinions) to help clinicians to take these steps the pendulum far in one direction towards the need (Del March 2000, p. 165). to consider clinical research evidence as central to Quite often there is no clear, certain path. This is the delivery of good health care. I think there has not a failure in EBM, but, as others have commented, been some good associated with this. The iconoclasm is inherent in the process of medical care itself (Mike has certainly caused a reaction, and part of this reac- 1999). tion has been the focusing and sharpening of reflec- I think of my own example. At a relatively young tion upon what the actual nature of medicine is, or age, I started to experience generalized muscle fas- could be. If there are good outcomes from EBM, one, ciculations and leg weakness. After finally heeding I would argue is the thoughtful and reflective writing good (spousal) advice to actually see a doctor, it was exemplified by Dr Tonelli’s paper. determined that I had a cervical myelopathy. In this case, I decided to search the literature for what the References best choice of future therapy would be: medical, Ashcroft R.E. (2004) Current epistemological problems in physical or surgical. To say that the literature was evidence-based medicine. Journal of Medical Ethics 30, confusing and not reassuring is to understate the 131–135. problem. Taking a purely evidence-based approach Buetow S. (2002) Beyond evidence-based medicine: and looking at randomized trials, there are few that bridge-building a medicine of meaning. Journal of Eval- look at the appropriate management of cervical uation in Clinical Practice 8, 103–108. myopathy and compare conservative and operative Cohen A.M., Stavri P.Z. & Hersh W.R. (2004) A catego- approaches. Most patients are older, and the length rization and analysis of the criticisms of Evidence-Based of follow-up is limited to three to five years. Facing Medicine. International Journal of Medical Informatics an uncertain future of a worsening spasticity and leg 73, 35–43. weakness, being told by a colleague that a mere tap Del Mar C. (2000) Evidence-based case report. Asymp- tomatic haematuria . . . in the doctor. British Medical from a rear-end accident would be enough to send Journal 320, 165–166. my cervical disc through my spinal cord, and finding Dickinson H.D. (1998) Evidence-based decision-making: inconsistent results in the literature, none of which an argumentative approach. International Journal of could answer the question of what it really means to Medical Informatics 51, 71–81. have titanium plates in your cervical spine for three Feinstein A.R. & Horwitz R.I. (1997) Problems in the or four decades, I hope, of life expectancy, I came ‘evidence’ of ‘evidence-based medicine’. American up with a process that included discussions with my Journal of Medicine 103, 529–535.

© 2006 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice, 12, 3, 281–288 287

R.E.G. Upshur

Guyatt G.H., Haynes B., Jaeschke R., Cook D., Green Naylor C.D. (1995) Grey zones of clinical practice: some L., Naylor C., Wilson M. & Richardson W. for the limits to evidence-based medicine. Lancet 345, 840–842. Evidence-based Medicine Working Group (2000) EBM: Sehon S. & Stanley D. (2003) A philosophical analysis principles of applying users’ guides to patient care. of the evidence-based medicine debate. BMC Health Based on the users’ guides to evidence-based medicine Services Research 3, 14. (2000). The Journal of the American. Medical Association Straus S.E. & McAlister F.A. (2000) Evidence-based 284 (10), 1290–1296. Available at: medicine: a commentary on common criticisms. Cana- http://www.cche.net/usersguides/applying.asp dian Medical Association Journal 163, 837–841. Haynes R. (2002) What kind of evidence is it that evidence- Tonelli M.R. (1998) The philosophical limits of evidence- based medicine advocates want health care providers based medicine. Academic Medicine 73, 1234–1240. and consumers to pay attention to? BMC Central Health Tonelli M.R. (2001) The limits of evidence-based medicine. Services Research 2, 3. Respiratory Care 46, 1435–1440; discussion 1440–1431. Horton R. (2000) Common sense and figures: the rhetoric Toulmin S. (1958) The Uses of Argument. Cambridge Uni- of validity in medicine (Bradford Hill Memorial Lecture versity Press, Cambridge. 1999). Statistics in Medicine 19, 3149–3164. Upshur R. (2003) Are all evidence-based practices alike? Jonsen A.R. & Toulmin S. (1988) The Abuse of Casuistry. Problems in the ranking of evidence. Canadian Medical University of California Press, Berkely, CA. Association Journal 169, 672–673. Langman M.J. (1997) Homoeopathy trials: reason for good Upshur R. & Colak E. (2003) Evidence and argumenta- ones but are they warranted? Lancet 350, 825. tion. Theoretical Medicine and Bioethics 24, 283–299. Linde K., Clausius N., Ramirez G., Melchart D., Eitel Upshur R. & Tracy C. (2004) Legitimacy, authority, and F., Hedges L.V. & Jonas W.B. (1997) Are the clini- hierarchy: Critical challenges for evidence-based medi- cal effects of homeopathy placebo effects? A meta- cine. Brief Treatment and Crisis Intervention 4, 297–204. analysis of placebo-controlled trials. Lancet 350, 834– Upshur R.E.G., VandenKerkof L. & Goel V. (2001) Mean- 843. ing and Measurement: a new model of evidence in health Malterud K. (2002) Reflexivity and metapositions: strate- care. Journal of Evaluation in Clinical Practice 7, 91–96. gies for appraisal of clinical evidence. Journal of Evalu- Vandenbroucke J.P. (1997) Homoeopathy trials: going ation in Clinical Practice 8, 121–126. nowhere. Lancet 350, 824. Mike V. (1999) Outcomes research and the quality of Whitcomb M.E. & Cohen J.J. (2004) The future of primary health care: The beacon of an ethics of evidence. Evalu- care medicine. The New England Journal of Medicine ation and the Health Professions 22, 3–32. 351, 710–712.

288 © 2006 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice, 12, 3, 281–288