RARE BOOKS LIB. &!:bM The University of Sydney Copyright and use of this thesis This thesis must be used in accordance with the provisions of the Copyright Act 1968. Reproduction of material protected by copyright may be an infringement of copyright and copyright owners may be entitled to take legal action against persons who infringe their copyright. Section 51 (2) of the Copyright Act permits an authorized officer of a university library or archives to provide a copy (by communication or otherwise) of an unpublished thesis kept in the library or archives, to a person who satisfies the authorized officer that he or she requires the reproduction for the purposes of research or study. The Copyright Act grants the creator of a work a number of moral rights, specifically the right of attribution, the right against false attribution and the right of integrity. You may infringe the author's moral rights if you: • fail to acknowledge the author of this thesis if you quote sections from the work • attribute this thesis to another author • subject this thesis to derogatory treatment which may prejudice the author's reputation For further information contact the University's Director of Copyright Services Telephone: 02 9351 2991 e-mail: [email protected] Evidence Based Medicine: Evolution, Revolution, or Illusion? A philosophical examination of the foundations of Evidence Based Medicine Adam LaCaze A thesis submitted for the degree of Doctor of Philosophy Department of Philosophy, The University of Sydney, 2009 ii Evidence based medicine: Evolution, Revolution, or Illusion? Preface Declaration of Originality. The content of this thesis represents my own original contribution. Where the work of others is discussed it is clearly referenced. No part of this thesis has been submitted for another de­ gree. Published Papers. The contents of Chapter 2, Evidence based medicine can't be ... , is to be published in Social Epistemology (LaCaze 2008a). iii iv Evidence based medicine: Evolution, Revolution, or Illusion? Abstract Evidence based medicine (EBM) has become the dominant model for inform­ ing and justifying therapeutic decisions. There is increasing philosophical interest in whether the central claims of EBM can be justified. EBM puts forward a methodological hierarchy as a criterion for justifying therapeutic decisions. This hierarchy privileges evidence from randomised interventional studies as a basis for inference over alternative forms of evidence such as observational studies and basic medical science. Proponents of EBM have provided surprisingly little justification for their methodological claims. The early chapters of the thesis examine the justification and interpretation of the hierarchy provided by proponents of EBM. The only sustainable justi­ fication of the hierarchy is as a hierarchy of comparative internal validity. Study dBsigns higher up the hierarchy have the capacity to rule out more sources of systematic error than study designs lower down. While there are good reasons for preferring randomised interventional studies for testing the efficacy of drugs, high internal validity is not sufficient for informing thera­ peutic decisions. In the later sections of the thesis, I turn to the question of external validity. The crucial role that observational studies and basic sci­ ence play in the application of clinical research is demonstrated. In the final chapters, I argue that some of the frequentist methods currently employed to analyse clinical data are ill suited to the task of informing therapeutic deci­ sions. EBM is promoted as a rationalist turn in medicine. If EBM is to fulfil this promise, more attention is needed on the foundations of the approach. This thesis examines the foundational arguments of EBM, and observes the limits of these arguments in informing therapeutic decisions. v vi Evidence based medicine: Evolution, Revolution, or Illusion? Acknowledgements I have been fortunate to receive much support and advice in completing this work. I especially thank my supervisors. From the project's inception to its current status, Mark Colyvan, my principle supervisor, has been an exceptional guide. I have gained much from witnessing Mark's approach to philosophy, both as an enjoyable activity of intrinsic merit and as a profession. I am a grateful beneficiary of Mark's generosity, optimism and wisdom. Jason Grossman, my associate supervisor, has been an tireless source of helpful advice and optimism. I am indebted to Jason for his passionate and patient guidance through the challenging terrain of philosophy of statistics. Jason's expertise has benefited the thesis immensely. Katie Steele has been my unofficial third supervisor and confidant. Katie graciously took me under her wing early in my PhD. Our many conversations on matters of formal and informal philosophy have enlightened me as much as I have enjoyed them. For coffee, conversations, ideas and inspiration I thank Neil Cottrell, Damian Cox, Chris Cutts, Steve Duffull, Fiona Fidler, Christopher Crannell, Paul Griffiths, Margeurite La Caze, Aidan Lyon, Frank May, John Matthew­ son, Hatha McDivitt, Fabien Medvecky, Lisa Nissen, Debra Rowett, Nick Shaw, Sue Tett, Neil Thomason, Scott Tyler, Karl Winckel, Bonnie Wintle, and John Worrall. I have worked with a number of excellent pharmacists and other health professionals who have shaped the questions broached in this thesis. In par­ ticular, I thank Chris Cutts, Neil Cottrell, Ian Coombes, Steve Duffull, Debra Rowett and Frank May. Through previous work I had the opportunity to meet regularly with a group of general practitioners working in rural Queens­ land, I thank them for sharing their insights on the challenges of therapeutic decision making. Fabien Medvecky, Karl Winckel and Scott Tyler kindly read and com- Vll viii Evidence based medicine: Evolution, Revolution, or Illusion? mented on sections of the thesis in its current form. For long-term support, and for supplying all the preconditions that are impossible to list, I thank David, Maureen, Megan, Damien and Sarah. And, for her unconditional and unwavering support, I thank Sanam. Prologue Rofecoxib was a heavily marketed, and widely used, anti-inflammatory agent that was withdrawn from the worldwide market in September 2004. Now, rofecoxib is a well recognised, and widely used, example of the challenges of regulatory and clinical decision making. The rofecoxib story is striking. Rofecoxib was withdrawn following post­ marketing evidence that it increased the risk of heart attacks and strokes. Prior to this, rofecoxib had progressed through the phases of drug develop­ ment without significant hitch. That is, its risks were not uncovered until after it had passed what we accept to be the 'best tests' of a drug's effi­ cacy and safety. But this is not what is especially striking about rofecoxib. Despite the rigours of drug development, some rare, though catastrophic, ad­ verse effects can only be detected once a drug is used by a very large number of people. What is especially striking in this case is that the risks of rofe­ coxib are relatively common, both in terms of presentation and frequency. The cardiovascular effects of rofecoxib are not entirely idiosyncratic. Indeed, the possibility of an increased risk of blood clots (leading to heart attacks or strokes) is foreseeable on pharmacological grounds. And, based on current data, the frequency of heart attacks and strokes caused by rofecoxib is in the range of 30 to 80 additional events per 10,000 patients per year; by con­ trast, a 'rare' adverse effect, say, for example penicillin anaphylaxis, occurs in about one patient in every 10,000. Even more disturbing is that rofecoxib was developed in anticipation of its safety benefits. In many ways rofecoxib, and the class of drugs it belongs to, had all the hallmarks of being a success story for biomedical science. Tra­ ditional anti-inflammatory agents, despite their effectiveness in relieving pain and inflammation, pose significant public health risks due to their propen­ sity to cause gastrointestinal damage. The COX-2 inhibitor class, of which rofecoxib is a member, were developed out of an increased pharmacological IX X Evidence based medicine: Evolution, Revolution, or Illusion? and physiological understanding of how anti-inflammatories work. COX-2 inhibitors are pharmacologically targeted to work just as well as traditional anti-inflammatories while minimising gastrointestinal risks. On this basis, these drugs were eagerly awaited, and once available, heavily prescribed. The anticipated improvements in gastrointestinal safety were eventually es­ tablished. Unfortunately, the cardiovascular risks identified a couple of years later are of a similar magnitude to the gastrointestinal benefits. Rofecoxib was approved, and spent four and a half years on the market, before its propensity for grave side effects was established. Clearly, this is an undesirable state of affairs. A number of questions arise immediately, perhaps the most urgent among these is: what went wrong? The answer is unsettling; in a strong sense, nothing went wrong. Rofecoxib was tested according to the methods we accept to be our 'gold standard'. Specifically, a significant number of reasonably sized randomised controlled trails con­ ducted prior to and soon after market approval failed to demonstrate the cardiovascular risks. The VIGOR study (Bombardier et al. 2000) provided some (disputed) evidence that rofecoxib may increase the risk of heart at­ tacks and strokes, but it