Valuing and Evaluating Evidence in Medicine by Kirstin Borgerson A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy Graduate Department of Philosophy University of Toronto © Copyright by Kirstin Borgerson (2008) Abstract Valuing and Evaluating Evidence in Medicine Kirstin Borgerson Doctor of Philosophy (2008) Department of Philosophy, University of Toronto Medical decisions should be based on good evidence. But this does not mean that health care professionals should practice evidence-based medicine. This dissertation explores how these two positions come apart, why they come apart, and what we should do about it. I begin by answering the descriptive question, what are current standards of evidence in medicine? I then provide a detailed critique of these standards. Finally, I address the more difficult normative question, how should we determine standards of evidence in medicine? In medicine, standards of evidence have been established by the pervasive evidence-based medicine (EBM) movement. Until now, these standards have not been subjected to comprehensive philosophical scrutiny. I outline and defend a theory of knowledge – a version of Helen Longino’s Critical Contextual Empiricism (CCE) – which enables me to critically evaluate EBM. My version of CCE emphasizes the critical evaluation of background assumptions. In accordance with this, I identify and critically evaluate the three substantive assumptions underlying EBM. First, I argue that medicine should not be held to the restrictive definition of science assumed by proponents of EBM. Second, I argue that epidemiological evidence should not be the only “base” of medical decisions. Third, I argue that not only is the particular hierarchy of evidence assumed by EBM unjustified, but that any attempt to hierarchically rank ii research methods is incoherent and unjustifiably restricts medical knowledge. Current standards of evidence divert attention from many legitimate sources of evidence. This distorts medical research and practice. In the remainder of the dissertation I propose means for improving not only current standards of medical evidence but also the process of producing and defending future standards. On the basis of four CCE norms, I argue that we have reason to protect and promote those features of the medical community that facilitate diversity, transparency, and critical interaction. Only then can we ensure that the medical community retains its ability to produce evidence that is both rigorous and relevant to practice. iii Acknowledgements I wish to thank my supervisor, Margaret Morrison, for her good judgment on everything from the scope and content of the thesis to the inner workings of academia. Thanks also to Ross Upshur for his good humour, love of philosophy and heretical insider perspective on EBM, and Barry Brown for sharing his wealth of experience on everything related to alternative health care. In the course of a PhD in philosophy you expect to learn a lot about your topic of research. In addition you hope that you will learn about what it means to be a philosopher and what value philosophy brings to the world. I learned the most about these deeply important issues from my graduate student colleagues. Particular thanks to Joseph Millum, Michael Garnett, Danielle Bromwich, Marika Warren, Vida Panitch and Lauren Bialystok. I would also like to thank my first philosophy professor, Rudy Krutzen, for his persistence; Kathryn Morgan for advocating for me when it really mattered; Heather Boon for her encouragement; Jim Brown for his mentorship; Bob Perlman from Perspectives in Biology and Medicine for believing that two graduate students in philosophy had something important to say and for giving us a venue to say it; Joshua Goldstein for his unwavering support; the EBM infidels, Robyn Bluhm and Maya Goldenberg, who have become not only colleagues and collaborators, but also friends; fellow members of the Varsity Rowing Team – particularly Kiran van Rijn – for sharing sunrises over Lake Ontario; Shannon Urbaniak for setting such a fine example in so many ways; and Murray Enkin for reminding me that there are some advances that have to be made one step at a time, whether in medicine or in life. Thanks also to Joanne Beckett and Dave White, Donna and Neil Larsen, Pat and Trent Cooley, and all the Svenkes. I was generously supported, both financially and intellectually, by several programs while at the University of Toronto. I received funding from the Ontario Graduate Scholarship program on several occasions. In addition, the Comparative Program on Health and Society (CPHS) awarded me two doctoral fellowships and one research associateship and provided me with opportunities to give my first public seminar presentation and to publish my first working paper. I am deeply grateful to everyone at the CPHS for their support. I was also awarded a CIHR Strategic iv Research and Training Doctoral Fellowship in Health Care, Technology and Place in the last two years of my PhD. Through this program I met a number of researchers from other disciplines, many of whom introduced me to new areas of research. I am confident that these encounters will shape my future research for the better. Finally, my deepest thanks go to my family. Thanks to my grandparents, two of whom experienced the worst of health care, and two of whom experienced the best. I hope that my research goes some way to shifting those odds for future patients. Thanks to my mother, Val Drummond, whose intelligence is equaled only by her compassion. I am completing a PhD today because she is such a strong feminist role model and because she has provided me with unconditional support at every stage of my life. Sharing equal credit for phenomenal acts of parenting is my endlessly energetic and incredibly generous father, Lon Borgerson, who always knew I was bright but who pushed me to be more than that: to be creative, to be passionate, and to make a difference. Thanks to my sister, Erika Borgerson, for being my best friend for as long as I can remember and for being there every time I needed to talk. Finally, thanks to my partner, Kieran Cooley, for so many things but most importantly because he always knows how to make me smile. v Contents Valuing and Evaluating Evidence in Medicine Abstract ii Acknowledgements iv Contents vi Introduction 1 1. Why Standards Matter: The ECMO Case 1 2. Standards of Evidence and the Rise of Evidence-based Medicine 4 3. Locating the Project within Philosophy 5 3.1. The Role of Philosophy 3.2. Bioethics and Philosophy of Medicine 3.3. Epistemological Framework 3.4. Context 4. The Argument Ahead 11 4.1. Chapter One: Social Epistemology 4.2. Chapter Two: Critical Contextual Empiricism 4.3. Chapter Three: Evidence-based Medicine 4.4. Chapter Four: Critiquing the Background Assumptions of Evidence-based Medicine: Part I 4.5. Chapter Five: Critiquing the Background Assumptions of Evidence-based Medicine: Part II 4.6. Chapter Six: The Future of Evidence-based Medicine: Diversity and Evidence-based Complementary and Alternative Medicine 4.7. Chapter Seven: The Future of Evidence-based Medicine: Interactive Objectivity and the Open Science Movement 5. Aim of the Dissertation 15 Chapter One Social Epistemology 17 1. Individual and Social Epistemology 17 2. Social Epistemology 18 2.1. Past and Present 2.2. Reasons for Social Epistemology vi 3. Criteria for Choosing a Theory Within Social Epistemology 23 4. Applying the Criteria 26 4.1. Focus on Epistemic Justification 4.2. Preservation of Individualistic Intuitions 4.2.1. Collective Entities 4.2.2. Inter-individual Relationships 4.3. Mechanisms for Managing Social Values 4.4. Robust Relationship between the Rational and the Social 4.4.1. Observation 4.4.2. Reason 5. Critical Contextual Empiricism 34 5.1. Naturalized Epistemology 5.2. Feminist Epistemology 6. Summary 38 Chapter Two Critical Contextual Empiricism 39 1. Constitutive and Contextual Values 40 2. Background Assumptions 41 3. CCE Norms 44 4. Modifications to CCE 45 5. Cultivating Diverse Perspectives 48 5.1. Preliminaries 5.2. Standard Epistemological Argument for Diversity 5.3. CCE Argument for Diversity 6. Shared Public Standards 54 6.1. The Tension 6.2. CCE and Community Standards 6.3. Local Community Standards and Regulatory Standards 6.4. Requirements for Inter-community Debate 6.5. Conclusions 7. Clarification: Three Senses of Knowledge 61 7.1. Agency: Who Knows? 7.2. Content: What Do They Know? 7.3. Warrant: How Do They Prove That They Know? 7.4. The Traditional Triumvirate 7.5. Avoiding Misunderstandings 8. Objections and Replies 65 8.1. Failure to Engage with Messy “Real” World of Science vii 8.1.1. Intuition 8.1.2. Empirical Evidence 8.1.3. History of Science 8.1.4. Contemporary Cases 8.2. Failure to capture what is Distinctive about Scientific Evidence 8.3. Improper Use of Liberal Democratic Principles 8.4. Dogmatism 9. Summary 73 Chapter Three Evidence-based Medicine 75 1. History and Development of Evidence-based Medicine 75 1.1. Rationalism and Empiricism 1.2. Clinical Epidemiology 1.3. Motivations 2. EBM Version One 79 3. The Hierarchy of Evidence 81 4. Epidemiological Terminology 83 4.1. Experimental Designs 4.2. Observational Designs 4.3. General Terminology 5. Assessment of Evidence in Three Stages 86 5.1. Validity 5.2. Importance 5.3. Relevance 6. The Reaction 88 7. EBM Version Two 89 8. The Background Assumptions of EBM 90 8.1. Medicine Should Be More Scientific 8.2. Clinical Research Evidence Should Form the Basis of Medical Decision-making 8.2.1. Not Authority 8.2.2. Not Pathophysiology 8.3. A Hierarchy of Evidence is Necessary 9. Summary 93 Chapter Four Critiquing the Background Assumptions of Evidence-based Medicine: Part I 95 viii 1. Should Medicine Be More Scientific? 95 1.1.