Drug and Therapeutics Committee Training Course Session 3. Assessing Medicine Efficacy Participants’ Guide Drug and Therapeutics Committee Training Course—Participants’ Guide This document was made possible through support provided by the U.S. Agency for International Development, under the terms of cooperative agreement number HRN-A-00-00- 00016-00. The opinions expressed herein are those of the author(s) and do not necessarily reflect the views of the U.S. Agency for International Development. About RPM Plus RPM Plus works in more than 20 developing and transitional countries to provide technical assistance to strengthen pharmaceutical and health commodity management systems. The program offers technical guidance and assists in strategy development and program implementation both in improving the availability of health commodities—pharmaceuticals, vaccines, supplies, and basic medical equipment—of assured quality for maternal and child health, HIV/AIDS, infectious diseases, and family planning, and in promoting the appropriate use of health commodities in the public and private sectors. Recommended Citation The materials may be freely abstracted, quoted and translated in part or in whole by non-profit or educational organizations (for reference or teaching only) provided the original source is acknowledged. They should not be sold nor used for any other commercial purpose. Management Sciences for Health and World Health Organization. 2007. Drug and Therapeutics Committee Training Course. Submitted to the U.S. Agency for International Development by the Rational Pharmaceutical Management Plus Program. Arlington, VA: Management Sciences for Health. Rational Pharmaceutical Management Plus Center for Pharmaceutical Management Management Sciences for Health 4301 North Fairfax Drive, Arlington, VA 22203 USA Phone: 703.524.6575 Fax: 703.524.7898 E-mail: [email protected] Web: www.msh.org/rpmplus Developed in Collaboration with the World Health Organization Geneva, Switzerland ii ABBREVIATIONS AND ACRONYMS ACS acute coronary syndrome CI confidence interval DTC Drug and Therapeutics Committee HR hazard ratio ITT intention to treat MI myocardial infarction n or N number NNT number needed to treat RCT randomized controlled trial RRR relative risk reduction iii Drug and Therapeutics Committee Training Course—Participants’ Guide iv CONTENTS Session 3. Assessing Medicine Efficacy ......................................................................................... 7 Purpose and Content ................................................................................................................... 7 Objectives ............................................................................................................................... 7 Preparation and Materials ....................................................................................................... 7 Further Readings ..................................................................................................................... 8 Introduction ................................................................................................................................. 8 Formulating the Question ........................................................................................................... 9 Finding the Evidence .................................................................................................................. 9 What Sort of Evidence? ............................................................................................................ 10 Assessing the Quality of the Evidence—What Makes a Good Clinical Trial? ........................ 10 Why Was the Study Done? What Clinical Questions Were the Authors Addressing? ........ 12 What Type of Study Was Done? .......................................................................................... 12 Was the Study Design Appropriate to the Research Question? ............................................ 14 Was the Design of the Study Sensible? ................................................................................ 15 Who Is Studied? .................................................................................................................... 16 How Well Was the Study Conducted? Was Systematic Bias Avoided or Minimized? ....... 16 Quality of Randomized Controlled Trials: What to Look for ............................................... 17 Randomization and Concealment of Allocation ................................................................... 17 Double-Blind vs. Open Trials ............................................................................................... 18 Inclusion of All Patients in the Statistical Analysis .............................................................. 18 Quality of Non-Randomized Trials—What to Look for .......................................................... 19 Understanding the Numbers ..................................................................................................... 20 What Did the Authors Think They Would Find? ................................................................. 20 What Were the Results? ........................................................................................................ 20 Different Types of Data Need Different Statistical Tests ..................................................... 21 Understanding Metrics or the Bottom Line .......................................................................... 23 What are the Standard Statistical Tests? ............................................................................... 26 Summary Statement on Single Trials ....................................................................................... 29 Systematic Reviews and Meta-Analysis ............................................................................... 29 Search Strategies and Inclusion Criteria ................................................................................... 30 The Use of Systematic Reviews and Meta-Analysis ................................................................ 31 The Use of Sensitivity Analysis in Interpreting the Results ................................................. 32 Interpreting Inconsistent Results (Heterogeneity) ................................................................ 32 Summary of Systematic Reviews ......................................................................................... 33 v Drug and Therapeutics Committee Training Course—Participants’ Guide Use of Evidence in Decision-Making ....................................................................................... 33 Activity 1. Comparing Antimicrobial Medicines for Pneumonia ............................................. 34 Activity 2. Interpreting the Data: The Helsinki Heart Study .................................................... 35 Activity 3. Critically Evaluating an Article .............................................................................. 35 Activity 4. Interpreting the Data: A Medicine Trial to Compare Artesunate with Mefloquine to Treat Malaria .......................................................................................................... 35 Annex 1. Glossary ..................................................................................................................... 38 Annex 2. Checklist to Detect Common Problems Encountered in Articles ............................. 41 vi SESSION 3. ASSESSING MEDICINE EFFICACY Purpose and Content Session 3 is designed to provide participants with a basic guide on how to determine medicine efficacy, primarily through review of the pharmaceutical literature with an emphasis on evaluating randomized controlled trials (RCTs), systematic reviews, and meta-analyses. Systematic and thorough evaluations of the pharmaceutical literature will provide the Drug and Therapeutics Committee (DTC) with the unbiased information necessary to select appropriate medicines for the formulary. In most countries, evaluating the pharmaceutical literature is commonly done by physicians and pharmacists. Unfortunately, this review is often done incorrectly. With the tools presented in this session and practice at home, participants will be better equipped to evaluate the literature systematically and scientifically. Objectives After attending this session, participants will be able to— • Understand the importance of determining efficacy and evaluating the clinical literature • Discuss the major types of medicine study design • Describe the key components of a journal article • Understand how to evaluate and interpret results of a randomized controlled trial • Discuss the use of systematic reviews and meta-analyses in evaluating medicines Preparation and Materials Read the following— • This session—A thorough understanding of this session is necessary before the presentation. • Review session 2, “Developing and Maintaining a Formulary” of the Participants’ Guide. • Activity article—Randomized trial of artesunate and mefloquine alone and in sequence for acute uncomplicated falciparum malaria. 7 Drug and Therapeutics Committee Training Course—Participants’ Guide Further Readings Gehlbach, S. H. 1993. Interpreting the Medical Literature 3rd ed. New York: McGraw-Hill. Greenhalgh, T. 2006. How to Read a Paper: The Basics of Evidence-Based Medicine 3rd ed. Blackwell Publishing. Grimes, D. A., and K. F. Schulz. 2002. An Overview of Clinical Research: The Lay of the Land. Lancet 359: 57–61. Kirkwood,