Standards for the Design, Conduct, and Evaluation of Adaptive Randomized Clinical Trials Michelle A. Detry, PhD, 1 Roger J. Lewis, MD, PhD, 1-4 Kristine R. Broglio, MS, 1 Jason T. Connor, PhD,1,5 Scott M. Berry, PhD, 1 Donald A. Berry, PhD 1,6 1. Berry Consultants, LLC, Austin, TX 2. Department of Emergency Medicine, Harbor-UCLA Medical Center, Torrance, CA 3. David Geffen School of Medicine at UCLA, Los Angeles, CA 4. Los Angeles Biomedical Research Institute, Torrance, CA 5. College of Medicine, University of Central Florida, Orlando, FL 6. Department of Biostatistics, The University of Texas MD Anderson Cancer Center, Houston, TX Revision Date: March 15, 2012 Contact Information: Donald A. Berry, PhD Berry Consultants, LLC, 4301 Westbank Drive, Bldg B, Suite 140, Austin, TX 78746 Telephone: 713-817-5586 Email: [email protected] DISCLAIMER All statements in this report, including its findings and conclusions, are solely those of the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute (PCORI), its Board of Governors or Methodology Committee. PCORI has not peer-reviewed or edited this content, which was developed through a contract to support the Methodology Committee’s development of a report to outline existing methodologies for conducting patient- centered outcomes research, propose appropriate methodological standards, and identify important methodological gaps that need to be addressed. The report is being made available free of charge for the information of the scientific community and general public as part of PCORI’s ongoing research programs. Questions or comments about this report may be sent to PCORI at [email protected] or by mail to 1828 L St., NW, Washington, DC 20036. Minimum Standards for Adaptive Trials Table of Contents Introduction .......................................................................................................................... 1 Literature Review ................................................................................................................ 3 Minimum Standards ........................................................................................................... 5 Minimum Standard 1. Explicit Prospective Specification of Planned Adaptations and Primary Analysis ............................................................................................................................... 7 Minimum Standard 2. Evaluation of Statistical Properties ...................................................... 10 Minimum Standard 3. Standards for Bayesian Adaptive Randomized Clinical Trial Designs ......................................................................................................................................................... 15 Minimum Standard 4. Communication and Vetting of Trial Design with Key Stakeholders 18 Minimum Standard 5. Ensure Clinical Trial Infrastructure is Adequate to Support Planned Adaptation(s) ................................................................................................................................. 21 Minimum Standard 6. Consideration of Operational Bias ........................................................ 24 Minimum Standard 7. Ensure Proper Oversight of Adaptive Randomized Clinical Trial ...... 29 Minimum Standard 8. The Reporting of Adaptive Randomized Clinical Trials Should be Consistent with the CONSORT Statement ................................................................................... 32 Gaps in Knowledge ............................................................................................................ 39 Discussion ............................................................................................................................ 41 Appendix 1: Adaptive Design Terminology ................................................................ 45 Appendix 2: PCORI Literature Review Abstraction Tool: Items Collected ........ 49 Appendix 3. Table of Guidance Documents ................................................................ 50 Acknowledgments ............................................................................................................. 51 References ........................................................................................................................... 52 Page ii Minimum Standards for Adaptive Trials Introduction Patient-centered outcomes research (PCOR) is intended to help physicians and patients make better treatment decisions using comparative information regarding the effectiveness of commonly used treatments in typical care settings.1, 2 Comparative effectiveness research (CER) is defined by the Institute of Medicine as “the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care.” 3 While CER can include many types of studies 4, including traditional randomized clinical trials (RCTs), adaptive RCTs hold particular promise for PCOR. 5-8 An adaptive clinical trial is one in which key trial characteristics (e.g., randomization proportions, sample size, treatment arms, eligibility criteria) evolve according to prespecified rules during the trial, in response to information accruing within the trial itself. Goals of this approach include higher statistical efficiency, improved patient outcomes, or better ethical balance. 9-11 There is a range of roles for adaptive trials, including dose-finding trials, personalized medicine trials in which treatments are developed to be tailored to specific patient traits, and informative comparative effectiveness trials. Adaptive approaches can lead to more effective treatments for participants in the CER trial itself, but also to better treatment of future patients having the disease or condition in question. Adaptive trials are ideal for PCOR because enrollment criteria may be altered to focus on patient subpopulations of particular interest. Appendix 1 lists common terminology related to adaptive trial design. Adaptive designs are not new to clinical trials. Group sequential designs have Page 1 Minimum Standards for Adaptive Trials been in use for decades. However, in recent years, increased emphasis has been placed on streamlining drug development, including by regulatory agencies. 12 In 2010, the Center for Drug Evaluation and Research and the Center for Biologics Evaluation and Research of the Food and Drug Administration (FDA) issued a draft guidance for industry regarding the use of adaptive designs for “adequate and well controlled” phase III trials. 13 This guidance supplemented a similarly focused 2007 “Reflection Paper” from the European Medicines Agency.14 On the medical device side, since the passage of the FDA Modernization Act of 1997 15 the FDA’s Center for Devices and Radiologic Health has been encouraging Bayesian adaptive approaches and issued its Bayesian guidance for medical device trials in 2010. 16 Innovations in adaptive trial design for drug and medical device development have led to highly flexible approaches that extend in a natural way to PCOR. The standard, modern approach to clinical trial design is steeped in hypothesis testing. This tradition is important and it has transformed medical research from case studies and clinical experience into a scientific discipline. But hypothesis testing is not as applicable to CER and PCOR as it is to the medical product registration setting because, estimation of treatment effects is of greater importance than simply determining superiority. Adaptive trial designs do not abandon the tradition of hypothesis testing, but controlling type I and type II errors may be secondary when addressing questions such as, “How do we build a trial with a goal of treating patients as effectively as possible, including those in the trial as well as those who present later?” Recognizing the need for innovation in clinical trial design, representatives Page 2 Minimum Standards for Adaptive Trials from Clinical and Translational Science Award programs have identified adaptive clinical trial design as a high priority methodological issue “to increase the efficiency of comparative effectiveness trials.”17 In October 2011, the Patient-Centered Outcomes Research Institute requested proposals to review applicable guidance documents and available literature and to propose minimum standards for the design, conduct, and reporting of adaptive clinical trials. This report was produced in response to that request. In this report, we describe the results of a comprehensive review of existing literature relative to adaptive RCTs and propose standards for the design, conduct and reporting of adaptive RCTs in PCOR. We augment published information based on our experience in designing and helping to conduct adaptive RCTs. We identify existing gaps in the literature that currently preclude identifying standards in specific areas. Literature Review We conducted a literature search with three goals. The first goal was to identify any applicable guidance or draft guidance documents from regulatory bodies or government agencies. We identified three of these, as mentioned above.13, 14, 16 Although adaptive design is not the main focus of the Bayesian guidance from FDA’s Center for Devices and Radiologic Health,16 the Bayesian approach is inherently adaptive, and thus the guidance addresses general concepts for adaptive clinical trials. Page 3 Minimum Standards for Adaptive Trials