Final Research Report
Total Page:16
File Type:pdf, Size:1020Kb
New Methods and Software for Designing Adaptive Clinical Trials of New Medical Treatments Michael Rosenblum1, PhD, Jon Arni Steingrimsson2, PhD, Josh Betz1, M.S., Affiliations: 1Department of Biostatistics, Johns Hopkins University, Baltimore, MD 21205, USA 2Department of Biostatistics, Brown University, Providence, RI 02903, USA Original Project Title: Innovative Randomized Trial Designs to Generate Stronger Evidence about Subpopulation Benefits and Harms PCORI ID: ME-1306-03198 HSRProj ID: 20143600 Institution: Johns Hopkins University _______________________________ To cite this document, please use: Rosenblum M, Steingrimsson JA, Betz J. (2019). New Methods and Software for Designing Adaptive Clinical Trials of New Medical Treatments. Washington, DC: Patient- Centered Outcomes Research Institute (PCORI). https://doi.org/10.25302/10.2019.ME.130603198 1 Table of Contents Abstract ............................................................................................................................................... 3 Background ......................................................................................................................................... 4 Specific Aims ................................................................................................................................... 7 Participation of patients and/or other stakeholders in the design and conduct of research and dissemination of findings .................................................................................................................... 7 Methods .............................................................................................................................................. 8 Aim 1: Develop and evaluate adaptive enrichment designs for time-to-event and other delayed endpoints. ....................................................................................................................................... 8 Aim 2: Conduct extensive simulation studies. .............................................................................. 12 Aim 3: Produce user-friendly, free, open-source software to optimize our adaptive enrichment designs and compare performance versus standard designs. ...................................................... 22 Discussion ......................................................................................................................................... 24 Study results in context: ............................................................................................................... 24 Uptake of study results: ................................................................................................................ 24 Study limitations: .......................................................................................................................... 25 Future research: ............................................................................................................................ 25 Conclusions: ...................................................................................................................................... 26 References ........................................................................................................................................ 27 Related Publications Published Manuscripts: ................................................................................... 30 Acknowledgments: ........................................................................................................................... 31 2 Abstract Background: Standard clinical trial designs aim to determine whether a treatment is beneficial, on average, for a target population. Such trials can have low power if the treatment only benefits a subpopulation, e.g., defined by disease severity, a biomarker, or a risk score at baseline. Randomized trial designs that adaptively change enrollment criteria during a trial, called adaptive enrichment designs, have potential to provide improved information about which subpopulations benefit from new treatments. Objectives: We aimed to (i) develop new adaptive enrichment designs and prove their key statistical properties; (ii) conduct simulations that mimic features of completed trial data sets in order to evaluate the new trial designs’ performance (such as sample size, duration, power, bias); the data sets are from trials involving treatments for HIV, stroke, and heart failure; (iii) develop user-friendly software for optimizing the performance of our new adaptive designs and comparing them to standard designs. The goal was to construct designs that satisfy power and Type I error requirements at the minimum cost in terms of expected sample size, i.e., average sample size over a set of plausible scenarios. We also considered the maximum sample size, i.e., the number of participants enrolled if there is no early stopping. Methods: We constructed new adaptive trial designs (including new rules for modifying enrollment and new procedures for testing multiple hypotheses) and proved key statistical properties such as control of the study-wide Type I error rate. Results: For the simulation study involving stroke, the new adaptive design reduced expected sample size by 32% compared to standard designs; the tradeoff is that the maximum sample size was 22% larger for the adaptive design. For the simulation study involving the cardiac resynchronization device for treating heart failure, the benefit of the adaptive design was a 2 5 % reduction in expected sample size but an 8% increase in maximum sample size versus standard designs. For the simulation study involving HIV, the adaptive designs did not provide substantial benefits. Conclusions: Optimized, adaptive enrichment designs can lead to reduced expected sample size compared to standard designs, in some settings. For adaptive enrichment to substantially add value, a sufficient number of primary outcomes need to be observed before enrollment is exhausted; this depends on the enrollment rate and the time from enrollment to observation of the primary outcome. Adaptive designs often involve tradeoffs such as reduced expected sample size at the price of greater maximum sample size, compared to standard designs. Our software can reveal these tradeoffs and determine whether certain adaptive enrichment designs substantially add value for a given trial design problem; this enables trial statisticians to make informed decisions among trial design options. Our designs assumed that subpopulations are defined before the trial starts, which requires prior data and scientific understanding of who may be more likely to benefit from the treatment. The sample size required to determine treatment effects for subpopulations can be substantially greater than for the overall population. 3 Background Adaptive designs involve preplanned rules for modifying how the trial is conducted based on accruing data. For example, adaptations could be made to the number enrolled, the probability of being randomized to treatment or control, the inclusion criteria, the length of follow-up, etc. According to the Patient-Centered Outcomes Research Institute (PCORI) Methodology Report1 “Adaptive designs are particularly appealing for PCOR because they could maintain many of the advantages of randomized clinical trials while minimizing some of the disadvantages.” We focus on one type of adaptive design called adaptive enrichment designs. Adaptive enrichment designs involve preplanned rules for modifying enrollment criteria in an ongoing trial.2 They typically involve multiple, pre-planned stages, each ending with an analysis of the cumulative data and a decision as to enrollment in the subsequent stage. These designs have potential to learn more about treatment effects in subpopulations.3 For example, enrollment of a subpopulation for which there is sufficient evidence of treatment efficacy, futility, or harm could be stopped, while enrollment continues for the remaining subpopulations. Figure 1, based on a similar figure in our paper4, gives a schematic of a 2- stage adaptive enrichment design in the context of a trial to evaluate a surgical treatment for stroke in 2 subpopulations. 4 Figure 1:4 Schematic of 2 stage adaptive enrichment design. 2 Stage Adaptive Enrichment Design Flow of Enrollment and Decision Stage 1 Stage 2 Enroll Both Pop. Enroll Both Subpopulation 1 Subpopulations Option 1 Subpopulation 2 Subpopulation 1 Enroll Only Subpop.1 Subpopulation 2 Option 2 Subpopulation 1 Enroll Only Subpop.2 Option 3 Subpopulation 2 Option 4 Trial A decision is made after stage 1 to: (1) continue enrolling both subpopulations; (2) enroll only subpopulation 1; (3) enroll only subpopulation 2; (4) stop the trial. The adaptive enrichment designs considered in this paper generally involve more than 2 stages, where similar decisions as in this figure are made at the interim analysis after each stage using the cumulative data available. We developed new statistical methodology and an open-source, freely available, software tool that optimizes new adaptive enrichment designs and compares their performance (via simulation) versus standard designs. Our designs aimed to determine treatment benefits and harms for subpopulations defined by a risk factor such as age, disease severity, or a biomarker measured at baseline. We also assessed tradeoffs involved in using adaptive enrichment designs versus standard designs. Our project addressed research priorities of the U.S Food and Drug Administration