ADAPTIVE CLINICAL TRIALS

P. Ranganath Nayak, Chief Executive Officer, Cytel Inc James A. Bolognese, Senior Director, Biostatistics, Cytel, Inc

The Adaptive Concept The clinical development of drugs and devices is a challenging task, and more so than product development activity in most other industries. The technical challenges are huge because understanding of the biological systems involved is complex and usually incomplete. The complexity is significant because patients can vary substantially from one another and their responses are difficult to predict. Additionally, every sponsor has to contend with the need for speed because of competition and the existence of budgetary pressures arising from the need to get a good return on investment in clinical development. The result is well known: success rates overall are low, New Drug Applications are too few, and the return on investment is abysmal. See Exhibit 1 for data on problems in clinical development.

A highly promising approach to re-thinking how clinical development is done is the adaptive clinical trial. What is it? In a conventional or “fixed” trial, all of the important details of the trial are decided before the trial starts, and are then not changed. The trial is completed, the data is analyzed, and if the results are positive, an NDA is submitted to the regulatory authorities. Unfortunately, in a large percentage of Phase 3 or “confirmatory” trials, the results are not positive, the trial is a failure, and the investment is unproductive. In an adaptive trial, in contrast, it is possible to look at the accruing data from patients at one or more times (called “interim looks” at the data) as the trial is on-going, and adjust certain parameters of the trial in order to increase the probability of success when the trial comes to an end.

See Exhibit 2 for some of the adaptations now being used.

Benefits of Adaptive Trials The benefits of adaptive trials fall into three categories: commercial, ethical, and budgetary.

The commercial benefit is that much better information is obtained on the drug and its dosage, resulting in a much greater chance that the phase III confirmatory trials will be successful, with a reduced probability that the phase III dose will either be toxic or show inadequate efficacy. Exhibit 3 illustrates the potential for better information to reduce the rate of failure in Phase 3.

Exhibit 1. Problems in clinical development  The biopharmaceutical industry worldwide spends $130 - $140 Billion per year on the clinical development of drugs, up from $33 Billion in 1999. Projected growth over the next 3 years is 2% [1].  The average R&D cost per approved new molecule is approximately $899 million for pharmaceuticals, while the R&D costs per approved new molecule for biotechs is over $1.2 billion [2] and [3].  The number of NCEs and New BLAs approved by the FDA was 35 in 1999 and 20 in 2008 [4]. 27 new drugs were approved by the FDA in 2013, as compared with 39 in 2012 and 30 in 2011. These include diagnostic and imaging agents. 24 new therapeutic drugs were approved in 2013, compared with 35 in 2012 and 28 in 2011 [5] and [6].  Over 60% of phase 2 trials and between 30% - 40% of phase 3 trials fail [7].  93% of trials in oncology fail, with 50% of failures in phase 3 [7].  45% of phase 3 industry trials have the wrong dose [8]. In addition, a new JAMA study shows that nearly 16% of trials which initially fail to gain approval, (but which are eventually approved,) fail initially because of improper dose selection [9].  Although drug development times have reportedly been decreasing in the past two years, evidence suggests that this is due primarily to changes in R&D portfolios rather than any significant improvements in development times [10].  Obtaining accurate estimates of variance in patient response data can reduce sample size by as much as 50% [11].  Wasted medical supplies in typical conventional trials are 30% to 60% of the optimum. In an adaptive dose-finding trial, conventional planning methods can yield overages of 300% to 500% [12]. In some oncology trials, the cost of medical supplies can be as much as $100,000 per patient per year.  10% to 30% of clinical sites never recruit a patient, but nevertheless incur a set-up cost of around $70,000 per site. Data are not analyzed during the implementation stage of a trial to identify these sites [13].  80% of trials registered at clinicaltrials.gov fail to meet enrollment timelines; and 50% of sites enroll 1 or 0 patients [14].  The largest single cost in clinical trials is site monitoring—as much as 30% of the total. However, real-time data are not available to optimize visit schedules, leading to large amounts of money wasted [15].

Exhibit 2. Adaptive designs currently in use Continual Reassessment Method and modified CRM designs (Phase 1) Simon 2-stage designs and modifications thereof (Screening) Seamless phase 1 / 2 designs (Dose-toxicity and early Proof-of-Concept) Adaptive designs for drug combination therapies Response-adaptive designs (for Proof-of-Concept and dose-ranging) Frequentist dose-response modeling Bayesian dose-response modeling Group sequential designs with early stopping for efficacy, harm or futility Sample size adjustments (Phase 3) Seamless phase 2b/3 designs (dose-selection plus confirmatory) Dropping / Adding doses (Phase 3) Population enrichment (Phase 3) 2-stage bioequivalence designs.

Exhibit 3:

Source: John Arrowsmith and Philip Miller ‘Trial Watch: Phase II and Phase III attrition rates 2011 – 2013,’ Nature Review Drug Discovery 12.8 (2013): 569 – 569. http://www.nature.com/nrd/journal/v12/n8/images_article/nrd4090-f1.jpg

Another commercial benefit is that the time-to-market for a drug can be reduced by combining proof-of-concept trials (to show that the drug works) with dose- finding trials (to pick the right dose) or by combining dose-finding trials with confirmatory trials. This reduces or eliminates the “lost” time (also known as “white space”) between the end of one trial and the startup of the next. Being faster to market increases the Net Present Value (NPV) of the product.

The ethical benefit is that in both early-stage and late-stage trials, more patients will receive doses that work because doses with inadequate efficacy are adapted out. This benefit is both immediate (early stage) and deferred (late stage).

The budgetary benefit is that, frequently, the number of patients required in a phase II trial can be reduced. See Exhibit 4 for an illustrative example from Eli Lilly.

Exhibit 4: Sample Size Reduction with Adaptive Designs

At its core, the idea behind adaptive trials is simple and compelling. Reduce the uncertainty in clinical development that leads to flawed trial designs by getting additional information (beyond what one had at the beginning of the trial) from the ongoing trial. Additional information is always better, leading to better decisions about what to do during the course of clinical development. More opportunities to refine decisions are also a benefit. See Exhibit 5.

Exhibit 5:

In practice, however, adaptive designs have attendant challenges so that the adaptive route may not be the right one for every trial. A thorough evaluation is needed to determine in any given instance the best choice of design, whether adaptive or not.

Adaptive or Not? It is useful to think of the decision on whether to make a trial adaptive or not as passing through three gates with gatekeepers who may not let you through.

The first gate is owned by regulatory agencies. One needs to show them the logic for doing an adaptive design and, if they accept the logic, that the statistics have been done correctly (“demonstrate control of the Type I error”), and if they accept that, that the possibility of bias (both intentional AND unintentional) and manipulation of the trial have been minimized. See Exhibit 6 for a summary of regulatory concerns.

Exhibit 6. Regulatory Concerns – A Summary

Rationale for adaptive – is it a cover for sloppiness in design or is there a provable need and benefit? Control of Type I error. “Show me.” Adaptations defined in advance Protocol for interim monitoring known in advance, proves that potential for intentional bias is virtually eliminated Appropriate firewalls in place to maintain appropriate blinding of accumulating study results After-the-fact proof that the protocol and interim monitoring procedures were followed (audit capability)

The clinical operations team owns the second gate. They will want to know whether all of the operational challenges of an adaptive trial have been dealt with. These range from the need to make accurate data available on time for interim analyses, to the need to ensure there is adequate medical supply at all sites (but not too much, for overages could be very expensive), and to the need to put in place an interim monitoring body such as a Data Monitoring Committee and make sure that the “unblinded” data they see are not seen by those who should not.

The finance staff owns the third gate. They will want to know if the adaptive trial is going to cost more or less than a conventional trial and, if it is potentially more, why the added cost is justified.

We find it useful to take these three gates and convert them into ten questions to answer in deciding “Adaptive Or Not”.

1. First, and most importantly, what is the timing of observation of the key endpoint upon which to base adaptation and what is the expected enrollment rate? More specifically, is there sufficient time between the observation of a sufficient number of patients’ endpoints and the enrollment of the last patient to afford opportunity for adaptation? Cytel can assist you in making this assessment.

2. If answer to 1 is “no”, is there a sufficiently reliable surrogate or biomarker that is observable early enough upon which adaptation could be based?

3. Are there any regulatory concerns / reservations which would prohibit use of an Adaptive Design (AD)? To answer this question, consultation with regulatory affairs personnel, and/or those experienced in dealing with regulatory agencies, may be required. Cytel can help you in making this assessment.

4. Will sufficient safety data be available at filing if Adaptive Design is used? That is, will the number of patients required by regulatory agencies to support the safety of the candidate product be available at expected time of filing? If the adaptive design provides fewer patients than required for completion of the required safety database, the sample size may need to be increased beyond that necessary for the adaptive design.

5. Is there sufficient drug supply (for all potential doses & durations) to support all possible adaptations? Cytel has the technology to help you to make this assessment.

6. Can the needed drug supply be made available at the sites when needed to support all possible adaptations? How can one do this without either the risk of stockouts or the cost of massive overage and waste? See Cytel’s presentation on how to meet the challenges of drug supply in an adaptive trial.

7. Is data acquisition to inform on the planned adaptations rapid enough so the required analyses to inform the adaptation can be completed in time? Cytel can help you make this assessment.

8. Is there sufficient statistical expertise to support the evaluation of design options so that the Adaptive Design can be properly evaluated and implemented? Cytel can provide this expertise if a sponsor does not possess it.

9. Is the software necessary for evaluation and implementation of Adaptive Design options available?

Adaptive design performance characteristics must be thoroughly summarized prior to protocol approval. Software for closed form calculations and/or simulations is necessary to document these characteristics, and must be available in time to support the protocol development, and then to execute the necessary adaptive design calculations during the trial. Cytel’s East, Compass and SiZ software are industry standards for both adaptive and non-adaptive designs. In addition, Cytel has a large suite of validated proprietary software that can be applied to unusual situations where the standard software is not adequate.

Software is needed that facilitates the interim monitoring process while satisfying regulatory concerns about bias. Software is also needed to make the adaptive changes happen – communicating what the changes are to the IVRS system, and making sure that the necessary doses are available at every site where they may be needed.

See the white paper on ACES (click icon below). ACES is our best- in-class system for adaptive trial implementation, which securely manages communications with independent trial recommendation committees (DMCs/DSMBs):

10. Does the use of Adaptive Design in comparison to a classical alternative non-Adaptive Design afford sufficient improvement on cost, time to complete, and/or quantity / quality of information favor the use of Adaptive Design? In order to answer this question, one should have a full evaluation of the costs, the time to complete, and the quantity and quality of information expected from the candidate adaptive design and the non-adaptive-design alternative. With this information, a design can be chosen to balance among these aspects of each of the design candidates. Cytel can help you in making this comparison.

When determining the sample size for a trial, most clinical trialists focus on statistical power or the desired precision for each specific clinical trial separately. However, trials should ideally be considered in the context of the development program in which the results from previous trials guide decision-making and design of subsequent trials. A framework was presented by Patel et al (2012) to evaluate the impact that several phase 2 design features have on the probability of phase 3 success (PoS) and the expected net present value (eNPV) of the product. The factors considered include the phase 2 sample size, decision rules to select a dose for phase 3 trials and the sample size for phase 3 trials. The development program for a drug to treat neuropathic pain was used as an example.

A hypbrid frequentist/Bayesian approach was used via simulation to evaluate the impact of the aforementioned design factors on Phase 3 program success, clinical utility, and eNPV. Statistical analysis of data from each of the two Phase 3 trials used traditional frequentist methods, and go/no-go decisions from Phase 2 to Phase 3 and from Phase 3 to regulatory submission used Bayesian posterior probabilities. The findings from their simulations of low, moderate, and high levels of each of efficacy and safety suggest that optimal sample sizes for Phase 2 can be obtained to maximize eNPV and clinical utility. These optimal sample sizes are a trade-off between increasing Phase 2 sample size to improve Phase 3 dose selection, which can increase time and cost, and maximizing marketing time to increase revenues. This methodology offers an objective approach to solving that trade-off. It is not suggested to be the sole determinant of late phase drug development strategy, but rather a benchmark to inform decisions.

Their framework included only non-adaptive, equal allocation study designs, and carrying a single dose into Phase 3. It can be straight-forwardly extended to assess the impact of adaptive dose-finding designs in Phase 2, and strategies for carrying more than 1 dose into the Phase 3 trials.

The framework of simulating sequences of clinical trials to optimize the design of each trial was extended earlier in development to evaluate how to optimally incorporate biomarkers into an early clinical development program. Cytel collaborative research investigated a framework that leverages the relationship between biomarkers and target clinical endpoint to optimize an early development plan. Different biomarker designs were assessed for proof-of- concept (PoC) and dose-finding to improve Phase 2b design and improve Phase 3 dose choice. A case study using a Bayesian tri-variate normal distribution model for 2 biomarkers and a clinically relevant endpoint was used with simulation to assess performance characteristics. That work found (1) at typical sample sizes for early development trials, biomarkers appear useful for PoC, but not for clinical endpoint dose-finding; (2) even with large amounts of prior information and near perfect correlation between biomarkers and clinical endpoints, Ph2b variability is only overcome by increased Ph2b sample sizes to improve Ph3 dose choice. Thus, for highly variable clinical endpoints, the fastest path appeared to be to demonstrate PoC by biomarkers, then go directly to Ph2b to measure the target clinical endpoint. This framework could also be extended to assess the impact of adaptive biomarker trials and adaptive phase 2 trials on probability of choosing the correct phase 3 dose(s).

Given the potential benefits of adaptive designs, we believe it makes sense for a sponsor to evaluate the adaptive alternative for their clinical trial to determine whether the benefits offered are significant and are worth the additional regulatory and operational complexity. Cytel has the experience to help you make this assessment. We will help you make the correct choice.

Lastly, when the entire development program is optimized for each of several candidate drugs in a portfolio, it remains to assign resources from a fixed research budget across the portfolio. When funding is limited to less than the total potential research budget to fully fund all candidates in the portfolio, it may be useful for management to use the projected timing of each drug candidate project along with its projected PoC to optimally allocate the budget across candidates to maximize their potential total value. This problem has been address via a linear programming optimization algorithm by Patel, et al., [2013]

They incorporated inputs from R&D and commercial functions by simultaneously addressing internal and external factors. The assessed the impact of study design parameters, sample size in particular, on the portfolio value via an integer programming (IP) model as the basis for Bayesian decision analysis to optimize phase 3 development portfolios using eNPV as the criterion. They showed how their framework could be used to determine optimal sample sizes and clinical trial schedules to maximize the total value of a portfolio under budget constraints. They illustrated flexibility of their IP model to answer a variety of ‘what-if’ questions that reflect situations that arise in practice. They extended their IP model to a stochastic IP model to incorporate uncertainty in the availability of drugs from earlier development phases for phase 3 development in the future, and showed how to use stochastic IP to re-optimize the portfolio development strategy over time as new information accumulates and budget changes occur.

In summary, adaptive design can improve the performance of clinical trials. Sequences of trials can be optimized via simulation and use of adaptive clinical trial designs. Finally a fixed research budget can be optimally allocated across a portfolio of drugs to maximize its value. Cytel has experience and associated software to do each of these separately, or in combination.

For more information on Cytel’s experience with adaptive design, please visit our strategic consulting website or have a look at our case studies:

 First FDA approved drug with an Adaptive Re-estimation Trial Design  Adaptive Trial in Orphan Product Development  Adaptive Cardiovascular Trial Case Study

References 1. Mathieu, Mark P (ed). Parexel’s Bio/Pharmaceutical R&D Statistical Sourcebook 2011/2012. 2. DiMasi, Joseph A., and Grabowski, Henry G. "The cost of biopharmaceutical R&D: is biotech different?" Managerial and Decision Economics, 28.4‐5 (2007): 469-479. 3. DiMasi, Joseph A., Hansen, Ronald W., and Grabowski, Henry G. "The price of innovation: new estimates of drug development costs." Journal of Health Economics, 22.2 (2003): 151- 185. 4. FDA. “Drug and Biologic Approval and IND Activity Approvals.” April 2014. http://www.fda.gov/Drugs/DevelopmentApprovalProcess/HowDrugsareDevelopedandAppr oved/DrugandBiologicApprovalReports/default.htm. 5. HBM Partners. “Trends in US New Drug Approvals.” 2014. http://www.hbmpartners.com/wAssets/docs/industry-reports/Trends-in-New-Drug- Approvals-by-US-FDA-2003-2013.pdf. 6. FDA. “New Molecular Entity Approvals for 2013.” 2014. http://www.fda.gov/Drugs/DevelopmentApprovalProcess/DrugInnovation/ucm381263.htm 7. Hay, Michael, et al. “Clinical development success rates for investigational drugs.” Nature Biotechnology, 32.1 (2014): 40 – 51. 8. Kola, Ismail and Landis, John. “Can the pharmaceutical industry reduce attrition rates?” Nature Reviews Drug Discovery, 3.1 (2004): 711-716. 9. Sacks LV, Shamsuddin HH, et al. “Scientific and Regulatory Reasons for Delay and Denial of FDA Approval of Initial Applications for New Drugs, 2000-2012.” JAMA, 311.4 (2014): 378- 384. 10. Thomson Reuters. 2013 CMR International Pharmaceutical R&D Factbook. 2013. http://cmr.thomsonreuters.com/pdf/fb-exec-2013.pdf 11. PHT and palmOne Client Success Story, “Merck Research Laboratories Insomnia Study.” http://www.phtcorp.com/Resources/Case-Studies/PDFs/Merck-Insomnia-case-study.aspx 12. Patel, Nitin R. “Simulation tools for planning drug supply in clinical trials.” Drug Supply Workshop Presentation at Wyeth, February 2009 (available upon request). 13. Center Watch. Wyeth Overhauling Phase II Program, July 18, 2006. http://www.centerwatch.com/news-resources/clinical-trials-today/headline- details.aspx?HeadlineID=598. 14. Vandevald, S and Thayer, J. “Trends in Clinical Trial Site Selection and Patient Recruitment.” Clinipace Worldwide Webcast February 27, 2014. 15. Private Communication from Robert Bosserman, CEO, Medelis, and Irving Dark, VP, Clinical Research Services, Cytel Inc. June 2009. 16. Patel N, Bolognese J, Chuang-Stein C, Hewitt D, Gammaitoni A, Pinheiro J. “Designing phase 2 trials based on program-level considerations: A case study for neuropathic pain.” Drug Information Journal, 46.4 (2012): 439-454. 17. Patel NR, Ankolekar S, Antonijevic Z, Rajicic N. “A mathematical model for maximizing the value of phase 3 drug development portfolios incorporating budget constraints and risk.” Statistics in Medicine , 32.10 (2013): 1763-1777. 18. Arrowsmith, John, and Miller, Philip. "Trial watch: phase ii and phase iii attrition rates 2011- 2012." Nature Reviews Drug Discovery, 12.8 (2013): 569-569.