Pubh 8482: Sequential Analysis Course Introduction Joseph S. Koopmeiners Division of Biostatistics University of Minnesota Week 1 Who am I? • Joe Koopmeiners • Assistant Professor in Division of Biostatistics • PhD - University of Washington, 2009 • Research Interests: • Sequential and adaptive methods for translational cancer research • statistical evaluation of diagnostic tests • Collaborative Projects: • Tobacco cessation • MRI as a diagnostic tool for prostate cancer • Statistical support for early phase clinical trials Who are you? • Name • Year in program (program, if not in biostatistics) • Advisor, if known Pre-requisites • Stat 8101-8102 • Students must be comfortable with the multivariate normal distribution Text Books There are no required texts for this course but lecture notes will draw heavily from: • Jennison, C. and Turnbull, B. (1999) Group Sequential Methods with Applications to Clinical Trials, Boca Raton: CRC Press. ISBN 0849303168 • Berry, S.M., Carlin, B.P., Lee, J.J. and Muller, P. (2010) Bayesian Adaptive Methods for Clinical trials, Boca Raton: CRC Press. ISBN 1439825483 Text Books Other useful textbooks on reserve in the biostat reading room include: • Whitehead, J. (1997) The Design and Analysis of Sequential Clinical Trials, 2nd Ed., New York: John Wiley & Sons. ISBN 0471975508 • Proschan, M. A., Lan, K.K.G. and Wittes, J.T. (2006) Statistical Monitoring of Clinical Trials: A Unified Approach, New York: Springer. ISBN 0387300597 • Yin, G. (2012) Clinical Trial Design: Bayesian and Frequentist Adaptive Methods, Hoboken: John Wiley & Sons. ISBN 0470581719 Evaluation and Grades • Homework - 40% • Mid-term exam - 30% • Final project - 30% Homework • About 5 homework assignments • You will have at least 2 weeks to work on the assignments • Homeworks will be due on Tuesdays Mid-term Exam • Take-home mid-term about halfway through the course • Will cover what I consider “classic” group sequential methodology • You will have one week to complete the exam Class Project • Pick a specific topic related to sequential and adaptive clinical trials • Writen report and 20 - 30 minute presentation • Discuss statistical challenges specific to your topic • Literature review of current research • Identify open research areas Office Hours • Monday 3:00 p.m. - 4:00 p.m. • Thursday 1:00 - 2:00 p.m. Course Website • Course Website http://www.biostat.umn.edu/ josephk/courses/pubh8482 fall2012/ • Linked from my faculty webpage Course Title • Current Title: • Sequential Analysis • More Accurate Title: • Sequential and Adaptive Methods for Clinical Trials What is a clinical trial? • Clinical Trial: A controlled experiment to test the safety or efficacy of a treatment or intervention • Usually randomized. Although, this is not always the case. Especially in phases 1 and 2. Fixed-sample Design • Most studies utilize fixed-sample designs • Fixed-sample design: collect a pre-specified number of subjects and test our hypthesis • Randomize 200 patients to either a novel treatment or placebo and compare survival using a log-rank test Sequential Design • An alternate approach is a sequential design • Sequential design: sequentially monitor the primary endpoint and continue to enroll subjects based on the interim results • Randomize an initial cohort of patients to a novel treatment or placebo and compare survival using a log-rank test • Determine if more patients should be enrolled based on pre-specified stopping rule Continuous Monitoring • Early sequential methods focused on continuous monitoring • Evaluating the endpoint after each new experimental unit • Quality control for bombs during WWII • These methods are not practical in the setting of clinical trials Group Sequential Methods • Sequential clinical trials generally rely on group sequential methodology • Group sequential methods: interim analyses are completed at pre-specified intervals throughout the study • Randomized the first 50 patients to either a novel treatment or placebo and compare survival using a log-rank test • Determine if more patients should be enrolled based on pre-specified stopping rule • Re-evaluate endpoint after every 50 patients until a total of 200 have been enrolled Adaptive Design • An adaptive design is a design that uses accumulating data from the ongoing trial to modify certain aspects of the study • Sample size • Treatment dose • Randomization ratio • Study arms Sequential vs. Adaptive Designs • There is no clear distinction between what constitutes a sequential and what constitutes an adaptive design • Both rely on interim analyses to modify the design • Sequential designs generally only modify the sample size (by stopping early), while adaptive designs are used to describe designs with more broad modifications • Both face similar statistical challenges Sequential and Adaptive Designs: Challenges • Clinical trials are designed to achieve desired operating characteristics • Type-I error • Power • Sequential and adaptive methods alter the operating characteristics of the study • Challenge: Incorporate sequential and adaptive methods while maintaining the desired operating characteristics • Goal: Show that sequential and adaptive methods have the same type-I error rate and power as a fixed-sample design but smaller sample size or other desirable property Phases of Drug Development • Phase 1: Safety trials • Phase 2: Efficacy trials • Phase 3: Confirmatory trials • Phase 4: Post-marketing surveillance Phase 1 Clinical Trials • First-in-human trials • Primary objective is to evaluate safety • Efficacy is a secondary concern • Small trials: 10 - 50 subjects Phase 1 Trials Example Phase 1 clinical trial in oncology • Estimate the probability of dose limiting toxicity (DLT) • We assume that as dose increases, the probability of DLT and the probability of efficacy will also increase • Maximum tolerated dose (MTD): Highest dose with probability of DLT less than some pre-specified cut-off (usually 0.2 or 0.33) Dose Escalation • We would like to do a randomized study • Too dangerous for first-in-human studies • Instead, we complete non-randomized dose escalation studies • Patients receive progressively higher doses until MTD has been identified • Adaptive designs are used to guide dose escalation Patients vs. Healthy Volunteers • Healthy volunteers are used in other settings where toxicities are less severe • Phase 1 oncology trials include patients for whom standard treatments have failed • Added goal treating patients with efficacious dose • Adaptive designs are used to treat patients at dose levels that are more likely to be efficacous Phase 2 Clinical Trials • Goal of Phase 2: evaluate the efficacy of a novel therapeutic agent • Surrogate endpoints are often used in place of hard endpoints • Phase 2 oncology trial: tumor response instead of overall survival • Continue to evaluate safety of new drug Stopping for futility • The majority of novel therapeutic agents will not be adequately efficacious • Clinical trials for ineffective treatments are expensive and fail to provide adequate care for study subjects • Early termination for futility allows ineffective treatments to be abonded if initial estimates of treatment efficacy are not promising Dropping Study Arms • The optimal dose/treatment schedule is unlikely to be known after Phase 1 • We might run a multi-arm study to investigate multiple dose levels/treatment schedules and adaptively drop arms to save time/money/etc. Safety Monitoring • The safety profile of a new drug continues to be evaluated in Phase 2 • Dual goals of answering scientific question/protecting study subjects • Sequential stopping rules are used to monitor the rate of adverse events Personalized Medicine • Personalized medicine refers to customized treatment decisions based on patient characteristics such as genetic or other information • A phase 2 trial could be designed to investigate the effectiveness of a new treatment in different subpopulations • We might design a study to adaptively assign subjects to one of several treatment or drop subgroups for which the drug is not effective Phase 3 Clinical Trials • Final, confirmatory trial for new therapeutic agent • Much larger than phase 2 • Hard endpoints • Overall survival instead of clinical response Sequential Monitoring in Phase 3 • Most common setting for sequential monitoring • Stopping rules set in advance to allow early termination for efficacy or futility • Get new treatments onto the market faster • Save time and money when treatments are not promising Adaptive Randomization • Typically, subjects are randomized to treatment or control using a fixed randomization ratio • Alternately, we could change the randomization ratio over time so that more subjects are assigned to “better” treatment • Better outcomes for study subjects Sample Size Re-estimation • Sample size is often calculated based on nuisance parameters for which there is little information • Incorrectly specified nuisance parameters can lead to under-powered studies • Re-estimate sample size using updated estimates of nuisance parameters at interim analyses Sample Size Re-estimation • What if the true effect size is smaller than we anticipated? • We can re-estimate the sample size using an updated estimate of the effect size at interim analyses • This is controversial In Summary Motivations for using sequential/adaptive designs can be group into the following: • Ethical • Economic • Administrative Ethical • Minimize the number of subjects treated with ineffective treatments • Make new treatments available to the public more quickly • Protect