Design of Randomized Clinical Trials October 15, 2009 Scott S. Emerson, M.D., Ph.D. Part
Total Page:16
File Type:pdf, Size:1020Kb
Design of Randomized Clinical Trials October 15, 2009 Design, Monitoring, and Analysis Preliminaries of Clinical Trials Scott S. Emerson, M.D., Ph.D. Conflicts of Interest / Disclaimers Professor of Biostatistics, University of Washington October 15 – 16, 2009 1 2 Conflicts of Interest Personal Defects Commercially available software for sequential Physical clinical trials Personality • S+SeqTrial (Emerson) • In the perjorative sense of the words • PEST (Whitehead) – Then: • EaSt (Mehta) • A bent twig • SAS (I provided some limited advice) –Now: • Old and male • University professor 3 4 A More Descriptive Title A More Descriptive Title All my talks All my talks The Use of Statistics to Answer The Use of Statistics to Answer Scientific Questions Scientific Questions (Confessions of a Former Statistician) (Confessions of a Former Statistician) Clinical Trial Design talks The Use of Statistics to Answer Scientific Questions Ethically and Efficiently (Confessions of a Former Statistician) 5 6 Scott S. Emerson, M.D., Ph.D. Part 1:1 Design of Randomized Clinical Trials October 15, 2009 Clinical Trials Overview of Clinical Trial Design Experimentation in human volunteers • Investigates a new treatment/preventive agent – Safety: • Are there adverse effects that clearly outweigh any Science and Statistics potential benefit? – Efficacy: • Can the treatment alter the disease process in a beneficial Where am I going? way? In the real world, clinical trial design must consider – scientific theory – Effectiveness: – statistical theory • Would adoption of the treatment as a standard affect – logistical issues morbidity / mortality in the population? – game theory I make an argument (plea?) for clinical trial design to consider science first, then statistics (Game theory is a necessary evil) 7 8 Carrying Coals to Newcastle Clinical Trial Design Wiley Act (1906) Finding an approach that best addresses the often • Labeling competing goals: Science, Ethics, Efficiency Food, Drug, and Cosmetics Act of 1938 • Basic scientists: focus on mechanisms • Safety • Clinical scientists: focus on overall patient health Kefauver – Harris Amendment (1962) •Ethical: focus on patients on trial, future patients • Efficacy / effectiveness • Economic: focus on profits and/or costs – " [If] there is a lack of substantial evidence that the drug will have the • Governmental: focus on safety of public: treatment effect ... shall issue an order refusing to approve the application. “ safety, efficacy, marketing claims – “...The term 'substantial evidence' means evidence consisting of adequate and well-controlled investigations, including clinical •Statistical: focus on questions answered precisely investigations, by experts qualified by scientific training” • Operational: focus on feasibility of mounting trial FDA Amendments Act (2007) • Registration of RCTs, Pediatrics, Risk Evaluation and Mitigation Strategies (REMS) 9 10 Statistical Planning Statistics and Science Satisfy collaborators as much as possible Statistics is about science • Discriminate between relevant scientific hypotheses • Science in the broadest sense of the word – Scientific and statistical credibility • Protect economic interests of sponsor Science is about proving things to people – Efficient designs • Science is necessarily adversarial – Economically important estimates – Competing hypotheses to explain the real world • Protect interests of patients on trial • Proof relies on willingness of the audience to believe it – Stop if unsafe or unethical • Science is a process of successive studies – Stop when credible decision can be made • Promote rapid discovery of new beneficial treatments Game theory: Accounting for conflicts of interest • Financial • Academic / scientific 11 12 Scott S. Emerson, M.D., Ph.D. Part 1:2 Design of Randomized Clinical Trials October 15, 2009 Science vs Statistics “Parameter” vs “Sample” Relationships Recognizing the difference between The true scientific relationship (“parameter space”) • Summary measures of the effect in population • The parameter space – Means, medians, geometric means, proportions… – What is the true scientific relationship? Scientific “sample space” scales: • Estimates attempting to assess scientific importance • The sample space – Point estimate is a statistic estimating a “parameter” – What data will you / did you gather? – Interval estimates • CI describes the values in the “parameter space” that are consistent with the data observed (the “sample space”) Purely statistical “sample space” scales • The precision with which you know the true effect – Power, P values, posterior probabilities 13 14 The Problem of Clinical Trial Design The Problem of Clinical Trial Design Happy families are all alike; every unhappy family is Unbiased clinical trials are all alike; every biased unhappy in its own way. clinical trial is biased in its own way. Leo Tolstoy, Anna Karenina, 1873-77 15 16 Statistical Sampling Plan Demystification Ethical and efficiency concerns are addressed All we are doing is statistics through sequential sampling • Planning a study • During the conduct of the study, data are analyzed at periodic intervals and reviewed by the DMC • Gathering data • Using interim estimates of treatment effect – Decide whether to continue the trial • Analyzing it – If continuing, decide on any modifications to • scientific / statistical hypotheses and/or • sampling scheme 17 18 Scott S. Emerson, M.D., Ph.D. Part 1:3 Design of Randomized Clinical Trials October 15, 2009 Sequential Studies Major Issue All we are doing is statistics Frequentist operating characteristics are based on • Planning a study the sampling distribution – Added dimension of considering time required • Stopping rules do affect the sampling distribution of the • Gathering data usual statistics – Sequential sampling allows early termination – MLEs are not normally distributed • Analyzing it – Z scores are not standard normal under the null – The same old inferential techniques • (1.96 is irrelevant) – The null distribution of fixed sample P values is not – The same old statistics uniform – But new sampling distribution • (They are not true P values) 19 20 Sampling Distribution of MLE Sampling Distribution of MLE 21 22 Sampling Distributions Familiarity and Contempt For any known stopping rule, however, we can compute the correct sampling distribution with specialized software • Standalone programs – PEST (some integration with SAS) –EaSt • Within statistical packages – S-Plus S+SeqTrial – SAS PROC SEQDESIGN 23 24 Scott S. Emerson, M.D., Ph.D. Part 1:4 Design of Randomized Clinical Trials October 15, 2009 Familiarity and Contempt Example: P Value From the computed sampling distributions we then Null sampling density tail compute – Bias adjusted estimates – Correct (adjusted) confidence intervals – Correct (adjusted) P values Candidate designs can then be compared with respect to their operating characteristics 25 26 Distinctions without Differences Perpetual Motion Machines Sequential sampling plans Discerning the hyperbole in much of the recent • Group sequential stopping rules statistical literature • Error spending functions • “Self-designing” clinical trials • Conditional / predictive power • Other adaptive clinical trial designs Statistical treatment of hypotheses – (But in my criticisms, I do use a more restricted definition than some in my criticisms) • Superiority / Inferiority / Futility • Two-sided tests / bioequivalence 27 28 Bottom Line Experimentation Directed Toward Adopting New Treatments You better think (think) about what you’re trying to do… -Aretha Franklin 29 30 Scott S. Emerson, M.D., Ph.D. Part 1:5 Design of Randomized Clinical Trials October 15, 2009 Overall Goal Phases of Investigation “Drug discovery” Series of studies support adoption of new treatment • More generally • Preclinical – a therapy or preventive strategy – Epidemiology incl risk factors – for some disease – Basic science: Physiologic mechanisms – in some population of patients – Animal experiments: Toxicology A series of experiments to establish •Clinical • Safety of investigations / dose – Phase I: Initial safety / dose finding • Safety of therapy – Phase II: Preliminary efficacy / further safety • Measures of efficacy – Phase III: Confirmatory efficacy / effectiveness – Treatment, population, and outcomes • Approval of indication • Confirmation of efficacy – (Phase IV: Post-marketing surveillance, REMS) • Confirmation of effectiveness 31 32 The Enemy First Where do we want to be? • Find a new treatment that improves health of individuals “Let’s start at the very beginning, a very • Find a new treatment that improves health of the good place to start…” population - Maria von Trapp (as quoted by Rodgers and Hammerstein) 33 34 Treatment “Indication” Experimental Results •Disease Start with the science – Putative cause vs signs / symptoms • No “experiment” is ethical if it cannot answer a relevant • May involve method of diagnosis, response to therapies question • Population – Restrict by risk of AEs or actual prior experience Hypotheses are the spectrum of possible results • Treatment or treatment strategy • An “experiment” discriminates among the possible – Formulation, administration, dose, frequency, hypotheses duration, ancillary therapies • The Scientist Game •Outcome – Clinical vs surrogate; timeframe; method of measurement 35 36 Scott S. Emerson, M.D., Ph.D. Part 1:6 Design of Randomized Clinical Trials October 15, 2009 Scientific Method Specific Aim Planned experiment includes protocol specified