Page 1 of 181 a Modelling Framework for Estimation of Comparative
Total Page:16
File Type:pdf, Size:1020Kb
A modelling framework for estimation of comparative effectiveness in pharmaceuticals using uncontrolled clinical trials Anthony James Hatswell UCL Statistical Science Page 1 of 181 I, Anthony James Hatswell confirm that the work presented in this thesis is my own. Where information has been derived from other sources, I confirm that this has been indicated in the thesis. Anthony James Hatswell Date Page 2 of 181 ABSTRACT Pharmaceuticals are most commonly studied in randomised controlled trials (RCTs) against a control arm (either active, or placebo). On occasions however treatments are licensed exclusively on the basis of uncontrolled study data - this thesis investigates how comparative effectiveness can be estimated under such circumstances. The role of RCTs in the approval and estimation of comparative effectiveness in pharmaceuticals is discussed, as well as potential methods for analysis where RCT data are not available. A review of all drug approvals from 1999-2014 by the European Medicines Agency and the US Food and Drug Administration is then presented. Performing literature searches in the majority of cases (80%), historical controls have been the primary source of estimates of comparative effectiveness, frequently without attempts to adjust for differences between studies. Given the high usage of historical controls, I focussed on the role of adjustment. This included a simulation study to understand where the method of Matching Adjusted Indirect Comparison (MAIC) is likely to be of use, looking specifically at the effect of model misspecification. Three novel methods (with practical examples) for the creation of historical controls are then presented; using extrapolation from the previous line of therapy, using non- responders to therapy as a surrogate, and comparing to a patient’s own prior data. The conclusion of the work is that there are clearly situations where RCTs cannot, or will not be used – regardless of the statistical issues this raises. In such cases by proactively identifying appropriate historical data, and using appropriate analysis methods – the downsides can be ameliorated, at least in part. A flowchart presenting the available methods (split by data access) is presented. Further research is required on the appropriateness of different sources of historical control data (e.g. registries versus RCT arms), and how to synthesize multiple estimates of effectiveness (e.g. multiple MAICs). Page 3 of 181 TABLE OF CONTENTS ABSTRACT ...................................................................................................................... 3 TABLE OF CONTENTS ................................................................................................... 4 TABLE OF TABLES......................................................................................................... 7 TABLE OF FIGURES ....................................................................................................... 7 TABLE OF EQUATIONS ................................................................................................. 9 ACKNOWLEDGEMENTS .............................................................................................. 10 PUBLICATIONS ............................................................................................................. 11 IMPACT STATEMENT ................................................................................................... 14 ABBREVIATIONS .......................................................................................................... 15 INTRODUCTION ..................................................................................................... 17 1.1 THE HISTORY AND THEORETICAL UNDERPINNING OF RANDOMISED TRIALS ...................... 19 1.2 ‘CONVENTIONAL’ RANDOMISED DOUBLE BLIND TRIALS ...................................................... 21 1.2.1 Randomisation and exchangeability ........................................................................ 21 1.2.2 Blinding of patients and physicians ......................................................................... 22 1.2.3 Multi-centre trials ........................................................................................................ 23 1.3 MATHEMATICAL NOTATION FOR RANDOMISED CLINICAL TRIALS ........................................ 23 1.4 PREVIOUS WORK IN ASSESSING EVIDENCE WITHOUT RANDOMISED TRIALS ..................... 26 1.4.1 Assessing the comparative effectiveness of treatments studied without a randomised control ..................................................................................................................... 26 1.4.2 Effect sizes seen in observational data, compared to RCTs ............................... 27 1.4.3 Historical controls ....................................................................................................... 29 1.4.4 Methodologies for the use of observational data in estimation of efficacy ........ 32 1.5 THE ROLE OF UNCONTROLLED STUDIES IN DRUG APPROVALS .......................................... 33 1.5.1 ‘Obviousness’ .............................................................................................................. 34 1.5.2 Clinical equipoise........................................................................................................ 35 1.5.3 The benefit-risk of trial participants and patients ................................................... 36 1.6 THE USE OF UNCONTROLLED STUDIES IN MODELLING AND HEALTH TECHNOLOGY APPRAISAL – GUIDANCE FROM AGENCIES ....................................................................................... 37 1.7 SUMMARY OF INTRODUCTION AND RESEARCH QUESTION ................................................. 41 EXISTING METHODOLOGIES THAT COULD BE USED TO ESTIMATE EFFECTIVNESS FROM UNCONTROLLED STUDIES ................................................ 44 2.1 METHODOLOGIES FOR THE ANALYSIS OF HISTORICAL CONTROLS .................................... 44 2.1.1 Methodologies for use with published historical controls ..................................... 44 2.1.2 Where individual level data (ILD) are available for either the intervention or the historical data .............................................................................................................................. 51 2.1.3 Where individual level data (ILD) are not available to a researcher for the intervention or the historical data ............................................................................................. 55 2.2 METHODOLOGIES FOR THE SYNTHESIS OF MULTIPLE HISTORICAL CONTROLS ................. 55 2.2.1 Meta-regression .......................................................................................................... 55 2.2.2 Meta-analysis of historical controls .......................................................................... 56 2.2.3 The Bayesian ‘power prior’ ....................................................................................... 57 2.2.4 Comparison between approaches for combing historical controls, and applicability to uncontrolled studies ......................................................................................... 59 2.3 WHERE NO HISTORICAL DATA ARE AVAILABLE .................................................................... 59 Page 4 of 181 2.3.1 The use of expert opinion.......................................................................................... 59 2.3.2 Threshold analysis and the ‘E-value’ ...................................................................... 60 2.4 EMERGING METHODOLOGIES............................................................................................... 61 2.4.1 The use of ‘real world data’ to establish control arms .......................................... 61 2.4.2 Machine learning ........................................................................................................ 62 2.5 SUMMARY OF EXISTING METHODOLOGIES FOR ESTIMATING COMPARATIVE EFFECTIVENESS USING UNCONTROLLED STUDIES ......................................................................... 63 IDENTIFICATION OF THE NUMBER OF TREATMENTS INVOLVED, AND METHODS USED FOR MODELLING ............................................................................................... 64 3.1 TREATMENTS APPROVED ON THE BASIS OF UNCONTROLLED CLINICAL STUDIES ............. 64 3.1.1 Regulatory processes in the United States and the European Union ................ 64 3.1.2 Details of the search of the EMA and FDA drug approval databases................ 65 3.1.3 Consolidated list of treatments licensed on the basis of uncontrolled studies from 1999 to 2014 ...................................................................................................................... 70 3.1.4 Disease areas where uncontrolled studies have most frequently been the basis for drug approvals ...................................................................................................................... 73 3.1.5 Comparison between the FDA and EMA on the number of approvals, and the dates of reviews .......................................................................................................................... 73 3.1.6 Congruence of findings with the existing literature ............................................... 78 3.1.7 Subsequent work performed by others in the area ............................................... 80 3.2 METHODS USED FOR ESTIMATING EFFECTIVENESS FROM UNCONTROLLED STUDIES ...... 81 3.2.1 Literature