DOCSLIB.ORG

Sources of Evidence for Assessing the Safety, Efficacy and Effectiveness Of

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Sources of Evidence for Assessing the Safety, Efficacy and Effectiveness Of Sources of evidence for assessing the safety, efficacy and effectiveness of medicines June 2017 Acknowledgements and disclaimer The Academy of Medical Sciences is most grateful to Sir Michael Rutter CBE FRS FBA and to the members of the working group for undertaking this study. We thank the Academy’s Officers, Council members and staff, the external review group, study observers, as well as our Fellows and all those who have contributed through the call for evidence, stakeholder meetings, or by providing oral evidence. We thank the study secretariat led by Dr Claire Cope. The Academy is grateful to Arthritis Research UK, the British Heart Foundation, the British Pharmacological Society, the British Society for Immunology, the Medical Research Council, the Naji Foundation, and the National Institute for Health Research Health Technology Assessment Programme for their financial contribution to the workstream as a whole. Funding from a core grant from the Department for Business, Energy and Industrial Strategy to the Academy was also used to support this project. This report is published by the Academy of Medical Sciences and has been endorsed by its Officers and Council. Contributions by the working group were made purely in an advisory capacity. The members of the working group participated in an individual capacity and not as representatives of, or on behalf of, their affiliated hospitals, universities, organisations or associations. Their participation should not be taken as endorsement by these bodies. All web references were accessed in February 2017. This work is © The Academy of Medical Sciences and is licensed under Creative Commons Attribution 4.0 International. The Academy of Medical Sciences 2 Sources of evidence for assessing the safety, efficacy and effectiveness of medicines Contents Executive summary ................................................................................................... 4 Recommendations .................................................................................................... 8 Glossary of abbreviations ........................................................................................ 11 1. Introduction ......................................................................................................... 13 1.1 Scope and terms of reference ......................................................................................... 15 1.2 Conduct of the study ....................................................................................................... 15 1.3 Overview and audience ................................................................................................... 16 2. The research issues that all study types should address .................................... 18 2.1 Bias .................................................................................................................................. 19 2.2 Confounding .................................................................................................................... 20 2.3 Blinding ........................................................................................................................... 20 2.4 Internal/external validity and relevance .......................................................................... 21 2.5 Moderating variables ....................................................................................................... 21 2.6 Absolute risk, relative risk, attributable risk and number needed to treat ....................... 22 2.7 Causality .......................................................................................................................... 23 2.8 Choice of comparator ...................................................................................................... 25 2.9 Participant attrition and adherence to treatments .......................................................... 25 2.10 The ‘placebo’ and ‘nocebo’ effects ................................................................................ 27 2.11 Surrogate endpoints ...................................................................................................... 28 3. Addressing the challenges of research designs .................................................. 29 3.1 Randomised controlled trials ........................................................................................... 29 3.2 Observational studies ...................................................................................................... 32 3.3 Attrition, adherence, choice of comparator and endpoints ............................................. 34 3.4 Qualitative research ........................................................................................................ 35 3.5 Meta-analyses and systematic reviews ............................................................................ 35 3.6 Hierarchies of evidence ................................................................................................... 36 4. Generating evidence – evolving approaches and special cases ......................... 39 4.1 Propensity scores as a way of dealing with social selection ............................................. 40 4.2 Natural experiments ........................................................................................................ 42 The Academy of Medical Sciences 3 4.3 Mendelian randomisation as a way of inferring causality ................................................ 44 4.4 Areas where new strategies are required ........................................................................ 44 4.5 Future challenges ............................................................................................................ 47 5. Further issues for consideration when assessing research findings .................. 51 5.1 Research reproducibility and reliability............................................................................ 51 5.2 Publication bias ............................................................................................................... 54 5.3 Working with industry ..................................................................................................... 54 5.4 Concerns about over or underuse of medicines .............................................................. 56 6. Evaluation of research findings when considering their application in clinical practice .................................................................................................................... 60 6.1 Step 1: Quality of evidence on efficacy and harms .......................................................... 60 6.2 Step 2: Combining RCT data from multiple studies .......................................................... 62 6.3 Step 3: Balancing benefits and harms .............................................................................. 62 6.4 Step 4: Consideration of moderating effects ................................................................... 63 6.5 Conclusions ..................................................................................................................... 63 7. Conclusions and recommendations .................................................................... 65 7.1 The research design and conduct issues that studies should seek to address ................. 65 7.2 Trial designs ..................................................................................................................... 66 7.3 Evolving approaches for the study of the benefits and harms of treatments .................. 67 7.4 Evaluation of research findings for clinical practice ......................................................... 68 7.5 Associated issues ............................................................................................................. 68 7.6 Recommendations from the report ................................................................................. 69 7.7 Guidelines for different stakeholder groups .................................................................... 71 Annex I. Report preparation .................................................................................... 74 Annex II. Consultation and evidence gathering ...................................................... 77 Annex III. Hierarchies of evidence ........................................................................... 79 Annex IV. Alternative trial designs .......................................................................... 81 Annex V. References ................................................................................................ 84 The Academy of Medical Sciences 4 Executive summary Recent high-profile media debates over the use of statins to prevent heart disease, anti-virals to treat influenza, and the human papilloma virus (HPV) vaccine to prevent cervical cancer, have brought to the fore discussions around the validity of the methods of collecting and analysing evidence, and the way that it is presented. To provide some clarity to the debate, the Academy of Medical Sciences convened a working group to explore the strengths and limitations of different methods of assessing evidence and to describe evolving approaches that are being, or have been, developed to address some of the major limitations of current methodologies. This working group project is part of a wider workstream considering how we can all best use evidence to judge the potential benefits and harms of medicines.
Load more

Recommended publications
  • Understanding Relative Risk, Odds Ratio, and Related Terms: As Simple As It Can Get Chittaranjan Andrade, MD
    Understanding Relative Risk, Odds Ratio, and Related Terms: As Simple as It Can Get Chittaranjan Andrade, MD Each month in his online Introduction column, Dr Andrade Many research papers present findings as odds ratios (ORs) and considers theoretical and relative risks (RRs) as measures of effect size for categorical outcomes. practical ideas in clinical Whereas these and related terms have been well explained in many psychopharmacology articles,1–5 this article presents a version, with examples, that is meant with a view to update the knowledge and skills to be both simple and practical. Readers may note that the explanations of medical practitioners and examples provided apply mostly to randomized controlled trials who treat patients with (RCTs), cohort studies, and case-control studies. Nevertheless, similar psychiatric conditions. principles operate when these concepts are applied in epidemiologic Department of Psychopharmacology, National Institute research. Whereas the terms may be applied slightly differently in of Mental Health and Neurosciences, Bangalore, India different explanatory texts, the general principles are the same. ([email protected]). ABSTRACT Clinical Situation Risk, and related measures of effect size (for Consider a hypothetical RCT in which 76 depressed patients were categorical outcomes) such as relative risks and randomly assigned to receive either venlafaxine (n = 40) or placebo odds ratios, are frequently presented in research (n = 36) for 8 weeks. During the trial, new-onset sexual dysfunction articles. Not all readers know how these statistics was identified in 8 patients treated with venlafaxine and in 3 patients are derived and interpreted, nor are all readers treated with placebo. These results are presented in Table 1.
    [Show full text]
  • Measures of Effect & Potential Impact
    Measures of Effect & Potential Impact Madhukar Pai, MD, PhD Assistant Professor McGill University, Montreal, Canada [email protected] 1 Big Picture Measures of Measures of Measures of potential disease freq effect impact First, we calculate measure of disease frequency in Group 1 vs. Group 2 (e.g. exposed vs. unexposed; treatment vs. placebo, etc.) Then we calculate measures of effect (which aims to quantify the strength of the association): Either as a ratio or as a difference: Ratio = (Measure of disease, group 1) / (Measure of disease, group 2) Difference = (Measure of disease, group 1) - (Measure of disease, group 2) Lastly, we calculate measures of impact, to address the question: if we removed or reduced the exposure, then how much of the disease burden can we reduce? Impact of exposure removal in the exposed group (AR, AR%) Impact of exposure remove in the entire population (PAR, PAR%) 2 Note: there is some overlap between measures of effect and impact 3 Measures of effect: standard 2 x 2 contingency epi table for count data (cohort or case-control or RCT or cross-sectional) Disease - Disease - no Column yes total (Margins) Exposure - a b a+b yes Exposure - c d c+d no Row total a+c b+d a+b+c+d (Margins) 4 Measures of effect: 2 x 2 contingency table for a cohort study or RCT with person-time data Disease - Disease - Total yes no Exposure - a - PYe yes Exposure – c - PY0 no Total a+c - PYe + PY0 Person-time in the unexposed group = PY0 5 Person-time in the exposed group = PYe STATA format for a 2x2 table [note that exposure
    [Show full text]
  • Attributable Risk Estimation Using Sas/Stat Software
    ATTRIBUTABLE RISK ESTIMATION USING SAS/STAT SOFTWARE Olaf Gefeller Abteilung Medizinische Statistik, Georg-August-Universitat Gottingen Abstract In epidemiologic studies devot~d to .the analysis of the relationship between an exposure characteristic and a disease the attributable risk constitutes an interesting epidemiologic risk measure. It can be interpreted as the proportion of cases of disease due to the exposure under study among all cases of disease in the study population. Thus, it indicates the impact of the exposure factor on the disease load of the population and can aid public health administrators in determining promising targets for disease preventive strategies in the population. The estimation of attributable risk from data arranged in contingency tables has received much attention in the statistical literature during recent years. However, none of the leading statistical software packages including SAS has incorporated the attributable risk in the catalogue of measures of association to be automatically calculated in some procedure. In the paper a brief introduction to the theory and application of attributable risk estimation from epidemiologic data is provided. Then, a general method is shown how to calculate attributable risk estimators under the multinomial sampling model usingSAS/STAT* software. Data from the Luebeck Blood Pressure Study, a cross-sectional study in the field of cardiovascular disease epidemiology, are used to illustrate the practical application of the method. Keywords: attributable risk, epidemiologic methods, measure of association 1. Introduction The quantification and measurement of disease risk associated with some exposure characteristic constitute a major goal of epidemiologic research. However, different conceptual approaches to describe the association between exposure and disease have been put forward in the ·epidemiologic literature.
    [Show full text]
  • ACCEA Personal Statements
    ACCEA BRONZE AWARD 2016 Name: PAUL AVEYARD GMC/GDC Number: 3332407 Speciality: ACADEMIC GP My research showed for the first time that primary care teams were ineffective at supporting weight loss, but referral to commercial weight management services was effective and cheaper. This research is cited in NICE guidelines and informed Department of Health guidance, leading to a doubling of referral to commercial providers and decline in primary care treatment. I trained GPs to deliver a 30-second intervention to promote these referrals and showed that screening for obesity is popular with patients, easy for GPs to deliver, and led to 2.5kg of weight loss. My research created a new approach to GP brief interventions to motivate smoking cessation. The national online training video has been used by 95,000 clinicians. My subsequent work with the Department of Health changed national guidelines, increasing referrals by 39%. My research showed that smoking reduction creates lasting cessation and influenced NICE guidelines. I was the GP member of the guideline group and launched the guidance at the NICE press conference, making England the first country in the world to implement a harm reduction strategy. I am one of four editors of the Cochrane Tobacco Addiction Group and our reviews are cited four times more than the Cochrane average. Our work is cited in UK and many other countries' national guidelines. ACCEA BRONZE AWARD 2016 Name: TREVOR THOMPSON GMC/GDC Number: 3494806 Speciality: ACADEMIC GP I am an NHS clinician dedicated to creating "Tomorrow's Doctors": 1. In 2015 I was awarded a National Teaching Fellowship by the Higher Education Academy (Google: HEA Trevor Thompson).
    [Show full text]
  • Perspectives on 'Communicating Evidence About Medicines'
    Perspectives on 'Communicating evidence about medicines' A report of a one-day workshop held by the Academy of Medical Sciences on 6 June 2016 Disclaimer This report does not represent a formal Academy of Medical Sciences position on how best to communicate evidence about medicines. Rather this document reflects the wide-ranging discussions that took place at the workshop. The report of this meeting will feed into the Academy’s workstream on ‘How can we all best use evidence to judge the potential benefits and harms of medicines?'.1 We hope that this report will also encourage wider discussion about communicating evidence about medicines. We would therefore welcome feedback on the report. For further information, please contact Dr Rachel Brown, Policy Officer at the Academy of Medical Sciences ([email protected], 020 3141 3223). We are most grateful to Professor Theresa Marteau FMedSci for Chairing this workshop and to Professor Sir David Spiegelhalter OBE FRS for his helpful contributions in the development of this workshop. We are also especially thankful to Dr Paul Robinson, Dr Jacintha Sivarajah, and Dr Catherine Harvey for their contributions to the workshop which were given at short notice, and to all other individuals who contributed to the event. All web references were accessed in June/July 2016. This work is © Academy of Medical Sciences and is licensed under Creative Commons Attribution 4.0 International. 1. Led by Professor Sir John Tooke FMedSci, the workstream also includes workshops on evaluating evidence in health, conflicts of interest, and communicating evidence about medicines in the media.
    [Show full text]
  • Special Topics in (Genetic) Epidemiology
    Special Topics in (Genetic) Epidemiology Li Hsu [email protected] Review: Epidemiologic Studies for Complex Diseases I Complex diseases are contributed by both genetic and environmental risk factors I Observational studies are powerful tools in studying these risk factors. Investigators have no control over exposure assignment. As a result, exposure of interest is often confounded by a third factor that is associated with exposure and the disease. Genetic Factors I Breakthroughs in high throughput genotyping and sequencing technologies have allowed researchers to assess genome-wide genetic effects on disease risk I High-dimension I Accurate measurements I Confounding can be effectively accounted for by principal components derived from genome-wide SNPs I Areas that need further development: set-based association; high-dimensional risk prediction; GxE (or GxG) interaction Environmental Risk Factors I Besides confounding, measurement error is an issue for environmental risk factors I Technologies such as wearable devices and metabolomics are being developed to better quantify aspects of environmental risk factors (e.g., diet and exercise) I Becoming more and more high-dimensional I Measurement error I Confounding remains to be a tricky issue I Areas that need further development: functional data analysis to better characterize the effects of environmental covariates; measurement error; Mendelian randomization/instrumental variables Study Designs I Two commonly used study designs: I Case-control studies I Case-control studies are restrospective
    [Show full text]
  • Benefit-Risk Assessment: IMI-PROTECT Case Study Example
    Benefit-Risk Assessment: IMI-PROTECT case study example Juhaeri Juhaeri, Ph.D. Head of Pharmacoepidemiology, Sanofi Adjunct Assistant Professor, UNC Chapel Hill 30th International Conference on Pharmacoepidemiology & Therapeutic Risk Management, Taipei, 24 October 2014 Disclaimers Juhaeri Juhaeri is an employee of Sanofi. The views and opinions represented in this presentation are solely those of the presenter and are not endorsed by nor necessarily representative of those of Sanofi The 5th Annual Meeting: Patient Safety - A Sustained Focus from Scientific Ideas to Innovative Medicines May 12-15, 2013 2 Objectives • To understand existing frameworks for a structured BR assessment • To describe quantitative BR methods to integrate benefits and risks Using rimonabant as an example 3 FDA Structured approach Decision Evidence and Conclusions Factor Uncertainties and Reasons Analysis of Condition Current Treatment Options Benefit Risk Risk Management Benefit-Risk Summary Assessment http://www.fda.gov/ForIndustry/UserFees/PrescriptionDrugUserFee/ucm326192.htm 4 EMA Project: 2 level approach • Qualitative approach • Quantitative approach: Multi-Criteria Decision Analysis (MCDA) method to derive a numerical value for the benefit–risk balance recommended for more complex situations *Zafiropoulos N, Phillips L, Pignatti F, Luria X. Evaluating benefit-risk: an agency perspective. Regulatory Rapporteur 2012 (9): 5-8 5 ISPE: Benefit-Risk Assessment, Communication and Evaluation (BRACE) 6 The Innovative Medicines Initiative (IMI) • The largest public-
    [Show full text]
  • Promoting the Use of Complex Innovative Designs in Clinical Trials
    Meeting March 20, 2018 Page 1 1 2 U.S. FOOD AND DRUG ADMINISTRATION 3 CENTER FOR DRUG EVALUATION AND RESEARCH (CDER) 4 CENTER FOR BIOLOGIC EVALUATION AND RESEARCH (CBER) 5 6 7 Public Meeting: 8 Promoting the Use of Complex Innovative Designs 9 in Clinical Trials 10 11 Tuesday, March 20, 2018 12 8:35 a.m. 13 14 FDA White Oak Campus 15 10903 New Hampshire Avenue 16 Silver Spring, Maryland 20903 17 18 19 20 Reported by: KeVon Congo, 21 Capital Reporting Company 22 www.CapitalReportingCompany.com 202-857-3376 Meeting March 20, 2018 Page 2 1 A P P E A R A N C E S 2 Deborah Ashby, PhD 3 Imperial College London 4 Julie Beitz, MD 5 Office of Drug Evaluation III, CDER, FDA 6 Robyn Bent 7 Office of Biostatistics, CDER, FDA 8 Scott Berry, PhD 9 Berry Consultants, LLC 10 Frank Bretz, PhD 11 Novartis 12 Aloka Chakravarty, PhD 13 Office of Biostatistics, CDER, FDA 14 Shein-Chung Chow, PhD 15 Office of Biostatistics, CDER, FDA 16 Ivan Chan, PhD 17 AbbVie Inc. 18 Scott Emerson, MD, PhD 19 University of Washington 20 Steven Goodman, MD, PhD 21 Stanford University School of Medicine 22 www.CapitalReportingCompany.com 202-857-3376 Meeting March 20, 2018 Page 3 1 A P P E A R A N C E S 2 Frank Harrell, Jr., PhD 3 Vanderbilt University School of Medicine 4 Telba Irony, PhD 5 Office of Biostatistics and Epidemiology, CBER, 6 FDA 7 Laura Lee Johnson, PhD 8 Office of Biostatistics, CDER, FDA 9 Stefanie Kraus 10 Office of Regulatory Counsel, CDER, FDA 11 Lisa LaVange, PhD 12 University of North Carolina at Chapel Hill 13 J.
    [Show full text]
  • Practical Help for Specifying the Target Difference in Sample Size Calculations for Rcts : the DELTA2 Five-Stage Study, Including a Workshop
    This is a repository copy of Practical help for specifying the target difference in sample size calculations for RCTs : the DELTA2 five-stage study, including a workshop. White Rose Research Online URL for this paper: http://eprints.whiterose.ac.uk/152823/ Version: Published Version Article: Cook, J.A., Julious, S.A., Sones, W. et al. (18 more authors) (2019) Practical help for specifying the target difference in sample size calculations for RCTs : the DELTA2 five-stage study, including a workshop. Health Technology Assessment, 23 (60). pp. 1-88. ISSN 1366-5278 https://doi.org/10.3310/hta23600 © Queen’s Printer and Controller of HMSO 2019. This work was produced by Cook et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK. Reuse Items deposited in White Rose Research Online are protected by copyright, with all rights reserved unless indicated otherwise. They may be downloaded and/or printed for private study, or other acts as permitted by national copyright laws. The publisher or other rights holders may allow further reproduction and re-use of the full text version.
    [Show full text]
  • Tackling the Structural Drivers of HIV
    ISSUE 31 • SUMMER 2012 Tackling the structural drivers of HIV In this issue… New MSc degrees in Nutrition and Global Mental Health Introducing the TB Centre Olympics and trafficking: the myths and the evidence CONTENTS Message from our Director 2012 Honorary Fellowships 3 Life at the School has been busy, with another Congratulations to our generation of students graduating from the School MSc class of 2011 4 during the Masters Graduation Day on 10 March and the Research Degrees Graduation Ceremony Health in the aftermath of war 5 on 8 June. These events are an important rite of passage for our graduates, marking a new phase Pioneering Bluetongue research 6 in their careers which will bring new experiences, Uncovering the myths and challenges and opportunities that will test the evidence around Olympics tools and expertise our graduates have been and trafficking 7 taught at the School. STRIVE: Tackling the During the Masters Graduation Day, we were very pleased to award honorary structural drivers of HIV 8-9 fellowships to Professor Sir Brian Greenwood, who this year has been knighted and awarded the Canada Gairdner Prize for his contribution to medicine Dramatic drop in child deaths 10 and public health, Professor Dame Valerie Beral, Director of Oxford’s Cancer Launch of collaborative Epidemiology Unit and formerly at the School, John Godfrey, a long standing infection research institute 11 School Board and Council member and Dr Tedros Adhanom Ghebreyesus, alumnus and the Ethiopian Minister for Health, whose leadership in Introducing the TB centre 12-13 malaria research and public health policy is internationally recognised.
    [Show full text]
  • Attributable Risk Percent in Case-Control Studies
    Br J Prev Soc Med: first published as 10.1136/jech.25.4.242 on 1 November 1971. Downloaded from Brit. J. prev. soc. Med. (1971), 25, 242- 244 ATTRIBUTABLE RISK PERCENT IN CASE-CONTROL STUDIES PHILIP COLE AND BRIAN MacMAHON Department ofEpidemiology, Harvard School ofPublic Health, 665 Huntington Avenue, Boston, Massachusetts 02115 The difference between the disease rates of two groups, one exposed and one not exposed to a (2) Ae% = X 100% suspected aetiological factor, is a useful measure of R the risk associated with the factor. This measure is where R, the relative risk, is the ratio of the risk of most interest when the association is considered among the exposed to that among the unexposed. causal, and has been termed the attributable risk The numerator of (2), termed the excess relative (MacMahon and Pugh, 1970). Expressed as a risk, is the segment of the relative risk among the percentage of the total disease rate in a group (the exposed which exceeds the risk among the un- attributable risk percent), the measure describes exposed. Since relative risk reflects total risk, the percentage of the group's total risk which is in expressing a group's excess relative risk as a per- excess of the risk among persons not exposed to centage of its relative risk yields the attributable the suspect factor. Both measures (attributable risk risk percent. Expression (2) applies then to relative and attributable risk percent) may be computed risk estimates such as are available from the usual either for persons exposed to the suspect factor or case-control study.
    [Show full text]
  • Study Designs and Their Outcomes
    © Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION © Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC CHAPTERNOT FOR SALE 3 OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION © JonesStudy & Bartlett Designs Learning, LLC and Their Outcomes© Jones & Bartlett Learning, LLC NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION “Natural selection is a mechanism for generating an exceedingly high degree of improbability.” —Sir Ronald Aylmer Fisher Peter Wludyka © Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION OBJECTIVES ______________________________________________________________________________ • Define research design, research study, and research protocol. • Identify the major features of a research study. • Identify© Jonesthe four types& Bartlett of designs Learning,discussed in this LLC chapter. © Jones & Bartlett Learning, LLC • DescribeNOT nonexperimental FOR SALE designs, OR DISTRIBUTIONincluding cohort, case-control, and cross-sectionalNOT studies. FOR SALE OR DISTRIBUTION • Describe the types of epidemiological parameters that can be estimated with exposed cohort, case-control, and cross-sectional studies along with the role, appropriateness, and interpreta- tion of relative risk and odds ratios in the context of design choice. • Define true experimental design and describe its role in assessing cause-and-effect relation- © Jones &ships Bartlett along with Learning, definitions LLCof and discussion of the role of ©internal Jones and &external Bartlett validity Learning, in LLC NOT FOR evaluatingSALE OR designs. DISTRIBUTION NOT FOR SALE OR DISTRIBUTION • Describe commonly used experimental designs, including randomized controlled trials (RCTs), after-only (post-test only) designs, the Solomon four-group design, crossover designs, and factorial designs.
    [Show full text]