DOCSLIB.ORG

Good Practices for Real‐World Data Studies of Treatment And/Or

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Good Practices for Real‐World Data Studies of Treatment And/Or VALUE IN HEALTH 20 (2017) 1003– 1008 Available online at www.sciencedirect.com journal homepage: www.elsevier.com/locate/jval Original Report Good Practices for Real‐World Data Studies of Treatment and/or Comparative Effectiveness: Recommendations from the Joint ISPOR‐ISPE Special Task Force on Real‐World Evidence in Health Care Decision Making à Marc L. Berger1, , Harold Sox2, Richard J. Willke3, Diana L. Brixner4, Hans‐Georg Eichler5, Wim Goettsch6, David Madigan7, Amr Makady6, Sebastian Schneeweiss8, Rosanna Tarricone9, Shirley V. Wang8, John Watkins10, C. Daniel Mullins11 1New York City, NY, USA; 2Patient-Centered Outcomes Research Institute, Washington, DC, USA; 3International Society for Pharmacoeconomics and Outcomes Research, Lawrenceville, NJ, USA; 4University of Utah, Salt Lake City, UT, USA; 5European Medicines Agency, London, UK; 6Zorginstituut Nederland and University of Utrecht, Utrecht, The Netherlands; 7Columbia University, New York City, NY, USA; 8Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; 9Bocconi University, Milan, Italy; 10Premera Blue Cross, Mountlake Terrace, WA, USA; 11University of Maryland, Baltimore, MD, USA ABSTRACT Purpose: Real‐world evidence (RWE) includes data from retrospective involvement in RWE studies. These recommendations, in concert with or prospective observational studies and observational registries and earlier recommendations about study methodology, provide a trust- provides insights beyond those addressed by randomized controlled worthy foundation for the expanded use of RWE in health care decision trials. RWE studies aim to improve health care decision making. making. Conclusion: The focus of these recommendations is good Methods: The International Society for Pharmacoeconomics and Out- procedural practices for studies that test a specifichypothesisina comes Research (ISPOR) and the International Society for Pharmacoepi- specific population. We recognize that some of the recommendations in demiology (ISPE) created a task force to make recommendations this report may not be widely adopted without appropriate incentives regarding good procedural practices that would enhance decision from decision makers, journal editors, and other key stakeholders. makers’ confidence in evidence derived from RWD studies. Peer review Keywords: comparative effectiveness, decision making, guidelines, by ISPOR/ISPE members and task force participants provided a consen- pharmacoepidemiology, real‐world data, treatment effectiveness. sus‐building iterative process for the topics and framing of recommen- dations. Results: The ISPOR/ISPE Task Force recommendations cover & 2017 Published by Elsevier Inc. on behalf of International Society for seventopicssuchasstudyregistration, replicability, and stakeholder Pharmacoeconomics and Outcomes Research (ISPOR). Introduction and interpreted accordingly, whereas data is but one component of the research plan. Evidence is shaped, while data simply are Real‐world evidence (RWE) is obtained from analyzing real‐world raw materials and alone are non‐informative.” RWE can inform data (RWD). The RWD is defined here briefly as data obtained the application of evidence from RCTs to health care decision outside the context of randomized controlled trials (RCTs) gen- making and provide insights beyond those addressed by RCTs. erated during routine clinical practice [1,2]. This includes data RWD studies assess both the care and health outcomes of from retrospective or prospective observational studies and patients in routine clinical practice and produce RWE. In contrast observational registries; some consider data from single arm to RCTs, patients and their clinicians choose treatments on the clinical trials as RWD. As stated in a 2007 International Society basis of the patient's clinical characteristics and preferences. for Pharmacoeconomics and Outcomes Research (ISPOR) task However, since the factors that influence treatment choice in force report, “Evidence is generated according to a research plan clinical practice may also influence clinical outcomes, RWD Marc L. Berger is a self‐employed part‐time consultant. This article is a joint publication by Pharmacoepidemiology and Drug Safety and Value in Health. * Address correspondence to: Marc L. Berger, New York, NY, USA. E-mail: [email protected] 1098-3015$36.00 – see front matter & 2017 Published by Elsevier Inc. on behalf of International Society for Pharmacoeconomics and Outcomes Research (ISPOR). http://dx.doi.org/10.1016/j.jval.2017.08.3019 1004 VALUE IN HEALTH 20 (2017) 1003– 1008 studies generally cannot yield definitive causal conclusions about We recognize that procedural integrity is necessary but not the effects of treatment. sufficient for including RWD studies in the body of evidence that Currently, most regulatory bodies and Health Technology health policy makers use to make a decision. Other factors play a Assessment (HTA) organizations use RWE for descriptive analyses role, including the study design, adjudication of outcomes (eg, disease epidemiology, treatment patterns, and burden of illness) (if relevant), transparency and methodological rigor of the and to assess treatment safety (the incidence of adverse effects) but analyses, and size of the treatment effect among others. None- not treatment effectiveness, whereas other decision makers also theless, the focus of this report is the role of procedural practices leverage RWD to varying extents with respect to effectiveness and that enhance the trustworthiness of the larger process, with a compartive effectiveness [3]. With increasing attention paid to the particular focus on the effectiveness or comparative effectiveness applicability of evidence to specific target populations, however, of pharmaceutical and biologic treatments. clinicians, payers, HTA organizations, regulators, and clinical guide- The scope of this report does not, by design, directly address linedevelopersarelikelytoturntoRWEtosharpendecisionmaking several important considerations when RWD studies are that heretofore had been guided principally by RCTs (frequently reviewed by decision makers: using a placebo control) in narrowly defined populations. Commonly voiced concerns about RWD studies include uncer- • Good practices for the design, analysis, and reporting of tainty about their internal validity, inaccurate recording of health observational RWD studies (ie, methodological standards). events, missing data, and opaque reporting of conduct and results These issues have been addressed by ISPOR, ISPE, the US Food [4]. Critics of such studies also worry that the RWE literature is and Drug Administration (FDA), the European Medicines biased because of “data dredging” (ie, conducting multiple analyses Agency, the European Network of Centres for Pharmacoepi- until one provides the hoped‐for result) and selective publication demiology and Pharmacovigilance (ENCePP®), and the Euro- (ie, journals' preference for publishing positive results) [5–8].Asa pean Network for Health Technology Assessment [11–23]. Their first step toward addressing these concerns, the ISPOR and the recommendations aim to improve the validity and relevance of International Society for Pharmacoepidemiology (ISPE) created a RWD studies by adequately addressing methodological issues task force to make recommendations regarding good procedural including confounding, bias, and uncertainty, which are the practices that would enhance decision makers' confidence in explicit objections raised by payers and regulatory bodies to evidence derived from RWD studies. using RWD studies to inform decision making. We define good procedural practices (or good “study hygiene”)as • Good practices for the conduct of pragmatic clinical trials policies about the planning, execution, and dissemination of RWD (pRCTs or PCTs). pRCTs are randomized trials with varying studies that help to assure the public of the integrity of the research amounts of pragmatic (real world) elements, including the use process and enhance confidence in the RWE produced from RWD of electronic health records or claims data to obtain study studies. Journal editors and regulatory agencies have established outcomes [24]. good procedural practices for pre‐approval RCTs, including study • We take no position on whether, and in which context, policy registration of trials on a public website prior to their conduct, the makers may deem observational studies sufficiently transpar- completion of an a priori protocol and data analysis plan, account- ent to routinely support initial regulatory approval and ability for documenting any changes in study procedures, and the reimbursement. expectation that all RCT results will be made public [9,10].A statement of complementary practices for RWD studies of treat- The focus of this report is RWE examining the effectiveness of ment effectiveness is lacking. Our report aims to fill this gap. pharmaceutical or biologic therapies. Although the recommenda- We differentiate 2 categories of RWD studies aiming to tions of this report will have relevance for all types of therapeutic provide data on treatment effectiveness that may differ in their health technologies, we recognize that applying these recommen- procedural approach: dations to technologies different from drugs (ie, medical devices) might require further refinement because of differences in the • Exploratory Treatment Effectiveness Studies. These studies evidence required for regulatory approval [25]. typically do not hypothesize the presence of a specific treat-
Load more

Recommended publications
  • Certification Accredited Schools List
    = Academic Clinical Research Programs Which Currently Meet Waiver Requirements for the Academy of Clinical Research Professional (Updated October 15, 2015) This list includes academic clinical research programs which, as of the above date, have been evaluated and found to meet the current waiver requirements established by the Academy of Clinical Research Professionals (Academy) Board of Trustees. Programs that meet the waiver requirements allow a candidate to waive 1,500 hours of hands-on experience performing the Essential Duties of the program to which the candidate has made application. This list is not to be considered exhaustive and is not designed to represent all the academic offerings available. Appearance on this list is not to be construed as an endorsement of its content or format by the Academy or a guarantee that a candidate will be eligible for certification. All individuals interested in enrolling in a program of study should evaluate any program on the basis of his or her personal needs. (* - programs marked with an asterisk MAY be accepted but acceptance will be dependent on the complete listing of courses completed) Individuals seeking additional information about each program should contact the institution directly. Academic Institution Program Name ( Click Title for Web Navigation) Academy of Applied Pharmaceutical Sciences Clinical Research Diploma Program American University of Health Sciences Master of Science in Clinical Research Anoka-Ramsey Community College Clinical Research Professional - Certificate Graduate
    [Show full text]
  • Placebo-Controlled Trials of New Drugs: Ethical Considerations
    Reviews/Commentaries/Position Statements COMMENTARY Placebo-Controlled Trials of New Drugs: Ethical Considerations DAVID ORENTLICHER, MD, JD als, placebo controls are not appropriate when patients’ health would be placed at significant risk (8–10). A placebo- controlled study for a new hair- uch controversy exists regarding sufficiently more effective than placebo to thickening agent could be justified; a the ethics of placebo-controlled justify its use. Finally, not all established placebo-controlled study for patients M trials in which an experimental therapies have been shown to be superior with moderate or severe hypertension therapy will compete with an already es- to placebo. If newer drugs are compared would not be acceptable (11). Similarly, if tablished treatment (or treatments). In with the unproven existing therapies, an illness causes problems when it goes such cases, argue critics, patients in the then patients may continue to receive untreated for a long period of time, a 52- control arm of the study should receive an drugs that are harmful without being week study with a placebo control is accepted therapy rather than a placebo. helpful. much more difficult to justify than a By using an active and effective drug, the Moreover, say proponents of placebo 6-week study (12). control patients would not be placed at controls, patients can be protected from When David S.H. Bell (13) explains risk for deterioration of their disease, and harm by “escape” criteria, which call for why placebo controls are unacceptable the study would generate more meaning- withdrawal from the trial if the patient for new drugs to treat type 2 diabetes, he ful results for physicians.
    [Show full text]
  • Generating Real-World Evidence by Strengthening Real-World Data Sources
    Generating Real-World Evidence by Strengthening Real-World Data Sources Using “real-world evidence” to bring new “Every day, health care professionals are updating patients’ treatments to patients as part of the 21st electronic health records with data on clinical outcomes Century Cures Act (the “Cures Act”) is a key resulting from medical interventions used in routine clinical priority for the Department of Health and practice. As our experience with new medical products Human Services (HHS). Specifically, the Cures expands, our knowledge about how to best maximize their Act places focus on the use of real-world data benefits and minimize potential risks sharpens with each data to support regulatory decision-making, including point we gather. Every clinical use of a product produces data the approval of new indications for existing that can help better inform us about its safety and efficacy.” drugs in order to make drug development faster jacqueline corrigan-curay, md, jd and more efficient. As such, the Cures Act director of the office of medical policy in fda’s center for drug evaluation and research has tasked the Food and Drug Administration (FDA) to develop a framework and guidance for evaluating real-world evidence in the context of drug regulation.1 Under the FDA’s framework, real-world evidence (RWE) is generated by different study designs or analyses, including but not limited to, randomized trials like large simple trials, pragmatic trials, and observational studies. RWE is the clinical evidence about the use, potential benefits, and potential risks of a medical product based on an analysis of real-world data (RWD).
    [Show full text]
  • Studies of Staphylococcal Infections. I. Virulence of Staphy- Lococci and Characteristics of Infections in Embryonated Eggs * WILLIAM R
    Journal of Clinical Investigation Vol. 43, No. 11, 1964 Studies of Staphylococcal Infections. I. Virulence of Staphy- lococci and Characteristics of Infections in Embryonated Eggs * WILLIAM R. MCCABE t (From the Research Laboratory, West Side Veterans Administration Hospital, and the Department of Medicine, Research and Educational Hospitals, University of Illinois College of Medicine, Chicago, Ill.) Many of the determinants of the pathogenesis niques still require relatively large numbers of and course of staphylococcal infections remain staphylococci to produce infection (19). Fatal imprecisely defined (1, 2) despite their increas- systemic infections have been equally difficult to ing importance (3-10). Experimental infections produce in animals and have necessitated the in- in suitable laboratory animals have been of con- jection of 107 to 109 bacteria (20-23). A few siderable assistance in clarifying the role of host strains of staphylococci have been found that are defense mechanisms and specific bacterial virulence capable of producing lethal systemic infections factors with a variety of other infectious agents. with inocula of from 102 to 103 bacteria (24) and A sensitive experimental model would be of value have excited considerable interest (25-27). The in defining the importance of these factors in virulence of these strains apparently results from staphylococcal infections, but both humans and an unusual antigenic variation (27, 28) which, the usual laboratory animals are relatively re- despite its interest, is of doubtful significance in sistant. Extremely large numbers of staphylo- human staphylococcal infection, since such strains cocci are required to produce either local or sys- have been isolated only rarely from clinical in- temic infections experimentally.
    [Show full text]
  • Using Real-World Evidence to Accelerate Safe and Effective Cures Advancing Medical Innovation for a Healthier America June 2016 Leadership Senator William H
    Using Real-World Evidence to Accelerate Safe and Effective Cures Advancing Medical Innovation for a Healthier America June 2016 Leadership Senator William H. Frist, MD Former U.S. Senate Majority Leader Chair, FDA: Advancing Medical Innovation Bipartisan Policy Center Representative Bart Gordon Former Member, U.S. House of Representatives Chair, FDA: Advancing Medical Innovation Bipartisan Policy Center Advisory Committee Marc M. Boutin, JD Chief Executive Officer National Health Council Mark McClellan, MD, PhD Director, Robert J. Margolis Center for Health Policy Duke University Patrick Soon-Shiong, MD Chairman and Chief Executive Officer Institute for Advanced Health Andrew von Eschenbach, MD President Samaritan Health Initiatives 1 Sta G. William Hoagland Ann Gordon Senior Vice President Writer Bipartisan Policy Center Michael Ibara, PharmD Janet M. Marchibroda Independent Consultant Director, Health Innovation Initiative and Executive Director, CEO Council on Health and Innovation Bipartisan Policy Center Tim Swope Senior Policy Analyst Bipartisan Policy Center Sam Watters Administrative Assistant Bipartisan Policy Center 2 FDA: ADVANCING MEDICAL INNOVATION EFFORT The Bipartisan Policy Center’s initiative, FDA: Advancing Medical Innovation, is developing viable policy options to advance medical innovation and reduce the time and cost associated with the discovery, development, and delivery of safe and effective drugs and devices for patients in the United States. Key areas of focus include the following: Improving the medical product development process; Increasing regulatory clarity; Strengthening the Food and Drug Administration’s (FDA) ability to carry out its mission; Using information technology to improve health and health care; and Increasing investment in medical products to address unmet and public health needs.
    [Show full text]
  • Short List of Commonly Used Acronyms in Clinical Research
    Short List of Commonly Used Acronyms in Clinical Research ACRP Association of Clinical Research Professionals AE Adverse Event ADR Adverse Drug Reaction AMA American Medical Association BAA Business Associate Agreement BIND Biological IND CA Coverage Analysis CAP College of American Pathologists CBCTN Community Based Clinical Trials Network CCRA Certified Clinical Research Associate (ACRP) CCRC Certified Clinical Research Coordinator (ACRP) CCRT Center for Clinical Research & Technology CCRP Certified Clinical Research Professional (SoCRA) CDA Confidential Disclosure Agreement CDC Center for Disease Control CFR Code of Federal Regulations CIRBI Center for IRB Intelligence CITI Collaborative Institutional Training Initiative CLIA Clinical Laboratory Improvement Amendments CME Continuing Medical Education CMS Central Management System COI Conflict of Interest CRA Clinical Research Associate CRC Clinical Research Coordinator CREC Continuing Research Education Credit CRF Case Report Form CRO Clinical Research Organization CTA Clinical Trial Agreement CTSC Clinical & Translational Science Collaborative CV Curriculum Vitae DCF Data Correction Form / Data Clarification Form DEA Drug Enforcement Agency (law enforcement division of FDA) DHHS Department of Health and Human Services DOS Date of Service DSMB or Data and Safety Monitoring Board (Plan or Committee) DSMP/C EAB Ethical Advisory Board (similar to IRB, used by other nations) EDC Electronic Data Capture FDA Food and Drug Administration FDA-482 Notice of Inspection FDA-483 Notice of Adverse
    [Show full text]
  • A Framework to Assess Empirical Reproducibility in Biomedical Research Leslie D
    McIntosh et al. BMC Medical Research Methodology (2017) 17:143 DOI 10.1186/s12874-017-0377-6 RESEARCH ARTICLE Open Access Repeat: a framework to assess empirical reproducibility in biomedical research Leslie D. McIntosh1, Anthony Juehne1*, Cynthia R. H. Vitale2, Xiaoyan Liu1, Rosalia Alcoser1, J. Christian Lukas1 and Bradley Evanoff3 Abstract Background: The reproducibility of research is essential to rigorous science, yet significant concerns of the reliability and verifiability of biomedical research have been recently highlighted. Ongoing efforts across several domains of science and policy are working to clarify the fundamental characteristics of reproducibility and to enhance the transparency and accessibility of research. Methods: The aim of the proceeding work is to develop an assessment tool operationalizing key concepts of research transparency in the biomedical domain, specifically for secondary biomedical data research using electronic health record data. The tool (RepeAT) was developed through a multi-phase process that involved coding and extracting recommendations and practices for improving reproducibility from publications and reports across the biomedical and statistical sciences, field testing the instrument, and refining variables. Results: RepeAT includes 119 unique variables grouped into five categories (research design and aim, database and data collection methods, data mining and data cleaning, data analysis, data sharing and documentation). Preliminary results in manually processing 40 scientific manuscripts indicate components of the proposed framework with strong inter-rater reliability, as well as directions for further research and refinement of RepeAT. Conclusions: The use of RepeAT may allow the biomedical community to have a better understanding of the current practices of research transparency and accessibility among principal investigators.
    [Show full text]
  • Independent Data and Safety Oversigh for DMID Clinical Research
    Policy No.: National Institute of Allergy and DMID Policy-010 - NCRS 1.2 v 2.0 Infectious Diseases / Division of Independent Data and Safety Oversight Microbiology and Infectious for DMID Clinical Research Diseases/Office of Clinical Research Affairs Effective Date: 13-JUL-2016 Version: 2.0 1.0 Purpose: To describe the Division of Microbiology and Infectious Diseases (DMID) policy for establishing independent data and safety oversight for DMID-supported clinical research. 2.0 Scope: This policy applies to DMID-supported clinical research studies where there are concerns about participant safety or data integrity. These types of studies require independent safety assessment. 3.0 Policy: An appropriate safety oversight plan will be described in the protocol. If applicable, an appropriate independent safety charter for all clinical research commensurate with risks, the size, and complexity of the study will be prepared. This plan will provide a description of the review of safety and/or efficacy data to assess volunteer safety and data integrity. The plan will state the level of independent oversight necessary or that no independent oversight is required. For grant-funded clinical research that does not require a formal protocol, the awarded grant will serve as the document describing the appropriate level of oversight. 4.0 Background: NIH policy requires that each Institute and Center (IC) have a system for managing/implementing the appropriate oversight of the conduct of clinical trials that ensures the safety of participants and the validity and integrity of the data. DMID extends this requirement to other clinical research where there are safety concerns. Data and safety oversight provides independent and objective review of the overall conduct of the study.
    [Show full text]
  • Module Catalogue Clinical Research and Epidemiology
    Module Catalogue for the Subject Clinical Research and Epidemiology as a Master’s with 1 major with the degree "Master of Science" (90 ECTS credits) Examination regulations version: 2015 Responsible: Faculty of Medicine JMU Würzburg • generated 23-Aug-2021 • exam. reg. data record 88|g73|-|-|H|2015 Module Catalogue for the Subject Clinical Research and Epidemiology Master’s with 1 major, 90 ECTS credits Contents The subject is divided into 3 Content and Objectives of the Programme 4 Abbreviations used, Conventions, Notes, In accordance with 5 Compulsory Courses 6 Compulsory Courses I: Basics of Epidemiology and Biometry 7 Introduction into epidemiology and biometry 8 Biometrical methods 9 Applied biometry 10 Epidemiological methods and evidence-based medicine 11 Compulsory Courses II: Clinical and Epidemiological Research 12 Topical clinical research questions and interpretation of clinical trials 13 Disease-specific epidemiology and prognostic modelling 14 Research methodology (I: Clinical trial methodology, II: Transferable skills training) 15 Compulsory Courses III: Research Placement 16 Practical training I 17 Practical training II 18 Thesis 19 Master Thesis Clinical Research and Epidemiology 20 Colloquium Master Thesis 21 Master’s with 1 major Clinical Research and Epide- JMU Würzburg • generated 23-Aug-2021 • exam. reg. data record page 2 / 21 miology (2015) Master (90 ECTS) Klinische Forschung und Epidemiologie - 2015 Module Catalogue for the Subject Clinical Research and Epidemiology Master’s with 1 major, 90 ECTS credits The subject is divided into starting section / sub-section ECTS credits page Compulsory Courses 60 6 Compulsory Courses I: Basics of Epidemiology and Biometry 22 7 Compulsory Courses II: Clinical and Epidemiological Research 18 12 Compulsory Courses III: Research Placement 20 16 Thesis 30 19 Master’s with 1 major Clinical Research and Epide- JMU Würzburg • generated 23-Aug-2021 • exam.
    [Show full text]
  • Clinical Research Milestones
    Clinical research milestones: Prior to award, investigators conducting clinical studies or clinical trials will be required to provide detailed interim performance measures and timelines for completing key objectives and administrative functions for the proposed clinical study or trial, as applicable. Milestones should be easily measurable and realistic. Milestones may include, as applicable, but are not limited to: • Finalization of the clinical study/trial protocol(s) and informed consent/assent forms (with NIDDK program official agreement, if applicable) and IRB approval • Registration of the clinical trial(s) in ClinicalTrials.gov • Completion of all required regulatory approvals (e.g., Investigational New Drug Application from the Food and Drug Administration) • Contracts/third party agreements, including intervention product supply • Good Clinical Practice (GCP) and other relevant technical training of study staff • Anticipated date of enrollment of the first participant • Recruitment and/or randomization of 25%, 50%, 75% and 100% of the target sample size • Follow-up visit completion, if applicable, of 25%, 50%, 75% and 100% of the enrolled or randomized study population, including women, minorities and children • Completion of data collection • Completion of primary endpoint and secondary endpoint data analyses • Completion of final study report and manuscript submission • Closeout plans/communication of results to participants • Reporting of results in ClinicalTrials.gov. These milestones will be negotiated at the time of the award, as appropriate. Future year support is contingent on satisfactory achievement of performance milestones. If milestones are not achieved fully, NIDDK may request development of a remedial plan and more frequent monitoring of progress, or take other remedial actions. .
    [Show full text]
  • NIAID's Antibiotic Resistance Research Framework: Current Status and Future Directions
    NIAID’S Antibiotic Resistance Research Framework: NIAID Current Status and Future Directions 2019 NIAID's Antibiotic Resistance Research Framework: Current Status and Future Directions Table of Contents Executive Summary ............................................................................................................................ 1 Introduction ......................................................................................................................................... 2 The Evolution and Accelerating Pace of Antibiotic Resistance (AR) ................................................2 The Antibiotic Development Pipeline ................................................................................................. 3 NIAID: Stimulating Research on AR .................................................................................................. 3 Basic Research................................................................................................................................. 3 Translational Research .................................................................................................................... 4 Clinical Research............................................................................................................................. 6 Research Resources ......................................................................................................................... 8 Partnerships Facilitate a Concerted Approach ...................................................................................
    [Show full text]
  • Reproducibility of Clinical Research in Critical Care: a Scoping Review Daniel J
    Niven et al. BMC Medicine (2018) 16:26 https://doi.org/10.1186/s12916-018-1018-6 RESEARCH ARTICLE Open Access Reproducibility of clinical research in critical care: a scoping review Daniel J. Niven1*, T. Jared McCormick2, Sharon E. Straus3, Brenda R. Hemmelgarn4, Lianne Jeffs5, Tavish R. M. Barnes1 and Henry T. Stelfox1 Abstract Background: The ability to reproduce experiments is a defining principle of science. Reproducibility of clinical research has received relatively little scientific attention. However, it is important as it may inform clinical practice, research agendas, and the design of future studies. Methods: We used scoping review methods to examine reproducibility within a cohort of randomized trials examining clinical critical care research and published in the top general medical and critical care journals. To identify relevant clinical practices, we searched the New England Journal of Medicine, The Lancet, and JAMA for randomized trials published up to April 2016. To identify a comprehensive set of studies for these practices, included articles informed secondary searches within other high-impact medical and specialty journals. We included late-phase randomized controlled trials examining therapeutic clinical practices in adults admitted to general medical-surgical or specialty intensive care units (ICUs). Included articles were classified using a reproducibility framework. An original study was the first to evaluate a clinical practice. A reproduction attempt re-evaluated that practice in a new set of participants. Results: Overall, 158 practices were examined in 275 included articles. A reproduction attempt was identified for 66 practices (42%, 95% CI 33–50%). Original studies reported larger effectsthanreproductionattempts(primary endpoint, risk difference 16.0%, 95% CI 11.6–20.5% vs.
    [Show full text]