PROTOCOL: Evaluating Reproducibility and Robustness of Real World Evidence from Large Healthcare Databases Version: Jun 16, 2018
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Reproducibility, Replicability, and Generalization in the Social, Behavioral, and Economic Sciences
REPRODUCIBILITY, REPLICABILITY, AND GENERALIZATION IN THE SOCIAL, BEHAVIORAL, AND ECONOMIC SCIENCES Report of the Subcommittee on Replicability in Science of the SBE Advisory Committee to the National Science Foundation 13 May 2015 Presentation at SBE AC Spring Meeting by K. Bollen. Report of the Subcommittee on Replicability in Science Advisory Committee to the NSF SBE Directorate Subcommittee Members: Kenneth Bollen (University of North Carolina at Chapel Hill, Cochair) John T. Cacioppo (University of Chicago, Cochair) Robert M. Kaplan (Agency for Healthcare Research and Quality) Jon A. Krosnick (Stanford University) James L. Olds (George Mason University) Staff Assistance Heather Dean (National Science Foundation) TOPICS I. Background II. Subcommittee Report III. Definitions IV. Recommendations V. Final Comments Background • Spring, 2013 o NSF SBE Advisory Committee establishes subcommittee on how SBE can promote robust research practices • Summer & Fall 2013 o Subcommittee proposal for workshop on “Robust Research in the Social, Behavioral, and Economic Sciences.” • February 20-21, 2014 o Workshop convened o Participants drawn from variety of universities, funding agencies, scientific associations, and journals o Cover broad range of issues from extent and cause of problems to possible solutions o Details are in the appendix of the circulated report • Post-workshop period o Document & digest workshop content o Discuss and propose recommendations o Complete report Subcommittee Report • DEFINITIONS o No consensus in science on the meanings -
FDA Oncology Experience in Innovative Adaptive Trial Designs
Innovative Adaptive Trial Designs Rajeshwari Sridhara, Ph.D. Director, Division of Biometrics V Office of Biostatistics, CDER, FDA 9/3/2015 Sridhara - Ovarian cancer workshop 1 Fixed Sample Designs • Patient population, disease assessments, treatment, sample size, hypothesis to be tested, primary outcome measure - all fixed • No change in the design features during the study Adaptive Designs • A study that includes a prospectively planned opportunity for modification of one or more specified aspects of the study design and hypotheses based on analysis of data (interim data) from subjects in the study 9/3/2015 Sridhara - Ovarian cancer workshop 2 Bayesian Designs • In the Bayesian paradigm, the parameter measuring treatment effect is regarded as a random variable • Bayesian inference is based on the posterior distribution (Bayes’ Rule – updated based on observed data) – Outcome adaptive • By definition adaptive design 9/3/2015 Sridhara - Ovarian cancer workshop 3 Adaptive Designs (Frequentist or Bayesian) • Allows for planned design modifications • Modifications based on data accrued in the trial up to the interim time • Unblinded or blinded interim results • Control probability of false positive rate for multiple options • Control operational bias • Assumes independent increments of information 9/3/2015 Sridhara - Ovarian cancer workshop 4 Enrichment Designs – Prognostic or Predictive • Untargeted or All comers design: – post-hoc enrichment, prospective-retrospective designs – Marker evaluation after randomization (example: KRAS in cetuximab -
FORMULAS from EPIDEMIOLOGY KEPT SIMPLE (3E) Chapter 3: Epidemiologic Measures
FORMULAS FROM EPIDEMIOLOGY KEPT SIMPLE (3e) Chapter 3: Epidemiologic Measures Basic epidemiologic measures used to quantify: • frequency of occurrence • the effect of an exposure • the potential impact of an intervention. Epidemiologic Measures Measures of disease Measures of Measures of potential frequency association impact (“Measures of Effect”) Incidence Prevalence Absolute measures of Relative measures of Attributable Fraction Attributable Fraction effect effect in exposed cases in the Population Incidence proportion Incidence rate Risk difference Risk Ratio (Cumulative (incidence density, (Incidence proportion (Incidence Incidence, Incidence hazard rate, person- difference) Proportion Ratio) Risk) time rate) Incidence odds Rate Difference Rate Ratio (Incidence density (Incidence density difference) ratio) Prevalence Odds Ratio Difference Macintosh HD:Users:buddygerstman:Dropbox:eks:formula_sheet.doc Page 1 of 7 3.1 Measures of Disease Frequency No. of onsets Incidence Proportion = No. at risk at beginning of follow-up • Also called risk, average risk, and cumulative incidence. • Can be measured in cohorts (closed populations) only. • Requires follow-up of individuals. No. of onsets Incidence Rate = ∑person-time • Also called incidence density and average hazard. • When disease is rare (incidence proportion < 5%), incidence rate ≈ incidence proportion. • In cohorts (closed populations), it is best to sum individual person-time longitudinally. It can also be estimated as Σperson-time ≈ (average population size) × (duration of follow-up). Actuarial adjustments may be needed when the disease outcome is not rare. • In an open populations, Σperson-time ≈ (average population size) × (duration of follow-up). Examples of incidence rates in open populations include: births Crude birth rate (per m) = × m mid-year population size deaths Crude mortality rate (per m) = × m mid-year population size deaths < 1 year of age Infant mortality rate (per m) = × m live births No. -
Incidence and Secondary Transmission of SARS-Cov-2 Infections in Schools
Prepublication Release Incidence and Secondary Transmission of SARS-CoV-2 Infections in Schools Kanecia O. Zimmerman, MD; Ibukunoluwa C. Akinboyo, MD; M. Alan Brookhart, PhD; Angelique E. Boutzoukas, MD; Kathleen McGann, MD; Michael J. Smith, MD, MSCE; Gabriela Maradiaga Panayotti, MD; Sarah C. Armstrong, MD; Helen Bristow, MPH; Donna Parker, MPH; Sabrina Zadrozny, PhD; David J. Weber, MD, MPH; Daniel K. Benjamin, Jr., MD, PhD; for The ABC Science Collaborative DOI: 10.1542/peds.2020-048090 Journal: Pediatrics Article Type: Regular Article Citation: Zimmerman KO, Akinboyo IC, Brookhart A, et al. Incidence and secondary transmission of SARS-CoV-2 infections in schools. Pediatrics. 2021; doi: 10.1542/peds.2020- 048090 This is a prepublication version of an article that has undergone peer review and been accepted for publication but is not the final version of record. This paper may be cited using the DOI and date of access. This paper may contain information that has errors in facts, figures, and statements, and will be corrected in the final published version. The journal is providing an early version of this article to expedite access to this information. The American Academy of Pediatrics, the editors, and authors are not responsible for inaccurate information and data described in this version. Downloaded from©2021 www.aappublications.org/news American Academy by of guest Pediatrics on September 27, 2021 Prepublication Release Incidence and Secondary Transmission of SARS-CoV-2 Infections in Schools Kanecia O. Zimmerman, MD1,2,3; Ibukunoluwa C. Akinboyo, MD1,2; M. Alan Brookhart, PhD4; Angelique E. Boutzoukas, MD1,2; Kathleen McGann, MD2; Michael J. -
Estimated HIV Incidence and Prevalence in the United States
Volume 26, Number 1 Estimated HIV Incidence and Prevalence in the United States, 2015–2019 This issue of the HIV Surveillance Supplemental Report is published by the Division of HIV/AIDS Prevention, National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention, Centers for Disease Control and Prevention (CDC), U.S. Department of Health and Human Services, Atlanta, Georgia. Estimates are presented for the incidence and prevalence of HIV infection among adults and adolescents (aged 13 years and older) based on data reported to CDC through December 2020. The HIV Surveillance Supplemental Report is not copyrighted and may be used and reproduced without permission. Citation of the source is, however, appreciated. Suggested citation Centers for Disease Control and Prevention. Estimated HIV incidence and prevalence in the United States, 2015–2019. HIV Surveillance Supplemental Report 2021;26(No. 1). http://www.cdc.gov/ hiv/library/reports/hiv-surveillance.html. Published May 2021. Accessed [date]. On the Web: http://www.cdc.gov/hiv/library/reports/hiv-surveillance.html Confidential information, referrals, and educational material on HIV infection CDC-INFO 1-800-232-4636 (in English, en Español) 1-888-232-6348 (TTY) http://wwwn.cdc.gov/dcs/ContactUs/Form Acknowledgments Publication of this report was made possible by the contributions of the state and territorial health departments and the HIV surveillance programs that provided surveillance data to CDC. This report was prepared by the following staff and contractors of the Division of HIV/AIDS Prevention, National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention, CDC: Laurie Linley, Anna Satcher Johnson, Ruiguang Song, Sherry Hu, Baohua Wu, H. -
Understanding Relative Risk, Odds Ratio, and Related Terms: As Simple As It Can Get Chittaranjan Andrade, MD
Understanding Relative Risk, Odds Ratio, and Related Terms: As Simple as It Can Get Chittaranjan Andrade, MD Each month in his online Introduction column, Dr Andrade Many research papers present findings as odds ratios (ORs) and considers theoretical and relative risks (RRs) as measures of effect size for categorical outcomes. practical ideas in clinical Whereas these and related terms have been well explained in many psychopharmacology articles,1–5 this article presents a version, with examples, that is meant with a view to update the knowledge and skills to be both simple and practical. Readers may note that the explanations of medical practitioners and examples provided apply mostly to randomized controlled trials who treat patients with (RCTs), cohort studies, and case-control studies. Nevertheless, similar psychiatric conditions. principles operate when these concepts are applied in epidemiologic Department of Psychopharmacology, National Institute research. Whereas the terms may be applied slightly differently in of Mental Health and Neurosciences, Bangalore, India different explanatory texts, the general principles are the same. ([email protected]). ABSTRACT Clinical Situation Risk, and related measures of effect size (for Consider a hypothetical RCT in which 76 depressed patients were categorical outcomes) such as relative risks and randomly assigned to receive either venlafaxine (n = 40) or placebo odds ratios, are frequently presented in research (n = 36) for 8 weeks. During the trial, new-onset sexual dysfunction articles. Not all readers know how these statistics was identified in 8 patients treated with venlafaxine and in 3 patients are derived and interpreted, nor are all readers treated with placebo. These results are presented in Table 1. -
Disease Incidence, Prevalence and Disability
Part 3 Disease incidence, prevalence and disability 9. How many people become sick each year? 28 10. Cancer incidence by site and region 29 11. How many people are sick at any given time? 31 12. Prevalence of moderate and severe disability 31 13. Leading causes of years lost due to disability in 2004 36 World Health Organization 9. How many people become sick each such as diarrhoeal disease or malaria, it is common year? for individuals to be infected repeatedly and have several episodes. For such conditions, the number The “incidence” of a condition is the number of new given in the table is the number of disease episodes, cases in a period of time – usually one year (Table 5). rather than the number of individuals affected. For most conditions in this table, the figure given is It is important to remember that the incidence of the number of individuals who developed the illness a disease or condition measures how many people or problem in 2004. However, for some conditions, are affected by it for the first time over a period of Table 5: Incidence (millions) of selected conditions by WHO region, 2004 Eastern The Mediter- South- Western World Africa Americas ranean Europe East Asia Pacific Tuberculosisa 7.8 1.4 0.4 0.6 0.6 2.8 2.1 HIV infectiona 2.8 1.9 0.2 0.1 0.2 0.2 0.1 Diarrhoeal diseaseb 4 620.4 912.9 543.1 424.9 207.1 1 276.5 1 255.9 Pertussisb 18.4 5.2 1.2 1.6 0.7 7.5 2.1 Measlesa 27.1 5.3 0.0e 1.0 0.2 17.4 3.3 Tetanusa 0.3 0.1 0.0 0.1 0.0 0.1 0.0 Meningitisb 0.7 0.3 0.1 0.1 0.0 0.2 0.1 Malariab 241.3 203.9 2.9 8.6 0.0 23.3 2.7 -
Metabolic Sensor Governing Bacterial Virulence in Staphylococcus Aureus
Metabolic sensor governing bacterial virulence in PNAS PLUS Staphylococcus aureus Yue Dinga, Xing Liub, Feifei Chena, Hongxia Dia, Bin Xua, Lu Zhouc, Xin Dengd,e, Min Wuf, Cai-Guang Yangb,1, and Lefu Lana,1 aDepartment of Molecular Pharmacology and bChinese Academy of Sciences Key Laboratory of Receptor Research, Shanghai Institute of Materia Medica, Chinese Academy of Sciences, Shanghai 201203, China; cDepartment of Medicinal Chemistry, School of Pharmacy, Fudan University, Shanghai 201203, China; dDepartment of Chemistry and eInstitute for Biophysical Dynamics, The University of Chicago, Chicago, IL 60637; and fDepartment of Basic Sciences University of North Dakota School of Medicine and Health Sciences, Grand Forks, ND 58203 Edited by Richard P. Novick, New York University School of Medicine, New York, NY, and approved October 14, 2014 (received for review June 13, 2014) An effective metabolism is essential to all living organisms, in- To survive and replicate efficiently in the host, S. aureus has cluding the important human pathogen Staphylococcus aureus.To developed exquisite mechanisms for scavenging nutrients and establish successful infection, S. aureus must scavenge nutrients adjusting its metabolism to maintain growth while also coping with and coordinate its metabolism for proliferation. Meanwhile, it also stress (6, 11). On the other hand, S. aureus produces a wide array must produce an array of virulence factors to interfere with host of virulence factors to evade host immune defenses and to derive defenses. However, the ways in which S. aureus ties its metabolic nutrition either parasitically or destructively from the host during state to its virulence regulation remain largely unknown. Here we infections (6). -
Principles of Validation of Diagnostic Assays for Infectious Diseases1
PRINCIPLES OF VALIDATION OF DIAGNOSTIC ASSAYS FOR INFECTIOUS DISEASES1 R. H. JACOBSON, New York State College of Veterinary Medicine, Cornell University, Ithaca, New York, USA Abstract PRINCIPLES OF VALIDATION OF DIAGNOSTIC ASSAYS FOR INFECTIOUS DISEASES. Assay validation requires a series of inter-related processes. Assay validation is an experimental process: reagents and protocols are optimized by experimentation to detect the analyte with accuracy and precision. Assay validation is a relative process: its diagnostic sensitivity and diagnostic specificity are calculated relative to test results obtained from reference animal populations of known infection/exposure status. Assay validation is a conditional process: classification of animals in the target population as infected or uninfected is conditional upon how well the reference animal population used to validate the assay represents the target population; accurate predictions of the infection status of animals from test results (PV+ and PV−) are conditional upon the estimated prevalence of disease/infection in the target population. Assay validation is an incremental process: confidence in the validity of an assay increases over time when use confirms that it is robust as demonstrated by accurate and precise results; the assay may also achieve increasing levels of validity as it is upgraded and extended by adding reference populations of known infection status. Assay validation is a continuous process: the assay remains valid only insofar as it continues to provide accurate and precise results as proven through statistical verification. Therefore, the work required for validation of diagnostic assays for infectious diseases does not end with a time-limited series of experiments based on a few reference samples rather, to assure valid test results from an assay requires constant vigilance and maintenance of the assay, along with reassessment of its performance characteristics for each unique population of animals to which it is applied. -
New NIH Guidelines to Improve Experimental Rigor and Reproducibility
New NIH guidelines to improve Experimental Rigor and Reproducibility Oswald Steward Professor, Department of Anatomy & Neurobiology Senior Associate Dean for Research University of California School of Medicine What I’ll tell you 1) Background: Increasing reports of failures to replicate 2) Previous actions by NIH to enhance rigor 3) New requirements for grants submitted after January 2016 4) What NIH will be looking for in new required sections 5) Some suggested best practices to meet new standards for scientific rigor • The Problem: Numerous recent reports of findings that can’t be replicated. Failure to replicate key results from preclinical cancer studies: 1) Prinz et al. (2011) Believe it or not: How much can we rely on published data on potential drug targets? Nature Reviews Drug Discovery. Inconsistencies were found in 2/3 studies 2) Begley and Ellis (2012) Raise standards for preclinical cancer research. Nature. Only 6/53 “landmark” papers replicated 3) Begley (2013) Six red flags for suspect work. Nature. -Were experiments performed blinded? -Were basic experiments repeated? -Were all the results presented? -Were there positive and negative controls? -Were reagents validated? -Were statistical tests appropriate? • In ALS field, Scott et al., (2008) re-assessed 70 drugs that had been reported to increase lifespan in an animal model of ALS (SOD1-G93A knockout mice). • 221 separate studies were carried out over a five year period that involved over 18,000 mice. • Some drugs were already in clinical trials. • Outcome: There was no statistically significant positive effect for any of the 70 compounds tested. • Conclusion: Previously published effects were measurements of noise in the distribution of survival means as opposed to an actual drug effect. -
Incidence Density Sampling for Nested Case-Control Study Designs
Paper AS09 Incidence Density Sampling for Nested Case-Control Study Designs Quratul Ann, IQVIA, London, United Kingdom Rachel Tham, IQVIA, London, United Kingdom ABSTRACT The nested case-control design is commonly used in safety studies, as it is an efficient approach to an epidemiological investigation. When the design is implemented appropriately, it can yield unbiased estimates of relative risk; and in exchange for a small loss in precision, considerable improved efficiency and reduction in costs can be achieved. There are several ways of sampling controls to cases in a nested case-control design. The most sophisticated way is via incidence density sampling. Incidence density sampling matches cases to controls based on the dynamic risk set at the time of case occurrence, where the probability of control selection is proportional to the total person-time at risk. Incidence density sampling alleviates the rare disease assumption; however, it is rarely used due to its computational complexity. This paper presents a novel and simple Statistical Analysis System (SAS) program for incidence density sampling, which could minimise bias and introduce a more appropriate way of optimising the matching of controls to cases. INTRODUCTION Retrospective studies gather available previous exposure information for a clearly defined source population and the outcome event is determined for all members of the cohort4. Retrospective studies facilitate an efficient approach to draw inferences and determine the association between exposure and disease prevalence particularly for rare outcomes and with long-term disease conditions. Real-world data (RWD) are a valuable source of information for investigating health-related outcomes in human populations. -
Incidence, Risk, and Case Fatality of First Ever Stroke in the Elderly Population
J Neurol Neurosurg Psychiatry: first published as 10.1136/jnnp.74.3.317 on 1 March 2003. Downloaded from 317 PAPER Incidence, risk, and case fatality of first ever stroke in the elderly population. The Rotterdam Study M Hollander, P J Koudstaal, M L Bots, D E Grobbee, A Hofman,MMBBreteler ............................................................................................................................. J Neurol Neurosurg Psychiatry 2003;74:317–321 See end of article for authors’ affiliations ....................... Objective: To estimate the incidence, survival, and lifetime risk of stroke in the elderly population. Methods: The authors conducted a study in 7721 participants from the population based Rotterdam Correspondence to: Dr M M.B. Breteler, Study who were free from stroke at baseline (1990–1993) and were followed up for stroke until Department of 1 January 1999. Age and sex specific incidence, case fatality rates, and lifetime risks of stroke were Epidemiology and calculated. Biostatistics, Erasmus Results: Mean follow up was 6.0 years and 432 strokes occurred. The incidence rate of stroke per Medical Centre Rotterdam, 1000 person years increased with age and ranged from 1.7 (95% CI 0.4 to 6.6) in men aged 55 to PO Box 1738, 3000 DR Rotterdam, Netherlands; 59 years to 69.8 (95% CI 22.5 to 216.6) in men aged 95 years or over. Corresponding figures for [email protected] women were 1.2 (95% CI 0.3 to 4.7) and 33.1 (95% CI 17.8 to 61.6). Men and women had similar absolute lifetime risks of stroke (21% for those aged 55 years). The survival after stroke did not differ Received 17 April 2002 Accepted in revised form according to sex.