DOCSLIB.ORG

Assessment of the Impact of the Electronic Submission and Review Environment on the Efficiency and Effectiveness of the Review of Human Drugs – Final Report

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Assessment of the Impact of the Electronic Submission and Review Environment on the Efficiency and Effectiveness of the Review of Human Drugs – Final Report Evaluations and Studies of New Drug Review Programs Under PDUFA IV for the FDA Contract No. HHSF223201010017B Task No. 2 Assessment of the Impact of the Electronic Submission and Review Environment on the Efficiency and Effectiveness of the Review of Human Drugs – Final Report September 9, 2011 Booz | Allen | Hamilton Assessment of the Impact of the Electronic Submission and Review Environment Final Report Table of Contents EXECUTIVE SUMMARY .............................................................................................................. 1 Assessment Overview .......................................................................................................... 1 Degree of Electronic Implementation ................................................................................... 1 Baseline Review Performance ............................................................................................. 2 Exchange and Content Data Standards ............................................................................... 4 Review Tools and Training Analysis .................................................................................... 5 Current Organization and Responsibilities ........................................................................... 6 Progress against Ideal Electronic Review Environment ....................................................... 6 Recommendations ............................................................................................................... 8 ASSESSMENT OVERVIEW ......................................................................................................... 9 Objectives and Scope .......................................................................................................... 9 Assessment Cohorts ............................................................................................................ 9 Methodology ....................................................................................................................... 11 FINDINGS ................................................................................................................................... 12 Degree of Electronic Implementation ................................................................................. 12 Baseline Review Performance ........................................................................................... 21 Exchange and Content Data Standards ............................................................................. 37 Review Tools and Training Analysis .................................................................................. 43 Current Organization and Responsibilities ......................................................................... 50 PROGRESS AGAINST IDEAL ELECTRONIC REVIEW ENVIRONMENT ................................ 53 RECOMMENDATIONS .............................................................................................................. 56 APPENDIX A: ADDITIONAL ANALYSES ................................................................................. 61 APPENDIX B: METHODOLOGY ............................................................................................... 90 APPENDIX C: HYPOTHESES ................................................................................................... 94 APPENDIX D: REVIEW DIVISION ACRONYMS ..................................................................... 101 APPENDIX E: EXCHANGE AND CONTENT DATA STANDARDS ........................................ 102 APPENDIX F: DETAILED ELECTRONIC REVIEW TOOL AND TRAINING ANALYSIS ....... 140 APPENDIX G: SURVEY ........................................................................................................... 172 APPENDIX H: PDUFA IV ANALYSIS AND DEEP DIVE COHORT APPLICATIONS ............. 183 i Assessment of the Impact of the Electronic Submission and Review Environment Final Report Exhibits Exhibit 1: Overview of Study Cohorts ......................................................................................... 10 Exhibit 2: Data Gathering and Analysis Methodology ................................................................. 11 Exhibit 3: Format of NDAs by Fiscal Year ................................................................................... 13 Exhibit 4: Format of BLAs by Fiscal Year ................................................................................... 14 Exhibit 5: Format of NDA Submission by Review Division ......................................................... 15 Exhibit 6: Format of BLA Submissions by Review Division ........................................................ 16 Exhibit 7: Format of NDAs/BLAs from Large Applicants ............................................................. 17 Exhibit 8: Format of NDAs/BLAs from Medium Applicants ......................................................... 19 Exhibit 9: Format of NDAs/BLAs from Small Applicants ............................................................. 20 Exhibit 10: NDA and BLA Approval Rates .................................................................................. 22 Exhibit 11: Priority NDA Approval Rate by Fiscal Year, by Submission Format ......................... 23 Exhibit 12: Standard NDA Approval Rate by Fiscal Year, by Submission Format ...................... 24 Exhibit 13: Priority and Standard BLA Approval Rate by Submission Format, by Fiscal Year ... 25 Exhibit 14: First Cycle Action for NDAs by Submission Format .................................................. 26 Exhibit 15: First Cycle Action for BLAs by Submission Format ................................................... 27 Exhibit 16: Average Time to Approval for Priority NDAs, by Fiscal Year and Submission Format ...................................................................................................................... 28 Exhibit 17: Average Time to Approval for Standard NDAs, by Fiscal Year and Submission Format ...................................................................................................................... 29 Exhibit 18: Average Time to Approval for CDER BLAs, by Fiscal Year and Submission Format ...................................................................................................................... 30 Exhibit 19: Average Time to Approval for CBER BLAs, by Fiscal Year and Submission Format ...................................................................................................................... 31 Exhibit 20: Average Time to First Action for NDAs by Submission Format................................. 32 Exhibit 21: Average Time to First Action for BLAs by Submission Format, by Center ................ 33 Exhibit 22: Responses to Survey Question: Does a fully electronic application improve your review experience? .................................................................................................. 34 Exhibit 23: Printing Habits of Review Staff .................................................................................. 35 Exhibit 24: Characteristics of Review Staff that Print Full Sections and/or Applications of Electronic Applications ............................................................................................. 36 Exhibit 25: Suggestions for Improving Applicants’ Electronic Submissions ................................ 37 Exhibit 26: Exchange and Content Data Standards Role in NDA/BLA Review Process Lifecycle ................................................................................................................... 39 Exhibit 27: Adoption of eCTD Format and Gateway for Electronic NDAs ................................... 40 Exhibit 28: Adoption of SDTM for NDAs ..................................................................................... 41 Exhibit 29: Overall Approval Rates and Time to Approval for SDTM and Non-SDTM Applications .............................................................................................................. 42 Exhibit 30: Exchange and Content Data Standard Experience by Preference for Applications with Standardized Data ............................................................................................ 43 Exhibit 31: Review Tools Associated with the Application Review Lifecycle .............................. 44 Exhibit 32: Perceived Accessibility of Process Tools .................................................................. 46 Exhibit 33: Survey Respondents Tool Usage, by Discipline ....................................................... 47 Exhibit 34: Reviewers’ Perceived Benefits and Proficiency of Most Used Analysis Tools .......... 48 Exhibit 35: Training Course Attendance and Satisfaction with Course Format .......................... 49 Exhibit 36: Most Preferred Training Formats .............................................................................. 50 ii Assessment of the Impact of the Electronic Submission and Review Environment Final Report Exhibit 37: Responsibilities of CBER and CDER Groups Involved with Electronic Submissions ............................................................................................................. 51 Exhibit 38: Level of Progress towards Potential Ideal Future State ............................................ 55 Exhibit 39: Format of NDA Submissions by Review Division ...................................................... 62 Exhibit 40: Format of BLA Submissions by Review Division .....................................................
Load more

Recommended publications
  • An Evaluation of Statistical Software for Research and Instruction
    Behavior Research Methods, Instruments, & Computers 1985, 17(2),352-358 An evaluation of statistical software for research and instruction DARRELL L. BUTLER Ball State University, Muncie, Indiana and DOUGLAS B. EAMON University of Houston-Clear Lake, Houston, Texas A variety of microcomputer statistics packages were evaluated. The packages were compared on a number of dimensions, including error handling, documentation, statistical capability, and accuracy. Results indicated that there are some very good packages available both for instruc­ tion and for analyzing research data. In general, the microcomputer packages were easier to learn and to use than were mainframe packages. Furthermore, output of mainframe packages was found to be less accurate than output of some of the microcomputer packages. Many psychologists use statistical programs to analyze ware packages available on the VAX computers at Ball research data or to teach statistics, and many are interested State University: BMDP, SPSSx, SCSS, and MINITAB. in using computer software for these purposes (e.g., Versions of these programs are widely available and are Butler, 1984; Butler & Kring, 1984). The present paper used here as points ofreference to judge the strengths and provides some program descriptions, benchmarks, and weaknesses of the microcomputer packages. Although evaluations of a wide variety of marketed statistical pro­ there are many programs distributed by individuals (e.g., grams that may be useful in research and/or instruction. see Academic Computer Center of Gettysburg College, This review differs in several respects from other re­ 1984, and Eamon, 1983), none are included in the present cent reviews of statistical programs (e. g., Carpenter, review.
    [Show full text]
  • SQSTM1 Mutations in Familial and Sporadic Amyotrophic Lateral Sclerosis
    ORIGINAL CONTRIBUTION SQSTM1 Mutations in Familial and Sporadic Amyotrophic Lateral Sclerosis Faisal Fecto, MD; Jianhua Yan, MD, PhD; S. Pavan Vemula; Erdong Liu, MD; Yi Yang, MS; Wenjie Chen, MD; Jian Guo Zheng, MD; Yong Shi, MD, PhD; Nailah Siddique, RN, MSN; Hasan Arrat, MD; Sandra Donkervoort, MS; Senda Ajroud-Driss, MD; Robert L. Sufit, MD; Scott L. Heller, MD; Han-Xiang Deng, MD, PhD; Teepu Siddique, MD Background: The SQSTM1 gene encodes p62, a major In silico analysis of variants was performed to predict al- pathologic protein involved in neurodegeneration. terations in p62 structure and function. Objective: To examine whether SQSTM1 mutations con- Results: We identified 10 novel SQSTM1 mutations (9 tribute to familial and sporadic amyotrophic lateral scle- heterozygous missense and 1 deletion) in 15 patients (6 rosis (ALS). with familial ALS and 9 with sporadic ALS). Predictive in silico analysis classified 8 of 9 missense variants as Design: Case-control study. pathogenic. Setting: Academic research. Conclusions: Using candidate gene identification based on prior biological knowledge and the functional pre- Patients: A cohort of 546 patients with familial diction of rare variants, we identified several novel (n=340) or sporadic (n=206) ALS seen at a major aca- SQSTM1 mutations in patients with ALS. Our findings demic referral center were screened for SQSTM1 muta- provide evidence of a direct genetic role for p62 in ALS tions. pathogenesis and suggest that regulation of protein deg- radation pathways may represent an important thera- Main Outcome Measures: We evaluated the distri- peutic target in motor neuron degeneration. bution of missense, deletion, silent, and intronic vari- ants in SQSTM1 among our cohort of patients with ALS.
    [Show full text]
  • Archives of Agriculture and Environmental Science
    ISSN (Online) : 2456-6632 Archives of Agriculture and Environmental Science An International Journal Volume 4 | Issue 2 Agriculture and Environmental Science Academy www.aesacademy.org Scan to view it on the web Archives of Agriculture and Environmental Science (Abbreviation: Arch. Agr. Environ. Sci.) ISSN: 2456-6632 (Online) An International Research Journal of Agriculture and Environmental Sciences Volume 4 Number 2 2019 Abstracted/Indexed: The journal AAES is proud to be a registered member of the following leading abstracting/indexing agencies: Google Scholar, AGRIS-FAO, CrossRef, Informatics, jGate @ e-Shodh Sindhu, WorldCat Library, OpenAIRE, Zenodo ResearchShare, DataCite, Index Copernicus International, Root Indexing, Research Gate etc. All Rights Reserved © 2016-2019 Agriculture and Environmental Science Academy Disclaimer: No part of this booklet may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without written permission of the publisher. However, all the articles published in this issue are open access articles which are distributed under the terms of the Creative Commons Attribution 4.0 License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited For information regarding permission, write us [email protected]. An official publication of Agriculture and Environmental Science Academy 86, Gurubaksh Vihar (East) Kankhal Haridwar-249408 (Uttarakhand), India Website: https://www.aesacademy.org Email: [email protected] Phone: +91-98971-89197 Archives of Agriculture and Environmental Science (An International Research Journal) (Abbreviation: Arch. Agri. Environ. Sci.) Aims & Objectives: The journal is an official publication of Agriculture and Environmental Science Academy.
    [Show full text]
  • Strategies for Non-Invasive Management of High-Grade Cervical Intraepithelial Neoplasia
    Strategies for non-invasive management of high-grade cervical intraepithelial neoplasia Citation for published version (APA): Koeneman, M. M. (2019). Strategies for non-invasive management of high-grade cervical intraepithelial neoplasia: prognostic biomarkers and immunotherapy. Maastricht University. https://doi.org/10.26481/dis.20190116mk Document status and date: Published: 01/01/2019 DOI: 10.26481/dis.20190116mk Document Version: Publisher's PDF, also known as Version of record Please check the document version of this publication: • A submitted manuscript is the version of the article upon submission and before peer-review. There can be important differences between the submitted version and the official published version of record. People interested in the research are advised to contact the author for the final version of the publication, or visit the DOI to the publisher's website. • The final author version and the galley proof are versions of the publication after peer review. • The final published version features the final layout of the paper including the volume, issue and page numbers. Link to publication General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal.
    [Show full text]
  • Statistics with Free and Open-Source Software
    Free and Open-Source Software • the four essential freedoms according to the FSF: • to run the program as you wish, for any purpose • to study how the program works, and change it so it does Statistics with Free and your computing as you wish Open-Source Software • to redistribute copies so you can help your neighbor • to distribute copies of your modified versions to others • access to the source code is a precondition for this Wolfgang Viechtbauer • think of ‘free’ as in ‘free speech’, not as in ‘free beer’ Maastricht University http://www.wvbauer.com • maybe the better term is: ‘libre’ 1 2 General Purpose Statistical Software Popularity of Statistical Software • proprietary (the big ones): SPSS, SAS/JMP, • difficult to define/measure (job ads, articles, Stata, Statistica, Minitab, MATLAB, Excel, … books, blogs/posts, surveys, forum activity, …) • FOSS (a selection): R, Python (NumPy/SciPy, • maybe the most comprehensive comparison: statsmodels, pandas, …), PSPP, SOFA, Octave, http://r4stats.com/articles/popularity/ LibreOffice Calc, Julia, … • for programming languages in general: TIOBE Index, PYPL, GitHut, Language Popularity Index, RedMonk Rankings, IEEE Spectrum, … • note that users of certain software may be are heavily biased in their opinion 3 4 5 6 1 7 8 What is R? History of S and R • R is a system for data manipulation, statistical • … it began May 5, 1976 at: and numerical analysis, and graphical display • simply put: a statistical programming language • freely available under the GNU General Public License (GPL) → open-source
    [Show full text]
  • Statistical Software
    Statistical Software A. Grant Schissler1;2;∗ Hung Nguyen1;3 Tin Nguyen1;3 Juli Petereit1;4 Vincent Gardeux5 Keywords: statistical software, open source, Big Data, visualization, parallel computing, machine learning, Bayesian modeling Abstract Abstract: This article discusses selected statistical software, aiming to help readers find the right tool for their needs. We categorize software into three classes: Statisti- cal Packages, Analysis Packages with Statistical Libraries, and General Programming Languages with Statistical Libraries. Statistical and analysis packages often provide interactive, easy-to-use workflows while general programming languages are built for speed and optimization. We emphasize each software's defining characteristics and discuss trends in popularity. The concluding sections muse on the future of statistical software including the impact of Big Data and the advantages of open-source languages. This article discusses selected statistical software, aiming to help readers find the right tool for their needs (not provide an exhaustive list). Also, we acknowledge our experiences bias the discussion towards software employed in scholarly work. Throughout, we emphasize the software's capacity to analyze large, complex data sets (\Big Data"). The concluding sections muse on the future of statistical software. To aid in the discussion, we classify software into three groups: (1) Statistical Packages, (2) Analysis Packages with Statistical Libraries, and (3) General Programming Languages with Statistical Libraries. This structure
    [Show full text]
  • STAT 3304/5304 Introduction to Statistical Computing
    STAT 3304/5304 Introduction to Statistical Computing Statistical Packages Some Statistical Packages • BMDP • GLIM • HIL • JMP • LISREL • MATLAB • MINITAB 1 Some Statistical Packages • R • S-PLUS • SAS • SPSS • STATA • STATISTICA • STATXACT • . and many more 2 BMDP • BMDP is a comprehensive library of statistical routines from simple data description to advanced multivariate analysis, and is backed by extensive documentation. • Each individual BMDP sub-program is based on the most competitive algorithms available and has been rigorously field-tested. • BMDP has been known for the quality of it’s programs such as Survival Analysis, Logistic Regression, Time Series, ANOVA and many more. • The BMDP vendor was purchased by SPSS Inc. of Chicago in 1995. SPSS Inc. has stopped all develop- ment work on BMDP, choosing to incorporate some of its capabilities into other products, primarily SY- STAT, instead of providing further updates to the BMDP product. • BMDP is now developed by Statistical Solutions and the latest version (BMDP 2009) features a new mod- ern user-interface with all the statistical functionality of the classic program, running in the latest MS Win- dows environments. 3 LISREL • LISREL is software for confirmatory factor analysis and structural equation modeling. • LISREL is particularly designed to accommodate models that include latent variables, measurement errors in both dependent and independent variables, reciprocal causation, simultaneity, and interdependence. • Vendor information: Scientific Software International http://www.ssicentral.com/ 4 MATLAB • Matlab is an interactive, matrix-based language for technical computing, which allows easy implementation of statistical algorithms and numerical simulations. • Highlights of Matlab include the number of toolboxes (collections of programs to address specific sets of problems) available.
    [Show full text]
  • Software Packages – Overview There Are Many Software Packages Available for Performing Statistical Analyses. the Ones Highlig
    APPENDIX C SOFTWARE OPTIONS Software Packages – Overview There are many software packages available for performing statistical analyses. The ones highlighted here are selected because they provide comprehensive statistical design and analysis tools and a user friendly interface. SAS SAS is arguably one of the most comprehensive packages in terms of statistical analyses. However, the individual license cost is expensive. Primarily, SAS is a macro based scripting language, making it challenging for new users. However, it provides a comprehensive analysis ranging from the simple t-test to Generalized Linear mixed models. SAS also provides limited Design of Experiments capability. SAS supports factorial designs and optimal designs. However, it does not handle design constraints well. SAS also has a GUI available: SAS Enterprise, however, it has limited capability. SAS is a good program to purchase if your organization conducts advanced statistical analyses or works with very large datasets. R R, similar to SAS, provides a comprehensive analysis toolbox. R is an open source software package that has been approved for use on some DoD computer (contact AFFTC for details). Because R is open source its price is very attractive (free). However, the help guides are not user friendly. R is primarily a scripting software with notation similar to MatLab. R’s GUI: R Commander provides limited DOE capabilities. R is a good language to download if you can get it approved for use and you have analysts with a strong programming background. MatLab w/ Statistical Toolbox MatLab has a statistical toolbox that provides a wide variety of statistical analyses. The analyses range from simple hypotheses test to complex models and the ability to use likelihood maximization.
    [Show full text]
  • On Reproducible Econometric Research
    On Reproducible Econometric Research Roger Koenkera Achim Zeileisb∗ a Department of Economics, University of Illinois at Urbana-Champaign, United States of America b Department of Statistics and Mathematics, WU Wirtschaftsuniversit¨at Wien, Austria SUMMARY Recent software developments are reviewed from the vantage point of reproducible econo- metric research. We argue that the emergence of new tools, particularly in the open-source community, have greatly eased the burden of documenting and archiving both empirical and simulation work in econometrics. Some of these tools are highlighted in the discussion of two small replication exercises. Keywords: reproducibility, replication, software, literate programming. 1. INTRODUCTION The renowned dispute between Gauss and Legendre over priority for the invention of the method of least squares might have been resolved by Stigler(1981). A calculation of the earth's ellipticity reported by Gauss in 1799 alluded to the use of meine Methode; had Stigler been able to show that Gauss's estimate was consistent with the least squares solution using the four observations available to Gauss, his claim that he had been using the method since 1795 would have been strongly vindicated. Unfortunately, no such computation could be reproduced leaving the dispute in that limbo all too familiar in the computational sciences. The question that we would like to address here is this: 200 years after Gauss, can we do better? What can be done to improve our ability to reproduce computational results in econometrics and related fields? Our main contention is that recent software developments, notably in the open- source community, make it much easier to achieve and distribute reproducible research.
    [Show full text]
  • Introducing SPM® Infrastructure
    Introducing SPM® Infrastructure © 2019 Minitab, LLC. All Rights Reserved. Minitab®, SPM®, SPM Salford Predictive Modeler®, Salford Predictive Modeler®, Random Forests®, CART®, TreeNet®, MARS®, RuleLearner®, and the Minitab logo are registered trademarks of Minitab, LLC. in the United States and other countries. Additional trademarks of Minitab, LLC. can be found at www.minitab.com. All other marks referenced remain the property of their respective owners. Salford Predictive Modeler® Introducing SPM® Infrastructure Introducing SPM® Infrastructure The SPM® application is structured around major predictive analysis scenarios. In general, the workflow of the application can be described as follows. Bring data for analysis to the application. Research the data, if needed. Configure and build a predictive analytics model. Review the results of the run. Discover the model that captures valuable insight about the data. Score the model. For example, you could simulate future events. Export the model to a format other systems can consume. This could be PMML or executable code in a mainstream or specialized programming language. Document the analysis. The nature of predictive analysis methods you use and the nature of the data itself could dictate particular unique steps in the scenario. Some actions and mechanisms, though, are common. For any analysis you need to bring the data in and get some understanding of it. When reviewing the results of the modeling and preparing documentation, you can make use of special features embedded into charts and grids. While we make sure to offer a display to visualize particular results, there’s always a Summary window that brings you a standard set of results represented the same familiar way throughout the application.
    [Show full text]
  • Minitab Connecttm
    TM MinitabMinitab Connect ConnectTM Data integration, automation, and governance for comprehensive insights 1 Minitab ConnectTM Minitab ConnectTM Access, blend, and enrich data from all your critical sources, for meaningful business intelligence and confident, informed decision making. Minitab ConnectTM empowers data users from across the enterprise with self-serve tools to transform diverse data into a governed network of data pipelines, to feed analytics initiatives and foster organization-wide collaboration. Users can effortlessly blend and explore data from databases, cloud and on-premise apps, unstructured data, spreadsheets, and more. Flexible, automated workflows accelerate every step of the data integration process, while powerful data preparation and visualization tools help yield transformative insights. 1 Data integration, automation, and governance for comprehensive insights Accelerate your digital transformation Minitab ConnectTM is empowering insights-driven organizations with tools to gather and share trusted data and actionable insights that drive their company forward. Enable next-generation analysis and reporting “The things we measure today are not even close to the things we measured three years ago. We’re able to do new things and iterate into the next generation of where we need to be instead of being mired in spreadsheets and PowerPoint decks.” - Senior Manager, Digital Search Analytics Telecommunications Industry Focus on business drivers, not manual tasks “Bringing this into our environment has really freed our analytics resources from the tactical tasks, and really allowed them to think strategically.” - Marketing Technology Manager, Finance Industry Empower users with sophisticated, no-code tools “The platform is really powerful. It has an ETL function but also has a front-end face to it.
    [Show full text]
  • Software in the Scientific Literature: Problems with Seeing, Finding, And
    Software in the Scientific Literature: Problems with Seeing, Finding, and Using Software Mentioned in the Biology Literature James Howison School of Information, University of Texas at Austin, 1616 Guadalupe Street, Austin, TX 78701, USA. E-mail: [email protected] Julia Bullard School of Information, University of Texas at Austin, 1616 Guadalupe Street, Austin, TX 78701, USA. E-mail: [email protected] Software is increasingly crucial to scholarship, yet the incorporates key scientific methods; increasingly, software visibility and usefulness of software in the scientific is also key to work in humanities and the arts, indeed to work record are in question. Just as with data, the visibility of with data of all kinds (Borgman, Wallis, & Mayernik, 2012). software in publications is related to incentives to share software in reusable ways, and so promote efficient Yet, the visibility of software in the scientific record is in science. In this article, we examine software in publica- question, leading to concerns, expressed in a series of tions through content analysis of a random sample of 90 National Science Foundation (NSF)- and National Institutes biology articles. We develop a coding scheme to identify of Health–funded workshops, about the extent that scientists software “mentions” and classify them according to can understand and build upon existing scholarship (e.g., their characteristics and ability to realize the functions of citations. Overall, we find diverse and problematic Katz et al., 2014; Stewart, Almes, & Wheeler, 2010). In practices: Only between 31% and 43% of mentions particular, the questionable visibility of software is linked to involve formal citations; informal mentions are very concerns that the software underlying science is of question- common, even in high impact factor journals and across able quality.
    [Show full text]