Evaluations and Studies of New Drug Review Programs Under PDUFA IV for the FDA Contract No. HHSF223201010017B Task No. 2 Assessment of the Impact of the Electronic Submission and Review Environment on the Efficiency and Effectiveness of the Review of Human Drugs – Final Report September 9, 2011 Booz | Allen | Hamilton Assessment of the Impact of the Electronic Submission and Review Environment Final Report Table of Contents EXECUTIVE SUMMARY .............................................................................................................. 1 Assessment Overview .......................................................................................................... 1 Degree of Electronic Implementation ................................................................................... 1 Baseline Review Performance ............................................................................................. 2 Exchange and Content Data Standards ............................................................................... 4 Review Tools and Training Analysis .................................................................................... 5 Current Organization and Responsibilities ........................................................................... 6 Progress against Ideal Electronic Review Environment ....................................................... 6 Recommendations ............................................................................................................... 8 ASSESSMENT OVERVIEW ......................................................................................................... 9 Objectives and Scope .......................................................................................................... 9 Assessment Cohorts ............................................................................................................ 9 Methodology ....................................................................................................................... 11 FINDINGS ................................................................................................................................... 12 Degree of Electronic Implementation ................................................................................. 12 Baseline Review Performance ........................................................................................... 21 Exchange and Content Data Standards ............................................................................. 37 Review Tools and Training Analysis .................................................................................. 43 Current Organization and Responsibilities ......................................................................... 50 PROGRESS AGAINST IDEAL ELECTRONIC REVIEW ENVIRONMENT ................................ 53 RECOMMENDATIONS .............................................................................................................. 56 APPENDIX A: ADDITIONAL ANALYSES ................................................................................. 61 APPENDIX B: METHODOLOGY ............................................................................................... 90 APPENDIX C: HYPOTHESES ................................................................................................... 94 APPENDIX D: REVIEW DIVISION ACRONYMS ..................................................................... 101 APPENDIX E: EXCHANGE AND CONTENT DATA STANDARDS ........................................ 102 APPENDIX F: DETAILED ELECTRONIC REVIEW TOOL AND TRAINING ANALYSIS ....... 140 APPENDIX G: SURVEY ........................................................................................................... 172 APPENDIX H: PDUFA IV ANALYSIS AND DEEP DIVE COHORT APPLICATIONS ............. 183 i Assessment of the Impact of the Electronic Submission and Review Environment Final Report Exhibits Exhibit 1: Overview of Study Cohorts ......................................................................................... 10 Exhibit 2: Data Gathering and Analysis Methodology ................................................................. 11 Exhibit 3: Format of NDAs by Fiscal Year ................................................................................... 13 Exhibit 4: Format of BLAs by Fiscal Year ................................................................................... 14 Exhibit 5: Format of NDA Submission by Review Division ......................................................... 15 Exhibit 6: Format of BLA Submissions by Review Division ........................................................ 16 Exhibit 7: Format of NDAs/BLAs from Large Applicants ............................................................. 17 Exhibit 8: Format of NDAs/BLAs from Medium Applicants ......................................................... 19 Exhibit 9: Format of NDAs/BLAs from Small Applicants ............................................................. 20 Exhibit 10: NDA and BLA Approval Rates .................................................................................. 22 Exhibit 11: Priority NDA Approval Rate by Fiscal Year, by Submission Format ......................... 23 Exhibit 12: Standard NDA Approval Rate by Fiscal Year, by Submission Format ...................... 24 Exhibit 13: Priority and Standard BLA Approval Rate by Submission Format, by Fiscal Year ... 25 Exhibit 14: First Cycle Action for NDAs by Submission Format .................................................. 26 Exhibit 15: First Cycle Action for BLAs by Submission Format ................................................... 27 Exhibit 16: Average Time to Approval for Priority NDAs, by Fiscal Year and Submission Format ...................................................................................................................... 28 Exhibit 17: Average Time to Approval for Standard NDAs, by Fiscal Year and Submission Format ...................................................................................................................... 29 Exhibit 18: Average Time to Approval for CDER BLAs, by Fiscal Year and Submission Format ...................................................................................................................... 30 Exhibit 19: Average Time to Approval for CBER BLAs, by Fiscal Year and Submission Format ...................................................................................................................... 31 Exhibit 20: Average Time to First Action for NDAs by Submission Format................................. 32 Exhibit 21: Average Time to First Action for BLAs by Submission Format, by Center ................ 33 Exhibit 22: Responses to Survey Question: Does a fully electronic application improve your review experience? .................................................................................................. 34 Exhibit 23: Printing Habits of Review Staff .................................................................................. 35 Exhibit 24: Characteristics of Review Staff that Print Full Sections and/or Applications of Electronic Applications ............................................................................................. 36 Exhibit 25: Suggestions for Improving Applicants’ Electronic Submissions ................................ 37 Exhibit 26: Exchange and Content Data Standards Role in NDA/BLA Review Process Lifecycle ................................................................................................................... 39 Exhibit 27: Adoption of eCTD Format and Gateway for Electronic NDAs ................................... 40 Exhibit 28: Adoption of SDTM for NDAs ..................................................................................... 41 Exhibit 29: Overall Approval Rates and Time to Approval for SDTM and Non-SDTM Applications .............................................................................................................. 42 Exhibit 30: Exchange and Content Data Standard Experience by Preference for Applications with Standardized Data ............................................................................................ 43 Exhibit 31: Review Tools Associated with the Application Review Lifecycle .............................. 44 Exhibit 32: Perceived Accessibility of Process Tools .................................................................. 46 Exhibit 33: Survey Respondents Tool Usage, by Discipline ....................................................... 47 Exhibit 34: Reviewers’ Perceived Benefits and Proficiency of Most Used Analysis Tools .......... 48 Exhibit 35: Training Course Attendance and Satisfaction with Course Format .......................... 49 Exhibit 36: Most Preferred Training Formats .............................................................................. 50 ii Assessment of the Impact of the Electronic Submission and Review Environment Final Report Exhibit 37: Responsibilities of CBER and CDER Groups Involved with Electronic Submissions ............................................................................................................. 51 Exhibit 38: Level of Progress towards Potential Ideal Future State ............................................ 55 Exhibit 39: Format of NDA Submissions by Review Division ...................................................... 62 Exhibit 40: Format of BLA Submissions by Review Division .....................................................