Panel Discussion FDA and Industry Collaboration on Computer Software Assurance (CSA) 20th Annual Computer And IT Systems Validation, April 23, 2019 Panelists: ➢ Khaled Moussally, Compliance Group – Global Head of QMS ➢ Jason Spiegler, Siemens Digital Industries Software – Senior Director, Strategic Initiatives & Customers, Life Sciences Practice ➢ Kurt Weber, Vericel Corporation - Director of Information Technology ➢ Harsha Chulki, ICU Medical - Head of Global IT Quality & CSV ➢ Ken Shitamoto, Gilead Sciences – Sr Director, IT Material Contributors: ➢ Cisco Vicenty, Office of Compliance, FDA (CDRH) Objectives: ➢ Create awareness to accelerate innovation ➢ Inspire action so you can begin to realize value Agenda: ➢ Quick recap on FDA CSV Team’s journey and CSA Draft Guidance ➢ Panel discussion on: o Recent recommendations to the Draft Guidance o Success Stories: Vericel, Gilead Sciences and ICU Medical o “Test More; Document Less” “Computer Software Assurance for Manufacturing, Operations, and ➢ Q&A and Open Discussion Quality System Software” Computer System Validation! What do you love about it? What do you dislike about it? What’s the background? Journey of FDA CSV Team FDA Case for Siemens – Fresenius Quality begins Executive Exchange w/ FDA: Industry team formed / 2011 - 2015 2012 recommendation development begins Q2 CSV Barrier identified 2016 Q1 Begin promoting recommendations: 2017 Zoll Lifevest + Medtronic value examples • More industry adoption • FDA “A List” status for CSA • CSA Draft Guidance Draft Guidance 2018 release targeted for 2019 • More examples developed 2019 • More firms applying recommendations (Vericel, ICU Medical, Gilead, etc) www.fda.gov CSV identified as a barrier for the FDA… For your technology investments, what are the barriers for Realizing Value? CSV!!! www.fda.gov The Industry CSV Team Company Name Company Name Baxter Healthcare Tina Koepke Johnson and Johnson Dana Guarnaccia Boston Scientific Damien McPhillips Johnson and Johnson Ron Schardong Boston Scientific Ray Murphy Lantheus Imaging Lou Poirier Compliance Group Khaled Moussally Medtronic Frankie Bill Edwards Lifesciences Penny Sangkhavichith Medtronic Michael Branch Edwards Lifesciences Andy Lee Medtronic April Francis FDA Cisco Vicenty NeuroVision Imaging Pepe Davis FDA John Murray Ortho-Clinical Diagnostics Des Chesterfield Fresenius Medical Care Bill D'Innocenzo Siemens PLM Jason Spiegler Fresenius Medical Care Curt Curtis Siemens PLM Greg Robino Fresenius Medical Care Marc Koetter Siemens PLM Thorsten Ruehl Zoll Lifevest Frank Meledandri Sr. Contributions also provided by past team members: Stacey Allen, Jason Aurich, Sean Benedik, Laura Clayton, Bill Hargrave, Joe Hens , Scott Moeller & Mark Willis Non-Product CSV is foundational … for enabling the “Digital Thread” of Smart Manufacturing www.fda.gov What are the pain points in CSV? Common CSV Pain points • Takes forever • Too much focus on document generation • Perceived Regulatory burden - doing things for the auditor • Test Scripts that often run into tens of pages • Test Script errors & costly defect management • Paper based processes • Complex, hard to use Risk-based approaches Quick recap of FDA – Industry CSV Team’s recommendations and what’s new? Key Take Aways WhyKey Now? Take Aways Med Dev lags other industries • Lack of clarity • Outdated compliance approach 80% • Perceived regulatory burden 20% • Reduces manufacturer’s Test Document capability to learn, react, & % Time Spent improve www.fda.gov Key Take Aways WhyKey Now? Take Aways Defining Risk • Clearly define “intended use”. Create a Paradigm Shift… • Focus on the “direct impact on device safety and • Streamline with value-driven, device quality”, and does it result in “patient/user patient focused approaches 80% safety risk?” See examples. • Critical thinking & risk-based 20% ➢ LMS vs Manufacturing Equipment Software agile approaches Test Document • For PMA Products, CDRH is exploring using risk • Improve manufacturer’s determination to make implementation of systems an % Time Spent capabilities with automation annually reportable change no 30-Day Notice Risk Based Assurance Strategies Assurance Evidence Capture • Take credit for work already done • Use CSV tools to automate assurance activities ➢ Leverage existing activities and trusted supplier data Note: FDA does not intend to review validation of • Use Agile test methods (e.g. unscripted testing) support tools. when appropriate • Use electronic data capture & • Mitigate risk with downstream process controls record creation vs paper • Leverage continuous data and information for documentation, screen shots, monitoring and assurance etc. www.fda.gov Acceptable record of results Assurance Approach Test Plan Test Results Record (Digital Acceptable) Unscripted Testing: • Testing of features and functions • Details regarding any • Summary description of features and functions tested Ad-hoc (with least- with no test plan failures/deviations found • Issues found and disposition burdensome • Conclusion statement documentation) • Record of who performed testing and date Unscripted Testing: • Testing of feature and function fail- • Details regarding any • Summary description of fail-modes tested Error guessing modes with no test plan failures/deviations found • Issues found and disposition • Conclusion statement • Record of who performed testing and date Unscripted Testing: • Establish high level test plan • Pass/fail for each test plan • Summary description of features and functions tested Exploratory Testing objectives for features and functions objective • Result for each test plan objective – only indication of pass/fail (no step-by-step procedure is • Details regarding any • Issues found and disposition necessary) failures/deviations found • Conclusion statement • Record of who performed testing and date Scripted Testing: • Limited Test cases (step-by-step • Pass/fail for test case identified • Summary description of features and functions tested Limited procedure) identified • Details regarding any • Result for each test case - only indication of pass/fail • Expected results for the test cases failures/deviations found and • Issues found and disposition • Identify unscripted testing applied disposition regarding fails • Conclusion statement • Independent review and approval of • Record of who performed testing and date test plan. • Signature and date of appropriate signatory authority Scripted Testing: • Test objectives • Pass/fail for test case • Detailed report of assurance activity Robust • Test cases (step-by-step procedure) • Details regarding any • Result for each test case - only indication of pass/fail • Expected results failures/deviations found and • Issues found and disposition • Independent review and approval of disposition regarding fails • Conclusion statement test cases. • Record of who performed testing and date • Signature and date of appropriate signatory authority www.fda.gov Spreadsheet to Analyze and Graph Non-conformances The manufacturer developed a spreadsheet used to analyze, and graph non-conformances stored in a controlled system. Intended use of the spreadsheet is found to have a low patient risk. • Intended Use: Analyze and graph non-conformances data stored in a controlled system • Risk Assessment: The intended use of the spreadsheet is for analyzing process quality outcomes and is identified as a high-risk function. The manufacturing process includes additional changes and inspections that assure non- conformances do not escape therefore the patient risk is low. • Tested: Spreadsheet X, Version 1.2 • Test type: Unscripted testing – exploratory testing • Goal: Ensure that analyses can be Created/Read/Updated/Deleted • When/Who: July 9, 2019, by John Smith • Testing activities: Created, updated, and deleted analyses and observed that all calculated fields were correctly updated • Conclusion: No errors observed That’s it! No more documentation required! www.fda.gov Complaint Handling Spreadsheet example Patient/Quality System Risk Examples Errors in the spreadsheet outputs • Manufacturing equipment controls will cause death, life-threatening Implementation Method • Automated product inspection Out of The Box Configured Custom harm or irreversible damage to High Limited Scripted Robust Scripted • Label management & automation Unscripted Testing High patient and / or fail the direct Testing Testing implementation of a quality • QC Laboratory test calculations /Quali Medium Ad-Hoc Testing (With Limited Scripted • Adverse event tracking Unscripted Testing system activity defined in a Record) Testing • Clinical trial results Low Ad-Hoc Testing (With Ad-Hoc Testing regulation Unscripted Testing ty System Risk Errors in the spreadsheet outputs • Product Complaint Tracking Patient Record) (With Record) will cause significant but • Product quality status management temporary harm or reversible • Product Complaints Trending Medium damage to patient and / or fail to • Spreadsheet supporting managers review Spreadsheet Description / Intended Use Patient/Quality Implementation Assurance Activity support the indirect the training assigned to either to their Risk Level Method implementation of a quality employees (the system that directly Product Recall Decision Matrix – Assessment High Custom Robust Scripted system activity defined in a implements the quality system activity is form used to decide the “recall” status of a Testing regulation. the Learning Management System product based on predefined trending criteria Errors in the spreadsheet outputs