Updated DEL2.2: Good Practices for Health Applications of Machine
Total Page:16
File Type:pdf, Size:1020Kb
INTERNATIONAL TELECOMMUNICATION UNION FG-AI4H-K-039 TELECOMMUNICATION STANDARDIZATION SECTOR ITU-T Focus Group on AI for Health STUDY PERIOD 2017-2020 Original: English WG(s): DAISAM E-meeting, 27-29 January 2021 DOCUMENT Source: Editors Title: Updated DEL2.2: Good practices for health applications of machine learning: Considerations for manufacturers and regulators Purpose: Discussion Contact: Luis Oala Email: [email protected] HHI Fraunhofer Germany Contact: Christian Johner Email: [email protected] Johner Institute for Healthcare IT Germany Contact: Pradeep Balachandran Email: [email protected] Technical Consultant (Digital Health) India Abstract: This document contains the latest draft of the FG-AI4H deliverable DEL02.2 "Good practices for health applications of machine learning: Considerations for manufacturers and regulators". This deliverable defines a set of guidelines intended to serve the AI solution developers/manufacturers on how to do conduct a comprehensive requirements analysis and to streamline the conformity assessment procedures to ensure regulatory compliance for the AI based Medical Devices (AI/ML-MD). International Telecommunication Union ITU-T FG-AI4H Deliverable TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (draft 27-01-2021) DEL2.2 Good practices for health applications of machine learning: Considerations for manufacturers and regulators - i - FG-AI4H-K-039 Summary This technical paper recommends a set of good machine learning practice guidelines to the manufacturers and regulators of data driven Artificial Intelligence based healthcare solutions on conducting comprehensive requirements analysis and streamlining conformity assessment procedures for continual product improvement in an iterative and adaptive manner. This set of good machine learning practice guidelines gives prime priority to the factor of patient safety and focuses on a streamlined process for risk minimization and quality assurance for AI/ML based health solutions and tries to establish a system of transparency and accountability of all the processes involved in AI/ML based health solutions. The proposed set of good machine learning practices adopts, extends and leverages the best practices and recommendations provided by internationally recognized medical device regulatory agencies such as the IMDRF and the FDA. These guidelines are devoid any legally binding or statutory requirements applicable to any specific regulatory framework or specific geographic jurisdiction. Keywords Regulatory Checklist, Software-as-a-Medical Device, AI/ML based Medical Devices Change Log This document contains latest version of the Deliverable DEL2.2 on "Good practices for health applications of machine learning: Considerations for manufacturers and regulators"[approved at the ITU-T Focus Group on AI for Health (FG-AI4H) meeting held in(draft 27-01-2021)]. Editors: LuisOala Email: [email protected] HHI Fraunhofer, Germany Christian Johner Email: [email protected] Johner Institute for Healthcare IT, Germany Pradeep Balachandran Email: [email protected] Technical Consultant (Digital Health), India Contributors: (in alphabetical order) Aaron .Y. Lee Email:[email protected] University of Washington Alixandro Werneck Leite Machine Learning Email: [email protected] Laboratory of Finance and Organizations, University of Brasilia, Brazil Andrew Murchison Email: [email protected] Company Oxford University Hospitals NHS Foundation Trust, United Kingdom AnleLin,Health Sciences Authority, Email: [email protected] Singapore Christoph Molnar Email: [email protected] Expert for Interpretable Machine Learning Technische Universität München Juliet Rumball-Smith Email: [email protected] New Zealand Ministry of Health DEL2.2(draft 27-01-2021) i - ii - FG-AI4H-K-039 Pat Baird Email: [email protected] Philips, U S A Peter. G. Goldschmidt Email: [email protected] World Development Group Inc., USA Pierre QuartaroloDanish Medicines Agency Email: [email protected] Shan Xu Email: [email protected] China Academy of Information and Communications Technology (CAICT) China Sven Piechottka Email: [email protected] Merantix Healthcare Germany Zack Hornberger Email: [email protected] Medical Imaging & Technology Alliance DEL2.2(draft 27-01-2021) ii - iii - FG-AI4H-K-039 CONTENTS Page 1 Scope ......................................................................................................................................... 2 2 References ................................................................................................................................. 4 3 Terms and definitions ............................................................................................................... 5 3.1 Terms defined elsewhere ............................................................................................ 5 3.2 Terms defined in this document ................................................................................. 6 4 Abbreviations and acronyms..................................................................................................... 6 5 Conventions .............................................................................................................................. 6 6 General requirements ................................................................................................................ 7 6.1 Process requirements .................................................................................................. 7 6.2 Competency requirements .......................................................................................... 8 7 Pre-market requirements ......................................................................................................... 10 7.1 Intended use and stakeholder requirements .............................................................. 10 7.1.1 Intended medical purpose ............................................................................. 10 7.1.2 Intended users and context of use ................................................................. 14 7.1.3 Stakeholder requirements ............................................................................. 16 7.1.4 Risk management and clinical evaluation .................................................... 19 7.2 Product and software requirements........................................................................... 22 7.2.1 Functionality and performance ..................................................................... 22 7.2.2 User interface................................................................................................ 28 7.2.3 Additional software aspects .......................................................................... 29 7.2.4 Risk management ......................................................................................... 30 7.3 Data management requirements................................................................................ 37 7.3.1 Data collection .............................................................................................. 37 7.3.2 Data annotation ............................................................................................. 43 7.3.3 Data pre-processing ...................................................................................... 45 7.3.4 Documentation and version control.............................................................. 47 7.4 Model development requirements ............................................................................ 50 7.4.1 Model preparation ........................................................................................ 50 7.4.2 Model training .............................................................................................. 51 7.4.3 Model evaluation .......................................................................................... 52 7.4.4 Model documentation ................................................................................... 55 7.5 Product development requirements .......................................................................... 58 7.5.1 Software development .................................................................................. 58 7.5.2 Risk Management ......................................................................................... 61 7.5.3 Accompanying materials .............................................................................. 64 DEL2.2(draft 27-01-2021) iii - iv - FG-AI4H-K-039 Page 7.6 Product validation requirements ............................................................................... 67 7.6.1 Usability validation ...................................................................................... 67 7.6.2 Clinical evaluation ........................................................................................ 71 7.7 Product release requirements .................................................................................... 72 8 Post-market requirements ....................................................................................................... 73 8.1 Production, distribution & installation requirements ................................................ 73 8.2 Post-market surveillance requirements ..................................................................... 75 8.3 Decommissioning requirements ..............................................................................