5566316.Pdf (3.644Mb)
Total Page：16
File Type：pdf, Size：1020Kb
Can machine learning complement traditional medical device surveillance? A case study of dual-chamber implantable cardioverter–defibrillators The Harvard community has made this article openly available. Please share how this access benefits you. Your story matters Citation Ross, J. S., J. Bates, C. S. Parzynski, J. G. Akar, J. P. Curtis, N. R. Desai, J. V. Freeman, et al. 2017. “Can machine learning complement traditional medical device surveillance? A case study of dual-chamber implantable cardioverter–defibrillators.” Medical Devices (Auckland, N.Z.) 10 (1): 165-188. doi:10.2147/ MDER.S138158. http://dx.doi.org/10.2147/MDER.S138158. Published Version doi:10.2147/MDER.S138158 Citable link http://nrs.harvard.edu/urn-3:HUL.InstRepos:34375278 Terms of Use This article was downloaded from Harvard University’s DASH repository, and is made available under the terms and conditions applicable to Other Posted Material, as set forth at http:// nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of- use#LAA Journal name: Medical Devices: Evidence and Research Article Designation: ORIGINAL RESEARCH Year: 2017 Volume: 10 Medical Devices: Evidence and Research Dovepress Running head verso: Ross et al Running head recto: Machine learning and medical device surveillance open access to scientific and medical research DOI: http://dx.doi.org/10.2147/MDER.S138158 Open Access Full Text Article ORIGINAL RESEARCH Can machine learning complement traditional medical device surveillance? A case study of dual- chamber implantable cardioverter–defibrillators Joseph S Ross1–4, Jonathan Background: Machine learning methods may complement traditional analytic methods for Bates4, Craig S Parzynski4, medical device surveillance. 4,5 Joseph G Akar , Jeptha P Methods and results: Using data from the National Cardiovascular Data Registry for implant- 4,5 4,5 Curtis , Nihar R Desai , able cardioverter–defibrillators (ICDs) linked to Medicare administrative claims for longitudinal 4,5 James V Freeman , Ginger M follow-up, we applied three statistical approaches to safety-signal detection for commonly used 4 6 Gamble , Richard Kuntz , Shu- dual-chamber ICDs that used two propensity score (PS) models: one specified by subject-matter Xia Li4, Danica Marinac-Dabic7, experts (PS-SME), and the other one by machine learning-based selection (PS-ML). The first Frederick A Masoudi8, Sharon- approach used PS-SME and cumulative incidence (time-to-event), the second approach used Lise T Normand9,10, Isuru Ranasinghe11, Richard E Shaw12, PS-SME and cumulative risk (Data Extraction and Longitudinal Trend Analysis [DELTA]), and the third approach used PS-ML and cumulative risk (embedded feature selection). Safety-signal Harlan M Krumholz2–5 surveillance was conducted for eleven dual-chamber ICD models implanted at least 2,000 times 1Section of General Medicine, Department of Medicine, 2Robert Wood over 3 years. Between 2006 and 2010, there were 71,948 Medicare fee-for-service beneficiaries Johnson Foundation Clinical Scholars who received dual-chamber ICDs. Cumulative device-specific unadjusted 3-year event rates Program, Yale School of Medicine, 3Department of Health Policy and varied for three surveyed safety signals: death from any cause, 12.8%–20.9%; nonfatal ICD- Management, Yale School of Public related adverse events, 19.3%–26.3%; and death from any cause or nonfatal ICD-related adverse Health, 4Center for Outcomes Research and Evaluation, Yale–New Haven event, 27.1%–37.6%. Agreement among safety signals detected/not detected between the time- Hospital, 5Section of Cardiovascular to-event and DELTA approaches was 90.9% (360 of 396, k=0.068), between the time-to-event Medicine, Department of Medicine, Yale School of Medicine, New Haven, and embedded feature-selection approaches was 91.7% (363 of 396, k=–0.028), and between 6 CT, Medtronic Inc, Minneapolis, MN, the DELTA and embedded feature selection approaches was 88.1% (349 of 396, k=–0.042). 7Division of Epidemiology, Center for Devices and Radiological Health, US Conclusion: Three statistical approaches, including one machine learning method, identified Food and Drug Administration, Silver important safety signals, but without exact agreement. Ensemble methods may be needed to Spring, MD, 8Division of Cardiology, Department of Medicine, University of detect all safety signals for further evaluation during medical device surveillance. 9 Colorado, Aurora, CO, Department of Keywords: implanted cardioverter–defibrillator, methodology, surveillance Health Care Policy, Harvard Medical School, 10Department of Biostatistics, Harvard TH Chan School of Public Health, Boston, MA, USA; 11Discipline Introduction of Medicine, University of Adelaide, Adelaide, SA, Australia; 12Department of Implantable cardioverter–defibrillators (ICDs) may be indicated for survivors of Clinical Informatics, California Pacific cardiac arrest or those who have experienced prior sustained ventricular arrhyth- Medical Center, San Francisco, CA, USA mias, and as primary prophylaxis for patients with heart failure and other condi- tions according to current guidelines.1–4 While ICD therapy has been associated with lower all-cause mortality,5 implantation also involves risk. Approximately 1% Correspondence: Joseph S Ross of ICD implantation procedures are associated with in-hospital death, and at least Section of General Internal Medicine, Yale 5% of patients experience other complications, including device malfunctions, University School of Medicine, PO Box 5–7 208093, New Haven, CT 0520, USA lead problems, and infections. Data from the American College of Cardiology’s Tel +1 203 785 2987 National Cardiovascular Data Registry (NCDR) demonstrated that 1% of patients Fax +1 203 737 3306 Email [email protected] experience pneumothorax or hematoma formation within 30 days, and 3% have a site submit your manuscript | www.dovepress.com Medical Devices: Evidence and Research 2017:10 165–188 165 Dovepress © 2017 Ross et al. This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms. php and incorporate the Creative Commons Attribution – Non Commercial (unported, v3.0) License (http://creativecommons.org/licenses/by-nc/3.0/). By accessing the work http://dx.doi.org/10.2147/MDER.S138158 you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms (https://www.dovepress.com/terms.php). Ross et al Dovepress infection or require ICD revision because of a mechanical prespecify “clinically important” patient characteristics for complication within 90 days.6 risk standardization, raising the question of whether novel However, for nearly all high-risk medical devices, includ- machine learning (ML) analytic methods might complement ing ICDs, long-term device safety, product performance, current medical device surveillance efforts by more fully and effectiveness in improving patient outcomes are not leveraging additional detailed patient, operator, and hospital well defined.8 Current medical device surveillance efforts to information collected within registry data and employing address these issues include mandatory reporting of certain data-driven, rather than expert-based, approaches to variable device-related adverse events and miscellaneous product selection for risk standardization.21 While ML algorithms are problems to the US Food and Drug Administration (FDA) typically applied to curated data sets where ground truth is Manufacturer and User Facility Device Experience program, known, in the context of real-world surveillance, the “truth” the hospital-based Medical Product Safety Network, and of medical device safety must be empirically investigated. required postmarket surveillance studies. Although each To understand potential advantages and disadvantages of has well-described limitations,9–12 data from these sources applying such methods to medical device surveillance better, have contributed unique insights that have helped to inform we used data from the NCDR-ICD registry in conjunction patient and physician decision-making and improve quality with Medicare administrative claims to conduct quarterly of care. In particular, required studies and registries offer an analyses of commonly used dual-chamber ICDs, surveying opportunity to collect detailed information about patients, over 3 years of follow-up for important safety signals: death procedures, and devices not routinely collected by electronic from any cause, nonfatal ICD-related adverse events, and a health records or administrative claims data. composite of the two. We applied three statistical approaches Importantly, the methods used to analyze data collected as to safety-signal detection: time-to-event, DELTA, and embed- part of medical device surveillance efforts have implications ded feature selection, each of which can be applied to reg- for the success of these efforts, and there are limitations to istry data to evaluate multiple device types, outcomes, and the traditional approaches used to analyze registry data. First, evaluation periods, and compared when the three approaches evaluations usually focus on overall product-class safety and identified safety signals. effectiveness, rather than examining differences across manu- facturers. Second, many analyses are cross-sectional and do Materials