BMJ Surg Interv Health Technologies: first published as 10.1136/bmjsit-2019-000024 on 25 March 2020. Downloaded from Open access Data source How real- world evidence can really deliver: a case study of data source development and use Joseph Drozda ,1 Angelique Zeringue,2 Benjamin Dummitt,2 Byron Yount,2 Frederic Resnic3 To cite: Drozda J, Zeringue A, PURPOSE AND GOALS OF ORIGINAL DATA evaluations of its utility appear in the demon- Dummitt B, et al. How real- COLLECTION stration report on the FDA’s website.9 world evidence can really deliver: a case study of Electronic health record (EHR) data hold data source development great promise for evaluating the quality and and use. BMJ Surg Interv efficiency of care. Realization of the poten- D ATA COLLECTED Health Technologies tial for EHRs requires that data be captured Development of the UDIR 2020;2:e000024. doi:10.1136/ without interfering with clinical workflow bmjsit-2019-000024 Mercy is an integrated delivery network in and without requiring manual data entry Additional material is Missouri, Kansas, Oklahoma and Arkansas ► into databases. Efforts to extract EHR data published online only. To view with over 40 hospitals and a total of 4396 to date have largely depended on natural please visit the journal online licensed beds. At the time of the demon- (http:// dx. doi. org/ 10. 1136/ language processing and have not scaled stration, Mercy operated five Cath Labs bmjsit- 2019- 000024). beyond the settings in which they were at its hospitals in Joplin, Missouri; Rogers, developed.1–4 Arkansas; Springfield, Missouri; St Louis, copyright. In 2012, the US Food and Drug Admin- Received 30 August 2019 Missouri; and Washington, Missouri. Mercy Revised 02 December 2019 istration (FDA) funded a demonstration manages supply chain information with the Accepted 13 January 2020 performed by Mercy Health (Mercy) whereby Infor Lawson (Infor Lawson, New York, USA) unique device identifiers (UDIs) were incor- enterprise resource processing (ERP) system porated into Mercy’s electronic information and has the same EHR in all its inpatient systems, enabling assessments of cardiovas- 5 facilities and clinician practices. In addition, cular device performance using EHR data. during the time of the demonstration, all The goals and design of the demonstra- 6–8 Mercy Cath Labs used Merge Hemo as their tion have been previously described. In hemodynamic software for documentation of summary, between May and December 2012, all procedures, including stent implantation. a barcode scanning system was installed in A point of use barcode scanning inventory http://sit.bmj.com/ 5 Mercy cardiac catheterization laboratories system (OptiFlex CL, Omnicell, Mountain (Cath Labs) for capturing data on all consum- View, California, USA) was installed for able supplies, including coronary stents. © Author(s) (or their capturing information on all Cath Lab employer(s)) 2020. Re- use Coronary stent barcodes were used as proto- consumable items (including the prototype permitted under CC BY-NC. No type UDIs and were linked at the patient level coronary stent UDIs) at the point of care.7 commercial re- use. See rights with clinical data from the hemodynamic and permissions. Published by When coronary stents are brought to the on September 24, 2021 by guest. Protected BMJ. system (Merge Hemo, Merge, Chicago, Illi- Cath Lab, their prototype UDIs (and now nois, USA) and Mercy’s EHR (EpicCare, 1Mercy Research, Mercy Health, actual UDIs) are scanned into OptiFlex CL. Chesterfield, Missouri, USA Epic, Verona, Wisconsin, USA). A database When a stent is implanted in a patient, its 2Department of Data Analytics & termed the UDI research database (UDIR) UDI is scanned and automatically uploaded Decision Intelligence Solutions, was built that contains EHR-extracted data, into Merge Hemo and OptiFlex CL. The Mercy Health, Chesterfield, UDI-associated coronary stent attributes, Missouri, USA device is linked to the patient in both Merge 3Division of Cardiovascular and data from other sources, for example, Hemo and OptiFlex CL. After case comple- Medicine, Lahey Hospital and the Social Security Death Master File. The tion, Cath Lab personnel transmit the case Medical Center, Burlington, UDIR is updated weekly with EHR data record (including UDI) from Merge Hemo Massachusetts, USA enabling longitudinal follow-up for purposes to the UDIR; OptiFlexCL data are automat- Correspondence to of device evaluation, including safety surveil- ically transmitted to the UDIR. Conflicts Dr Joseph Drozda; lance. Details of UDIR content and structure in patient identification are then recon- [email protected] have been published previously8 and initial ciled through integration between UDIR Drozda J, et al. BMJ Surg Interv Health Technologies 2020;2:e000024. doi:10.1136/bmjsit-2019-000024 1 BMJ Surg Interv Health Technologies: first published as 10.1136/bmjsit-2019-000024 on 25 March 2020. Downloaded from Open access and Mercy’s integrated patient data mart. Additionally, a preset group of baseline patient characteristics (labo- ratory values, diagnoses and so on) are automatically extracted from the EHR and uploaded to the UDIR on a weekly basis. Finally, UDI- associated stent attributes are uploaded to the UDIR from the FDA’s Global UDI Database (GUDID)10 and from a supplemental database termed the Supplemental UDI Database (SUDID). The structure of the UDIR, then, supports analyses utilizing variables in three broad categories: baseline patient char- acteristics, coronary stent attributes and patient outcomes including adverse events. Assessment of the UDIR The UDIR contains data from multiple sources: Epic- Care (Clinical and Billing), Merge Hemo, OptiFlex CL, GUDID, SUDID and the Social Security Death Master File. Additionally, two of Mercy’s Cath Labs (Springfield and St Louis) participate in the National Cardiovascular Data Registry’s (NCDR) CathPCI Registry into which data Figure 1 Patient and procedure selection. from all percutaneous coronary interventions (PCIs) are entered by personnel from the hospitals’ clinical quality departments. These data are available for comparison Our analytic plan called for the UDIR to be an active with UDIR data. surveillance dataset and we anticipated using unadjusted UDIR data completeness was assessed using compari- data to identify safety signals as we felt it would be diffi- sons of data within the database as extracted from each cult to apply propensity scoring to prospectively captured of the data sources and with data obtained from reports EHR data. When a safety signal was identified, we used submitted to the CathPCI Registry. Cath Lab personnel propensity matching methods to adjust for poten- copyright. bar code scanning compliance was also assessed because tial selection bias in the choice of the stent’s drug. An it is a major contributor to data completeness. Details of implanter might, for instance, have chosen a BMS for a data completeness and scan compliance evaluations are patient in shock because of the poor prognosis. Separate included in the online supplementary appendix B and analyses were conducted using propensity matching and online supplementary figures 1; 2. In summary, both were weighting techniques. Propensity scores were calculated found to be at an acceptable level for safety surveillance using covariates based on the adjustment models used purposes. in the Massachusetts CathPCI data (Mass-DAC registry) with presenting diagnosis based on International Classifi- cation of Diseases, Ninth Revision, Clinical Modification (ICD-9 CM) codes.11 Twelve of the clinical covariates, as DATA RESOURCE USE listed in table 1, were available in the UDIR database and http://sit.bmj.com/ Example case study were used in our final model. To demonstrate the functionality of the database and face Finally, Cox proportional hazard models were conducted validity of UDIR data, analyses were performed of ‘all to compare mortality between patients receiving BMS comers’ receiving coronary stents from 1 November 2012 versus DES. Analyses were performed using SAS V.9.3 to 26 October 2013. We tracked the major adverse cardiac and Kaplan Meier curves were created using R V.3.1. The events of death, acute myocardial infarction (AMI), total Mercy Hospital St Louis Institutional Review Board deter- on September 24, 2021 by guest. Protected coronary revascularization and stent thrombosis. Deaths mined that the demonstration was exempt from review. were captured in the Social Security Death Master File A total of 2250 patients were found undergoing PCI with and the other outcomes were found in Epic Clarity (the stent implantation. Except for those patients who died Epic data warehousing utility) using diagnosis and billing within the first 30 days, a minimum of 30 days of follow-up codes.9 We selected mortality as the outcome of interest data were available for each patient with data censored at because of its objectivity and data completeness and ‘drug’ 1 year post- PCI. The mean follow-up time was 320.9±107.0 as the stent attribute of interest. We hypothesized that days. After applying exclusions, there remained 1555 there would be no difference in mortality among stents procedures performed on 1445 patients with an average that eluted various drugs (drug eluting stent (DES))—or of 1.1 implanted stents per patient. Of these patients, 300 no drug in the case of bare- metal stents (BMS). During (20.8%) had received BMS. The remaining received DES, the time of the study, all implanted DES contained ever- consisting