UNIVERSITY OF AMSTERDAM

MASTERS THESIS

Impact of Medication Grouping on Fall
Risk Prediction in Elders: A Retrospective
Analysis of MIMIC-III Critical Care
Database

Student:

Noman Dormosh Student No. 11412682

SRP Mentor:

Dr. Martijn C. Schut
–

SRP Tutor:

Prof. dr. Ameen Abu-Hanna

SRP Address:

Amsterdam University Medical Center - Location AMC Department Medical Informatics Meibergdreef 9, 1105 AZ Amsterdam

Practice teaching period: November 2018 - June 2019

A thesis submitted in fulfillment of the requirements for the degree of Master of Medical Informatics

iii

Abstract

Background: Falls are the leading cause of injury in elderly patients. Risk factors for falls including among others history of falls, old age, and female gender. Research studies have also linked certain medications with an increased risk of fall in what is called fall-risk-increasing drugs (FRIDs), such as psychotropics and cardiovascular drugs. However, there is a lack of consistency in the definitions of FRIDs between the studies and many studies did not use any systematic classification for medications.

Objective: The aim of this study was to investigate the effect of grouping medications at different levels of granularity of a medication classification system on the performance of fall risk prediction models.

Methods: This is a retrospective analysis of the MIMIC-III cohort database. We created seven prediction models including demographic, comorbidity and medication variables. Medications were grouped using the anatomical therapeutic chemical classification system (ATC) starting from the most specific scope of medications and moving up to the more generic groups: one model used individual medications (ATC level 5), four models used medication grouping at levels one, two, three and four of the ATC and one model did not include medications. We compared the predictive performance of these models with the performance of a model with a predefined ATC groups of FRIDs based on the literature and expert review (Expert-Opinion). Medications and comorbidities were extracted from the discharge summaries. Logistic regression with least absolute shrinkage and selection operator (LASSO) was used to construct the prediction models. The main performance measure was the area under the receiver operating characteristic curve (AUC-ROC). Additionally, we systematically evaluated the performance by including area under the precision-recall curve (AUC- PR), Akaike Information Criterion (AIC), sensitivity, specificity, precision and the Brier score. Calibration was assessed using calibration plots. In order to internally validate the results, we performed bootstrapping to obtain optimism-corrected estimates and Wilcoxon signedrank test to test the significance of the median differences of estimates between all the models and the expert-opinion FRIDs model.

Results: The highest statistically significant AUC-ROC was achieved by grouping medications at level two of the ATC [0.7143 (CI 95%: 0.710-0.718) compared to 0.698 (CI 95%: 0.6927-0.7018) of the Expert-Opinion model]. The AUC-PR of all the models with medications significantly outperformed the Expert-Opinion model except the ATC-1 model. The values of the other performance measures were varied between the models depending on the grouping level. All the models showed satisfactory calibration.

Conclusion: The performance of fall risk prediction model is significantly affected by medication grouping. The discrimination performance is enhanced by grouping the medications using the ATC classification system instead of using a FRIDs list. The optimal grouping level can be determined through experimentation.

v

Samenvatting

Achtergrond: Vallen bij ouderen is een belangrijkste oorzaak van letsel. Risicofactoren voor vallen, waaronder onder meer zijn geschiedenis van vallen, ouderdom en vrouwelijk geslacht. Onderzoeksstudies hebben ook bepaalde medicijnen in verband gebracht met een verhoogd valrisico bij zogenaamde valrisicoverhogende geneesmiddelen (FRIDs), zoals psychotropen en cardiovasculaire geneesmiddelen. Er is echter een gebrek aan consistentie in de definities van FRIDs tussen de studies en veel studies hebben geen systematische classificatie voor medicijnen gebruikt.

Doel: Het doel van deze studie was om het effect te onderzoeken van het groeperen van medicijnen op verschillende niveaus van granulariteit van een medicatieclassificatiesysteem op de prestaties van val-risicovoorspellingsmodellen

Methoden: Dit is een retrospectieve analyse van de MIMIC-III cohortdatabase. We hebben zeven voorspellingsmodellen gemaakt, waaronder demografische, comorbiditeits- en medicatievariabelen. Medicijnen werden gegroepeerd met behulp van het anatomische therapeutische chemische classificatiesysteem (ATC) uitgaande van de meest specifieke reikwijdte van medicijnen en oplopend naar de meer generieke groepen: één model gebruikte individuele medicijnen (ATC niveau 5), vier modellen gebruikten medicatiegroepering op niveau één, twee, drie en vier van de ATC en één model zonder medicijnen. We hebben de voorspellende prestaties van deze modellen vergeleken met de prestaties van een model met vooraf gedefinieerde ATC groepen van FRIDs op basis van de literatuur en expert review (Expert-Opinion). Medicijnen en comorbiditeiten werden uit ontslagsamenvattingen gehaald. Logistieke regressie met minst absolute krimp en selectie-operator (LASSO) werd gebruikt om de voorspellingsmodellen te construeren. De belangrijkste prestatiemaatstaf was “area under the receiver operating characteristic curve” (AUC-ROC). Bovendien hebben we de prestaties systematisch geëvalueerd door Akaike Information Criterion (AIC) op te nemen, area under precision recall (AUC-PR), sensitiviteit, specificiteit, precisie en de Brier-score. Om de resultaten intern te valideren, hebben we bootstrapping uitgevoerd om optimisme corrigeerde schattingen te verkrijgen en Wilcoxon signed-rank test om de significantie van de mediaan te testen op verschillen in schattingen tussen alle modellen en het (Expert-Opinion) model.

Resultaten: De hoogste statistisch significante AUC-ROC werd bereikt door medicatie te groeperen op niveau twee van de ATC [0.7143 (CI 95 %: 0.710-0.718) vergeleken met 0.698 (CI 95 %: 0.6927-0.7018) van het Expert-Opinion model]. De AUC-PR van alle modellen met medicijnen overtrof aanzienlijk het Expert-Opinion-model behalve het ATC-1 model. De waarden van de andere prestatiemetingen varieerden tussen de modellen, afhankelijk van het groeperingsniveau. Alle modellen vertoonden een bevredigende kalibratie.

Conclusie: De prestaties van val-risicovoorspelling worden aanzienlijk beïnvloed door medicatiegroepering. De discriminatieprestaties van de val voorspellen is verbeterd door de hele medicatie te groeperen met behulp van het ATC classificatiesysteem in plaats van de FRIDs lijst. Het optimale groeperingsniveau kan worden bepaald door middel van experimenten.

vii

Acknowledgements

I would like first of all to thank my wife Heba. She has been extremely supportive of me throughout my life and has made innumerable sacrifices to help me get to this point. Thanks to my children, Boshra, Mohammad, and the triplets Omar, Huda and Hamza, who have continually provided the requisite motivation to finish my degree. My parents who taught me to pursue perfection instead of excellence, deserve special thanks. Without such a team behind me, I doubt that I would be in this place today.

I would like to thank my tutor Prof. Dr. Ameen Abu Hanna and the daily mentor, Dr. Martijn Schut, for their prompt guidance and support through the SRP-process. I am also grateful to Prof. Dr. Nathalie van der Velde who provided valuable inputs and feedback to the SRP. Special thanks go to Dr. Ronald Cornet who granted me the permission to access the data used in this thesis, the MIMIC database.

Last but not least, thanks to you reader. You have read at least one page of my thesis. Thank You.. . ix

Contents

• Abstract
• iii

• v
• Abstract (Dutch)

• Acknowledgements
• vii

• 1
• Introduction
• 1

23
1.1 Research Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Outline of this Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

• 2
• Background
• 5

5678
2.1 Fall-Risk-Increasing Drugs (FRIDs) . . . . . . . . . . . . . . . . . . . . . 2.2 Medical Information Mart for Intensive Care III (MIMIC-III) database . 2.3 Anatomical Therapeutic Chemical Classification System (ATC) . . . . 2.4 Least Absolute Shrinkage and Selection Operator (LASSO) . . . . . . .

• 3
• Materials and Methods
• 11

3.1 Study Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 3.2 Population . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 3.3 Patient and Event Inclusion . . . . . . . . . . . . . . . . . . . . . . . . . 12 3.4 Data Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 3.5 Data Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 3.6 Clinical Outcome . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.7 Outcome Measurements: Predictive Performance . . . . . . . . . . . . 15 3.8 Statistical Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.9 Models Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

45

• Results
• 17

4.1 Study Population . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 4.2 Extracted Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 4.3 Missing Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 4.4 Measures of Predictive Performance . . . . . . . . . . . . . . . . . . . . 19

• Discussion and Future Direction
• 25

• 29
• Appendix

x

A Common Abbreviations Used in Medication Extraction B Average Predictive Comparison (APC) Bibliography
31 33 49

1

Chapter 1

Introduction

About one-third of community-dwelling elderly people aged 65 years and more fall at least once per year [1]. Falls are the major cause of injury among elders and result in high healthcare demand. Age-related problems, including falls, are expected to increase in the coming decades due to aging population. As a consequence, falls are expected to become one of the major public health problems worldwide [2–4]. There are several risk factors for falls including among others history of falls, old age, female gender and mobility problems [5–8]. Research studies have also linked certain medications with an increased risk of fall among older adults in what is called fallrisk-increasing drugs (FRIDs), which include psychotropics such as antidepressants, sedative hypnotics, cardiovascular drugs such as diuretics and antihypertensives and a miscellaneous group including analgesics [9–11].

Although numerous published studies have sought to establish an association between medication use and risk of fall, substantial number of these studies have not used any systematic classification of medications [9, 12]. Moreover, many studies lacked the consistency in the way medications have been classified. For instance, the class ‘antihypertensives’ sometimes includes diuretics and sometimes does not [76].

Several attempts have been made to establish a list of FRIDs that can serve as a tool in fall prevention programs [13, 14] or to improve medication safety in older adults [15– 17]. However, much uncertainty still exists about which medication is supported by evidence to be included and how should these medications be classified accordingly. In spite of this, the lack of the consistency in the definition of FRIDs hinders the effort of the researchers to compare results between different studies. To better illustrate this, consider the various prediction models created to estimate the future risk of fall [18–21]. Because FRIDs were inconsistently defined in these studies, it remains difficult to compare the results and it is uncertain whether a different definition of FRIDs could have affected the obtained results. In fact, a recent systematic review and meta-analysis [22] found that pharmacological subgroups (e.g. loop diuretics) could lead to increased risk of fall whereas the broader pharmacological group (e.g. diuretics) did not.
2

Chapter 1. Introduction

To increase the quality, consistency, comparability and clinical implementation, medications are grouped according to their therapeutic effects or using the Anatomical Therapeutic Chemical (ATC) classification system [23]. The ATC classification system, recommended by the World Health Organization (WHO), is the most widely recognized classification system for drugs. This classification system has five levels; the first level allows active substances to be classified into fourteen main groups according to the organ or system which they act. The first level is divided into pharmacological/therapeutic subgroups (2nd level), chemical/pharmacological/therapeutic (3rd and 4th) and the 5th level is the chemical substance. For example, captopril is coded C09AA01 (C=cardiovascular system, C09=agents acting on the renin-angiotensin system, C09A=ACE inhibitors, C09AA=ACE inhibitors, C09AA01=captopril).

Because medications represent one of the most modifiable risk factors for falls, there is an opportunity to address medication related issues especially in the prevention programs. In fact, previous studies in community settings have shown that reducing FRIDs can result in a 39–66% reduction in falls [13, 16, 24, 25]. This thesis seeks to address the problem of FRIDs classification inconsistency with the focus on fall risk prediction models. We hypothesize that we can obtain better discrimination performance results (between fallers and non-fallers) by the use of the entire medications instead of a predetermined list of FRIDs. We exploit the hierarchical connection between medications and their groups using the ATC to create a fall risk prediction model. Because subgroups and individual medications may have different fall-riskincreasing properties, we explore the effect of grouping the entire medications at varying levels of the ATC.

The aim of this study is to compare the performance of fall risk prediction by considering expert-based FRIDs grouping, individual medications, and by clustering the entire medications at varying ATC levels.

1.1 Research Questions

The research question addressed in this thesis is the following:
1. To what extent is grouping medications affects the performance of fall risk prediction model ?

Performance assessment is evaluated systematically using various performance measures. These performance measures are: the area under the receiver operating characteristic curve (AUC-ROC), the area under the precision-recall curve (AUC-PR), sensitivity, specificity, precision (positive predictive value), Akaike Information Criterion (AIC), calibration plots and the Brier score.

1.2. Outline of this Thesis

3

1.2 Outline of this Thesis

The next chapter provides all the necessary background for the reader of this thesis: information on fall-risk-increasing drugs (FRIDs), an overview of the Medical Information Mart for Intensive Care III (MIMIC-III) database that we used in our experiment, a description of the Anatomical Therapeutic Chemical (ATC) classification system and a brief overview of the Least Absolute Shrinkage and Selection Operator (LASSO). Chapter 3 presents the methodology of the experiment and chapter 4 shows the results. Detailed discussion, future recommendations and the conclusion are provided in chapter 5. In order to facilitate the reading, long tables were placed in an appendix at the end of the thesis.
5

Chapter 2

Background

In this chapter we discuss FRIDs, the Medical Information Mart for Intensive Care III (MIMIC-III) database, the ATC classification system and the Least Absolute Shrinkage and Selection Operator (LASSO).

2.1 Fall-Risk-Increasing Drugs (FRIDs)

Many widely used drugs are associated with falls in elderly patients [26–29]. Drugs that increase fall risk are summarized under the acronym FRIDs (fall-risk-increasing drugs). This may alleviate further analysis of fall risk and also help to update recommendations.

Drugs can induce falling in different ways, including the expected and adverse effects of either single drugs or drug–drug interactions [30] and dosage and change of medications [31]. Both pharmacokinetics and pharmacodynamics properties of drugs change with aging [28]. As age increases the half-life of drugs increases due to the increase of body fat mass. In fragile and malnourished elders, serum albumin is often reduced, hence increasing the free drugs concentration. Some of the intended and unintended pharmacotherapeutic effects of drugs (e.g., sedation, cognitive changes, dizziness, and orthostatic hypotension) might increase the risk of falls.

Numerous published studies have sought to establish an association between medication use and risk of falling. These medications include certain anti-hypertensives (e.g., diuretics, beta-blockers), anti-arrhythmics, anti-cholinergics, anti-histamines, sedatives and hypnotics (e.g., benzodiazepines), neuroleptics, antidepressants, narcotics, and nonsteroidal anti-inflammatory drugs (NSAIDs) [9, 10, 32–34]. However, this evidence is based primarily on observational data with few adjustment for confounders, dosage, or duration of therapy. Therefore it is unclear whether the observed increase in falls is truly related to the use of these drugs or the underlying conditions that the drugs are treating.
6

Chapter 2. Background

Much effort has been made to establish and a list of FRIDs that can be used in falls prevention programme or to improve medication safety in older adults [16, 24, 35]. However, for many drugs there is only minimal scientific evidence of their impact on falls; therefore, there is no concrete list of FRIDs and they are inconsistent. Table 2.1 shows an example of a list of FRIDs based on the literature and expert reviews, according to Swedish National Board of Health and Welfare [15] (NBHW).

• ATC* code
• Drugs/group of drugs

Increase the fall risk

• NO2A
• Opioids

N05A (NO5AN excluded) N05B
Antipsychotics (lithium excluded) Anxiolytics
N05C N06A
Hypnotics and sedatives Antidepressants

May cause or worsen orthostatism

C01D C02
Vasodilators used in cardiac diseases Antihypertensives
C03 C07
Diuretics Beta blocking agents
C08 C09 G04CA N04B
Calcium channel blockers Renin-angiotensin system inhibitors Alpha-adrenoreceptor antagonists Dopaminergic agents
N05A (NO5AN excluded) N06A
Antipsychotics (lithium excluded) Antidepressants

TABLE 2.1: Fall risk-increasing drugs (FRIDs) and drugs that may cause or worsen orthostatism (ODs) according to the list from the Swedish National Board of Health and Welfare (NBHW) *Anatomical Therapeutic Chemical classification system.

2.2 Medical Information Mart for Intensive Care III (MIMIC-
III) database

Medical Information Mart for Intensive Care (MIMIC-III) is a large database comprising information relating to patients admitted to critical care units at a large tertiary care hospital. The database was established in October 2003 as a Bioengineering Research Partnership between MIT, Philips Medical Systems, and Beth Israel Deaconess Medical Center.

The current release of the database, MIMIC-III, currently contains more than forty thousand patients who were admitted to Intensive Care Units (ICU) at Beth Israel Deaconess Medical Center (BIDMC), an academic, urban tertiary-care hospital. The data is de-identified, annotated and is made openly available to the research community.

2.3. Anatomical Therapeutic Chemical Classification System (ATC)

7
Beside patient information driven from the hospital, the MIMIC-III database contains detailed physiological and clinical charted data. One of the purposes of MIMIC- III is to develop and evaluate advanced ICU patient monitoring and decision support systems that will improve the efficiency, accuracy, and timeliness of clinical decision-making in critical care.

There are mainly two types of data in the MIMIC-III database; clinical data driven from the electronic health record (EHR) such as patients’ demographics, diagnoses, laboratory values, vital signs, etc. The second type is the free-text of various categories of clinical notes. Among other categories, it comprises discharge summaries, radiology reports and nursing notes.

MIMIC-III is an open access database available to any researchers around the world. However, researchers seeking to use the database must formally request access. The database is maintained by PhysioNet (http://physionet.org), a diverse group of computer scientists, physicists, mathematicians, biomedical researchers, clinicians, and educators around the world.