View to Evaluate the Algorithm Outputs

Home , Adverse drug reaction

Detecting Adverse Drug Reactions in Electronic Health Records by using the Food and Drug Administration’s Adverse Event Reporting System

A thesis submitted to the Division of Research and Advanced Studies of the University of Cincinnati

in partial fulfillment of the requirements for the degree of Master of Science for the Major of Computer Science in the Department of Electrical Engineering and Computing Systems of the College of Engineering and Applied Science 2016 By Huaxiu Tang

Committee Members: Jaroslaw Meller, Ph.D., Chair Raj Bhatnagar, Ph.D. Yizhao Ni, Ph.D.

Abstract

The objective of this study was to detect adverse drug reactions (ADRs) in Electronic Health Records

(EHR) by using the Food and Drug Administration’s (FDA) Adverse Event Reporting System (FAERS).

The null hypothesis stated that “leveraging drug-reaction pairs from the FAERS will not improve the performance of EHR-based ADR detection”. Both structured data and unstructured clinical notes

(narrative text) were used in this study. Natural Language Processing (NLP) was used to identify medical conditions in clinical notes. First, I identified the 41 most frequently used medications by calculating the frequencies of medications in the EHR data and FDA ADR reports. Focusing on the 41 medications, I used the FDA drug-reaction pairs as candidate ADRs and searched for them in the clinical notes to identify Drug-Reaction Pair Sentences (DRPSs). Then, NLP techniques were performed on the DRPSs to identify ADR-related DRPSs. Finally, a clinical informatics expert with medical degree and I performed manual review to evaluate the algorithm outputs. A total of 2475 ADR-related DRPSs were identified, which contained 1492 unique DRPSs. Five hundred and twenty eight sentences were randomly sampled from the 1492 DRPSs for manual review. The positive predictive value on the reviewed cases was 42.4%

(224 out of 528). The performance of ADR identification leveraging FDA ADE reports (PPV = 42.4%) was significantly better than not using FDA ADE reports (PPV = 8.7%), the p-value was 3.67 × 10-36. The null hypothesis was rejected. The promising results have potential for significant impact in automated or semi-automated ADR detection, particularly as growing adoption of EHRs greatly expands the size of, and increases the complexity of, the EHR data.

Key works: adverse drug reaction; clinical notes; electronic health records; natural language processing

iii

Acknowledgement

This thesis was completed with the help of many people. Firstly, I would like to thank my advisors, Dr. Imre Solti and Dr. Jaroslaw Meller, for their great help and insightful guidance. Dr. Solti suggested the overall scope of this thesis and guided me until the thesis was finished. I appreciate the resources, mentoring and opportunity given to me by my advisers, Dr. Solti and Dr. Meller.

I would like to especially thank Dr. Yizhao Ni for the mentoring and guidance during my three years’ of study and the great effort he made for my thesis project. He also provided tremendous technical help which made this study possible.

I would like to give my special thanks to Dr. Eric Kirkendall and Lisa

Garrity, who gave me helpful clinical suggestions and comments when I hit roadblocks during the study.

I would give my special thanks to Todd G. Lingren and Dr. Qi Li, who shared their experience and resources related to this study with me.

I also want to thank Dr. Raj Bhatnagar who were kind enough to agree to serve on my thesis committee and provided many valuable suggestions.

I want to thank Ms Julie Muenchen, Ms Vicki Baker, Ms Jennifer Diallo and Ms Melissa Hogan for their administrative help. I would also thank my classmates Lirong Tan and Xiaoting Zhu, who provided help while I was not at Cincinnati.

Lastly and most importantly, I want to thank my dear husband and daughter. They were always there for me and I cannot say my thanks enough for their support and love.

Table of Contents Abstract ...... ii Acknowledgement ...... iv Table of Contents ...... vi List of Tables ...... vii List of Figures ...... viii Abbreviations ...... ix 1 Introduction ...... 1 2 Related work ...... 3 2.1 Definitions related to ADR and scope of the study ...... 3 2.2 Adverse drug reaction detection ...... 4 2.3 Natural language processing on clinical notes ...... 9 2.3.1 Medical condition identification ...... 9 2.3.2 Contextual property detection ...... 12 3 Data ...... 13 3.1 Electronic Health Records (EHR) ...... 13 3.2 FDA FAERS Reports ...... 14 4 Methods ...... 15 4.1 Most frequently used medications ...... 17 4.1.1 Data preprocessing on FDA FAERS reports ...... 18 4.1.2 Medication name mapping ...... 20 4.1.3 Most frequently used medications ...... 27 4.2 Preparation for DDSS/reaction-CUI matching ...... 29 4.2.1 Preprocessing the FDA drug-reaction pairs ...... 30 4.2.2 FDA reaction-CUI mapping ...... 31 4.2.3 Preprocessing the clinical notes ...... 32 4.2.4 NLP of the clinical notes ...... 34 4.3 Identifying ADR-related DRPSs ...... 43 4.3.1 ADR time window ...... 44 4.3.2 Identifying ADR-related DRPSs ...... 44 4.3.3 Thresholding reactions for a medication ...... 54 4.4 Identifying baseline ADR-related DRPSs ...... 57 5 Results ...... 60 5.1 Evaluation metrics ...... 60

5.2 Performance of ADR detection with FAERS reports ...... 60 5.3 Performance of ADR detection using the baseline ...... 69 6 Discussion...... 71 7 Conclusion ...... 75 8 References ...... 75 9 Appendix ...... 80

List of Tables

Table 1: EHR data ...... 13 Table 2: FDA ADE Reports ...... 14 Table 3: Frequencies and cumulative percentages for the 41 most frequently used medications ..... 28 Table 4: Criteria to skip reactions ...... 30 Table 5: Number of reactions reported by FDA for the 41 most frequently used medications ...... 31 Table 6: Types of notes used in this study ...... 33 Table 7: Complicated encounters to be filtered out ...... 33 Table 8: Excluded sections ...... 36 Table 9: Key words and captures of plan/ hypothetical /instructions/therapy ...... 38 Table 10: Patterns indicating relations between DDSSs and medications and their indication ...... 39 Table 11: ADR-related DRPSs identified for each medication ...... 45 Table 12: ADR-related DRPSs identified for each note type ...... 47 Table 13: ADR-related DRPSs identified for each medication and each note type ...... 48 Table 14: Number of ADR-related DRPSs at each cut-off percentage ...... 54 Table 15: Baseline ADR-related DRPSs identified for each medication ...... 58 Table 16: Results for each medication ...... 61 Table 17: Results for each note type ...... 62 Table 18: Results for each cut-off percentage ...... 62 Table 19: Results for each medication in each note type ...... 63 Table 20: Number of medication-note type pairs and the corresponding percentages at each PPV level ...... 67 Table 21: Identified ADRs and their frequencies ...... 68 Table 22: Baseline review results for each medication ...... 69

Appendix Table 1: Frequencies and cumulative percentages for the top 50 most frequently used medications in EHR data ...... 80 Appendix Table 2: Excluded reactions ...... 82 Appendix Table 3: Excluded FDA drug-reaction pairs ...... 84 Appendix Table 4: Included FDA drug-reaction pairs ...... 85

vii

List of Figures Figure 1: cTAKES components and their dependencies (Masanz, 2013) ...... 11 Figure 2: Overview of the study...... 17 Figure 3: Example of one single one-drug-ADE and two multi-drug-ADE cases ...... 18 Figure 4: Age code and report source code (a. Age code; b. Report source code) ...... 19 Figure 5: Data preprocessing on the FDA reports ...... 20 Figure 6: Brand/generic name searching in the Lexicomp Online Database ...... 20 Figure 7: Preprocessing of medication names ...... 21 Figure 8: Medication name tokenization and the corresponding searching of the candidates ...... 21 Figure 9: Example of search queries ...... 22 Figure 10: Examples of returned pages ...... 24 Figure 11: Parsing rules for identifying generic name from the returned HTML page ...... 24 Figure 12: Parsing rules for identifying the FDA generic names from the returned HTML pages .. 25 Figure 13: User interfaces for the manual review tool ...... 27 Figure 14: Medication names (left column) mapping to generic names (right column) ...... 27 Figure 15: The UMLS Terminology Browser ...... 32 Figure 16: Data preprocessing for clinical notes ...... 34 Figure 17: Workflow of the NLP pipeline ...... 35 Figure 18: Example sentences and the cTAKES output of DDSS terms and CUIs ...... 38 Figure 19: Pipeline for identifying ADR-related DRPSs ...... 45 Figure 20: Plots for ADR-related DRPSs at each cut-off percentage ...... 56 Figure 21: Pipeline for identifying baseline ADR-related DRPSs ...... 57

viii

Abbreviations

ADE ------Adverse Drug Event

ADR ------Adverse Drug Reaction

CCHMC ------Cincinnati Children’s Hospital Medical Center cTAKES ------clinical Text Analysis and Knowledge Extraction System

CUI ------Concept Unique Identifier

DDSSs ------Diseases/Disorders and Signs/Symptoms

DRPS ------Drug-Reaction Pair Sentence

EHR ------Electronic Health Record

FAERS ------Food and Drug Administration’s Adverse Event Reporting System

FDA ------Food and Drug Administration

NLP ------Natural Language Processing

UIMA ------Unstructured Information Management Architecture

1 Introduction

Detecting post-marketing safety signals earlier is increasingly important and of great interest to industry, regulators, and the public (Honig, 2013). However, it remains challenging to incorporate clinical evidence in electronic adverse drug reaction alerting systems, especially in pediatric settings (Carspecken et al.,

2013).

An adverse drug reaction (ADR) is defined as “an appreciably harmful or unpleasant reaction, resulting from an intervention related to the use of a medicinal product, which predicts hazard from future administration and warrants prevention or specific treatment, or alteration of the dosage regimen, or withdrawal of the product.” (Edwards & Aronson, 2000). ADRs can happen in all health care settings including inpatient, outpatient and nursing homes, causing more than 700,000 injuries and deaths in the

United States annually (Classen et al., 1997; Cullen et al., 1997; HHS, 2015). Many ADRs could be detected and mitigated if we pay more attention to patients’ clinical status changes. It is a big challenge to detect such changes and eventually ADRs from complex clinical documents, however, it is worthwhile for clinical researchers to contribute to this field because the improvement can prevent harm to the patients and even save their lives.

Clinical researchers have conducted a lot of work in identifying ADRs. In the early phase, researchers manually reviewed the clinical documents in order to identify ADRs. With the rapid development of information technology, the researchers started leveraging the advantages of Electronic Health Records

(EHRs) and the newly emerging techniques to identify ADRs. The EHRs store both structured and unstructured information related to a patient over time, such as demographics, admission and discharge information, medications taken, and clinical notes. The ADR-related information is hidden in the massive amount of the EHR data. Therefore, the EHR data is a rich resource for data mining and knowledge discovery on ADR (Roque et al., 2011).

Clinical narratives (clinical notes) are essential resources for ADR detection. However, using clinical notes presents a challenge because it is error-prone and labor-intensive to manually process the narrative text. To address this problem, I used the clinical Text Analysis and Knowledge Extraction System

(cTAKES), a modular and open-sourced natural language processing (NLP) system designed to process and extract information from clinical text (Savova et al., 2010).

In addition, the Food and Drug Administration’s Adverse Event Reporting System (FAERS) collects adverse drug events (ADEs) from all kinds of report sources (FDA, 2015) such as health professionals and patients. Both structured reports and unstructured plain text are provided to describe ADR-related information in the Food and Drug Administration (FDA) website. These sources, particularly the structured reports, are good resource for clinical research and pharmacovigilance.

In this study, I focused on exploring both structured and unstructured data from the EHR to detect ADR events, by using NLP techniques and FAERS reports. Since the FAERS reports have specified which drug is suspected to cause a reaction, I used this information to detect ADRs in the EHR data. That is, I used the information from FAERS reports to automatically identify reactions that might be caused by a specific medication. Focusing on the most frequently used medications in out EHR data, I first performed

NLP to identify DDSS/reactions information from clinical notes. Then I identified drug-reaction pair sentences (DRPSs) in clinical narratives by searching a medication name and the corresponding FAERS reported reaction(s). Finally, a sentence-based ADR detection by applying NLP methods was conducted to exclude non-ADR related DRPSs. The remaining ADR-related DRPSs were ranked based on thresholding reactions for a medication. Manual review was performed on a randomly sampled set of

ADR-related DRPSs to evaluate the performance.

The objective of this project was to build a pipeline to detect ADRs in the EHR data. By leveraging the information from FDA reports, I built an NLP-based, sentence-level ADR detector. I proposed the following null hypothesis:

H0: leveraging drug-reaction pairs from the FAERS will not improve the performance of EHR-based

ADR detection.

Ha: leveraging drug-reaction pairs from the FAERS will improve the performance of EHR-based ADR detection.

In order to verify the null hypothesis, I generated a baseline ADR detection algorithm and compared the results with that of the algorithm using FAERS reports.

The next chapters describe my work in details. Chapter 2 introduces the related work for the study.

Chapter 3 describes the data sets used to detect ADRs. Chapter 4 describes the rationale and the implementation processes of the ADR detector. Chapter 5 presents the results. Chapter 6 discusses the findings, specifies the limitations, and proposes potential improvements that could be made in the future.

Finally, Chapter 7 concludes the work.

2 Related work

This section describes the literature studies related to my work. I first introduce the definitions of ADR and specify the scope of the study (Section 2.1). I then review the literature studies related to ADR detection. Finally, I review the NLP technologies which are the fundamental basis of the study.

2.1 Definitions related to ADR and scope of the study

There are several terms describing medication related harms: medication error, adverse event, adverse drug event, and adverse drug reaction. Medication error is defined as “any preventable event that may cause or lead to inappropriate medication use or patient harm” (NCCMERP, 2015). An adverse event is a undesirable event that may or may not be caused by a medication (Trotti et al., 2003). An adverse drug event is an injury related to a particular medication. ADE includes harm caused by the medication

(including adverse drug reactions and overdoses), and harm from the use of the medication (including

3 dose reductions and discontinuations of drug therapy) (Ramesh et al., 2014). An adverse drug reaction is defined as “an appreciably harmful or unpleasant reaction, resulting from an intervention related to the use of a medicinal product, which predicts hazard from future administration and warrants prevention or specific treatment, or alteration of the dosage regimen, or withdrawal of the product.” (Edwards &

Aronson, 2000). Nebeker et al. addressed the main difference between ADE and ADR (Nebeker, Barach,

& Samore, 2004). That is, ADR indicates the harm directly caused by a medication at normal doses, while

ADE includes all harms caused by the use of a medication, including overdoses. In other words, ADRs are a subset of ADEs.

In this study I focused on identifying ADR events in clinical notes. Dosage information is not considered in the study.

2.2 Adverse drug reaction detection

Data models and algorithms were proposed to facilitate ADR detection. Chazard et al. proposed a data model for ADE detection in EHRs (Chazard et al., 2009). Liyanage & De Lusignan applied ontologies to capture adverse events following immunization (AEFI) from health data (Liyanage & De Lusignan, 2014).

The rationale was to use method of ontological representations of clinical concepts to map similar concepts (AEFI across different coding systems). Lependu et al. presented a novel method to annotate clinical notes and transform them into a patient-feature matrix (Lependu et al., 2013). The matrix could be used to detect drug-ADE associations and ADEs associated with drug-drug interactions. Kirkendall et al. contributed to the ADE and medication error research by conducting a retrospective, quantitative analysis to evaluate the accuracy of medication dosing rules from a vendor-supplied set of electronic rules

(Kirkendall et al., 2014).

Rule-based ADR detection were widely used in recent work. Chazard et al. generated ADE detection rules using decision trees and association rules based on structured and unstructured data from the EHR

4 for 115,447 inpatient encounters (Chazard et al., 2011). The rules were then filtered, validated, and reorganized by experts. Finally, they applied 236 validated ADE-detection rules to automatically detect

27 types of ADEs. One of the limitations for Chazard’s method was that producing ADE detection rules by data mining was complex. By applying the 236 validated ADE-detection rules and statistical analysis,

Chazard et al. concluded from another work that contextualization was necessary for ADE detection and prevention (Chazard et al., 2012). Sohn et al. developed a system to extract physician-asserted drug side effect sentences from EHR clinical notes for psychiatry and psychology patients (Sohn et al., 2011). They first manually developed pattern matching rules by examining keywords and expression patterns of side effects to discover a side effect and its causative drug relationship. Then, a combination of decision trees

(C4.5) using side effect keyword features and pattern matching rules was used to extract sentences that contain side effect and causative drug pairs. The system achieved a PPV of 64%. There were two limitations for Sohn’s study. First, they focused on a small corpus of a particular domain (psychiatry).

The generalizability of the rules for identifying side effects on other domain-corpus was unknown.

Second, a complicated or indirect description of side effect might be missed by the system because the system only extracted clearly defined side effects and causative drugs with limited indication words and patterns. My study was inspired from this work and the differences were: first, I used FDA ADR reports to extract drug-reaction pairs from the reports. Second, I applied NLP and regular expressions to identify sentences containing the extracted reaction and the presumably causative drug pair in the EHR data.

Finally, I utilized NLP tools to identify complicated or indirect Diseases/Disorders and Signs/Symptoms

(DDSSs) from clinical notes.

NLP was used in several ADR detection studies before. Aramaki et al. studied identification of ADEs from clinical records and the results showed that 7.7% of clinical records include adverse event information, in which 59% (4.5% in total) can be extracted automatically by NLP systems. The result was encouraging due to the massive amounts of clinical records (Aramaki et al., 2010). Wang et al. performed

NLP based method to detect ADEs in narrative clinical discharge summaries and applied contextual

5 filtering on the reports to improve performance (Wang et al., 2010). Evaluation indicated that applying the filters improved recall (from 43% to 75%) and PPV (from 16% to 31%) of ADE detection. One of the limitations for Wang’s study was that they only studied discharge summaries. Park et al. proposed an algorithm for detecting laboratory abnormalities after treatment with medications (Park et al., 2011). The

PPV of the algorithm was 22-75%. Epstein et al. developed a high-performance NLP algorithm in

Transact-SQL to identify medication and food allergies and sensitivities from unstructured allergy entries in EHR systems (Epstein et al., 2013). Their algorithm was developed on a training set of 9445 records and tested on a separated set of 9430 records. PPV for medication allergy matches was 99.99% on both the training and the test sets. A limitation of Epstein’s study was that the dictionaries developed might not be fully portable to other institutions. A functional evaluation for data mining tasks was performed by

Jung et al. (Jung et al., 2014). Their work came up with two conclusions: first, using simple term recognition methods instead of advanced NLP methods resulted in little or no impact on accuracy when using large datasets; second, simpler dictionary-based methods had the advantage of scaling well to very large datasets.

The literature shows that data mining and machine learning methods were applied to detect ADRs in EHR data. Ji et al. proposed a data mining approach to identify potential ADRs from EHRs (Ji et al., 2011).

They introduced causal association rules to represent potential causal relationships between a drug and

ICD-9 coded signs or symptoms representing potential ADRs. Eight out of fifty top event pairs were validated to represent true causal associations. The PPV for Ji’s study was low (PPV=16%). Lingren classified the sentences into informative and non-informative categories, then applied support vector machine (SVM) to detect ADEs on informative sentences through sentence classification (Lingren, 2013).

The PPV for identifying ADEs in his work was 80.4%. However, the work only focused on the data from ten patients.

Other methods were used to detect ADRs. Yoon et al. proposed a quantitative post marketing surveillance algorithm, the Comparison of Laboratory Extreme Abnormality Ratio (CLEAR), for detecting ADR

6 events from EHR data (Yoon et al., 2012). They tested the algorithm on 470 randomly selected drug- event pairs from a 10-year EHR data set. 120 out of 150 detected events were ADRs (PPV = 83.7%). The

CLEAR algorithm had several limitations: first, it focused only on laboratory results. Symptom-related

ADRs such as dizziness, dry mouth, and weight gain, which were usually recorded in free-text were not detected; second, concomitant drugs and dose-related effects were not considered. Liu et al. designed a study to correlate abnormal laboratory results with specific drug administrations by comparing the outcomes of a drug-exposed group and a matched unexposed group (Liu et al., 2013). The highest PPV achieved by the authors was 77%. The weakness of Liu’s study included that the gold standards used were neither fully authoritative nor complete, and that the prescribed drugs usually counteracted known anticipated reactions. Henao et al. used a patient clustering model to identify ADEs in clinical notes.

Performance of the ADE detection was not evaluated in Henao’s study (Henao et al., 2013). Pathak et al. applied Linked Data principles and technologies to represent patient EHR data as Resource Description

Framework, and then identified potential drug-drug interactions for cardiovascular and gastroenterology drugs (Pathak et al., 2013). However, a more robust and rigorous evaluation was required before the system could be deployed. Another disadvantage of the study was that they only used drug prescription data. In contrast, I used both drug prescription and administration data of patients in my study. Li et al. presented two phenotyping algorithms to detect adverse event and medical errors in a neonatal intensive care unit (Li et al., 2014). Their automated ADE detection algorithms provided higher recall and positive predictive value (PPV) than currently used trigger tools or voluntary incident-reporting systems.

Other work focused on a specific or a specific type of drug or reaction. Overby et al. developed an EHR phenotyping algorithm for drug-induced liver injury (Overby et al., 2013). The algorithm identified 117 cases and 23 confirmed true positive cases (PPV was 19.7%). Cheetham et al. developed an automated causality assessment algorithm to identify drug-induced liver injury in EHR data (Cheetham et al., 2014).

They only applied the algorithm on structured EHR data. Kirkendall et al. focused on electronic acute kidney injury (AKI) triggers to identify pediatric patients at risk for nephrotoxic medication (NTMx)-

7 associated harm (Kirkendall et al., 2014). They built a risk-stratifying system based on electronic triggers that detected patients at-risk for NTMx-AKI before injury occurs. The PPV increased over the study, from ≥78% to ≥95% consistently. By identifying upper gastrointestinal bleeding misclassification (UGIB) cases using free text or disease codes in four health-care databases, Valkhoff et al. proved that UGIB affected PPV but not caused magnitude of drug-related upper strointestinal bleeding risk (Valkhoff et al.,

2014). From each database, they randomly selected and manually reviewed 200 cases to calculate PPVs.

The PPVs for evaluating free text in two databases was 22% and 47%, respectively. Observation-based methods were applied to study some specific medications such as Levetiracetam, which always caused severe reactions as were reported by physicians and patients. For example, Kirmani et al. conducted a study on the role of Intravenous Levetiracetam in seizure prophylaxis of severe traumatic brain injury patients (Kirmani et al., 2013). The main limitation for these algorithms was that they focused on a particular drug or reaction. The algorithms and the conclusions might not generalize to a large scope of medications and reactions.

Time window (time course, censoring periods) was used to exclude false positive ADRs in the following two studies. Sai et al. developed a detection algorithm for statin-induced myopathy (SIM) (Sai et al.,

2013). Five subjects with suspected SIM were identified (the PPV was 100%). The authors concluded that combined use of exclusion criteria for disease, medical practice data and time course of medication contributes to better prediction of SIM. However, it need study a larger population to confirm the effectiveness of the current algorithm. Sauzet et al. illustrated the usability and examined the reliability of the weibull shape parameter (WSP) tool as a method for event detection in EHR (Sauzet et al., 2013).

Censoring periods (time window) were used and anlyzed in the study. There were three limitations for this study: first, when the events were uncommon the success of this method might depend on the acuuracy of reporting time; second, further work was required to define the censoring periods (time window); third, all patients who were prescribed were considered at risk, even if they did not take the drug.

Researchers also utilized public resource to facilitate ADR detection. Harpaz et al. tailored an association rule mining algorithm to identify multi-item ADE associations (associations relating multiple drugs to possibly multiple adverse events) in FAERS (Harpaz et al., 2010). Ramesh et al. developed an annotated corpus of FAERS narratives and used machine learning-based algorithms to identify drugs and adverse event information from the FAERS narratives (Ramesh et al., 2014). Their study enriched the FAERS data for post marketing pharmacovigilance.

2.3 Natural language processing on clinical notes

There is a huge amount of EHR data and the size is increasing every day. However, a large portion of the

EHR data is stored as narrative clinical notes. A lot of clinical information (e.g., symptoms, treatment, and ADRs) is described in clinical notes. As such, NLP plays an important role in extracting clinical information from the EHR data. NLP-based methods have been applied to medical condition identification, negation detection, patterns extraction, and clinical note de-identification (L. Deleger et al.,

2012), and so on. In this section, I introduce three main NLP techniques used in this study.

2.3.1 Medical condition identification

Medical conditions are defined as all diseases, illnesses, and injuries except for mental disorders

(Association, 2013). Medical conditions include indications, contraindications, overdosage, and adverse reactions (sign/symptoms and disease/disorders). Identifying medical conditions in clinical notes is a substantial step for ADR detection.

Based on existing open-source technologies of the Unstructured Information Management Architecture

(UIMA) framework and OpenNLP natural language processing toolkit, Savova et al. developed an open- source NLP system, named the clinical Text Analysis and Knowledge Extraction System (cTAKES), for information extraction from EHR clinical notes (Savova et al., 2010). cTAKES was specifically trained

9 for clinical text and it could create rich linguistic and semantic annotations that can be utilized in clinical research. The created annotations were the foundation for higher-level processing and application of clinical notes. Garla et al. developed the Yale cTAKES Extensions (YTEX) --- extensions to the cTAKES that simplified feature extraction, experimentation with various feature representations, and the development of both rule and machine-learning based document classifiers (Garla et al., 2011). The

YTEX was available at http://code.google.com/p/ytex.

The main function of cTAKES used in my study is processing clinical notes and identifying clinical named entities from the clinical notes. cTAKES can extract several types of clinical entities, including medications, diseases/disorders, signs/symptoms. cTAKES can also extract attribute information for the identified named entity from the text span, including the map of ontology code (e.g., SNOMED-CT code), affirmative/negated expression, and context information such as family history, current status, unrelated to patient. cTAKES consists of a number of function-based components. Each component includes at least one analysis engine (annotator) and has certain function(s). Users can chose to use one or more component(s) to help extract information from clinical notes. The UIMA framework of cTAKES allows the users to select annotators and the output order of the annotations. Dependencies for these components are shown in Figure 1 (Masanz, 2013). My study used Clinical Documents Pipeline which was the top-level and main project for processing a clinical document through the entire cTAKES pipeline. The Clinical

Documents Pipeline included sentence detection, part of speech tagging POS, chunking, named entity recognition, context detection, and negation detection (Bleeker, 2012).

Figure 1: cTAKES components and their dependencies (Masanz, 2013)

Based on the open-sourced cTAKES, Li et al. developed a hybrid NLP system, named AutoMCExtractor, to identify medical conditions (e.g., signs, symptoms, diseases and disorders) from FDA drug labels (Li et al., 2013). The PPV, recall, and F-measure of AutoMCExtractor for the token-level evaluation were 92%,

73%, and 82%, respectively.

2.3.2 Contextual property detection

Identifying medical conditions in the clinical notes is important but not sufficient for identifying ADRs in clinical notes. The context information of the medical condition is critical to determine an ADR.

Therefore, the contextual properties for a medical condition including certainty, negation, hypothesis, and the relation with the medication should be analyzed as well.

Approximately half of the medical conditions described in clinical notes are negated (Chapman et al.,

2001b). Methods for identifying negated medical conditions have been developed in recent years

(Auerbuch et al., 2004; Chapman et al., 2001a; Elkin et al., 2005; Mutalik et al., 2001). Chapman et al. developed a regular expression based algorithm, NegEx, to identify negated sentences by looking for negation phrase appearing before or after a medical condition (Chapman et al., 2001a; Chapman et al.,

2013). A list of negation phrases were defined before negation detection. Averbuch et al. developed a machine learning-based algorithm to learn negative patterns in clinical notes and then applied the patterns to detect negation medical conditions (Auerbuch et al., 2004). After evaluating different methods of negation detection, Goryachev et al. concluded that the accuracy of regular expressions methods (91.9-

92.3%) was higher than that of machine learning-based methods (83.5-89.9%) (Goryachev et al., 2006).

Based on this conclusion, I used the regular expression-based algorithm NegEx to detect negation in the study.

Building on NegEx, Harkema et al. developed an algorithm ConText to determine if a clinical condition mentioned in clinical reports was negated, hypothetical, historical, or experienced by someone other than

12 the patient (Harkema et al., 2010). ConText was evaluated and had reasonably good performance for negated, historical, and hypothetical conditions across all reports.

3 Data

3.1 Electronic Health Records (EHR)

This study was performed on the data of 42,995 pediatric patients treated as inpatients at Cincinnati

Children’s Hospital Medical Center (CCHMC) between January 1, 2010 and August 31, 2012. I retrospectively extracted data for 71,909 inpatient visits (referred to as “encounters”) from the EHR. This data set included demographics, clinical notes and medication administration records (MARs) during the hospitalization of the 42,995 pediatric patients. Description of the data set is in Table 1.

Table 1: EHR data

File Name Number of Description

Entries

Demographics 71,909 Demographics about a patient, such as birth date, name, address, etc.

MAR 6,957,393 Medication Administration Record. This data contained date and time

the medications were given to the patients during the patients’

hospitalization.

Notes 2,786,410 Clinical notes (narrative text) written by doctors or nurses.

In order to detect ADR, I need to identify both medications taken by the patients and reactions caused by the medications. I focused on MAR to get the information of medications given to the patients during the inpatient period. There were 3,915 unique medications recorded in the MAR data. The free text clinical

13 notes were used for retrieving information of medical conditions (Diseases/Disorders and

Signs/Symptoms, DDSS).

3.2 FDA FAERS Reports

The FDA Adverse Event Reporting System (FAERS) is a database that contains information on adverse event and medication error reports submitted to the FDA (FDA, 2015). Adverse events and medication errors in FAERS are coded to terms in the Medical Dictionary for Regulatory Activities (MedDRA) terminology. Some ADE reports were received from healthcare professionals (e.g., physicians, pharmacists, nurses), while others from consumers (e.g., patients, family members, lawyers) and manufacturers.

I downloaded 10 years (2004-2013) of ADE reports from the FDA online database. Reports from 2004 to

2012 were from the older Quarterly Legacy AERS Data Files; while reports after 2012 (including the last quarter of 2012) were from the new FAERS database. Both databases update ADE report files quarterly, e.g., file name “DEMO12Q2.TXT” indicates a file for “demographic information of the second quarter

(from April to June) of the year 2012”. A total of 2.94 GB files organized as 308 Quarterly ASCII data files were downloaded for the study. Information such as demographics, drug, reaction, patient outcomes, report sources, drug therapy and indications for use (diagnoses) for the reported drugs, were included in the files. Table 2 shows the FDA ADE Reports files and their descriptions for this study.

Table 2: FDA ADE Reports

File Number of Description

Entries

Demographics 5,332,404 Contains patient demographics (e.g., ages) and administrative

information, a single record for each event report.

Drug 19,366,958 Contains information of medications reported for the event.

Reaction 19,221,277 Contains adverse event coded to terms in the Medical Dictionary for

Regulatory Activities (MedDRA) terminology.

Report Sources 1,951,787 Contains report sources for event.

Therapy 7,818,140 Contains start dates and end dates of drug therapy for the reported drugs.

4 Methods

The objective of this study was to detect adverse drug reaction in the EHR data, with the aid of the FDA

FAERS Reports. Since the FDA ADE reports have specified which drug is suspected to cause a reaction,

I used this information to detect ADRs in the EHR data. That is, I matched medication names between the

FDA ADE reports and the EHR data, and used the associated FDA reactions reported for that particular drug as the potential ADR candidates. I then searched for the reactions in the clinical notes after a mediation was given to identify ADRs in the EHR.

In the study I applied five main steps to achieve my goal:

Firstly, I identified the most frequently used medications based on frequencies counted on generic names of the medications. Since our study was data-driven and frequency-based, the results would be unreliable if the occurrence of a medication was rare in either source of the data. Hence, I calculated the frequencies of medications for both data sources. The most frequently used medications were then identified. I focused on detecting ADRs for the most frequently used medications.

Secondly, I performed NLP to identify DDSS/reactions information from clinical notes. Medical conditions were not recorded in the EHR as structured data. Therefore, I relied on the unstructured

15 clinical notes to identify DDSS/reactions. Natural language processing techniques were leveraged to identify reactions information from clinical notes.

Thirdly, I identified Drug-Reaction Pair Sentences (DRPSs) in the clinical notes. I matched medication names between the FDA ADE reports and the EHR data and used the associated FDA reactions as the potential ADR candidates. According to the potential ADRs from the FDA reports for each most frequently used medications, I searched drug-reaction pairs for potential ADRs in each clinical note sentence. If both a medication name and the corresponding FAERS reported reaction(s) were found in that sentence, I considered the sentence a Drug-Reaction Pair Sentence (DRPS).

Then, I performed a sentence-based ADR detection to identify ADR-related DRPSs. The rationale was to find DRPSs which were most likely to indicate ADRs. To identify such ADR-related DRPSs from all detected DRPSs, I performed NLP steps (section analysis, sentence-based post-processing) on the DRPSs to rule out non-ADR DRPSs. The remaining sentences were considered ADR-related DRPSs.

Finally, two independent reviewers manually identified ADRs from the ADR-related DRPSs generated by my algorithm. Positive Predictive Value (PPV) was calculated based on the results of the manual review.

The overview of the study is illustrated in Figure 2.

Figure 2: Overview of the study

4.1 Most frequently used medications

My study focused on pediatric patients. However, the FDA reports contained drug-reaction pairs related to both pediatric and adult populations. In addition, I considered the FDA reports from non-professional sources unreliable for the purpose of the current research. To address these problems, data preprocessing had to be done on the FDA reports. Section 4.1.1 describes the process of data preprocessing.

Medications in both data sources contained both generic name and brand name, with additional dosage terms and some typos. I mapped/projected all medication names to corresponding generic names to group

17 medications before calculating their frequencies. Section 4.1.2 describes the details of medication name mapping process. Finally, I counted the medication frequencies and identified the most frequently used medications in section 4.1.3.

4.1.1 Data preprocessing on FDA FAERS reports

I focused on one-drug-ADE (ADE caused by only one drug) to avoid complicated drug interactions.

Figure 3 illustrates a one-drug-ADE case (case 100206491) and two multi-drug-ADE cases (cases

100206501 and 100206512). Case 100206491 was a one-drug-ADE because the patient was given only one medication (“GLASSIA”) according to FDA reports. The medication (“GLASSIA”) reportedly caused four reactions, “Blood glucose decreased”, “Hypotension”, “Renal failure” and “Urticaria”. Cases

100206501 and 100206512 were not one-drug-ADE cases because the patients were given more than one medication.

Figure 3: Example of one single one-drug-ADE and two multi-drug-ADE cases

I focused on pediatric patients between 0 and 18 years old and professional sourced FDA reports. Figure

4 shows the key fields for excluding adults and non-professional FDA reports. The Age_Code (Figure 4a) along with Age_Number from the Demographics file were used to rule out patients with ages greater than

18 years old. That is, if the Age_Code was “YR” and Age_Number was ≤18, it was classified as a pediatric ADE report. The RPSR_Code field (Figure 4b) from the Report Sources file was used to identify all professional reports (RPSR_Code = HP, Health Professional).

a B

Figure 4: Age code and report source code (a. Age code; b. Report source code)

Figure 5 illustrates the data preprocessing procedure on the FDA reports. Among the 5,332,243 unique

FDA-reported ADE cases, 43% (2,298,093 cases) were one-drug-ADE cases. 10.9% of the one-drug-

ADE cases were reported by a health professional, among which, 7.7% were pediatric cases. Hence, a total of 19,247 cases were one-drug-ADE, pediatric cases, and were reported by a health professional.

Figure 5: Data preprocessing on the FDA reports

4.1.2 Medication name mapping

In both FDA reports and EHR data, the medications were recorded using both generic name and brand name. If I would have matched these medication names literally, I would get many false negatives (e.g.,

“Tylenol” does not match “Acetaminophen”). To solve this problem, I mapped each medication name to the corresponding generic name in order to group medications, count medication frequencies, and perform medication matching.

I used Lexicomp Online Database (Lexicomp, 2015) to construct a medication name mapping dictionary.

Lexicomp Online Database is a commercial online database that provides searching for medication names and clinical information. Figure 6 illustrates the searching interface of the Lexicomp Online Database.

Figure 6: Brand/generic name searching in the Lexicomp Online Database

I preprocessed the medication names and developed tools and scripts to look up their generic names automatically. The details of the preprocessing and automated searching are provided below:

Step 1. I preprocessed medication names by removing dosage information. Figure 7 illustrates this preprocessing procedure.

Figure 7: Preprocessing of medication names

Step 2. I developed query commands and ran them using an open source tool (curl.exe) to automatically extract HTML pages from Lexicomp online. The preprocessed medication names for EHR data could be directly used as search terms in Lexicomp online. However, I couldn’t develop proper regular expression

(rules) to process various cases for FDA medication names. E.g., “TYLENOL CAPLET” in Figure 8 can’t be identified from Lexicomp because it contains term “CAPLET”, which was not removed in medication name preprocessing. To address this problem, I tokenized the medication names and created searching candidates (each individual or combination of tokens). Figure 8 shows an example of tokenizing medication names into searching candidates.

Figure 8: Medication name tokenization and the corresponding searching of the candidates

Figure 9 shows the queries for searching three medication names, “TYLENOL”, “CAPLET”, and

“TYLENOL CAPLET”, respectively. A total of 19,762 queries were generated and executed to automatically extract and return HTML pages to our server. The HTML pages were saved as .txt files.

Figure 9: Example of search queries

Step 3. I created rules to parse the returned HTML pages and got the corresponding generic names. Figure

10 shows examples of returned pages for brand name searching (Figure 10a), generic name searching

(Figure 10b) and a non-medication name searching (Figure 10c), respectively.

Figure 10: Examples of returned pages

(a. Brand name searching; b. Generic name searching; c. Non-medication name searching)

Figure 11: Parsing rules for identifying generic name from the returned HTML page

Figure 11 shows the parsing rules for identifying generic name from the returned HTML page. The medication names in the EHR data were easier to parse since each medication name was returned by one

HTML page. However, the medication names in the FDA reports have been tokenized and combined as several search candidates. As such, I parsed the returned results for the search candidates using additional parsing rules (Figure 12).

Figure 12: Parsing rules for identifying the FDA generic names from the returned HTML pages

Step 4. I performed a manual review to identify the correct generic name for each medication. In order to facilitate the manual review, I developed a tool with convenient user interface. Figure 13 shows snapshots of the developed tool for the manual review. As shown in Figure 13a, the tool loaded the medication mapping dictionary using the “Load” button; then it used the “Config” button to configure the returned

HTML pages folder. There were two columns in the text field. The preprocessed medication names were listed in the left column. The generic names identified in step 3 were listed in the right column as candidate generic names for the manual review. Double clicking a word in the text field opened another window (Figure 13b) to review this word. Word or phrase election would trigger an event of loading the corresponding returned HTML pages. Based on the content shown at the bottom of the window, the user could easily decide if the candidate was the correct generic name for the searched text (word or phrase). If the candidate was correct, I clicked the button “next” to confirm the candidate and loaded the next item;

25 otherwise, I copied the correct generic name and pasted it to the candidate textbox. Updates were automatically saved to the medication name mapping dictionary.

Figure 13: User interfaces for the manual review tool

(a. Main window for loading medication name mapping dictionary and configuration; b. Window for manual review with the aid of the content from the returned HTML pages)

4.1.3 Most frequently used medications

Based on the medication name mapping dictionary created by the previous section, the medications from the EHR MAR and the FDA reports were grouped to calculate frequencies. For instance, the medication names with dosage information or the brand names (left column in Figure 14) were mapped to the same generic name(right column in Figure 14) after the original medication name mapping.

Figure 14: Medication names (left column) mapping to generic names (right column)

After calculating the medication frequencies, I generated a list of the top 50 most frequently used medications from the EHR data. Appendix Table 1 shows the frequencies of the top 50 most frequently used medications from the EHR data. Finally, I focused on the 41 medications (Table 3) which were among the top 50 most frequently used medications in EHR data and the number of FDA ADE report(s) of the medications were greater than 0.

Table 3: Frequencies and cumulative percentages for the 41 most frequently used medications

Cumulative Cumulative Cumulative Number Number Percentage Number of Percentage Percentage of EHR of EHR Drug Name over all EHR MAR over all over all MAR MAR EHR MAR Encounters EHR MAR EHR MAR records Patients records Encounters Patients SODIUM CHLORIDE 139,941 10.9% 48,363 71.4% 29,790 73.5%

ACETAMINOPHEN 69,163 16.3% 37,333 82.9% 24,281 85.0%

HEPARIN 46,373 19.9% 13,185 83.6% 6,585 85.5%

MORPHINE SULFATE 34,606 22.6% 13,483 84.5% 11,077 86.6%

FENTANYL CITRATE 26,924 24.7% 10,293 85.2% 8,722 87.3%

MIDAZOLAM 24,817 26.7% 6,749 85.5% 5,627 87.6%

ONDANSETRON 24,710 28.6% 14,021 86.1% 9,520 88.1%

DIPHENHYDRAMINE 24,087 30.5% 11,225 86.9% 5,203 88.9%

FUROSEMIDE 22,822 32.2% 4,222 87.0% 2,542 88.9%

LORAZEPAM 22,635 34.0% 5,216 87.2% 3,287 89.0%

CEFAZOLIN 18,811 35.5% 9,176 87.4% 7,622 89.2%

OXYCODONE 14,455 36.6% 7,297 87.5% 5,299 89.3%

IBUPROFEN 14,126 37.7% 9,974 88.4% 8,305 90.2%

RANITIDINE 13,354 38.7% 8,771 89.1% 5,817 90.7%

VANCOMYCIN 12,824 39.7% 4,487 89.1% 3,011 90.7%

IOVERSOL 12,331 40.7% 8,136 89.2% 6,045 90.8%

CLINDAMYCIN 12,290 41.7% 5,389 89.4% 4,550 90.9%

INSULIN ASPART 12,247 42.6% 1,028 89.5% 681 91.0%

ALBUMIN 11,233 43.5% 2,135 89.5% 1,491 91.0%

METHYLPREDNISOLONE 10,640 44.3% 4,517 89.6% 2,622 91.0%

LANSOPRAZOLE 10,593 45.2% 7,847 90.4% 3,570 91.3%

HYDROMORPHONE 10,584 46.0% 2,845 90.4% 2,108 91.3%

LIDOCAINE 10,503 46.8% 7,656 90.6% 5,125 91.5%

DIAZEPAM 10,195 47.6% 4,567 90.7% 3,433 91.5%

INSULIN LISPRO 9,728 48.4% 789 90.8% 640 91.6%

HYDROCORTISONE 8,848 49.0% 3,287 91.0% 1,982 91.8%

EPINEPHRINE 8,832 49.7% 3,819 91.2% 3,168 92.0%

CEFTRIAXONE 8,485 50.4% 5,601 91.2% 4,413 92.0%

BUPIVACAINE 8,255 51.0% 7,378 91.4% 6,500 92.1%

KETOROLAC 7,891 51.7% 5,410 91.5% 4,606 92.2%

AMOXICILLIN 7,876 52.3% 6,099 92.0% 5,231 92.8%

LIDOCAINE & PRILOCAINE 7,726 52.9% 5,618 92.3% 3,969 93.2%

METHADONE 7,564 53.5% 877 92.4% 710 93.2%

NYSTATIN 6,841 54.0% 4,862 92.5% 2,400 93.4%

PIPERACILLIN&TAZOBACTAM 6,702 54.5% 3,396 92.5% 2,127 93.4% DEXAMETHASONE SODIUM 6,458 55.0% 4,080 92.6% 3,430 93.4% PHOSPHATE SODIUM BICARBONATE 6,378 55.5% 1,511 92.6% 1,234 93.4%

KETAMINE 6,377 56.0% 1,358 92.6% 1,262 93.4%

EPINEPHRINE&LIDOCAINE 6,262 56.5% 5,478 92.6% 4,496 93.5%

LEVETIRACETAM 6,177 57.0% 2,720 93.0% 1,308 93.7%

MORPHINE 6,146 57.5% 2,573 93.0% 2,282 93.7%

The 41 medications in Table 3 guaranteed that the magnitude of the samples in the EHR data was sufficiently high and the number of samples in the FDA reports was greater than 0. I then focused on detecting ADRs for the 63,043 encounters (cumulative 93% encounters) in which the patients (cumulative

93.7% patients) were given these most frequently used medications.

4.2 Preparation for DDSS/reaction-CUI matching

The FDA reports and the EHR data could use different terms to express the same DDSSs and reactions. .

For instance, the DDSS of “headache” in the EHR did not match the FDA reaction “pain in head”, although they expressed the same meaning and had the same concept unique identifier (CUI,

C0018681)(NIH, 2015a). In order to match FDA reactions with DDSSs in the EHR data, I standardized the terms from both sources using the same terminology - CUI from Unified Medical Language System

(UMLS) (Bodenreider, 2004).

The next few sections describes how I processed and mapped the FDA reactions to CUIs (section 4.2.1 and section 4.2.2). Then section 4.2.3 and section 4.2.4 describes the steps performed on the clinical notes to identify the DDSS terms and the corresponding CUIs. Finally, Section 4.2.4 describes other NLP steps performed on clinical notes in order to detect ADEs.

4.2.1 Preprocessing the FDA drug-reaction pairs

I manually reviewed and excluded non-ADR drug-reaction pairs from the FDA reports. E.g., overdoses of any medication were reported as drug-reaction pairs in FDA reports. However, overdose is not an ADR according to the definition of ADR. Some drug-reaction pairs in the FDA reports (such as

LEVETIRACETAM - seizures) indicated that the medication was used to treat the reaction other than causing the reaction. Criteria to exclude reactions were listed in table 4. Appendix Table 2 lists the unique reactions excluded according to the criteria lists in Table 4. Appendix Table 3 lists the drug-reaction pairs excluded for the 41 medications according to indication information for the 41 medications obtained from the pharmaceutical website (www.Drugs.com, 2015). Appendix Table 4 lists the unique reactions included for the 41 medications.

Table 4: Criteria to skip reactions

Criteria to exclude reactions Criteria to exclude reactions Reactions mostly relevant for adults Known wrong medication or medication errors

Accidents Irrelevant or vague description of the reaction

Overdose

4.2.2 FDA reaction-CUI mapping

After preprocessing, a total of 2646 unique FDA drug-reaction pairs were included in this study. Table 5 lists the number of unique reactions for the 41 most frequently used medications. To correctly identify the corresponding CUIs for these reactions, I manually searched them in the UMLS terminology browser

(NIH, 2015b) and extracted the returned CUIs. Figure 15 shows the interface for using the UMLS

Terminology Browser. A reaction-CUI mapping dictionary was created after searching all included reaction terms for the 41 most frequently used medications.

Table 5: Number of reactions reported by FDA for the 41 most frequently used medications

Number of Unique Number of Reaction Drug Name Unique Reactions Drug Name s IBUPROFEN 319 INSULIN LISPRO 38

ACETAMINOPHEN 262 KETAMINE 38

LEVETIRACETAM 231 FUROSEMIDE 35

OXYCODONE 193 IOVERSOL 34

CEFTRIAXONE 162 DIAZEPAM 33

AMOXICILLIN 151 PIPERACILLIN&TAZOBACTAM 31

MORPHINE SULFATE 125 MIDAZOLAM 25

METHADONE 116 BUPIVACAINE 23

DIPHENHYDRAMINE 113 MORPHINE 23

ONDANSETRON 84 SODIUM CHLORIDE 21

LIDOCAINE 60 RANITIDINE 19 DEXAMETHASONE SODIUM PHOSPHATE 54 LIDOCAINE&PRILOCAINE 15

FENTANYL CITRATE 54 INSULIN ASPART 12

LANSOPRAZOLE 49 NYSTATIN 11 METHYLPREDNISOLONE 46 CEFAZOLIN 8

HEPARIN 43 EPINEPHRINE&LIDOCAINE 8

VANCOMYCIN 43 HYDROMORPHONE 4

EPINEPHRINE 40 KETOROLAC 4

CLINDAMYCIN 39 ALBUMIN 1

HYDROCORTISONE 39 SODIUM BICARBONATE 1

LORAZEPAM 39 Total 2646

Figure 15: The UMLS Terminology Browser

4.2.3 Preprocessing the clinical notes

Clinical text is a valuable resource for detecting ADR because medical conditions are usually mentioned in the unstructured clinical notes. If I could identify DDSS terms from the clinical notes, I could further analyze whether the DDSS is caused by a medication given to the patient. If yes, an ADR is detected.

There were several types of clinical notes in the EHR data. Based on the guideline created by the physician champion, I focused on processing four types of clinical notes: History & Physical (H&P),

Discharge Summaries, Emergency Department (ED), and Progress notes. Table 6 shows the descriptive statistics of the four note types.

Table 6: Types of notes used in this study

Note Types Number of Notes Number of Encounters

Progress notes 857,258 58,539

Emergency Department (ED) 121,758 28,648

Discharge Summaries 91,511 56,340

History & Physical (H&P) 97,870 53,071

Total (unique) 1,168,397 60,085

In addition, I excluded complicated encounters that had more than 1000 notes. Table 7 shows three examples of complicated encounters.

Table 7: Complicated encounters to be filtered out

Encounter ID # of Notes Register date Discharge date Days in hospital

E001 1918 22-Sep-2010 11-Oct-2011 12 months

E002 1277 28-May-2010 28-Jul-2010 2 months

E003 1175 28-Feb-2010 21-Sep-2010 7 months

Finally, I excluded the encounters in which the patients were not given any one of the 41 top frequently used medications during the hospitalization. Figure 16 illustrats the series of data preprocessing steps for the clinical notes.

Figure 16: Data preprocessing for clinical notes

4.2.4 NLP of the clinical notes

The aim of this study is to investigate the feasibility of using FDA reports to detect ADRs in EHR data.

Given each drug-reaction pair from the FDA reports, I searched both the medication name and the standardized reaction (i.e., CUI) in the cTAKES output for each sentence in a note. If both of them were found in the sentence, the sentence was identified as a DRPS.

However, the presence of a DRPS did not necessarily indicate an ADR due to several reasons. First, the

DRPS could be a negated sentence that represented the absence of the reaction (e.g., “no overnight events-pain well controlled with ibuprofen with no nausea/vomiting”). Second, some sections in a note would never mention an ADR event, such as section of “plan of care”. In addition, a sentence could indicate a hypothetical event, which was commonly used in discharge instruction and plan of care (i.e., “I will increase dose of ibuprofen if pain continues.”). Finally, based on the context, the sentence describing

34 the therapy could indicate that the medication was used to cure the reaction. For instance, the sentence

“still having headaches 7/10 and emesis, receiving ibuprofen, oxycodone and tylenol prn yesterday for pain” was followed by “pain was well controlled by oxycodone”.

To reduce the false positives, I developed a series of NLP processes to rule out the DRPSs that were not related to ADRs. I first applied section-based analysis to exclude non-ADR sections. Then I split the rest of the sections into sentences and ran cTAKES and CUI-matching to identify DRPSs. Finally, I conducted sentence-based post-processing to exclude DRPSs that were not related to ADRs (e.g., negation, hypothetical events, therapy). Figure 17 illustrates the workflow of the NLP processes I performed.

Figure 17: Workflow of the NLP pipeline

4.2.4.1 Section-based analysis

A clinical note contains several sections, such as plan, history, instruction, medication, and goal. Some sections (e.g., plan, medication) specified the future plan or prescription information. The sentences in

35 these sections would never indicate an ADR event even if they contained the medication name-reaction pairs from the FDA reports. Hence, I excluded these sections from the ADR analysis. Based on our manual review of 50 sample of clinical notes, I developed a list of regular expressions on section headers to rule out the following sections (Table 8):

Table 8: Excluded sections

Section headers Section headers Section headers Section headers

"Recommendations:"Synonyms "Recommend:"Synonyms "Treatments:"Synonyms "Treatment:"Synonyms "Recommendation:" "Plan:" "Plans:" "Goal:" "Assessment and Plan:" "Call If:" "A/P:" "Goals:" "Medication Information "Instruction:" "Instructions:" "Medications:"

""The following "Consults:" "Consult:" "Medication:" recommendations"Home Medication are "Current Medications" "Current Medication" "PRN Medications" "PRN Medication" "Current IP PRN "PRN Reasons:" "Drug Data:" offered:"Instructions HAR:" "Medication Data:" Medications""OBJECTIVE:" "antibiotics:" "antibiotics (day):" "Immune "Prophylaxis:" "scheduled Medications" "scheduled Medication"

Compromised:""Hospital Medications:" "Med List:" "Past Medical History" "Meds at time of "SCHEDULED:" "Home meds as above:" discharge:"

4.2.4.2 Splitting notes into sentences

I split the content of the notes into sentences using the period (.) symbol and more than two consecutive white spaces, because more than two white spaces represented the end of a section header in some clinical notes. I provided an example to illustrate the process. A portion of the original note is shown below:

"Mom reports some previous vomiting with taking ibuprofen. for any increase abdominal pain, decrease or no urine output Discharge Medications: Carrier, Marshall Home Medication Instructions

HAR:XXXXXX Printed on: XX/XX/XXXX 1425 Medication Information acetaminophen

(TYLENOL) 80 MG/0"

This portion of note was split into the following sentences:

1. Mom reports some previous vomiting with taking ibuprofen.

2. for any increase abdominal pain, decrease or no urine output

3. Discharge Medications: Carrier, Marshall

4. Home Medication Instructions HAR:XXXXXX

5. Printed on:XX/XX/XXXX 1425 Medication Information

6. acetaminophen (TYLENOL) 80 MG/0

4.2.4.3 Identifying DDSSs in clinical notes

I used the Clinical Text Analysis and Knowledge Extraction System (cTAKES) toolkit (Savova et al.,

2010) for medical condition identification documented in clinical notes. cTAKES used UMLS dictionaries to search tokenized clinical notes and assigned UMLS codes to the matched tokens. I split the notes into encounter-based folders, ran cTAKES on each folder, and returned all identified Systematized

Nomenclature of Medicine Clinical Terms (SNOMED CT, including DDSSs) DDSSfor an encounter to a text file. I recorded all DDSS texts identified from the notes and the corresponding CUIs. Figure 18 shows example sentences and the corresponding output DDSS terms and CUIs. Each DDSS term had a

CUI representation, for example, “C0042963” for “vomiting”, “C0000737” for “abdominal pain”,

“C0042963” for “nausea” and “C0013144” for “drowsiness”. Negated DDSS terms were identified by a prefix symbol “-”. E.g., “-C0013604 edema” indicates the patient had no symptom of edema.

Figure 18: Example sentences and the cTAKES output of DDSS terms and CUIs

4.2.4.4 Sentence-based post-processing

In the EHR, some types of notes (discharge summaries and H&P notes) had clear section boundaries (e.g., double/triple/more spaces) such that I could focus on the sections that described patients’ symptoms and medication usage (e.g. Interval History, Subjective/Interval History). However, the other notes (progress and ED notes) usually did not have clear boundaries between sections. In order to exclude hypothetical

DDSSs that often occurred in the instruction and plan of care sections, I developed an additional list of regular expressions as described below.

First, I performed a regular expression search to exclude sentences with key words that indicated plan/ hypothetical /instructions etc. The list of key words is presented in Table 9.

Table 9: Key words and captures of plan/ hypothetical /instructions/therapy

Keywords Capture "plan" plan of care "plans" plan of care "goal" the goal of therapy

"goals" the goal of therapy "will" hypothetical/future event "In context of" hypothetical "Provided instructions to family" instructions "premedicated with" therapy "premed" therapy "pre-med" therapy

I then identified patterns indicating relations between DDSSs and medications and excluded the sentences with the patterns presented in Table 10. For the patterns in Table 10, some indicated therapy relation for medication and DDSS, e.g., "for breakthrough " + DDSS, DDSS + " improved with " + medication-name; some suggested that the DDSS was an existing symptom, e.g., "due to " + DDSS; others presented a prescription, e.g., "scheduled " + medication-name.

Table 10: Patterns indicating relations between DDSSs and medications and their indication

Pattern Indication medication-name + " which helps" therapy medication-name + " with improving clinical status" therapy "resolved with " + medication-name therapy "scheduled " + medication-name prescription medication-name + "which helped" therapy "responsive to" + medication-name therapy "respond to" + medication-name therapy "responded to" + medication-name therapy "r/t" + medication-name therapy "improve with prn" + medication-name therapy "controlled on" + medication-name therapy "Continue" + medication-name therapy "continuing" + medication-name therapy

"require" + medication-name therapy "requiring" + medication-name therapy "treat with" + medication-name therapy "treating with" + medication-name therapy "treated with" + medication-name therapy "treated with low-dose" + medication-name therapy "treated with high-dose" + medication-name therapy "controlled by" + medication-name therapy "improved with use of" + medication-name therapy "controlled with" + medication-name therapy "responds nicely to" + medication-name therapy medication-name + "with good response" therapy medication-name + "prn" prescription "prn " + medication-name prescription "needed" + medication-name prescription "need" + medication-name prescription "relieved with" + medication-name therapy "well managed on" + medication-name therapy "improved with administration of" + medication-name therapy medication-name + "scheduled" prescription "relieved with" + medication-name therapy "Added" + medication-name + "for" therapy "transitioned to" + medication-name therapy "good response to" + medication-name therapy "switched to" + medication-name therapy "continuing on" + medication-name therapy "improving with" + medication-name therapy "doing well on" + medication-name therapy "changed to" + medication-name therapy

"switching to " + medication-name therapy "achieved with " + medication-name therapy "treated successfully with " + medication-name therapy "improves with " + medication-name therapy "for breakthrough " + DDSS therapy "for increased " + DDSS therapy "prevent additional " + DDSS therapy "for any " + DDSS therapy "for " + DDSS therapy "prevent " + DDSS therapy "with resolution of " + DDSS therapy "improvement in " + DDSS therapy "improvement in his " + DDSS therapy "improvement in her " + DDSS therapy DDSS + " improved" therapy DDSS + " well controlled " + medication-name therapy DDSS + " improved with " + medication-name therapy "current " + DDSS + " treatments" therapy "history of " + DDSS existing symptom "prolonged " + DDSS existing symptom "recurrent " + DDSS existing symptom "because of " + DDSS existing symptom "persistent " + DDSS existing symptom "continued " + DDSS existing symptom "again had " + DDSS existing symptom "increased incidence of " + DDSS existing symptom "due to " + DDSS existing symptom "d/t " + DDSS existing symptom medication-name + "prn" + DDSS prescription

"for additional" + DDSS therapy "for spiking a" + DDSS therapy "for recent" + DDSS therapy "for her" + DDSS therapy "for his" + DDSS therapy "avoid" + DDSS therapy "avoid breakdown with the" + DDSS therapy "improved" + DDSS therapy "improved her" + DDSS therapy "improved his" + DDSS therapy DDSS + "improved" therapy DDSS + "resolved" therapy DDSS + "has improved" therapy DDSS + "has resolved" therapy "resolved" + DDSS therapy "resolved his" + DDSS therapy "resolved her" + DDSS therapy "increasing" + DDSS existing symptom DDSS + "control:" therapy "for potential" + DDSS hypothetical "only use when" + DDSS hypothetical "avoid masking" + DDSS hypothetical "prn" + DDSS prescription "lack of" + DDSS negation "non" + DDSS negation medication-name + "with improvement of " + DDSS therapy "lock" + DDSS therapy

I observed that there were many cases in which DDSSs were concatenated in one sentence. These DDSSs could not be matched by the patterns provided in Table 9. For example, “Zofran scheduled q8 for nausea/retching/vomiting, Ibuprofen or Morphine PRN pain”. In this sentence, the DDSS “vomiting” could not be matched if I used the pattern of “for vomiting”. To deal with these cases (key words + reaction synonyms concatenated with symbol “/”), I developed a regular expression rule ("\\Wfor

(\\w+/)*" + DDSS + "(/\\w+)*\\W") to match them. Another example is “With continued nausea and vomiting in room, unable to tolerate po ibuprofen”, where “continued nausea and vomiting” (or “nausea or vomiting”) could be matched with the regular expression "\\W(\\w+\\sand\\s)? " + DDSS + "\\W" (or

"\\W(\\w+\\sor\\s)? " + DDSS + "\\W"). These two regular expression rules were applied to all the patterns in Table 10 to assist excluding the sentences.

Finally, I used the Java implementation (GeneralNegEx) (Solti & Kye, 2010) of the NegEx algorithm

( Chapman et al., 2013) to identify and rule out DRPSs with negated medical conditions. The NegEx algorithm identified negated conditions by applying regular expressions over DDSSs. However, phrases of concatenated DDSSs (e.g., “nausea/vomiting”) could not be processed by NegEx. For instance, NegEx identified the term “vomiting” in “patient has no vomiting” as negated. However, it identified the same term in “patient has no nausea/vomiting” as affirmed. I supplemented the GeneralNegEx by adding regular expression rules (described in previous paragraph) to avoid this type of error.

4.3 Identifying ADR-related DRPSs

This section addresses how I identified ADR-related DRPSs in the EHR data (section 4.3.2). ADR time window (4.3.1) and threshold of reactions for a medication (section 4.3.3) were used to detect the ADR- related DRPSs.

4.3.1 ADR time window

Time windows are important because an ADR happens within a specific time period after the medication is given. If I used all notes created during the entire encounter to detect an ADR for a medication, I might capture wrong signals and cause false positive alerts. For example, I might capture something before the medication administration which was not an ADR. Different ADRs could have different ADR time windows even for the same medication. As a proof of concept, I used a flexible time window for all medications. That is, I only analyzed the clinical notes that were written between the medication was given and the patient discharge.

4.3.2 Identifying ADR-related DRPSs

Figure 19 illustrates the pipeline for identifying ADR-related DRPSs in clinical notes. As described in the previous sections, I first searched for FDA drug-reaction pairs in clinical notes to identify DRPSs.

Medication names were mapped to generic names and reactions were mapped to CUIs before ADR matching. I then performed NLP steps (section-based analysis, CUI matching, sentence-based post- processing) on DRPSs to detect ADR-related DRPSs.

Figure 19: Pipeline for identifying ADR-related DRPSs

I identified 1055 encounters with ADR-related DRPS sentences. The descriptive statistics of these ADR- related DRPSs for each medication were presented in Table 11. The number of DRPSs detected for the most frequently used medication “SODIUM CHLORIDE” (which included 1,020,782 clinical notes among 44,499 encounters) was 0. There were 572 DRPSs detected for the medication “VANCOMYCIN” and 556 for “OXYCODONE”. However, the percentage of DRPS per note on “CLINDAMYCIN” was the highest (20.3%) among the 41 most frequently used medications.

Table 11: ADR-related DRPSs identified for each medication

Percent Percentage age of Number of DRPS Numb Numb DRPS Number of of DRPS Encounters Number of er of er of Drug Name Notes Encounters Encount over notes DRPS DRPS over ers Number of Notes s Number Encounters of Notes

SODIUM CHLORIDE 44,499 0 0.00% 1,020,782 0 0.000% 0

ACETAMINOPHEN 34,465 26 0.08% 835,005 58 0.007% 65

HEPARIN 11,312 12 0.11% 565,966 20 0.004% 20

MORPHINE SULFATE 12,892 4 0.03% 469,159 14 0.003% 16

FENTANYL CITRATE 9,754 0 0.00% 380,979 0 0.000% 0

MIDAZOLAM 6,132 3 0.05% 303,594 3 0.001% 3

ONDANSETRON 12,527 1 0.01% 380,549 1 0.000% 1

DIPHENHYDRAMINE 9,319 25 0.27% 348,279 34 0.010% 48

FUROSEMIDE 4,051 8 0.20% 312,921 10 0.003% 10

LORAZEPAM 4,757 8 0.17% 326,046 9 0.003% 10

CEFAZOLIN 4,661 0 0.00% 159,769 0 0.000% 0

OXYCODONE 7,042 227 3.22% 252,209 442 0.175% 556

IBUPROFEN 9,497 76 0.80% 215,687 102 0.047% 136

RANITIDINE 7,955 3 0.04% 318,090 3 0.001% 3

VANCOMYCIN 3,416 224 6.56% 263,477 526 0.200% 572

IOVERSOL 4,942 0 0.00% 300,079 0 0.000% 0

CLINDAMYCIN 4,394 177 4.03% 146,696 298 0.203% 330

INSULIN ASPART 907 0 0.00% 33,805 0 0.000% 0

ALBUMIN 1,938 0 0.00% 199,676 0 0.000% 0 METHYLPREDNISOLON E 3,498 17 0.49% 188,106 33 0.018% 38

LANSOPRAZOLE 7,406 4 0.05% 309,870 7 0.002% 7

HYDROMORPHONE 2,512 1 0.04% 130,507 1 0.001% 1

LIDOCAINE 3,498 3 0.09% 187,038 3 0.002% 3

DIAZEPAM 3,854 4 0.10% 140,971 5 0.004% 6

INSULIN LISPRO 665 1 0.15% 27,560 9 0.033% 9

HYDROCORTISONE 2,821 17 0.60% 217,480 40 0.018% 40

EPINEPHRINE 1,386 7 0.51% 84,209 8 0.010% 8

CEFTRIAXONE 4,144 85 2.05% 135,146 138 0.102% 158

BUPIVACAINE 869 0 0.00% 28,028 0 0.000% 0

KETOROLAC 5,163 0 0.00% 112,979 0 0.000% 0

AMOXICILLIN 5,770 83 1.44% 138,273 156 0.113% 183 LIDOCAINE & PRILOCAINE 4,573 0 0.00% 265,180 0 0.000% 0

METHADONE 855 74 8.65% 123,041 169 0.137% 228

NYSTATIN 4,549 1 0.02% 280,734 1 0.000% 1 PIPERACILLIN & TAZOBACTAM 2,947 1 0.03% 230,557 1 0.000% 1 DEXAMETHASONE SODIUM PHOSPHATE 3,827 0 0.00% 140,019 0 0.000% 0 SODIUM BICARBONATE 1,388 0 0.00% 140,863 0 0.000% 0

KETAMINE 1,200 4 0.33% 96,200 5 0.005% 5 EPINEPHRINE & LIDOCAINE 201 0 0.00% 10,305 0 0.000% 0

LEVETIRACETAM 2,562 3 0.12% 102,182 10 0.010% 12

MORPHINE 612 4 0.65% 56,495 4 0.007% 5

Total (unique) 60,085 1,055 1.76% 1,168,397 2089 0.179% 2475

The descriptive statistics of the ADR-related DRPSs for each note type is presented in Table 12. Take

progress notes as an example, there were a total of 132,650,746 sentences from 857,258 notes in 58,539

encounters. The algorithm detected 1,813 ADR-related DRPSs from 1,522 notes in 691 encounters. The

percentage of ADR-related DRPSs per progress note was 0.001%.

Table 12: ADR-related DRPSs identified for each note type

Percentage Percentag Percentag of DRPS e of e of Number of Numbe Number of Encounter DRPS Numbe DRPS DRPS Number r of Number of Note Type Encounter s over Notes r of over Encounter of notes DRPS Sentences s Number of over DRPSs Number s Notes Encounter Number of s of Notes Sentences Progress Notes 58,539 691 1.18% 857,258 1,522 0.18% 132,650,746 1,813 0.001% ED Notes 28,648 8 0.03% 121,758 8 0.01% 2,139,671 8 0.0004% Discharge Summaries 56,340 360 0.64% 91,511 369 0.40% 11,341,952 432 0.004% H&P Notes 53,071 181 0.34% 97,870 190 0.19% 15,685,367 222 0.001% Total (unique) 60,085 1055 1.76% 1,168,397 2089 0.18% 161,817,736 2475 0.002%

The descriptive statistics of the ADR-related DRPSs for each medication-note type pair is presented in

Table 13. Take Acetaminophen for an example, the algorithm analyzed a total of 660,367 progress notes in 33,792 encounters and detected 49 ADR-related DRPSs from 44 notes.

Table 13: ADR-related DRPSs identified for each medication and each note type

Percentage Percentag Num of DRPS e of Number Number ber Encounter DRPS Drug of of DRPS Number of Number of Note Type s over Notes Name Encounte Encount of notes DRP DRPSs Number of over rs ers S Encounter Number Notes s of Notes Progress Notes 43,745 0 0.000% 774,712 0 0.000% 0 ED Notes SODIUM 23,053 0 0.000% 99,822 0 0.000% 0 CHLORID Discharge E Summaries 42,286 0 0.000% 70,229 0 0.000% 0 H&P Notes 39,770 0 0.000% 76,019 0 0.000% 0 Progress Notes 33,792 14 0.041% 660,367 44 0.007% 49 ED Notes ACETAMI 15,874 1 0.006% 67,935 1 0.001% 1 NOPHEN Discharge Summaries 32,235 5 0.016% 51,547 5 0.010% 5 H&P Notes 30,336 8 0.026% 55,156 8 0.015% 10 Progress Notes 11,045 11 0.100% 509,741 18 0.004% 18 ED Notes 3,564 0 0.000% 15,804 0 0.000% 0 HEPARIN Discharge Summaries 10,745 2 0.019% 19,434 2 0.010% 2 H&P Notes 10,560 0 0.000% 20,987 0 0.000% 0 Progress Notes 12,750 4 0.031% 402,984 11 0.003% 13 ED Notes MORPHIN 6,057 0 0.000% 26,573 0 0.000% 0 E Discharge SULFATE Summaries 12,612 0 0.000% 19,066 0 0.000% 0 H&P Notes 11,199 1 0.009% 20,536 3 0.015% 3 Progress Notes 9,581 0 0.000% 338,781 0 0.000% 0 ED Notes FENTANY 2,941 0 0.000% 12,634 0 0.000% 0 L Discharge CITRATE Summaries 9,425 0 0.000% 13,909 0 0.000% 0 H&P Notes 8,764 0 0.000% 15,655 0 0.000% 0

Progress Notes 5,949 2 0.034% 274,372 2 0.001% 2 ED Notes MIDAZOL 1,805 1 0.055% 8,053 1 0.012% 1 AM Discharge Summaries 5,926 0 0.000% 9,633 0 0.000% 0 H&P Notes 5,751 0 0.000% 11,536 0 0.000% 0 Progress Notes 12,363 0 0.000% 313,580 0 0.000% 0 ED Notes ONDANS 5,592 0 0.000% 25,565 0 0.000% 0 ETRON Discharge Summaries 12,216 1 0.008% 20,261 1 0.005% 1 H&P Notes 11,464 0 0.000% 21,143 0 0.000% 0 Progress Notes 9,124 9 0.099% 304,396 18 0.006% 31 ED Notes DIPHENH 3,564 0 0.000% 15,546 0 0.000% 0 YDRAMI Discharge NE Summaries 7,771 1 0.013% 13,865 1 0.007% 1 H&P Notes 7,680 15 0.195% 14,472 15 0.104% 16 Progress Notes 4,024 6 0.149% 292,764 7 0.002% 7 ED Notes FUROSEM 1,154 0 0.000% 4,807 0 0.000% 0 IDE Discharge Summaries 3,870 2 0.052% 7,206 2 0.028% 2 H&P Notes 3,877 1 0.026% 8,144 1 0.012% 1 Progress Notes 4,741 8 0.169% 299,041 9 0.003% 10 ED Notes LORAZEP 1,980 0 0.000% 9,183 0 0.000% 0 AM Discharge Summaries 4,622 0 0.000% 8,277 0 0.000% 0 H&P Notes 4,584 0 0.000% 9,545 0 0.000% 0 Progress Notes 4,601 0 0.000% 140,900 0 0.000% 0 ED Notes CEFAZOL 958 0 0.000% 4,112 0 0.000% 0 IN Discharge Summaries 4,616 0 0.000% 6,981 0 0.000% 0 H&P Notes 4,242 0 0.000% 7,776 0 0.000% 0 Progress Notes 7,010 184 2.625% 218,061 376 0.172% 483 ED Notes OXYCOD 2,444 0 0.000% 10,904 0 0.000% 0 ONE Discharge Summaries 6,932 44 0.635% 11,150 45 0.404% 52 H&P Notes 6,484 21 0.324% 12,094 21 0.174% 21

Progress Notes 9,344 34 0.364% 156,373 50 0.032% 73 ED Notes IBUPROF 6,341 3 0.047% 26,688 3 0.011% 3 EN Discharge Summaries 9,303 24 0.258% 15,690 24 0.153% 28 H&P Notes 8,851 24 0.271% 16,936 25 0.148% 32 Progress Notes 7,906 1 0.013% 273,308 1 0.000% 1 ED Notes RANITIDI 3,597 0 0.000% 15,976 0 0.000% 0 NE Discharge Summaries 7,758 2 0.026% 13,571 2 0.015% 2 H&P Notes 7,636 0 0.000% 15,235 0 0.000% 0 Progress Notes 3,409 144 4.224% 242,375 392 0.162% 429 ED Notes VANCOM 1,831 2 0.109% 8,091 2 0.025% 2 YCIN Discharge Summaries 3,402 97 2.851% 6,165 97 1.573% 102 H&P Notes 3,283 32 0.975% 6,846 35 0.511% 39 Progress Notes 4,785 0 0.000% 267,916 0 0.000% 0 ED Notes IOVERSO 2,859 0 0.000% 14,389 0 0.000% 0 L Discharge Summaries 4,845 0 0.000% 8,233 0 0.000% 0 H&P Notes 4,526 0 0.000% 9,541 0 0.000% 0 Progress Notes 4,328 98 2.264% 120,209 178 0.148% 190 ED Notes CLINDAM 2,939 0 0.000% 11,165 0 0.000% 0 YCIN Discharge Summaries 4,364 85 1.948% 7,348 88 1.198% 100 H&P Notes 4,081 32 0.784% 7,974 32 0.401% 40 Progress Notes 906 0 0.000% 26,940 0 0.000% 0 ED Notes INSULIN 563 0 0.000% 2,689 0 0.000% 0 ASPART Discharge Summaries 891 0 0.000% 1,986 0 0.000% 0 H&P Notes 868 0 0.000% 2,190 0 0.000% 0 Progress Notes 1,928 0 0.000% 189,422 0 0.000% 0 ED Notes 661 0 0.000% 2,742 0 0.000% 0 ALBUMIN Discharge Summaries 1,784 0 0.000% 3,359 0 0.000% 0 H&P Notes 1,861 0 0.000% 4,153 0 0.000% 0

Progress Notes 3,470 13 0.375% 169,943 27 0.016% 31 ED Notes METHYLP 1,525 0 0.000% 6,835 0 0.000% 0 REDNISO Discharge LONE Summaries 2,878 4 0.139% 5,330 5 0.094% 6 H&P Notes 2,884 1 0.035% 5,998 1 0.017% 1 Progress Notes 7,346 4 0.054% 272,625 7 0.003% 7 ED Notes LANSOPR 2,691 0 0.000% 12,165 0 0.000% 0 AZOLE Discharge Summaries 7,075 0 0.000% 12,050 0 0.000% 0 H&P Notes 6,811 0 0.000% 13,030 0 0.000% 0 Progress Notes 2,498 1 0.040% 118,165 1 0.001% 1 ED Notes HYDROM 893 0 0.000% 4,173 0 0.000% 0 ORPHONE Discharge Summaries 2,459 0 0.000% 3,963 0 0.000% 0 H&P Notes 2,258 0 0.000% 4,206 0 0.000% 0 Progress Notes 3,380 1 0.030% 167,695 1 0.001% 1 ED Notes LIDOCAI 1,609 1 0.062% 6,993 1 0.014% 1 NE Discharge Summaries 3,278 1 0.031% 5,798 1 0.017% 1 H&P Notes 3,231 0 0.000% 6,552 0 0.000% 0 Progress Notes 3,837 3 0.078% 124,264 4 0.003% 4 ED Notes DIAZEPA 1,016 0 0.000% 4,818 0 0.000% 0 M Discharge Summaries 3,746 1 0.027% 5,649 1 0.018% 2 H&P Notes 3,435 0 0.000% 6,240 0 0.000% 0 Progress Notes 665 1 0.150% 22,861 9 0.039% 9 ED Notes INSULIN 439 0 0.000% 1,879 0 0.000% 0 LISPRO Discharge Summaries 639 0 0.000% 1,226 0 0.000% 0 H&P Notes 651 0 0.000% 1,594 0 0.000% 0 Progress Notes 2,793 11 0.394% 202,310 32 0.016% 32 ED Notes HYDROC 1,137 0 0.000% 4,823 0 0.000% 0 ORTISON Discharge E Summaries 2,701 4 0.148% 4,917 4 0.081% 4 H&P Notes 2,685 4 0.149% 5,430 4 0.074% 4

Progress Notes 1,359 4 0.294% 75,551 4 0.005% 4 ED Notes EPINEPH 832 0 0.000% 3,444 0 0.000% 0 RINE Discharge Summaries 1,371 2 0.146% 2,338 2 0.086% 2 H&P Notes 1,354 2 0.148% 2,876 2 0.070% 2 Progress Notes 4,096 49 1.196% 105,653 90 0.085% 103 ED Notes CEFTRIA 3,209 0 0.000% 14,064 0 0.000% 0 XONE Discharge Summaries 4,137 31 0.749% 7,394 31 0.419% 36 H&P Notes 4,038 17 0.421% 8,035 17 0.212% 19 Progress Notes 698 0 0.000% 24,433 0 0.000% 0 ED Notes BUPIVAC 166 0 0.000% 684 0 0.000% 0 AINE Discharge Summaries 840 0 0.000% 1,475 0 0.000% 0 H&P Notes 809 0 0.000% 1,436 0 0.000% 0 Progress Notes 5,106 0 0.000% 85,382 0 0.000% 0 ED Notes KETOROL 2,736 0 0.000% 11,929 0 0.000% 0 AC Discharge Summaries 4,996 0 0.000% 7,654 0 0.000% 0 H&P Notes 4,507 0 0.000% 8,014 0 0.000% 0 Progress Notes 5,662 54 0.954% 109,592 106 0.097% 120 ED Notes AMOXICI 2,708 0 0.000% 10,942 0 0.000% 0 LLIN Discharge Summaries 5,632 36 0.639% 8,573 38 0.443% 49 H&P Notes 5,380 12 0.223% 9,166 12 0.131% 14 Progress Notes 4,540 0 0.000% 238,731 0 0.000% 0 LIDOCAI ED Notes NE & 2,189 0 0.000% 10,294 0 0.000% 0 PRILOCAI Discharge NE Summaries 4,257 0 0.000% 7,483 0 0.000% 0 H&P Notes 4,189 0 0.000% 8,672 0 0.000% 0 Progress Notes 853 57 6.682% 118,478 135 0.114% 178 ED Notes METHAD 290 0 0.000% 1,136 0 0.000% 0 ONE Discharge Summaries 819 22 2.686% 1,487 23 1.547% 34 H&P Notes 818 9 1.100% 1,940 11 0.567% 16

Progress Notes 4,524 1 0.022% 255,829 1 0.000% 1 ED Notes NYSTATI 1,824 0 0.000% 8,055 0 0.000% 0 N Discharge Summaries 4,428 0 0.000% 8,186 0 0.000% 0 H&P Notes 4,393 0 0.000% 8,664 0 0.000% 0 Progress Notes 2,944 1 0.034% 212,691 1 0.000% 1 PIPERACI ED Notes LLIN & 1,554 0 0.000% 6,892 0 0.000% 0 TAZOBAC Discharge TAM Summaries 2,939 0 0.000% 5,259 0 0.000% 0 H&P Notes 2,776 0 0.000% 5,715 0 0.000% 0 Progress Notes 3,752 0 0.000% 124,757 0 0.000% 0 DEXAME ED Notes THASONE 1,026 0 0.000% 4,126 0 0.000% 0 SODIUM Discharge PHOSPHA Summaries 3,696 0 0.000% 5,245 0 0.000% 0 TE H&P Notes 3,463 0 0.000% 5,891 0 0.000% 0 Progress Notes 1,386 0 0.000% 133,761 0 0.000% 0 ED Notes SODIUM 425 0 0.000% 1,680 0 0.000% 0 BICARBO Discharge NATE Summaries 1,330 0 0.000% 2,479 0 0.000% 0 H&P Notes 1,344 0 0.000% 2,943 0 0.000% 0 Progress Notes 1,183 2 0.169% 88,073 2 0.002% 2 ED Notes KETAMIN 838 0 0.000% 4,045 0 0.000% 0 E Discharge Summaries 1,160 0 0.000% 1,909 0 0.000% 0 H&P Notes 1,069 3 0.281% 2,173 3 0.138% 3 Progress Notes 198 0 0.000% 9,125 0 0.000% 0 EPINEPH ED Notes RINE & 124 0 0.000% 515 0 0.000% 0 LIDOCAI Discharge NE Summaries 195 0 0.000% 315 0 0.000% 0 H&P Notes 181 0 0.000% 350 0 0.000% 0 Progress Notes 2,525 3 0.119% 86,890 8 0.009% 10 ED Notes LEVETIR 1,216 0 0.000% 5,870 0 0.000% 0 ACETAM Discharge Summaries 2,461 1 0.041% 4,250 1 0.024% 1 H&P Notes 2,430 1 0.041% 5,172 1 0.019% 1

Progress Notes 612 2 0.327% 53,969 2 0.004% 3 ED Notes MORPHIN 92 0 0.000% 380 0 0.000% 0 E Discharge Summaries 596 2 0.336% 966 2 0.207% 2 H&P Notes 571 0 0.000% 1,180 0 0.000% 0

Total (unique) 60,085 1055 1.756% 1,168,397 2089 0.179% 2475

4.3.3 Thresholding reactions for a medication

A total of 2475 ADR-related DRPSs from 2089 notes (1055 encounters) were detected. Before manual review I proposed an assumption that certain drug-reaction pairs in the FDA reports were more likely to be ADRs. The possibility is estimated by the conditional probability of a reaction given a medication. For instance, AGITATION was reported as a reaction after taking OXYCODONE, but it was only reported in

2% of the FDA reports with OXYCODONE. In contrast, VOMITING was reported in 6.8% of the FDA reports with OXYCODONE. The statistics implied that a DRPS with both VOMITING and

OXYCODONE was more likely to be an ADR compared with a DRPS with AGITATION and

OXYCODONE. Based on this hypothesis, I counted and plotted a curve of the numbers of ADR-related

DRPSs (see Table 14 and Figure 20) for different cut-off percentages on the drug-reaction-pair distribution. The cut-off percentage was calculated by dividing the number of the drug-reaction pairs with the number of the cases related to this drug in the FDA reports (equation 1).

푛푢푚푏푒푟 표푓 푑푟푢푔−푟푒푎푐푡𝑖표푛 푝푎𝑖푟푠 푐푢푡 − 표푓푓 푝푒푟푐푛푡푒푎푔푒 = × 100 (1) 푛푢푚푏푒푟 표푓 푡ℎ푒 푐푎푠푒푠 푟푒푙푎푡푒푑 푡표 푡ℎ𝑖푠 푑푟푢푔

Table 14: Number of ADR-related DRPSs at each cut-off percentage

Number of Number of ADR- Cut-off Number of ADR-related Number of ADR- related DRPS Percentage (%) DRP pairs DRPS related DRPSs Notes Encounters

100 6 1 1 1

67 6 1 1 1

66 7 1 1 1

58 7 1 1 1

57 9 1 1 1

51 9 1 1 1

50 26 1 1 1

43 26 1 1 1

42 31 1 1 1

40 31 1 1 1

39 32 1 1 1

34 32 1 1 1

33 34 9 13 14

32 36 9 13 14

31 36 9 13 14

30 37 9 13 14

29 37 9 13 14

28 57 13 18 19

26 57 13 18 19

25 77 14 19 20

24 77 14 19 20

23 83 14 19 20

21 83 14 19 20

20 103 16 21 22

19 110 63 169 172

18 112 63 169 172

17 115 81 220 225

16 142 82 221 226 15 161 83 222 227

14 252 96 248 254

13 258 103 258 265

12 270 108 267 275

11 298 109 268 276

10 388 146 311 320

9 438 178 362 378

8 477 221 440 458

7 522 228 447 470

6 656 500 930 1001

5 829 559 1044 1124

4 988 776 1506 1661

3 1147 843 1703 1920

2 1445 916 1816 2113

1 2006 1004 1980 2335

0 2646 1055 2089 2475

Figure 20: Plots for ADR-related DRPSs at each cut-off percentage

4.4 Identifying baseline ADR-related DRPSs

In order to verify the null hypothesis of “leveraging drug-reaction pairs from the FAERS will not improve the performance of EHR-based ADR detection”, I generated a set of baseline ADR-related DRPSs, which included both medication name and DDSS term in each sentence. Instead of leveraging FDA ADE reports for detecting ADRs, I used a list of CUI represented DDSS set (which from a gold standard annotation for another project(Louise Deleger et al., 2012)) as candidate reactions that might be caused by each medication. Figure 21 shows workflow and steps I used to generate the baseline ADR-related DRPSs.

The workflow for generating baseline ADR-related DRPSs was similar as workflow of generating ADR- related DRPSs described in previous sections, except that I used an existing DDSS CUI set for each medication as candidate drug-reaction pairs for baseline ADR-related DRPSs identification, other than leveraging FDA ADE reports as candidate drug-reaction pairs.

Figure 21: Pipeline for identifying baseline ADR-related DRPSs

The baseline reaction set included 4345 CUI represented DDSSs. As a comparison with the 29 medications which had identified ADR-related ADRs, I identified baseline ADR-related DRPSs for the

29 medications. The combination of 29 medications and 4345 DDSSs generated a total number of

126,005 (4345 multiply 29) baseline candidate drug-reaction pairs. Finally, a total of 93,598 baseline

ADR-related DRPSs were detected for the 29 medications. The descriptive statistics of these baseline

ADR-related DRPSs for each medication were presented in Table 15. Take Sodium Chloride for an example, after analyzing 1,020,782 clinical notes from 44,499 encounters, the baseline algorithm detected

8243 ADR-related DRPSs in 7683 clinical notes from 843 encounters.

Table 15: Baseline ADR-related DRPSs identified for each medication

Percent Percentage Number Numb age of Numb of Baseline Number of er of Baseline er of DRPS of Baseline Number of Baseli DRPS Baseli Drug Name Encounters Encounte DRPS notes ne Notes ne over rs Encount DRPS over DRPS Number of ers Notes Number s Encounters of Notes

SODIUM CHLORIDE 44,499 843 1.89% 1,020,782 7683 0.753% 8243

ACETAMINOPHEN 34,465 217 0.63% 835,005 321 0.038% 456

HEPARIN 11,312 1071 9.47% 565,966 4270 0.754% 6843

MORPHINE SULFATE 12,892 18 0.14% 469,159 68 0.014% 110

FENTANYL CITRATE 9,754 0 0.00% 380,979 0 0.000% 0

MIDAZOLAM 6,132 23 0.38% 303,594 34 0.011% 55

ONDANSETRON 12,527 81 0.65% 380,549 252 0.066% 337

DIPHENHYDRAMINE 9,319 123 1.32% 348,279 170 0.049% 215

FUROSEMIDE 4,051 35 0.86% 312,921 63 0.020% 78

LORAZEPAM 4,757 148 3.11% 326,046 398 0.122% 488

CEFAZOLIN 4,661 82 1.76% 159,769 210 0.131% 253

OXYCODONE 7,042 2150 30.53% 252,209 5877 2.330% 8484

IBUPROFEN 9,497 1043 10.98% 215,687 1725 0.800% 2496

RANITIDINE 7,955 85 1.07% 318,090 299 0.094% 445

VANCOMYCIN 3,416 1855 54.30% 263,477 7973 3.026% 13981

IOVERSOL 4,942 7 0.14% 300,079 7 0.002% 7

CLINDAMYCIN 4,394 1913 43.54% 146,696 4683 3.192% 8634

INSULIN ASPART 907 3 0.33% 33,805 3 0.009% 8

ALBUMIN 1,938 713 36.79% 199,676 2935 1.470% 7756

METHYLPREDNISOLONE 3,498 267 7.63% 188,106 1293 0.687% 1931

LANSOPRAZOLE 7,406 35 0.47% 309,870 111 0.036% 132

HYDROMORPHONE 2,512 66 2.63% 130,507 172 0.132% 193

LIDOCAINE 3,498 166 4.75% 187,038 1032 0.552% 1438

DIAZEPAM 3,854 145 3.76% 140,971 254 0.180% 379

INSULIN LISPRO 665 3 0.45% 27,560 11 0.040% 11

HYDROCORTISONE 2,821 951 33.71% 217,480 4797 2.206% 6849

EPINEPHRINE 1,386 356 25.69% 84,209 641 0.761% 1069

CEFTRIAXONE 4,144 1049 25.31% 135,146 2087 1.544% 3482

BUPIVACAINE 869 0 0.00% 28,028 0 0.000% 0

KETOROLAC 5,163 53 1.03% 112,979 65 0.058% 88

AMOXICILLIN 5,770 772 13.38% 138,273 1497 1.083% 2865 LIDOCAINE & PRILOCAINE 4,573 2 4.37E-04 265,180 2 0.001% 3

METHADONE 855 524 61.29% 123,041 3618 2.940% 5248

NYSTATIN 4,549 1176 25.85% 280,734 5346 1.904% 8074 PIPERACILLIN & TAZOBACTAM 2,947 60 2.04% 230,557 127 0.055% 193 DEXAMETHASONE SODIUM PHOSPHATE 3,827 0 0.00% 140,019 0 0.000% 0

SODIUM BICARBONATE 1,388 58 4.18% 140,863 112 0.080% 169

KETAMINE 1,200 151 12.58% 96,200 502 0.522% 840 EPINEPHRINE & LIDOCAINE 201 2 1.00% 10,305 2 0.019% 3

LEVETIRACETAM 2,562 27 1.05% 102,182 37 0.036% 45

MORPHINE 612 324 52.94% 56,495 1233 2.182% 1697

Total (unique) 60,085 10,874 18.10% 1,168,397 51445 4.403% 93599

5 Results

5.1 Evaluation metrics

Given the ADR-related DRPSs predicted by the algorithms, one clinical informatics expert with a medical degree and I performed manual chart review independently to assess the numbers of true positive (TP) and false positive (FP) in the predictions. The inter reviewer agreement was 83.7%. The positive predictive value PPV=TP/(TP+FP) as well as p-value and confidence interval were then used as the evaluation of the study. The p-value and confidence interval for PPV of algorithms are commonly computed by z-test (or t-test if the number of samples was less than 30). The output of algorithms is viewed as a data set, which are independently, identically sampled from a binomial distribution with probability p=ppv. According to the central limit theorem, p (ppv) can be computed by normal distribution as long as n is sufficiently large (adjusted normal distribution can be used for small n).

5.2 Performance of ADR detection with FAERS reports

The algorithm identified 2475 ADR-related DRPSs, in which there were total of 1492 unique ADR- related DRPSs. I randomly sampled 528 (35.4% of unique cases) ADR-related DRPSs for manual review.

Table 16 -Table 19 show the review results. In the tables, I excluded the 12 medications that did not have

ADR-related DRPSs. For the remaining 29 medications I counted the number of TP ADRs, FP ADRs for each medication (Table 16), each note type (Table 17), each cut-off threshold (Table 18) and each medication-note type combination (Table 19). All identified DRPSs were predicted as positive sentences by the algorithm. If the sentence mentioned an ADR, it was a TP sentence; otherwise it was a FP.

Table 16 shows the TP, FP and PPV for each medication. Two hundred and twenty-four identified DRPSs were marked as TP ADR sentences (PPV=42.4%, 95% CI = 42.4±4.2). Two medications achieved a high

PPV of greater than or equal to 70%: detecting MORPHINE SULFATE achieved a PPV of 75% and detecting LORAZEPAM achieved a PPV of 70%. Total of eight medications had their PPV greater than

60 or equal to 60%, while total of eleven medications had their PPVs no less than 40%. Eight medications had their PPV equal to 0 since no TP cases were found among the reviewed ADR-related DRPSs.

However, the total number of the reviewed DRPSs related to the 0-PPV medications was less than ten.

The PPVs for other eleven medications were between 3.6% and 37.5%.

Table 16: Results for each medication

Number of Reviewed Drug Name Number of TP Number of FP PPV ADR-related DRPSs MORPHINE SULFATE 4 3 1 75.0%

LORAZEPAM 10 7 3 70.0%

MIDAZOLAM 3 2 1 66.7%

FUROSEMIDE 9 6 3 66.7%

OXYCODONE 135 86 49 63.7%

ACETAMINOPHEN 27 17 10 63.0%

DIAZEPAM 5 3 2 60.0%

MORPHINE 5 3 2 60.0%

METHADONE 47 21 26 44.7%

CLINDAMYCIN 45 20 25 44.4%

VANCOMYCIN 48 21 27 43.8%

HEPARIN 16 6 10 37.5%

LIDOCAINE 3 1 2 33.3%

LEVETIRACETAM 6 2 4 33.3%

AMOXICILLIN 29 9 20 31.0%

HYDROCORTISONE 10 3 7 30.0%

CEFTRIAXONE 26 6 20 23.1%

DIPHENHYDRAMINE 19 4 15 21.1%

KETAMINE 5 1 4 20.0% METHYLPREDNISOL ONE 26 2 24 7.7%

IBUPROFEN 28 1 27 3.6%

ONDANSETRON 1 0 1 0.0%

RANITIDINE 3 0 3 0.0%

LANSOPRAZOLE 6 0 6 0.0%

HYDROMORPHONE 1 0 1 0.0%

INSULIN LISPRO 2 0 2 0.0%

EPINEPHRINE 7 0 7 0.0%

NYSTATIN 1 0 1 0.0% PIPERACILLIN&TAZ OBACTAM 1 0 1 0.0%

Total 528 224 304 42.4%

Table 17 shows the results based on note types. ADE-related DRPSs detected from ED Notes had the highest PPV (80%). The DRPSs detected from Progress notes, Discharge summaries and H & P notes had

PPVs of 45.8%, 38.1% and 25.9%, respectively.

Table 17: Results for each note type

Number of Reviewed Note Type Number of TP Number of FP PPV ADR-related DRPSs Progress Notes 360 165 195 45.8%

ED Notes 5 4 1 80.0%

Discharge Summaries 105 40 65 38.1%

H&P Notes 58 15 43 25.9%

Total 528 224 304 42.4%

Table 18 shows the PPVs for each cut-off percentage. The 15% and 25% thresholds achieved higher

PPVs of 69.6% and 66.7% respectively.

Table 18: Results for each cut-off percentage

Cut-off Number of Reviewed Number of TP Number of FP PPV Percentage ADR-related DRPSs 40% 1 0 1 0.0% 35% 1 0 1 0.0%

30% 1 0 1 0.0%

25% 6 4 2 66.7%

20% 8 4 4 50.0%

15% 23 16 7 69.6%

10% 59 33 26 55.9%

5% 236 93 143 39.4%

0% 528 224 304 42.4%

Table 19 shows the detailed results for each medication-note type pair. There were nine medication-note type pairs achieving 100% PPVs. The pairs were: ACETAMINOPHEN, MIDAZOLAM,

VANCOMYCIN and LIDOCAINE in the ED notes; HEPARIN, DIPHENHYDRAMINE,

FUROSEMIDE and MORPHINE in the discharge summaries; and METHADONE in the H&P notes.

Among the total of 116 medication-note type pairs, 48 pairs had no identified ADR-related DRPSs.

Table 19: Results for each medication in each note type

Number of Number Number Drug Name Note Type Reviewed ADR- PPV of TP of FP related DRPSs

Progress Notes 15 10 5 66.7%

ED Notes 1 1 0 100.0% ACETAMINOPHEN Discharge Summaries 5 2 3 40.0%

H&P Notes 6 4 2 66.7%

Progress Notes 14 4 10 28.6%

ED Notes 0 0 0 N/A HEPARIN Discharge Summaries 2 2 0 100.0%

H&P Notes 0 0 0 N/A

Progress Notes 4 3 1 75.0%

MORPHINE ED Notes 0 0 0 N/A SULFATE Discharge Summaries 0 0 0 N/A

H&P Notes 0 0 0 N/A

Progress Notes 2 1 1 50.0%

ED Notes 1 1 0 100.0% MIDAZOLAM Discharge Summaries 0 0 0 N/A

H&P Notes 0 0 0 N/A

Progress Notes 0 0 0 N/A

ED Notes 0 0 0 N/A ONDANSETRON Discharge Summaries 1 0 1 0.0%

H&P Notes 0 0 0 N/A

Progress Notes 8 3 5 37.5%

DIPHENHYDRAMI ED Notes 0 0 0 N/A NE Discharge Summaries 1 1 0 100.0%

H&P Notes 10 0 10 0.0%

Progress Notes 7 5 2 71.4%

ED Notes 0 0 0 N/A FUROSEMIDE Discharge Summaries 1 1 0 100.0%

H&P Notes 1 0 1 0.0%

Progress Notes 10 7 3 70.0%

ED Notes 0 0 0 N/A LORAZEPAM Discharge Summaries 0 0 0 N/A

H&P Notes 0 0 0 N/A

Progress Notes 109 67 42 61.5%

ED Notes 0 0 0 N/A OXYCODONE Discharge Summaries 19 13 6 68.4%

H&P Notes 7 6 1 85.7%

Progress Notes 12 1 11 8.3%

ED Notes 1 0 1 0.0% IBUPROFEN Discharge Summaries 4 0 4 0.0%

H&P Notes 11 0 11 0.0% RANITIDINE Progress Notes 1 0 1 0.0%

ED Notes 0 0 0 N/A

Discharge Summaries 2 0 2 0.0%

H&P Notes 0 0 0 N/A

Progress Notes 27 15 12 55.6%

ED Notes 1 1 0 100.0% VANCOMYCIN Discharge Summaries 14 3 11 21.4%

H&P Notes 6 2 4 33.3%

Progress Notes 28 14 14 50.0%

ED Notes 0 0 0 N/A CLINDAMYCIN Discharge Summaries 15 6 9 40.0%

H&P Notes 2 0 2 0.0%

Progress Notes 20 1 19 5.0%

METHYLPREDNIS ED Notes 0 0 0 N/A OLONE Discharge Summaries 5 1 4 20.0%

H&P Notes 1 0 1 0.0%

Progress Notes 6 0 6 0.0%

ED Notes 0 0 0 N/A LANSOPRAZOLE Discharge Summaries 0 0 0 N/A

H&P Notes 0 0 0 N/A

Progress Notes 1 0 1 0.0%

HYDROMORPHON ED Notes 0 0 0 N/A E Discharge Summaries 0 0 0 N/A

H&P Notes 0 0 0 N/A

Progress Notes 1 0 1 0.0%

ED Notes 1 1 0 100.0% LIDOCAINE Discharge Summaries 1 0 1 0.0%

H&P Notes 0 0 0 N/A

Progress Notes 4 3 1 75.0% DIAZEPAM ED Notes 0 0 0 N/A

Discharge Summaries 1 0 1 0.0%

H&P Notes 0 0 0 N/A

Progress Notes 2 0 2 0.0%

ED Notes 0 0 0 N/A INSULIN LISPRO Discharge Summaries 0 0 0 N/A

H&P Notes 0 0 0 N/A

Progress Notes 7 2 5 28.6%

HYDROCORTISON ED Notes 0 0 0 N/A E Discharge Summaries 2 1 1 50.0%

H&P Notes 1 0 1 0.0%

Progress Notes 4 0 4 0.0%

ED Notes 0 0 0 N/A EPINEPHRINE Discharge Summaries 1 0 1 0.0%

H&P Notes 2 0 2 0.0%

Progress Notes 12 3 9 25.0%

ED Notes 0 0 0 N/A CEFTRIAXONE Discharge Summaries 10 2 8 20.0%

H&P Notes 4 1 3 25.0%

Progress Notes 19 6 13 31.6%

ED Notes 0 0 0 N/A AMOXICILLIN Discharge Summaries 8 3 5 37.5%

H&P Notes 2 0 2 0.0%

Progress Notes 36 17 19 47.2%

ED Notes 0 0 0 N/A METHADONE Discharge Summaries 10 3 7 30.0%

H&P Notes 1 1 0 100.0%

Progress Notes 1 0 1 0.0%

NYSTATIN ED Notes 0 0 0 N/A Discharge Summaries 0 0 0 N/A

H&P Notes 0 0 0 N/A

Progress Notes 1 0 1 0.0%

PIPERACILLIN & ED Notes 0 0 0 N/A TAZOBACTAM Discharge Summaries 0 0 0 N/A

H&P Notes 0 0 0 N/A

Progress Notes 2 0 2 0.0%

ED Notes 0 0 0 N/A KETAMINE Discharge Summaries 0 0 0 N/A

H&P Notes 3 1 2 33.3%

Progress Notes 4 2 2 50.0%

ED Notes 0 0 0 N/A LEVETIRACETAM Discharge Summaries 1 0 1 0.0%

H&P Notes 1 0 1 0.0%

Progress Notes 3 1 2 33.3%

ED Notes 0 0 0 N/A MORPHINE Discharge Summaries 2 2 0 100.0%

H&P Notes 0 0 0 N/A

Total 528 224 304 42.4%

Table 20 show the number of medication-note type pairs and the corresponding percentages at each PPV level. For example, 14.7% (10 pairs out of 68) of the pairs had PPVs equal or above 80%; 20.6% (14 out of 68) had PPVs equal or greater than 70%.

Table 20: Number of medication-note type pairs and the corresponding percentages at each PPV level

Number of PPV medication-note Percentage type pairs >=0% 68 100.0% >=10% 42 61.8%

>=20% 40 58.8%

>=30% 33 48.5%

>=40% 26 38.2%

>=50% 23 33.8%

>=60% 18 26.5%

>=70% 14 20.6%

>=80% 10 14.7%

>=90% 9 13.2%

>=100% 9 13.2%

Table 21 lists the detected ADRs for each medication and their frequencies counted from the 224 TP cases of the review set. For example, for ACETAMINOPHEN the most frequent ADR identified in the reviewed set was VOMITING. It was detected in six ADR-related DRPSs; the second most frequent

ADR for ACETAMINOPHEN was NAUSEA, which was detected in three ADR-related DRPSs.

Table 21: Identified ADRs and their frequencies

Drug Name Reactions and their Frequencies VOMITING:6; NAUSEA:3; PRURITUS:2; CHOLESTASIS:1; ACETAMINOPHEN HEPATOTOXICITY:1; INFECTION:1; URTICARIA:1; HEART RATE INCREASED:1; RESPIRATORY DISTRESS:1; AMOXICILLIN RASH:4; NAUSEA:3; VOMITING:2;

CEFTRIAXONE PRURITUS:3; CHILLS:1; CYANOSIS:1; VOMITING:1;

CLINDAMYCIN EXANTHEM:15; HYPERSENSITIVITY:2; VOMITING:2; NAUSEA:1;

DIAZEPAM APNOEA:2; RESPIRATORY DEPRESSION:1; DIPHENHYDRAMI AGITATION:2; ERYTHEMA:1; HEART RATE INCREASED:1; NE HYPOKALAEMIA:3; DIZZINESS:1; HEART RATE INCREASED:1; FUROSEMIDE METABOLIC ALKALOSIS:1; EPISTAXIS:1; GASTROINTESTINAL HAEMORRHAGE:2; HEPARIN HAEMORRHAGE INTRACRANIAL:1; PLEURAL EFFUSION:1; RETROPERITONEAL HAEMORRHAGE:1; HYDROCORTISON HYPERTENSION:3; E

IBUPROFEN VOMITING:1;

KETAMINE VOMITING:1;

LEVETIRACETAM AGITATION:1; INSOMNIA:1;

LIDOCAINE HYPOAESTHESIA:1;

LORAZEPAM SOMNOLENCE:5; HALLUCINATION:1; RESPIRATORY ARREST:1; SOMNOLENCE:7; VOMITING:7; APNOEA:3; BRADYCARDIA:1; METHADONE FATIGUE:1; BLOOD PRESSURE DECREASED:1; MUSCLE RIGIDITY:1; METHYLPREDNIS VOMITING:2; OLONE MIDAZOLAM URTICARIA:1; RESPIRATORY DEPRESSION:1;

MORPHINE HYPOTENSION:1; SOMNOLENCE:1; HEART RATE DECREASED:1; MORPHINE DEPRESSED MOOD:1; NAUSEA:1; PRURITUS:1; SULFATE PRURITUS:28; VOMITING:18; NAUSEA:17; CONSTIPATION:9; SOMNOLENCE:5; ANXIETY:1; APNOEA:1; CONVULSION:1; OXYCODONE DIZZINESS:1; DYSPNOEA:1; SEDATION:1; CONFUSIONAL STATE:2; RESPIRATORY DEPRESSION:1; PRURITUS:10; HYPERSENSITIVITY:5; NAUSEA:1; RED MAN VANCOMYCIN SYNDROME:5;

5.3 Performance of ADR detection using the baseline

The algorithm identified 93,598 baseline ADR-related DRPSs, in which there were total of 32,778 unique baseline ADR-related DRPSs. I randomly sampled 528 baseline ADR-related DRPSs from the unique baseline ADR-related DRPSs for manual review. For each medication, the number of review cases sampled from baseline set was the same as the number of reviewed cases sampled from the identified

ADR-related DRPSs by leveraging FDA ADE reports. Based on my review, total of 46 out of 528 sentences were TPs. The PPV for baseline review set based on my review was 8.7% (95% CI = 8.7±2.4).

Table 22 presents the descriptive statistics of the PPVs for each medication.

Table 22: Baseline review results for each medication

Number of Drug Name Reviewed Baseline Number of TP Number of FP PPV ADR-related DRPSs

MORPHINE SULFATE 27 4 23 14.8%

LORAZEPAM 16 2 14 12.5%

MIDAZOLAM 4 0 4 0.0%

FUROSEMIDE 3 1 2 33.3%

OXYCODONE 1 0 1 0.0%

ACETAMINOPHEN 19 3 16 15.8%

DIAZEPAM 9 4 5 44.4%

MORPHINE 10 0 10 0.0%

METHADONE 135 6 129 4.4%

CLINDAMYCIN 28 1 27 3.6%

VANCOMYCIN 3 0 3 0.0%

HEPARIN 48 7 41 14.6%

LIDOCAINE 45 4 41 8.9%

LEVETIRACETAM 26 3 23 11.5%

AMOXICILLIN 6 0 6 0.0%

HYDROCORTISONE 1 0 1 0.0%

CEFTRIAXONE 3 0 3 0.0%

DIPHENHYDRAMINE 5 0 5 0.0%

KETAMINE 2 0 2 0.0%

METHYLPREDNISOLONE 10 0 10 0.0%

IBUPROFEN 7 0 7 0.0%

ONDANSETRON 26 0 26 0.0%

RANITIDINE 29 2 27 6.9%

LANSOPRAZOLE 47 6 41 12.8%

HYDROMORPHONE 1 0 1 0.0%

INSULIN LISPRO 1 0 1 0.0%

EPINEPHRINE 5 2 3 40.0%

NYSTATIN 6 1 5 16.7% PIPERACILLIN&TAZOBA CTAM 5 0 5 0.0%

Total 528 46 482 8.7%

6 Discussion

By leveraging the FDA ADR reports, I developed an NLP-based algorithm to automatically detect adverse drug reactions retrospectively in the 2010-2012 EHR data of Cincinnati Children’s Hospital

Medical Center. One objective of the study was to reduce the cost of manual evaluation of clinical notes.

Among the total of 60,085 encounters that contained 1,168,397 clinical notes and 161,817,736 sentences, my algorithm identified a total of 2,475 sentences (from 2,089 notes, 1,055 encounters) as positive ADR cases.

The use of the FDA reported drug-reaction pairs for detecting ADRs in the EHR data is novel and the results are promising. The performance of ADR identification leveraging FDA ADE reports (PPV =

42.4%) was significantly better than not using FDA ADE reports (PPV = 8.7%), the p-value was 3.67 ×

10-36. The null hypothesis of “leveraging drug-reaction pairs from the FAERS will not improve the performance of EHR-based ADR detection.” was rejected. The overall PPV is 42.4% and the best PPV is

75% for MORPHINE SULFATE. Although eight medications had PPVs equal to 0%, the numbers of positive cases detected for these medications were rare (less than 10). The assumption that “the reactions with a high cut-off percentage are more likely to be an ADR” was rejected. I postulate one reason could be that some unreasonable candidates in the FDA reports were not excluded due to my limited clinical knowledge. However, the PPV at the 15% cut-off achieved a high PPV of 69.6%. In addition, we could achieve higher PPVs if we apply this method to detect ADRs for certain medications (e.g., MORPHINE

SULFATE and LORAZEPAM), note types (e.g., ED and progress notes). We could also use a cut-off percentage threshold (e.g., 15%) to refine the candidate ADRs (from FDA reported drug-reaction pairs) for ADR detection.

The PPV (42.4%) for this study was higher than the NLP-based algorithm developed by Wang et al

(achieved a PPV from 16% to 31%) to detect ADEs on discharge summaries (Wang et al., 2010). Park’s

NLP algorithm for detecting laboratory abnormalities after treatment with medications achieved PPVs of

22-75% (Park et al., 2011). When compared with data mining and machine learning methods, the performance of this study was better than Ji’s data mining-based approach, which achieved a PPV of 16% to identify potential ADRs from EHRs (Ji et al., 2011). However, the PPV for my method was relatively lower than supervised learning methods developed by Lingren (Lingren, 2013): The PPV for identifying

ADEs from ten patients in Lingren’s work was 80.4%. However, the method developed by Lingren was only evaluated on the data of ten patients. Indeed, most of the existing algorithms for ADR detection focused on certain medications. On the contrary, my method provided a more generalizable solution for

ADR detection that was not limited to certain medications or reactions. Compared with the machine learning algorithms that require manual annotation, my approach was completely automated and hence more cost-effective.

The algorithm performed better on the ED notes and the progress notes (Table 17), suggesting that the medication-reaction pairs found in these notes are more likely to indicate ADR events. This is because the

ADR events occurred during the patients’ hospitalization were usually documented in these notes other than H&P notes and discharge summaries. In addition, the algorithm achieved 100% PPV on nine medication-note type pairs (Table 20). The promising results show the great potential for the integration of the algorithm into clinical practice to detect ADR events for these medications. The TP ADRs listed in

Table 21 could potentially benefit the clinicians. It concludes which reaction(s) could be caused by a medication as well as the frequency/probability of the reactions on the medication.

By conducting an error analysis on the FP ADR predictions, I found that the majority of the errors were due to the lack of semantic analysis. The current algorithm enumerated all medication-reaction pairs from a DRPS without analyzing the syntax of the sentence. That is, one ADR-related DRPS would appear as a

TP for a medication-reaction pair in a DRPS but FPs for all other medication-reaction pairs. For instance,

72 the algorithm would detected four ADR-related events from the sentence “patient had vomiting last night, now having itching related to oxycodone use, discontinued oxycodone and started ibuprofen.”: oxycodone-vomiting, oxycodone-itching, ibuprofen-vomiting and ibuprofen-itching. Although it captured the TP ADR (oxycodone-itching), it also caused three FPs due to the negligence of the sentence structure.

To alleviate this problem, I will apply advanced NLP algorithms to analyze the semantic and temporal relations within the context in our future work.

Most of the existing ADR/ADE detection algorithms could only be applied to specific medication(s) or drug-reaction pair(s). Although we focused on the 41 most frequently used medications in our institution’s EHR data, the algorithm could be generalizable and applied to potentially all drugs and their reactions. In addition, all processes conducted in this study were automated. The generality and the programmable automation of the algorithm suggest that the study could eventually lead to an integrated

ADR event identification system for clinical decision support.

The findings of the study were limited in several ways. The severity of the reaction was not addressed in this study. Some severe reactions (e.g., severe pains) should be treated immediately; other reactions (e.g., vomiting) need attention and followed up by clinicians; while some mild reactions (e.g., a little sleepy) might be assigned a low severity score and given low priority to be reviewed and clinically followed.

Future work is in progress to identify severity and attribution of the ADR to the medication.

Another limitation is that the study did not consider drug-drug interaction and dosage information. The patients in the study were more likely to take multiple drugs, and some drugs might interact with others and cause ADRs. Although drug dosage information existed in the EHR data, I did not consider it due to complexity and computational power. Future studies should consider drug-drug interaction and dosage information.

In addition, I only detected ADRs which happened during the patients’ hospitalization because the study design required ADRs to occur during the patients’ hospital stay. Therefore, the algorithm did not detect

ADRs that were caused by medications prescribed and used mainly in an outpatient setting. What’s more,

I simplified the ADR time windows in this study. Different ADRs may have different ADR time windows even for the same medication. The PPV would increase if I could collect specific and precise time window from professionals for each drug-reaction pair.

The candidate drug-reaction pairs could be further refined by professionals. I have excluded dozens of indications as well as unrelated reactions. However, due to the limitation of my clinical knowledge, many non-ADR drug-reaction pairs might still exist in the candidate list. A more refined candidate drug- reaction pairs will definitely improve the performance of the algorithm.

Finally, this study has limitations with respect to the algorithm evaluation. Selection bias may exist since

I only used the EHR data from one pediatric institution. The result and conclusion might not apply to

EHR data from other hospitals with mainly adult patients. I could not evaluate the recall of the algorithm because no gold standard existed. I used an evaluation criteria derived from independent review by one clinical informatics expert with a medical degree and myself to calculate the PPVs. Although the inter reviewer agreement (83.7%) was high, the criteria is not optimal because I do not have a medical training.

Future studies should involve several clinical workers or pharmacists who work full-time to work out a gold standard data set and perform manual review for the identified ADR-related DRPSs. EHR data from other hospital settings might be utilized if it is available in future work.

I would perform future research in the following directions: I would look into using non-professional

(such as patients, layperson) reported drug-reaction pairs in the FAERS database and see how the PPV changes then; I would refine the candidate drug-reaction pairs and their corresponding time windows to improve the performance; I would try leveraging other public resources to obtain candidate ADRs and then identify ADRs from EHR data based on the candidate ADRs; I would also refine my algorithm for medical data from other clinical settings (e.g., outpatients, other types of clinical notes, EHR from other hospitals) in future studies.

7 Conclusion

By leveraging candidate drug-reaction pairs from the FDA ADE reports, I developed an NLP-based, automated algorithm to identify ADRs in the EHR data for the most frequently used medications. Using a clinician generated, reference standard based evaluation of real-world clinical data, the proposed algorithm achieved promising performance and showed great potential in identifying the ADRs accurately.

The rejection of the null hypothesis conclude that leveraging drug-reaction pairs from the FAERS will improve the performance of EHR-based ADR detection. Although the study focused only on the most frequently used medications in our institution, the algorithm is generalizable to all medications. The study will pave the way to automate ADR detection in all hospitalized settings.

8 References

Aramaki, E., Miura, Y., Tonoike, M., Ohkuma, T., Masuichi, H., Waki, K., & Ohe, K. (2010). Extraction of adverse drug effects from clinical records. Studies in Health Technology and Informatics, 160(PART 1), 739–743. http://doi.org/10.3233/978-1-60750-588-4-739

Association, A. P. (2013). Diagnostic and Statistical Manual of Mental Disorders (DSM-5®). Washington, D.C.: American Psychiatric Publishing.

Auerbuch, M., Karson, T. H., Ben-Ami, B., Maimon, O., & Rokach, L. (2004). Context-sensitive medical information retrieval. Studies in Health Technology and Informatics, 107(Pt 1), 282–6. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/15360819

Bleeker, T. (2012). Clinical Documents Pipeline. Retrieved January 1, 2015, from https://cwiki.apache.org/confluence/display/CTAKES/cTAKES+3.0+- +Clinical+Documents+Pipeline

Bodenreider, O. (2004). The Unified Medical Language System (UMLS): integrating biomedical terminology. Nucleic Acids Research, 32(Database issue), D267–D270. http://doi.org/10.1093/nar/gkh061

Carspecken, C. W., Sharek, P. J., Longhurst, C., & Pageler, N. M. (2013). A Clinical Case of Electronic Health Record Drug Alert Fatigue: Consequences for Patient Outcome. Pediatrics, 131(6), e1970– e1973. http://doi.org/10.1542/peds.2012-3252

Chapman, W. W., Bridewell, W., Hanbury, P., Cooper, G. F., & Buchanan, B. G. (2001a). A simple algorithm for identifying negated findings and diseases in discharge summaries. Journal of Biomedical Informatics, 34(5), 301–310. http://doi.org/10.1006/jbin.2001.1029

Chapman, W. W., Bridewell, W., Hanbury, P., Cooper, G. F., & Buchanan, B. G. (2001b). Evaluation of negation phrases in narrative clinical reports. Proceedings / AMIA ... Annual Symposium. AMIA Symposium, 105–109. http://doi.org/D010001471 [pii]

Chapman, W. W., Hillert, D., Velupillai, S., Kvist, M., Skeppstedt, M., Chapman, B. E., … Deleger, L. (2013). Extending the NegEx lexicon for multiple languages. Studies in Health Technology and Informatics, 192, 677–81. Retrieved from http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3923890&tool=pmcentrez&rendertype= abstract

Chazard, E., Bernonville, S., Ficheur, G., & Beuscart, R. (2012). A statistics-based approach of contextualization for adverse drug events detection and prevention. Studies in Health Technology and Informatics, 180, 766–770. http://doi.org/10.3233/978-1-61499-101-4-766

Chazard, E., Ficheur, G., Bernonville, S., Luyckx, M., & Beuscart, R. (2011). Data mining to generate adverse drug events detection rules. IEEE Transactions on Information Technology in Biomedicine, 15(6), 823–830. http://doi.org/10.1109/TITB.2011.2165727

Chazard, E., Merlin, B., Ficheur, G., Sarfati, J. C., & Beuscart, R. (2009). Detection of adverse drug events: Proposal of a data model. Studies in Health Technology and Informatics, 148, 63–74. http://doi.org/10.3233/978-1-60750-043-8-63

Cheetham, T. C., Lee, J., Hunt, C. M., Niu, F., Reisinger, S., Murray, R., … Papay, J. (2014). An automated causality assessment algorithm to detect drug-induced liver injury in electronic medical record data. Pharmacoepidemiology and Drug Safety, 23(6), 601–8. http://doi.org/10.1002/pds.3531

Classen, D. C., Pestotnik, S. L., Evans, R. S., Lloyd, J. F., & Burke, J. P. Adverse drug events in hospitalized patients. Excess length of stay, extra costs, and attributable mortality. JAMA, 277(4), 301–6. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/9002492

Cullen, D. J., Sweitzer, B. J., Bates, D. W., Burdick, E., Edmondson, A., & Leape, L. L. (1997). Preventable adverse drug events in hospitalized patients: a comparative study of intensive care and general care units. Critical Care Medicine, 25(8), 1289–97. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/9267940

Deleger, L., Li, Q., Lingren, T., Kaiser, M., Molnar, K., Stoutenborough, L., … Solti, I. (2012). Building gold standard corpora for medical natural language processing tasks. AMIA ... Annual Symposium Proceedings / AMIA Symposium. AMIA Symposium, 2012, 144–53. Retrieved from http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3540456&tool=pmcentrez&rendertype= abstract

Deleger, L., Molnar, K., Savova, G., Xia, F., Lingren, T., Li, Q., … Solti, I. (2012). Large-scale evaluation of automated clinical note de-identification and its impact on information extraction. Journal of the American Medical Informatics Association, (2), 84–94. http://doi.org/10.1136/amiajnl-2012-001012

Edwards, I., & Aronson, J. (2000). Adverse drug reactions: definitions, diagnosis, and management. Lancet, 356(9237), 1255–1259. http://doi.org/10.1016/S0140-6736(00)02799-9

Elkin, P. L., Brown, S. H., Bauer, B. a, Husser, C. S., Carruth, W., Bergstrom, L. R., & Wahner-Roedler, D. L. (2005). A controlled trial of automated classification of negation from clinical notes. BMC Medical Informatics and Decision Making, 5, 13. http://doi.org/10.1186/1472-6947-5-13

Epstein, R. H., St Jacques, P., Stockin, M., Rothman, B., Ehrenfeld, J. M., & Denny, J. C. (2013). Automated identification of drug and food allergies entered using non-standard terminology. Journal of the American Medical Informatics Association : JAMIA, 20(5), 962–8. http://doi.org/10.1136/amiajnl-2013-001756

FDA. (2015). The FDA Adverse Event Reporting System (FAERS). Retrieved from http://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Surveillance/AdverseDrugE ffects/

Garla, V., Lo Re, V., Dorey-Stein, Z., Kidwai, F., Scotch, M., Womack, J., … Brandt, C. (2011). The Yale cTAKES extensions for document classification: architecture and application. Journal of the American Medical Informatics Association : JAMIA, 18(5), 614–620. http://doi.org/10.1136/amiajnl- 2011-000093

Goryachev, S., Sordo, M., Zeng, Q., & Ngo, L. (2006). Implementation and Evaluation of Four Different Methods of Negation Detection. Boston, MA: DSG. Retrieved from https://www.i2b2.org/software/projects/hitex/negation.pdf

Harkema, H., Dowling, J. N., Thornblade, T., & Chapman, W. W. (2010). and Temporal Status from Clinical Reports, 42(5), 839–851. http://doi.org/10.1016/j.jbi.2009.05.002.Context

Harpaz, R., Chase, H. S., & Friedman, C. (2010). Mining multi-item drug adverse effect associations in spontaneous reporting systems. BMC Bioinformatics, 11 Suppl 9(Suppl 9), S7. http://doi.org/10.1186/1471-2105-11-S9-S7

Henao, R., Murray, J., Ginsburg, G., Carin, L., & Lucas, J. E. (2013). Patient clustering with uncoded text in electronic medical records. AMIA ... Annual Symposium Proceedings / AMIA Symposium. AMIA Symposium, 2013(3), 592–9. Retrieved from http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3900202&tool=pmcentrez&rendertype= abstract

HHS. (2015). Reducing and Preventing Adverse Drug Events To Decrease Hospital Costs. Retrieved January 1, 2015, from http://archive.ahrq.gov/research/findings/factsheets/errors- safety/aderia/ade.html

Honig, P. K. (2013). Advancing the science of pharmacovigilance. Clinical Pharmacology and Therapeutics, 93(6), 474–5. http://doi.org/10.1038/clpt.2013.60

Ji, Y., Ying, H., Dews, P., Mansour, A., Tran, J., Miller, R. E., & Massanari, R. M. (2011). A potential causal association mining algorithm for screening adverse drug reactions in postmarketing surveillance. IEEE Transactions on Information Technology in Biomedicine, 15(3), 428–437. http://doi.org/10.1109/TITB.2011.2131669

Jung, K., LePendu, P., Iyer, S., Bauer-Mehren, a., Percha, B., & Shah, N. H. (2014). Functional evaluation of out-of-the-box text-mining tools for data-mining tasks. Journal of the American Medical Informatics Association, 121–131. http://doi.org/10.1136/amiajnl-2014-002902

Kirkendall, E. S., Spires, W. L., Mottes, T. a., Schaffzin, J. K., Barclay, C., & Goldstein, S. L. (2014). Development and Performance of Electronic Acute Kidney Injury Triggers to Identify Pediatric Patients at Risk for Nephrotoxic Medication-associated Harm. Applied Clinical Informatics, 5(2), 313–333. http://doi.org/10.4338/ACI-2013-12-RA-0102

Kirkendall, E. S., Spooner, S. A., & Logan, J. R. (2014). Evaluating the accuracy of electronic pediatric drug dosing rules. Journal of the American Medical Informatics Association : JAMIA, 21(e1), e43–9. http://doi.org/http://dx.doi.org/10.1136/amiajnl-2013-001793

Kirmani, B. F., Mungall, D., & Ling, G. (2013). Role of intravenous levetiracetam in seizure prophylaxis of severe traumatic brain injury patients. Frontiers in Neurology, 4 NOV(November), 4–7. http://doi.org/10.3389/fneur.2013.00170

Lependu, P., Iyer, S. V, Bauer-Mehren, A., Harpaz, R., Ghebremariam, Y. T., Cooke, J. P., & Shah, N. H. (2013). Pharmacovigilance using Clinical Text. AMIA Joint Summits on Translational Science Proceedings AMIA Summit on Translational Science, 2013(2011), 109. Retrieved from http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3814501&tool=pmcentrez&rendertype= abstract

Lexicomp. (2015). Lexicomp Online Database. Retrieved from http://online.lexi.com/

Li, Q., Deleger, L., Lingren, T., Zhai, H., Kaiser, M., Stoutenborough, L., … Solti, I. (2013). Mining FDA drug labels for medical conditions. BMC Medical Informatics and Decision Making, 13, 53. http://doi.org/10.1186/1472-6947-13-53

Li, Q., Melton, K., Lingren, T., Kirkendall, E. S., Hall, E., Zhai, H., … Solti, I. (2014). Phenotyping for patient safety: algorithm development for electronic health record based automated adverse event and medical error detection in neonatal intensive care. Journal of the American Medical Informatics Association : JAMIA, 776–784. http://doi.org/10.1136/amiajnl-2013-001914

Lingren, T. G. (2013). Detecting Adverse Events in Clinical Trial Free Text.

Liu, M., McPeek Hinz, E. R., Matheny, M. E., Denny, J. C., Schildcrout, J. S., Miller, R. a, & Xu, H. (2013). Comparative analysis of pharmacovigilance methods in the detection of adverse drug reactions using electronic medical records. Journal of the American Medical Informatics Association : JAMIA, 20(3), 420–426. http://doi.org/10.1136/amiajnl-2012-001119

Liyanage, H., & De Lusignan, S. (2014). Ontologies to capture adverse events following immunisation (AEFI) from real world health data. Studies in Health Technology and Informatics, 197, 15–19. http://doi.org/10.3233/978-1-61499-389-6-15

Masanz, J. (2013). cTAKES 3.1 Component Use Guide. Retrieved January 1, 2015, from https://cwiki.apache.org/confluence/display/CTAKES/cTAKES+3.1+Component+Use+Guide

Mutalik, P. G., Deshpande, A., & Nadkarni, P. M. (2001). Use of General-purpose Negation Detection to Augment Concept Indexing of Medical Documents. Journal of the American Medical Informatics Association, 8(6), 598–609. http://doi.org/10.1136/jamia.2001.0080598

NCCMERP. (2015). What is a Medication Error? Retrieved January 1, 2015, from http://www.nccmerp.org/about-medication-errors

Nebeker, J. R., Barach, P., & Samore, M. H. (2004). Improving Patient Care Clarifying Adverse Drug Events : A Clinician ’ s Guide to Terminology , VERSUS, 795–802.

NIH. (2015a). Concept unique identifier. Retrieved January 1, 2015, from http://www.nlm.nih.gov/research/umls/new_users/online_learning/Meta_005.html

NIH. (2015b). UMLS terminology browser. Retrieved January 1, 2015, from https://uts.nlm.nih.gov/home.html

Overby, C. L., Pathak, J., Gottesman, O., Haerian, K., Perotte, A., Murphy, S., … Weng, C. (2013). A collaborative approach to developing an electronic health record phenotyping algorithm for drug- induced liver injury. Journal of the American Medical Informatics Association : JAMIA, 1–10. http://doi.org/10.1136/amiajnl-2013-001930

Park, M. Y., Yoon, D., Lee, K., Kang, S. Y., Park, I., Lee, S.-H., … Park, R. W. (2011). A novel algorithm for detection of adverse drug reaction signals using a hospital electronic medical record database. Pharmacoepidemiology and Drug Safety, 20(6), 598–607. http://doi.org/10.1002/pds.2139

Pathak, J., Kiefer, R. C., & Chute, C. G. (2013). Using linked data for mining drug-drug interactions in electronic health records. Studies in Health Technology and Informatics, 192(1-2), 682–686. http://doi.org/10.3233/978-1-61499-289-9-682

Ramesh, B. P., Belknap, S. M., Li, Z., Frid, N., West, D. P., & Yu, H. (2014). Automatically Recognizing Medication and Adverse Event Information From Food and Drug Administration’s Adverse Event Reporting System Narratives. JMIR Medical Informatics, 2(1), e10. http://doi.org/10.2196/medinform.3022

Roque, F. S., Jensen, P. B., Schmock, H., Dalgaard, M., Andreatta, M., Hansen, T., … Brunak, S. (2011). Using electronic patient records to discover disease correlations and stratify patient cohorts. PLoS Computational Biology, 7(8). http://doi.org/10.1371/journal.pcbi.1002141

Sai, K., Hanatani, T., Azuma, Y., Segawa, K., Tohkin, M., Omatsu, H., … Saito, Y. (2013). Development of a detection algorithm for statin-induced myopathy using electronic medical records. Journal of Clinical Pharmacy and Therapeutics, 38(3), 230–235. http://doi.org/10.1111/jcpt.12063

Sauzet, O., Carvajal, A., Escudero, A., Molokhia, M., & Cornelius, V. R. (2013). Illustration of the weibull shape parameter signal detection tool using electronic healthcare record data. Drug Safety : An International Journal of Medical Toxicology and Drug Experience, 36(10), 995–1006. http://doi.org/10.1007/s40264-013-0061-7

Savova, G. K., Masanz, J. J., Ogren, P. V, Zheng, J., Sohn, S., Kipper-Schuler, K. C., & Chute, C. G. (2010). Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture,

component evaluation and applications. Journal of the American Medical Informatics Association : JAMIA, 17(5), 507–513. http://doi.org/10.1136/jamia.2009.001560

Sohn, S., Kocher, J.-P. a., Chute, C. G., & Savova, G. K. (2011). Drug side effect extraction from clinical narratives of psychiatry and psychology patients. Journal of the American Medical Informatics Association, 18(Suppl 1), i144–i149. http://doi.org/10.1136/amiajnl-2011-000351

Solti, I., & Kye, J. (2010). General NegEx Java Implementation v.2.0. Retrieved from https://code.google.com/p/negex/downloads/detail?name=GeneralNegEx.Java.v.2.0_10272010.zip

Trotti, A., Colevas, a. D., Setser, A., Rusch, V., Jaques, D., Budach, V., … Rubin, P. (2003). CTCAE v3.0: Development of a comprehensive grading system for the adverse effects of cancer treatment. Seminars in Radiation Oncology, 13(3), 176–181. http://doi.org/10.1016/S1053-4296(03)00031-6

Valkhoff, V. E., Coloma, P. M., Masclee, G. M. C., Gini, R., Innocenti, F., Lapi, F., … Trifirò, G. (2014). Validation study in four health-care databases: Upper gastrointestinal bleeding misclassification affects precision but not magnitude of drug-related upper gastrointestinal bleeding risk. Journal of Clinical Epidemiology, 67(8), 921–931. http://doi.org/10.1016/j.jclinepi.2014.02.020

Wang, X., Chase, H., Markatou, M., Hripcsak, G., & Friedman, C. (2010). Selecting information in electronic health records for knowledge acquisition. Journal of Biomedical Informatics, 43(4), 595– 601. http://doi.org/10.1016/j.jbi.2010.03.011 www.Drugs.com. (2015). Indications for drugs. Retrieved January 1, 2015, from http://www.drugs.com/

Yoon, D., Park, M. Y., Choi, N. K., Park, B. J., Kim, J. H., & Park, R. W. (2012). Detection of adverse drug reaction signals using an electronic health records database: Comparison of the Laboratory Extreme Abnormality Ratio (CLEAR) algorithm. Clinical Pharmacology and Therapeutics, 91(3), 467–74. http://doi.org/10.1038/clpt.2011.248

9 Appendix

Appendix Table 1: Frequencies and cumulative percentages for the top 50 most frequently used medications in EHR data

Cumulati Cumulat Cumulati Numb Numb ve Numb ive ve Numbe er of er of Percentag er of Percenta Percentag r of EHR EHR e over all EHR Drug Name ge over e over all FDA MAR MAR EHR MAR all EHR EHR ADE record Encou MAR Patient MAR MAR reports s nters Encounte s records Patients rs SODIUM CHLORIDE 139941 10.9% 48363 71.4% 29790 73.5% 7

ACETAMINOPHEN 69163 16.3% 37333 82.9% 24281 85.0% 200

HEPARIN 46373 19.9% 13185 83.6% 6585 85.5% 43 MORPHINE 34606 22.6% 13483 84.5% 11077 86.6% 85 SULFATE ALBUTEROL 30669 25.0% 9890 87.5% 6685 89.4% 0 SULFATE FENTANYL 26924 27.1% 10293 88.1% 8722 90.1% 158 CITRATE MIDAZOLAM 24817 29.0% 6749 88.5% 5627 90.3% 13

ONDANSETRON 24710 31.0% 14021 88.9% 9520 90.8% 43 DIPHENHYDRAMIN 24087 32.9% 11225 89.6% 5203 91.5% 35 E FUROSEMIDE 22822 34.6% 4222 89.8% 2542 91.5% 7

LORAZEPAM 22635 36.4% 5216 90.0% 3287 91.6% 19

CEFAZOLIN 18811 37.9% 9176 90.2% 7622 91.8% 2

OXYCODONE 14455 39.0% 7297 90.2% 5299 91.8% 147

IBUPROFEN 14126 40.1% 9974 90.8% 8305 92.5% 192

RANITIDINE 13354 41.1% 8771 91.4% 5817 92.9% 10

VANCOMYCIN 12824 42.1% 4487 91.4% 3011 92.9% 21

IOVERSOL 12331 43.1% 8136 91.5% 6045 92.9% 7

CLINDAMYCIN 12290 44.1% 5389 91.6% 4550 93.0% 16

INSULIN ASPART 12247 45.0% 1028 91.7% 681 93.1% 24 POLYETHYLENE 11492 45.9% 7911 92.1% 5371 93.4% 0 GLYCOL ALBUMIN 11233 46.8% 2135 92.1% 1491 93.4% 1 METHYLPREDNISO 10640 47.6% 4517 92.3% 2622 93.4% 19 LONE LANSOPRAZOLE 10593 48.4% 7847 92.9% 3570 93.7% 21

HYDROMORPHONE 10584 49.3% 2845 92.9% 2108 93.7% 1

LIDOCAINE 10503 50.1% 7656 93.1% 5125 93.8% 26

DIAZEPAM 10195 50.9% 4567 93.1% 3433 93.9% 18 ROCURONIUM 9876 51.7% 1719 93.1% 1531 93.9% 0 BROMIDE INSULIN LISPRO 9728 52.4% 789 93.2% 640 93.9% 31 HYDROCORTISONE 8848 53.1% 3287 93.3% 1982 94.0% 19

EPINEPHRINE 8832 53.8% 3819 93.5% 3168 94.2% 39

CEFTRIAXONE 8485 54.4% 5601 93.5% 4413 94.2% 73

CEFOXITIN 8446 55.1% 3704 93.5% 3315 94.3% 0 POTASSIUM 8365 55.8% 2193 93.6% 1496 94.3% 0 CHLORIDE BUPIVACAINE 8255 56.4% 7378 93.7% 6500 94.3% 7

KETOROLAC 7891 57.0% 5410 93.8% 4606 94.4% 2

AMOXICILLIN 7876 57.6% 6099 94.1% 5231 94.8% 58 PREDNISOLONE & SODIUM 7800 58.2% 4697 94.2% 3462 94.8% 0 PHOSPHATE LIDOCAINE&PRILO 7726 58.8% 5618 94.5% 3969 95.2% 6 CAINE METHADONE 7564 59.4% 877 94.5% 710 95.2% 65

GENTAMICIN 7418 60.0% 3569 94.5% 2623 95.3% 0

NYSTATIN 6841 60.5% 4862 94.7% 2400 95.4% 4 LACTATED 6707 61.1% 6480 94.8% 5873 95.5% 0 RINGERS PIPERACILLIN & 6702 61.6% 3396 94.8% 2127 95.5% 12 TAZOBACTAM DEXAMETHASONE SODIUM 6458 62.1% 4080 94.8% 3430 95.6% 10 PHOSPHATE SODIUM 6378 62.6% 1511 94.9% 1234 95.6% 1 BICARBONATE KETAMINE 6377 63.1% 1358 94.9% 1262 95.6% 15 EPINEPHRINE & 6262 63.6% 5478 94.9% 4496 95.6% 4 LIDOCAINE LEVETIRACETAM 6177 64.1% 2720 95.2% 1308 95.8% 204

MORPHINE 6146 64.5% 2573 95.2% 2282 95.8% 13

DOCUSATE 5900 65.0% 4577 95.2% 3079 95.9% 0

Appendix Table 2: Excluded reactions

Excluded Reactions Excluded Reactions ABORTION SPONTANEOUS FOETAL HEART RATE ABNORMAL ACCIDENT FOETAL HEART RATE DECELERATION

ACCIDENTAL DEATH FOETAL HYPOKINESIA

ACCIDENTAL DRUG INTAKE BY CHILD GUN SHOT WOUND

ACCIDENTAL EXPOSURE ILL-DEFINED DISORDER ACCIDENTAL EXPOSURE TO PRODUCT BY IMPAIRED DRIVING ABILITY CHILD ACCIDENTAL NEEDLE STICK INDUCED LABOUR

ACCIDENTAL OVERDOSE INTENTIONAL DRUG MISUSE

ACCIDENTAL POISONING INTENTIONAL MISUSE

ALCOHOL USE INTENTIONAL OVERDOSE

AUTISM INTRA-UTERINE DEATH MATERNAL DRUGS AFFECTING BRADYCARDIA FOETAL FOETUS BRADYCARDIA NEONATAL MEDICATION ERROR

CAESAREAN SECTION NO ADVERSE DRUG EFFECT

CHILD ABUSE NO ADVERSE EFFECT

CHILD MALTREATMENT SYNDROME NO ADVERSE EVENT CONTRAINDICATION TO MEDICAL NO REACTION ON PREVIOUS TREATMENT EXPOSURE TO DRUG DIET REFUSAL OFF LABEL USE

DISEASE COMPLICATION OVERDOSE

DISEASE PROGRESSION PREMATURE DELIVERY PREMATURE SEPARATION OF DISEASE RECURRENCE PLACENTA DRUG ABUSER PRESCRIBED OVERDOSE DRUG ADMINISTERED TO PATIENT OF ROAD TRAFFIC ACCIDENT INAPPROPRIATE AGE DRUG ADMINISTRATION ERROR SELF-MEDICATION

DRUG DEPENDENCE SOCIAL PROBLEM

DRUG DETOXIFICATION STILLBIRTH

DRUG DISPENSING ERROR SUBSTANCE ABUSE

DRUG DOSE OMISSION TACHYCARDIA FOETAL

DRUG EXPOSURE DURING PREGNANCY TOXICITY TO VARIOUS AGENTS DRUG EXPOSURE VIA BREAST MILK TREATMENT NONCOMPLIANCE

DRUG INEFFECTIVE UNEVALUABLE EVENT

DRUG LEVEL VICTIM OF ABUSE

DRUG PRESCRIBING ERROR VICTIM OF CHILD ABUSE

DRUG WITHDRAWAL SYNDROME VICTIM OF CRIME

DRUG WITHDRAWAL SYNDROME NEONATAL VICTIM OF HOMICIDE

FOETAL CEREBROVASCULAR DISORDER WHEELCHAIR USER

FOETAL DISORDER WRONG DRUG ADMINISTERED WRONG TECHNIQUE IN DRUG USAGE FOETAL EXPOSURE DURING PREGNANCY PROCESS FOETAL GROWTH RESTRICTION FOETAL GROWTH RETARDATION

Appendix Table 3: Excluded FDA drug-reaction pairs

Drug Name Excluded Reactions ACETAMINOPHE ABDOMINAL PAIN; ABDOMINAL PAIN UPPER; AGITATION; N HEADACHE; IRRITABILITY; PYREXIA; FUNGAL INFECTION; PSEUDOMONAS INFECTION; PYREXIA; AMOXICILLIN RESPIRATORY TRACT INFECTION; ABSCESS; CHOLANGITIS; COUGH; DIARRHOEA; ESCHERICHIA CEFTRIAXONE INFECTION; LEUKOCYTOSIS; PLEURAL EFFUSION; PYREXIA; RESPIRATORY DISTRESS; SEPTIC SHOCK; SKIN INFECTION; CLINDAMYCIN PYREXIA;

DIAZEPAM CONVULSION; ANAPHYLACTIC REACTION; BLOOD PRESSURE DECREASED; BRADYCARDIA; CARDIAC ARREST; ERYTHEMA; HEART RATE EPINEPHRINE DECREASED; HEART RATE INCREASED; HYPERSENSITIVITY; HYPERTENSION; HYPOTENSION; RESPIRATORY DISTRESS; SINUS TACHYCARDIA; TACHYCARDIA; HEPARIN DEEP VEIN THROMBOSIS; THROMBOSIS; HYDROCORTISO ANAEMIA; ASTHMA; ECZEMA; EXANTHEM; HYPERSENSITIVITY; NE PRURITUS; SEPSIS; THROMBOCYTOPENIA; VOMITING; ABDOMINAL PAIN; AGITATION; ARTHRALGIA; ARTHRITIS; CELLULITIS; CHEST PAIN; EAR PAIN; FRACTURE; INFECTION; MUSCLE IBUPROFEN CRAMP; NECK PAIN; PAIN; PAIN IN EXTREMITY; PERICARDIAL EFFUSION; PHARYNGITIS; PHARYNGOLARYNGEAL PAIN; PYREXIA; TENDERNESS; TORTICOLLIS; ULCER; VIRAL INFECTION; INSULIN LISPRO DIABETES MELLITUS INADEQUATE CONTROL; KETAMINE SEDATION;

LEVETIRACETA CONVULSION; PARTIAL SEIZURES; M LIDOCAINE MYALGIA;

LORAZEPAM AGITATION; IRRITABILITY;

METHADONE HEART RATE INCREASED; METHYLPREDNI ASTHMA; HYPERSENSITIVITY; INFECTION; SOLONE MIDAZOLAM AGITATION; SEDATION;

MORPHINE AGITATION; PAIN IN EXTREMITY; TACHYCARDIA; MORPHINE ABDOMINAL PAIN; AGITATION; HEADACHE; SULFATE NYSTATIN DIARRHOEA;

ONDANSETRON NAUSEA; VOMITING; ABDOMINAL PAIN; AGITATION; EAR PAIN; IRRITABILITY; PAIN; OXYCODONE PYREXIA; PIPERACILLIN& HYPERTHERMIA; PYREXIA; RASH; TAZOBACTAM BRONCHOPNEUMONIA; COUGH; HYPOTENSION; MYOCARDITIS; PAIN; VANCOMYCIN PYREXIA; RASH; SCRATCH;

Appendix Table 4: Included FDA drug-reaction pairs

Drug Name Included Reactions ABNORMAL BEHAVIOUR; ACIDOSIS; ACTIVATED PARTIAL THROMBOPLASTIN TIME PROLONGED; ACUTE HEPATIC FAILURE; AGGRESSION; ALANINE AMINOTRANSFERASE INCREASED; AMINO ACID LEVEL DECREASED; AMINO ACID METABOLISM DISORDER; AMMONIA INCREASED; ANALGESIC DRUG LEVEL ABOVE THERAPEUTIC; ANALGESIC DRUG LEVEL INCREASED; ANAPHYLACTIC REACTION; ANGIOEDEMA; ANION GAP INCREASED; ANURIA; APNOEA; ARGININOSUCCINATE SYNTHETASE DEFICIENCY; ARRHYTHMIA; ARTHRALGIA; ASCITES; ASPARTATE AMINOTRANSFERASE ACETAMINOPH INCREASED; ASPIRATION; BLOOD BICARBONATE DECREASED; BLOOD EN BILIRUBIN INCREASED; BLOOD CREATINE INCREASED; BLOOD CREATINE PHOSPHOKINASE INCREASED; BLOOD CREATININE INCREASED; BLOOD GLUCOSE DECREASED; BLOOD LACTATE DEHYDROGENASE INCREASED; BLOOD LACTIC ACID INCREASED; BLOOD PH DECREASED; BLOOD PRESSURE DECREASED; BLOOD PRESSURE DIASTOLIC DECREASED; BLOOD SODIUM DECREASED; BLOOD UREA DECREASED; BLOOD UREA INCREASED; BODY TEMPERATURE DECREASED; BODY TEMPERATURE INCREASED; BRADYCARDIA; BRAIN DEATH; BRAIN OEDEMA; BRONCHIOLITIS;

CARBOXYHAEMOGLOBIN INCREASED; CARDIAC ARREST; CARDIAC FAILURE CONGESTIVE; CARDIOVASCULAR DISORDER; CARNITINE DEFICIENCY; CEREBRAL HAEMORRHAGE; CEREBROVASCULAR ACCIDENT; CHOLESTASIS; CIRCULATORY COLLAPSE; COAGULOPATHY; COMA; COMA ACIDOTIC; COMPLETED SUICIDE; CONFUSIONAL STATE; CONGENITAL ATRIAL SEPTAL DEFECT; CONJUNCTIVAL HYPERAEMIA; CONJUNCTIVITIS; CONTUSION; CONVULSION; CRYING; CSF CELL COUNT INCREASED; CSF GLUCOSE; CSF LYMPHOCYTE COUNT INCREASED; CSF PROTEIN; CSF PROTEIN INCREASED; CSF TEST ABNORMAL; CYANOSIS; DEAFNESS; DEATH NEONATAL; DEHYDRATION; DELIRIUM; DEPRESSED LEVEL OF CONSCIOUSNESS; DERMATITIS ALLERGIC; DIALYSIS; DIARRHOEA; DIZZINESS; DRUG ERUPTION; DRUG HYPERSENSITIVITY; DRUG INTOLERANCE; DRUG LEVEL ABOVE THERAPEUTIC; DRUG LEVEL INCREASED; DRUG SCREEN POSITIVE; DRUG SPECIFIC ANTIBODY PRESENT; DRUG TOXICITY; DYSPNOEA; ECCHYMOSIS; ELECTROCARDIOGRAM QT SHORTENED; ELECTROCARDIOGRAM T WAVE ABNORMAL; ELECTROENCEPHALOGRAM ABNORMAL; ELECTROLYTE IMBALANCE; EMPHYSEMA; ENCEPHALOPATHY; EOSINOPHILIA; EXPOSURE DURING BREAST FEEDING; EXPOSURE DURING PREGNANCY; EYE ROLLING; EYE SWELLING; FACE OEDEMA; FATIGUE; FEAR; FEBRILE CONVULSION; FEEDING DISORDER NEONATAL; FEELING ABNORMAL; FLUSHING; FOOD POISONING; GASTROINTESTINAL DISORDER; GASTROINTESTINAL HAEMORRHAGE; GENERAL PHYSICAL HEALTH DETERIORATION; GLARE; GRAND MAL CONVULSION; HAEMATOCRIT ABNORMAL; HAEMATOCRIT DECREASED; HAEMATURIA; HAEMODIALYSIS; HAEMOGLOBIN ABNORMAL; HAEMOGLOBIN DECREASED; HAEMOLYSIS; HAEMORRHAGE; HAEMORRHAGIC DISORDER; HEAD CIRCUMFERENCE ABNORMAL; HEART RATE INCREASED; HEPATIC ENZYME INCREASED; HEPATIC FAILURE; HEPATIC FUNCTION ABNORMAL; HEPATIC NECROSIS; HEPATIC STEATOSIS; HEPATITIS; HEPATITIS ACUTE; HEPATOMEGALY; HEPATORENAL FAILURE; HEPATOTOXICITY; HERNIA; HYPERAEMIA; HYPERAMMONAEMIA; HYPERGLYCAEMIA; HYPERSENSITIVITY; HYPOCHROMIC ANAEMIA; HYPOGLYCAEMIA; HYPOKALAEMIA; HYPONATRAEMIA; HYPOPNOEA; HYPOPROTHROMBINAEMIA; HYPOTENSION; IATROGENIC INJURY; INAPPROPRIATE SCHEDULE OF DRUG ADMINISTRATION; INCORRECT DOSE ADMINISTERED; INCORRECT DRUG DOSAGE FORM ADMINISTERED; INFECTION; INFLAMMATION; INTERNATIONAL NORMALISED RATIO INCREASED; INTRACRANIAL PRESSURE INCREASED; JAUNDICE; JAUNDICE NEONATAL; KETONURIA; KETOSIS; LABORATORY TEST ABNORMAL; LETHARGY; LEUKOPENIA; LIP SWELLING; LIVEDO RETICULARIS; LIVER DISORDER; LIVER FUNCTION TEST ABNORMAL; LIVER INJURY; LIVER TRANSPLANT; LOCAL SWELLING; LOSS OF CONSCIOUSNESS; LUNG INFILTRATION; LYMPHOCYTE STIMULATION TEST POSITIVE; MALAISE; MEMORY IMPAIRMENT; MENTAL STATUS CHANGES; METABOLIC ACIDOSIS; METABOLIC DISORDER; MIOSIS; MOUTH HAEMORRHAGE; MOVEMENT DISORDER; MUSCULOSKELETAL STIFFNESS; NAUSEA; NECROSIS; NEPHROLITHIASIS; NIGHTMARE; NORMOCHROMIC NORMOCYTIC

ANAEMIA; OEDEMA PERIPHERAL; OESOPHAGEAL VARICES HAEMORRHAGE; ORAL MUCOSAL PETECHIAE; PALLOR; PAPILLOEDEMA; PCO2 DECREASED; PETECHIAE; PLATELET COUNT INCREASED; PLEURAL EFFUSION; PNEUMOMEDIASTINUM; PO2 INCREASED; POISONING; POOR QUALITY SLEEP; POST INFLAMMATORY PIGMENTATION CHANGE; POST PROCEDURAL HAEMORRHAGE; PREMATURE BABY; PRODUCT PACKAGING ISSUE; PRODUCTIVE COUGH; PROTEIN TOTAL DECREASED; PROTHROMBIN LEVEL INCREASED; PROTHROMBIN TIME PROLONGED; PRURITUS; PULSE ABNORMAL; PULSE ABSENT; PUPILLARY REFLEX IMPAIRED; RASH ERYTHEMATOUS; RED BLOOD CELL COUNT DECREASED; RENAL FAILURE; RENAL FAILURE ACUTE; RENAL IMPAIRMENT; RESPIRATORY ARREST; RESPIRATORY DISORDER; RESPIRATORY DISTRESS; RESPIRATORY FAILURE; RESPIRATORY RATE INCREASED; RESPIRATORY SYNCYTIAL VIRUS BRONCHIOLITIS; RESPIRATORY SYNCYTIAL VIRUS TEST POSITIVE; RESPIRATORY TRACT INFECTION; RESPIRATORY TRACT INFECTION VIRAL; RETICULOCYTOSIS; REYE'S SYNDROME; RHABDOMYOLYSIS; SEPSIS; SHOCK; SINUS TACHYCARDIA; SKIN DISCOLOURATION; SKIN HYPERPIGMENTATION; SKIN WARM; SOMNOLENCE; SPEECH DISORDER; STEVENS-JOHNSON SYNDROME; SUICIDAL IDEATION; SUICIDE ATTEMPT; SWELLING FACE; SYNCOPE; TACHYCARDIA; TACHYPNOEA; THROAT IRRITATION; THROMBOCYTOPENIA; TINNITUS; TOXIC EPIDERMAL NECROLYSIS; TRANSAMINASES ABNORMAL; TRANSAMINASES INCREASED; TREMOR; TRICHOTILLOMANIA; UNDERDOSE; UNRESPONSIVE TO STIMULI; UPPER RESPIRATORY TRACT INFECTION; URINE OUTPUT DECREASED; URTICARIA; VASODILATATION; VENTRICULAR FIBRILLATION; VENTRICULAR TACHYCARDIA; VOMITING; WEIGHT DECREASED; WHEEZING; WHITE BLOOD CELL COUNT INCREASED; XANTHOCHROMIA; ALBUMIN LATEX ALLERGY; ABDOMINAL PAIN; ABDOMINAL PAIN UPPER; ABNORMAL BEHAVIOUR; ACUTE GENERALISED EXANTHEMATOUS PUSTULOSIS; AGGRESSION; AGITATION; ALANINE AMINOTRANSFERASE INCREASED; ALLERGIC MYOCARDITIS; ANAPHYLACTIC SHOCK; ANGIOEDEMA; ANURIA; ANXIETY; APGAR SCORE LOW; ARTHRALGIA; ASCITES; ASPARTATE AMINOTRANSFERASE INCREASED; BACK PAIN; BLEPHAROSPASM; BLOOD ALBUMIN DECREASED; BLOOD ALKALINE PHOSPHATASE INCREASED; BLOOD CHLORIDE DECREASED; BLOOD CREATININE INCREASED; BLOOD PH DECREASED; BLOOD POTASSIUM AMOXICILLIN INCREASED; BLOOD PRESSURE DECREASED; BLOOD PRESSURE INCREASED; BLOOD SODIUM DECREASED; BLOOD URIC ACID INCREASED; BODY TEMPERATURE INCREASED; BRADYCARDIA; BURNING SENSATION; CARDIAC FAILURE; CEREBRAL INFARCTION; CEREBRAL ISCHAEMIA; CHAPPED LIPS; CHOKING; CHROMATURIA; CLOSTRIDIUM DIFFICILE COLITIS; COMA; CONJUNCTIVAL HYPERAEMIA; CONVULSION; CORNEAL NEOVASCULARISATION; CORNEAL PERFORATION; CORNEAL THINNING; C-REACTIVE PROTEIN INCREASED; CRYSTAL URINE PRESENT; CYANOSIS; DARK CIRCLES UNDER EYES; DERMATITIS ALLERGIC; DIZZINESS; DRUG ERUPTION;

DRUG HYPERSENSITIVITY; DRUG TOXICITY; DYSPNOEA; EMOTIONAL DISORDER; ENCEPHALOPATHY; ENTEROCOLITIS HAEMORRHAGIC; EOSINOPHILIC MYOCARDITIS; EPISTAXIS; EPSTEIN-BARR VIRUS ANTIBODY POSITIVE; ERYTHEMA; FATIGUE; FLANK PAIN; FLUSHING; GAIT DISTURBANCE; HAEMATOCRIT DECREASED; HAEMATURIA; HAEMODIALYSIS; HAEMOGLOBIN DECREASED; HEART RATE INCREASED; HEPATIC FAILURE; HEPATIC FUNCTION ABNORMAL; HEPATOCELLULAR INJURY; HYPERCAPNIA; HYPERHIDROSIS; HYPERSENSITIVITY; HYPERTENSION; HYPOTONIA; INFECTIOUS MONONUCLEOSIS; INFLAMMATION; LETHARGY; LEUKOCYTOSIS; LIP SWELLING; LISTLESS; LIVER FUNCTION TEST ABNORMAL; LOCAL SWELLING; LOSS OF CONSCIOUSNESS; LUPUS NEPHRITIS; MALAISE; MUSCULOSKELETAL STIFFNESS; MYOCARDIAL FIBROSIS; MYOCARDIAL INFARCTION; NAUSEA; NEONATAL RESPIRATORY DISTRESS SYNDROME; NEPHROPATHY TOXIC; NEPHROTIC SYNDROME; OBLITERATIVE BRONCHIOLITIS; OEDEMA; OEDEMA MOUTH; OEDEMA PERIPHERAL; PAIN; PALLOR; PCO2 DECREASED; PCO2 INCREASED; PERICARDIAL EFFUSION; PERIORBITAL OEDEMA; PHARYNGEAL DISORDER; PITTING OEDEMA; PLEURAL EFFUSION; PNEUMOTHORAX; PO2 DECREASED; PREMATURE BABY; PRODUCT QUALITY ISSUE; PROTEINURIA; PRURITUS; PRURITUS GENERALISED; PSYCHOMOTOR HYPERACTIVITY; PUSTULAR PSORIASIS; RASH; RASH ERYTHEMATOUS; RASH GENERALISED; RASH MACULAR; RASH PRURITIC; REACTION TO DRUG EXCIPIENTS; RECTAL HAEMORRHAGE; RENAL FAILURE ACUTE; RENAL FUNCTION TEST ABNORMAL; RENAL PAIN; RESPIRATORY RATE INCREASED; SCROTAL OEDEMA; SHIFT TO THE LEFT; SKIN DEPIGMENTATION; SKIN EXFOLIATION; SKIN LESION; SOMNOLENCE; SPLENOMEGALY; STEVENS-JOHNSON SYNDROME; SUDDEN DEATH; SWELLING FACE; SWOLLEN TONGUE; SYSTEMIC LUPUS ERYTHEMATOSUS; TOOTH DISCOLOURATION; TOXIC EPIDERMAL NECROLYSIS; TYMPANIC MEMBRANE DISORDER; URINARY INCONTINENCE; URINE OXALATE; URTICARIA; VOMITING; VULVOVAGINAL CANDIDIASIS; WHEEZING; YELLOW SKIN; ALTERED STATE OF CONSCIOUSNESS; BLOOD PRESSURE DECREASED; CARDIO-RESPIRATORY ARREST; COMA; CONVULSION; FEEDING DISORDER NEONATAL; HEADACHE; HYPOTONIA; INCORRECT ROUTE OF DRUG ADMINISTRATION; INJECTION SITE PAIN; INJECTION SITE BUPIVACAINE SWELLING; LETHARGY; MENINGITIS; NAUSEA; NUCHAL RIGIDITY; PHARMACEUTICAL PRODUCT COMPLAINT; PULSE ABSENT; RESTLESSNESS; SOMNOLENCE; STRABISMUS; TACHYCARDIA; URINARY RETENTION; VOMITING; CHORIOAMNIONITIS; ENDOMETRITIS DECIDUAL; FUNISITIS; PERITONEAL FLUID ANALYSIS ABNORMAL; POST PROCEDURAL CEFAZOLIN DISCHARGE; STREPTOCOCCUS TEST POSITIVE; TONIC CONVULSION; VASCULITIS; ABDOMINAL PAIN; ABDOMINAL PAIN UPPER; ABDOMINAL TENDERNESS; ACIDOSIS; ACTIVATED PARTIAL THROMBOPLASTIN TIME PROLONGED; ACUTE CHEST SYNDROME; ALANINE CEFTRIAXONE AMINOTRANSFERASE INCREASED; ANAEMIA; ANAEMIA HAEMOLYTIC AUTOIMMUNE; ANAPHYLACTIC REACTION; ANAPHYLACTIC SHOCK; ANGIOEDEMA; ANTIBODY TEST POSITIVE; ANURIA; APALLIC

SYNDROME; APATHY; AREFLEXIA; ARTHRALGIA; ASPARTATE AMINOTRANSFERASE INCREASED; ASTHENIA; BACK PAIN; BACTERIA STOOL IDENTIFIED; BILE DUCT STONE; BILIARY DILATATION; BILIARY TRACT DISORDER; BLOOD BILIRUBIN UNCONJUGATED INCREASED; BLOOD CREATININE INCREASED; BLOOD FIBRINOGEN INCREASED; BLOOD IMMUNOGLOBULIN E INCREASED; BLOOD LACTATE DEHYDROGENASE INCREASED; BLOOD PH DECREASED; BLOOD PHOSPHORUS INCREASED; BLOOD PRESSURE DECREASED; BURNING SENSATION; CALCULUS URINARY; CAPILLARY NAIL REFILL TEST ABNORMAL; CARDIAC ARREST; CARDIO-RESPIRATORY ARREST; CEREBRAL HAEMORRHAGE; CEREBRAL ISCHAEMIA; CHILLS; CHOLECYSTITIS; CHOLECYSTITIS ACUTE; CHOLELITHIASIS; CHOLESTASIS; CONDITION AGGRAVATED; CONVULSION; C-REACTIVE PROTEIN INCREASED; CYANOSIS; DECREASED APPETITE; DEHYDRATION; DIALYSIS; DIPLOPIA; DISSEMINATED INTRAVASCULAR COAGULATION; DIZZINESS; DRUG HYPERSENSITIVITY; DRUG SPECIFIC ANTIBODY PRESENT; DRUG- INDUCED LIVER INJURY; DYSPNOEA; EMOTIONAL DISTRESS; EOSINOPHILIA; ERYTHEMA; EYE SWELLING; FAECAL INCONTINENCE; FALL; FATIGUE; FEMORAL PULSE ABNORMAL; FIBRIN D DIMER INCREASED; FOAMING AT MOUTH; GALLBLADDER DISORDER; GALLBLADDER NECROSIS; GASTROENTERITIS VIRAL; GASTROINTESTINAL TUBE INSERTION; GENERAL PHYSICAL HEALTH DETERIORATION; HAEMATOCRIT DECREASED; HAEMATURIA; HAEMODIALYSIS; HAEMOGLOBIN DECREASED; HAEMOGLOBINURIA; HAEMOLYSIS; HAEMOLYTIC ANAEMIA; HAEMOLYTIC URAEMIC SYNDROME; HAPTOGLOBIN DECREASED; HEADACHE; HEART RATE INCREASED; HEPATITIS; HEPATITIS B SURFACE ANTIGEN POSITIVE; HEPATITIS TOXIC; HYPERAEMIA; HYPERGLYCAEMIA; HYPERSENSITIVITY; HYPERTENSION; HYPOTENSION; HYPOTONIA; HYPOXIC-ISCHAEMIC ENCEPHALOPATHY; INCONTINENCE; INFUSION RELATED REACTION; INTRACRANIAL PRESSURE INCREASED; INTRAVASCULAR HAEMOLYSIS; ISCHAEMIC STROKE; JOINT EFFUSION; JOINT SWELLING; LEUKOCYTOCLASTIC VASCULITIS; LISTLESS; LIVER DISORDER; LOCAL SWELLING; LUNG INFILTRATION; MALAISE; MEDICAL DEVICE COMPLICATION; MENINGOCOCCAL BACTERAEMIA; MENTAL STATUS CHANGES; MOANING; MULTI-ORGAN FAILURE; MUSCULOSKELETAL STIFFNESS; MYDRIASIS; MYOCLONUS; MYOGLOBIN URINE PRESENT; NECROSIS; NERVOUS SYSTEM DISORDER; NORMOCHROMIC NORMOCYTIC ANAEMIA; OEDEMA; OXYGEN SATURATION DECREASED; PAIN IN EXTREMITY; PALLOR; PANIC REACTION; PARAESTHESIA; PERITONEAL DIALYSIS; PHARYNGEAL DISORDER; PHOTOPHOBIA; PLATELET COUNT DECREASED; PRODUCT QUALITY ISSUE; PRODUCT SUBSTITUTION ISSUE; PROTEINURIA; PROTHROMBIN TIME PROLONGED; PRURITUS; PRURITUS GENERALISED; PULSE ABSENT; PURPURA; RASH MACULO- PAPULAR; REFUSAL OF TREATMENT BY PATIENT; RENAL DISORDER; RENAL FAILURE; RENAL FAILURE ACUTE; RENAL TUBULAR NECROSIS; RESPIRATORY ARREST; RESPIRATORY FAILURE; SALIVARY HYPERSECRETION; SCRATCH; SOMNOLENCE; SWELLING FACE; TACHYCARDIA; THROAT TIGHTNESS; TONGUE OEDEMA; TOXIC

ENCEPHALOPATHY; TREMOR; ULTRASOUND ABDOMEN ABNORMAL; UNRESPONSIVE TO STIMULI; URINARY INCONTINENCE; VISUAL DISTURBANCE; VOMITING; WHEEZING; WHITE BLOOD CELL COUNT INCREASED; ACUTE CORONARY SYNDROME; ACUTE MYOCARDIAL INFARCTION; ARNOLD-CHIARI MALFORMATION; BACK PAIN; CAUSTIC INJURY; CHEST PAIN; CORNEAL SCAR; DEAFNESS UNILATERAL; DERMATITIS ALLERGIC; DYSARTHRIA; DYSGEUSIA; DYSPHAGIA; DYSPNOEA; ELECTROCARDIOGRAM ST SEGMENT ELEVATION; EXANTHEM; FOREIGN BODY ASPIRATION; GAIT DISTURBANCE; HAEMANGIOMA; CLINDAMYCIN HEADACHE; HYPERPHOSPHATAEMIA; HYPERSENSITIVITY; IRITIS; KOUNIS SYNDROME; LEG AMPUTATION; LYMPHOCYTOSIS; MALAISE; MUSCLE SPASTICITY; MYOCARDIAL ISCHAEMIA; NAUSEA; OESOPHAGITIS; PERITONEAL DIALYSIS; PRURITUS; REACTION TO COLOURING; RENAL FAILURE ACUTE; SALMONELLOSIS; SEPSIS; STEVENS-JOHNSON SYNDROME; TINNITUS; VOMITING; 17-HYDROXYPROGESTERONE INCREASED; ABSCESS; ADRENOGENITAL SYNDROME; ARRHYTHMIA; ARTHRALGIA; BACTERIAL INFECTION; BLOOD PRESSURE DECREASED; BODY FAT DISORDER; BRAIN OEDEMA; CENTRAL NERVOUS SYSTEM LESION; CEREBROVASCULAR ACCIDENT; CONDITION AGGRAVATED; CONVULSION; COUGH; DEPRESSED LEVEL OF CONSCIOUSNESS; DIARRHOEA; DYSKINESIA; ENCEPHALOPATHY; EYE MOVEMENT DISORDER; EYE OEDEMA; GASTROENTERITIS; HEADACHE; HEMIANOPIA; HEMIPARESIS; DEXAMETHASO HYPERKALAEMIA; HYPERREFLEXIA; HYPERSENSITIVITY; NE SODIUM HYPERTENSION; HYPOGLYCAEMIA; HYPONATRAEMIA; LETHARGY; PHOSPHATE LOSS OF CONSCIOUSNESS; MANIA; MENTAL STATUS CHANGES; MYOCLONUS; NEUROLOGICAL DECOMPENSATION; OSTEONECROSIS; PAIN; PALPITATIONS; PARTIAL SEIZURES; POSTERIOR REVERSIBLE ENCEPHALOPATHY SYNDROME; PUPILLARY REFLEX IMPAIRED; PYREXIA; SKIN STRIAE; SOMNOLENCE; STAPHYLOCOCCAL INFECTION; STRESS; STRIDOR; TACHYCARDIA; TENDON RUPTURE; UPPER RESPIRATORY TRACT INFECTION; VISUAL IMPAIRMENT; VOMITING; WHITE BLOOD CELL COUNT INCREASED; APNOEA; APNOEIC ATTACK; CONDITION AGGRAVATED; DEATH; DEVICE LEAKAGE; DIARRHOEA; DYSPNOEA; FLUSHING; HEADACHE; HYPERCHLORAEMIA; HYPOTONIA NEONATAL; INCORRECT DOSE ADMINISTERED; INFANTILE APNOEIC ATTACK; INJECTION SITE REACTION; MEDICAL DEVICE COMPLICATION; METABOLIC ACIDOSIS; NEONATAL RESPIRATORY FAILURE; NERVOUS SYSTEM DISORDER; NO DIAZEPAM THERAPEUTIC RESPONSE; NYSTAGMUS; OLIGURIA; PHARMACEUTICAL PRODUCT COMPLAINT; PHLEBOTHROMBOSIS; PROCEDURAL VOMITING; PRODUCT QUALITY ISSUE; RESPIRATORY ACIDOSIS; RESPIRATORY DEPRESSION; SOMNOLENCE; SPECIFIC GRAVITY URINE DECREASED; THERAPEUTIC RESPONSE DELAYED; UNDERDOSE; URINE OUTPUT DECREASED; VISUAL ACUITY REDUCED; AGEUSIA; AGITATION; ALANINE AMINOTRANSFERASE INCREASED; ASPARTATE AMINOTRANSFERASE INCREASED; BLOOD BILIRUBIN DIPHENHYDRA INCREASED; BLOOD CREATINE PHOSPHOKINASE INCREASED; BLOOD MINE CULTURE POSITIVE; BLOOD GLUCOSE INCREASED; BLOOD PH DECREASED; BLOOD PRESSURE DECREASED; BODY TEMPERATURE

INCREASED; BRADYCARDIA; BRAIN DEATH; BRAIN OEDEMA; BUCCAL CAVITY PAPILLOMA; BUNDLE BRANCH BLOCK RIGHT; CARDIAC ARREST; CARDIO-RESPIRATORY ARREST; CENTRAL VENOUS PRESSURE INCREASED; COMA; COMPLETED SUICIDE; COUGH; CRYING; CYANOSIS; DEATH; DELIRIUM; DIARRHOEA; DILATATION VENTRICULAR; DISORIENTATION; DROOLING; DRUG LEVEL ABOVE THERAPEUTIC; DRUG SCREEN POSITIVE; DRUG TOXICITY; DRY SKIN; DYSKINESIA; DYSPHORIA; DYSPNOEA; ELECTROCARDIOGRAM ABNORMAL; ELECTROCARDIOGRAM QRS COMPLEX ABNORMAL; ELECTROCARDIOGRAM QT PROLONGED; ELECTROCARDIOGRAM ST-T CHANGE; ERYTHEMA; EYE SWELLING; FAECES DISCOLOURED; FALL; FEELING COLD; FEELING HOT; FLUSHING; FORMICATION; FUNGAL INFECTION; GAMMA-GLUTAMYLTRANSFERASE INCREASED; GLOSSODYNIA; GRAND MAL CONVULSION; HALLUCINATION; HALLUCINATIONS, MIXED; HEART RATE INCREASED; HEART SOUNDS ABNORMAL; HEPATIC ENZYME INCREASED; HYPERHIDROSIS; HYPERREFLEXIA; HYPOTENSION; HYPOXIC-ISCHAEMIC ENCEPHALOPATHY; INCOHERENT; INCORRECT DOSE ADMINISTERED; INTERCOSTAL RETRACTION; LABORATORY TEST ABNORMAL; LETHARGY; LOSS OF CONSCIOUSNESS; MENTAL STATUS CHANGES; METABOLIC ACIDOSIS; MUSCLE RIGIDITY; MYDRIASIS; MYOCARDIAL ISCHAEMIA; MYOCLONUS; NYSTAGMUS; ORAL DISCOMFORT; ORAL MUCOSAL BLISTERING; PAPILLOMA; PCO2 DECREASED; PO2 DECREASED; POSTICTAL STATE; PRODUCT QUALITY ISSUE; PULMONARY OEDEMA; PULSE ABNORMAL; PUPIL FIXED; PUPILLARY REFLEX IMPAIRED; RASH ERYTHEMATOUS; RESPIRATORY ARREST; RESPIRATORY RATE INCREASED; RESTLESSNESS; RHABDOMYOLYSIS; RIGHT VENTRICULAR SYSTOLIC PRESSURE INCREASED; SCAR; SHOCK; SINUS TACHYCARDIA; SKIN PAPILLOMA; SPEECH DISORDER; STAPHYLOCOCCAL INFECTION; STATUS EPILEPTICUS; SUDDEN INFANT DEATH SYNDROME; SUICIDE ATTEMPT; SWOLLEN TONGUE; TACHYCARDIA; TACHYPNOEA; THERAPEUTIC AGENT TOXICITY; THERMAL BURN; TONGUE BLISTERING; TONGUE DISORDER; TRICUSPID VALVE INCOMPETENCE; UNRESPONSIVE TO VERBAL STIMULI; VENTRICULAR TACHYCARDIA; VOMITING; WHITE BLOOD CELL COUNT INCREASED; ASTHMA; BLOOD GLUCOSE INCREASED; BLOOD PRESSURE INCREASED; CARDIO-RESPIRATORY ARREST; COMA; DEVICE BREAKAGE; DEVICE CONNECTION ISSUE; DEVICE FAILURE; DEVICE MALFUNCTION; DRUG EFFECT DECREASED; FALSE POSITIVE LABORATORY RESULT; FEELING COLD; FLUID OVERLOAD; FOOD ALLERGY; HEADACHE; HYPOKALAEMIA; INCORRECT DOSE ADMINISTERED; INCORRECT DRUG ADMINISTRATION DURATION; EPINEPHRINE INCORRECT ROUTE OF DRUG ADMINISTRATION; INJECTION SITE ANAESTHESIA; INJECTION SITE DISCOLOURATION; INJECTION SITE HAEMORRHAGE; INJECTION SITE STINGING; INJURY; LONG QT SYNDROME; MEDICAL DEVICE COMPLICATION; NEONATAL DISORDER; NO THERAPEUTIC RESPONSE; NODAL RHYTHM; PALLOR; PHARMACEUTICAL PRODUCT COMPLAINT; PNEUMOTHORAX; POOR PERIPHERAL CIRCULATION; POSTERIOR REVERSIBLE ENCEPHALOPATHY SYNDROME; PRODUCT QUALITY ISSUE;

PULMONARY OEDEMA; RHABDOMYOLYSIS; SCREAMING; TREATMENT FAILURE; VENTRICULAR TACHYCARDIA; CYANOSIS; FEELING JITTERY; HEART RATE DECREASED; EPINEPHRINE& HYPERHIDROSIS; METHAEMOGLOBINAEMIA; PALLOR; POST LIDOCAINE PROCEDURAL PAIN; SINUS BRADYCARDIA; ABDOMINAL DISTENSION; ABDOMINAL PAIN; ABNORMAL DREAMS; ACIDOSIS; BLOOD GLUCOSE INCREASED; BLOOD PRESSURE DIASTOLIC DECREASED; CARDIAC ARREST; COMA; CONFUSIONAL STATE; CONVULSION; COORDINATION ABNORMAL; CYANOSIS; DEATH; DEPRESSED LEVEL OF CONSCIOUSNESS; DISSOCIATION; DIZZINESS; DRUG LEVEL INCREASED; DRUG TOXICITY; DYSPNOEA; FEELING COLD; GINGIVAL DISCOLOURATION; HALLUCINATION; HEART RATE INCREASED; HYPERTENSION; HYPOTENSION; INCORRECT FENTANYL DOSE ADMINISTERED; LACERATION; LETHARGY; LOSS OF CITRATE CONSCIOUSNESS; MIOSIS; MUSCLE RIGIDITY; MYDRIASIS; NASAL DISCOMFORT; NAUSEA; PERIPHERAL COLDNESS; PNEUMONIA ASPIRATION; PNEUMONITIS; PYREXIA; REFUSAL OF TREATMENT BY RELATIVE; RESPIRATORY ARREST; RESPIRATORY DEPRESSION; RESPIRATORY RATE INCREASED; SCREAMING; SKIN DISCOLOURATION; SLUGGISHNESS; SOMNOLENCE; STOMACH DISCOMFORT; SUICIDE ATTEMPT; TACHYCARDIA; TONGUE DISCOLOURATION; UNRESPONSIVE TO STIMULI; VASCULAR ACCESS COMPLICATION; VERTIGO; VOMITING; ABDOMINAL PAIN UPPER; ALKALOSIS HYPOCHLORAEMIC; ALKALOSIS HYPOKALAEMIC; BACK PAIN; BLOOD ALDOSTERONE INCREASED; BLOOD ELECTROLYTES DECREASED; BLOOD PRESSURE DECREASED; COUGH; DEPRESSED LEVEL OF CONSCIOUSNESS; DIZZINESS; DRUG ABUSE; ELECTROCARDIOGRAM NORMAL; ENCEPHALOPATHY; HEADACHE; HEART RATE INCREASED; HYPERCAPNIA; FUROSEMIDE HYPERCHOLESTEROLAEMIA; HYPOALBUMINAEMIA; HYPOCHLORAEMIA; HYPOKALAEMIA; HYPONATRAEMIA; HYPOTENSION; HYPOVOLAEMIA; METABOLIC ACIDOSIS; METABOLIC ALKALOSIS; MUSCULAR WEAKNESS; NEPHROTIC SYNDROME; NOCTURIA; PYELOCALIECTASIS; PYELOPLASTY; RESPIRATORY FAILURE; URINE CALCIUM/CREATININE RATIO INCREASED; URINE CHLORIDE INCREASED; VOMITING; WEIGHT DECREASED; ACTIVATED PARTIAL THROMBOPLASTIN TIME PROLONGED; AMAUROSIS FUGAX; ASCITES; BLEEDING TIME PROLONGED; CARDIAC ARREST; CARDIOGENIC SHOCK; DEATH; DEATH NEONATAL; DEVICE OCCLUSION; DRUG EFFECT DELAYED; DRUG EFFECT INCREASED; EPISTAXIS; GASTROINTESTINAL HAEMORRHAGE; GENERAL PHYSICAL HEALTH DETERIORATION; HAEMODYNAMIC INSTABILITY; HAEMORRHAGE INTRACRANIAL; HEMIPLEGIA TRANSIENT; HEPARIN- HEPARIN INDUCED THROMBOCYTOPENIA; HYPOXIA; INTERNATIONAL NORMALISED RATIO INCREASED; INTRACARDIAC THROMBUS; MULTI- ORGAN FAILURE; NEONATAL DISORDER; PERICARDIAL EFFUSION; PERITONITIS; PHARMACEUTICAL PRODUCT COMPLAINT; PHLEBITIS; PLEURAL EFFUSION; PNEUMONIA; POST PROCEDURAL COMPLICATION; PROCEDURAL COMPLICATION; PRODUCT QUALITY ISSUE; PULMONARY EMBOLISM; PULMONARY HAEMORRHAGE; PULMONARY INFARCTION; RETROPERITONEAL HAEMORRHAGE;

SEPSIS; SHUNT OCCLUSION; SHUNT THROMBOSIS; SUBCLAVIAN VEIN THROMBOSIS; SUPERIOR VENA CAVAL OCCLUSION; THROMBOSIS IN DEVICE; VENA CAVA THROMBOSIS; ADRENOCORTICAL INSUFFICIENCY ACUTE; ANGIONEUROTIC OEDEMA; APPLICATION SITE BURN; APPLICATION SITE ERYTHEMA; BRONCHOSPASM; CARDIAC DISORDER; CARDIAC HYPERTROPHY; CATARACT; CATARACT SUBCAPSULAR; CELLULITIS; CONDITION AGGRAVATED; DIARRHOEA; DYSPHONIA; ENDOPHTHALMITIS; EYE INFECTION STAPHYLOCOCCAL; GASTROENTERITIS ROTAVIRUS; GENERAL PHYSICAL HEALTH DETERIORATION; HYPERTENSION; HYDROCORTIS HYPERTROPHIC CARDIOMYOPATHY; INJURY; INTRAOCULAR LENS ONE IMPLANT; KLEBSIELLA BACTERAEMIA; LEFT VENTRICLE OUTFLOW TRACT OBSTRUCTION; LEFT VENTRICULAR HYPERTROPHY; LYMPHADENOPATHY; LYMPHANGITIS; LYMPHOPENIA; NEONATAL RESPIRATORY DISTRESS SYNDROME; NEUTROPHILIA; NOONAN SYNDROME; OSTEOPOROTIC FRACTURE; POST PROCEDURAL INFECTION; PRODUCT SUBSTITUTION ISSUE; PSYCHOTIC DISORDER; STAPHYLOCOCCAL INFECTION; SUICIDAL IDEATION; TREMOR; VISION BLURRED; VISUAL ACUITY REDUCED; HYDROMORPH COMPLETED SUICIDE; NARCOTIC INTOXICATION; PNEUMONIA; ONE PULMONARY CONGESTION; ABDOMINAL DISTENSION; ABDOMINAL RIGIDITY; ABDOMINAL TENDERNESS; ACIDOSIS; ACQUIRED DIAPHRAGMATIC EVENTRATION; ACTIVATED PARTIAL THROMBOPLASTIN TIME PROLONGED; ACUTE RESPIRATORY DISTRESS SYNDROME; AGGRESSION; ALANINE AMINOTRANSFERASE INCREASED; ANAPHYLACTIC REACTION; ANAPHYLACTIC SHOCK; ANGIOEDEMA; ANION GAP ABNORMAL; ANION GAP INCREASED; AORTIC VALVE REPAIR; APNOEA; AREFLEXIA; ASPARTATE AMINOTRANSFERASE INCREASED; ASPIRATION; ASTHENIA; ASTHMA; ATAXIA; ATELECTASIS; ATRIAL FIBRILLATION; BACTERIAL INFECTION; BALANITIS; BILE OUTPUT; BILIARY TRACT DISORDER; BLISTER; BLOOD ALKALINE PHOSPHATASE INCREASED; BLOOD AMYLASE INCREASED; BLOOD ANTIDIURETIC HORMONE INCREASED; BLOOD BICARBONATE ABNORMAL; BLOOD BILIRUBIN INCREASED; BLOOD COPPER INCREASED; BLOOD CREATINE INCREASED; BLOOD IBUPROFEN IMMUNOGLOBULIN E INCREASED; BLOOD LACTIC ACID INCREASED; BLOOD POTASSIUM DECREASED; BLOOD PRESSURE DECREASED; BLOOD PRESSURE DIASTOLIC DECREASED; BLOOD SODIUM DECREASED; BLOOD UREA INCREASED; BRADYCARDIA; BURNING SENSATION; CARDIAC DISORDER; CARDIAC ENZYMES INCREASED; CARDIAC PERFORATION; CARDIAC VALVE ABSCESS; CARDIAC VALVE VEGETATION; CARDIOMYOPATHY; CARDIOTOXICITY; CEREBRAL HAEMORRHAGE; CEREBRAL HAEMORRHAGE FOETAL; CEREBRAL HAEMORRHAGE NEONATAL; CERULOPLASMIN INCREASED; CHEST X- RAY ABNORMAL; CHOLESTASIS; CIRCULATORY COLLAPSE; COAGULATION FACTOR VIII LEVEL DECREASED; COAGULOPATHY; COLLAGEN DISORDER; COMA; COMPLETED SUICIDE; CONDITION AGGRAVATED; CONFABULATION; CONFUSIONAL STATE; CONJUNCTIVITIS; CONVULSION; COUGH; C-REACTIVE PROTEIN INCREASED; CRYING; CYANOSIS; CYTOLYTIC HEPATITIS; DEATH;

DEATH NEONATAL; DECREASED APPETITE; DEHYDRATION; DEPRESSED LEVEL OF CONSCIOUSNESS; DIARRHOEA; DILATATION VENTRICULAR; DIPLOPIA; DISORIENTATION; DIZZINESS; DRUG EFFECT DECREASED; DRUG ERUPTION; DRUG HYPERSENSITIVITY; DRUG INTOLERANCE; DRUG LEVEL ABOVE THERAPEUTIC; DRUG LEVEL INCREASED; DRUG RASH WITH EOSINOPHILIA AND SYSTEMIC SYMPTOMS; DRUG SCREEN FALSE POSITIVE; DRUG TOXICITY; DRY SKIN; DUODENAL PERFORATION; DUODENAL ULCER PERFORATION; DYSKINESIA; DYSPNOEA; ELECTROCARDIOGRAM QT PROLONGED; ELECTROCARDIOGRAM ST SEGMENT DEPRESSION; ELECTROENCEPHALOGRAM ABNORMAL; ELECTROLYTE IMBALANCE; ENCEPHALOPATHY; ENDOCARDITIS STAPHYLOCOCCAL; ERYSIPELAS; ERYTHEMA; ERYTHEMA MULTIFORME; EXTRASYSTOLES; EYE DISORDER; EYE SWELLING; EYELID OEDEMA; FACE OEDEMA; FEELING COLD; FLATULENCE; FLUID RETENTION; FOOD INTOLERANCE; FOREIGN BODY TRAUMA; GALLBLADDER DISORDER; GASTRIC DILATATION; GASTRIC HAEMORRHAGE; GASTRIC MUCOSAL HYPERTROPHY; GASTRIC ULCER; GASTRIC ULCER HAEMORRHAGE; GASTRITIS; GASTRODUODENITIS; GASTROINTESTINAL HAEMORRHAGE; GENERAL PHYSICAL HEALTH DETERIORATION; GLASGOW COMA SCALE ABNORMAL; GOITRE; GRAND MAL CONVULSION; HAEMARTHROSIS; HAEMATEMESIS; HAEMATOCHEZIA; HAEMODIALYSIS; HAEMOGLOBIN ABNORMAL; HAEMOGLOBIN DECREASED; HAEMOGLOBIN INCREASED; HAEMOLYSIS; HAEMOPTYSIS; HAEMORRHAGE; HAEMORRHAGE NEONATAL; HEART RATE INCREASED; HEPATIC ENZYME INCREASED; HEPATIC FAILURE; HEPATIC HAEMORRHAGE; HEPATITIS; HYDRONEPHROSIS; HYPERCHOLESTEROLAEMIA; HYPERNATRAEMIA; HYPERSENSITIVITY; HYPERTENSION; HYPERVENTILATION; HYPONATRAEMIA; HYPOPHAGIA; HYPOREFLEXIA; HYPOTENSION; HYPOTHERMIA; HYPOTONIA; HYPOXIA; IDIOPATHIC THROMBOCYTOPENIC PURPURA; IDIOSYNCRATIC DRUG REACTION; IMPAIRED GASTRIC EMPTYING; INCORRECT DOSE ADMINISTERED; INFECTION IN AN IMMUNOCOMPROMISED HOST; INTENTIONAL SELF-INJURY; INTERNATIONAL NORMALISED RATIO INCREASED; INTESTINAL PERFORATION; INTRA-ABDOMINAL HAEMORRHAGE; INTRAVENTRICULAR HAEMORRHAGE NEONATAL; JOINT RANGE OF MOTION DECREASED; JOINT SWELLING; KIDNEY ENLARGEMENT; LETHARGY; LEUKOCYTOSIS; LIPASE INCREASED; LIVER DISORDER; LIVER INJURY; LOCAL SWELLING; LOSS OF CONSCIOUSNESS; LUNG CONSOLIDATION; LUNG DISORDER; MALAISE; MENINGITIS ASEPTIC; MENTAL STATUS CHANGES; METABOLIC ACIDOSIS; METHAEMOGLOBINAEMIA; MIOSIS; MITRAL VALVE REPAIR; MOBILITY DECREASED; MUCOSAL EROSION; MUCOSAL INFLAMMATION; MULTIPLE DRUG OVERDOSE; MUSCLE CONTRACTURE; MUSCLE RIGIDITY; NASAL OEDEMA; NASOPHARYNGITIS; NAUSEA; NECROTISING COLITIS; NECROTISING ENTEROCOLITIS NEONATAL; NECROTISING FASCIITIS; NEONATAL DISORDER; NEUTROPHIL MORPHOLOGY ABNORMAL; NEUTROPHIL PELGER-HUET ANOMALY PRESENT; NEUTROPHILIA; NO THERAPEUTIC RESPONSE; OCULAR HYPERAEMIA; OCULAR ICTERUS; ODYNOPHAGIA;

OEDEMA; OEDEMA MOUTH; OEDEMA NEONATAL; OESOPHAGEAL INJURY; OESOPHAGEAL PERFORATION; OLIGURIA; OPERATIVE HAEMORRHAGE; OTITIS MEDIA; OXYGEN SATURATION DECREASED; OXYGEN SATURATION NORMAL; PALLOR; PANCREATITIS ACUTE; PATENT DUCTUS ARTERIOSUS; PELVIC VENOUS THROMBOSIS; PERICARDITIS; PERITONEAL DIALYSIS; PERITONITIS; PETECHIAE; PETIT MAL EPILEPSY; PHARYNGEAL ABSCESS; PLATELET COUNT DECREASED; PLATELET FUNCTION TEST ABNORMAL; PLEURAL DISORDER; PLEURAL EFFUSION; PLEURISY; PNEUMONIA ASPIRATION; PNEUMONIA PNEUMOCOCCAL; PNEUMONIA STREPTOCOCCAL; PNEUMONITIS; PNEUMOPERITONEUM; POOR PERIPHERAL CIRCULATION; PROTHROMBIN LEVEL DECREASED; PULMONARY EMBOLISM; PULMONARY HAEMORRHAGE; PULMONARY HYPERTENSION; PULMONARY OEDEMA; PULMONARY OEDEMA NEONATAL; PULMONARY VASCULAR DISORDER; PULSE ABSENT; PULSE PRESSURE DECREASED; PUPILLARY REFLEX IMPAIRED; PYELONEPHRITIS; PYLORIC STENOSIS; RASH; RASH GENERALISED; RECTAL HAEMORRHAGE; RENAL FAILURE; RENAL FAILURE ACUTE; RENAL FAILURE NEONATAL; RENAL FUNCTION TEST ABNORMAL; RENAL IMPAIRMENT; RENAL PAPILLARY NECROSIS; RENAL TUBULAR NECROSIS; RESPIRATORY ACIDOSIS; RESPIRATORY DISTRESS; RESPIRATORY RATE INCREASED; RETCHING; RHABDOMYOLYSIS; RIGHT ATRIAL DILATATION; RIGHT VENTRICULAR SYSTOLIC PRESSURE; RIGHT VENTRICULAR SYSTOLIC PRESSURE INCREASED; SCAR; SEPSIS; SEPTIC EMBOLUS; SINUS BRADYCARDIA; SINUS TACHYCARDIA; SKIN DISCOLOURATION; SKIN HAEMORRHAGE; SKIN LESION; SKIN NECROSIS; SKIN REACTION; SKIN SWELLING; SMOOTH MUSCLE ANTIBODY POSITIVE; SPEECH DISORDER; STAPHYLOCOCCAL ABSCESS; STAPHYLOCOCCAL INFECTION; STAPHYLOCOCCAL SEPSIS; STATUS ASTHMATICUS; STEVENS-JOHNSON SYNDROME; STREPTOCOCCAL INFECTION; SUBCUTANEOUS ABSCESS; SUBCUTANEOUS EMPHYSEMA; SUICIDE ATTEMPT; SUPERINFECTION; SWELLING FACE; SYSTEMIC LUPUS ERYTHEMATOSUS; TACHYCARDIA; TACHYPNOEA; THERAPEUTIC RESPONSE DECREASED; THERAPEUTIC RESPONSE UNEXPECTED; THROMBOCYTOPENIA; TOXIC EPIDERMAL NECROLYSIS; TOXIC SHOCK SYNDROME STREPTOCOCCAL; TRANSAMINASES INCREASED; TRICUSPID VALVE INCOMPETENCE; TROPONIN INCREASED; UNRESPONSIVE TO STIMULI; URINE ANALYSIS ABNORMAL; URINE SODIUM DECREASED; URTICARIA; URTICARIA GENERALISED; VANISHING BILE DUCT SYNDROME; VARICELLA; VASCULAR RESISTANCE PULMONARY INCREASED; VOMITING; VULVAL ULCERATION; VULVITIS; WEIGHT DECREASED; WHITE BLOOD CELL COUNT DECREASED; WHITE BLOOD CELL COUNT INCREASED; XANTHOGRANULOMA; ANAPHYLACTIC REACTION; BLOOD GLUCOSE DECREASED; BLOOD GLUCOSE INCREASED; CONVULSION; DIABETIC KETOACIDOSIS; DRUG INSULIN EFFECT DECREASED; HYDRONEPHROSIS; HYPERSENSITIVITY; ASPART HYPOGLYCAEMIA; INCORRECT DOSE ADMINISTERED; KETOACIDOSIS; PHARYNGOLARYNGEAL PAIN; INSULIN ACTIVITIES OF DAILY LIVING IMPAIRED; BLOOD GLUCOSE LISPRO DECREASED; BLOOD GLUCOSE FLUCTUATION; BLOOD GLUCOSE

INCREASED; BLOOD INSULIN INCREASED; COMMUNICATION DISORDER; CONDITION AGGRAVATED; CONGENITAL ATRIAL SEPTAL DEFECT; CONGENITAL VENTRICULAR SEPTAL DEFECT; CONVULSION; DEVICE FAILURE; DIABETIC KETOACIDOSIS; DIPLOPIA; DIZZINESS; FATIGUE; HEADACHE; HEREDITARY DISORDER; HYPERBILIRUBINAEMIA; HYPERGLYCAEMIA; HYPOGLYCAEMIA NEONATAL; HYPOGLYCAEMIC SEIZURE; INCORRECT DOSE ADMINISTERED; JAUNDICE NEONATAL; KETOACIDOSIS; MALAISE; MENINGOMYELOCELE; NEONATAL ASPHYXIA; NEONATAL RESPIRATORY ARREST; PHARMACEUTICAL PRODUCT COMPLAINT; PHOTOPHOBIA; PREGNANCY; PREGNANCY WITH ADVANCED MATERNAL AGE; PREMATURE BABY; READING DISORDER; VIRAL INFECTION; VIRAL PHARYNGITIS; VISION BLURRED; VOMITING; ASTHMA; BLOOD PRESSURE DECREASED; BRAIN OEDEMA; CARDIAC PROCEDURE COMPLICATION; CHEST DISCOMFORT; CLONUS; COUGH; CYANOSIS; DYSPNOEA; ELECTROENCEPHALOGRAM ABNORMAL; HAEMATOCRIT DECREASED; HEART RATE INCREASED; HYPERHIDROSIS; HYPERREFLEXIA; HYPERSENSITIVITY; IOVERSOL HYPERTENSION; HYPOKINESIA; HYPOTENSION; IRRITABILITY; LETHARGY; NAUSEA; OXYGEN SATURATION DECREASED; PROCEDURAL COMPLICATION; PRURITUS; RESPIRATORY DISTRESS; RESPIRATORY FAILURE; SIMPLE PARTIAL SEIZURES; SNEEZING; SOMNOLENCE; SWELLING FACE; TACHYCARDIA; THROAT TIGHTNESS; URTICARIA; VOMITING; ABDOMINAL PAIN; AIRWAY COMPLICATION OF ANAESTHESIA; ALANINE AMINOTRANSFERASE INCREASED; ARRHYTHMIA; ASCITES; ASPARTATE AMINOTRANSFERASE INCREASED; BILIARY DILATATION; BLADDER DISORDER; BLADDER HYPERTROPHY; CARDIAC ARREST NEONATAL; CHOLEDOCHAL CYST; CYSTITIS; CYSTITIS ULCERATIVE; DRUG ABUSE; DYSURIA; ELECTROCARDIOGRAM ST SEGMENT DEPRESSION; EYE SWELLING; FOOD INTOLERANCE; GALLBLADDER DISORDER; GAMMA-GLUTAMYLTRANSFERASE INCREASED; GASTRIC KETAMINE MUCOSA ERYTHEMA; HYPERSENSITIVITY; INCORRECT DOSE ADMINISTERED; LARYNGOSPASM; MICTURITION URGENCY; POLLAKIURIA; PULMONARY ARTERIAL PRESSURE INCREASED; RASH; REDUCED BLADDER CAPACITY; SALIVARY HYPERSECRETION; SPHINCTER OF ODDI DYSFUNCTION; SWELLING FACE; TORSADE DE POINTES; URINARY INCONTINENCE; VENTRICULAR FIBRILLATION; VENTRICULAR TACHYCARDIA; VOMITING; WHITE BLOOD CELL COUNT INCREASED; KETOROLAC DYSPNOEA; LIP SWELLING; OLIGURIA; RENAL FAILURE ACUTE; ABDOMINAL PAIN UPPER; ACTIVATED PARTIAL THROMBOPLASTIN TIME PROLONGED; AGITATION NEONATAL; BACTERIAL INFECTION; BLOOD CALCIUM INCREASED; BLOOD CREATININE INCREASED; BLOOD MAGNESIUM INCREASED; BLOOD PARATHYROID HORMONE LANSOPRAZOL DECREASED; BLOOD UREA INCREASED; CHILLS; COELIAC DISEASE; E COLITIS; CROHN'S DISEASE; DEHYDRATION; DIARRHOEA; EPISTAXIS; EYE DISORDER; EYE HAEMORRHAGE; FALL; GAIT DISTURBANCE; HALLUCINATION, VISUAL; HEADACHE; INCORRECT DOSE ADMINISTERED; LETHARGY; MICROCYTIC ANAEMIA; MYALGIA; NO

THERAPEUTIC RESPONSE; NYSTAGMUS; POST PROCEDURAL HAEMORRHAGE; PREMATURE BABY; PRODUCT FORMULATION ISSUE; PROTEIN-LOSING GASTROENTEROPATHY; PROTEINURIA; PROTHROMBIN TIME PROLONGED; PYREXIA; RENAL DISORDER; RENAL FAILURE; RICKETS; SEPSIS; SUDDEN INFANT DEATH SYNDROME; THROMBOCYTOPENIA; TIBIA FRACTURE; TORUS FRACTURE; TREATMENT FAILURE; TUBULOINTERSTITIAL NEPHRITIS; URTICARIA; VIRAL INFECTION; VITAMIN D DEFICIENCY; VOMITING; ABASIA; ABDOMINAL PAIN; ABDOMINAL PAIN UPPER; ABNORMAL BEHAVIOUR; ACETONAEMIC VOMITING; AGGRESSION; AGITATION; AKINESIA; ALANINE AMINOTRANSFERASE INCREASED; ALOPECIA; ANAPHYLACTIC REACTION; ANGIONEUROTIC OEDEMA; ANOREXIA; ANTICONVULSANT DRUG LEVEL ABOVE THERAPEUTIC; ANTICONVULSANT DRUG LEVEL BELOW THERAPEUTIC; ANTICONVULSANT DRUG LEVEL DECREASED; ANXIETY; APATHY; APHASIA; APNOEA; APNOEIC ATTACK; ARTHROPATHY; ASPARTATE AMINOTRANSFERASE INCREASED; ASPHYXIA; ATAXIA; ATHETOSIS; ATONIC SEIZURES; ATTENTION DEFICIT/HYPERACTIVITY DISORDER; AUTOIMMUNE THYROIDITIS; BALANCE DISORDER; BLINDNESS; BLINDNESS UNILATERAL; BLOOD ALKALINE PHOSPHATASE INCREASED; BLOOD AMYLASE INCREASED; BLOOD CREATINE PHOSPHOKINASE INCREASED; BLOOD CREATINE PHOSPHOKINASE MB INCREASED; BLOOD CREATININE INCREASED; BLOOD LACTATE DEHYDROGENASE INCREASED; BLOOD UREA NITROGEN/CREATININE RATIO INCREASED; BODY TEMPERATURE; BONE MARROW FAILURE; BREAST MASS; CARDIO-RESPIRATORY ARREST; CHOLECYSTECTOMY; CHOREA; CLUMSINESS; COGNITIVE DISORDER; COMMUNICATION DISORDER; COMPLETED SUICIDE; CONCUSSION; CONDITION AGGRAVATED; CONFUSIONAL STATE; CONGENITAL FOOT LEVETIRACETA MALFORMATION; CONTUSION; CRYING; CYANOSIS; DEATH; DEATH M NEONATAL; DEHYDRATION; DEPRESSED LEVEL OF CONSCIOUSNESS; DEPRESSION; DEVELOPMENTAL DELAY; DIARRHOEA; DIPLOPIA; DIZZINESS; DROPPED HEAD SYNDROME; DROWNING; DRUG EFFECT DECREASED; DRUG HYPERSENSITIVITY; DRUG INTOLERANCE; DRUG LEVEL INCREASED; DRUG TOXICITY; DYSKINESIA; DYSPHEMIA; DYSTONIA; ELECTROENCEPHALOGRAM ABNORMAL; ENCEPHALITIS; ENCEPHALITIS AUTOIMMUNE; ENCEPHALOPATHY; ENTEROCOLITIS HAEMORRHAGIC; EOSINOPHILIA; EPILEPTIC PSYCHOSIS; EPSTEIN- BARR VIRUS INFECTION; ERYTHEMA; ESCHERICHIA SEPSIS; EYELID OEDEMA; FACE OEDEMA; FALL; FALSE POSITIVE INVESTIGATION RESULT; FATIGUE; FEAR; FEAR OF DEATH; FEEDING DISORDER; FLANK PAIN; FLAT AFFECT; FRACTURE; GAIT DISTURBANCE; GASTROINTESTINAL HAEMORRHAGE; GAZE PALSY; GLOMERULONEPHRITIS PROLIFERATIVE; GRAND MAL CONVULSION; GROWTH RETARDATION; HAEMATOMA; HAEMOCHROMATOSIS; HAEMOGLOBIN DECREASED; HAEMOLYTIC ANAEMIA; HALLUCINATION; HEAD INJURY; HEADACHE; HEART RATE INCREASED; HEPATIC ENZYME INCREASED; HEPATIC FUNCTION ABNORMAL; HEPATITIS; HYDRONEPHROSIS; HYPERSENSITIVITY; HYPOPHOSPHATAEMIA; HYPORESPONSIVE TO STIMULI; HYPOTHERMIA; HYPOTONIA; IATROGENIC INJURY; IMPULSE-

CONTROL DISORDER; INCORRECT DOSE ADMINISTERED; INCORRECT ROUTE OF DRUG ADMINISTRATION; INFECTION; INFLAMMATION; INJURY; INSOMNIA; IRRITABILITY; JAUNDICE; JUVENILE MYOCLONIC EPILEPSY; KIDNEY FIBROSIS; LACTIC ACIDOSIS; LETHARGY; LEUKAEMIA; LEUKOPENIA; LIMB REDUCTION DEFECT; LIPASE INCREASED; LIVER DISORDER; MAJOR DEPRESSION; MENTAL DISABILITY; MENTAL STATUS CHANGES; METABOLIC ACIDOSIS; METABOLIC ENCEPHALOPATHY; MITOCHONDRIAL CYTOPATHY; MOOD ALTERED; MOTOR DYSFUNCTION; MUSCLE TWITCHING; MYDRIASIS; MYOCLONIC EPILEPSY; MYOCLONUS; NAUSEA; NEONATAL DISORDER; NEUTROPENIA; NO THERAPEUTIC RESPONSE; NUCLEAR MAGNETIC RESONANCE IMAGING ABNORMAL; OLIGOHYDRAMNIOS; OPSOCLONUS MYOCLONUS; OPTIC ATROPHY; OXYGEN SATURATION DECREASED; PALLOR; PANCREATITIS; PANIC ATTACK; PARADOXICAL DRUG REACTION; PERSONALITY CHANGE; PETIT MAL EPILEPSY; PHARMACEUTICAL PRODUCT COMPLAINT; PHYSICAL ASSAULT; PLACENTA PRAEVIA; PLEOCYTOSIS; POSTURE ABNORMAL; PRECOCIOUS PUBERTY; PREGNANCY; PREMATURE BABY; PRESYNCOPE; PRODUCT QUALITY ISSUE; PRODUCT SUBSTITUTION ISSUE; PROTEIN URINE PRESENT; PRURITUS; PSYCHOTIC DISORDER; PYREXIA; RASH; RED BLOOD CELLS URINE POSITIVE; REGRESSIVE BEHAVIOUR; RENAL DISORDER; RENAL FAILURE; RENAL FAILURE ACUTE; RENAL PAPILLARY NECROSIS; RENAL TUBULAR DISORDER; RESPIRATORY ARREST; RESPIRATORY DISTRESS; RESPIRATORY FAILURE; RHINITIS; SALIVA ALTERED; SALIVARY HYPERSECRETION; SCAR; SENSORY INTEGRATIVE DYSFUNCTION; SEPSIS; SKIN DISCOLOURATION; SOCIAL AVOIDANT BEHAVIOUR; SOMNOLENCE; SPEECH DISORDER; SPEECH DISORDER DEVELOPMENTAL; STAPHYLOCOCCAL SEPSIS; STATUS EPILEPTICUS; STEREOTYPY; SUBARACHNOID HAEMORRHAGE; SUDDEN DEATH; SUDDEN UNEXPLAINED DEATH IN EPILEPSY; SUICIDAL IDEATION; SUPRAVENTRICULAR TACHYCARDIA; SYRINGE ISSUE; THERMAL BURN; THROMBOCYTOPENIA; TONIC CONVULSION; TOXIC ENCEPHALOPATHY; TRANSAMINASES INCREASED; TREATMENT FAILURE; TREMOR; TUBULOINTERSTITIAL NEPHRITIS; URETERAL STENT INSERTION; URETERIC OBSTRUCTION; URINARY TRACT INFECTION; VENTOUSE EXTRACTION; VESICOURETERIC REFLUX; VIRAL INFECTION; VISUAL ACUITY REDUCED; VISUAL IMPAIRMENT; VOMITING; WHITE BLOOD CELL COUNT DECREASED; WHITE BLOOD CELL COUNT INCREASED; WHITE BLOOD CELLS URINE POSITIVE; ANAPHYLACTIC REACTION; APNOEA; AREFLEXIA; BRADYCARDIA; BRAIN INJURY; CARDIAC ARREST; CARDIO-RESPIRATORY ARREST; COMA; COMPLETED SUICIDE; CONVULSION; CONVULSION NEONATAL; CYANOSIS; DEATH NEONATAL; DECEREBRATION; DIZZINESS; DRUG HYPERSENSITIVITY; DRUG TOXICITY; DYSPNOEA; EAR PAIN; LIDOCAINE ELECTROENCEPHALOGRAM ABNORMAL; EPIGLOTTITIS; EYELID DISORDER; EYELID OEDEMA; FALL; FATIGUE; GRAND MAL CONVULSION; HAEMORRHAGE; HEADACHE; HYPERTONIA; HYPOACUSIS; HYPOAESTHESIA; HYPOTENSION; INFLAMMATION; INSOMNIA; LAGOPHTHALMOS; LEUKOCYTOSIS; LUNG INFILTRATION; METHAEMOGLOBINAEMIA; NAUSEA; NECK PAIN; NEONATAL

DISORDER; NERVOUS SYSTEM DISORDER; OSTEOMYELITIS; PALLOR; PARAESTHESIA; PENILE HAEMORRHAGE; POST PROCEDURAL COMPLICATION; PRODUCT QUALITY ISSUE; PUPIL FIXED; RESPIRATORY ARREST; RESPIRATORY DISTRESS; SCAR; SINUSITIS; SKIN DISCOLOURATION; SKIN EXFOLIATION; SKIN NECROSIS; SUDDEN DEATH; TREMOR; TUNNEL VISION; VISUAL IMPAIRMENT; ACQUIRED METHAEMOGLOBINAEMIA; BRADYCARDIA; CONVULSION; CYANOSIS; DEPRESSED LEVEL OF CONSCIOUSNESS; DRUG TOXICITY; LIDOCAINE&PR HYPOTENSION; HYPOTONIA; MALAISE; OXYGEN SATURATION ILOCAINE DECREASED; PALLOR; PENILE ERYTHEMA; PENILE SWELLING; RASH ERYTHEMATOUS; SALIVARY HYPERSECRETION; ABNORMAL BEHAVIOUR; ACIDOSIS; BALANCE DISORDER; CONDITION AGGRAVATED; CONFUSIONAL STATE; CYANOSIS NEONATAL; DAYDREAMING; DEPRESSED LEVEL OF CONSCIOUSNESS; DIZZINESS; DRUG SCREEN POSITIVE; DYSARTHRIA; DYSSTASIA; ELECTROENCEPHALOGRAM ABNORMAL; EYE DISORDER; FATIGUE; GAIT DISTURBANCE; HALLUCINATION; HEADACHE; HEART RATE INCREASED; HEART RATE IRREGULAR; HYPOTONIA NEONATAL; LORAZEPAM HYPOVENTILATION; INFANTILE APNOEIC ATTACK; LOSS OF CONSCIOUSNESS; METABOLIC ACIDOSIS; METABOLIC DISORDER; MOVEMENT DISORDER; MYDRIASIS; NEONATAL DISORDER; NYSTAGMUS; POOR SUCKING REFLEX; RESPIRATORY ACIDOSIS; RESPIRATORY ARREST; RESPIRATORY DEPRESSION; SEROTONIN SYNDROME; SOMNOLENCE; SYSTOLIC HYPERTENSION; TACHYCARDIA; VISUAL DISTURBANCE; ACIDOSIS; ACUTE MYOCARDIAL INFARCTION; ACUTE RESPIRATORY DISTRESS SYNDROME; APNOEA; ASPHYXIA; BAROTRAUMA; BASE EXCESS ABNORMAL; BASE EXCESS NEGATIVE; BLOOD BICARBONATE ABNORMAL; BLOOD BICARBONATE DECREASED; BLOOD GLUCOSE ABNORMAL; BLOOD GLUCOSE INCREASED; BLOOD LACTIC ACID INCREASED; BLOOD PH DECREASED; BLOOD PH INCREASED; BLOOD PRESSURE DECREASED; BODY TEMPERATURE DECREASED; BRADYCARDIA; BRADYPNOEA; BRAIN HERNIATION; BRAIN OEDEMA; BREATH SOUNDS ABNORMAL; BRONCHOPNEUMONIA; CARDIAC ARREST; CARDIAC DISORDER; CARDIOGENIC SHOCK; CARDIO- RESPIRATORY ARREST; CEREBRAL ATROPHY; CEREBRAL CALCIFICATION; CEREBRAL INFARCTION; CEREBRAL ISCHAEMIA; CIRCULATORY COLLAPSE; COMA; COMPLETED SUICIDE; METHADONE COMPUTERISED TOMOGRAM ABNORMAL; CONVULSION; COORDINATION ABNORMAL; CYANOSIS; DEATH; DEPRESSED LEVEL OF CONSCIOUSNESS; DIABETES MELLITUS INSULIN-DEPENDENT; DRUG SCREEN POSITIVE; DRUG TOXICITY; DYSARTHRIA; DYSKINESIA; DYSPNOEA; ELECTROMECHANICAL DISSOCIATION; EMPHYSEMA; ENCEPHALITIS; ENCEPHALOPATHY; FATIGUE; GAZE PALSY; GENERALISED OEDEMA; GLASGOW COMA SCALE ABNORMAL; HEAD CIRCUMFERENCE ABNORMAL; HEMIPLEGIA; HEPATIC FAILURE; HEPATOMEGALY; HYDROCEPHALUS; HYPERCAPNIA; HYPERGLYCAEMIA; HYPERHIDROSIS; HYPOPNOEA; HYPOTENSION; HYPOTHERMIA; HYPOXIA; HYPOXIC ENCEPHALOPATHY; INJURY; INTRACRANIAL PRESSURE INCREASED; JUGULAR VEIN DISTENSION; LIVIDITY; LOSS OF CONSCIOUSNESS; LUNG INFILTRATION;

METABOLIC ACIDOSIS; MIOSIS; MULTI-ORGAN DISORDER; MULTI- ORGAN FAILURE; MULTIPLE DRUG OVERDOSE; MUSCLE RIGIDITY; MUSCLE SPASTICITY; MUTISM; MYDRIASIS; NARCOTIC INTOXICATION; NASOPHARYNGITIS; NERVOUS SYSTEM DISORDER; NUCLEAR MAGNETIC RESONANCE IMAGING BRAIN ABNORMAL; OCULOCEPHALOGYRIC REFLEX ABSENT; OXYGEN SATURATION DECREASED; PCO2 INCREASED; PNEUMONIA; PNEUMONIA ASPIRATION; PNEUMONIA HAEMOPHILUS; PO2 DECREASED; POISONING; PULMONARY HAEMORRHAGE; PULMONARY OEDEMA; PUPIL FIXED; PUPILLARY REFLEX IMPAIRED; REFLEXES ABNORMAL; RESPIRATION ABNORMAL; RESPIRATORY ACIDOSIS; RESPIRATORY ARREST; RESPIRATORY DEPRESSION; RESPIRATORY DISORDER; RESPIRATORY FAILURE; RESPIRATORY RATE DECREASED; RESPIRATORY RATE INCREASED; SOMNOLENCE; STUPOR; SUDDEN INFANT DEATH SYNDROME; UNRESPONSIVE TO STIMULI; VENTRICULAR DYSFUNCTION; VENTRICULAR FIBRILLATION; VENTRICULAR HYPERTROPHY; VENTRICULAR TACHYCARDIA; VOMITING; ABDOMINAL PAIN; ANAPHYLACTIC REACTION; ANGIONEUROTIC OEDEMA; ASTHENIA; BLOOD GLUCOSE INCREASED; BLOOD PRESSURE INCREASED; BRADYCARDIA; BRAIN HERNIATION; BRAIN MASS; BRAIN OEDEMA; BRONCHOSPASM; CARDIAC ARREST; CEREBRAL VENOUS THROMBOSIS; CONSTIPATION; CONVULSION; CSF PROTEIN INCREASED; DEPRESSED LEVEL OF CONSCIOUSNESS; GENERAL PHYSICAL HEALTH DETERIORATION; GRAND MAL CONVULSION; HAEMORRHAGE INTRACRANIAL; HAEMORRHAGIC DIATHESIS; HEMIPLEGIA; HYPERTROPHIC CARDIOMYOPATHY; METHYLPREDN HYPERVENTILATION; HYPOTENSION; MENTAL DISORDER; MUSCULAR ISOLONE WEAKNESS; NEOPLASM; NEPHROCALCINOSIS; NEUROGENIC BLADDER; NUCLEAR MAGNETIC RESONANCE IMAGING BRAIN ABNORMAL; OXYGEN SATURATION DECREASED; PALLOR; PERITONEAL DIALYSIS; PNEUMONIA LEGIONELLA; RED BLOOD CELL SEDIMENTATION RATE INCREASED; RENAL FAILURE ACUTE; RESPIRATORY ARREST; RESPIRATORY DISORDER; REVERSIBLE POSTERIOR LEUKOENCEPHALOPATHY SYNDROME; SKIN TEST POSITIVE; TACHYCARDIA; TUMOUR LYSIS SYNDROME; UROSEPSIS; URTICARIA; VOMITING; AGGRESSION; ASPHYXIA; BLOOD CREATINE PHOSPHOKINASE INCREASED; CARDIAC ARREST; CONVULSION NEONATAL; CRYING; DYSTONIA; HYPERSENSITIVITY; HYPORESPONSIVE TO STIMULI; HYPOTONIA NEONATAL; INFUSION RELATED REACTION; INTERCOSTAL RETRACTION; NO THERAPEUTIC RESPONSE; MIDAZOLAM OPISTHOTONUS; OXYGEN SATURATION DECREASED; PERIORBITAL OEDEMA; PROCEDURAL COMPLICATION; PRODUCT QUALITY ISSUE; PRURITUS; REFUSAL OF TREATMENT BY PATIENT; RESPIRATORY DEPRESSION; RETROGRADE AMNESIA; SCREAMING; SKIN OEDEMA; URTICARIA; ACIDOSIS; BLOOD PH INCREASED; BLOOD PRESSURE DIASTOLIC DECREASED; BODY TEMPERATURE DECREASED; BRAIN DEATH; MORPHINE CARDIO-RESPIRATORY ARREST; COMPLETED SUICIDE; COORDINATION ABNORMAL; DECEREBRATION; DRUG ABUSE; DRUG

100

SCREEN POSITIVE; DRUG TOXICITY; FALL; HEART RATE DECREASED; HYPOTENSION; LETHARGY; MENTAL STATUS CHANGES; MIOSIS; MYOCLONUS; RESPIRATORY RATE DECREASED; SOMNOLENCE; SUICIDAL IDEATION; VENTRICULAR FIBRILLATION; ABDOMINAL DISTENSION; ABNORMAL BEHAVIOUR; ACIDOSIS; ACUTE RESPIRATORY FAILURE; ALANINE AMINOTRANSFERASE INCREASED; ANALGESIC DRUG LEVEL ABOVE THERAPEUTIC; APNOEA; ASPARTATE AMINOTRANSFERASE INCREASED; ASPIRATION; ASTHENIA; BLOOD CREATININE INCREASED; BLOOD GLUCOSE DECREASED; BLOOD PRESSURE DIASTOLIC DECREASED; BLOOD PRESSURE DIASTOLIC INCREASED; BLOOD PRESSURE INCREASED; BLOOD PRESSURE SYSTOLIC INCREASED; BLOOD SODIUM DECREASED; BODY TEMPERATURE INCREASED; BOWEL SOUNDS ABNORMAL; BRADYCARDIA; BRAIN HERNIATION; BRAIN OEDEMA; BREATH SOUNDS DECREASED; CARDIAC ARREST; CARDIO-RESPIRATORY ARREST; CEREBRAL VENTRICLE DILATATION; CHEST PAIN; CLONUS; COAGULOPATHY; COMA; COMPLETED SUICIDE; CONFUSIONAL STATE; CONGENITAL HYDROCEPHALUS; CRYING; CYANOSIS; DEAFNESS; DEATH; DEPRESSED LEVEL OF CONSCIOUSNESS; DEPRESSED MOOD; DEPRESSION; DEVELOPMENTAL DELAY; DRUG ABUSE; DRUG SCREEN POSITIVE; DRUG TOXICITY; DYSPNOEA; ELECTROCARDIOGRAM P WAVE ABNORMAL; ELECTROCARDIOGRAM QT PROLONGED; ELECTROCARDIOGRAM T WAVE PEAKED; ELECTROENCEPHALOGRAM ABNORMAL; ENCEPHALOPATHY; FAECES DISCOLOURED; HAEMODIALYSIS; HEART RATE INCREASED; HYPERKALAEMIA; HYPOKALAEMIA; HYPOTENSION; HYPOTONIA; HYPOVENTILATION; HYPOXIC-ISCHAEMIC ENCEPHALOPATHY; INAPPROPRIATE AFFECT; MORPHINE INTERNATIONAL NORMALISED RATIO INCREASED; LETHARGY; SULFATE LOGORRHOEA; LOSS OF CONSCIOUSNESS; LUNG INFILTRATION; MAXIMAL VOLUNTARY VENTILATION DECREASED; MENTAL STATUS CHANGES; METABOLIC ACIDOSIS; MIOSIS; MULTIPLE DRUG OVERDOSE; MUSCULOSKELETAL STIFFNESS; MYDRIASIS; MYOCLONUS; NAUSEA; NEUROTOXICITY; NO THERAPEUTIC RESPONSE; OEDEMA; OPIATES POSITIVE; OXYGEN SATURATION DECREASED; PAIN IN EXTREMITY; PANCREATITIS; PERIPHERAL COLDNESS; PNEUMONIA ASPIRATION; PO2 DECREASED; POISONING; PROTHROMBIN TIME PROLONGED; PRURITUS; PULMONARY OEDEMA; PUPIL FIXED; PUPILLARY REFLEX IMPAIRED; PYELOCALIECTASIS; PYREXIA; RENAL FAILURE ACUTE; RENAL IMPAIRMENT; RESPIRATORY ACIDOSIS; RESPIRATORY ARREST; RESPIRATORY DEPRESSION; RESPIRATORY DISTRESS; RESPIRATORY FAILURE; RESPIRATORY RATE DECREASED; RHONCHI; SEDATION; SINUS TACHYCARDIA; SOMNOLENCE; SPEECH DISORDER; STOOL ANALYSIS ABNORMAL; STRIDOR; STUPOR; SUICIDAL IDEATION; SUICIDE ATTEMPT; TACHYCARDIA; TONGUE DISCOLOURATION; TRANSAMINASES INCREASED; TREMOR; TROPONIN INCREASED; UMBILICAL CORD VASCULAR DISORDER; UNRESPONSIVE TO STIMULI; URINARY RETENTION; URINE OUTPUT DECREASED; VASOCONSTRICTION; VENTRICULAR FIBRILLATION; VOMITING; WHEEZING; WHITE BLOOD CELL COUNT DECREASED; WHITE BLOOD CELL COUNT INCREASED;

101

AGEUSIA; CONVULSION; DEVICE BREAKAGE; EYE ROLLING; HYPERSOMNIA; HYPOXIC ENCEPHALOPATHY; LOSS OF NYSTATIN CONSCIOUSNESS; MUSCULOSKELETAL STIFFNESS; PHARMACEUTICAL PRODUCT COMPLAINT; SEPSIS; VIRAL INFECTION; ABDOMINAL PAIN; AGITATION; ALANINE AMINOTRANSFERASE INCREASED; ALTERED STATE OF CONSCIOUSNESS; APNOEA; ARRHYTHMIA; ASPARTATE AMINOTRANSFERASE INCREASED; ATRIAL SEPTAL DEFECT; BLOOD POTASSIUM INCREASED; BLOOD PRESSURE INCREASED; BODY TEMPERATURE INCREASED; BRADYCARDIA; BRONCHOPNEUMONIA; BRUXISM; BURNING SENSATION; CARDIAC ARREST; CONVULSION; CREPITATIONS; DEATH NEONATAL; DEPRESSED LEVEL OF CONSCIOUSNESS; DIZZINESS; DRUG TOXICITY; DYSTONIA; ECHOVIRUS TEST POSITIVE; ELECTROCARDIOGRAM QRS COMPLEX PROLONGED; ELECTROCARDIOGRAM QT PROLONGED; ENTEROVIRUS INFECTION; ENTEROVIRUS TEST POSITIVE; EXTENSOR PLANTAR RESPONSE; FLUSHING; GRAND MAL CONVULSION; HAEMATOCRIT DECREASED; HAEMATOCRIT INCREASED; HEART RATE INCREASED; HEPATIC ENZYME INCREASED; HEPATIC INFECTION BACTERIAL; HEPATOTOXICITY; HYDROCELE; HYPERHIDROSIS; ONDANSETRON HYPERREFLEXIA; HYPERSENSITIVITY; HYPERTHERMIA; HYPERTHERMIA MALIGNANT; HYPERTONIA; HYPOGLYCAEMIA; HYPOGLYCAEMIA NEONATAL; HYPOTENSION; INFECTIVE THROMBOSIS; INTESTINAL OBSTRUCTION; IRRITABILITY; JAUNDICE NEONATAL; LIVER DISORDER; LOSS OF CONSCIOUSNESS; LYMPHADENOPATHY; MENTAL STATUS CHANGES; MOVEMENT DISORDER; MUSCLE RIGIDITY; MUSCULOSKELETAL PAIN; MYALGIA; MYDRIASIS; MYOCLONUS; NYSTAGMUS; OXYGEN SATURATION DECREASED; PELVIC KIDNEY; PLATELET COUNT INCREASED; PREMATURE BABY; PUPILLARY REFLEX IMPAIRED; RASH ERYTHEMATOUS; RASH PRURITIC; REFLEXES ABNORMAL; RESPIRATORY RATE INCREASED; SEROTONIN SYNDROME; SHOCK; SINUS BRADYCARDIA; SINUS TACHYCARDIA; SOMNOLENCE; STRIDOR; TACHYCARDIA; THERAPEUTIC RESPONSE UNEXPECTED; THROMBOSIS; TORSADE DE POINTES; VENTRICULAR FIBRILLATION; VENTRICULAR SEPTAL DEFECT; VENTRICULAR TACHYCARDIA; ABNORMAL BEHAVIOUR; ACTIVATED PARTIAL THROMBOPLASTIN TIME PROLONGED; AGGRESSION; ALANINE AMINOTRANSFERASE INCREASED; ANAEMIA; ANAL DILATATION; ANALGESIC DRUG LEVEL INCREASED; ANORECTAL DISORDER; ANOXIA; ANOXIC ENCEPHALOPATHY; ANTIBODY TEST POSITIVE; ANXIETY; AORTIC DISSECTION; APNOEA; ARRHYTHMIA; ASPARTATE AMINOTRANSFERASE INCREASED; ASPIRATION; ASTHENIA; ASTHMA; ATAXIA; BALANCE DISORDER; BASE EXCESS DECREASED; BICUSPID OXYCODONE AORTIC VALVE; BLOOD ALKALINE PHOSPHATASE INCREASED; BLOOD BICARBONATE DECREASED; BLOOD CALCIUM DECREASED; BLOOD CHLORIDE INCREASED; BLOOD CREATINE PHOSPHOKINASE MB INCREASED; BLOOD CREATININE INCREASED; BLOOD GLUCOSE ABNORMAL; BLOOD GLUCOSE DECREASED; BLOOD GLUCOSE INCREASED; BLOOD IMMUNOGLOBULIN E INCREASED; BLOOD LEAD INCREASED; BLOOD PH DECREASED; BLOOD PHOSPHORUS INCREASED; BLOOD POTASSIUM INCREASED; BLOOD PRESSURE

102

DECREASED; BLOOD PRESSURE DIASTOLIC DECREASED; BLOOD PRESSURE DIASTOLIC INCREASED; BLOOD PRESSURE IMMEASURABLE; BLOOD SODIUM DECREASED; BODY TEMPERATURE DECREASED; BODY TEMPERATURE INCREASED; BRAIN DEATH; BRAIN OEDEMA; BRONCHITIS ACUTE; BRONCHOPNEUMONIA; CARDIAC ARREST; CARDIAC ENZYMES INCREASED; CARDIAC TAMPONADE; CARDIOMEGALY; CARDIO-RESPIRATORY ARREST; CEPHALHAEMATOMA; CEREBRAL ISCHAEMIA; CEREBRAL VENTRICLE DILATATION; CIRCULATORY COLLAPSE; COMA; COMPLETED SUICIDE; CONFUSIONAL STATE; CONSTIPATION; CONTUSION; CONVULSION; COORDINATION ABNORMAL; COUGH; CULTURE POSITIVE; CYANOSIS; DEATH; DEPRESSED LEVEL OF CONSCIOUSNESS; DIARRHOEA; DIFFICULTY IN WALKING; DIZZINESS; DRUG LEVEL INCREASED; DRUG SCREEN POSITIVE; DRUG TOXICITY; DYSPNOEA; ELECTROCARDIOGRAM CHANGE; ELECTROLYTE IMBALANCE; ENCEPHALOPATHY; EXCORIATION; EXPOSURE DURING BREAST FEEDING; EXPOSURE TO TOXIC AGENT; FACE INJURY; FATIGUE; FEEDING DISORDER NEONATAL; FEEDING DISORDER OF INFANCY OR EARLY CHILDHOOD; FOREIGN BODY ASPIRATION; GAIT DISTURBANCE; GENERAL PHYSICAL CONDITION ABNORMAL; GENERAL PHYSICAL HEALTH DETERIORATION; GENERAL SYMPTOM; GLIOSIS; GRAM STAIN POSITIVE; HAEMATOCRIT DECREASED; HAEMOGLOBIN INCREASED; HAEMORRHAGE; HALLUCINATION; HEAD INJURY; HEAD LAG ABNORMAL; HEART RATE INCREASED; HEPATITIS; HYPERHIDROSIS; HYPOPNOEA; HYPOTENSION; HYPOTHERMIA NEONATAL; HYPOTONIA; HYPOVENTILATION; HYPOXIA; HYPOXIC ENCEPHALOPATHY; INCORRECT DOSE ADMINISTERED; INCREASED BRONCHIAL SECRETION; INJURY; INTERNATIONAL NORMALISED RATIO INCREASED; LABORATORY TEST ABNORMAL; LETHARGY; LIMB INJURY; LOSS OF CONSCIOUSNESS; LUNG INJURY; MALAISE; MENTAL STATUS CHANGES; MIOSIS; MOTOR DYSFUNCTION; MULTI-ORGAN DISORDER; MULTIPLE DRUG OVERDOSE; MULTIPLE DRUG OVERDOSE INTENTIONAL; MYDRIASIS; NARCOTIC INTOXICATION; NAUSEA; NEONATAL COMPLICATIONS OF SUBSTANCE ABUSE; NERVOUS SYSTEM DISORDER; NERVOUSNESS; NO THERAPEUTIC RESPONSE; OBSTRUCTIVE AIRWAYS DISORDER; ORAL DISCOMFORT; ORAL INTAKE REDUCED; OXYGEN SATURATION DECREASED; PALLOR; PNEUMONIA; PNEUMONIA ASPIRATION; PNEUMONIA STAPHYLOCOCCAL; PNEUMONITIS; POISONING; POSTICTAL STATE; PREMATURE BABY; PROTHROMBIN TIME PROLONGED; PRURITUS; PULMONARY ALVEOLAR HAEMORRHAGE; PULMONARY CONGESTION; PULMONARY OEDEMA; PULSE ABSENT; PULSE PRESSURE DECREASED; PULSE PRESSURE INCREASED; PUPIL FIXED; PUPILLARY REFLEX IMPAIRED; RASH ERYTHEMATOUS; RESPIRATION ABNORMAL; RESPIRATORY ARREST; RESPIRATORY DEPRESSION; RESPIRATORY DISORDER; RESPIRATORY DISTRESS; RESPIRATORY FAILURE; RESPIRATORY RATE DECREASED; RESPIRATORY RATE INCREASED; RETINAL HAEMORRHAGE; RHABDOMYOLYSIS; SCRATCH; SEDATION; SEPSIS; SINUS TACHYCARDIA; SKIN WARM; SLEEP DISORDER; SLEEP WALKING; SLUGGISHNESS; SNORING; SOMNOLENCE; SPUTUM DISCOLOURED; STAPHYLOCOCCAL INFECTION; STRIDOR;

103

SUBCUTANEOUS HAEMATOMA; SUICIDE ATTEMPT; TEARFULNESS; THERAPEUTIC AGENT TOXICITY; THERAPY NON-RESPONDER; THERMAL BURN; TOXICOLOGIC TEST ABNORMAL; TREMOR NEONATAL; UNRESPONSIVE TO STIMULI; UNRESPONSIVE TO VERBAL STIMULI; VENTRICULAR FIBRILLATION; VENTRICULAR HYPERTROPHY; VOMITING; WHEEZING; WHITE BLOOD CELL COUNT INCREASED; ALANINE AMINOTRANSFERASE INCREASED; ANAEMIA HAEMOLYTIC AUTOIMMUNE; ASPARTATE AMINOTRANSFERASE INCREASED; BLOOD LACTATE DEHYDROGENASE INCREASED; BRADYCARDIA; BRAIN OEDEMA; CHEST PAIN; CHILLS; COUGH; CYANOSIS; DRUG REACTION WITH EOSINOPHILIA AND SYSTEMIC SYMPTOMS; DRUG TOXICITY; PIPERACILLIN& DYSPNOEA; ELECTROCARDIOGRAM ST SEGMENT DEPRESSION; TAZOBACTAM ENCEPHALOPATHY; EOSINOPHILIA; FLUSHING; GRAND MAL CONVULSION; HEPATORENAL FAILURE; HYPOKALAEMIA; INCORRECT ROUTE OF DRUG ADMINISTRATION; LEUKOCYTOCLASTIC VASCULITIS; LEUKOPENIA; LYMPHOPENIA; METABOLIC ACIDOSIS; NEUTROPENIA; PARAESTHESIA; PERIPHERAL COLDNESS; SOMNOLENCE; THROMBOCYTOPENIA; TONIC CLONIC MOVEMENTS; ANXIETY; ASTHENIA; CHEST PAIN; CONVULSION; DYSKINESIA; ERYTHEMA; FONTANELLE BULGING; HEAD TITUBATION; HELICOBACTER INFECTION; INJECTION SITE ERYTHEMA; RANITIDINE IRRITABILITY; MUSCLE CONTRACTIONS INVOLUNTARY; NEUROTOXICITY; POOR QUALITY SLEEP; PRODUCT QUALITY ISSUE; PRODUCT TASTE ABNORMAL; RESTLESSNESS; TOXIC EPIDERMAL NECROLYSIS; TREMOR; SODIUM DEVICE MALFUNCTION; BICARBONATE CATHETER RELATED INFECTION; CEREBRAL ARTERY EMBOLISM; CHILLS; DEVICE LEAKAGE; DRUG HYPERSENSITIVITY; GENERALISED ERYTHEMA; HEADACHE; HYPERSENSITIVITY; IATROGENIC INJURY; SODIUM INFUSION RELATED REACTION; INJECTION SITE URTICARIA; OEDEMA; CHLORIDE PHARMACEUTICAL PRODUCT COMPLAINT; PULMONARY EMBOLISM; PYREXIA; RASH; RASH PAPULAR; SEPSIS; STAPHYLOCOCCAL INFECTION; TRANSFUSION REACTION; VOMITING; ABDOMINAL PAIN; ACUTE HEPATIC FAILURE; ANAPHYLACTIC REACTION; BLOOD CREATININE INCREASED; CARDIAC ARREST; CARDIAC FAILURE; CARDIAC FAILURE CONGESTIVE; CARDIO- RESPIRATORY ARREST; CONJUNCTIVAL HYPERAEMIA; CYANOSIS; DRUG HYPERSENSITIVITY; DRUG REACTION WITH EOSINOPHILIA AND SYSTEMIC SYMPTOMS; DRUG TOXICITY; EOSINOPHILIC MYOCARDITIS; ERYTHEMA MULTIFORME; GENERALISED OEDEMA; VANCOMYCIN HAEMOFILTRATION; HAEMOGLOBIN DECREASED; HEPATIC ENCEPHALOPATHY; HEPATIC NECROSIS; HEPATOMEGALY; HYPERHIDROSIS; HYPERSENSITIVITY; HYPOALBUMINAEMIA; INCORRECT DOSE ADMINISTERED; INCORRECT DRUG ADMINISTRATION RATE; KIDNEY ENLARGEMENT; LIVER TRANSPLANT; LUPUS NEPHRITIS; LYMPHADENOPATHY; NAUSEA; NEONATAL DISORDER; NEPHROPATHY TOXIC; PRURITUS; PULMONARY OEDEMA; RED MAN SYNDROME; RENAL FAILURE

104

ACUTE; RENAL TUBULAR NECROSIS; RESPIRATORY ARREST; THROMBOCYTOPENIA; T-LYMPHOCYTE COUNT INCREASED; TUBULOINTERSTITIAL NEPHRITIS; WHITE BLOOD CELL COUNT INCREASED;

105