Comparison of the Mortality Prediction of Different ICU Scoring Systems

Egyptian Journal of Chest Diseases and Tuberculosis (2015) xxx, xxx–xxx

HOSTED BY The Egyptian Society of Chest Diseases and Tuberculosis Egyptian Journal of Chest Diseases and Tuberculosis

www.elsevier.com/locate/ejcdt www.sciencedirect.com

ORIGINAL ARTICLE Comparison of the mortality prediction of diﬀerent ICU scoring systems (APACHE II and III, SAPS II, and SOFA) in a single-center ICU subpopulation with acute respiratory distress syndrome

Abdelbaset Saleh a,*, Magda Ahmed b, Intessar Sultan c, Ahmed Abdel-lateif d a Chest Department, Saudi German Hospital Al-Madinah, Chest Medicine, Mansoura University, Egypt b Chest Medicine, Taiba University and Mansoura University, Egypt c Internal Medicine, Taiba University and Cairo University, Egypt d ICU, Saudi German Hospital Al-Madinah, Egypt

Received 14 April 2015; accepted 17 May 2015

KEYWORDS Abstract Background: Scoring systems can be used to define critically ill patients, estimate their Acute respiratory distress prognosis, help in clinical decision making, guide the allocation of resources and estimate the qual- syndrome; ity of care in the ICU. APACHE II; Purpose: This study compared the predictive accuracy of four predictive scoring systems in the APACHE III; ICU. SAPS II; Methods: A prospective cohort study including consecutively admitted 110 adult ICU patients SOFA (88 males) with ARDS from Saudi German Hospital, Madinah, was performed from June 2013 to January 2015. The median age of the patients was 38 years, the median duration of illness before ICU admission was 6 days, and the median duration of ICU admission was 27 days. The APACHE II, APACHE III, SAPS II, and SOFA scores were calculated based on the worst values during the first 24 h of admission. Results: The actual mortality rate (27.3%) was higher than the estimated mortality rates, with the highest predicted rate of 11.3% obtained using the APACHE II. All four severity scores were significantly associated with mortality (F = 62.772, p = 0.000) and explained 83% of its variability (R2 = 0.834). However, after adjustment, only the APACHE III scoring system was a significant predictor (Beta = À0.753, p = 0.000). Three scoring systems were significantly associated with mortality (F = 42.055, p = 0.000) and explained almost 70% of its variability (R2 = 0.712), but after adjustment, only the APACHE II was a significant predictor (Beta = À0.631, p = 0.041).

* Corresponding author at: Saudi German Hospital Al-Madinah, Department of Chest, Mansoura University, Mansoura 35516, Egypt. E-mail address: [email protected] (A. Saleh). Peer review under responsibility of The Egyptian Society of Chest Diseases and Tuberculosis. http://dx.doi.org/10.1016/j.ejcdt.2015.05.012 0422-7638 ª 2015 Production and hosting by Elsevier B.V. on behalf of The Egyptian Society of Chest Diseases and Tuberculosis. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

The combination of the severity score and mortality prediction was a signiﬁcant predictor of mortality (Beta = À1.397, p = 0.000 and Beta = 0.517, p = 0.036, respectively). Conclusion: The accuracy of the studied scoring systems for predicting ICU mortality in ARDS patients is limited. The performance of the APACHE II/III scoring systems was superior to that of other systems in terms of predicting the severity and mortality, and the combination of scores improved the performance. There is a need to develop ARDS-speciﬁc scoring systems. Until a new system is developed, it is better to use the updated versions of the APACHE scoring system or a combination of all ICU scoring systems. ª 2015 Production and hosting by Elsevier B.V. on behalf of The Egyptian Society of Chest Diseases and Tuberculosis. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/ licenses/by-nc-nd/4.0/).

Introduction to the SAPS II and SOFA systems in predicting ICU mortality [9]. In a previous study, the accuracy of both the SAPS II and Clinical assessment of the severity of illness is an essential com- APACHE instruments improved when combined with the ponent of medical practice to predict the mortality and mor- assessment of sequential SOFA scores. bidity of critically ill patients, especially in the intensive care Four predictive scoring systems were evaluated in the pre- unit (ICU) [1,2]. Physiologically based scoring systems are sent study to determine, which system is the best for predicting more applicable than diagnosis-based scoring systems and the outcomes of ARDS patients. These were the Acute can estimate the risk based on the degree of variation from Physiology and Chronic Health Evaluation (APACHE) sys- the normal function of major organ systems [1,2]. Acute respi- tems II and III, the Simplified Acute Physiology Score II ratory distress syndrome (ARDS) is among the leading causes (SAPS II), and the Sequential Organ Failure Assessment score of mortality in critically ill patients. ARDS is an acute, diffuse, (SOFA). According to a Medline search, no study comparing inflammatory lung injury, leading to increased pulmonary vas- these four scoring systems had been reported in our region cular permeability, increased lung weight, and a loss of aerated (Middle East). Thus, we designed this study to compare the lung tissue. The clinical hallmarks of ARDS are hypoxemia performance of the four scoring systems in ARDS patients. and bilateral radiographic opacities, while the pathological hallmark is diffuse alveolar damage [3]. Subjects and methods The Acute Physiology And Chronic Health Evaluation (APACHE), introduced in 1981, takes into consideration var- This prospective cohort study was conducted in the Adult ICU ious parameters, such as physiological variables, vital signs, at the Saudi German Hospital Al-Madinah, KSA. One hun- urine output, the neurological score, age and co-morbid condi- dred and ten consecutively admitted adult patients admitted tions, which may have a significant impact on the outcome of between June 2013 and January 2015 were included in the critically ill patients [4]. The APACHE II, formulated in 1985, study after obtaining consent from the institutional review estimates the risk based on the worst variables available within board. The consent for participation was obtained at time of the first 24 h of admission. The APACHE II is widely used to admission from the patients themselves or from the relatives quantify the severity of illness in the ICU, and has been vali- who were most acquainted with the patient; a waiver of dated in many clinical trials. The APACHE III scoring system informed consent was granted by the IRB due to the minimal is similar to the APACHE II system, except that several vari- risk of the observational study. ables have been added (e.g., diagnosis, prior treatment loca- The diagnostic evaluation was performed on admission to tion) [5]. exclude other differential diagnoses and to identify the specific The Simplified Acute Physiology Score (SAPS) streamlines causes of ARDS. We included patients who met the 2012 data collection and analysis without compromising the diag- Berlin definition criteria for ARDS [10]. nostic accuracy. The SAPS II is the most widely used version. The exclusion criteria included acute interstitial pneumonia It calculates a severity score using the worst values measured (Hamman–Rich syndrome), disseminated cancer, eosinophilic during the initial 24 h in the ICU for 17 variables [6]. The pneumonia, diffuse alveolar hemorrhage associated with colla- Sequential Organ Failure Assessment (SOFA) uses simple gen vascular diseases, and cardiogenic pulmonary edema. measurements of major organ functions to calculate a severity Patients were also excluded if they refused to participate, died score. The scores are calculated 24 h after admission to the within 24 h of the ARDS diagnosis, or when the diagnosis of ICU and every 48 h thereafter. The mean and the highest ARDS or other alternative causes of acute hypoxemic respira- scores are most predictive of mortality. In addition, scores that tory failure could not be established based on the clinical con- increase by approximately 30% are associated with a mortality text, symptoms, signs, and bronchoscopy in patients whose rate of at least 50% [7]. The original SOFA instrument was acute hypoxemic respiratory failure remained of uncertain etiol- derived from a cohort of 1449 patients admitted to 40 ICUs ogy after non-diagnostic flexible bronchoscopy if one or more of in 16 countries [8]. the diagnostic possibilities under consideration might warrant There have been no large, prospective studies that have targeted therapy or would substantially change the prognosis. compared the different ICU predictive scoring systems in a Scoring systems: The APACHE II, APACHE III, SAPS II, subset of patients with ARDS. A systemic review of the and SOFA scores were calculated based on the worst values SOFA, SAPS II, APACHE II, and APACHE III scoring sys- recorded during the first 24 h of admission. All enrolled tems found that the APACHE systems were slightly superior patients were followed during their ICU stay, and the outcome

(due to their abnormal distribution) or percentages. The statistical analysis was performed using non-parametric statistics to compare the two mortality groups (survivors/non-survivors). A linear regression analysis was performed, and Microsoft Excel was used to generate Fig. 1. The statistical tests were two-tailed and a value of p < 0.05 was considered to be the cut-off value of signiﬁcance.

Results

This study included 22 females (20%) and 88 males (80%), with a median age of 38 years (26–84 years), who had a median Figure 1 Results of linear regression analysis of the mortality duration of illness before ICU admission of 6 (2–15) days prediction of the individual and combined ICU scoring systems (Table 1). After being monitored for a median period of 27 (APACHE III, APACHE II, SAPS II, and SOFA) in ARDS (8–45) days, 30 of the patients died (mortality rate: 27.3%; patients. p = 0.036. Table 1). However, the predicted mortality rates were lower than the actual mortality rate, with the highest rate of 11.3% was recorded as survivors or non-survivors. For each of the calculated using the APACHE II and the lowest rates calcu- identified patients, the institutional APACHE III database lated with the SAPS (7.9%) and SOFA (<10%) scoring was searched using the software program provided by Cerner systems. Corporation (Kansas City, Missouri), and APACHE III data A comparison of the different ICU scoring systems for the were abstracted. The online combination ICU Mortality disease severity and estimated mortality in ARDS patients Calculator (APACHE II, SAPS II, and SOFA scores to pre- (Table 2) showed that all four ICU scoring systems had statis- dict hospital mortality; http://clincalc.com/IcuMortality/ tically significant differences between survivors and non- Default.aspx) was used to calculate the corresponding score survivors. for each patient. The data included age, gender, vital signs, The multivariate regression showed that the four different Glasgow Coma Scale scores, and urine output, which were scores were all significantly associated with mortality abstracted at the bedside by trained nurses according to a for- (F = 131.82, p = 0.000), and explained 83% of its variability malized protocol, and which were entered into the computer (R2 = 0.834). However, after adjustment, only the APACHE by trained specialists. The APACHE III system score was cal- III scoring system was a significant predictor of mortality culated separately using the http://www.quesgen.com/ (Beta = À0.753, p = 0.000). Three different methods signifi- ApacheIII.php database. cantly estimated the mortality (F = 42.055, p = 0.000) and explained almost 70% of its variability (R2 = 0.712). Statistical analysis However, after adjustment, only the APACHE II was a significant predictor of mortality (Beta = À0.631, p = 0.041). The The data analysis was performed using the Statistical Package combination of the severity score and mortality prediction for the Social Sciences software program version 20 (SPSS was a significant predictor of mortality (Beta = À1.397, Inc., Chicago, IL). Descriptive data were expressed as medians p = 0.000 and Beta = 0.517, p = 0.036, respectively) (Fig. 1).

Table 1 Characteristic data, ICU scoring systems of severity and estimated mortality, and mortality in ARDS patients. Median Minimum Maximum Gender: n (%) Females: n (%) 22 (20) Males: n (%) 88 (80) Age: years 38 26 84 Duration of illness before ICU admission: days 6 2 15 Duration of ICU admission: days 27 8 45 APACHE II Severity score 10 5.00 34.00 Estimated mortality: % 11.3 5.8 81 SAPS II Severity score 26 16.00 75.00 Estimated mortality: % 7.9 2.30 88.90 SOFA Severity score 4 2 11 Estimated mortality: % <10 <10 40–50 APACHE III Severity score 53 29.00 112.00 Combined scoring systems Severity score 22 16.25 54.75 Estimated mortality: % 8.9 5.7 63.13 Mortality Died: n (%) 30 (27.3) Survived: n (%) 80 (72.7)

Table 2 Comparison of different ICU scoring systems of severity and estimated mortality in ARDS patients. Survivors n = 80 Non-survivors n =30 p APACHE II Score (median) 8 27 0.000 Estimated mortality (%) 8.7 60.5 0.000 SAPS II Score (median) 21 60 0.000 Estimated mortality (%) 4.2 68.10 0.000 SOFA Score (median) 4 9 0.000 Estimated mortality (%) <10 10–20 0.000 APACHE III Score (median) 43 96 0.000 Combined scoring systems Score (median) 19.88 49.25 0.000 Estimated mortality (%) 8.17 47.43 0.000

Discussion patients. In the Escarce et al. study [24], the calculated APACHE II score underestimated the predicted mortality ARDS is among the main causes of mortality in critically ill risks for patients who were transferred from different settings patients [3]. The mortality rate in this study (27.3%) was lower (hospital floors: 38% vs 55%, intermediate care unit: 32% vs than the previously reported estimates ranging from 26% to 59%, and other hospitals: 21% vs 36%) but was close to the 58% [11–13]. This agrees with other studies which found better actual rate for patients who were admitted directly from the survival rates over time [14–16]. However, the mortality rate in emergency department (25% vs 22%). this study was higher than that predicted by different scoring Another study [25] validated SAPS II and APACHE II and systems. The Severity scores and mortality prediction in ICU found that the observed mortality of high risk mainly surgical patients are important for facilitating the management of hos- patients (above 60%) was over-predicted by SAPS II and pital resources [17], evaluating the quality of care [18–20], and under-predicted by APACHE II. The same study also found most importantly evaluating therapeutic interventions [21], that the actual mortality was significantly higher than that pre- especially for patients with ARDS [22,23]. dicted by any of the 2 scores. Another study of patients from As observed in other studies [24,25], our survivors had sig- Scottish ICUs [19] compared 6 scoring systems and found significantly lower median scores for all scoring systems com- nificantly different observed mortalities from those predicted pared to non-survivors (Table 2). with the best overall performance by SAPS II and better cali- Moreover, linear regression analysis confirmed an indepen- bration by APACHE II. dent association between the severity scores and mortality pre- The difference between the predicted and actual mortality diction by different scoring systems and mortality. rates may be explained by the limited accuracy of the mortality Our present results showed that the only significant predic- prediction models because they are limited by the items tors of mortality were the APACHE II and III and the com- included, and subjected to interpretation and influenced by bined score of the four systems (Fig. 1). The APACHE II many factors including local admission, discharge and man- score in this study ranged from 5 to 34, with a median of 10. agement policies [32]. They are likely to under- or over- These results agree with those described by Parajuli et al. in estimate mortality in selected patient subpopulations because 2015 [26] who reported a range of 6–35, with a mean score the original cohort was mixed. Another explanation is that of 18.26 ± 7.40. Moreover this range is comparable to that the single performance of these scoring systems during the ini- reported from other areas of the world, including India [27], tial 24 h as predictive models requires periodic retesting, revis- Bangladesh [28] and Turkey [29]. ing, and updating, because its accuracy decreases as treatments The APACHE III system is similar to APACHE II system and other factors influencing the mortality rate change. [30] but with more data and daily updates of the clinical infor- The main strength of this study is the inclusion of a subpop- mation used to recalculate the estimated mortality [30].Itis ulation of ICU patients for accuracy of mortality prediction by expected that APACHE III mortality prediction will be more otherwise non-specific scoring systems. However, the study accurate than a single projection based upon the first 24 h of was limited by its small size. Another limitation is the non- ICU admission [31]. Unfortunately, the mortality prediction inclusion of the updated versions of some of the scoring sys- by this score was not included in this study due to the use of tems, such as the APACHE IV, SAPS III, and the Mortality the initial 24 h data by other scoring systems, which could have Prediction Model (MPM), in this comparative study. interfered with comparisons between different scoring systems. The combined score of the four systems improved the mortal- Conclusion ity prediction. Since different scoring systems have different aims and measurements, they should be combined, rather than The four studied ICU predictive scoring systems (APACHE II, competing against each other [32]. APACHE III, SAPS II, and SOFA) gave significantly different In this study, only the APACHE II gave a mortality predic- severity scores and mortality prediction in survivors compared tion >10%, while the other scoring systems gave a mortality to non-survivors among ICU patients with ARDS. However, prediction <10%. The three scoring systems and their com- their accuracy in predicting the actual mortality was limited. bined scores underestimated the actual mortality in ICU The performance of the APACHE II/III scoring systems was

Please cite this article in press as: A. Saleh et al., Comparison of the mortality prediction of different ICU scoring systems (APACHE II and III, SAPS II, and SOFA) in a single-center ICU subpopulation with acute respiratory distress syndrome, Egypt. J. Chest Dis. Tuberc. (2015), http://dx.doi.org/10.1016/j.ejcdt.2015.05.012 Accuracy of scoring systems in ARDS 5 superior to that of the other systems in mortality prediction, [10] The ARDS Definition Task Force, V.M. Ranieri, G.D. and applying a combination of scoring systems improved the Rubenfeld, B.T. Thompson, N.D. Ferguson, E. Caldwell, E. performance. Fan, L. Camporota, A.S. Slutsky, Acute respiratory distress syndrome: the Berlin definition, J. Am. Med. Assoc. 307 (2012) 2526–2533. Recommendation [11] J. Villar, J. Blanco, J.M. Anõń, et al, The ALIEN study: incidence and outcome of acute respiratory distress syndrome in There is a need to develop an ARDS-specific scoring system the era of lung protective ventilation, Intensive Care Med. for better severity grading and mortality prediction. Until such (2011) 37–1932. a system is developed, it is better to use the updated versions of [12] A. Esteban, F. Frutos-Vivar, A. Muriel, et al, Evolution of the APACHE scoring system including daily updates or a mortality over time in patients receiving mechanical ventilation, combination of the available ICU scoring systems. To improve Am. J. Respir. Crit. Care Med. (2013) 188–220. [13] C.Y. Wang, C.S. Calfee, D.W. Paul, et al, One-year mortality the predictions, it is necessary to update the APACHE III and predictors of death among hospital survivors of acute score daily. At present, it is recommended that the APACHE respiratory distress syndrome, Intensive Care Med. (2014) 40– scoring system (APACHE IV if available) be used rather than 388. other systems for predicting the mortality because of its supe- [14] R.D. Stapleton, B.M. Wang, L.D. Hudson, et al, Causes and rior accuracy. It is recommended that clinicians use the most timing of death in patients with ARDS, Chest (2005) 128–525. recent version of the APACHE scoring system because it [15] S.E. Erickson, G.S. Martin, J.L. Davis, et al, Recent trends in includes frequent daily updates to the data. acute lung injury mortality: 1996–2005, Crit. Care Med. (2009) 37–1574. [16] M. Zambon, J.L. Vincent, Mortality rates for patients with Competing interests acute lung injury/ARDS have decreased over time, Chest (2008) 133–1120. The authors declare that they have no competing interests. [17] J.E. Zimmerman, D.P. Wagner, W.A. Knaus, et al, The use of risk predictions to identify candidates for intermediate care Acknowledgment units. Implications for intensive care utilization and cost, Chest (1995) 108–490. [18] B. Afessa, M.T. Keegan, R.D. Hubmayr, et al, Evaluating the This research paper is made possible through the help and sup- performance of an institution using an intensive care unit port from all ICU staff. benchmark, Mayo Clin. Proc. (2005) 80–174. [19] J.E. Zimmerman, C. Alzola, K.T. Von Rueden, The use of References benchmarking to identify top performing critical care units: a preliminary assessment of their policies and practices, J. Crit. [1] M. Palazo, Severity of illness, in: J.L. Vincent, E. Abraham, Care (2003) 18–76. F.A. Moore, P.M. Kochanek, M.P. Fink (Eds.), Text Book of [20] J.E. Zimmerman, S.M. Shortell, W.A. Knaus, et al, Value and Critical Care, sixth ed., Elsevier Publishers, Albany (NY), 1996, cost of teaching hospitals: a prospective, multicenter, inception pp. 17–29. cohort study, Crit. Care Med. (1993) 21–1432. [2] D.C. Bouch, J.P. Thompson, Severity score system in critically [21] W.A. Knaus, D.P. Wagner, J.E. Zimmerman, E.A. Draper, ill, Oxford J. 8 (2008) 181–185. Variations in mortality and length of stay in intensive care units, [3] ARDS Definition Task Force, V.M. Ranieri, G.D. Rubenfeld, Ann. Intern. Med. (1993) 118–753. B.T. Thompson, N.D. Ferguson, E. Caldwell, E. Fan, L. [22] Ventilation with lower tidal volumes as compared with Camporota, A.S. Slutsky, Acute respiratory distress syndrome: traditional tidal volumes for acute lung injury and the acute the Berlin Definition, JAMA 307 (23) (2012) 2526–2533, http:// respiratory distress syndrome. The Acute Respiratory Distress dx.doi.org/10.1001/jama.2012.5669. Syndrome Network, N. Engl. J. Med. (2000) 342–1301. [4] W.A. Knaus, J.E. Zimmerman, D.P. Wagner, E.A. Draper, [23] A. Anzueto, R.P. Baughman, K.K. Guntupalli, et al, D.E. Lawrence, APACHE acute physiology and chronic health Aerosolized surfactant in adults with sepsis-induced acute evaluation: a physiologically based classification system, Crit. respiratory distress syndrome. Exosurf Acute Respiratory Care Med. 9 (1981) 591–597. Distress Syndrome Sepsis Study Group, N. Engl. J. Med. [5] W.A. Knaus, E.A. Draper, D.P. Wagner, J.E. Zimmerman, (1996) 334–1417. APACHE II: a severity of disease classification system, Crit. [24] J.J. Escarce, M.A. Kelley, Admission source to the medical Care Med. 13 (1985) 818–829. intensive care unit predicts hospital death independent of [6] J.R. Le Gall, S. Lemeshow, F. Saulnier, et al, A new simplified APACHE II score, JAMA 264 (18) (1990) 2389. acute physiology Score (SAPSII) based on European/North [25] M. Capuzzo, V. Valpondi, A. Sgarbi, S. Bortolazzi, V. Pavoni, American multicenter study, JAMA (1993) 270–2957. G. Gilli, G. Candini, G. Gritti, R. Alvisi, Validation of severity [7] F.L. Ferreira, D.P. Bota, A. Bross, et al, Serial evaluation of the scoring systems SAPS II and APACHE II in a single-center SOFA score to predict outcome in critically ill patients, JAMA population, Intensive Care Med. 26 (12) (2000) 1779. (2001) 286–1754. [26] B.D. Parajuli, G.S. Shrestha, B. Pradhan, R. Amatya, [8] J.L. Vincent, A. de Mendonça, F. Cantraine, et al, Use of the Comparison of acute physiology and chronic health evaluation SOFA score to assess the incidence of organ dysfunction/failure II and acute physiology and chronic health evaluation IV to in intensive care units: results of a multicenter, prospective predict intensive care unit mortality, Indian J. Crit. Care Med. study. Working group on ‘‘sepsis-related problems’’ of the 19 (2015) 87–91. European Society of Intensive Care Medicine, Crit. Care Med. [27] R. Gupta, V.K. Arora, Performance evaluation of APACHE II (1998) 26–1793. score for an Indian patient with respiratory problems, Indian J. [9] L. Minne, A. Abu-Hanna, E. de Jonge, Evaluation of SOFA- Med. Res. 119 (2004) 273–282. based models for predicting mortality in the ICU: a systematic [28] M.O. Faruq, M.R. Mahmud, T. Begum, T.S. Ahsan, K. review, Crit. Care (2008) R12–R161. Fatema, F. Ahmed, et al, A comparison of severity systems

APACHE II and SAPS II in critically ill patients, Bangladesh of hospital mortality for critically ill hospitalized adults, Chest Crit. Care J. 1 (2013) 27–32. 100 (6) (1991) 1619. [29] T.A. Ayazoglu, A comparison APACHE II and APACHE IV [31] D.P. Wagner, W.A. Knaus, F.E. Harrell, J.E. Zimmerman, C. scoring systems in predicting outcome in patients admitted with Watts, Daily prognostic estimates for critically ill adults in stroke to an intensive care unit, Anaesth Pain Intensive Care 15 intensive care units: results from a prospective, multicenter, (2011) 7–12. inception cohort analysis, Crit. Care Med. 22 (9) (1994) 1359. [30] W.A. Knaus, D.P. Wagner, E.A. Draper, J.E. Zimmerman, M. [32] Vincent, Moreno, Clinical review: scoring systems in the Bergner, P.G. Bastos, C.A. Sirio, D.J. Murphy, T. Lotring, A. critically ill, Crit. Care 14 (2010) 207, .