Comparison of the Performance of APACHE II, SOFA, and mNUTRIC Scoring Systems in Critically Ill Patients: A 2-year Cross-sectional Study
1,2,4Department of Medicine, Jawaharlal Nehru Medical College, Datta Meghe Institute of Medical Sciences (Deemed to be University), Wardha, Maharashtra, India
3Department of General Surgery, Jawaharlal Nehru Medical College, Datta Meghe Institute of Medical Sciences (Deemed to be University), Wardha, Maharashtra, India
Corresponding Author: Sunil Kumar, Department of Medicine, Jawaharlal Nehru Medical College, Datta Meghe Institute of Medical Sciences (Deemed to be University), Wardha, Maharashtra, India, Phone: +91 9850393787, e-mail: firstname.lastname@example.org
How to cite this article Kumar S, Gattani SC, Baheti AH, Dubey A. Comparison of the Performance of APACHE II, SOFA, and mNUTRIC Scoring Systems in Critically Ill Patients: A 2-year Cross-sectional Study. Indian J Crit Care Med 2020;24(11):1057–1061.
Source of support: Nil
Conflict of interest: None
Aims and objectives: Different severity scores are being used to assess outcomes in intensive care unit, but variable data had been reported so far per their performance. Main objective of this study is to compare performance of acute physiology and chronic health evaluation II (APACHE II), sequential organ failure assessment (SOFA), and modified nutrition risk in critically ill (mNUTRIC) scoring systems regarding the outcomes in the form of morbidity and mortality in medical intensive care unit (MICU) at rural tertiary-care health center.
Materials and methods: In this cross-sectional study, 1,990 patients older than 18 years admitted in the ICU were enrolled. Age, gender, diagnosis, intubation, comorbidities, APACHE II, SOFA scores, m NUTRIC score, MICU stays in days, and need of mechanical ventilation were noted.
Results: When we compared different score with mortality, APACHE-II was having sensitivity of 89.9% and specificity of 97.6%; SOFA had 90.1% sensitivity and 96.6% specificity; while mNUTRIC score had 97.2% sensitivity and 74.0% specificity. APACHE-II score had sensitivity of 93.4%, SOFA had 90.5%, and mNUTRIC score 92.3% with low specificity of 76.5% in predicting requirement of mechanical ventilation. mNUTRIC score and ICU length of stay showed moderate positive correlation (p value = %3C;0.001).
Conclusion: All the three scores were comparable in sensitivity and specificity in predicting outcomes in the form of mortality, need of mechanical ventilation, and length of ICU stays. mNUTRIC score was more sensitive than others, and as it was based on nutritional status, hence more weightage should be given on this score.
Keywords: Acute physiology and chronic health evaluation II, Intensive care unit, Modified nutrition risk in critically ill, Scoring systems, Sequential organ failure assessment..
For the assessment of patient outcome such as morbidity and mortality, as well as severity of illness in most of the medical intensive care units (MICU), various scoring systems had been studied.1–6 To date, no consensus has been reached to point out the foremost tool for identifying the propensity of risk in critically ill patients.3,4 More studies are required to evaluate various scoring tools, as their sensitivity and specificity are not defined to predict mortality in different patients across medical ICU setup. Nowadays sequential organ failure assessment (SOFA) and acute physiology and chronic health evaluation II (APACHE II) are being used for comparisons of different outcomes in different ICUs.7–9 In 2011, Heyland et al. proposed, developed, and validated a widely accepted unique scoring system known as the NUTRIC or the Nutrition Risk in Critically ill score comprises of 6 variables, most of which are relatively obtained effortlessly in the ICUs.10–12 Later on studies showed that eliminating interleukin 6 levels from the original NUTRIC Score did not have drastic effects on its performance, and a revised mNUTRIC (modified NUTRIC score) was formulated.13–15 The patients who had a score ≥6 were considered to be at high risk. No studies had been reported so far while comparing APACHE II, SOFA, and mNUTRIC scoring systems as far as their performance is concerned. In this study, we had tried to evaluate performance APACHE II, SOFA, and m NUTRIC scoring systems regarding the outcomes in ICU, such as mortality, need of mechanical ventilations, and length of ICU stay.
MATERIALS AND METHODS
This prospective cross-sectional study was conducted in the 42-bedded medical intensive care unit (MICU) under Department of Medicine at Jawaharlal Nehru Medical College, Wardha, in central India, from October 2017 to September 2019. Ethical clearance for this study was obtained from local institutional ethical committee, and written informed consent was obtained from the participants before commencement of the study.
A point biserial correlation for power analysis was conducted in G*Power to determine a sufficient sample size with an alpha of 0.05, a power of 0.80, a small effect size (p value = 0.1), and two tails.16 The optimal sample size was 779 based on the abovemnetioned assumptions. In this study, all the consecutive/sequential patients older than 18 years of age (n = 2100) admitted in the MICU were enrolled in this study. During this 2 year period, a total of 1,990 patients were analyzed in the present study. Patients who died within 24 hours were excluded. Critically ill surgical and patients of transplants were excluded because of admission at separate special ICU. Patients referred from other ICUs were also excluded. For patients with multiple admissions in MICU, first admission was included in data analysis. The study algorithm flow diagram is shown in Flowchart 1.
mNUTRIC SCORE was calculated in critically ill patients admitted to the MICU by taking various variables that are a part of the scoring system like age, thorough history and clinical examination, and preexisting comorbidities. SOFA score and APACHE II score were calculated using online software.
The cutoffs of mNUTRIC score may differ in different ICUs depending on the patient’s characteristics, the disease dynamics, cultural, social, and environmental factors, and the type of treatment facilities available. Based on the data analyzed over 2 years, the results showed that a cutoff of mNUTRIC score %3E;5.5 was associated with adverse outcome. The range of mNUTRIC score ranges from 0 to 9. Since the value 5.5 was not a whole number, we rounded of the value to the closest approximate higher value, i.e., 6. Hence, we divided our study population into two groups with patients having score ≥6 as high mNUTRIC score and those having score ≤5 as low mNUTRIC score. Accuracy of performance of APACHE II, SOFA, and mNUTRIC scoring system was measured by the area under the ROC curve for mortality and requirement of mechanical ventilation in MICU.
Data were coded and recorded in MS Excel. Descriptive statistics were elaborated in the form of means ± standard deviation for continuous variables and frequencies/percentages for categorical variables. Group differences were compared using t test for continuously distributed data, and chi-square test for categorical data. IBM SPSS v23 was used for statistical analysis. Statistical significance was set at p value <0.05.
In this study, out of 1,990 patients, 395 died while 1,595 survived and 799 had a high (≥6) mNUTRIC score, while 1,191 had a low (≤5) mNUTRIC score. Mean age of the patients were 50.48 ± 17.58 years, male being 63.2% and female being 36.8%. Other baseline characteristics are shown in Table 1.
When we compared different scores with mortality, APACHE-II was having sensitivity of 89.9% and specificity of 97.6% (AUC = 0.983), SOFA had 90.1% and 96.6% (AUC = 0.986), whereas mNUTRIC score had 97.2% and 74.0% (AUC = 0.938), respectively, shown in Figure 1. As far as the need for mechanical ventilation were considered, APACHE-II score had sensitivity of 93.4% and specificity of 89.7% (AUC = 0.966); SOFA had sensitivity of 90.5% and specificity of 95.8% (AUC = 0.976); and mNUTRIC Score had sensitivity of 92.3% and specificity of 76.5% (AUC = 0.901) as shown in Figure 2.
Scatterplot shown in Figures 3 to 5 depicts the correlation between all three scoring systems and ICU length of stay (days). Individual points represented as individual cases. The blue trend line represented the general trend of correlation between the two variables. The shaded gray area represented the 95% confidence interval of this trend line. There was a weak positive correlation between APACHE-II score and SOFA score with ICU length of stay (days), and this correlation was statistically significant (ρ = 0.24, p value = <0.001), whereas between mNUTRIC Score and ICU length of stay (days), there was moderate positive correlation (ρ = 0.31, p value = <0.001). For every 1 unit increase in mNUTRIC score, the ICU length of stay in days increases by 0.98 units.
Treating patient and predicting mortality in the ICUs is always a challenge as well as great concern for physicians. The effect of this prediction is on various aspects of patient care, such as medical treatment, triage, end-of-life ICU care, and many more. In recent years, there has been a growing interest in finding new prognostic markers like mNUTRIC score in critically ill ICU patients. Primary outcomes of this study were to compare mortality by different scoring system in critically ill patients. Secondary outcome was to analyze the need of mechanical ventilation and length of ICU stays which may have affected mortality. Most of the time because of variable institutional and population parameters, ICU mortality rates also varies which ranges from 6.4 to 46%.17,18 In our study, mortality rate was 14.7% in less than 50 years of age, 20.7% among 50–75 years of age, and 42.0% in older than 75 years of age. Study by Leong and Tai and Ursavaş et al. had not shown any direct relation as far as increasing age and mortality rates are concerned. They further quoted that age was not a deciding factor in a ICU patient.17–19
|Parameters (n = 1990)||Nonsurvivors (n = 395)||Survivors (n = 1595)||p value|
|Age (years)||56.67 ± 16.92||48.95 ± 17.40||<0.0011|
|Male||269 (68.1%)||988 (61.9%)|
|Female||126 (31.9%)||607 (38.1%)|
|APACHE-II score||40.00 ± 7.13||22.17 ± 4.61||<0.0011|
|SOFA score||15.86 ± 3.36||6.62 ± 1.93||<0.0011|
|ICU length of stay (days)||6.02 ± 6.04||6.76 ± 5.35||<0.0011|
|mNUTRIC score||7.15 ± 0.94||4.64 ± 1.50||<0.0011|
|mNUTRIC score category||<0.0012|
|High (≥6)||384 (97.2%)||415 (26.0%)|
|Low (≤5)||11 (2.8%)||1180 (74.0%)|
|Mechanical ventilation||345 (87.3%)||137 (8.6%)||<0.0012|
|Hypertension||282 (71.4%)||826 (51.8%)||<0.0012|
|Diabetes mellitus||45 (11.4%)||127 (8.0%)||0.0302|
|Renal diseases||267 (67.6%)||459 (28.8%)||<0.0012|
|Neurological diseases||168 (42.5%)||348 (21.8%)||<0.0012|
|Cardiovascular diseases||356 (90.1%)||1102 (69.1%)||<0.0012|
|Respiratory diseases||348 (88.1%)||248 (15.5%)||<0.0012|
|Sepsis||268 (67.8%)||603 (37.8%)||<0.0012|
|Other disorders||297 (75.2%)||893 (55.9%)||<0.0012|
1 Kruskal-Wallis test;
2 Chi-squared test. APACHE II, acute physiology and chronic health evaluation II; ICU, intensive care unit; mNUTRIC, modified nutrition risk in critically ill; SOFA, sequential organ failure assessment
Estimated mortality by APACHE-II score was 46.47 ± 22.81%, 56.43 ± 20.21%, 64.04 ± 20.46% (p value < 0.0011) in the same groups, respectively. In this study, we had found trend of increase in APACHE II scores and mortality as patient’s age increases.
Several studies have regarded mNUTRIC score to be one of the best scoring systems to adjudge nutritional adequacy in critically unwell patients and so the outcomes.8,9,11,12 This score was valuable in categorizing illness according to its severity and estimating the approximate duration of ICU stay and need of mechanical ventilation.
On comparing different scoring system with mortality, APACHE-II and SOFA had better sensitivity and specificity than mNUTRIC Score, which had high sensitivity but low specificity. Same observation was in case of need for mechanical ventilation. In this study, there was a weak positive correlation between APACHE-II score and SOFA score with ICU length of stay but moderate positive correlation with mNUTRIC score.
Strength and Limitation
The present study had strength of single-center study with large number of patients having standard care. Main limitation of the study was not having predefined sample size, as we had compared the different scores regarding outcome in terms of mortality.
The discriminative performance of mNUTRIC score for assessing overall outcomes was found to be comparable in our study. Although having lower specificity, mNUTRIC score was more sensitive than others, and as it was based on nutritional status, more weightage should be given on this score. Further studies with multicenter trial are needed to develop national agreement and better clarification of the issues along with ICUs from different parts of the world taking more concern on nutritional status.
This study had been approved by Institutional Ethics Committee review; with approval number DMIMS (DU)/IEC/2018-19/6759.
2. Karabıyık L. Intensive care scoring systems. J Med Surg Intensive Care Med 2010;9:129–143.
6. Juneja D, Singh O, Nasa P, Dang R. Comparison of newer scoring systems with the conventional scoring systems in general intensive care population. Minerva Anestesiol 2012;78(2):194–200.
7. Franchi F, Cubattoli L, Mongelli P, Porciani C, Nocci M, Casadei E, et al. Validation of simplified acute physiology score II and simplified acute physiology score III as mortality and morbidity risk models. Crit Care 2009;13 (Suppl 1):P508. DOI: 10.1186/cc7672.
8. Ata ur-Rehman HM, Ishtiaq W, Yousaf M, Bano S, Mujahid AM, Akhtar A. Modified nutrition risk in critically ill (mNUTRIC) score to assess nutritional risk in mechanically ventilated patients: a prospective observational study from the pakistani population. Cureus 2018;10(12):e3786. DOI: 10.7759/cureus.3786.
9. Rahman A, Hasan RM, Agarwala R, Martin C, Day AG, Heyland DK. Identifying critically-ill patients who will benefit most from nutritional therapy: further validation of the “modified NUTRIC” nutritional risk assessment tool. Clin Nutr 2016;35(1):158–162. DOI: 10.1016/j.clnu.2015.01.015.
10. Heyland DK, Dhaliwal R, Jiang X, Day AG. Identifying critically ill patients who benefit the most from nutrition therapy: the development and initial validation of a novel risk assessment tool. Crit Care 2011;15(6):R268. DOI: 10.1186/cc10546.
11. Kalaiselvan MS, Renuka MK, Arunkumar AS. Use of nutrition risk in critically ill (NUTRIC) score to assess nutritional risk in mechanically ventilated patients: a prospective observational study. Indian J Crit Care Med 2017;21(5):253–256. DOI: 10.4103/ijccm.IJCCM_24_17.
12. Mukhopadhyay A, Henry J, Ong V, Leong CSF, Teh AL, van Dam RM, et al. Association of modified NUTRIC score with 28 day mortality in critically ill patients. Clin Nutr 2017;36(4):1143–1148. DOI: 10.1016/j.clnu.2016.08.004.
13. Kılıç YA. Intensive care scoring systems: why, how, where we are? J Med Surg Intensive Care Med 2002;2:26–31.
14. Öizbilgin Ş, Demirağ K, Sargın A, Uyar M, Moral AR. Comparing scoring systems used in intensive care unit regarding predictive roles mortality. J Med Surg Intensive Care Med 2001;9:8–13.
15. Balcı C, Sungurtekin H, Gürses E, Sungurtekin U. APACHE II, APACHE III, SOFA scoring systems, platelet counts and mortality in septic and non-septic patients. Turk J Trauma Emerg Surg 2005;11(1):29–34.
16. Faul F, Erdfelder E, Buchner A, Lang A. Statistical power analyses using G*power 3.1: tests for correlation and regression analyses. Behav Res Methods 2013;41(4):1149–1160. DOI: 10.3758/BRM.41.4.1149.
18. Leong IY, Tai DY. Is increasing age associated with mortality in the critically ill elderly. Singapore Med J 2002;43(1):33–36.
19. Ursavaş A, Ege E, Yüksel EG. The evaluation of the factors affecting mortality in respiratory intensive care unit. J Med SurgIntensive Care Med 2006;6:43–48.
© The Author(s). 2020 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by-nc/4.0/), which permits unrestricted use, distribution, and non-commercial reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.