BRIEF RESEARCH COMMUNICATION


https://doi.org/10.5005/jp-journals-10071-24094
Indian Journal of Critical Care Medicine
Volume 26 | Issue 1 | Year 2022

Prediction of Mortality in the Medical Intensive Care Unit with Serial Full Outline of Unresponsiveness Score in Elderly Patients

Jamileh Ramazani1https://orcid.org/0000-0003-4645-780X, Mohammad Hosseini2https://orcid.org/0000-0002-5977-1921

1Department of Nursing, Nursing and Midwifery College, Bojnourd Branch, Islamic Azad University, Bojnourd, Iran

2Department of Nursing, Nursing and Midwifery College, North Khorasan University of Medical Sciences, Bojnurd, Iran

Corresponding Author: Jamileh Ramazani, Department of Nursing, Nursing and Midwifery College, Bojnourd Branch, Islamic Azad University, Bojnourd, Iran, Phone: +98-058-32730053, e-mail: ramazani182@yahoo.com

How to cite this article: Ramazani J, Hosseini M. Prediction of Mortality in the Medical Intensive Care Unit with Serial Full Outline of Unresponsiveness Score in Elderly Patients. Indian J Crit Care Med 2022;26(1):94–99.

Source of support: Nil

Conflict of interest: None

ABSTRACT

Background: Advanced age is one of the key risk factors for mortality and morbidity in intensive care units. The full outline of unresponsiveness (FOUR) score has been developed and introduced to address the limitations of the Glasgow Coma Scale (GCS). The current study aimed to evaluate the ability of the FOUR score in predicting the outcomes (survivors, nonsurvivors).

Materials and methods: This observational study of 168 consecutive elderly patients admitted to medical intensive care during the 14 months carried out prospectively. FOUR score in the 24, 48, and 72 hours of admission, and demographic characteristics of all elderly patients were calculated, then recorded. The receiver operating characteristic (ROC) curve, logistic regression, and Hosmer-Lemeshow test were used (95% confidence interval) for statistical analysis.

Results: FOUR scores in 24, 48, and 72 hours between survivors and nonsurvivors (p <0.0001, p <0.0001, and p <0.0001, respectively) were statistically different. The discrimination power of FOUR score 24 hours of admission was excellent [area under ROC (AUC): 85.7% [standard error (SE)]: 2.8%]; it was acceptable for 48 and 72 hours of admission [AUC: 76.3% (SE: 3.6%), AUC: 75/0% (SE: 3.8%), respectively]. The FOUR score of 24 and 48 hours (x2 = 10.06, p = 0.261, x2 = 6.82, p = 0.448, respectively) showed acceptable calibration.

Conclusions: The FOUR score is a suitable scoring system for prognostication of outcomes in critically ill elderly patients. The FOUR score 24 hours of admission was superior in terms of discrimination power than 48 and 72 hours, but better calibration power belonged to FOUR score 48 hours.

Keywords: Elderly, Full outline of unresponsiveness score, Medical intensive care, Nonsurvivors, Survivors.

INTRODUCTION

A large number of predictive models have been developed and are widely used in intensive care practice over the past 35 years. They have valuable advantages such as continuous improvement of care and management in the intensive care units.1 In addition to their strengths, scoring systems have weaknesses, and we know there is no perfect model that can be used in all different settings.2,3 Customizing the model coefficients to the specific population could be beneficial for obtaining more accurate probabilities or the development of new scoring systems specialized to the target population.4 Developed in 1974, the Glasgow Coma Scale (GCS) used to objectively describe the neurological status. Although it is considered as the gold standard in determining the level of consciousness, but in the literature, limitations for this score have been mentioned several times.58 From the past decade, the full outline of unresponsiveness (FOUR) score was validated and introduced to calculate four functional dimensions including eye, brainstem reflexes, motor responses, and respiration.9 By using this new structured objective scoring model, the brainstem function can be examined, even for patients who cannot communicate verbally.10,11 Decreasing FOUR score (with ranging from 0 to 16) is associated with a falling level of consciousness.12,13

The validity of the FOUR score with reference to GCS has been shown in many studies.5,6,11,14 However, there is no consensus on which of these scoring systems is best.1517 External validation should be obtained when applying the scoring system to a new setting.2

Nyam et al.18 examined whether the FOUR score is superior to GCS in predicting ICU mortality in traumatic brain injury (TBI) patients. They compared the ability of four scoring systems by using AUC-ROC curve. Of the 55 TBI patients, 72.73% were men, and the mean age of subjects was 63.1 ± 17.9. The overall mortality rate was 14.6%. The AUCs were 74.47, 74.73, 81.78, and 53.32% for the FOUR score, GCS, APACHE II, and TISS, respectively. They found that the FOUR score was similar to the GCS and APACHE II, in terms of predictive power, also the FOUR score could predict early mortality in TBI patients. In another study, Said et al.19 compared the usefulness of GCS and FOUR score in predicting extubation failure. A total of 86 patients with impaired consciousness were included in their study. The median GCS and FOUR score were 7 (3–10) and 8.5 (2.3–11), respectively, on admission. The AUC with FOUR score was significantly greater than GCS score, AUC: 0.867 (0790–0.944), confidence interval (CI): 95% and AUC: 0.832 (0.741–0.923), CI: 95%; respectively, (p = 0.014). They found the FOUR has a better ability to predict successful extubation than GCS.

Khanal et al.20 in a study of 97 patients (age >16 years), prospectively compared the outcome prediction of these two models. Measuring both scores within 24 hours of ICU admission, data analyzing showed that the mean scores of nonsurvivors were lower than survivors (p <0.001). The AUCs of 0.79 and 0.82 (for GCS and FOUR score, respectively) indicated to fair discrimination power for both. Considering the Youden index, 6.5 was the best cutoff point for both scores. The Hosmer–Lemeshow Chi-square coefficient test showed the FOUR score had better calibration power than GCS. The Spearman test of 0.91 pointed to a strong correlation between two scores (p <0.001). Some studies suggest that these two predictive models have the same ability in prognostication of the outcome; however, some studies have pointed to the superiority of either model. Accordingly, it is recommended that these models should be repeatedly calibrated.2 As far as we know, there have been no studies assessing the use of the FOUR score in predicting outcomes in MICU for the elderly patients. The present study aimed to evaluate the FOUR score ability to predict outcomes in elderly patients admitted to the MICU.

MATERIALS AND METHODS

In this cohort study (July 2016–October 2017), 168 consecutive elderly patients (age ≤65 years) admitted to the MICU were included to study. Patients with brain death and who stayed in the ICU for less than 24 hours were excluded.

Patients’ demographic data (including gender and age) were collected from the records of patients, and serial FOUR scores for the first 24, 48, and 72 hours of admission were calculated for each patient separately.

Eye and motor responses, respiration patterns and brainstem reflexes are the categories of the FOUR score. The ranges of score for each category is 0–4 points. The lower scores of the FOUR score denoting an increasing deviation from normal. Worst FOUR scores in 24 hours at the first 24, 48, and 72 hours after MICU admission were derived. After the initial recording of data in collection form, the data entered to the SPSS (IBM Corp., Version 22.0, New York). Then, using statistical tests, the relationship between patients’ outcomes and the FOUR score was assessed. To protect patients’ privacy, no information has been provided that could lead to patient identification. The primary outcomes for this nonintervention study were nonsurvivors and survivors.

Patients who died after 24 hours of admission to the MICU or were diagnosed with brain death were included in nonsurvivors. To summarize the data related to the study population, simple descriptive statistics were used for continuous variables and percentage for classified variables. The p <0.050 is considered significant. For assessing the discrimination power of the model, calculating the AUC-ROC curve, distinguishing between survivors and nonsurvivors, was evaluated. AUC of 0.5 means random chance; AUC more than 0.7 and 0.8 indicates a moderate and good prognostic models, respectively.21 The agreement between actual outcomes and individual probabilities shown with calibration power that assessed by Hasmer–Lemshaw goodness of fit (GOF) test; if P is greater than 0.05, it indicates that the model has a good calibration.22

RESULTS

A total of 168 elderly patients admitted to MICU, with a mean age of 74.97 ± 6.28 years (65–90 years), 70 (41.7%) men, and 98 (58.3%) women were enrolled in this study. In total, 31% (52 patients) died. Chronic obstructive pulmonary disease (COPD), gastrointestinal (GI) bleeding, cerebrovascular accident (CVA), sepsis, post-cardiopulmonary resuscitation (CPR), and cancer were the most diagnosed diseases, respectively. Table 1 shows the characteristics of the study population.

Table 1: Characteristics of the study samples
Characteristics Total (n = 168) Survivors (n = 116) Nonsurvivors (n = 52) p
Age (years, mean ± SD range) 74.97 ± 6.28 73.40 ± 6.16 78.48 ± 5.02 p <0.001
COPD 72 55 17
GI bleeding 25 20 5
CVA 20 16 4
Sepsis 23 15 8
Post CPR 17 5 12
Cancer 11 5 6
Sex (n, %)        
Men 70 (41.7) 48 (41.4) 22 (42.3) p = 0.91
Women 98 (58.3) 68 (58.6) 30 (57.7)  
Length of MICU stay (days, mean ± SD) 19.11 ± 11.81 18.97 ± 12.93 19.42 ± 8.94 p = 0.79
FOUR score 24 hours (mean ± SD) 8.53 ± 4.18 10.07 ± 3.99 5.10 ± 1.92 p <0.001
FOUR score 48 hours (mean ± SD) 9.55 ± 4.11 10.71 ± 4.05 6.98 ± 2.93 p <0.001
FOUR score 72 hours (mean ± SD) 9.28 ± 3.89 10.34 ± 3.86 6.92 ± 2.78 p <0.001
MICU, medical intensive care; FOUR score, full outline of unresponsiveness score

The survivors showed significantly higher scores at 24, 48, and 72 hours after admission than nonsurvivors (p <0.001, p <0.001, and p <0.001, respectively) (Table 1).

There are two known methods to examine the performance of predictive scoring systems: calculating the calibration and discrimination power of them. In Table 2, the performance of the GCS and FOUR score has compared.

Table 2: Comparison of FOUR score and GCS between survivors and nonsurvivors
Variables Cutoff score Youden index Sensitivity (%) Specificity (%) Positive predictive value (%) Negative predictive value (%) Accuracy (%) Area under ROC curve 95% CI SE Significant
FOUR score 24 hours 8.5 0.565 96.15 60.34 52.08 97.22 71.43 0.857 0.80–0.91 0.028 0.000
FOUR score 48 hours 10.5 0.432 92.31 50.86 45.71 93.65 63.69 0.763 0.69–0.83 0.036 0.000
FOUR score 72 hours 9.5 0.364 76.92 59.48 45.98 85.19 64.88 0. 750 0.68–0.82 0.038 0.000
FOUR score, full outline of unresponsiveness score; GCS, Glasgow Coma Scale

AUCs indicated that the FOUR score’s discrimination power for 24, 48, and 72 hours of admission was good (AUC = 0.857, AUC = 0.763, and AUC = 0.750, respectively). By the Youden index (sensitivity + specificity −1), the best cutoff point for models was determined. By cutoff point 8.5, FOUR score 24 hours of admission showed 96% sensitivity, 60% specificity, and 71% accuracy, with an AUC of 0.757 ± 0.028 standard error (SE) (95%; 0.80–0.91, p <0.001). A cutoff point 10.5 for FOUR score 48 hours of admission showed 92% sensitivity, 51% specificity, and 64% accuracy, also the AUC was 0.763 ± 0.036 SE (95%; 0.69–0.83, p <0.001), and a cutoff score 9.5 for FOUR score 72 hours of admission accompanied by 77% sensitivity, 59% specificity, and 65% accuracy, also the AUC was 0.750 ± 0.038 SE (95%; 0.68–0.82, p <0.001) (Table 2).

The calibration was good just for FOUR score 24 hours (χ2 = 10.06, p = 0.26) and 48 hours of admission (χ2 = 6.82, p = 0.45), and it was weak for FOUR score 72 hours. In Figure 1, the ROC curve shows the predictive accuracy of the two models. In terms of predictive accuracy, the FOUR score 24 hours of admission was superior to the FOUR score of other days although the FOUR score 48 hours showed a slightly better calibration than FOUR score 24 hours of admission. The survivors and nonsurvivors were homogeneous in terms of gender and LOS-MICU (p = 0.91, p = 0.79, respectively) (Table 1). In addition, the nonsurvivors were older than survivors (p <0.001).

Fig. 1: ROC curves for FOUR score 24, 48, and 72 hours of admission at the MICU. The AUC ROC was 0.857.0.763, and 0.750, respectively

DISCUSSION

The performance of serial FOUR score (24, 48, and 72 hours of admission) has been evaluated in the MICU. The survivors had a higher mean serial FOUR scores compared to nonsurvivors (p <0.001, p <0.001, and p <0.001, respectively). Moreover, the serial scores showed good discrimination power (p <0.001, p <0.001, and p <0.001, respectively). Also, the AUC of 0.857 for FOUR score 24 hours, the AUC of 0.763 for FOUR score 48 hours, and the AUC of 0.750 for FOUR score 72 hours of admission indicated that the discrimination power of FOUR score was excellent for FOUR score 24 hours and good for FOUR score 48 and 72 hours. The better discrimination power of FOUR score 24 hours of admission may represent that the 24 hours of admission in the intensive care unit is a critical time for assessing the severity of illness, as many predictive models use this time for calculating the score. Compared to FOUR score 72 hours (weak calibration), FOUR score 24 hours (χ2 = 10.06, p = 0.26) and FOUR score 48 hours (χ2 = 6.82, p = 0.45) had good calibration. It may represent that the FOUR score is suitable in short-term settings, and the adjusted model is needed for long-term settings.

The best cutoff point for FOUR score 24, 48, and 72 hours was 8.5, 10.5, and 9.5, respectively. In Akavipat et al.’s23 study, the cutoff points for in-hospital mortality and the poor outcome were 10 and 14, respectively.

In line with our results, many studies have reported that patients with low FOUR scores had a poorer prognosis or higher mortality rates.5,11,18,24,25 Many studies have also pointed to the appropriate prognostication power of FOUR score (Table 3).

Table 3: Similar studies to findings
Authors Year FOUR score 24 hours (mean ± SD) Sensitivity Specificity Positive predictive value Negative predictive value Area under ROC curve Findings
Fugate et al. 2010 FOUR score ≤4
FOUR score ≤8
FOUR score ≤10
47.6
71.4
92.9
100
97.8
80.0
100
96.8
81.3
67.2
78.6
92.3
Two points improvement in FOUR score was associated with survival. In addition, it is a simple and accurate predictor of outcome in patients surviving cardiac arrest.
Kocak et al. 2012 Survivors: 12.9 ± 2.48
Non-survivors: 8.6 ± 4.32
0.67 The FOUR score is a useful tool for evaluation of acute stroke patients and outcome estimation.
Mcnett et al. 2014 Median:15 0.91 FOUR is comparable to GCS in terms of predictive ability.
Sadaka et al. 2012 13   0.64 The accuracy of the FOUR score in predicting the outcome of TBI patients confirmed but they emphasized there is need further studies for standard use of this tool.
Okasha et al. 2014 Median:11 73.0 80.0 0.85 The predictive ability of FOUR score was better than GCS.
Khanal et al. 2016 Survivors: 9.13 ± 3.61 Non-survivors: 4.97 ± 2.76 79.31 79.41 62.16 90.0 0.82 Compared to GCS the sensitivity, specificity, positive predictive value, negative predictive value, and accuracy of FOUR score were better.
Ramazani and Hosseini 2018 7.21 ± 2.83 67.6 87.5 33.33 92.59 78.7 FOUR score sowed good performance for predicting outcomes in children admitted to medical/surgical ICUs.
FOUR score, full outline of unresponsiveness score; GCS, glasgow coma scale

Fugate et al.24 in a prospective, a cohort study from June 2006 to October 2009 assessed the predictive ability of the FOUR score in patients’ postcardiac arrest and compared its performance to GCS. Hospital mortality was the primary outcome. The total mortality rate was 65% (89). None of the patients with a FOUR score ≤4 survived at days 3–5 after cardiac arrest, and only one patient with a GCS of 3 survived. On the 3rd–5th days after cardiac arrest, 91% patients with FOUR score >8 (p <0.0001) and 87% patients with GCS >6 survived (p <0.0001). In serial exams, two-point improvement in FOUR score was associated with survival. This relationship did not apply to GCS score. They observed the FOUR score has good accuracy, and a simple clinical tool can predict outcomes reliably.

Kocak et al.’s26 study assessed the usefulness of the FOUR score in predicting the clinical outcomes of ICU patients with stroke. They also examined the associations between FOUR score and other coma scales like GCS and APACHE II. It was a prospective study of 100 acute stroke patients who were admitted to a neurology ICU. The mean age of the population was 70.49 ± 12.42 years. The FOUR scores of nonsurvivors on the 1st, 3rd, and 10th days of admission were significantly lower than survivors (p = 0.005, p <0.001, p <0.001, and p <0.001, respectively). Also, the AUC ranged from 0.675 (95% CI, 0.565–0.786) for the 1st day of admission, 0.922 (95% CI, 0.867–0.977) for the 3rd day, and 0.981 (95% CI, 0.947–1.015) for the10th day. They suggested that the FOUR score as a useful scale can be used in the evaluation of acute stroke patients and outcome estimation.

Mcnett et al.27 evaluated the correlations between GCS and FOUR scores of 24 and 72 hours with cognitive/functional outcomes. Also, they assessed the correlation between both scores and mortality; 136 patients with a mean age of 53.1 enrolled in their study. In their study, the FOUR score was similar to GCS in terms of discrimination power.

In this study, FOUR score 24 hours of admission showed excellent discrimination power and for 48 and 72 hours of admission, the power was good. The FOUR score of 24 and 48 hours had acceptable calibration power, but calibration of FOUR score 72 hours was weak. Most of the studies, in line with our findings, have mentioned that FOUR score has an acceptable to excellent level of discrimination power.16,18,28

Sadaka et al.28 in a prospective study examined the predictive value of FOUR score in TBI and compared its performance to GCS; 51 patients were enrolled in their study. In-hospital mortality and poor neurologic outcome in the 3rd to 6th months were the primary outcomes. The FOUR score showed an AUC of 0.67 for poor neurologic outcomes, it was 0.65 for GCS. Similarly, for in-hospital mortality it was 0.64 for FOUR score and 0.63 for GCS. They confirmed the appropriate accuracy of the FOUR score in predicting the outcome of TBI patients, also they emphasized further studies for standard use of this model.

In another prospective cohort study, Okasha et al.16 compared the ability of these two models in predicting TBI outcomes; 60 patients with TBI were included in their study. Both scores were calculated on arrival to the emergency room. Unfavorable outcome, in-hospital mortality, endotracheal intubation, and LOS were primary outcomes. The overall mortality rate was 25% (15) and 58% (35) had an unfavorable outcome. The AUC for FOUR score was significantly higher than AUC for the GCS (0.850 vs 0.796, p = 0.025). The models were similar in predicting unfavorable outcome (p = 0.136) and endotracheal intubation (p = 0.06). The ability of the FOUR score was greater in distinguishing survivors and non-survivors.

In our study, the calibration power of FOUR score was acceptable in 24 and 48 hours, but it was weak for FOUR score 72 hours of admission (p = 0.26, p = 0/45, and p <0.001, respectively). The calibration power of FOUR score was varied in several studies, depending on the diagnosis of the patients and the setting in which the study was performed.29,30 This inconsistency and dissimilarity in calibration can be explained in this way that Predictive models of outcome should be calibrated before being used in new settings to avoid compromising their accuracy.2 Recalibrating outcome prediction models take into account the adjustments needed to cope with population changes, and the quality of care in different settings thus can improve the estimation of the outcome.

The overall mortality rate observed in our study was 31%, which was 65% in Fugate et al.24 study and 14.6% in Nyam et al.18 study. Differences in inclusion criteria, patients’ diagnosis, severity of their illness, and quality of care between studies can be the reasons for the existing differences in mortality rates. The sample size, quality of care, and institutional policies can influence model calibration. Therefore, it is necessary to adapt the model to these conditions. Ethical considerations have been considered in conducting this study.

CONCLUSION

The FOUR score showed good predictive performance in predicting mortality in elderly patients who were admitted to MICU. FOUR score 24 hours of admission had excellent discrimination and good calibration power. For 48 hours of admission, the FOUR score had acceptable discrimination with good calibration power. For FOUR score 72 hours of admission, just discrimination of model was acceptable, and calibration was weak. Utilizing such a valid predictive model in critically elderly patients is beneficial.

ORCID

Jamileh Ramazani https://orcid.org/0000-0003-4645-780X

Mohammad Hosseini https://orcid.org/0000-0002-5977-1921

REFERENCES

1. Murthy JM, Meena AK, Kumar SR. Severity-of-illness scoring systems and models: neurological and neurosurgical intensive care units. Neurol India 2001;Suppl. 1:S91–S94. PMID: 11889480.

2. Hosseini M, Ramazani J. Evaluation of acute physiology and chronic health evaluation II and sequential organ failure assessment scoring systems for prognostication of outcomes among intensive care unit’s patients. Saudi J Anaesth 2016;10(2):168–173. DOI: 10.4103/1658-354X.168817.

3. Ramazani J, Hosseini M. Comparison of the predictive ability of the pediatric risk of mortality III, pediatric index of Mortality3, and pediatric logistic organ dysfunction-2 in medical and surgical intensive care units. J Comprehensive Pediatrics 2019;10(2):e82830. DOI: 10.5812/compreped.82830.

4. Sikka P, Jaafar WM, Bozkanat E, El-Solh AA. A comparison of severity of illness scoring systems for elderly patients with severe pneumonia. Intensive Care Med 2000;26(12):1803–1810. DOI: 10.1007/s001340000719.

5. Cohen J. Interrater reliability and predictive validity of the FOUR score coma scale in a pediatric population. J Neurosci Nurs 2009;41(5):261–267; quiz 268–269. DOI: 10.1097/jnn.0b013e3181b2c766.

6. Chen B, Grothe C, Schaller K. Validation of a new neurological score (FOUR Score) in the assessment of neurosurgical patients with severely impaired consciousness. Acta Neurochir (Wien) 2013;155(11):2133–2139; discussion 9. DOI: 10.1007/s00701-013-1854-2.

7. Marcati E, Ricci S, Casalena A, Toni D, Carolei A, Sacco S. Validation of the Italian version of a new coma scale: the FOUR score. Intern Emerg Med 2012;7(2):145–152. DOI: 10.1007/s11739-011-0583-x.

8. Ramazani J, Hosseini M. Comparison of Glasgow Coma Scale and GCS-Age Prognosis Score in Older Adult Patients/Yasli Hastalarda Glasgow Koma Skalasi ve GCS-Yas Prognoz Skorunun Karsilastirilmasi. J Dahili ve Cerrahi Bilimler Yoğun Bakım Dergisi 2018;10(2):35–41. DOI: 10.33381/dcbybd.2019.2068.

9. Peng J, Deng Y, Chen F, Zhang X, Wang X, Zhou Y, et al. Validation of the Chinese version of the FOUR score in the assessment of neurosurgical patients with different level of consciousness. BMC Neurol 2015;15:254. DOI: 10.1186/s12883-015-0508-9.

10. Zeiler FA, Lo BWY, Akoth E, Silvaggio J, Kaufmann AM, Teitelbaum J, et al. Predicting outcome in Subarachnoid Hemorrhage (SAH) utilizing the Full Outline of UnResponsiveness (FOUR) score. Neurocrit Care 2017;27(3):381–391. DOI: 10.1007/s12028-017-0396-5.

11. Ramazani J, Hosseini M. Prediction of ICU mortality in critically ill children: comparison of SOFA, GCS, and FOUR score. Med Klin Intensivmed Notfmed 2019;114(8):717–723. DOI: 10.1007/s00063-018-0484-0.

12. Saika A, Bansal S, Philip M, Devi BI, Shukla DP. Prognostic value of FOUR and GCS scores in determining mortality in patients with traumatic brain injury. Acta Neurochir 2015;157(8):1323–1328. DOI: 10.1007/s00701-015-2469-6.

13. Ramazani J, Hosseini M. Comparison of full outline of unresponsiveness score and Glasgow Coma Scale in Medical Intensive Care Unit. Ann Card Anaesth 2019;22(2):143–148. DOI: 10.4103/aca.ACA_25_18.

14. Kasprowicz M, Burzynska M, Melcer T, Kübler A. A comparison of the Full Outline of UnResponsiveness (FOUR) score and Glasgow Coma Score (GCS) in predictive modelling in traumatic brain injury. Br J Neurosurg 2016;30(2):211–220. DOI: 10.3109/02688697.2016.1161173.

15. Iyer VN, Mandrekar JN, Danielson RD, Zubkov AY, Elmer JL, Wijdicks EF. Validity of the FOUR score coma scale in the medical intensive care unit. Mayo Clin Proc 2009;84(8):694–701. DOI: 10.1016/S0025-6196(11)60519-3.

16. Okasha AS, Fayed AM, Saleh AS. The FOUR score predicts mortality, endotracheal intubation and ICU length of stay after traumatic brain injury. Neurocrit Care 2014;21(3):496–504. DOI: 10.1007/s12028-014-9995-6.

17. Akavipat P. Endorsement of the FOUR score for consciousness assessment in neurosurgical patients. Neurol Med Chir (Tokyo) 2009;49(12):565–571. DOI: 10.2176/nmc.49.565.

18. Nyam TTE, Ao KH, Hung SY, Shen ML, Yu TC, Kuo JR. FOUR score predicts early outcome in patients after traumatic brain injury. Neurocrit Care 2017;26(2):225–231. DOI: 10.1007/s12028-016-0326-y.

19. Said T, Chaari A, Hakim K, Hamama D, Casey W. Usefulness of full outline of unresponsiveness score to predict extubation failure in intubated critically-ill patients: a pilot study. Int J Crit Illn Inj Sci 2016;6(4):172–177. DOI: 10.4103/2229-5151.195401.

20. Khanal K, Bhandari SS, Shrestha N, Acharya SP, Marhatta MN. Comparison of outcome predictions by the Glasgow coma scale and the Full Outline of UnResponsiveness score in the neurological and neurosurgical patients in the Intensive Care Unit. Indian J Crit Care Med 2016;20(8):473–476. DOI: 10.4103/0972-5229.188199.

21. Metz CE. Basic principles of ROC analysis. Semin Nucl Med 1978;8(4):283–298. DOI: 10.1016/s0001-2998(78)80014-2.

22. Hosseini M, Ramazani J. Comparison of acute physiology and chronic health evaluation II and Glasgow Coma Score in predicting the outcomes of Post Anesthesia Care Unit’s patients. Saudi J Anaesth 2015;9(2):136–141. DOI: 10.4103/1658-354X.152839.

23. Akavipat P, Sookplung P, Kaewsingha P, Maunsaiyat P. Prediction of discharge outcome with the full outline of unresponsiveness (FOUR) score in neurosurgical patients. Acta Med Okayama 2011;65(3):205–210. DOI: 10.18926/AMO/46633.

24. Fugate JE, Rabinstein AA, Claassen DO, White RD, Wijdicks EF. The FOUR score predicts outcome in patients after cardiac arrest. Neurocrit Care 2010;13(2):205–210. DOI: 10.1007/s12028-010-9407-5.

25. Weiss N, Venot M, Verdonk F, Chardon A, Le Guennec L, Llerena MC, et al. Daily FOUR score assessment provides accurate prognosis of long-term outcome in out-of-hospital cardiac arrest. Rev Neurol (Paris) 2015;171(5):437–444. DOI: 10.1016/j.neurol.2015.02.013.

26. Kocak Y, Ozturk S, Ege F, Ekmekci H. A useful new coma scale in acute stroke patients: FOUR score. Anaesth Intensive Care 2012;40(1):131–136. DOI: 10.1177/0310057X1204000115.

27. McNett M, Amato S, Gianakis A, Grimm D, Philippbar SA, Belle J, et al. The FOUR score and GCS as predictors of outcome after traumatic brain injury. Neurocrit Care 2014;21(1):52–57. DOI: 10.1007/s12028-013-9947-6.

28. Sadaka F, Patel D, Lakshmanan R. The FOUR score predicts outcome in patients after traumatic brain injury. Neurocrit Care 2012;16(1):95–101. DOI: 10.1007/s12028-011-9617-5.

29. Cheng K, Bassil R, Carandang R, Hall W, Muehlschlegel S. The estimated verbal GCS subscore in intubated traumatic brain injury patients: is it really better? J Neurotrauma 2017;34(8):1603–1609. DOI: 10.1089/neu.2016.4657.

30. Cerro L, Valencia J, Calle P, Leon A, Jaimes F. [Validation of APACHE II and SOFA scores in 2 cohorts of patients with suspected infection and sepsis, not admitted to critical care units]. Rev Esp Anestesiol Reanim 2014;61(3):125–132. DOI: 10.1016/j.redar.2013.11.014.

________________________
© The Author(s). 2022 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by-nc/4.0/), which permits unrestricted use, distribution, and non-commercial reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.