Interrater Reliability of Four Neurological Scales for Patients Presenting to the Emergency Department

INTRODUCTION

Altered mental status (AMS) is a cluster of clinical symptoms that includes reduced levels of consciousness and deranged attention, arousal, and cognitive abilities.¹ Patients with undifferentiated AMS in who the exact etiology is unknown present frequently to the emergency department (ED). They often manifest vague symptoms; thus, evaluation and management of such patients may become highly challenging for the emergency physician.

To assess AMS, various neurological scales have been developed which include the Advanced Trauma Life Support AVPU (alert, responds to verbal and painful stimuli or unresponsive) scale,² ACDU (alert, confused, drowsy, unresponsive) scale 3, Glasgow coma scale (GCS),³ and Swedish reaction level scale.⁴

In patients with altered levels of consciousness from traumatic and nontraumatic causes, the GCS has been found to exhibit moderate interrater reliability when tested in the ED.⁵ Neurological assessment scales other than the GCS have never been assessed or studied to ensure reliability and validity.⁶ Gill et al. recently proposed the simplified motor scale (SMS) which is a statistically derived scale as a possible replacement for the GCS to overcome limitations of the GCS score without losing its prognostic performance.⁷ SMS score is assigned as: 2—obeys commands, 1—localizes pain, and 0—withdrawal to pain or less response. In comparison to the total GCS score or its three individual components, SMS has shown to have equivalent performance for the prediction of adverse neurological outcomes after traumatic brain injury with varying mechanisms of injury and levels of injury severities.

It has been previously suggested to simplify the GCS score to one of its three components. The motor component of the GCS had similar accuracy as a total GCS score in predicting severe traumatic brain injury (TBI), as found by Ross et al.⁸ In 200,000 patients, Healey et al. found that the motor component preserved the power of the total GCS to predict survival.⁹ Similarly, Meredith et al. found that a motor score of less than 6 correlated with an increased risk of mortality in the out-of-hospital setting.¹⁰

The SMS was found to be equivalent to the GCS in predicting clinical outcomes in 8,000 patients of TBI.⁷ Despite the abovementioned conclusions of various authors pointing toward the possibility of the SMS to replace the more complex GCS, it would require prospective validation in a new sample of patients with altered levels of consciousness before clinical application. In this study, we chose to test the interrater reliability of the SMS, GCS, AVPU scale and ACDU scale prospectively when applied to patients presenting to the ED with altered mental status.

MATERIALS AND METHODS

STATISTICAL ANALYSIS

All statistical testing was two-sided and performed using a significance (alpha) level of 0.05. Percentage agreement (percentage of patients who received the same score by both the observers for the different scales) and kappa coefficient score were used to determine interrater reliability for SMS, individual components of GCS, AVPU, and ACDU. STATA System, version 9.0 was used to conduct all statistical analysis.

RESULTS

In the 174 patients who were included in the study, 108 (62.06%) were males and 66 (37.93%) were females. The median age was 58 years (19–98 years).

The SMS had the best interrater agreement in terms of percent agreements with 83.91% (95% CI 77.7–88.6%), followed by ACDU which was 76.44% (95% CI 69.0–81.6%), AVPU 75.29% (95% CI 67.8–80.6%), GCS Eye 74.14% (95% CI 66.6–79.6%), GCS Verbal 67.82% (95% CI 60.6–74.3%), and GCS Motor was 64.94% (95% CI 57.6–71.6%) (Fig. 1).

The kappa coefficient score for SMS was 0.75 (95% CI 0.67–0.83), followed by GCS Eye which was 0.63 (95% CI 0.54–0.72), AVPU 0.62 (95% CI 0.52–0.72), ACDU 0.60 (95% CI 0.49–0.71) and GCS Verbal 0.58 (0.49–0.66) and GCS Motor was 0.53 (95% CI 0.44–0.63) (Fig. 2).

The causes of AMS in the study population were found to be infections (26%), trauma (16%), metabolic encephalopathy (13%), and cerebral infarctions (13%). And 12% of the population presented with other causes such as non-traumatic bleed, alcohol and drug intoxications, tumors of the brain, and cerebral venous thrombosis (Fig. 3).

DISCUSSION

The study found the SMS outperform other neurological scales when tested prospectively for interrater reliability in ED patients presenting with AMS.

Subjective evaluation of patients in AMS may vary among physicians which may result in bias. Neurological scales should ideally provide information regarding arousability, level of awareness, and ability to interact.

The GCS was introduced in 1974 as a tool to use in patients with altered levels of consciousness by Teasdale and Jennett, but physicians and investigators have over the many years repeatedly questioned its reliability and practicality.^3,5,7 In acute-care settings, it has been proposed that the use of simpler scales may be more useful and reliable than the GCS.

The motor component of the GCS has been found to be as sensitive as the GCS total to predict patients with severe TBI.⁸ The motor component alone has been shown to be equivalent to the total GCS at predicting mortality in the prehospital setting.¹⁰ The GCS and AVPU scales were compared to each other in toxicology patients, and it was found that the AVPU was simpler for nurses to implement, as concluded by Kelly et al.¹¹ In the neurosurgical setting, ACDU and AVPU scales were compared to the GCS. It was found that the ACDU was more reliable in identifying deteriorating patient conditions than the AVPU and GCS, and a recommendation to follow the ACDU was made.¹²

In patients with head injury, the SMS has been shown to be superior than the GCS and its components.⁵ It has also been shown to reliably predict traumatic brain injury and mortality.^13,14 In a study of trauma patients, it was observed that the SMS or AVPU can potentially replace the total GCS score for trauma triage.¹⁵

In our study, the interrater reliability of the neurological scales were examined for both traumatic and nontraumatic patients in AMS. The SMS displayed a higher reliability as compared to the ACDU, AVPU, and GCS scales. Further larger validation studies to evaluate the SMS for clinical use are recommended.

Since the investigators were not present at all times, possibility of physician bias cannot be overruled. The ease of using SMS compared to the other scales may have resulted in the apparent better performance, as it has fewer categories to be assigned during evaluation which could be a possible limitation of the study.

We conclude that the Simplified Motor Score possessed the best interrater reliability in comparison to the other neurological scales (GCS total and components, AVPU, and ACDU) in both traumatic and nontraumatic settings for patients presenting with AMS.

It may be recommended that the SMS could be used as the neurological assessment scale of choice in the ED setting. However, larger studies as mentioned before are required to validate our findings and to determine its utility as a prognostic tool.

DISCLAIMER

The views expressed in the submitted case report are the author’s own and not an official position of the institution.

CONTRIBUTORS

Haldar M is the principal author; Verma A, Jaiswal S and Sheikh WR are coauthors.

REFERENCES

1. American College of Emergency Physicians. Clinical policy for the initial approach to patients presenting with altered mental status. Ann Emerg Med 1999;33(2):251–280. DOI: 10.1016/S0196-0644(99)70406-3.

2. American College of Surgeons Committee on Trauma. Advanced trauma life support manual.Chicago, IL: American College of Surgeons; 1997.

3. Teasdale G, Jennett B. Assessment of coma and impaired consciousness: a practical scale. Lancet 1974;2(7872):81–84. DOI: 10.1016/S0140-6736(74)91639-0.

4. Johnstone AJ, Lohlun JC, Miller JD, McIntosh CA, Gregori A, Brown R, et al. A comparison of the Glasgow coma scale and the swedish reaction level scale. Brain Inj 1993;7(6):501–506. DOI: 10.3109/02699059309008177.

5. Gill MR, Reiley DG, Green SM. Inter-rater reliability of Glasgow coma scale scores in the emergency department. Ann Emerg Med 2004;43(2):215–223. DOI: 10.1016/S0196-0644(03)00814-X.

6. Gill M, Martens K, Lynch EL, Salih A, Green SM. Inter-rater reliability of 3 simplified neurologic scales applied to adults presenting to the emergency department with altered levels of consciousness. Ann Emerg Med 2007;49(4):403–407. DOI: 10.1016/j.annemergmed.2006.03.031.

7. Gill M, Windemuth R, Steele R, Green SM. A comparison of the Glasgow coma scale score to Simplified alternative scores for the prediction of traumatic brain injury outcomes. Ann Emerg Med 2005;45(1):37–42. DOI: 10.1016/j.annemergmed.2004.07.429.

8. Ross SE, Leipold C, Terregino C, O’Malley KF. Efficacy of the motor component of the Glasgow coma scale in trauma triage. J Trauma 1998;45(1):42–44. DOI: 10.1097/00005373-199807000-00008.

9. Healey C, Osler TM, Rogers FB, Healey MA. Improving the Glasgow coma scale score: motor score alone is a better predictor. J Trauma 2003;54(4):671–678. DOI: 10.1097/01.TA.0000058130.30490.5D.

10. Meredith W, Rutledge R, Hansen AR, Oller DW, Thomason M, Cunningham P, et al. Field triage of trauma patients based upon the ability to follow commands: a study in 29,573 injured patients. J Trauma 1995;38(1):129–135. DOI: 10.1097/00005373-199501000-00030.

11. Kelly CA, Upex A, Bateman DN. Comparison of consciousness level assessment in the poisoned patient using the alert/Verbal/painful/unresponsive scale and the Glasgow coma scale. Ann Emerg Med 2004;44(2):108–113. DOI: 10.1016/j.annemergmed.2004.03.028.

12. McNarry AF, Goldhill DR. Simple bedside assessment of level of consciousness: comparison of two simple assessment scales with the Glasgow coma scale. Anaesthesia 2004;59(1):34–37. DOI: 10.1111/j.1365-2044.2004.03526.x.

13. Buitendag JJP, Ras A, Kong VY, Bruce JL, Laing GL, Clarke DL, et al. Validation of the simplified motor score in patients with traumatic brain injury at a major trauma centre in South Africa. S Afr Med J 2018;108(2):90–93. DOI: 10.7196/SAMJ.2018.v108i2.12757.

14. Singh B, Murad MH, Prokop LJ, Erwin PJ, Wang Z, Mommer SK, et al. Meta-analysis of Glasgow coma scale and Simplified motor score in predicting traumatic brain injury outcomes. Brain Inj 2013;27(3):293–300. DOI: 10.3109/02699052.2012.743182.

15. Hopkins E, Green SM, Kiemeney M, Haukoos JS. A two-center validation of “Patient does not follow Commands” and three other Simplified measures to replace the Glasgow coma scale for field trauma triage. Ann Emerg Med 2018;72(3):259–269. DOI: 10.1016/j.annemergmed.2018.03.038.