Modified Early Warning Score vs Cardiac Arrest Risk Triage Score for Prediction of Cardiopulmonary Arrest: A Case–Control Study

HIGHLIGHTS

Tan and colleagues compared the accuracy of the well-accepted MEWS and the newer CART score and found that the CART score had higher sensitivity and specificity in predicting CP arrest compared to the MEWS. However, this difference was not statistically significant.

INTRODUCTION

Failure to recognize a decline in status among patients in the hospital wards is common and leads to delayed transfer to the ICU, subsequently increasing mortality rates.^1,2 This is a familiar scenario in many hospitals, where the ratio of patients to health providers greatly favors the former. Many patients who experience clinical deterioration have abnormal vital signs, laboratory findings, and clinical parameters several hours to days prior to CP arrest.^3,4 Several studies have thus developed tools to predict CP arrest and promote early ICU transfers.^4–9 The end-point of these studies was to provide health care providers with simple yet efficient criteria to activate rapid response teams (RRTs). The most well-known of these clinical tools is the MEWS.¹⁰

The original MEWS was developed by Morgan et al.¹¹ and refined by several studies for use in internal medicine, surgical, and even obstetric cases.^10,12,13 The introduction of this tool in hospitals has been associated with better clinical outcomes.¹³ The current version of the MEWS is the usual comparator by which other tools are measured against. It has been utilized in several studies and actual practice to activate RRTs.^14,15

The MEWS uses simple observable bedside parameters to predict the risk of CP arrest (Table 1). In 2012, Churpek et al. developed another tool using similar bedside parameters to predict CP arrest. This system, the CART Score, demonstrated greater accuracy than the MEWS (Table 2).¹⁶ Unlike other risk stratification scores,^6–9 the beauty of the CART Score and MEWS is their practicability and ease of use. These tools do not need laboratory tests or sophisticated equipment and instead rely on parameters gathered through basic bedside physical examination. These include age, heart rate (HR), respiratory rate (RR), blood pressure (BP), temperature, and neurologic status.

These factors can be observed universally in all settings, regardless of available equipment. This is an important consideration in developing countries, where many health facilities may lack advanced devices and laboratory capabilities.

To our knowledge, no study has validated the use of ward risk scores in hospitals in our country. It was, therefore, our goal to determine if these tools have practical value for predicting cardiac arrest in our local setting.

The main objective of this study was to validate the MEWS and CART scores and compare their accuracy in predicting CP arrest among patients admitted to the wards. The specific objectives were: (1) to determine the sensitivity and specificity of both tools in predicting cardiac arrest and ICU transfers; and (2) to determine the optimal cut-off points of both tools for predicting these events.

PATIENTS AND METHODS

RESULTS

A total of 82 patients were enrolled in the study. All patients who were eligible during the study period were enrolled. There were no significant differences between the case and control groups in terms of age, sex, height and weight, and main diagnosis classification (Table 3). Most of the patients enrolled in the study were admitted for cardiovascular disease (47.6%), followed by respiratory (25.8%) and infectious diseases (11%). The top comorbid conditions were hypertension, diabetes mellitus, and chronic kidney disease (CKD). Only among patients with CKD was a significant difference noted (p-value = 0.019) between the two groups. Sensitivity analysis, however, showed that removing these patients from the analysis did not significantly alter the measures of validity of the MEWS and CART among cases and controls.

Subjects were categorized into four groups based on their primary management: whether they were admitted for cardiac disease and managed medically (cardiac medical, n = 27) or surgically (cardiac surgical, n = 13); or whether they were admitted for a non-cardiac illness and managed medically (non-cardiac medical, n = 39) or surgically (non-cardiac surgical, n = 3). No significant difference was found between these groups among cases and controls (p-value = 0.91).

None of the patients had prior RRT calls. There was no difference between cases and controls in terms of prior ICU admissions (p-value = 0.48).

Table 4 presents the various cut-off points for the MEWS and CART score 8 hours prior to either event (CP arrest or ICU transfer post RRT). The CART score had a higher accuracy than the MEWS at all time points prior to any event. The highest accuracy of the CART score was 74.39%, noted 8 hours prior to an event. A cut-off of ≥12 at this time point had a specificity of 80.43% and a sensitivity of 66.67 (AUC 0.74). At this time point, a MEWS score of ≥3 had an accuracy of 69.51% with a lower specificity of 78.26% and a lower sensitivity of 30.56% (AUC 0.71).

Considering only cases who were ICU transfers post RRT, a CART score of ≥12 had an accuracy of 76.56% with a specificity of 80.43% and a sensitivity of 66.67% (AUC 0.82) 8 hours prior. At this time point, a MEWS score of ≥3 had an accuracy of 73% with a specificity of 78.26% and a sensitivity of 61.11% (AUC 0.71).

Looking at patients who experienced CP arrest, a CART score of ≥12 also had an accuracy of 76.56% with a specificity of 80.43% and a sensitivity of 66.67% (AUC 0.67) 8 hours prior. At this time point, a MEWS score of ≥3 had an accuracy of 71.88% with a specificity of 78.26% and a sensitivity of 55.56% (AUC 0.71).

Despite the above results, however, the AUC analysis of both scores showed that there was no significant difference between the MEWS and CART score across all time points prior to an event (Table 5). Among patients detected by both risk scores, a MEWS score of ≥3 detected CP arrest earlier than a CART score of ≥12 (median, 16 hours vs 8 hours), although this value was not statistically significant (p-value = 0.23).

DISCUSSION

Our reason for choosing the MEWS and CART scores for validation and comparison was the premise that simple bedside parameters should be enough to risk-stratify ward patients for ICU transfers. Both scores are primarily comprised of vital signs. These parameters have been found by research to become abnormal several hours prior to CP arrest, and to be individually correlated with CP arrest.^18–21

We determined the optimal MEWS cut-off as ≥3, which is less than the cut-off score in other studies.^22–26 While the sensitivity of this cut-off was quite low across all time points, it has better specificity than other values. Our argument was that it is more important to have a higher specificity than sensitivity because our intent is to “rule in” the probability of a CP arrest so that we can intervene and prevent its occurrence.

In all time points prior to CP arrest or ICU transfer, a CART score of ≥12 was found to have higher accuracy than a MEWS of ≥3. The highest accuracy for both scores was noted 8 hours prior to an event. However, unlike the study of Churpek et al. which demonstrated greater accuracy of the CART score than the MEWS with AUCs of 0.84 vs 0.76 (p = 0.001), respectively,¹⁶ there was no significant difference between the AUCs of both scores in our study. The former study found the median times of CP arrest detection of the CART score and MEWS to be 48 hours vs 42 hours prior to an event, respectively. By comparison, our study found that the median times of CP arrest detection by both scores were closer to the actual time of CP arrest (16 hours for the CART score and 8 hours for the MEWS). However, similar to their results, the difference between the two scores was also not statistically significant (p-value = 0.23).

The AUC values of both the MEWS and CART scores in our study progressively improved as the time-point neared the actual event (CP arrest or ICU transfer). At around 16 hours prior, the AUCs fall within moderate levels of discrimination (Table 5).²⁷ These AUCs are midway between the findings of other recent studies on the MEWS.^23–26 They are higher than the findings of Kruisselbrink et al. (AUC 0.69)²³ and Ho et al. (AUC 0.68)²⁴ but less than the findings of Xie et al. (AUC 0.83)²⁵ and Peng et al. (AUC 0.96).²⁶ Our inference is that while these risk stratification tools are useful predictors of CP arrest and ICU transfer, they should be viewed as part and parcel of the larger clinical picture of the individual patient.

Based on our study we recommend a MEWS threshold of 3 and a CART score threshold of 12. Both tools were proven to be reliable in our setting. The selection of which tool to use will thus depend on the staff’s preference. The CART score contains fewer variables but may take slightly longer to compute. Our personal opinion is that the MEWS may be easier to calculate because of the relatively short range of value options (0–3) assigned per parameter. It also has the advantage of being a tried and tested warning score as previously mentioned.

These scores should not be used in isolation, but rather taken as part of the overall clinical context of a particular patient. In practical terms, Filipino patients with a MEWS of ≥3 or a CART score of ≥12 will need close monitoring and frequent reevaluation by ward physicians and must be transferred to the ICU in the instance of further deterioration.

LIMITATIONS

We have identified two main limitations to our study. The first is the relatively small sample size. Although this was within the statistically determined sample size target, we acknowledge that a larger population could have provided more robust results. The second limitation is related to the study design. We recognize that our study could have benefited from a prospective design where the vital signs of all admitted patients would be recorded consecutively until an event occurred. However, our hospital does not have a digital system for tracking vital signs since admission. Hence, we felt that a more practical approach was to begin at the time of the event and trace the previous vital signs up to the 48th hour mark prior to the event. The basis for this was the ability of the CART score to detect the risk for CP arrest at a median of 48 hours, based on the study by Churpek et al.¹⁶

CONCLUSION

Having a valid and accurate risk score for predicting CP arrest is an invaluable tool for the management of patients in the wards. This is especially important in the setting of a developing country such as ours, where patients greatly outnumber the healthcare staff. Our study demonstrated that the CART score had statistically comparable accuracy to the MEWS, which is currently the internationally recognized tool for detecting CP arrest at the wards. This research proves that risk scores have great utility in clinical practice and must be used in tandem with a clinical assessment to detect patients at risk for clinical deterioration.

ORCID

Armand Delo Antone Tan https://orcid.org/0000-0003-2440-8345

Chito Caimoy Permejo https://orcid.org/0000-0002-0005-8072

Ma Consolacion Dolor Torres https://orcid.org/0000-0002-0570-5814

REFERENCES

1. Cardoso LTQ, Grion CMC, Matsuo T, Anami EHT, Kauss IAM, Seko L, et al. Impact of delayed admission to intensive care units on mortality of critically ill patients: a cohort study. Crit Care 2011;15(1):R28. DOI: 10.1186/cc9975.

2. Mardini L, Lipes J, Jayaraman D. Adverse outcomes associated with delayed intensive care consultation in medical and surgical inpatients. J Crit Care 2012;27(6):688–693. DOI: 10.1016/j.jcrc.2012.04.011.

3. Kang MA, Churpek MM, Zadravecz FJ, Adhikari R, Twu NM, Edelson DP. Real-time risk prediction on the wards: a feasibility study. Crit Care Med 2016;44(8):1468–1473. DOI: 10.1097/CCM.0000000000001716.

4. Churpek MM, Yuen TC, Edelson DP. Risk stratification of hospitalized patients on the wards. Chest 2013;143(6):1758–1765. DOI: 10.1378/chest.12-1605.

5. Churpek MM, Yuen TC, Winslow C, Robicsek AA, Meltzer DO, Gibbons RD, et al. Multicenter development and validation of a risk stratification tool for ward patients. Am J Respir Crit Care Med 2014;190(6):649–655. DOI: 10.1164/rccm.201406-1022OC.

6. Ong MEH, NgCHL, Goh K, Liu N, Koh ZX, Shahidah N, et al. Prediction of cardiac arrest in critically ill patients presenting to the emergency department using a machine learning score incorporating heart rate variability compared with the modified early warning score. Crit Care 2012;16(3):R108. DOI: 10.1186/cc11396.

7. Bulut M, Cebicci H, Sigirli D, Sak A, Durmus O, Top AA, et al. The comparison of modified early warning score with rapid emergency medicine score: a prospective multicentre observational cohort study on medical and surgical patients presenting to emergency department. Emerg Med J 2014;31(6):476–481. DOI: 10.1136/emermed-2013-202444.

8. Smith GB, Prytherch DR, Meredith P, Schmidt PE, Featherstone PI. The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation 2013;84(4):465–470. DOI: 10.1016/j.resuscitation.2012.12.016.

9. Prytherch DR, Smith GB, Schmidt PE, Featherstone PI. ViEWS—towards a national early warning score for detecting adult inpatient deterioration. Resuscitation 2010;81(8):932–937. DOI: 10.1016/j.resuscitation.2010.04.014.

10. Subbe CP, Kruger M, Rutherford P, Gemmel L. Validation of a modified Early Warning Score in medical admissions. QJM 2001;94(10):521–526. DOI: 10.1093/qjmed/94.10.521.

11. Morgan RJM, Williams F, Wright MM. An early warning scoring system for detecting developing critical illness. Clin Intensive Care 1997;8:100.

12. Khergade M, Suri J, Bharti R, Pandey D, Bachani S, Mittal P. Obstetric early warning score for prognostication of critically ill obstetric patient. Indian J Crit Care Med 2020;24(6):398. DOI: 10.5005/jp-journals-10071-23453.

13. Gardner-Thorpe J, Love N, Wrightson J, Walsh S, Keeling N. The value of Modified Early Warning Score (MEWS) in surgical in-patients: a prospective observational study. Ann R Coll Surg Engl 2006;88(6):571–575. DOI: 10.1308/003588406X130615.

14. Hillman K. Rapid response systems. Indian J Crit Care Med 2008;12(2):77–81. DOI: 10.4103/0972-5229.42561.

15. Devita MA, Bellomo R, Hillman K, Kellum J, Rotondi A, Teres D, et al. Findings of the first consensus conference on medical emergency teams. Crit Care Med 2006;34(9):2463–2478. DOI: 10.1097/01.CCM.0000235743.38172.6E.

16. Churpek MM, Yuen TC, Park SY, Meltzer DO, Hall JB, Edelson DP. Derivation of a cardiac arrest prediction model using ward vital signs. Critical Care Med 2012;40(7):2102–2108. DOI: 10.1097/CCM.0b013e318250aa5a.

17. Hulley SB, Cummings SR, Browner WS, Grady D, Newman TB. Designing Clinical Research: An Epidemiologic Approach. 4th ed. Philadelphia, PA: Lippincott Williams & Wilkins; 2013. p. 79.

18. Andersen LW, Kim WY, Chase M, Berg KM, Mortensen SJ, Moskowitz A, et al. The prevalence and significance of abnormal vital signs prior to in-hospital cardiac arrest. Resuscitation 2016;98:112–117. DOI: 10.1016/j.resuscitation.2015.08.016.

19. Ljunggren M, Castrén M, Nordberg M, Kurland L. The association between vital signs and mortality in a retrospective cohort study of an unselected emergency department population. Scand J Trauma, Resusc Emerg Med 2016;24(1):21. DOI: 10.1186/s13049-016-0213-8.

20. Barfod C, Lauritzen MM, Danker JK, Sölétormos G, Forberg JL, Berlac PA, et al. Abnormal vital signs are strong predictors for intensive care unit admission and in-hospital mortality in adults triaged in the emergency department-a prospective cohort study. Scand J Trauma, Resusc Emerg Med 2012;20(1):28. DOI: 10.1186/1757-7241-20-28.

21. Oh H, Lee K, Seo W. Temporal patterns of change in vital signs and Cardiac Arrest Risk Triage scores over the 48 hours preceding fatal in‐hospital cardiac arrest. J Adv Nursing 2016;72(5):1122–1133. DOI: 10.1111/jan.12897.

22. Nishijima I, Oyadomari S, Maedomari S, Toma R, Igei C, Kobata S, et al. Use of a modified early warning score system to reduce the rate of in-hospital cardiac arrest. J Intensive Care 2016;4(1):12. DOI: 10.1186/s40560-016-0134-7.

23. Kruisselbrink R, Kwizera A, Crowther M, Fox-Robichaud A, O’Shea T, Nakibuuka J, et al. Modified early warning score (MEWS) identifies critical illness among ward patients in a resource restricted setting in Kampala, Uganda: a prospective observational study. PLoS One 2016;11(3):e0151408. DOI: 10.1371/journal.pone.0151408.

24. Ho LO, Li H, Shahidah N, Koh ZX, Sultana P, Ong MEH. Poor performance of the modified early warning score for predicting mortality in critically ill patients presenting to an emergency department. World J Emerg Med 2013;4(4):273–278. DOI: 10.5847/wjem.j.issn.1920-8642.2013.04.005.

25. Xie X, Huang W, Liu Q, Tan W, Pan L, Wang L, et al. Prognostic value of modified early warning score generated in a Chinese emergency department: a prospective cohort study. BMJ Open 2018;8(12):e024120. DOI: 10.1136/bmjopen-2018-024120.

26. Peng LS, Hassan A, Bustam A, Noor Azhar M, Ahmad R. Using modified early warning score to predict need of lifesaving intervention in adult non-trauma patients in a tertiary state hospital. Hong Kong J Emerg Med 2018;25(3):146–151. DOI: 10.1136/bmjopen-2018-024120.

27. Mandrekar JN. Receiver operating characteristic curve in diagnostic test assessment. J Thorac Oncol 2010;5(9):1315–1316. DOI: 10.1097/JTO.0b013e3181ec173d.