Modified Early Warning Score vs Cardiac Arrest Risk Triage Score for Prediction of Cardiopulmonary Arrest: A Case–Control Study
Corresponding Author: Armand Delo Antone Tan, Department of Adult Cardiology, Philippine Heart Center, Quezon City, Philippines, e-mail: email@example.com
How to cite this article: Tan ADA, Permejo CC, Torres MCD. Modified Early Warning Score vs Cardiac Arrest Risk Triage Score for Prediction of Cardiopulmonary Arrest: A Case–Control Study. Indian J Crit Care Med 2022;26(7):780–785.
Source of support: Nil
Conflict of interest: None
Background: Delayed transfer to the intensive care unit (ICU) contributes to increased mortality. Clinical tools, developed to shorten this delay, are especially useful in hospitals where the ideal healthcare provider-to-patient ratio is not met. This study aimed to validate and compare the accuracy of the well-accepted modified early warning score (MEWS) and the newer cardiac arrest risk triage (CART) score in the Philippine setting.
Patients and methods: This case–control study involved 82 adult patients admitted to the Philippine Heart Center. Patients who had cardiopulmonary (CP) arrest at the wards and those transferred to the ICU were included. Vital signs and alert-verbal-pain-unresponsive (AVPU) scales were recorded from recruitment until 48 hours prior to CP arrest or ICU transfer. The MEWS and CART scores were computed at specific time points and compared using measures of validity.
Results: The highest accuracy was obtained by the CART score with a cut-off of ≥12 at 8 hours prior to CP arrest or ICU transfer, with a specificity of 80.43% and sensitivity of 66.67%. At this time point, the MEWS with a cut-off of ≥3 had a specificity of 78.26% but a lower sensitivity of 58.33%. The area under the curve (AUC) analysis revealed that these differences were not statistically significant.
Conclusion: We recommend an MEWS threshold of 3 and a CART score threshold of 12 to help identify patients at risk for clinical deterioration. The CART score had comparable accuracy to the MEWS, but the latter’s computation may be easier.
Keywords: Cardiac arrest, Critical care, Early warning score, Rapid response team.
Tan and colleagues compared the accuracy of the well-accepted MEWS and the newer CART score and found that the CART score had higher sensitivity and specificity in predicting CP arrest compared to the MEWS. However, this difference was not statistically significant.
Failure to recognize a decline in status among patients in the hospital wards is common and leads to delayed transfer to the ICU, subsequently increasing mortality rates.1,2 This is a familiar scenario in many hospitals, where the ratio of patients to health providers greatly favors the former. Many patients who experience clinical deterioration have abnormal vital signs, laboratory findings, and clinical parameters several hours to days prior to CP arrest.3,4 Several studies have thus developed tools to predict CP arrest and promote early ICU transfers.4–9 The end-point of these studies was to provide health care providers with simple yet efficient criteria to activate rapid response teams (RRTs). The most well-known of these clinical tools is the MEWS.10
The original MEWS was developed by Morgan et al.11 and refined by several studies for use in internal medicine, surgical, and even obstetric cases.10,12,13 The introduction of this tool in hospitals has been associated with better clinical outcomes.13 The current version of the MEWS is the usual comparator by which other tools are measured against. It has been utilized in several studies and actual practice to activate RRTs.14,15
The MEWS uses simple observable bedside parameters to predict the risk of CP arrest (Table 1). In 2012, Churpek et al. developed another tool using similar bedside parameters to predict CP arrest. This system, the CART Score, demonstrated greater accuracy than the MEWS (Table 2).16 Unlike other risk stratification scores,6–9 the beauty of the CART Score and MEWS is their practicability and ease of use. These tools do not need laboratory tests or sophisticated equipment and instead rely on parameters gathered through basic bedside physical examination. These include age, heart rate (HR), respiratory rate (RR), blood pressure (BP), temperature, and neurologic status.
These factors can be observed universally in all settings, regardless of available equipment. This is an important consideration in developing countries, where many health facilities may lack advanced devices and laboratory capabilities.
To our knowledge, no study has validated the use of ward risk scores in hospitals in our country. It was, therefore, our goal to determine if these tools have practical value for predicting cardiac arrest in our local setting.
The main objective of this study was to validate the MEWS and CART scores and compare their accuracy in predicting CP arrest among patients admitted to the wards. The specific objectives were: (1) to determine the sensitivity and specificity of both tools in predicting cardiac arrest and ICU transfers; and (2) to determine the optimal cut-off points of both tools for predicting these events.
PATIENTS AND METHODS
This single-center case–control study was conducted from May 2018 to May 2019. The study was approved by the Institutional Review Board of the institution which waived informed consent.
Flowchart 1 illustrates the methodology applied in this study. Patients were selected from those admitted to the wards. All adult patients (≥19 years of age) who were admitted for at least 48 hours were eligible for the study. Patients who were on a do not resuscitate (DNR) directive prior to arrest were excluded. Cases and their matched controls were simultaneously selected from the same ward. Cases were divided into two subsets: (1) patients who had a CP arrest at the wards despite the outcome (return of spontaneous circulation or death) and (2) patients who were attended by the RRT and subsequently transferred to the ICU. This second subset was included because the main purpose of the CART and MEWS tools is to detect clinical deterioration for immediate intervention (i.e., ICU transfer) before CP arrest occurs.
For each individual case enrolled, 1–2 patients were selected as controls. Controls were patients with similar or matched characteristics to the cases (age, co-morbidities, main diagnosis, and whether the management was medical or surgical) who did not experience CP arrest or ICU transfers.
For all patients, basic and demographic information, diagnoses, vital signs, and level of consciousness prior to the arrest were collected from the chart. If there were missing vital signs in the chart, the most recent recording was imputed as the parameter for that hour. This simulates what is usually done in clinical practice.
The corresponding MEWS and CART scores were computed for each 4-hour interval starting from the time of enrollment up to 48 hours prior. For cases, the starting point was the time of CP arrest or ICU transfer. Only the first arrest was considered when computing the risk scores for cases. Measures of diagnostic accuracy were computed and compared for each time point that the scores were measured.
Descriptive statistics were used to summarize the demographic and clinical characteristics of the patients. Frequency and proportion were used for categorical variables, and mean and SD for normally distributed continuous variables. Independent Sample t-test and Fisher’s Exact/Chi-square test were used to determine the difference in mean and frequency, respectively, between case and control. Sensitivity, specificity, and likelihood ratios as well as the AUCs were used to determine the diagnostic accuracy of CART and MEWS scores to predict CP arrest and ICU transfer. All statistical tests were two-tailed tests. Shapiro–Wilk was used to test the normality of the continuous variables. Missing variables were neither replaced nor estimated. Null hypotheses were rejected at a 0.05α-level of significance. STATA 13.1 was used for data analysis.
A total of 82 patients were enrolled in the study. All patients who were eligible during the study period were enrolled. There were no significant differences between the case and control groups in terms of age, sex, height and weight, and main diagnosis classification (Table 3). Most of the patients enrolled in the study were admitted for cardiovascular disease (47.6%), followed by respiratory (25.8%) and infectious diseases (11%). The top comorbid conditions were hypertension, diabetes mellitus, and chronic kidney disease (CKD). Only among patients with CKD was a significant difference noted (p-value = 0.019) between the two groups. Sensitivity analysis, however, showed that removing these patients from the analysis did not significantly alter the measures of validity of the MEWS and CART among cases and controls.
|Characteristic||Cases (n = 36)||Control (n = 46)|
|Age (years), mean ± SD||63.56 ± 16.40||59.76 ± 17.55|
|Height (cm), mean ± SD||162.06 ± 6.54||159.20 ± 9.18|
|Weight (kg), mean ± SD||64.15 ± 11.99||62.13 ± 12.40|
|Main diagnosis classification|
|Cardiovascular||17 (47.2)||22 (47.8)|
|Respiratory||8 (22.2)||14 (30.4)|
|Infectious||5 (13.9)||4 (8.6)|
|Gastrointestinal||2 (5.6)||3 (6.5)|
|Neurologic||1 (2.8)||2 (4.4)|
|Others||3 (8.3)||1 (2.2)|
|Hypertension||18 (50)||23 (50)|
|Diabetes||10 (27.8)||11 (23.9)|
|Chronic kidney disease||8 (22.2)*||2 (4.3)*|
|Cardiac medical||11 (30.6)||16 (34.8)|
|Cardiac surgical||5 (13.9)||8 (17.4)|
|Non-cardiac medical||19 (52.8)||20 (43.5)|
|Non-cardiac surgical||1 (2.8)||2 (4.3)|
|Prior ICU admission||13 (36.1)||13 (28.3)|
Subjects were categorized into four groups based on their primary management: whether they were admitted for cardiac disease and managed medically (cardiac medical, n = 27) or surgically (cardiac surgical, n = 13); or whether they were admitted for a non-cardiac illness and managed medically (non-cardiac medical, n = 39) or surgically (non-cardiac surgical, n = 3). No significant difference was found between these groups among cases and controls (p-value = 0.91).
None of the patients had prior RRT calls. There was no difference between cases and controls in terms of prior ICU admissions (p-value = 0.48).
Table 4 presents the various cut-off points for the MEWS and CART score 8 hours prior to either event (CP arrest or ICU transfer post RRT). The CART score had a higher accuracy than the MEWS at all time points prior to any event. The highest accuracy of the CART score was 74.39%, noted 8 hours prior to an event. A cut-off of ≥12 at this time point had a specificity of 80.43% and a sensitivity of 66.67 (AUC 0.74). At this time point, a MEWS score of ≥3 had an accuracy of 69.51% with a lower specificity of 78.26% and a lower sensitivity of 30.56% (AUC 0.71).
|Score||Sensitivity (%)||Specificity (%)||Accuracy (%)|
Considering only cases who were ICU transfers post RRT, a CART score of ≥12 had an accuracy of 76.56% with a specificity of 80.43% and a sensitivity of 66.67% (AUC 0.82) 8 hours prior. At this time point, a MEWS score of ≥3 had an accuracy of 73% with a specificity of 78.26% and a sensitivity of 61.11% (AUC 0.71).
Looking at patients who experienced CP arrest, a CART score of ≥12 also had an accuracy of 76.56% with a specificity of 80.43% and a sensitivity of 66.67% (AUC 0.67) 8 hours prior. At this time point, a MEWS score of ≥3 had an accuracy of 71.88% with a specificity of 78.26% and a sensitivity of 55.56% (AUC 0.71).
Despite the above results, however, the AUC analysis of both scores showed that there was no significant difference between the MEWS and CART score across all time points prior to an event (Table 5). Among patients detected by both risk scores, a MEWS score of ≥3 detected CP arrest earlier than a CART score of ≥12 (median, 16 hours vs 8 hours), although this value was not statistically significant (p-value = 0.23).
|Time point (hours)||AUC values||p-value|
Our reason for choosing the MEWS and CART scores for validation and comparison was the premise that simple bedside parameters should be enough to risk-stratify ward patients for ICU transfers. Both scores are primarily comprised of vital signs. These parameters have been found by research to become abnormal several hours prior to CP arrest, and to be individually correlated with CP arrest.18–21
We determined the optimal MEWS cut-off as ≥3, which is less than the cut-off score in other studies.22–26 While the sensitivity of this cut-off was quite low across all time points, it has better specificity than other values. Our argument was that it is more important to have a higher specificity than sensitivity because our intent is to “rule in” the probability of a CP arrest so that we can intervene and prevent its occurrence.
In all time points prior to CP arrest or ICU transfer, a CART score of ≥12 was found to have higher accuracy than a MEWS of ≥3. The highest accuracy for both scores was noted 8 hours prior to an event. However, unlike the study of Churpek et al. which demonstrated greater accuracy of the CART score than the MEWS with AUCs of 0.84 vs 0.76 (p = 0.001), respectively,16 there was no significant difference between the AUCs of both scores in our study. The former study found the median times of CP arrest detection of the CART score and MEWS to be 48 hours vs 42 hours prior to an event, respectively. By comparison, our study found that the median times of CP arrest detection by both scores were closer to the actual time of CP arrest (16 hours for the CART score and 8 hours for the MEWS). However, similar to their results, the difference between the two scores was also not statistically significant (p-value = 0.23).
The AUC values of both the MEWS and CART scores in our study progressively improved as the time-point neared the actual event (CP arrest or ICU transfer). At around 16 hours prior, the AUCs fall within moderate levels of discrimination (Table 5).27 These AUCs are midway between the findings of other recent studies on the MEWS.23–26 They are higher than the findings of Kruisselbrink et al. (AUC 0.69)23 and Ho et al. (AUC 0.68)24 but less than the findings of Xie et al. (AUC 0.83)25 and Peng et al. (AUC 0.96).26 Our inference is that while these risk stratification tools are useful predictors of CP arrest and ICU transfer, they should be viewed as part and parcel of the larger clinical picture of the individual patient.
Based on our study we recommend a MEWS threshold of 3 and a CART score threshold of 12. Both tools were proven to be reliable in our setting. The selection of which tool to use will thus depend on the staff’s preference. The CART score contains fewer variables but may take slightly longer to compute. Our personal opinion is that the MEWS may be easier to calculate because of the relatively short range of value options (0–3) assigned per parameter. It also has the advantage of being a tried and tested warning score as previously mentioned.
These scores should not be used in isolation, but rather taken as part of the overall clinical context of a particular patient. In practical terms, Filipino patients with a MEWS of ≥3 or a CART score of ≥12 will need close monitoring and frequent reevaluation by ward physicians and must be transferred to the ICU in the instance of further deterioration.
We have identified two main limitations to our study. The first is the relatively small sample size. Although this was within the statistically determined sample size target, we acknowledge that a larger population could have provided more robust results. The second limitation is related to the study design. We recognize that our study could have benefited from a prospective design where the vital signs of all admitted patients would be recorded consecutively until an event occurred. However, our hospital does not have a digital system for tracking vital signs since admission. Hence, we felt that a more practical approach was to begin at the time of the event and trace the previous vital signs up to the 48th hour mark prior to the event. The basis for this was the ability of the CART score to detect the risk for CP arrest at a median of 48 hours, based on the study by Churpek et al.16
Having a valid and accurate risk score for predicting CP arrest is an invaluable tool for the management of patients in the wards. This is especially important in the setting of a developing country such as ours, where patients greatly outnumber the healthcare staff. Our study demonstrated that the CART score had statistically comparable accuracy to the MEWS, which is currently the internationally recognized tool for detecting CP arrest at the wards. This research proves that risk scores have great utility in clinical practice and must be used in tandem with a clinical assessment to detect patients at risk for clinical deterioration.
Armand Delo Antone Tan https://orcid.org/0000-0003-2440-8345
Chito Caimoy Permejo https://orcid.org/0000-0002-0005-8072
Ma Consolacion Dolor Torres https://orcid.org/0000-0002-0570-5814
1. Cardoso LTQ, Grion CMC, Matsuo T, Anami EHT, Kauss IAM, Seko L, et al. Impact of delayed admission to intensive care units on mortality of critically ill patients: a cohort study. Crit Care 2011;15(1):R28. DOI: 10.1186/cc9975.
2. Mardini L, Lipes J, Jayaraman D. Adverse outcomes associated with delayed intensive care consultation in medical and surgical inpatients. J Crit Care 2012;27(6):688–693. DOI: 10.1016/j.jcrc.2012.04.011.
3. Kang MA, Churpek MM, Zadravecz FJ, Adhikari R, Twu NM, Edelson DP. Real-time risk prediction on the wards: a feasibility study. Crit Care Med 2016;44(8):1468–1473. DOI: 10.1097/CCM.0000000000001716.
5. Churpek MM, Yuen TC, Winslow C, Robicsek AA, Meltzer DO, Gibbons RD, et al. Multicenter development and validation of a risk stratification tool for ward patients. Am J Respir Crit Care Med 2014;190(6):649–655. DOI: 10.1164/rccm.201406-1022OC.
6. Ong MEH, NgCHL, Goh K, Liu N, Koh ZX, Shahidah N, et al. Prediction of cardiac arrest in critically ill patients presenting to the emergency department using a machine learning score incorporating heart rate variability compared with the modified early warning score. Crit Care 2012;16(3):R108. DOI: 10.1186/cc11396.
7. Bulut M, Cebicci H, Sigirli D, Sak A, Durmus O, Top AA, et al. The comparison of modified early warning score with rapid emergency medicine score: a prospective multicentre observational cohort study on medical and surgical patients presenting to emergency department. Emerg Med J 2014;31(6):476–481. DOI: 10.1136/emermed-2013-202444.
8. Smith GB, Prytherch DR, Meredith P, Schmidt PE, Featherstone PI. The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation 2013;84(4):465–470. DOI: 10.1016/j.resuscitation.2012.12.016.
9. Prytherch DR, Smith GB, Schmidt PE, Featherstone PI. ViEWS—towards a national early warning score for detecting adult inpatient deterioration. Resuscitation 2010;81(8):932–937. DOI: 10.1016/j.resuscitation.2010.04.014.
11. Morgan RJM, Williams F, Wright MM. An early warning scoring system for detecting developing critical illness. Clin Intensive Care 1997;8:100.
12. Khergade M, Suri J, Bharti R, Pandey D, Bachani S, Mittal P. Obstetric early warning score for prognostication of critically ill obstetric patient. Indian J Crit Care Med 2020;24(6):398. DOI: 10.5005/jp-journals-10071-23453.
13. Gardner-Thorpe J, Love N, Wrightson J, Walsh S, Keeling N. The value of Modified Early Warning Score (MEWS) in surgical in-patients: a prospective observational study. Ann R Coll Surg Engl 2006;88(6):571–575. DOI: 10.1308/003588406X130615.
15. Devita MA, Bellomo R, Hillman K, Kellum J, Rotondi A, Teres D, et al. Findings of the first consensus conference on medical emergency teams. Crit Care Med 2006;34(9):2463–2478. DOI: 10.1097/01.CCM.0000235743.38172.6E.
16. Churpek MM, Yuen TC, Park SY, Meltzer DO, Hall JB, Edelson DP. Derivation of a cardiac arrest prediction model using ward vital signs. Critical Care Med 2012;40(7):2102–2108. DOI: 10.1097/CCM.0b013e318250aa5a.
17. Hulley SB, Cummings SR, Browner WS, Grady D, Newman TB. Designing Clinical Research: An Epidemiologic Approach. 4th ed. Philadelphia, PA: Lippincott Williams & Wilkins; 2013. p. 79.
18. Andersen LW, Kim WY, Chase M, Berg KM, Mortensen SJ, Moskowitz A, et al. The prevalence and significance of abnormal vital signs prior to in-hospital cardiac arrest. Resuscitation 2016;98:112–117. DOI: 10.1016/j.resuscitation.2015.08.016.
19. Ljunggren M, Castrén M, Nordberg M, Kurland L. The association between vital signs and mortality in a retrospective cohort study of an unselected emergency department population. Scand J Trauma, Resusc Emerg Med 2016;24(1):21. DOI: 10.1186/s13049-016-0213-8.
20. Barfod C, Lauritzen MM, Danker JK, Sölétormos G, Forberg JL, Berlac PA, et al. Abnormal vital signs are strong predictors for intensive care unit admission and in-hospital mortality in adults triaged in the emergency department-a prospective cohort study. Scand J Trauma, Resusc Emerg Med 2012;20(1):28. DOI: 10.1186/1757-7241-20-28.
21. Oh H, Lee K, Seo W. Temporal patterns of change in vital signs and Cardiac Arrest Risk Triage scores over the 48 hours preceding fatal in‐hospital cardiac arrest. J Adv Nursing 2016;72(5):1122–1133. DOI: 10.1111/jan.12897.
22. Nishijima I, Oyadomari S, Maedomari S, Toma R, Igei C, Kobata S, et al. Use of a modified early warning score system to reduce the rate of in-hospital cardiac arrest. J Intensive Care 2016;4(1):12. DOI: 10.1186/s40560-016-0134-7.
23. Kruisselbrink R, Kwizera A, Crowther M, Fox-Robichaud A, O’Shea T, Nakibuuka J, et al. Modified early warning score (MEWS) identifies critical illness among ward patients in a resource restricted setting in Kampala, Uganda: a prospective observational study. PLoS One 2016;11(3):e0151408. DOI: 10.1371/journal.pone.0151408.
24. Ho LO, Li H, Shahidah N, Koh ZX, Sultana P, Ong MEH. Poor performance of the modified early warning score for predicting mortality in critically ill patients presenting to an emergency department. World J Emerg Med 2013;4(4):273–278. DOI: 10.5847/wjem.j.issn.1920-8642.2013.04.005.
25. Xie X, Huang W, Liu Q, Tan W, Pan L, Wang L, et al. Prognostic value of modified early warning score generated in a Chinese emergency department: a prospective cohort study. BMJ Open 2018;8(12):e024120. DOI: 10.1136/bmjopen-2018-024120.
26. Peng LS, Hassan A, Bustam A, Noor Azhar M, Ahmad R. Using modified early warning score to predict need of lifesaving intervention in adult non-trauma patients in a tertiary state hospital. Hong Kong J Emerg Med 2018;25(3):146–151. DOI: 10.1136/bmjopen-2018-024120.
© The Author(s). 2022 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by-nc/4.0/), which permits unrestricted use, distribution, and non-commercial reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.