Indian Journal of Critical Care Medicine

Register      Login



Volume / Issue

Online First

Related articles

VOLUME 27 , ISSUE 6 ( June, 2023 ) > List of Articles

Original Article

Comparison of Six Scoring Systems for Predicting In-hospital Mortality among Patients with SARS-COV2 Presenting to the Emergency Department

Benyamin Hoseini, Seyyed Mohammad Tabatabaei

Keywords : COVID-19, Emergency department, Mortality prediction, Performance measures, Scoring system

Citation Information : Hoseini B, Tabatabaei SM. Comparison of Six Scoring Systems for Predicting In-hospital Mortality among Patients with SARS-COV2 Presenting to the Emergency Department. Indian J Crit Care Med 2023; 27 (6):416-425.

DOI: 10.5005/jp-journals-10071-24463

License: CC BY-NC 4.0

Published Online: 31-05-2023

Copyright Statement:  Copyright © 2023; The Author(s).


Background: The study aimed to compare the prognostic accuracy of six different severity-of-illness scoring systems for predicting in-hospital mortality among patients with confirmed SARS-COV2 who presented to the emergency department (ED). The scoring systems assessed were worthing physiological score (WPS), early warning score (EWS), rapid acute physiology score (RAPS), rapid emergency medicine score (REMS), national early warning score (NEWS), and quick sequential organ failure assessment (qSOFA). Materials and methods: A cohort study was conducted using data obtained from electronic medical records of 6,429 confirmed SARS-COV2 patients presenting to the ED. Logistic regression models were fitted on the original severity-of-illness scores to assess the models’ performance using the Area Under the Curve for ROC (AUC-ROC) and Precision-Recall curves (AUC-PR), Brier Score (BS), and calibration plots were used to assess the models’ performance. Bootstrap samples with multiple imputations were used for internal validation. Results: The mean age of the patients was 64 years (IQR:50–76) and 57.5% were male. The WPS, REMS, and NEWS models had AUROC of 0.714, 0.705, and 0.701, respectively. The poorest performance was observed in the RAPS model, with an AUROC of 0.601. The BS for the NEWS, qSOFA, EWS, WPS, RAPS, and REMS was 0.18, 0.09, 0.03, 0.14, 0.15, and 0.11 respectively. Excellent calibration was obtained for the NEWS, while the other models had proper calibration. Conclusion: The WPS, REMS, and NEWS have a fair discriminatory performance and may assist in risk stratification for SARS-COV2 patients presenting to the ED. Generally, underlying diseases and most vital signs are positively associated with mortality and were different between the survivors and non-survivors.

  1. Ghale-Noie ZN, Salmaninejad A, Bergquist R, Mollazadeh S, Hoseini B, Sahebkar A. Genetic aspects and immune responses in COVID-19: Important organ involvement. Adv Exp Med Biol 2021;1327:3–22. DOI: 10.1007/978-3-030-71697-4_1.
  2. Bhatraju PK, Ghassemieh BJ, Nichols M, Kim R, Jerome KR, Nalla AK, et al. COVID-19 in critically ill patients in the seattle region - Case series. N Engl J Med 2020;382(21):2012–2022. DOI: 10.1056/NEJMoa2004500.
  3. Goshayeshi L, Akbari Rad M, Bergquist R, Allahyari A, Hashemzadeh K, Hoseini B. Demographic and clinical characteristics of severe COVID-19 infections: a cross-sectional study from Mashhad University of Medical Sciences, Iran. BMC Infect Dis 2021;21(1):656. DOI: 10.1186/s12879-021-06363-6
  4. Carlos WG, Dela Cruz CS, Cao B, Pasnick S, Jamil S. Novel wuhan (2019-nCoV) coronavirus. Am J Respir Crit Care Med 2020;201(4):P7–P8. DOI: 10.1164/rccm.2014P7.
  5. Paules CI, Marston HD, Fauci AS. Coronavirus infections—more than just the common cold. JAMA 2020;323(8):707–708. DOI: 10.1001/jama.2020.0757.
  6. Goshayeshi L, Milani N, Bergquist R, Sadrzadeh SM, Rajabzadeh F, Hoseini B. Covid-19 presented only with gastrointestinal symptoms: a case report of a 14-year-old patient. Govaresh. 2021;25(4): 300–304.
  7. Holshue ML, DeBolt C, Lindquist S, Lofy KH, Wiesman J, Bruce H, et al. First case of 2019 novel coronavirus in the united states. N Engl J Med 2020;382(10):929–936. DOI: 10.1056/NEJMoa2001191.
  8. Redd WD, Zhou JC, Hathorn KE, McCarty TR, Bazarbashi AN, Thompson CC, et al. Prevalence and characteristics of gastrointestinal symptoms in patients with sars-cov-2 infection in the United States: a multicenter cohort study. Gastroenterology 2020;159(2):765–767. DOI:
  9. Long B, Brady WJ, Koyfman A, Gottlieb M. Cardiovascular complications in covid-19. Emerg Med 2020;38(7):1504–1507. DOI: 10.1016/j.ajem.2020.04.048.
  10. Levin S, Toerper M, Hamrock E, Hinson JS, Barnes S, Gardner H, et al. Machine-learning-based electronic triage more accurately differentiates patients with respect to clinical outcomes compared with the emergency severity index. Ann Emerg Med 2018;71(5):565–574. e2. DOI: 10.1016/j.annemergmed.2017.08.005.
  11. Khodashahi R, Naderi H, Bojdy A, Heydari AA, Tavanaee SA, Javad MG, et al. Comparison the effect of arbidol plus hydroxychloroquine vs hydroxychloroquine alone in treatment of covid-19 disease: A randomized clinical trial. Current Respiratory Medicine Reviews 2020;16(4):252–262. DOI: 10.2174/1573398X17666210129125703.
  12. Mareiniss DP. The impending storm: COVID-19, pandemics and our overwhelmed emergency departments. Am J Emerg Med 2020;38(6):1293–1294. DOI: 10.1016/j.ajem.2020.03.033.
  13. Khodashahi R, Naderi HR, Sedaghat A, Allahyari A, Sarjamee AS, Eshaghi AS, et al. Intravenous immunoglobulin for treatment of patients with covid-19: A case-control study. research article. Arch Clin Infect Dis 2021;16(1):e108068. DOI: 10.5812/archcid.108068.
  14. Rahmatinejad Z, Hoseini B, Rahmatinejad F, Abu-Hanna A, Bergquist R, Pourmand A, et al. Internal validation of the predictive performance of models based on three ed and icu scoring systems to predict inhospital mortality for intensive care patients referred from the emergency department. Biomed Res Int 2022;2022:3964063. DOI: 10.1155/2022/3964063.
  15. Rahmatinejad Z, Rahmatinejad F, Sezavar M, Tohidinezhad F, Abu-Hanna A, Eslami S. Internal validation and evaluation of the predictive performance of models based on the PRISM-3 (Pediatric Risk of Mortality) and PIM-3 (Pediatric Index of Mortality) scoring systems for predicting mortality in Pediatric Intensive Care Units (PICUs). BMC Pediatr 2022;22(1):199. DOI: 10.1186/s12887-022-03228-y.
  16. Alireza A, Leila A, Zahra R, Mirmohammad M, Najmeh N, Saeid E. Development of a national core dataset for the Iranian ICU patients outcome prediction: a comprehensive approach. J Innov Health Inform 2018;25(2):71–76. DOI: 10.14236/jhi.v25i2.953.
  17. Rahmatinejad Z, Tohidinezhad F, Reihani H, Rahmatinejad F, Pourmand A, Abu-Hanna A, et al. Prognostic utilization of models based on the APACHE II, APACHE IV, and SAPS II scores for predicting in-hospital mortality in emergency department. The American Journal of Emergency Medicine 2020/09/01/2020;38(9):1841–1846. DOI:
  18. Rahmatinejad Z, Tohidinezhad F, Rahmatinejad F, Eslami S, Pourmand A, Abu-Hanna A, et al. Internal validation and comparison of the prognostic performance of models based on six emergency scoring systems to predict in-hospital mortality in the emergency department. BMC Emerg Med 2021;21(1):68. DOI: 10.1186/s12873-021-00459-7.
  19. Rahmatinejad Z, Reihani H, Tohidinezhad F, Rahmatinejad F, Peyravi S, Pourmand A, et al. Predictive performance of the SOFA and mSOFA scoring systems for predicting in-hospital mortality in the emergency department. The American Journal of Emergency Medicine 2019;37(7):1237–1241. DOI:
  20. Rahmatinejad F, Rahmatinejad Z, Kimiafar K, Eslami S, Hoseini B. Performance of pediatric risk of mortality and pediatric index of mortality in pediatric intensive care units: A case study of patients with digestive diseases. GOVARESH. 2022;26(3):132–142.
  21. Brabrand M, Folkestad L, Clausen NG, Knudsen T, Hallas J. Risk scoring systems for adults admitted to the emergency department: a systematic review. Scand J Trauma Resusc Emerg Med 2010;18(1):8. DOI: 10.1186/1757-7241-18-8.
  22. Olivia D, Nayak A, Balachandra M, John J. A classification model for prediction of clinical severity level using qSOFA medical score. Information Discovery and Delivery 2020; DOI: 10.1108/idd-02-2019-0013.
  23. Song C-Y, Xu J, He J-Q, Lu Y-Q. COVID-19 early warning score: A multi-parameter screening tool to identify highly suspected patients. medRxiv. 2020; DOI:
  24. Kim I, Song H, Kim HJ, Park KN, Kim SH, Oh SH, et al. Use of the National Early Warning Score for predicting in-hospital mortality in older adults admitted to the emergency department. Clin Exp Emerg Med 2020;7(1):61–66. DOI: 10.15441/ceem.19.036.
  25. Guo Y, Wang Y, Ma C, Li R, Li T. Performance of quick sequential organ failure assessment (qSOFA) score for prognosis of heat-related hospitalized patients. Heart Lung 2020;49(4):415–419. DOI: 10.1016/j.hrtlng.2020.02.040.
  26. Groarke J, Gallagher J, Stack J, Aftab A, Dwyer C, McGovern R, et al. Use of an admission early warning score to predict patient morbidity and mortality and treatment success. Emerg Med J 2008;25(12):803–806. DOI: 10.1136/emj.2007.051425.
  27. Duckitt R, Buxton-Thomas R, Walker J, Cheek E, Bewick V, Venn R, et al. Worthing physiological scoring system: derivation and validation of a physiological early-warning system for medical admissions. An observational, population-based single-centre study. Br J Anaesth 2007;98(6):769–774. DOI: 10.1093/bja/aem097.
  28. Rhee KJ, Fisher Jr CJ, Willitis NH. The rapid acute physiology score. Am J Emerg Med 1987;5(4):278–282. DOI: 10.1016/0735-6757(87)90350-0.
  29. Olsson T, Terént A, Lind L. Rapid emergency medicine score: A new prognostic tool for in-hospital mortality in nonsurgical emergency department patients. J Intern Med 2004;255(5):579–587. DOI: 10.1111/j.1365-2796.2004.01321.x.
  30. Sabetian G, Azimi A, Kazemi A, Hoseini B, Asmarian N, Khaloo V, et al. Prediction of patients with covid-19 requiring intensive care: a cross-sectional study based on machine-learning approach from iran. Indian J Crit Care Med 2022;26(6):688–695. DOI: 10.5005/jp-journals-10071-24226.
  31. Schomaker M, Heumann C. Bootstrap inference when using multiple imputation. Stat med 2018;37(14):2252–2266. DOI: 10.1002/sim.7654.
  32. Hu H, Kong W, Yao N, Qiu Y, Gu H, Xu W. Prognostic value of three rapid scoring scales and combined score for the assessment of patients with coronavirus disease 2019. Nurs Open 2022;9(3):1865–1872. DOI: 10.1002/nop2.934.
  33. Hu H, Yao N, Qiu Y. Comparing rapid scoring systems in mortality prediction of critically ill patients with novel coronavirus disease. Acad Emerg Med 2020;27(6):461–468. DOI: 10.1111/acem.13992.
  34. Liu S, Yao N, Qiu Y, He C. Predictive performance of SOFA and qSOFA for in-hospital mortality in severe novel coronavirus disease. Am J Emerg Med 2020;38(10):2074–2080. DOI: 10.1016/j.ajem.2020.07.019.
  35. Haimovich AD, Ravindra NG, Stoytchev S, Young HP, Wilson FP, van Dijk D, et al. Development and validation of the quick COVID-19 severity index: A prognostic tool for early clinical decompensation. Ann Emerg Med 2020;76(4):442–453. DOI: 10.1016/j.annemergmed.2020.07.022.
  36. Zhou F, Yu T, Du R, Fan G, Liu Y, Liu Z, et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet 2020;395(10229):1054–1062. DOI: 10.1016/S0140-6736(20)30566-3.
  37. Zhou Y, He Y, Yang H, Yu H, Wang T, Chen Z, et al. Development and validation a nomogram for predicting the risk of severe COVID-19: A multi-center study in Sichuan, China. Plos one 2020;15(5):e0233328. DOI: 10.1371/journal.pone.0233328.
  38. Ehmann MR, Zink EK, Levin AB, Suarez JI, Belcher HME, Daugherty Biddison EL, et al. Operational recommendations for scarce resource allocation in a public health crisis. Chest 2020;159:10231. DOI: 10.1016/j.chest.2020.09.246.
  39. Maves RC, Downar J, Dichter JR, Hick JL, Devereaux A, Geiling JA, et al. Triage of scarce critical care resources in COVID-19: An implementation guide for regional allocation an expert panel report of the task force for mass critical care and the American College of Chest Physicians. Chest 2020; 158(1):212–225. DOI: 10.1016/j.chest.2020.03.063.
  40. Cheng FY, Joshi H, Tandon P, Freeman R, Reich DL, Mazumdar M, et al. Using machine learning to predict ICU transfer in hospitalized COVID-19 patients. J Clin Med 2020;9(6):1668. DOI: 10.3390/jcm9061668.
  41. Heo J, Han D, Kim H-J, Kim D, Lee YK, Lim D, et al. Prediction of patients requiring intensive care for COVID-19: development and validation of an integer-based score using data from Centers for Disease Control and Prevention of South Korea. J Intensive Care 2021;9(1):16. DOI: 10.1186/s40560-021-00527-x.
  42. Saba T, Abunadi I, Shahzad MN, Khan AR. Machine learning techniques to detect and forecast the daily total COVID-19 infected and deaths cases under different lockdown types. Microsc Res Tech;84(7):1462–1474. DOI: 10.1002/jemt.23702.
  43. Surme S, Buyukyazgan A, Bayramlar OF, Cinar AK, Copur B, Zerdali E, et al. Predictors of intensive care unit admission or mortality in patients with coronavirus disease 2019 pneumonia in Istanbul, Turkey. Jpn J Infect Dis 2021;74(5):458–464. DOI: 10.7883/yoken.JJID.2020.1065.
PDF Share
PDF Share

© Jaypee Brothers Medical Publishers (P) LTD.