Does Neutrophil-to-lymphocyte Ratio at Admission Predict Severity and Mortality in COVID-19 Patients? A Systematic Review and Meta-analysis
1Department of Cardiology, Rajendra Institute of Medical Sciences, Ranchi, Jharkhand, India
2Department of Nephrology, Rajendra Institute of Medical Sciences, Ranchi, Jharkhand, India
3Department of Medicine, Palamu Medical College, Daltonganj, Jharkhand, India
4Department of Laboratory Medicine, Rajendra Institute of Medical Sciences, Ranchi, Jharkhand ,India
Corresponding Author: Prattay Guha Sarkar, Department of Cardiology, Rajendra Institute of Medical Sciences, Ranchi, Jharkhand, India, Phone: +91 7909098629, e-mail: firstname.lastname@example.org
How to cite this article: Sarkar PG, Pant P, Kumar J, Kumar A. Does Neutrophil-to-lymphocyte Ratio at Admission Predict Severity and Mortality in COVID-19 Patients? A Systematic Review and Meta-analysis. Indian J Crit Care Med 2022;26(3):361–375.
Source of support: Nil
Conflict of interest: None
Background: Coronavirus disease-2019 (COVID-2019) pandemic continues to be a significant public health problem. Severe COVID-19 cases have a poor prognosis and extremely high mortality. Prognostic factor evidence can help healthcare providers understand the likely prognosis and identify subgroups likely to develop severe disease with increased mortality risk so that timely treatments can be initiated. This meta-analysis has been performed to evaluate the neutrophil-to-lymphocyte ratio (NLR) at admission as a prognostic factor to predict severe coronavirus disease and mortality.
Materials and methods: A literature search was conducted through April 30, 2021, to retrieve all published studies, including gray literature and preprints, investigating the association between NLR and severity or mortality in COVID-19 patients. Screening of studies and data extraction have been done by two authors independently. The methodological quality of the included studies was assessed by the Quality in Prognosis Studies (QUIPS) tool.
Results: Twenty-four studies involving 4,080 patients reported the prognostic value of NLR for severe COVID-19. The pooled sensitivity (SEN), specificity (SPE), and area under the curve were 0.75 (95% CI 0.69–0.80), 0.74 (95% CI 0.70–0.78), and 0.81 (95% CI 0.77–0.84). Fifteen studies involving 4,071 patients reported the prognostic value of NLR for mortality in COVID-19. The pooled sensitivity (SEN), specificity (SPE), and area under curve were 0.80 (95% CI 0.72–0.86), 0.78 (95% CI 0.69–0.85), and 0.86 (95% CI 0.83–0.89).
Conclusion: The prognostic value of NLR at admission for severity and mortality in patients with COVID-19 is good. Evaluating the NLR at admission can assist treating clinicians to identify early the cases likely to worsen. This would help to conduct early triage, identify potentially high-risk cases, and start optimal monitoring and management, thus reducing the overall mortality of COVID-19.
Trial registry: This meta-analysis was prospectively registered on PROSPERO database (Registration Number: CRD42021247801).
Keywords: COVID-19 ARDS, COVID-19 mortality, Neutrophil-to-lymphocyte ratio, Prognosis.
Coronavirus disease-2019 (COVID-2019) pandemic continues to affect varied populations creating the greatest crisis faced by healthcare systems worldwide. COVID-19 is caused by a RNA virus, transmitted through respiratory droplets which enters the respiratory system by inhalation.1 The disease is generally mild in 80% of the patients with involvement restricted to upper and conducting airway.2 These patients can generally be managed at home with conservative management. Rest 20% of the patients develop pulmonary infiltrates, and among them, a subset develops severe disease.3 Mortality in the patients with severe COVID pneumonia may be as high as 49% as shown in an epidemiological study by China CDC.4
Early identification of the prognostic factors for severe disease can facilitate rapid access to intensive care units when required.5 Worsening status of a COVID patient might not be detected in time because symptoms and signs, such as fever, tachycardia, tachypnea, and leukocyte count, are nonspecific and may not be always present or appear late.6 Neutrophil-to-lymphocyte ratio (NLR) is an inflammatory biomarker and has prognostic value for severity of disease and mortality. This systematic review and meta-analysis were done to evaluate the prognostic value of NLR at admission for predicting severity and mortality in COVID-19.
MATERIALS AND METHODS
Preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement guidelines have been followed to perform this meta-analysis.7 The study was registered prospectively on PROSPERO database (Registration Number: CRD42021247801).
Selection of Studies
Authors PubMed, Google Scholar, Scirius, MEDLINE, Liliacs, Cochrane, CINAHIL, PLoS, and SIGLE databases through April 30, 2021. Following search terms were used: “coronavirus disease 2019” or “2019 novel coronavirus” or “SARS-CoV-2” or “2019-nCoV” or “COVID-19” and “NLR” or “neutrophil lymphocyte ratio”. No language restrictions were imposed. The reference lists of the included studies were further screened to find additional citations.
All the citations were independently screened by two authors (PGS and PP) to find studies to be included into the final analysis. Any disagreement was resolved through discussion. In case of persistent disagreement, a third reviewer (AK) was consulted for arbitration. Studies were selected if the following criteria were met: (1) The prognostic value of NLR on severity and mortality in patients of COVID-19 was evaluated; (2) sufficient information was available to calculate a 2 × 2 table for true-positive (TP), false-positive (FP), true-negative (TN), and false-negative (FN). Exclusion criteria were (1) inability to extract 2 × 2 table; (2) case reports, reviews, comment, letter, and animal studies.
Data Extraction and Quality Assessment
We prepared standard data extraction forms after discussion in between three reviewers. Pilot data extraction was done by two reviewers, and any shortcomings in the form were rectified by discussion with third reviewer. Final extraction of relevant information was done by two independent authors (PGS and PP). All extracted data were verified by another reviewer (JM). Following data have been extracted from individual studies: area under curve (AUC), cutoff value, TP, TN, FP, FN, sensitivity (SEN), and specificity (SPE). The extracted information was reviewed by a third author (AK). We used the quality in prognosis studies (QUIPS) tool to assess risk of bias (ROB) in six domains: participation, attrition, prognostic factor measurement, outcome measurement, confounding factors, and statistical analysis and reporting.8
We used random-effects model to compute the pooled sensitivity, pooled specificity with 95% CI considering the significant heterogeneity among the studies. Summary area under the curve was computed to determine the discriminating power of NLR for mortality. Diagnostic odds ratio was computed to provide the accuracy of NLR for the predicting mortality. Heterogeneity more than 50% was considered as statistically significant heterogeneity. Meta-regression analysis was done to determine the source of heterogeneity and subgroup effects. All the statistical analyses were completed using software STATA version 13.
Selection and Characteristic of Studies
Study selection process is shown in Flowchart 1. We reviewed PubMed, Google scholar, Scirius, MEDLINE, Liliacs, Cochrane, CINAHIL, PLoS, and SIGLE databases through April 30, 2021, and identified 56 studies. An additional 11 records were identified through other sources. Nine records found in duplicates were removed. The remaining 47 studies were scrutinized by reading the abstract. Three studies were excluded as they did not report prognostic value of NLR in COVID-19 patients. Full-text articles of 44 studies were evaluated. Four studies did not reроrt NLR, eight studies did not рrоvide RОС, and data were not extrасtаble and hence were excluded. Finally, 17 studies reроrting the sensitivity and specificity of NLR recorded at admission to predict development of severe СОVID-19 disease, 8 studies reроrting the prognostic value of NLR recorded at admission on mortality in СОVID-19 раtients, and 7 studies reроrting the prognostic value of NLR recorded at admission on both severity and mortality in СОVID-19 раtients were included in this systematic review and metа-аnаlysis.
The characteristics of each study and the prognostic value of NLR for severity in COVID-19 patients are presented in Table 1. All the studies were retrospective in nature. Out of the 24 studies, 15 were conducted in China and three were conducted in Turkey. Number of patients in the studies varied from 45 to 735. All the studies reported sensitivity, specificity, and AUC, which varied among the studies. Severe disease was defined as раtients with least one of the following features: shortness of breath, respiratory rate (RR) ≥30 times/minute or oxygen saturation (resting state) ≤93%, or РаО2/FiО2 ≤300 mm Hg.
|Study||Year||Country||Language||Patient number||Non severe (inpatient non-ICU)||Severe (ICU)||Male N (%)||T2DM N (%)||HTN N (%)||COPD N (%)||CHF/CAD N (%)||CKD N (%)||Mean age||Cutoff||AUC||TP||FN||TN||FP||SEN %||SPEC %|
|Yang et al.10||2020||CHINA||ENGLISH||93||69||24||60.2||22.5||24.7||NA||13.9||10.7||46.6||3.3||0.84||21||3||43||26||0.88||0.64|
|Wang et al.16||2020||CHINA||ENGLISH||45||35||10||51.1||9||8.9||NA||NA||4.4||39||13.4||0.89||8||2||28||7||0.83||0.82|
|Ok et al.11||2020||TURKEY||ENGLISH||139||85||54||44.4||17.9||23.7||NA||13.6||NA||55.5||3.3||0.87||42||12||60||25||0.79||0.71|
|Liu et al.12||2020||CHINA||ENGLISH||115||78||37||55.6||8.6||21.7||5.2||3.5||NA||NA||3.1||NA||28||9||64||14||0.76||0.83|
|Asghar et al.36||2020||PAKISTAN||ENGLISH||100||67||33||69||41||32||3||13||10||52.6||3.7||0.8||29||4||41||26||0.88||0.62|
|Sun et al.17||2020||CHINA||ENGLISH||116||89||27||51.7||NA||NA||NA||NA||NA||50||4.5||0.89||20||7||80||9||0.74||0.90|
|Shang et al.1||2020||CHINA||ENGLISH||443||139||304||49.66||14.22||29.57||2.7||9.93||NA||56||4.3||0.74||171||133||116||23||0.56||0.84|
|Liu et al.37||2020||CHINA||ENGLISH||84||61||23||55.95||8.3||19||2.4||9.5||6||53||4.9||0.76||12||11||53||8||0.56||0.87|
|Basbus et al.38||2020||SPAIN||ENGLISH||131||110||21||54.1||6.9||30.5||3.8||5.9||NA||52||3||NA||16||5||74||36||0.81||0.67|
|Li et al.39||2020||CHINA||CHINESE||93||50||43||59.13||10.75||12.9||6.45||NA||NA||62.1||11.3||NA||34||9||46||4||0.79||0.92|
|Zha et al.40||2020||CHINA||CHINESE||85||48||37||67.05||NA||NA||NA||NA||NA||54.2||5.6||0.77||25||12||37||11||0.69||0.78|
|Fei et al.41||2020||CHINA||CHINESE||72||52||20||44.44||NA||NA||NA||NA||NA||58||3||0.89||20||0||38||14||1.00||0.73|
|Xia et al.42||2020||CHINA||CHINESE||63||32||31||52.36||19.04||38.09||4.76||3.17||NA||63.4||4.8||0.83||26||5||24||8||0.84||0.75|
|Noor et al.43||2020||PAKISTAN||ENGLISH||735||365||370||88.8||17.4||26||5.7||13.5||3.5||46.3||8.544||0.773||249||121||264||101||0.68||0.73|
|Bastug et al.18||2020||TURKEY||ENGLISH||191||145||46||56||14.1||30.9||NA||10.5||2.6||49||3.2||0.861||32||14||105||40||0.70||0.73|
|Tatum et al.44||2020||AMERICA||ENGLISH||188||139||49||45.21||NA||NA||NA||NA||2.3||58.7||4.94||0.651||26||23||101||38||0.55||0.73|
|Fu et al.13||2020||CHINA||ENGLISH||75||59||16||60||5.3||9.3||5.3||NA||3.1||46.6||6.29||0.88||12||4||48||11||0.75||0.82|
|Seyit et al.45||2020||TURKEY||ENGLISH||110||35||75||56.36||NA||NA||NA||NA||NA||44.16||1.81||0.615||52||23||16||19||0.70||0.46|
|Lin et al.46||2020||CHINA||ENGLISH||68||22||46||58.82||5.9||26.5||1.5||NA||5.9||52.4||3.63||0.948||43||3||15||7||0.94||0.73|
|Mousavi-Nasab et al.47||2020||IRAN||ENGLISH||70||56||14||57.14||NA||NA||NA||NA||NA||42.7||NA||0.87||11||3||45||11||0.80||0.82|
|Zeng et al.14||2021||CHINA||ENGLISH||352||301||51||53.9||NA||NA||NA||NA||NA||NA||5.33||0.801||41||10||207||94||0.82||0.69|
|Ramos-Penafiel et al.15||2020||MEXICO||ENGLISH||125||81||44||64||21.6||19.2||NA||NA||NA||51||NA||NA||26||18||48||33||0.60||0.60|
|Cheng et al.19||2020||CHINA||ENGLISH||456||205||251||46.2||15.3||32.9||3.94||11.4||4.16||55||3.2||0.81||196||55||151||54||0.78||0.74|
|Wang et al.4||2020||CHINA||ENGLISH||131||119||12||42.7||21.4||39.7||NA||NA||NA||64||1.95||0.729||8||4||88||31||0.70||0.74|
The characteristics of each study and the prognostic value of NLR for mortality in COVID-19 patients are presented in Table 2. All the studies were retrospective in nature. Out of the 15 studies, nine were conducted in China. One study each were from America, Mexico, Iran, Turkey, Pakistan, and Spain. Number of patients in the studies varied from 76 to 1,004. All the studies reported sensitivity, specificity, and AUC, which varied among the studies.
|Study||Year||Country||Language||Patient number||Survivor||Nonsurvivor||Male N (%)||T2DM N (%)||HTN N (%)||COPD N (%)||CHF/CAD N (%)||CKD N (%)||Mean age||Cutoff||AUC||TP||FP||FN||TN||SEN %||SPEC %|
|Tatum et al.44||2020||AMERICA||ENGLISH||125||102||23||45.6||NA||NA||NA||NA||NA||58.7||10||0.71||12||11||98||4||0.52||0.97|
|Chen et al.48||2020||CHINA||ENGLISH||681||577||104||53.2||16.7||43||2.2||11.7||4||65||6.7||0.86||87||17||447||130||0.84||0.77|
|Ok et al.11||2020||TURKEY||ENGLISH||139||126||13||44.4||17.9||23.7||NA||13.6||NA||55.5||5.7||0.85||10||3||113||13||0.83||0.90|
|Asghar et al.36||2020||PAKISTAN||ENGLISH||100||78||22||69||41||32||3||13||10||52.6||4.2||0.81||19||3||48||30||0.91||0.63|
|Yan et al.49||2020||CHINA||ENGLISH||1,004||964||40||49.1||10.6||23.4||0.79||7.47||10.15||NA||11.8||0.95||39||1||752||212||0.98||0.78|
|Basbus et al.38||2020||SPAIN||SPANISH||131||112||9||54.1||6.9||30.5||3.8||5.9||NA||52||3||NA||7||2||69||43||0.78||0.62|
|Li et al.39||2020||CHINA||CHINESE||93||62||31||59.13||10.75||12.9||6.45||NA||NA||62.1||11.3||0.92||27||4||52||10||0.90||0.84|
|Song et al.50||2020||CHINA||CHINESE||84||42||42||66.66||NA||NA||NA||NA||NA||66.5||6.1||0.87||32||10||37||5||0.76||0.88|
|Zhang et al.30||2020||CHINA||CHINESE||154||127||27||52.59||13.63||13.63||5.84||10.38||7.79||69.2||9.4||0.86||20||7||116||11||0.76||0.92|
|Ramos-penafiel et al.15||2020||MEXICO||ENGLISH||125||81||44||64||21.6||19.2||NA||NA||NA||51||13||0.72||26||18||48||33||0.60||0.60|
|Xu et al.51||2020||CHINA||ENGLISH||76||44||32||60.53||19.74||35.53||2.63||9.21||6.58||59.1||3.59||0.69||30||2||17||27||0.94||0.39|
|Ye et al.52||2020||CHINA||ENGLISH||349||297||52||49.6||16.3||29.5||12.6||4.6||4||62||7.13||0.86||41||11||243||54||0.80||0.82|
|Wang et al.16||2020||CHINA||ENGLISH||131||119||12||42.7||21.4||39.7||NA||NA||NA||64||13.87||0.963||10||2||107||12||0.90||0.90|
|Zeng et al.53||2021||CHINA||ENGLISH||352||116||15||53.9||NA||NA||NA||NA||NA||NA||7.19||0.828||13||2||74||42||0.93||0.64|
|Eslamijouybari et al.54||2020||IRAN||ENGLISH||527||429||98||44||NA||NA||NA||NA||NA||NA||6.55||0.703||63||35||268||161||0.65||0.63|
Study Quality and Publication Bias
Risk of bias assessment was done by QUIPS tool (Fig. 1). Risk of bias domains evaluated include participation, attrition, prognostic factor measurement, outcome measurement, confounding factors, and statistical analysis and reporting.9 Risk of bias was highest in the domain of confounding factors as none of the studies adequately described other confounding variables. Studies by Yang et al.,10 Ok et al.,11 Liu et al.,12 Fu et al.,13 Zeng et al.,14 and Ramos-Penafiel et al.15 scored “high” in the QUIPS tool risk of bias domain 5, i.e., confounding factors. All the remaining studies had unclear risk in this domain. Overall, study by Fu et al.13 had high risk of bias on evaluation by QUIPS tool. Studies by Yang et al.,10 Wang et al.,16 Liu et al.,12 Sun et al.,17 Bastug et al.,18 and Cheng et al.19 had low risk of bias on evaluation by QUIPS tool. For rest of the studies, risk of bias was unclear. Deek’s funnel plot asymmetry test revealed publication bias to be nonsignificant in both the categories (Fig. 2).
Prognostic Value of NLR for Severe Disease
Twenty-four studies involving 4,080 patients reported the prognostic value of NLR for severity in COVID-19 patients. The pooled sensitivity (SEN) and specificity (SPE) were 0.75 (95% CI 0.69–0.80) and 0.74 (95% CI 0.70–0.78), respectively. The positive likelihood ratio was 2.9 (95% CI 2.5–3.4), and the negative likelihood ratio was 0.34 (95% CI 0.28–0.41). The DOR was 9 (95% CI 6–12). The SROC curve is shown in Figure 3. The AUC of NLR for predicting mortality was 0.81 (95% CI 0.77–0.84). This indicates that NLR has high prognostic value for severity in COVID-19. Fagan normogram shows that if the pretest probability was set to 50%, the posttest probability is more than 90% at NLR cutoff of 5 at admission. On the contrary, when the NLR was below 3, posttest probability was significantly lower.
Prognostic Value of NLR for Mortality
Fifteen studies involving 4,071 patients reported the prognostic value of NLR for mortality in COVID-19 patients. The pooled sensitivity (SEN) and specificity (SPE) were 0.80 (95% CI 0.72–0.86) and 0.78 (95% CI 0.69–0.85), respectively. The positive likelihood ratio was 3.7 (95% CI 2.6–5.3), and the negative likelihood ratio was 0.25 (95% CI 0.18–0.35). The DOR was 15 (95% CI 8–25). The SROC curve is shown in Figure 4. The AUC of NLR for predicting mortality was 0.86 (95% CI 0.83–0.89). This indicates that NLR has high prognostic value for severity in COVID-19.
Fagan normogram shows that if the pretest probability was set to 50%, the posttest probability is more than 90% at NLR cutoff of 6 at admission. On the contrary, when the NLR was below 3, posttest probability was significantly lower.
Goodness of Fit and Outlier Detection
Our goodness of fit analysis showed model calibrated well for both predicting severity and mortality outcomes. This shows that the sample data are representative of data we would expect to find in an actual population. We did not observe significant outlier effects of studies included in the present meta-analysis for mortality outcome; however, for severity analysis, we observe that two studies fell outside the two-standard deviation in the outlier detection analysis.
For the severity prediction (Table 3), our subgroup analysis revealed a consistent finding across studies in which the mean proportion of diabetes was greater than 15% (I2 = 43.8%, for specificity), mean proportion of hypertension more than 25% (I2 = 48.2% for specificity), and the mean proportion of CAD was greater than 10% (I2 = 26.2%, for specificity), and the mean age was less than 50 years. The findings have significant clinical implications and generate research hypotheses suggesting that the NLR test may be a viable predictive marker for the subgroups of hypertensive, diabetic, coronary artery disease (CAD), and younger COVID-19 subjects.
|Prediction of severity|
|Less severity population (≤29%)
N = 12 studies
|0.75 (0.66–0.82)||0.76 (0.71–0.80)||0.81 (0.80–0.84)||9 (6–14)||60.8% (Sen)
|Higher severity population (>29%)
N = 12 studies
|0.75 (0.68–0.81)||0.73 (0.66–0.79)||0.79 (0.75–0.82)||2.8 (2.2–3.6)||83.9% (Sen)
|Proportion of hypertensive <25%
N = 8
|0.73 (0.65–0.80)||0.78 (0.69–0.85)||0.82 (0.78–0.85)||10 (5–18)||51.2% (Sen)
|Proportion of hypertension 25% or more
N = 9
|0.77 (0.68–0.84)||0.71 (0.67–0.75)||0.85 (0.81–0.87)||8 (6–12)||86.1% (Sen)
|Diabetes 15% or less
N = 9
|0.74 (0.64–0.82)||0.79 (0.73–0.84)||0.84 (0.80–0.87)||11 (7–17)||81.7% (Sen)
|Diabetes 15% or more
N = 8
|0.76 (0.69–0.82)||0.70 (0.65–0.73)||0.76 (0.72–0.79)||7 (5–11)||69.1% (Sen)
|CAD 10% or less
N = 5
|0.69 (0.56–0.80)||0.79 (0.71–0.85)||0.81 (0.78–0.85)||8 (5–13)||76.1% (Sen)
|CAD 10% more
N = 6
|0.76 (0.72–0.80)||0.77 (0.70–0.84)||0.70 (0.66–0.74)||8 (6–11)||70.1% (Sen)
|Male 55% or less
N = –11
|0.74 (0.64–0.82)||0.75 (0.67–0.82)||0.75 (0.71–0.79)||9 (6–13)||78.3% (Sen)
|Male 55% or more
N = 13
|0.81 (0.77–0.84)||0.75 (0.67–0.81)||0.74 (0.67–0.80)||8 (5–13)||66.4% (Sen)
|Age less than 50
N = 8
|0.70 (0.65–0.74)||0.75 (0.66–0.82)||0.71 (0.67–0.75)||7 (4–11)||34.4% (Sen)
|Age more than 50
N = 14
|0.76 (0.67–0.83)||0.74 (0.69–0.83)||0.80 (0.77–0.84)||9 (6–14)||84.4% (Sen)
N = 9
|0.70 (0.63–0.76)||0.68 (0.64–0.73)||0.75 (0.71–0.78)||5 (4–7)||51.4% (Sen)
N = 15
|0.77 (0.70–0.83)||0.78 (0.73–0.82)||0.84 (0.81–0.87)||12 (9–17)||83.4% (Sen)
Similarly, for mortality prediction (Table 4), our subgroup analysis indicated that NLR has a consistent and reliable predictive accuracy in terms of sensitivity across studies with a mortality rate of less than or equal to 17% (I2 = 21.2%), among studies with a mean proportion of hypertensive individuals greater than 29% (I2 = 22.1%), and studies with a mean age greater than 50 years (I2 = 20.4%). These findings may have significant clinical implications, implying that NLR may have uniform predictive accuracy for patients in the older age-groups, those who are hypertensive, and less sick patients with probability of lower mortality incidence.
|Prediction of severity|
N = 7 studies
|0.75 (0.64–0.84)||0.77 (0.11–0.89)||0.82 (0.79–0.86)||21 (12–36)||21.2% (SEN)
N = 8 studies
|0.75 (0.68–0.81)||0.73 (0.66–0.79)||0.79 (0.75–0.82)||11 (5–23)||77.9% (SEN)
|Hypertension 29% or less
N = 5
|0.82 (0.66–0.92)||0.82 (0.71–0.90)||0.89 (0.86–0.92)||22 (7–72)||84.2% (SEN)
|Hypertension 29% or more
N = 6
|0.85 (0.78–0.90)||0.71 (0.56–0.83)||0.87 (0.84–0.90)||14 (8–24)||22.1% (SEN)
|Diabetes 16% or less
N = 4
|0.88 (0.75–0.95)||0.81 (0.68–0.89)||0.92 (0.89–0.94)||31 (12–79)||63.7% (SEN)
|Diabetes 16% or more
N = 7
|0.81 (0.71–0.88)||0.75 (0.60–0.85)||0.85 (0.82–0.88)||12 (6–27)||70.1% (SEN)
|Age less than 60
N = 6
|0.76 (0.59–0.87)||0.73 (0.51–0.88)||0.81 (0.78–0.84)||9 (4–20)||77.4% (SEN)
|Age more than 50
N = 6
|0.80 (0.74–0.85)||0.85 (0.80–0.89)||0.87 (0.84–0.90)||23 (15–36)||20.4% (SEN)
N = 9
|0.79 (0.68–0.82)||0.69 (0.64–0.74)||0.79 (0.70–0.88)||6 (2–9)||71.4% (SEN)
N = 6
|0.82 (0.72–0.87)||0.76 (0.71–0.81)||0.80 (0.71–0.83)||10 (6–114)||67.4% (SEN)
Our subgroup analysis observed that a higher cutoff value of NLR (>5 for severity and >6 for mortality) carries similar significance in predicting severity of disease and mortality (Table 5). We did not observe significant influence of mean age, hypertension, diabetes, CAD, heart failure, COPD and sex in the individual studies on the pooled effect size of NLR for predicting severity and mortality in COVID-19 (Fig. 5). We analyzed for differences between the pooled sensitivity and specificity reported by studies conducted in China versus outside China. Fifteen out of 24 studies reporting severity and nine out of six studies reporting mortality have been conducted in China. The pooled sensitivity and specificity of NLR at admission for predicting severity from studies conducted in China were 0.77 (95% CI 0.70–0.83) and 0.78 (95% CI 0.73–0.82), respectively, versus 0.70 (95% CI 0.63–0.76) and 0.68 (95% CI 0.64–0.73), respectively, for studies conducted outside China. The difference in the pooled specificity was found to be statistically significant, with studies from China reporting a higher specificity for NLR at admission to predict severity. The pooled sensitivity and specificity of NLR at admission for predicting mortality from studies conducted in China was 0.85 (95% CI 0.78–0.89) and 0.80 (95% CI 0.70–0.87), respectively, versus 0.65 (95% CI 0.57–0.72) and 0.76 (95% CI 0.58–0.88), respectively, for studies conducted outside China. The difference in the pooled sensitivity was found to be statistically significant for NLR at admission with studies from China reporting a higher sensitivity for NLR at admission to predict mortality.
|Category||No. of studies||Sensitivity (95% CI)||Specificity (95% CI)||PLR (95% CI)||NLR (95% CI)||DOR (95% CI)|
Certainty of Evidence
We have assessed certainty of evidence by the GRADE approach.20 The certainty of the evidence for the overall prognostic value of NLR at admission for severity was moderate (Table 6) due to significant indirectness in the studies reporting surrogate outcomes and significant heterogeneity with I2 >50%. The certainty of the evidence for the overall prognostic value of NLR at admission for mortality was high (Table 5). Studies included in the pooled analysis of NLR for mortality had low risk of bias, low indirectness, low imprecision, and undetected publication bias. However, significant heterogeneity with I2 >50% was reported between these studies.
|Question: Can neutrophil-to-lymphocyte ratio at admission predict severity in COVID-19?|
|Sensitivity||0.75 (95% CI: 0.69–0.80)||Prevalences||20%,||30%,||50%|
|Specificity||0.74 (95% CI: 0.70–0.78)|
|Factors that may decrease certainty of evidence||Effect per 1,000 patients tested|
|Outcome||No. of studies (No. of patients)||Study design||Risk of bias||Indirectness||Inconsistency||Imprecision||Publication bias||pretest probability of 20%||pretest probability of 30%||pretest probability of 50%||Test accuracy CoE|
|True-positives (patients with severity)||24 studies 1,638 patients||Cohort and case-control type studies||Not serious||seriousa||Seriousb||Not serious||All plausible residual confounding would reduce the demonstrated effect||150
|False-negatives (patients incorrectly classified as not having severity)||50 (40–62)||75 (60–93)||125
|True-negatives (patients without severity)||24 studies 2,442 patients||Cohort and case-control type studies||Not serious||seriousa||Seriousb||Not serious||All plausible residual confounding would reduce the demonstrated effect||592
|False-positives (patients incorrectly classified as having severity)||208 (176–240)||182 (154–210)||130 (110–150)|
|True positives (patients with mortality)||15 studies 564 patients||Cohort and case-control type studies||Not serious||Not serious||seriousa||Not serious||All plausible residual confounding would reduce the demonstrated effect||80 (72–85)||160 (144–170)||240
|False negatives (patients incorrectly classified as not having mortality)||20 (15–28)||40 (30–56)||60 (45–84)|
|True negatives (patients without mortality)||15 studies 3276 patients||Cohort and case-control type studies||Not serious||Not serious||seriousa||Not serious||All plausible residual confounding would reduce the demonstrated effect||702 (630–765)||624 (560–680)||546 (490–595)||⨁⨁⨁⨁
|False positives (patients incorrectly classified as having mortality)||198 (135–270)||176 (120–240)||154 (105–210)|
We observed evidence for good performance and discriminatory power of NLR for predicting outcomes in patients with COVID-19.
It has been seen that coronavirus infection causes a physiological stress on the human body which is characterized by elevated levels of cortisol and catecholamines. Increased endogenous cortisol and catecholamines in response to acute physiological stress (<6 hours) are known to cause leukocytosis and lymphopenia.21 Therefore, NLR has potential to identify the individuals at risk for adverse outcomes. NLR has also been used to predict prognosis, severity, and mortality in other inflammatory conditions, such as hepatocellular cancer, breast cancer, neonatal sepsis, and blood stream infections.22–26 NLR is calculated as absolute neutrophil count divided by absolute lymphocyte count.27 In a normal individual, its value is between 1 and 3. A value between 6 and 9 indicated mild stress (e.g., appendicitis). In the presence of sepsis, it is above 9 and may be as high as 100.27
Systemic inflammation triggered by SARS-CoV-2 in cases of severe coronavirus disease or nonsurviving cases causes progressive reductions in lymphocyte count and progressive increase in neutrophil count.28 Neutrophils are triggered by various inflammatory factors like interleukin 6 and interleukin.29 SARS-CoV-2 is known to depress cellular immunity significantly.30 This causes a reduction in CD3 + T cells, CD4 + T cells, and CD8 + T cells due to cytopathic effects.14,16,31 Therefore, NLR may be associated with progression of disease. Since changes in NLR appear before symptomatic worsening,21 it may be used to predict severity and mortality.
Our study indicates that NLR ≥5 at admission for severity and NLR ≥6 at admission for mortality have the optimal prognostic power in COVID-19. Meta-regression analysis revealed clinical factors, such as age, sex, diabetes, hypertension, CAD, heart failure, COPD, and CKD, did not affect the prognostic power of NLR at admission for severity and mortality in coronavirus disease. Further, NLR above 5 and 6 probably has similar prognostic significance for severity and mortality, respectively. Threshold effect of NLR cutoffs on sensitivity and specificity for severity and mortality was 6 and 12%, respectively. This indicates that variable cutoffs of NLR reported by different studies do not introduce significant heterogeneity in the results. However, studies conducted in China had a significantly higher pooled specificity for NLR predicting severity and significantly higher pooled sensitivity for NLR predicting mortality. This may have occurred due to differences in the study population and high number of studies in Chinese population included in analyses. Future studies conducted outside China will be needed to further assess whether our study findings can be generalized to different populations. The goodness-of-fit test appears to indicate that the model was well fit for assessing prognostic performance of NLR for predicting mortality and severity in COVID-19 patients.
To date, five systematic review and meta-analyses have been published to determine correlation of NLR with outcomes in COVID-19 patients.32–36 However, our meta-analysis has improvised upon certain aspects, as compared to the previous ones. We have presented the key differences in Table 7.
|Criteria||Simadibrata DM, 2020||Lagunas-rangel FA, 2020||Ghahramani, 2020||Li X, 2020||Ulloque-badaracco, 2021||Present meta-analysis|
|No. of studies (severity)||38||5||22||13||36||24|
|No. of subjects (severity)||5,699||828||3,396||1,579||8,732||4,080|
|No. of studies (mortality)||38||—||—||—||28||15|
|No. of subjects (mortality)||6,033||—||—||—||6,790||4,071|
|Recommended guidelines for prognostic meta-analysis reporting||Pooled sensitivity||×||×||×||√||×||√|
|Summary area under the curve||×||×||×||√||×||√|
|Diagnostic odds ratio||×||×||×||√||×||√|
|Methodological quality (QUIPS)||×||×||×||×||×||√|
|Analysis used pooled sensitivity, pooled specificity, summary area under the curve, and diagnostic odds ratio||×
Standard mean difference.
Standard mean difference.
Pooled weighted mean difference
Pooled sensitivity, pooled specificity, Summary Area under the curve, Diagnostic odds ratio.
Log odds ratio
Pooled sensitivity, pooled specificity, Summary Area under the curve, Diagnostic odds ratio.
Four meta-analyses have reported only pooled mean, standard deviation, or standard mean difference of NLR in COVID-19.32–34,36 High NLR levels on admission were associated with severe COVID-19 and mortality. However, sensitivity, specificity, AUC, and optimal cutoff of NLR at admission for predicting severity or mortality have not been evaluated in these studies. Authoritative bodies such as the Cochrane collaboration currently recommend the use of the bivariate parameters (sensitivity and specificity) and SROC curves in meta-analysis of diagnostic test accuracy studies.37 Since this is a meta-analysis of prognostic studies, we have reported the SROC curves and derived the sensitivity, specificity of a specific cutoff of NLR at admission for predicting severity and mortality, as supported by authoritative bodies. Similar approach has been used in the meta-analysis by Li et al.35 This meta-analysis reported the sensitivity, specificity, and AUC of NLR at admission for predicting severity or mortality in COVID-19.35 Thirteen studies involving 1,579 patients’ data on severity have been included in this analysis. Authors have used Quality Assessment of Diagnostic Accuracy Studies-2 (QUАDАS-2) limiting the validity of risk of bias assessment in the earlier conducted meta-analysis. Effect of confounding factors, such as age, sex, hypertension, САD, heart failure, СОРD, and СKD, was not evaluated by the authors. Our meta-analysis has included 24 studies involving 4,080 patients reporting the prognostic value of NLR at admission for severe COVID-19 and 15 studies involving 4,071 patients reporting the prognostic value of NLR at admission for mortality in COVID-19. Of these 15 out of 24 studies reporting severity and 9 out of 15 studies reporting mortality have been conducted in China. This highlights that we have included significantly higher number of studies and subjects with higher proportion of the studies from outside China. We have used QUIРS tool for assessment of methodological quality of included studies, which is the preferred tool for bias assessment in prognostic studies. We have evaluated pooled estimates for studies conducted in China and outside China and have documented significant differences. Further studies conducted outside China will be needed to further assess the prognostic accuracy of NLR for outcomes in patients with COVID-19 in other population groups.
This is the first meta-analysis which has evaluated the source of variation on pooled effect size using meta-regression analysis. We have assessed the certainty of the evidence using GRADE criteria for the first time. We have also used a goodness-of-fit model to evaluate the applicability of the results to actual population.
Our meta-analysis conducted following the prognostic studies meta-analysis guidelines to provide the clinically meaningful results.
The major shortcoming of our meta-analysis is the retrospective nature of the data due to which it is prone to various confounding factors. Subgroup analyses did not reveal signify interaction with confounding factors, such as age, sex, hypertension, diabetes, САD, heart failure, СОРD, and СKD. However, possibility of interaction with other confounding factors cannot be ruled out. A high proportion of the studies that have been included in this analysis are from China. This may limit the generalizability of the results and conclusions.
Prognostic value of NLR can be used to identify cases with potential of progression into severe category early. NLR ≥5 identifies a patient subset likely to develop severe COVID-19 with acceptable sensitivity and specificity. NLR ≥6 identifies a patient subset with high risk of mortality with high sensitivity and specificity. Since NLR can be calculated bedside easily, it can serve as a cost-effective method to identify COVID-19 patients at higher risk of severe disease and mortality. Early triage, aggressive monitoring, and management may help to reduce progression in these cases and reduce mortality.
Prattay Guha Sarkar https://orcid.org/0000-0002-9200-5751
Pragya Pant https://orcid.org/0000-0002-2430-1532
Jagmohan Kumar https://orcid.org/0000-0003-3792-922X
Amit Kumar https://orcid.org/0000-0003-0970-5333
2. Wu Z, McGoogan JM. Characteristics of and important lessons from the coronavirus disease 2019 (COVID-19) outbreak in China: summary of a report of 72 314 cases from the Chinese Center for Disease Control and Prevention. Journal of the American Medical Association 2020;323(13):1239–1242. DOI: 10.1001/jama.2020.2648.
4. Wang D, Hu B, Hu C, Zhu F, Liu X, Zhang J, et al. Clinical characteristics of 138 hospitalized patients with 2019 novel coronavirus-infected pneumonia in Wuhan, China. Journal of the American Medical Association 2020;323(11):1061–1069. DOI: 10.1001/jama.2020.1585.
6. Martin JB, Badeaux JE. Interpreting laboratory tests in infection: making sense of biomarkers in sepsis and systemic inflammatory response syndrome for intensive care unit patients. Crit Care Nurs Clin North Am 2017;29(1):119–130. DOI: 10.1016/j.cnc.2016.09.004.
11. Ok F, Erdogan O, Durmus E, Carkci S, Canik A. Predictive values of blood urea nitrogen/creatinine ratio and other routine blood parameters on disease severity and survival of COVID-19 patients. J Med Virol 2021;93(2):786–793. DOI: 10.1002/jmv.26300.
12. Liu J, Liu Y, Xiang P, Pu L, Xiong H, Li C, et al. Neutrophil-to-lymphocyte ratio predicts critical illness patients with 2019 coronavirus disease in the early stage. J Transl Med 2020;18(1):206. DOI: 10.1186/s12967-020-02374-0.
13. Fu J, Kong J, Wang W, Wu M, Yao L, Wang Z, et al. The clinical implication of dynamic neutrophil to lymphocyte ratio and D-dimer in COVID-19: a retrospective study in Suzhou China. Thromb Res 2020;192:3–8. DOI: 10.1016/j.thromres.2020.05.006.
14. Zeng Z-Y, Feng S-D, Chen G-P, Wu J-N. Predictive value of the neutrophil to lymphocyte ratio for disease deterioration and serious adverse outcomes in patients with COVID-19: a prospective cohort study. BMC Infect Dis 2021;21(1):80. DOI: 10.1186/s12879-021-05796-3.
15. Ramos-Peñafiel CO, Santos-González B, Flores-López EN, Galván-Flores F, Hernández-Vázquez L, Santoyo-Sánchez A, et al. Usefulness of the neutrophil-to-lymphocyte, monocyte-to-lymphocyte and lymphocyte-to-platelet ratios for the prognosis of COVID-19-associated complications. Gac Med Mex 2020;156(5):405–411. DOI: 10.24875/GMM.M20000428.
16. Wang X, Li X, Shang Y, Wang J, Zhang X, Su D, et al. Ratios of neutrophil-to-lymphocyte and platelet-to-lymphocyte predict all-cause mortality in inpatients with coronavirus disease 2019 (COVID-19): a retrospective cohort study in a single medical centre. Epidemiol Infect 2020;148:e211. DOI: 10.1017/S0950268820002071.
17. Sun S, Cai X, Wang H, He G, Lin Y, Lu B, et al. Abnormalities of peripheral blood system in patients with COVID-19 in Wenzhou, China. Clin Chim Acta Int J Clin Chem 2020;507:174–180. DOI: 10.1016/j.cca.2020.04.024.
18. Bastug A, Bodur H, Erdogan S, Gokcinar D, Kazancioglu S, Kosovali BD, et al. Clinical and laboratory features of COVID-19: predictors of severe prognosis. Int Immunopharmacol 2020;88:106950. DOI: 10.1016/j.intimp.2020.106950.
19. Cheng B, Hu J, Zuo X, Chen J, Li X, Chen Y, et al. Predictors of progression from moderate to severe coronavirus disease 2019: a retrospective cohort. Clin Microbiol Infect Off Publ Eur Soc Clin Microbiol Infect Dis 2020;26(10):1400–1405. DOI: 10.1016/j.cmi.2020.06.033.
20. Brozek JL, Canelo-Aybar C, Akl EA, Bowen JM, Bucher J, Chiu WA, et al. GRADE Guidelines 30: the GRADE approach to assessing the certainty of modeled evidence—An overview in the context of health decision-making. J Clin Epidemiol 2021;129:138–150. DOI: 10.1016/j.jclinepi.2020.09.018.
21. Benschop RJ, Rodriguez-Feuerhahn M, Schedlowski M. Catecholamine-induced leukocytosis: early observations, current research, and future directions. Brain Behav Immun 1996;10(2):77–91. DOI: 10.1006/brbi.1996.0009.
22. Can E, Hamilcikan Ş, Can C. The value of neutrophil to lymphocyte ratio and platelet to lymphocyte ratio for detecting early-onset neonatal sepsis. J Pediatr Hematol Oncol 2018;40(4):e229–e232. DOI: 10.1097/MPH.0000000000001059.
23. Dirican A, Kucukzeybek BB, Alacacioglu A, Kucukzeybek Y, Erten C, Varol U, et al. Do the derived neutrophil to lymphocyte ratio and the neutrophil to lymphocyte ratio predict prognosis in breast cancer? Int J Clin Oncol 2015;20(1):70–81. DOI: 10.1007/s10147-014-0672-8.
24. Loonen AJM, de Jager CPC, Tosserams J, Kusters R, Hilbink M, Wever PC, et al. Biomarkers and molecular analysis to improve bloodstream infection diagnostics in an emergency care unit. PLoS One 2014;9(1):e87315. DOI: 10.1371/journal.pone.0087315.
25. Yodying H, Matsuda A, Miyashita M, Matsumoto S, Sakurazawa N, Yamada M, et al. Prognostic significance of neutrophil-to-lymphocyte ratio and platelet-to-lymphocyte ratio in oncologic outcomes of esophageal cancer: a systematic review and meta-analysis. Ann Surg Oncol 2016;23(2):646–654. DOI: 10.1245/s10434-015-4869-5.
26. Acet H, Ertaş F, Akıl MA, Özyurtlu F, Polat N, Bilik MZ, et al. Relationship between hematologic indices and global registry of acute coronary events risk score in patients with ST-segment elevation myocardial infarction. Clin Appl Thromb 2016;22(1):60–68. DOI: 10.1177/1076029614533145.
32. Simadibrata DM, Calvin J, Wijaya AD, Ibrahim NAA. Neutrophil-to-lymphocyte ratio on admission to predict the severity and mortality of COVID-19 patients: a meta-analysis. Am J Emerg Med 2021;42:60–69. DOI: 10.1016/j.ajem.2021.01.006.
33. Lagunas‐Rangel FA. Neutrophil‐to‐lymphocyte ratio and lymphocyte‐to‐C‐reactive protein ratio in patients with severe coronavirus disease 2019 (COVID‐19): a meta‐analysis. J Med Virol 2020;92(10):1733–1734. DOI: 10.1002/jmv.25819.
34. Ghahramani S, Tabrizi R, Lankarani KB, Kashani SMA, Rezaei S, Zeidi N, et al. Laboratory features of severe vs non-severe COVID-19 patients in Asian populations: a systematic review and meta-analysis. Eur J Med Res 2020;25(1):30. DOI: 10.1186/s40001-020-00432-3.
35. Li X, Liu C, Mao Z, Xiao M, Wang L, Qi S, et al. Predictive values of neutrophil-to-lymphocyte ratio on disease severity and mortality in COVID-19 patients: a systematic review and meta-analysis. Crit Care Lond Engl 2020;24(1):647. DOI: 10.1186/s13054-020-03374-8.
36. Asghar A, Saqib U, Aijaz S, Bukhari KHS, Hayat A. Utility of neutrophil-to-lymphocyte ratio, platelets-to-lymphocyte ratio and call score for prognosis assessment in covid-19 patients. PAFMJ 2020 9;70(2):S590–S596.
37. Liu YP, Li GM, He J, Liu Y, Li M, Zhang R, et al. Combined use of the neutrophil-to-lymphocyte ratio and CRP to predict 7-day disease severity in 84 hospitalized patients with COVID-19 pneumonia: a retrospective cohort study. Ann Transl Med 2020;8(10):635.
38. Basbus L, Lapidus MI, Martingano I, Puga MC, Pollán J. [Neutrophil to lymphocyte ratio as a prognostic marker in COVID-19]. Medicina (Mex). 2020;80 Suppl 3:31–6.
39. Li H, Zhao M, Xu Y. [Biochemical analysis between common type and critical type of COVID-19 and clinical value of neutrophil/lymphocyte ratio]. Nan Fang Yi Ke Da Xue Xue Bao 2020 30;40(7):965–971.
40. Zha Q. Study on early laboratory warning of severe COVID-19. Lab Med 35(6):557–560.
41. Fei M, Tong F, Tao X, Wang J. [Value of neutrophil-to-lymphocyte ratio in the classification diagnosis of coronavirus disease 2019]. Zhonghua Wei Zhong Bing Ji Jiu Yi Xue. 2020 May;32(5):554–8.
42. Xia X, Wen M, Zhan S, He J, Chen W. [An increased neutrophil/lymphocyte ratio is an early warning signal of severe COVID-19]. Nan Fang Yi Ke Da Xue Xue Bao 2020 30;40(3):333–336.
43. Noor A, Akhtar F, Tashfeen S, Anwar N, Saleem B, Khan SA, et al. Neutrophil-to-Lymphocyte Ratio, derived Neutrophil-to-Lymphocyte Ratio, Platelet-to-Lymphocyte Ratio and Lymphocyte-to-Monocyte Ratio as risk factors in critically ill COVID-19 patients, a single centered study. J Ayub Med Coll Abbottabad 2020 ;32(Suppl 1)(4):S595–S601.
44. Tatum D, Taghavi S, Houghton A, Stover J, Toraih E, Duchesne J. Neutrophil-to-Lymphocyte Ratio and Outcomes in Louisiana COVID-19 Patients. Shock Augusta Ga 2020 ;54(5):652–658.
45. Seyit M, Avci E, Nar R, Senol H, Yilmaz A, Ozen M, et al. Neutrophil to lymphocyte ratio, lymphocyte to monocyte ratio and platelet to lymphocyte ratio to predict the severity of COVID-19. Am J Emerg Med 2021;40:110–114.
46. Lin S, Mao W, Zou Q, Lu S, Zheng S. Associations between hematological parameters and disease severity in patients with SARS-CoV-2 infection. J Clin Lab Anal 2021;35(1):e23604.
47. Mousavi-Nasab SD, Mardani R, Nasr Azadani H, Zali F, Ahmadi Vasmehjani A, Sabeti S, et al. Neutrophil to lymphocyte ratio and C-reactive protein level as prognostic markers in mild versus severe COVID-19 patients. Gastroenterol Hepatol Bed Bench 2020;13(4):361–366.
48. Chen F-F, Zhong M, Liu Y, Zhang Y, Zhang K, Su D-Z, et al. The characteristics and outcomes of 681 severe cases with COVID-19 in China. J Crit Care 2020 ;60:32–37.
49. Yan X, Li F, Wang X, Yan J, Zhu F, Tang S, et al. Neutrophil to lymphocyte ratio as prognostic and prognostic factor in patients with coronavirus disease 2019: A retrospective cross-sectional study. J Med Virol 2020 ;92(11):2573–2581.
50. Song H SH. Prognostic value of multiple inflammatory indexes on the prognosis of patients with corona virus disease 2019. Card Cerebr Pneumal Vasc Dis 2020;28(6):13–16.
51. Xu J-B, Xu C, Zhang R-B, Wu M, Pan C-K, Li X-J, et al. Associations of procalcitonin, C-reaction protein and neutrophil-to-lymphocyte ratio with mortality in hospitalized COVID-19 patients in China. Sci Rep 2020 14;10(1):15058.
52. Ye W, Chen G, Li X, Lan X, Ji C, Hou M, et al. Dynamic changes of D-dimer and neutrophil-lymphocyte count ratio as prognostic biomarkers in COVID-19. Respir Res 2020 3;21(1):169.
53. Zeng F, Li L, Zeng J, Deng Y, Huang H, Chen B, et al. Can we predict the severity of coronavirus disease 2019 with a routine blood test? Pol Arch Intern Med 2020 29;130(5):400–406.
54. Eslamijouybari M, Heydari K, Maleki I, Moosazadeh M, Hedayatizadeh-Omran A, Vahedi L, et al. Neutrophil-to-Lymphocyte and Platelet-to-Lymphocyte Ratios in COVID-19 Patients and Control Group and Relationship with Disease Prognosis. Casp J Intern Med 2020;11(Suppl 1):531–535.
© The Author(s). 2022 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by-nc/4.0/), which permits unrestricted use, distribution, and non-commercial reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.