KEYWORDS
MEWS, sepsis, SIRS, qSOFA
INTRODUCTION
Sepsis is a serious medical condition where infection leads to systemic inflammation and finally organ dysfunction. Based on incidence rates in seven high-income countries, the estimated global incidence of hospital-treated sepsis and severe sepsis is 437 and 270, respectively, per 100,000 inhabitants.1,2 The incidence of sepsis and severe sepsis has increased in the last few decades, probably due to better recognition and increasing age.3 The incidence of sepsis is age-related with an increased incidence in both infants (< 1 year) and the elderly (> 65 years).4,5 Mortality and long-term morbidity, especially among elderly patients with sepsis, is high.6
Since 1991, the Systemic Inflammatory Response Syndrome (SIRS) criteria have been used to classify sepsis (table 1).7 At an International Sepsis Definitions Conference in 2001 it was concluded that the SIRS criteria were too non-specific to diagnose systemic inflammation caused by an infection. However, due to the high sensitivity in predicting systemic inflammation, the SIRS criteria were maintained.8 As a result of growing criticism on the low specificity of the SIRS criteria, an update of the sepsis definition and criteria was needed.9 Early 2016, an international sepsis task force published a new international consensus for the definition of sepsis. They defined sepsis as life-threatening organ dysfunction caused by a dysregulated host response to infection.10 The new sepsis definition uses a set of clinical and biochemical criteria called the Sequential Organ Failure Assessment (SOFA) (table 2). The SOFA score is mainly based on biochemical criteria and therefore the task force developed the more clinical qSOFA screening tool which is based on respiratory rate, systolic blood pressure and an altered mental state (table 1). The new sepsis definition requires a change in the SOFA score of two or more points following an infection. According to the task force, the new sepsis definition is more focussed on the severity of organ dysfunction in patients with an acute infection than the traditional SIRS score. As such, it is supposed to have a higher specificity in identifying patients with more severe and life-threatening infections.
Early recognition and treatment of sepsis is important to reduce mortality, hospital length of stay and morbidity.11,12 While the qSOFA score is not part of the new sepsis definition, it is an important part of the sepsis work-up in the ED.13,14
Although the performance of qSOFA in relation to ICU hospitalisation and mortality has been studied in large prospective datasets, the interrelationship between various scores, i.e. the similarities and differences between the patient groups classified as septic by these scores, is unclear.15
The new definition can enhance research in the treatment of more critically ill sepsis patients. However, until now all research on early antibiotic treatment of sepsis has been based on the SIRS criteria. Thus it is not known what the consequences of the changed sepsis definition are for early antibiotic treatment in patients who are no longer classified as septic according to the new guideline.
This study aims to determine the effect of qSOFA and SOFA compared with the MEWS (table 3) and SIRS criteria on the classification of patients with an infection as having sepsis and its performance compared with the MEWS and SIRS criteria. 
METHODS
Study design and setting
A retrospective single-centre study was performed to evaluate the difference in the number of medical patients in the ED classified as septic by either the SIRS, SOFA criteria or MEWS and their mortality. Medical records of 600 medical patients (18.4% of all medical patients) who visited the ED of the Academic Medical Centre Amsterdam (AMC) between 1 November 2015 and 1 November 2016 were randomly included. The AMC is a 500-bed tertiary care hospital linked to the University of Amsterdam (UVA) in the southeast of Amsterdam, the Netherlands.
This study was approved by the review board for reuse of care data of the AMC. Due to the retrospective observational nature of the study, no formal approval from the Medical Ethics Committee was necessary.
Selection of participants
Patients had to meet the following criteria to be included: (1) patients were ≥ 18 years old; (2) patients visited the ED and were seen by the internal medicine department or its subspecialties, rheumatology, gastroenterology, pulmonary and respiratory diseases, vascular diseases, intensive unit or the department of geriatric medicine. Patients were included only once regardless of the number of consultations at the ED. However, the number of repeat visits was recorded. If patients consulted the ED more than once with a probable infection, the first consultation was selected. Only when a patient was hospitalised at a subsequent consultation at the ED was the consultation that led to hospitalisation selected. After randomisation and screening of the medical records, patients were only excluded from analysis if none of the vital signs (temperature, blood pressure and heart rate) were noted during the ED consultation.
Methods and measurements
All data necessary for the SIRS (temperature, heart rate, blood pressure, respiratory rate, PaCO2 , leukocytes), qSOFA, SOFA (Glasgow coma scale, altered mentation, blood pressure, respiratory rate, PaO2 /FIO2, serum thrombocytes, bilirubin, lactate and creatinine and the use of vasopressor agents) and MEWS, as well as general demographics such as age, gender and comorbidities were retrieved from the medical records. Also, data about laboratory tests, cultures, radiology examination and interventions performed on the ED and within the first 48 hours of admission to the hospital were collected.
Probable infection was based on the final diagnosis in the discharge letter after hospitalisation or the ED visit and confirmed by an independent review of the medical record by the principal investigator, who was not involved in patient care during this period. Where discrepancies seemed to exist between the discharge letter and the review by the principal investigator due to new insights, e.g. cultures turning positive after discharge, the case was subjected to further review by a second senior investigator. Proven infection was defined as a positive blood, urine, sputum, wound or ascites culture that was considered to be clinically relevant by the attending physician from the medical department.
SIRS, qSOFA, SOFA and MEWS scores were calculated from the collected data to classify patients with sepsis and severe sepsis. Sepsis based on the SIRS criteria was defined as a probable infection combined with a SIRS score of ≥ 2 points. Severe sepsis was defined as a probable infection, a SIRS score of ≥ 2 points and organ dysfunction (SOFA score ≥ 2). Since mortality risk increases strongly at MEWS ≥ 5, sepsis according to MEWS was based on a MEWS ≥ 5 + infection.16
For all calculated scores, the first noted laboratory results, temperature, heart rate, respiratory rate, lowest Glasgow Coma Score (GCS) and the lowest noted blood pressure on the ED were used. The GSC was based on the free text of the ED consultation if it was not explicitly noted. Other missing values were considered to be within the normal range.
Vital signs were measured by experienced nurses using automated blood pressure cuffs and a tympanic thermometer. FiO2 was determined based on conversion tables for the amount and route of oxygen administered. PaO2 was based on arterial blood gas when available, but was often missing. If no arterial blood gas was taken, no points on the SOFA score could be awarded for this parameter.
Group characteristics according to various sepsis definitions
The patient groups classified as having sepsis according to the various definitions (see table 4) were analysed for their characteristics, including outcome parameters such as hospitalisation and mortality.
Statistical analysis
Statistical and descriptive analytics were performed using the Statistical Package for Social Sciences (SPSS-PC version 23.0.0.3; IBM corporation). Continuous data are expressed as mean ± standard deviation (SD) and categorical data as number (%). Sensitivity and specificity were calculated to evaluate the accuracy of the clinical sepsis tools (SIRS, qSOFA and MEWS ≥ 5) in in-hospital mortality. Sensitivity and specificity are expressed as percentage with 95% exact Clopper-Pearson confidence intervals.
RESULTS
Characteristics of study subjects
Between 1 November 2015 and 1 November 2016 there were 29,615 patient visits to the ED of the AMC, of which 3254 patients visited the ED for a medical problem (for a total of 4632 ED visits). Seven patients were excluded from this study because they were less than 18 years old, therefore 3247 were included for randomisation. Out of the randomly selected 600 screened medical records, 577 records were included for analysis. In eight of the excluded records there were no vital signs noted for the ED consultation and additionally 15 excluded patients presented to one of the medical specialties but were not actually seen by a doctor of one of the medical departments. The patient characteristics of the 577 medical patients are summarised in table 5. 
Establishing infection and score measurements
Based on the final diagnosis in their discharge letter 198/577 patients (34.3%) had a probable infection. In 175/577 patients (30.3%), antibiotics were started on the ED and a culture was performed within the first 48 hours of admission (table 5).
The attending physician concluded in the discharge letter that there was no probable infection in 18 of these 175 patients. However, an additional, 41/577 patients had an infection even though antibiotics were not started and no culture was performed within the first 48 hours of admission. Of these 41 patients, 14 patients were treated for pneumonia only based on chest X-ray or clinical signs, 5 patients had malaria, 9 patients had a proven viral infection, 4 patients had a urinary tract infection, 7 patients had another kind of bacterial infection and 2 patients with a metastatic malignancy did not agree to receiving antibiotic treatment. So, in total 198 patients had a probable infection.
Eventually, 77 of these 198 patients had a proven infection defined as a positive blood, urine, stool, sputum, wound or ascites culture.
When calculating the scores, blood pressure, temperature and heart rate were available for almost all patients. Likewise, creatinine and leucocyte count were available in over 93% of cases.
On the other extreme, arterial blood gas analysis was frequently missing and in 129 patients with a probable infection no PaO2 /FiO2 was available. In 2 patients with a probable infection no blood pressure was noted and 32 patients had no noted respiratory rate. In all patients with a probable infection a GSC was scored.
The distribution of scores in the population studied is shown in table 5.
Main results
The SIRS criteria classified 141/198 patients as septic (71.2%) of which 55/198 patients (27.8%) also met the criteria for severe sepsis. The combined qSOFA and SOFA score for sepsis classified 17/198 patients (8.6%) as septic and a MEWS ≥ 5 classified 58/198 patients (29.3%) as septic.
Even though the number of patients with sepsis according to MEWS ≥ 5 and according to SIRS was approximately the same (58 vs 55 patients, respectively), the scores classified different patients. Of the patients with a probable infection, 24/198 (12.1%) had both severe sepsis and a MEWS of ≥ 5 points. Almost all patients with sepsis based on the SOFA definition (88%) also had a MEWS of ≥ 5 points and severe sepsis according to the SIRS criteria. The overlap between the various definitions is shown in figure 1.
Patients with sepsis classified by the different clinical tools had a mean age varying from 56.4 years ± 20.2 to 63.2 ± 15.7 (table 4). The respiratory tract was the most common site of infection (range 50% to 64.7%) followed by the urinary tract (range 5.9% to 15.5%). All patients with sepsis classified by SOFA criteria were hospitalised compared with 94.5% of the patients with severe sepsis, 89.7% of the patients with a MEWS of ≥ 5 and 78.7% of the patients with sepsis classified by the SIRS criteria.
Twelve patients with a probable infection (6.1%) died during hospitalisation. Five out of 17 patients (29.4%) with sepsis according to the qSOFA score died during hospitalisation. The in-hospital mortality in patients with sepsis (SIRS criteria), severe sepsis and MEWS ≥ 5 was 6.4%, 12.7% and 8.6%. The overlap in in-hospital mortality between the sepsis scores is shown in figure 1. 
Sensitivity and specificity analysis
The specificity of the different sepsis tools for predicting in-hospital mortality was 56.9% (95% CI: 52.7-61.1) for SIRS ≥ 2, 96.4% (94.5;97.8) for qSOFA ≥ 2, 87.0% (95% CI: 83.9-89.7) for MEWS ≥ 5 and 87.4% (84.3-90.0) for SIRS ≥ 2 and SOFA ≥ 2 (table 6). Sensitivity was poor for qSOFA [33.3% (14.6-57.0)] compared with SIRS [61.9% (38.4-81.9)]
DISCUSSION
Because of their high prevalence and mortality, infectious diseases and sepsis need to be quickly identified in the ED so that early interventions may improve outcome. As shown in table 5, one in three of our medical patients presented with a probable infection at the ED based on the final diagnosis in the discharge letter.
The new sepsis work-up based on the qSOFA and SOFA score classified only a small group of these patients as septic compared with the traditional SIRS criteria. In fact, 88% of the patients previously labelled as septic would no longer be considered septic due to the change in sepsis definition. However, as designed, the combination of qSOFA + SOFA ≥ 2 did succeed in classifying a small group of patients with life-threatening organ dysfunction and increased mortality (table 6). Patients with sepsis classified by these criteria had an in-hospital mortality of 29.4%.
Although the qSOFA score is not part of the sepsis-3 definition, it plays an important role in the sepsis work-up in the ED. Effectively, the qSOFA is meant to replace the SIRS score as a bedside tool to recognise sepsis in the ED. Only 15.7% of those with a probable infection scored ≥ 2 points on the qSOFA score, while 35.9% patients with a probable infection scored ≥ 2 points on the SOFA score (table 5). Thus, the qSOFA score failed to classify 40/71 patients (56.3%) as septic who would have been considered septic if only the SOFA score were used. This is in line with the study by Freund et al. who reported that 25% of ED patients with a probable infection have a qSOFA score of ≥ 2 while 34% of the patients have a SOFA score of more than two points.
The strict selection by the qSOFA score is mainly accounted for by the small number of patients with a probable infection who had an altered mental state (20 patients (10.1%)). Of all patients with a probable infection, 52 patients (26.3%) had a systolic blood pressure of ≤ 100 mmHg and 66 patients (33.3%) had a respiratory rate of ≥ 22/min. Recent studies by Churpek et al., Freund et al. and Williams et al. similarly found a positive qSOFA score (≥ 2) in only 9%, 25% and 10.2% of their patients at the time of suspicion of infection.15,17,18
MEWS is normally used for early recognition of clinical deterioration of hospitalised patients and was not originally designed for recognition of sepsis patients. However, MEWS ≥ 5 selected approximately the same number of patients as severe sepsis when compared with the SIRS criteria. Although these scores select different patients within the sepsis spectrum, there were no major differences in ICU admissions or site of infection (table 4). The overlap in patients classified by the various scores is shown in figure 1.
Due to the high prevalence of positive SIRS criteria in patients in the ICU, the SIRS score is not very useful as a clinical tool in the recognition of sepsis in ICU patients. Therefore, a score with a higher specificity in predicting infection and mortality like the qSOFA score in the ICU is needed. Despite this, recent studies report that the predictive value of the qSOFA score for in-hospital mortality is better outside the ICU compared with patients admitted to the ICU.18-20 In the ED on the other hand, a clinical tool with a high sensitivity in predicting organ dysfunction and mortality is more important, to avoid undertreatment of sepsis. In our data, neither MEWS nor qSOFA seems to meet this criterion.
qSOFA seems to meet this criterion. Finally, current guidelines for sepsis are not only aimed at early recognition but also at early treatment of sepsis. The Surviving Sepsis Campaign guidelines (2014) recommend to administer effective intravenous antibiotics within the first hour of recognition of severe sepsis and septic shock based on the SIRS criteria.21 One of the main questions is whether the new sepsis definition is only meant for the recognition of patients with a higher risk of mortality or also as guideline for early antibiotic treatment. One year after the introduction of the new sepsis definition, many hospitals still use the SIRS criteria for the recognition of sepsis and severe sepsis on the ED. This is due to concerns of physicians that the strict selection of the qSOFA score may lead to undertreatment of sepsis and thus possibly increase its mortality.22 Although no formal study has compared treatment regimens based on qSOFA to SIRS and MEWS, we share these concerns when looking at the drastic decrease in the number of patients who would be classified as septic in our ED.
Our study has a few limitations. First, a single-centre study was performed with a relative small number of patients compared with other recent studies. The number of randomised patients in our study was too low to detect small differences between the different sepsis scores. Particularly the low number of deaths in our cohort (only 12 patients with a probable infection) makes our sensitivity/ specificity analysis less precise.
However, the small number of patients made detailed clinical follow-up possible of patients throughout their hospitalisation to confirm the presence of infection.
Another limitation is that only medical patients at the ED were included in this study. It is unknown whether the classification of sepsis by the different sepsis tools differs between medical patients and patients who suffer from other types of infections such as wound infections after surgery.
In summary, there are no major differences in gender, age, comorbidity and site of infection between patients with sepsis and severe sepsis classified by the SIRS, qSOFA criteria or a MEWS of ≥ 5 with a probable infection in the ED. However, the new qSOFA work-up for sepsis classifies only a very small group of patients with a higher mortality compared with the SIRS criteria. Due to this strict selection, the qSOFA score seems unsuitable as bedside tool in the sepsis work-up at the ED. Until the therapeutic implications of the qSOFA and the downstream effects on mortality have been formally evaluated in randomised clinical trials, the SIRS criteria upon which current knowledge about early treatment, particularly antibiotic treatment, of sepsis was founded should still be leading in the emergency department.
DISCLOSURES
All authors declare no conflict of interest according to the ICJME guidelines. The study was supported by the University of Amsterdam, no grants or financial support were received.
REFERENCES