Acute Physiologic and Chronic Health Examination II and Sequential Organ Failure Assessment Scores for Predicting Outcomes of Out-of-Hospital Cardiac Arrest Patients Treated with Therapeutic Hypothermia
Article information
Abstract
Background
The aim of this study was to assess the relationship between acute physiologic and chronic health examination (APACHE) II and sequential organ failure assessment (SOFA) scores and outcomes of post-cardiac arrest patients treated with therapeutic hypothermia (TH).
Methods
Out-of-hospital cardiac arrest (OHCA) survivors treated with TH between January 2010 and December 2012 were retrospectively evaluated. We captured all components of the APACHE II and SOFA scores over the first 48 hours after intensive care unit (ICU) admission (0 h). The primary outcome measure was in-hospital mortality and the secondary outcome measure was neurologic outcomes at the time of hospital discharge. Receiver-operating characteristic and logistic regression analysis were used to determine the predictability of outcomes with serial APACHE II and SOFA scores.
Results
A total of 138 patients were enrolled in this study. The area under the curve (AUC) for APACHE II scores at 0 h for predicting in-hospital mortality and poor neurologic outcomes (cerebral performance category: 3–5) was more than 0.7, and for SOFA scores from 0 h to 48 h the AUC was less than 0.7. Odds ratios used to determine associations between APACHE II scores from 0 h to 48 h and in-hospital mortality were 1.12 (95% confidence interval [CI], 1.03–1.23), 1.13 (95% CI, 1.04–1.23), and 1.18 (95% CI, 1.07–1.30).
Conclusions
APACHE II, but not SOFA score, at the time of ICU admission is a modest predictor of in-hospital mortality and poor neurologic outcomes at the time of hospital discharge for patients who have undergone TH after return of spontaneous circulation following OHCA.
Introduction
The number of out-of-hospital cardiac arrests (OHCA) in Korea was estimated to be 97,291 between 2006 and 2010. The age-standardized incidence of OHCA increased from 37.5 cases to 46.8 cases per 100,000 persons in 2010, and survival to hospital discharge was 3.0% in 2010.[1] Still, survivors of cardiac arrest with return of spontaneous circulation (ROSC) after cardiopulmonary resuscitation (CPR) often die due to refractory shock or recurrent cardiac arrest (CA), or experience devastating neurological impairment as a result of ischemia/reperfusion-induced cerebral injury.[2,3] Therapeutic hypothermia (TH) has been shown to alleviate anoxic brain injury and improve neurologic outcomes in patients after ROSC.[4,5]
The prediction of neurological outcomes is a crucial factor in determining treatment strategies for CA patients with ROSC. Severity assessments help to reduce ineffective life-sustaining treatment and promote efficient distribution of health care resources.[6,7] When the risk of death is accurately predicted, patients and their families can prepare for an expected death and physicians can more effectively stratify patients based on adjusted predictive risks for their quality evaluation or research.[8] In previous studies, the survival of CA patients varied according to the location at which CA occurred, the presence or absence of a bystander who could perform CPR, access to a professional rescuer who could perform CPR, initial cardiac rhythm, and no-flow and low-flow times.[9,10] However, prediction tools such as the acute physiologic and chronic health examination (APACHE),[11] Simplified acute physiologic score (SAPS),[12] and sequential organ failure assessment (SOFA)[13] do not incorporate certain variables that may affect outcomes of CA patients because these scoring systems were developed for application to patients with critical illness. It is therefore necessary to verify the utility of these scoring systems for quantifying the severity of illness in CA patients.
This study aimed to assess the utility of APACHE II and SOFA scores in predicting outcomes of CA patients who underwent TH after ROSC.
Materials and Methods
1) Subjects
This study was conducted at a 1,300-bed university hospital with 100 intensive care unit (ICU) beds and nearly 90,000 emergency-room visits each year. Among adult patients who presented to the emergency room due to OHCA over the course of 36 months from January 1, 2010 to December 31, 2012, those who underwent TH after CPR and ROSC were screened, and their medical records were retrospectively analyzed. Patients with traumatic CA were excluded along with those who died before ICU admission and those with insufficient patient information.
2) Therapeutic hypothermia
An identical protocol was followed for the treatment of OHCA patients with ROSC during the study period. The target temperature of 33°C was achieved by either surface or endovascular cooling in addition to a palliative method with 4°C normal saline infusion, ice packs, and bladder irrigation. Surface cooling was performed using either a water blanket (Blanketrol® II Cincinnati Sub-Zero Products, OH, USA) or a pad (ArcticGelTM Pads & Arctic Sun® 2000, Medivance, CO, USA). Endovascular cooling was performed using a catheter (Cool Line® Catheter & CoolGard 3000®, ZOLL, MA, USA). Patient temperature was monitored with a rectal thermometer to ensure that the target temperature was achieved and maintained for 24 hours. After the initiation of cooling, rewarming began at a rate of 0.2–0.3°C every hour in order to increase body temperature to 36.5°C, and increased temperature was maintained for 72 hours. All patients required endotracheal intubation and mechanical ventilation before TH, and they were given sedatives, analgesics and muscle relaxants during TH.
3) Methods
All patients who presented with OHCA and underwent TH were enrolled in the cardiac arrest registry. From the registry, patients’ demographic variables including age, gender, past medical history, in-hospital mortality, neurologic outcomes at the time of discharge, and other variables associated with APACHE II and SOFA scores were analyzed. The APACHE II and SOFA scores were calculated at the time of admission (0 hour), at 24 hours, and at 48 hours based on abnormal vital signs and lab results. The Glasgow coma scale, which is included in the APACHE II and SOFA scores, represents the last value measured before sedation.[14] There were patients who received post cardiac arrest care (PACC) in the emergency room because there were no beds available in the ICU. In those cases, the time of initiation of PACC was considered to be the time of admission. Data related to CA were retrospectively collected according to the Utstein style: the time of CA occurrence, location of CA, witness status, bystander CPR, initial cardiac rhythm, medication used during CA, CPR, and time of ROSC.[15,16] The primary endpoint was in-hospital mortality and the secondary endpoint was neurologic outcomes at the time of hospital discharge. Neurologic status at the time of discharge was measured using a 5-point cerebral performance category (CPC): a CPC of 1 indicates a good cerebral performance with the ability to lead a normal life and/or mild neurologic symptoms; a CPC of 2 indicates sufficient neurologic function for independent activities of daily living; a CPC of 3 indicates consciousness, moderate cerebral disability, and inability to perform independent activities of daily living; a CPC of 4 indicates comatose or persistent vegetative state; and a CPC of 5 indicates death.[17] A CPC of 1–2 is considered a good neurologic outcome, and a CPC of 3–5 is considered a poor neurologic outcome.[4,5] The time from CA to ROSC is defined as the downtime, which is further divided into no-flow time (the time from CA to initiation of CPR) and low-flow time (the time from CPR to ROSC).[8]
4) Statistical analysis
Statistical analysis was performed using SPSS, version 18 (IBM, NY, USA) and MedCalc, version 12.7.7.0 (MedCalc Software, Ostend, Belgium). A p value less than 0.05 was considered to be statistically significant. Continuous variables were presented as mean values and standard deviations (SD) or median values and interquartile ranges (IQR) depending on distribution, and categorical variables were presented as relative frequencies (%). Either independent t-tests or Mann-Whitney U tests were used to compare differences between groups when the dependent variable was continuous. A chi-square test was used to examine the association between categorical variables. A logistic regression analysis was used in multivariate analysis of variables associated with in-hospital mortality and neurologic outcomes at the time of discharge by building a model with variables that had a p value of 0.2 or less on univariate analysis and those reported as significant in previous studies through backward stepwise elimination. To assess the discrimination power of each CPC score, the area under the receiver operating characteristic curve (ROC) was calculated (AUC) using in-hospital mortality and neurologic outcomes as independent variables. An AUC value > 0.9 indicates high discrimination power, an AUC value between 0.7 and 0.9 indicates moderate discrimination power, and an AUC value < 0.7 indicates low discrimination power.[18] To verify the utility of APACHE II and SOFA scores as predictors of in-hospital mortality and poor neurologic outcomes at the time of discharge, cutoff values were calculated using ROC curves, AUC values, and Youden’s index of the scores at each time point. The corresponding sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were also calculated with 95% CIs. To reduce false positive results, the cut-off value that allows 100% specificity was determined at each time point. Model calibration was assessed by the Hosmer-Lemeshow goodness of fit test. p values > 0.05 were considered to be calibrated.
Results
1) Subject characteristics and cardiac arrest data
A total of 138 CA patients participated in this study. Subject characteristics and cardiac arrest data are summarized in Table 1. There were 95 men (68.6%), and the mean age of all subjects was 52.9 years (SD ± 15.4). In-hospital mortality occurred in 54 patients (41.3%), and good neurologic outcomes were observed in 47 patients (34.1%) (42 with CPC 1, 5 with CPC 2, 5 with CPC 3, 29 patients with CPC 4 and 57 patients with CPC 5). Among patients who died in the hospital, 9 (15.7%) died within 48 hours. The median downtime was 30 min (IQR 23–42), and the mean lactate level was 8.4 mmol/L (SD: ± 3.4). The median time from ROSC to ICU admission was 145 min (IQR 88–227).
Baseline characteristics of the study population (N = 138)
2) Variables associated with in-hospital mortality and neurologic outcomes
In-hospital mortality was affected by initial cardiac rhythm, cardiogenic CA, vasopressor administration within 48 hours, and baseline lactate level (Table 2). Neurologic outcomes at the time of discharge were associated with gender, history of coronary artery disease, witnessed CA, initial cardiac rhythm, cardiogenic CA, duration of CA, no-flow time, and baseline lactate level (Table 3). APACHE II and SOFA scores at all time points were related to mortality whereas APACHE II scores at 0 hour and 48 hours were related to neurologic outcomes at the time of discharge (Table 4).
Comparison of variables between patients that died versus those that were alive at the time of hospital discharge (N = 138)
Comparisons of variables in patients with good versus poor neurologic outcomes at the time of hospital discharge (N = 138)
Serial APACHE II and SOFA scores according to mortality and neurologic outcomes (N = 138)
3) Receiver-operating characteristic curve analysis for APACHE II and SOFA scores
Time-dependent AUC values of APACHE II and SOFA scores in predicting in-hospital mortality and poor neurologic outcomes are presented in Table 5. The predictability of APACHE II scores was assessed at each time point using ROC curves as shown in Fig. 1. and 2. The AUC for APACHE II scores at 0 hour were 0.7 and 0.73, respectively, showing a moderate discriminatory power, whereas AUC values decreased to less than 0.7 at 24 and 48 hours. The AUC for SOFA scores was also low at less than 0.7 at all time points, showing a low discriminatory power. Hosmer-Lemeshow statistics were adequate for the scoring system since most p-values exceeded 0.05, although APACHE II scores associated with in-hospital mortality showed p values of 0.045 and 0.042 at 24 and 48 hours, respectively. The odds ratios (OR) with 95% CI for APACHE II scores in predicting in-hospital mortality at 0 hour, 24 hours, and 48 hours were 1.12 (1.03–1.23), 1.13 (1.04–1.23) and 1.18 (1.07–1.30), respectively. The OR of APACHE II scores for predicting neurologic outcomes at the time of discharge at the same time points were 1.15 (1.05–1.25), 1.14 (1.05–1.24) and 1.18 (1.08–1.29), respectively. The OR for SOFA scores in predicting in-hospital mortality at 48 hours was 1.33 (1.12–1.58), and the OR for SOFA scores in predicting neurologic outcomes at 0 hour, 24 hours, and 48 hours were 1.15 (1.01–1.32), 1.17 (1.01–1.35), and 1.33 (1.12–1.58), respectively. The cutoff value for baseline APACHE II scores in predicting in-hospital mortality was 21, with a sensitivity of 80.7%, specificity of 53.1%, PPV of 54.8% and NPV of 79.6%. The cutoff value for baseline APACHE II scores in predicting poor neurologic outcomes was 20, with a sensitivity of 82.4%, specificity of 59.6%, PPV of 79.8% and NPV of 63.6%. The cut-off values that corresponded to 100% specificity for baseline APACHE II scores in predicting in-hospital mortality and neurologic outcomes were 32 and 29, respectively.
Performance of APACHE II and SOFA scores in predicting mortality and neurologic outcomes
Comparison of receiver operating characteristic curves for serial APACHE II scores to predict in-hospital mortality using the area under the curve. The white dots on each curved line indicate the cutoff values at serial time points. APACHE: acute physiologic and chronic health examination; AUC: area under the curve; CV: cutoff values.
Discussion
The purpose of this study was to assess the time-dependent predictive performance of APACHE II and SOFA scores in predicting in-hospital mortality and neurologic outcomes at the time of discharge in OHCA patients who underwent TH after CPR with ROSC. As a result, APACHE II scores were moderate predictors of the two outcomes with AUC values of 0.7 and 0.73 respectively, indicating moderate discriminatory power. The overall model calibration was also supported by Hosmer-Lemeshow statistics (p value > 0.05)
Adrie et al.[9] identified initial cardiac rhythm, no-flow and low-flow times, serum lactate level, and creatinine level as being independently associated with poor neurologic outcomes at the time of discharge among OHCA patients. In this study, the same variables with the exception of creatinine played a similar role. According to a meta-analysis by Sasson et al.[10] rates of survival to hospital discharge were high among CA patients with witnessed CA, presence of emergency medical system (EMS), bystander CPR, CA caused by ventricular fibrillation or ventricular tachycardia, and ROSC at the site of initial collapse. In this study, initial cardiac rhythm was also identified as a variable associated with in-hospital mortality, though this was not statistically significant, likely due to low rates of witnessed CA, performance of CPR by bystanders,[1] and disparities in EMS presence.
The APACHE II scoring system was developed by Knaus et al.[11] in 1985 to classify the severity of illness. This scoring system was used for 5,815 patients in ICUs at 13 hospitals, and was found to correlate with in-hospital mortality. APACHE II scores have been effectively used to predict outcomes in patients with critical illness including sepsis,[19] pesticide poisoning,[20] and trauma,[21] and also in surgical patients.[22] In 2012, Donnino et al.[8] used APACHE II scores as a prognostic predictor in CA patients. In their study, 80 patients (35%) underwent TH among 228 patients who had CA outside or inside of a hospital. Also, the ORs of APACHE II scores for predicting in-hospital mortality at 0 hour, 24 hours, 48 hours, and 72 hours were 1.09 (1.05–1.13), 1.10 (1.06–1.15), 1.20 (1.12–1.28) and 1.22 (1.13–1.31), respectively. The ORs for APACHE II scores for predicting neurologic outcomes at the time of discharge at the same time points were 1.08 (1.03–1.12), 1.10 (1.06–1.15), 1.18 (1.11–1.25), and 1.21 (1.23–1.29), respectively. These findings were similar to those of our study, in that APACHE II scores had moderate discriminatory power in predicting in-hospital mortality and poor neurologic outcomes at 24 hours, 48 hours, and 72 hours. As previously mentioned, this study demonstrated that APACHE II scores at 0 hour had a better discriminatory power for outcome prediction than those at other time points. Thus, the results of these studies are inconsistent. These differences may be due to the fact that this study included only OHCA patients and scores were measured at the time of ICU admission instead of after ROSC. Also, TH can be accompanied by hypotension, bradycardia, increases in oxygen consumption and carbon dioxide production, hypothermia, electrolyte imbalance, and decreases in platelet and white blood cell counts.[23] The 24 hours following ICU admission is referred to as the maintenance phase in this study, meaning that APACHE II scores may elevate during this period, leading to low discriminatory power.
The SOFA score was proposed by Vincent et al.[13] in 1993 as a scoring system to assess the extent of organ dysfunction. It is an excellent tool for assessing the incidence of organ dysfunction/failure and predicting morbidity. In a study by Firreira et al.[24] conducted in ICU patients at a Belgian hospital in 2001, the mean and maximum SOFA scores which were measured over 96 hours were valid outcome predictors. SOFA scores have been reported to be reliable predictors of mortality and morbidity in several studies. Roberts et al.[25] used SOFA scores to determine the extent of multiple organ dysfunction in 208 CA patients after ROSC. Of those, 170 (84%) were in hospital cardiac arrest (IHCA) patients, and 77 of them (38%) underwent TH. To measure the extent of organ dysfunction/failure, serial SOFA scores were obtained at 24 hours intervals over 72 hours, and multivariate logistic regression was used to determine the association between SOFA scores and in-hospital mortality, with adjustments for cerebral dysfunction. According to the results of their study, organ dysfunction was the most common post-cardiac arrest sequelae, followed by cerebral injury, and respiratory or cardiovascular dysfunction. Their multivariate analysis also showed that the maximum baseline extracerebral SOFA score was associated with in-hospital mortality (OR 2.37 [95% CI, 1.28–4.38]). Also, the total maximum baseline extracerebral SOFA score was independently related to in-hospital mortality over 72 hours (OR 1.95 [95% CI, 1.15–3.29]). In particular, the total maximum SOFA scores for respiratory or cardiovascular disorders were independently associated with in-hospital mortality, reflecting hemodynamic instability and oxygenation failure. This study however demonstrated that SOFA scores had an OR of 1.33 (95% CI, 1.12–1.58) at 48 hours as a predictive value for in-hospital mortality. These results differed from those of Roberts et al.[25] which may be because our study calculated baseline SOFA scores at the time of ICU admission, and their association with in-hospital mortality and neurologic outcomes was assessed using the ROC curve and AUC values.
In addition to APACHE II and SOFA scores, the SAPS score is sometimes used to assess severity of illness. SAPS II, which was introduced by Le Gall et al.[12] is the most common version used in the clinical setting. Salciccioli et al.[26] compared the SAPS II and SAPS III in predicting outcomes in 274 OHCA/IHCA patients. Of those, 103 (38%) underwent TH. In their study, SAPS II was calculated based on the worst score for each variable during the first 24 hours after ROSC, whereas SAPS III was calculated based on variables at the time of ICU admission. AUC values for SAPS II in predicting in-hospital mortality and poor neurologic outcomes at the time of discharge were 0.7 and 0.71, respectively, showing moderate discrimination, while AUC values for SAPS III were 0.66 and 0.65. A recent study by Bisbal et al.[27] also found a higher performance of SAPS II in predicting mortality in CA patients when comparing the two scoring systems.
In 2006, Adrie et al.[9] proposed use of the OHCA score to assess severity of illness in OHCA patients using initial cardiac rhythm, no-flow and low-flow times, serum lactate level, and creatinine level. They reported an AUC value of 0.82 for the OHCA score, which was calculated for predicting poor neurologic outcomes. Hunziker et al.[28] also supported the utility of OHCA scores with an AUC value of 0.85 in their study conducted to verify the utility of OCHA scores. The OHCA score is easy to calculate and utilize because it uses variables predicted at the time of ICU admission, with no observation period necessary. However, it is difficult to estimate the no-flow and low-flow times accurately, making scoring calculations more complicated.
This study has several limitations. First, it was conducted at a single hospital, and a sample size calculation was not performed. In addition, scoring was performed within the first 48 hours after ICU admission according to our TH protocol, since the effects of hypothermia including rewarming would not resolve and medications such as sedatives and muscle relaxants would not metabolize with 48 hours. Given the trends in AUC values, higher discrimination could be achieved if APACHE and SOFA score calculation extended to 72 hours after ICU admission.
In conclusion, APACHE II scores at the time of ICU admission showed moderate discriminatory power in predicting in-hospital mortality and poor neurologic outcomes at the time of discharge in OHCA patients who underwent TH after CPR with ROSC. However, SOFA scores showed a low discriminatory power. Additionally, patients with APACHE II scores of 32 or higher had a high risk of in-hospital death. Therefore, this score may be useful in predicting in-hospital mortality in OHCA patients receiving TH. Further large scale studies are needed to explore the utility of various scoring systems as predictors of mortality and neurologic outcomes in OHCA patients receiving TH.
Comparison of receiver operating characteristic curves for serial APACHE II scores to predict neurologic outcomes at the time of hospital discharge using the area under the curve. The white dots on each curved line indicates the cutoff values at serial time points. CPC: cerebral performance category; APACHE: acute physiologic and chronic health examination; AUC: area under the curve; CV: cutoff values.
Notes
No potential conflict of interest relevant to this article was reported.