Skip Navigation
Skip to contents

ACC : Acute and Critical Care

OPEN ACCESS
SEARCH
Search
Advanced Search
mobile menu button

Articles

Page Path
HOME > Acute Crit Care > Forthcoming articles > Article
Original Article
Pulmonary Using machine learning techniques for early prediction of tracheal intubation in patients with septic shock: a multi-center study in Korea
Ji Han Heo1,*orcid, Taegyun Kim1,2orcid, Tae Gun Shin3orcid, Gil Joon Suh1,2orcid, Woon Yong Kwon1,2orcid, Hayoung Kim1,4orcid, Heesu Park1orcid, Heejun Kim1orcid, Sol Han4orcid, on behalf of the Korean Shock Society

DOI: https://doi.org/10.4266/acc.004776
Published online: April 30, 2025

1Department of Emergency Medicine, Seoul National University Hospital, Seoul, Korea

2Department of Emergency Medicine, Seoul National University College of Medicine, Seoul, Korea

3Department of Emergency Medicine, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea

4Department of Emergency Medicine, Incheon Red Cross Hospital, Incheon, Korea

Corresponding author: Taegyun Kim Department of Emergency Medicine, Seoul National University College of Medicine, 103 Daehak-ro, Jongno-gu, Seoul 03080, Seoul, Korea Tel: +82-2-2072-3257 Fax: +82-2-741-7855 E-mail: kimtaegyun@snuh.org
*Current affiliation: Interdisciplinary Program in Bioengineering, Graduate School, Seoul National University, Korea
• Received: December 23, 2024   • Revised: February 25, 2025   • Accepted: February 25, 2025

© 2025 The Korean Society of Critical Care Medicine

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

  • 574 Views
  • 31 Download
Full Article

Download PDF Download PDF

  • Background
    Patients with septic shock frequently require tracheal intubation in the emergency department (ED). However, the criteria for tracheal intubation are subjective, based on physician experience, or require serial evaluations over relatively long intervals to make accurate predictions, which might not be feasible in the ED. We used supervised learning approaches and features routinely available during the initial stages of evaluation and resuscitation to stratify the risks of tracheal intubation within a 24-hour time window.
  • Methods
    We retrospectively analyzed the data of patients diagnosed with septic shock based on the SEPSIS-3 criteria across 21 university hospital EDs in the Republic of Korea. A principal component analysis revealed a complex, non-linear decision boundary with respect to the application of tracheal intubation within a 24-hour time window. Stratified five-fold cross validation and a grid search were used with extreme gradient boost. Shapley values were calculated to explain feature importance and preferences.
  • Results
    In total, data for 4,762 patients were analyzed; within that population, 1,486 (31%) were intubated within a 24-hour window, and 3,276 (69%) were not. The area under the receiver operating characteristic curve and F1 scores for intubation within a 24-hour window were 0.829 (95% CI, 0.801–0.878) and 0.654 (95% CI, 0.627–0.681), respectively. The Shapley values identified lactate level after initial fluids, suspected lung infection, initial pH, Sequential Organ Failure Assessment score at enrollment, and respiratory rate at enrollment as important features for prediction.
  • Conclusions
    An extreme gradient boosting machine can moderately discriminate whether intubation is warranted within 24 hours of the recognition of septic shock in the ED.
  • Key Words: intubation; machine learning; septic shock
Tracheal intubation is frequently performed in emergency departments (EDs) as a life-saving intervention for patients deteriorating with septic shock. The pathophysiology underlying the need for the procedure includes failure in gas exchange due to consolidations and pulmonary edema and respiratory fatigue in response to compensatory tachypnea following high anion-gap metabolic acidosis (i.e., lactic acidosis) [1]. Both of those conditions are common in septic shock due to systemic inflammation, which impairs oxygen delivery and extraction, capillary integrity, myocardial function, and vascular tone. For such patients, international guidelines recommend the infusion of at least 30 ml/kg of a crystalloid solution, measurement of serum lactate levels, drawing of blood cultures, and the administration of broad-spectrum antibiotics. If hypotension persists, norepinephrine is indicated as the first-line vasopressor, with further additions as necessary [2,3]. Adherence to this bundle can differ depending on the time of presentation [4]; however, as the disease progresses, oxygenation worsens, requiring progressively higher levels of oxygen support, such as a high-flow nasal cannula [5]. At that point, physicians should consider tracheal intubation to prevent cardiovascular collapse and manage respiratory failure [6].
According to a survey of intensivists in Europe, the decision to intubate a patient is based on the physician’s subjective experience and the patient’s respiratory, neurological, and cardiovascular phenotypes [7]. However, physician experience can differ with their years of practice and qualifications, summarized colloquially as the clinical gestalt [8]; previous studies have investigated whether physician experience is more useful than clinical scoring rules, and the results have been mixed. Furthermore, in fast-paced environments such as EDs, where overcrowding and resource allocation must be accounted for concurrently, there might simply be insufficient time and resources to consider an elective procedure. In those situations, a risk stratification tool developed specifically for use during resuscitation could prove beneficial. We hypothesized that we could use supervised machine learning techniques with features routinely available during the initial stages of resuscitation to create an algorithm that could stratify the risks of intubation during the first 24 hours after the recognition of septic shock in the ED.
Research Ethics
This study adhered to the Declaration of Helsinki. The Institutional Review Board of Seoul National University Hospital approved the study (No. E-2406-115-1545) and waived the requirement for written informed consent.
Study Population
This multicenter retrospective observational study analyzed data from the Korean Shock Society septic shock registry [9]. This registry contains information such as demographics, vital signs, medical histories, blood test results (including initial arterial blood gas analysis), time and duration of tracheal intubation (if applicable), summaries of care provided, and outcomes of patients with sepsis who initially presented to the ED at any of 21 university-affiliated hospitals in the Republic of Korea. At the time of writing, the registry covered the period from September 2015 to December 2023.
Inclusion Criteria
A systolic blood pressure (SBP) <90 mm Hg, mean arterial pressure <70 mm Hg, or SBP reduction >40 mm Hg were defined as hypotension. If hypotension persisted after the (1) administration of a fluid challenge or (2) use of vasopressors to maintain a blood pressure (BP) ≥90 mm Hg or mean arterial pressure ≥70 mm Hg, the patient was deemed to be experiencing refractory hypotension [10,11]. Hypoperfusion was defined as a serum lactate level ≥4 mmol/L. Patients suspected to have an infection whose Sequential Organ Failure Assessment (SOFA) score changed ≥2 from baseline were included in the registry. The baseline SOFA score was assumed to be 0 for patients with missing values. Patients were enrolled when either their lactate level was ≥4 mmol/L or they exhibited ongoing hypotension despite initial fluids in the ED. In this study, we included patients from the registry who were ≥19 years old and screened positive for septic shock based on the SEPSIS-3 criteria [12].
Exclusion Criteria
Patients with confirmed documentation to withhold intensive care unit (ICU) care and/or tracheal intubation before study enrollment were excluded from the analysis. Those who underwent tracheal intubation before study enrollment and those who underwent tracheal intubation within 10 minutes of ED arrival were also excluded. Patients who underwent tracheal intubation after more than 24 hours were excluded because they were considered to have been intubated late. Patients with missing values in the final set of features were deleted in a list-wise manner. Multiple imputations of missing values were not conducted because visualization of the missing values map indicated that they were not missing at random.
Feature Selection
Features were screened based on availability during the initial ED evaluation and presumed relevance to the application of tracheal intubation. Therefore, features such as the maximum SOFA score in a 24-hour period were excluded because they could not be known a priori at the time of resuscitation and initial evaluation. Based on a review of the pathophysiology and literature [7,13 14], the following features were selected: age, SBP at enrollment, diastolic blood pressure (DBP) at enrollment, respiratory rate (RR) at enrollment, heart rate (HR) at enrollment, body temperature (BT) at enrollment, altered mental status at enrollment, initial pH, initial HCO3– level, initial partial pressure of CO2, lactate level after initial fluids, suspected lung infection, suspected hepatobiliary infection, suspected urinary tract infection, and suspected gastrointestinal infection. A (normalized) correlation matrix (Supplementary Figure 1) was computed and visualized to assess the degree of collinearity, indicated by an absolute value of the normalized correlation coefficient ≥0.7 [15].
Primary Outcome
The primary outcome was dichotomized to indicate whether the patient received tracheal intubation within the first 24 hours (early) after the recognition of septic shock in the ED.
Statistical Analysis
Dichotomized with respect to the outcome, continuous features were tested for normality and equality of variances using the Shapiro-Wilk test and Levene’s tests, respectively. Because normality and equal variances were not observed for all continuous features, the Mann-Whitney U-test was used to compute the median and interquartile ranges (25% and 75% quartiles) for these features. Categorical variables are presented as numbers and percentages and were compared using the chi-square test or Fisher’s exact test, as appropriate. Two-sided P-values <0.05 were deemed statistically significant. The 95% confidence intervals (CIs) for the sensitivity, specificity, positive predictive value, and negative predictive value were calculated based on the normal approximation. The 95% CIs for the area under the receiver operating characteristic curve (AUROC) were calculated using DeLong’s method. The 95% CIs for area under the precision-recall curve (AUPRC) were approximated using the logit transformation method. The 95% CIs for the F1 scores were approximated using adjusted Wilson score intervals.
Principal Component Analysis
To visualize and explore the presence or absence of a non-linear decision boundary, a principal component analysis (Supplementary Figure 2) was performed on the continuous features. Principal component–transformed features were not used for downstream prediction; instead, they were used to confirm the presence of a non-linear decision boundary, which would necessitate the use of models such as extreme gradient boosting machines that can capture complex, non-linear relationships.
Machine Learning Analysis
We divided the entire dataset into training and test sets (75%:25%) and stratified them with respect to the primary outcome. The means and variances of the feature vectors in the training split were calculated to scale each feature to a zero mean and unit variance. Subsequently, an element-wise operation was performed for each feature in the test set based on the means and variances obtained previously to prevent data leakage. An extreme gradient boosting machine with L2 regularization was subjected to a grid search with five-fold cross validation on the training set to obtain the best hyperparameters. The search space for the grid search and the best hyperparameters obtained are provided in Supplementary Table 1. Using the best model, the AUROC, AUPRC, sensitivity, specificity, positive predictive value, negative predictive value, and F1 scores were calculated after predictions were made for the test set, which was unseen during training. Shapley values were calculated for each feature to assess their preference levels and impacts on the model output. Precision and recall were plotted, and F1 was calculated at each threshold to determine the decision threshold that maximized it (Supplementary Figure 3).
Analysis Tools
All code was written in Python (3.11.8) within a Linux OS environment (Ubuntu 24.04 LTS, Intel Core i9-10900X CPU @ 3.70 GHz). Extreme gradient boosting with CUDA support was implemented using the XGBoost library [16], available at http://github.com/dmlc/xgboost. The Shapley values [17] were calculated using http://github.com/shap/shap. The grid search was implemented using nested for-loops, with the best hyperparameters updated based on the highest average AUROC across all five folds for all combinations in the search space. All matrix and vector computations were performed with single-precision using CuPy tensors on a single graphical processing unit (NVIDIA GeForce RTX 3090).
The study flowchart study is presented in Figure 1. In total, 10,376 patients were screened. Three of them were excluded due to age limitations, and 4,441 did not meet the SEPSIS-3 inclusion criteria, leaving 5,932 patients. An additional 402 patients were excluded based on advance directives that were effective prior to arrival, resulting in 5,530 patients. A further 257 patients were intubated prior to enrollment, and 306 were intubated after more than 24 hours, leaving 4,967 patients. Fifty-four more patients were excluded because they were intubated within 10 minutes of ED arrival. After removing patients with missing or erroneous values in the final set of features, we analyzed the data for 4,762 patients; 1,486 (31%) patients were intubated within the 24-hour window, and 3,276 (69%) were not. The frequency of tracheal intubation in aggregated bins of one unit hour after the diagnosis of septic shock is given in Supplementary Figure 4.
Baseline demographics for the entire set (100%) are shown in Table 1 with respect to tracheal intubation within 24 hours after enrollment. SBP, DBP, HR, RR, BT, and altered mental status at enrollment showed statistically significant differences (P<0.001) in relation to the primary outcome. In addition, all features related to the arterial blood gas analysis showed significant differences between those who did and did not receive tracheal intubation within 24 hours (P<0.001). In terms of the site of infection, suspected lung, urinary tract, gastrointestinal, and hepatobiliary infections differed with statistical significance between groups (P<0.001). Similar differences were recorded for known chronic lung disease, SOFA score at enrollment, maximum SOFA score over 24 hours, and the Acute Physiology and Chronic Health Evaluation (APACHE) II score. Examination of the APACHE II score and maximum SOFA score over 24 hours after enrollment reflected the a posteriori distribution of baseline comorbidities and severity of illness, respectively, in the subset of the population included in this study. Patients who were intubated had higher SOFA and APACHE II scores than those who were not.
Table 2 (training set, 75%) and Table 3 (test set, 25%) show trends similar to those in Table 1. A comparison of the training and test sets is given in Supplementary Table 2; all P-values were >0.05, indicating a random, adequate training/test split. Figures 2 and 3, and Table 4 summarize the performance of the best model, which was obtained through a grid search and evaluated on the test set. In the prediction of early tracheal intubation in patients with septic shock, the AUROC, AUPRC, sensitivity, specificity, positive predictive value, negative predictive value, and F1 score were 0.829 (95% CI, 0.801–0.878), 0.702 (95% CI, 0.676–0.728), 0.740 (95% CI, 0.695–0.784), 0.763 (95% CI, 0.734–0.792), 0.586 (95% CI, 0.542–0.631), 0.866 (95% CI, 0.841–0.891), and 0.654 (95% CI, 0.627–0.681), respectively. Figure 4 shows a confusion matrix for the best model evaluated on the test set; it produced 275 true positives, 625 true negatives, 194 false positives, and 97 false negatives.
Figure 5 shows that the top five features in the order of importance were lactate level after initial fluids, suspected lung infection, initial pH, SOFA score at enrollment, and RR at enrollment. As the lactate level after initial fluids became higher, the initial pH became lower, the SOFA score at enrollment became higher, and the RR at enrollment became higher, the model predictions became more likely to favor intubation. Suspected hepatobiliary and urinary tract infections worked against intubation, whereas suspected lung and suspected gastrointestinal infections favored intubation.
For this study, we trained an extreme gradient boosting machine to discriminate the need for tracheal intubation of patients with septic shock within the first 24 hours after diagnosis in the ED. Its performance in terms of the AUROC was good, whereas its precision and recall were suboptimal [18]. The F1 score revealed that the predictions of the model were moderately reliable. The principal component analysis in Supplementary Figure 2 showed that the dimension reduction of the feature space did not yield a clearly separable boundary, with overlaps between outcomes based on the features selected. Therefore, our choice of an extreme gradient boosting machine to discover non-linear decision boundaries was appropriate. Gradient boosting machines progressively accumulate estimators to correct previously misclassified labels, and thus they can capture complex, non-linear relationships. Overall, the gaps suggest the existence of additional features that still need to be discovered, in accordance with a previous study [14]. The identification of additional predictors should therefore be the focus of further research.
Tools such as the ROX index and HR, Acidosis, Consciousness, Oxygenation, and Respiratory Rate (HACOR) score require serial calculations to make accurate predictions, so they might not be directly applicable to patients who are rapidly deteriorating with septic shock when they first present to the ED [19,20]. For instance, serial measurements of the HACOR scale in patients who were successfully weaned from non-invasive ventilation exhibited decreasing scores over time, in contrast to the score of those who were intubated [21]. Observation for 48 hours is not a practical option in an ED. Those tools were designed to evaluate patients for elective procedures, not patients requiring resuscitation in the ED, and their performance has been limited when they have been tested in EDs [22]. Moreover, they were not designed for patients with septic shock.
Independent risk factors for mortality within 72 hours of ED arrival have been identified previously [23]. Although there is an ongoing debate about whether early tracheal intubation should be performed within 8 or 24 hours after the recognition of septic shock, early intubation does have potential benefits in lowering the 30- and 90-day all-cause mortality rates. An analysis of the Medical Information Mart for Intensive Care-IV data showed that the risk of death in patients with septic shock peaks at 50.5 hours post recognition, with an inverse U-shaped correlation to the 30-day all-cause mortality rate [24]. Another study showed that intubation within 24 hours of a sepsis diagnosis resulted in fewer 28-day hospital-free days [25]. In another study, intubating patients with septic shock within 8 hours after vasopressor initiation did not demonstrate definite survival benefits, although the number of patients analyzed might have limited the generalizability of those results [26]. Another study reported decreased ICU mortality rates, lower in-hospital mortality, shorter ICU stays, and lower tracheostomy rates in patients with septic shock who were intubated early [27]. The outcomes following early or late intubation were not the primary subject of our present analysis. We investigated only whether early intubation could be predicted. Therefore, future studies should continue to investigate how the timing of intubation in the ED correlates with long- and short-term outcomes.
Machine learning was previously used and validated for predicting reintubation rates after extubation [28]. It has also been applied to predict weaning from tracheal intubation in the ICU [29]. We used machine learning techniques to predict the need for tracheal intubation in patients with septic shock after they were recognized and evaluated and resuscitation was initiated. Physician experience, and therefore the decision to intubate a deteriorating patient, can differ depending on their years of practice and qualifications (for example, a subspecialty of critical care medicine in addition to emergency medicine). Our results in this study could serve as objective clinical decision support when the decision to intubate must be planned concurrently with ED overcrowding.
Previous studies have not evaluated feature importance, but instead relied on aggregated item scores to suggest cut-offs; Shapley values have only recently been used in emergency medicine research to enhance the interpretability, explainability, and transparency of artificial intelligence model outputs [30]. The Shapley values in this study indicate that the top five features that the extreme gradient boosting machine used to predict early tracheal intubation in patients with septic shock were: lactate level after initial fluids, suspected lung infection, initial pH, SOFA score at enrollment, and RR at enrollment.
This study has the following limitations. First, it is subject to selection bias because our patient population might not be representative of all patients with septic shock, given the different phenotypes of sepsis [31]. Second, direct comparison between our results and the ROX index, HACOR scale, or updated HACOR scale [32] was not feasible because the Korean Shock Society septic shock registry did not collect data on the fraction of inspired oxygen or the Glasgow Coma Scale. However, we have shown in this analysis that simple, dichotomized features combined with continuous numerical features routinely available at the time of ED evaluation and resuscitation enabled our model to make moderately accurate predictions. Third, evidence indicates that patients with cardiogenic pulmonary edema can be successfully weaned from non-invasive ventilation; however, due to a large number of missing values for cardiac markers in our dataset, we were unable to take them into consideration. Fourth, management levels might have differed across sites, and on-site differences in the management of patients with septic shock might have introduced implicit bias. Fifth, the model developed herein could benefit from further external validation before it is applied clinically.
In conclusion, an extreme gradient boosting machine moderately discriminated early tracheal intubation in patients with septic shock using features commonly obtained in EDs during the initial resuscitation and evaluation stages. However, early tracheal intubation cannot be conclusively predicted using only the model developed in this study; clinicians must continue to make careful judgements when determining whether early tracheal intubation is warranted.
▪ At the time of septic shock recognition in the emergency department, vital signs at enrollment, suspected infection focus, and arterial blood gas analysis were assessed to predict tracheal intubation within a 24-hour window.
▪ An extreme gradient boosting machine, coupled with five-fold cross-validation and grid search, was able to moderately discriminate the need for tracheal intubation within 24 hours.
▪ Shapley values revealed that the top five features for prediction were lactate level after initial fluids, suspected lung infection, initial pH, Sequential Organ Failure Assessment score at enrollment, and respiratory rate at enrollment.

CONFLICT OF INTEREST

No potential conflict of interest relevant to this article was reported.

FUNDING

This research was supported by the Basic Research Training Program for MD, Seoul National University College of Medicine, to Ji Han Heo and by a grant from the Patient-Centered Clinical Research Coordinating Center (PACEN) funded by the Ministry of Health & Welfare, Republic of Korea (grant No. RS-2024-00398566) to Tae Gun Shin.

ACKNOWLEDGMENTS

We would like to acknowledge and thank the following Korean Shock Society (KoSS) Investigators:

Won Young Kim, Sang-Min Kim, Seung Mok Ryoo (Asan Medical Center); Gun Tak Lee, Sung Yeon Hwang (Samsung Medical Center); Tae Ho Lim, Byuk Sung Ko (Hanyang University Seoul Hospital); Sung-Hyuk Choi, Sung-Joon Park (Korea University Guro Hospital); Yoo Seok Park, Jin Ho Beom (Severance Hospital); Yoon Sun Jung (Seoul National University Hospital); Juhyun Song, Kap Su Han (Korea University Anam Hospital); Sung Phil Chung, Taeyoung Kong, Eunah Han (Gangnam Severance Hospital); You Hwan Jo, Ji Eun Hwang (Seoul National University Bundang Hospital); Jonghwan Shin, Hui Jai Lee (Seoul Metropolitan Government–Seoul National University Boramae Medical Center); Gu Hyun Kang (Hallym University Kangnam Sacred Heart Hospital); Hanjin Cho, Sejoong Ahn (Korea University Ansan Hospital); Hong Joon Ahn (Chungnam National University Hospital); Kyuseok Kim (CHA Bundang Medical Center); Kihwan Choi (CHA Gumi Medical Center); Han Sung Choi, Ki Young Jeong, Seok Hun Ko (Kyung Hee University Hospital); Hyo Jin Bang (Seoul St. Mary's Hospital); Jinwoo Jeoung, Min Joon Seo (Dong-A University Hospital); Sangsoo Han, Sangchun Choi (University Bucheon Hospital); Heewon Yang (Ajou University Hospital); Chiwon Ahn (Chung-Ang University Hospital); Changsun Kim, Hyungoo Shin (Hanyang University Guri Hospital).

AUTHOR CONTRIBUTIONS

Conceptualization: TK, GJS, WYK. Methodology: JHH, HK, SH. Formal analysis: JHH, HK, HP, GJS, WYK. Data curation: TK, TGS, HK, HP, HK, SH. Visualization: JHH, SH. Project administration: TK. Funding acquisition: JHH, TGS. Writing – original draft: JHH, HK, SH. Writing – review & editing: TK, TGS, GJS, WYK, HK, HP. All authors read and agreed to the published version of the manuscript.

Supplementary materials can be found via https://doi.org/10.4266/acc.004776.
Supplementary Table 1.
The search space for 5-fold cross-validation grid search
acc-004776-Supplementary-Table-1.pdf
Supplementary Table 2.
Demographics and baseline features of the entire set based on train or test set
acc-004776-Supplementary-Table-2.pdf
Supplementary Figure 1.
Normalized correlation coefficient matrix of the entire set. SBP: systolic blood pressure; enroll: enrollment; DBP: diastolic blood pressure; HR: heart rate; RR: respiratory rate; BT: body temperature; pCO2: partial pressure of CO2; SOFA: Sequential Organ Failure Assessment.
acc-004776-Supplementary-Figure-1.pdf
Supplementary Figure 2.
(A) Principal components in the order of decreasing explained variance. (B) Visualization of the primary outcome based on the first two principal components. The first two principal components with the highest explained variance do not reveal a clearly separable boundary. PCA: principal component analysis.
acc-004776-Supplementary-Figure-2.pdf
Supplementary Figure 3.
The optimal decision threshold for maximizing the F1 score is 0.471.
acc-004776-Supplementary-Figure-3.pdf
Supplementary Figure 4.
Frequency of tracheal intubation versus time after enrollment in 1-hour bins.
acc-004776-Supplementary-Figure-4.pdf
Figure 1.
Study flowchart. KoSS: Korean Shock Society; ED: emergency department.
acc-004776f1.jpg
Figure 2.
Area under the receiver operating characteristics (AUROC) curve in relation to tracheal intubation within 24 hours after enrollment, evaluated on the test set.
acc-004776f2.jpg
Figure 3.
Area under the precision recall characteristics (AUPRC) curve in relation to tracheal intubation within 24 hours after enrollment, evaluated on the test set.
acc-004776f3.jpg
Figure 4.
Confusion matrix in relation to tracheal intubation within 24 hours after enrollment, evaluated on the test set.
acc-004776f4.jpg
Figure 5.
Shapley values for tracheal intubation within 24 hours. The top five features used by the extreme gradient boosting machine are lactate level after initial fluids, suspected lung infection, initial pH, Sequential Organ Failure Assessment (SOFA) score at enrollment, and respiratory rate (RR) at enrollment, with higher lactate levels after initial fluids, higher initial RR, and lower pH levels increasing the likelihood that early tracheal intubation will be applied. enroll: enrollment; HR: heart rate; DBP: diastolic blood pressure; pCO2: partial pressure of CO2; SBP: systolic blood pressure; BT: body temperature; RR: respiratory rate; SHAP: Shapley additive explanations.
acc-004776f5.jpg
Table 1.
Demographics and baseline features of the entire set on intubation within 24 hour window
Variable Overall Intubation (–) Intubation (+) P-value
Number 4,762 3,276 1,486 -
Age (yr) 70 (61–78) 69 (60–78) 70 (62–78) 0.017
Sex 2,887 (60.6) 1,923 (58.7) 964 (64.9) <0.001
 Male 1,875 (39.4) 1,353 (41.3) 522 (35.1)
Vital sign
 SBP at enroll 84.0 (74.0–97.0) 84.0 (75.0–94.0) 86.0 (74.0–106.0) <0.001
 DBP at enroll 50.0 (43.0–59.0) 50.0 (43.0–57.0) 52.0 (44.0–62.8) <0.001
 HR at enroll 107.5 (93.0–123.0) 105.0 (91.0–119.0) 114.0 (98.0–130.0) <0.001
 RR at enroll 22.0 (20.0–26.0) 20.0 (18.0–24.0) 24.0 (20.0–30.0) <0.001
 BT at enroll 37.4 (36.6–38.2) 37.5 (36.7–38.3) 37.2 (36.4–38.1) <0.001
 Altered mental status No 3,660 (76.9) 2,713 (82.8) 947 (63.7) <0.001
Yes 1,102 (23.1) 563 (17.2) 539 (36.3)
 Vasopressor use No 56 (1.2) 32 (1.0) 24 (1.6) 0.080
Yes 4,706 (98.8) 3244 (99.0) 1462 (98.4)
 Refractory hypotension No 896 (18.8) 499 (15.2) 397 (26.7) <0.001
Yes 3,866 (81.2) 2,777 (84.8) 1,089 (73.3)
 qSOFA ≥2 at enroll No 2,345 (49.2) 1,804 (55.1) 541 (36.4) <0.001
Yes 2,417 (50.8) 1,472 (44.9) 945 (63.6)
Arterial blood gas analysis
 Initial pH 7.4 (7.3–7.5) 7.4 (7.4–7.5) 7.4 (7.3–7.4) <0.001
 Initial pCO2 27.3 (23.0–32.0) 27.1 (23.1–31.1) 28.0 (22.6–35.0) <0.001
 Initial HCO3– 17.7 (14.3–20.9) 18.2 (15.2–21.1) 16.0 (12.4–19.8) <0.001
 Lactate at enroll 4.5 (3.1–6.8) 4.2 (2.8–5.8) 5.9 (4.2–8.8) <0.001
 Lactate after initial fluid 4.0 (2.8–6.1) 3.5 (2.6–5.2) 5.5 (3.5–8.3) <0.001
Site of infection
 Suspected lung infection No 3,433 (72.1) 2,660 (81.2) 773 (52.0) <0.001
Yes 1,329 (27.9) 616 (18.8) 713 (48.0)
 Suspected urinary tract infection No 3,635 (76.3) 2,376 (72.5) 1,259 (84.7) <0.001
Yes 1,127 (23.7) 900 (27.5) 227 (15.3)
 Suspected gastrointestinal infection No 3,744 (78.6) 2,637 (80.5) 1,107 (74.5) <0.001
Yes 1,018 (21.4) 639 (19.5) 379 (25.5)
 Suspected hepatobiliary infection No 3,610 (75.8) 2,313 (70.6) 1,297 (87.3) <0.001
Yes 1,152 (24.2) 963 (29.4) 189 (12.7)
 Suspected bone or soft tissue infection No 4,587 (96.3) 3,171 (96.8) 1,416 (95.3) 0.013
Yes 175 (3.7) 105 (3.2) 70 (4.7)
 Suspected central nervous system infection No 4,744 (99.6) 3,269 (99.8) 1,475 (99.3) 0.013
Yes 18 (0.4) 7 (0.2) 11 (0.7)
 Suspected catheter related infection No 4,708 (98.9) 3,238 (98.8) 1,470 (98.9) 0.917
Yes 54 (1.1) 38 (1.2) 16 (1.1)
 Suspected bloodstream infection No 4,572 (96.0) 3,141 (95.9) 1,431 (96.3) 0.545
Yes 190 (4.0) 135 (4.1) 55 (3.7)
Past medical history
 Hypertension No 2,680 (56.3) 1,831 (55.9) 849 (57.1) 0.442
Yes 2,082 (43.7) 1,445 (44.1) 637 (42.9)
 Diabetes mellitus No 3,027 (63.6) 2,067 (63.1) 960 (64.6) 0.332
Yes 1,735 (36.4) 1,209 (36.9) 526 (35.4)
 Cardiac disease No 4,123 (86.6) 2,858 (87.2) 1,265 (85.1) 0.053
Yes 639 (13.4) 418 (12.8) 221 (14.9)
 Known cerebrovascular accident No 4,202 (88.2) 2,916 (89.0) 1,286 (86.5) 0.016
Yes 560 (11.8) 360 (11.0) 200 (13.5)
 Known chronic lung disease No 4,453 (93.5) 3,094 (94.4) 1,359 (91.5) <0.001
Yes 309 (6.5) 182 (5.6) 127 (8.5)
 Known chronic renal disease No 4,330 (90.9) 3,006 (91.8) 1,324 (89.1) 0.004
Yes 432 (9.1) 270 (8.2) 162 (10.9)
 Known chronic liver disease No 4,166 (87.5) 2,845 (86.8) 1,321 (88.9) 0.053
Yes 596 (12.5) 431 (13.2) 165 (11.1)
Miscellaneous
 Referral from nursing center No 4,337 (91.1) 3,014 (92.0) 1,323 (89.0) 0.001
Yes 425 (8.9) 262 (8.0) 163 (11.0)
 SOFA score at enroll 7.0 (5.0–9.0) 6.0 (4.0–8.0) 8.0 (6.0–11.0) <0.001
 SOFA score (max over 24 hours) 9.0 (7.0–12.0) 8.0 (6.0–11.0) 12.0 (9.0–15.0) <0.001
 APACHE II score 22.0 (16.0–28.0) 20.0 (15.0–25.0) 28.0 (22.0–36.0) <0.001
 28-Day outcome
  Death 1,188 (24.9) 478 (14.6) 710 (47.8) <0.001
  Survival 3,416 (71.7) 2,663 (81.3) 753 (50.7)
  NA 158 (3.3) 135 (4.1) 23 (1.5)

Values are presented as median (interquartile range) or number (%).

SBP: systolic blood pressure; DBP: diastolic blood pressure; HR: heart rate; RR: respiratory rate; BT: body temperature; qSOFA: quick Sequential Organ Failure Assessment; APACHE: Acute Physiology and Chronic Health Evaluation; NA: not available.

Table 2.
Demographics and baseline features of the train set on intubation within 24 hour window
Variable Overall Intubation (–) Intubation (+) P-value
Number 3,571 2,457 1,114 -
Age (yr) 70 (61–78) 69 (61–78) 70 (62–78) 0.135
Sex 2,157 (60.4) 1,453 (59.1) 704 (63.2) 0.024
 Male 1,414 (39.6) 1,004 (40.9) 410 (36.8)
Vital sign
 SBP at enroll 84.0 (75.0–98.0) 84.0 (75.0–94.0) 86.0 (74.0–106.8) <0.001
 DBP at enroll 50.0 (43.0–59.0) 50.0 (43.0–57.0) 52.0 (43.0–63.0) <0.001
 HR at enroll 108.0 (93.0–123.0) 105.0 (91.0–119.0) 114.0 (98.0–130.0) <0.001
 RR at enroll 22.0 (20.0–26.0) 20.0 (18.0–24.0) 24.0 (20.0–30.0) <0.001
 BT at enroll 37.4 (36.6–38.2) 37.5 (36.7–38.3) 37.1 (36.4–38.1) <0.001
 Altered mental status No 2,744 (76.8) 2,044 (83.2) 700 (62.8) <0.001
Yes 827 (23.2) 413 (16.8) 414 (37.2)
 Vasopressor use No 41 (1.1) 22 (0.9) 19 (1.7) 0.053
Yes 3,530 (98.9) 2,435 (99.1) 1,095 (98.3)
 Refractory hypotension No 681 (19.1) 384 (15.6) 297 (26.7) <0.001
Yes 2,890 (80.9) 2,073 (84.4) 817 (73.3)
 qSOFA ≥2 at enroll No 1,750 (49.0) 1,341 (54.6) 409 (36.7) <0.001
Yes 1,821 (51.0) 1,116 (45.4) 705 (63.3)
Arterial blood gas analysis
 Initial pH 7.4 (7.3–7.5) 7.4 (7.4–7.5) 7.4 (7.3–7.4) <0.001
 Initial pCO2 27.2 (23.0–32.0) 27.0 (23.0–31.0) 28.0 (22.8–34.9) <0.001
 Initial HCO3– 17.7 (14.2–20.8) 18.1 (15.1–21.1) 15.9 (12.4–19.8) <0.001
 Lactate at enroll 4.6 (3.1–6.8) 4.2 (2.8–5.7) 6.1 (4.3–9.0) <0.001
 Lactate after initial fluid 4.0 (2.8–6.1) 3.5 (2.6–5.1) 5.6 (3.6–8.5) <0.001
Site of infection
 Suspected lung infection No 2,576 (72.1) 1,999 (81.4) 577 (51.8) <0.001
Yes 995 (27.9) 458 (18.6) 537 (48.2)
 Suspected urinary tract infection No 2,721 (76.2) 1,779 (72.4) 942 (84.6) <0.001
Yes 850 (23.8) 678 (27.6) 172 (15.4)
 Suspected gastrointestinal infection No 2,800 (78.4) 1,973 (80.3) 827 (74.2) <0.001
Yes 771 (21.6) 484 (19.7) 287 (25.8)
 Suspected hepatobiliary infection No 2,729 (76.4) 1,749 (71.2) 980 (88.0) <0.001
Yes 842 (23.6) 708 (28.8) 134 (12.0)
 Suspected bone or soft tissue infection No 3,434 (96.2) 2,379 (96.8) 1,055 (94.7) 0.003
Yes 137 (3.8) 78 (3.2) 59 (5.3)
 Suspected central nervous system infection No 3,559 (99.7) 2,452 (99.8) 1,107 (99.4) 0.058
Yes 12 (0.3) 5 (0.2) 7 (0.6)
 Suspected catheter related infection No 3,536 (99.0) 2,435 (99.1) 1,101 (98.8) 0.562
Yes 35 (1.0) 22 (0.9) 13 (1.2)
 Suspected bloodstream infection No 3,417 (95.7) 2,348 (95.6) 1,069 (96.0) 0.651
Yes 154 (4.3) 109 (4.4) 45 (4.0)
Past medical history
 Hypertension No 2,038 (57.1) 1,396 (56.8) 642 (57.6) 0.676
Yes 1,533 (42.9) 1,061 (43.2) 472 (42.4)
 Diabetes mellitus No 2,290 (64.1) 1,567 (63.8) 723 (64.9) 0.541
Yes 1,281 (35.9) 890 (36.2) 391 (35.1)
 Cardiac disease No 3,100 (86.8) 2,156 (87.7) 944 (84.7) 0.016
Yes 471 (13.2) 301 (12.3) 170 (15.3)
 Known cerebrovascular accident No 3,154 (88.3) 2,193 (89.3) 961 (86.3) 0.012
Yes 417 (11.7) 264 (10.7) 153 (13.7)
 Known chronic lung disease No 3,336 (93.4) 2,313 (94.1) 1,023 (91.8) 0.012
Yes 235 (6.6) 144 (5.9) 91 (8.2)
 Known chronic renal disease No 3,246 (90.9) 2,252 (91.7) 994 (89.2) 0.023
Yes 325 (9.1) 205 (8.3) 120 (10.8)
 Known chronic liver disease No 3,126 (87.5) 2,135 (86.9) 991 (89.0) 0.094
Yes 445 (12.5) 322 (13.1) 123 (11.0)
Miscellaneous
 Referral from nursing center 3,252 (91.1) 2,260 (92.0) 992 (89.0) 0.005
319 (8.9) 197 (8.0) 122 (11.0)
 SOFA score at enroll 6.0 (5.0–9.0) 6.0 (4.0–8.0) 8.0 (6.0–11.0) <0.001
 SOFA score (max over 24 hours) 9.0 (7.0–12.0) 8.0 (6.0–10.0) 12.0 (9.0–15.0) <0.001
 APACHE II score 22.0 (16.0–29.0) 20.0 (15.0–25.0) 28.0 (22.0–36.0) <0.001
 28-Day outcome <0.001
  Death 886 (24.8) 359 (14.6) 527 (47.3)
  Survival 2,566 (71.9) 1,996 (81.2) 570 (51.2)
  NA 119 (3.3) 102 (4.2) 17 (1.5)

Values are presented as median (interquartile range) or number (%).

SBP: systolic blood pressure; DBP: diastolic blood pressure; HR: heart rate; RR: respiratory rate; BT: body temperature; qSOFA: quick Sequential Organ Failure Assessment; APACHE: Acute Physiology and Chronic Health Evaluation; NA: not available.

Table 3.
Demographics and baseline features of the test set on intubation within 24 hour window
Variable Overall Intubation (–) Intubation (+) P-value
Number 1,191 819 372
Age (yr) 70 (61–78) 69 (60–77) 71 (62–79) 0.030
Sex 730 (61.3) 470 (57.4) 260 (69.9) <0.001
 Male 461 (38.7) 349 (42.6) 112 (30.1)
Vital sign
 SBP at enroll 85.0 (74.0–96.0) 84.0 (74.0–93.0) 86.5 (72.8–103.0) 0.002
 DBP at enroll 50.0 (43.0–58.0) 50.0 (42.0–56.0) 53.0 (44.0–62.0) <0.001
 HR at enroll 107.0 (93.0–123.0) 104.0 (91.0–119.0) 115.0 (100.0–130.2) <0.001
 RR at enroll 22.0 (19.0–26.0) 20.0 (18.0–24.0) 24.0 (20.0–30.0) <0.001
 BT at enroll 37.4 (36.6–38.2) 37.5 (36.7–38.2) 37.3 (36.5–38.1) 0.014
 Altered mental status No 916 (76.9) 669 (81.7) 247 (66.4) <0.001
Yes 275 (23.1) 150 (18.3) 125 (33.6)
 Vasopressor use No 15 (1.3) 10 (1.2) 5 (1.3) 1.000
Yes 1,176 (98.7) 809 (98.8) 367 (98.7)
 Refractory hypotension No 215 (18.1) 115 (14.0) 100 (26.9) <0.001
Yes 976 (81.9) 704 (86.0) 272 (73.1)
 qSOFA ≥2 at enroll No 595 (50.0) 463 (56.5) 132 (35.5) <0.001
Yes 596 (50.0) 356 (43.5) 240 (64.5)
Arterial blood gas analysis
 Initial pH 7.4 (7.4–7.5) 7.4 (7.4–7.5) 7.4(7.3–7.4) <0.001
 Initial pCO2 27.7 (23.3–32.0) 27.3 (23.7–31.4) 28.4 (22.5–35.0) 0.080
 Initial HCO3– 18.0 (14.6–21.0) 18.5 (15.4–21.2) 16.5 (12.5–20.1) <0.001
 Lactate at enroll 4.5 (3.1–6.5) 4.2 (2.8–5.9) 5.6(4.0–8.1) <0.001
 Lactate after initial fluid 3.9 (2.7–6.0) 3.5 (2.6–5.2) 5.0(3.4–7.8) <0.001
Site of infection
 Suspected lung infection No 857 (72.0) 661 (80.7) 196 (52.7) <0.001
Yes 334 (28.0) 158 (19.3) 176 (47.3)
 Suspected urinary tract infection No 914 (76.7) 597 (72.9) 317 (85.2) <0.001
Yes 277 (23.3) 222 (27.1) 55 (14.8)
 Suspected gastrointestinal infection No 944 (79.3) 664 (81.1) 280 (75.3) 0.027
Yes 247 (20.7) 155 (18.9) 92 (24.7)
 Suspected hepatobiliary infection No 881 (74.0) 564 (68.9) 317 (85.2) <0.001
Yes 310 (26.0) 255 (31.1) 55 (14.8)
 Suspected bone or soft tissue infection No 1,153 (96.8) 792 (96.7) 361 (97.0) 0.896
Yes 38 (3.2) 27 (3.3) 11 (3.0)
 Suspected central nervous system infection No 1,185 (99.5) 817 (99.8) 368 (98.9) 0.080
Yes 6 (0.5) 2 (0.2) 4 (1.1)
 Suspected catheter related infection No 1,172 (98.4) 803 (98.0) 369 (99.2) 0.224
Yes 19 (1.6) 16 (2.0) 3 (0.8)
 Suspected bloodstream infection No 1,155 (97.0) 793 (96.8) 362 (97.3) 0.786
Yes 36 (3.0) 26 (3.2) 10 (2.7)
Past medical history
 Hypertension No 642 (53.9) 435 (53.1) 207 (55.6) 0.454
Yes 549 (46.1) 384 (46.9) 165 (44.4)
 Diabetes mellitus No 737 (61.9) 500 (61.1) 237 (63.7) 0.417
Yes 454 (38.1) 319 (38.9) 135 (36.3)
 Cardiac disease No 1,023 (85.9) 702 (85.7) 321 (86.3) 0.861
Yes 168 (14.1) 117 (14.3) 51 (13.7)
 Known cerebrovascular accident No 1,048 (88.0) 723 (88.3) 325 (87.4) 0.724
Yes 143 (12.0) 96 (11.7) 47 (12.6)
 Known chronic lung disease No 1,117 (93.8) 781 (95.4) 336 (90.3) 0.001
Yes 74 (6.2) 38 (4.6) 36 (9.7)
 Known chronic renal disease No 1,084 (91.0) 754 (92.1) 330 (88.7) 0.077
Yes 107 (9.0) 65 (7.9) 42 (11.3)
 Known chronic liver disease No 1,040 (87.3) 710 (86.7) 330 (88.7) 0.381
Yes 151 (12.7) 109 (13.3) 42 (11.3)
Miscellaneous
 Referral from nursing center No 1,085 (91.1) 754 (92.1) 331 (89.0) 0.105
Yes 106 (8.9) 65 (7.9) 41 (11.0)
 SOFA score at enroll 7.0 (5.0–9.0) 6.0 (5.0–8.0) 8.0 (5.8–11.0) <0.001
 SOFA score (max over 24 hours) 9.0 (7.0–12.0) 9.0 (7.0–11.0) 12.0 (9.0–15.0) <0.001
 APACHE II score 21.0 (16.0–28.0) 19.0 (15.0–24.0) 27.0 (20.0–36.0) <0.001
 28-Day outcome <0.001
  Death 302 (25.4) 119 (14.5) 183 (49.2)
  Survival 850 (71.4) 667 (81.4) 183 (49.2)
  NA 39 (3.3) 33 (4.0) 6 (1.6)

Values are presented as median (interquartile range) or number (%).

SBP: systolic blood pressure; DBP: diastolic blood pressure; HR: heart rate; RR: respiratory rate; BT: body temperature; qSOFA: quick Sequential Organ Failure Assessment; APACHE: Acute Physiology and Chronic Health Evaluation; NA: not available.

Table 4.
Performance metrics regarding intubation within 24 hours after enroll, evaluated on the test set
Metrics Value (95% CI)
AUROC 0.829 (0.801–0.878)
AUPRC 0.702 (0.676–0.728)
Sensitivity 0.740 (0.695–0.784)
Specificity 0.763 (0.734–0.792)
Positive predictive value 0.586 (0.542–0.631)
Negative predictive value 0.866 (0.841–0.891)
F1 score 0.654 (0.627–0.681)

AUROC: area under the receiver operating characteristic curve; AUPRC: area under the precision-recall curve.

  • 1. Russell JA, Rush B, Boyd J. Pathophysiology of septic shock. Crit Care Clin 2018;34:43-61.ArticlePubMed
  • 2. Evans L, Rhodes A, Alhazzani W, Antonelli M, Coopersmith CM, French C, et al. Surviving sepsis campaign: international guidelines for management of sepsis and septic shock 2021. Intensive Care Med 2021;47:1181-247.ArticlePubMedPMC
  • 3. Suh GJ, Shin TG, Kwon WY, Kim K, Jo YH, Choi SH, et al. Hemodynamic management of septic shock: beyond the Surviving Sepsis Campaign guidelines. Clin Exp Emerg Med 2023;10:255-64.ArticlePubMedPMCPDF
  • 4. You JS, Park YS, Chung SP, Lee HS, Jeon S, Kim WY, et al. Relationship between time of emergency department admission and adherence to the Surviving Sepsis Campaign bundle in patients with septic shock. Crit Care 2022;26:43.ArticlePubMedPMCPDF
  • 5. Frat JP, Ragot S, Coudroy R, Constantin JM, Girault C, Prat G, et al. Predictors of intubation in patients with acute hypoxemic respiratory failure treated with a noninvasive oxygenation strategy. Crit Care Med 2018;46:208-15.ArticlePubMed
  • 6. Jaber S, Bellani G, Blanch L, Demoule A, Esteban A, Gattinoni L, et al. The intensive care medicine research agenda for airways, invasive and noninvasive mechanical ventilation. Intensive Care Med 2017;43:1352-65.ArticlePubMedPDF
  • 7. de Montmollin E, Aboab J, Ferrer R, Azoulay E, Annane D. Criteria for initiation of invasive ventilation in septic shock: an international survey. J Crit Care 2016;31:54-7.ArticlePubMed
  • 8. Li YL, Mo JR, Cheng NM, Chan SS, Lin PY, Chen XH, et al. Gestalt for shock and mortality in the emergency department: a prospective study. Am J Emerg Med 2018;36:988-92.ArticlePubMed
  • 9. Shin TG, Hwang SY, Kang GH, Kim WY, Ryoo SM, Kim K, et al. Korean Shock Society septic shock registry: a preliminary report. Clin Exp Emerg Med 2017;4:146-53.ArticlePubMedPMCPDF
  • 10. ProCESS Investigators; Yealy DM, Kellum JA, Huang DT, Barnato AE, Weissfeld LA, et al. A randomized trial of protocol-based care for early septic shock. N Engl J Med 2014;370:1683-93.ArticlePubMedPMC
  • 11. Mouncey PR, Osborn TM, Power GS, Harrison DA, Sadique MZ, Grieve RD, et al. Trial of early, goal-directed resuscitation for septic shock. N Engl J Med 2015;372:1301-11.ArticlePubMed
  • 12. Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA 2016;315:801-10.ArticlePubMedPMC
  • 13. Bauer PR, Kumbamu A, Wilson ME, Pannu JK, Egginton JS, Kashyap R, et al. Timing of intubation in acute respiratory failure associated with sepsis: a mixed methods study. Mayo Clin Proc 2017;92:1502-10.ArticlePubMed
  • 14. Darreau C, Martino F, Saint-Martin M, Jacquier S, Hamel JF, Nay MA, et al. Use, timing and factors associated with tracheal intubation in septic shock: a prospective multicentric observational study. Ann Intensive Care 2020;10:62.ArticlePubMedPMCPDF
  • 15. Dormann CF, Elith J, Bacher S, Buchmann C, Carl G, Carré G, et al. Collinearity: a review of methods to deal with it and a simulation study evaluating their performance. Ecography 2013;36:27-46.Article
  • 16. Chen T, Guestrin C. XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; 2016 Aug 13-17; San Francisco, CA. New York, NY: ACM; 2016. p. 785-94.
  • 17. Lundberg SM, Lee SI. A unified approach to interpreting model predictions. Adv Neural Inf Process Syst 30 2017;5:4768-77.
  • 18. Mandrekar JN. Receiver operating characteristic curve in diagnostic test assessment. J Thorac Oncol 2010;5:1315-6.ArticlePubMed
  • 19. Roca O, Messika J, Caralt B, García-de-Acilu M, Sztrymf B, Ricard JD, et al. Predicting success of high-flow nasal cannula in pneumonia patients with hypoxemic respiratory failure: the utility of the ROX index. J Crit Care 2016;35:200-5.ArticlePubMed
  • 20. Duan J, Han X, Bai L, Zhou L, Huang S. Assessment of heart rate, acidosis, consciousness, oxygenation, and respiratory rate to predict noninvasive ventilation failure in hypoxemic patients. Intensive Care Med 2017;43:192-9.ArticlePubMedPDF
  • 21. Carrillo A, Lopez A, Carrillo L, Caldeira V, Guia M, Alonso N, et al. Validity of a clinical scale in predicting the failure of non-invasive ventilation in hypoxemic patients. J Crit Care 2020;60:152-8.ArticlePubMed
  • 22. Chong CY, Bustam A, Noor Azhar M, Abdul Latif AK, Ismail R, Poh K. Evaluation of HACOR scale as a predictor of non-invasive ventilation failure in acute cardiogenic pulmonary oedema patients: a prospective observational study. Am J Emerg Med 2024;79:19-24.ArticlePubMed
  • 23. Reaven MS, Rozario NL, McCarter MS, Heffner AC. Incidence and risk factors associated with early death in patients with emergency department septic shock. Acute Crit Care 2022;37:193-201.ArticlePubMedPMCPDF
  • 24. Xu J, Chen J, Cai H, Zheng X. Effect of timing of intubation on clinical outcomes of patients with septic shock: a retrospective cohort study. Sci Rep 2024;14:22128.ArticlePubMedPMCPDF
  • 25. Yang T, Shen Y, Park JG, Schulte PJ, Hanson AC, Herasevich V, et al. Outcome after intubation for septic shock with respiratory distress and hemodynamic compromise: an observational study. BMC Anesthesiol 2021;21:253.ArticlePubMedPMCPDF
  • 26. Mellado-Artigas R, Ferrando C, Martino F, Delbove A, Ferreyro BL, Darreau C, et al. Early intubation and patient-centered outcomes in septic shock: a secondary analysis of a prospective multicenter study. Crit Care 2022;26:163.ArticlePubMedPMCPDF
  • 27. Kim G, Oh DK, Lee SY, Park MH, Lim CM; Korean Sepsis Alliance (KSA) investigators. Impact of the timing of invasive mechanical ventilation in patients with sepsis: a multicenter cohort study. Crit Care 2024;28:297.ArticlePubMedPMCPDF
  • 28. Wang H, Zhao QY, Luo JC, Liu K, Yu SJ, Ma JF, et al. Early prediction of noninvasive ventilation failure after extubation: development and validation of a machine-learning model. BMC Pulm Med 2022;22:304.ArticlePubMedPMCPDF
  • 29. Fabregat A, Magret M, Ferré JA, Vernet A, Guasch N, Rodríguez A, et al. A machine learning decision-making tool for extubation in intensive care unit patients. Comput Methods Programs Biomed 2021;200:105869.ArticlePubMed
  • 30. Okada Y, Ning Y, Ong ME. Explainable artificial intelligence in emergency medicine: an overview. Clin Exp Emerg Med 2023;10:354-62.ArticlePubMedPMCPDF
  • 31. Gårdlund B, Dmitrieva NO, Pieper CF, Finfer S, Marshall JC, Taylor Thompson B. Six subphenotypes in septic shock: Latent class analysis of the PROWESS Shock study. J Crit Care 2018;47:70-9.ArticlePubMedPMC
  • 32. Duan J, Chen L, Liu X, Bozbay S, Liu Y, Wang K, et al. An updated HACOR score for predicting the failure of noninvasive ventilation: a multicenter prospective observational study. Crit Care 2022;26:196.ArticlePubMedPMCPDF

Figure & Data

References

    Citations

    Citations to this article as recorded by  

      • PubReader PubReader
      • ePub LinkePub Link
      • Cite
        CITE
        export Copy
        Close
        Download Citation
        Download a citation file in RIS format that can be imported by all major citation management software, including EndNote, ProCite, RefWorks, and Reference Manager.
        Format:
        • RIS — For EndNote, ProCite, RefWorks, and most other reference management software
        • BibTeX — For JabRef, BibDesk, and other BibTeX-specific software
        Include:
        • Citation for the content below
        Using machine learning techniques for early prediction of tracheal intubation in patients with septic shock: a multi-center study in Korea
        Close
      • XML DownloadXML Download
      Figure
      • 0
      • 1
      • 2
      • 3
      • 4
      Using machine learning techniques for early prediction of tracheal intubation in patients with septic shock: a multi-center study in Korea
      Image Image Image Image Image
      Figure 1. Study flowchart. KoSS: Korean Shock Society; ED: emergency department.
      Figure 2. Area under the receiver operating characteristics (AUROC) curve in relation to tracheal intubation within 24 hours after enrollment, evaluated on the test set.
      Figure 3. Area under the precision recall characteristics (AUPRC) curve in relation to tracheal intubation within 24 hours after enrollment, evaluated on the test set.
      Figure 4. Confusion matrix in relation to tracheal intubation within 24 hours after enrollment, evaluated on the test set.
      Figure 5. Shapley values for tracheal intubation within 24 hours. The top five features used by the extreme gradient boosting machine are lactate level after initial fluids, suspected lung infection, initial pH, Sequential Organ Failure Assessment (SOFA) score at enrollment, and respiratory rate (RR) at enrollment, with higher lactate levels after initial fluids, higher initial RR, and lower pH levels increasing the likelihood that early tracheal intubation will be applied. enroll: enrollment; HR: heart rate; DBP: diastolic blood pressure; pCO2: partial pressure of CO2; SBP: systolic blood pressure; BT: body temperature; RR: respiratory rate; SHAP: Shapley additive explanations.

      Figure 1.

      Figure 2.

      Figure 3.

      Figure 4.

      Figure 5.

      Using machine learning techniques for early prediction of tracheal intubation in patients with septic shock: a multi-center study in Korea
      Variable Overall Intubation (–) Intubation (+) P-value
      Number 4,762 3,276 1,486 -
      Age (yr) 70 (61–78) 69 (60–78) 70 (62–78) 0.017
      Sex 2,887 (60.6) 1,923 (58.7) 964 (64.9) <0.001
       Male 1,875 (39.4) 1,353 (41.3) 522 (35.1)
      Vital sign
       SBP at enroll 84.0 (74.0–97.0) 84.0 (75.0–94.0) 86.0 (74.0–106.0) <0.001
       DBP at enroll 50.0 (43.0–59.0) 50.0 (43.0–57.0) 52.0 (44.0–62.8) <0.001
       HR at enroll 107.5 (93.0–123.0) 105.0 (91.0–119.0) 114.0 (98.0–130.0) <0.001
       RR at enroll 22.0 (20.0–26.0) 20.0 (18.0–24.0) 24.0 (20.0–30.0) <0.001
       BT at enroll 37.4 (36.6–38.2) 37.5 (36.7–38.3) 37.2 (36.4–38.1) <0.001
       Altered mental status No 3,660 (76.9) 2,713 (82.8) 947 (63.7) <0.001
      Yes 1,102 (23.1) 563 (17.2) 539 (36.3)
       Vasopressor use No 56 (1.2) 32 (1.0) 24 (1.6) 0.080
      Yes 4,706 (98.8) 3244 (99.0) 1462 (98.4)
       Refractory hypotension No 896 (18.8) 499 (15.2) 397 (26.7) <0.001
      Yes 3,866 (81.2) 2,777 (84.8) 1,089 (73.3)
       qSOFA ≥2 at enroll No 2,345 (49.2) 1,804 (55.1) 541 (36.4) <0.001
      Yes 2,417 (50.8) 1,472 (44.9) 945 (63.6)
      Arterial blood gas analysis
       Initial pH 7.4 (7.3–7.5) 7.4 (7.4–7.5) 7.4 (7.3–7.4) <0.001
       Initial pCO2 27.3 (23.0–32.0) 27.1 (23.1–31.1) 28.0 (22.6–35.0) <0.001
       Initial HCO3– 17.7 (14.3–20.9) 18.2 (15.2–21.1) 16.0 (12.4–19.8) <0.001
       Lactate at enroll 4.5 (3.1–6.8) 4.2 (2.8–5.8) 5.9 (4.2–8.8) <0.001
       Lactate after initial fluid 4.0 (2.8–6.1) 3.5 (2.6–5.2) 5.5 (3.5–8.3) <0.001
      Site of infection
       Suspected lung infection No 3,433 (72.1) 2,660 (81.2) 773 (52.0) <0.001
      Yes 1,329 (27.9) 616 (18.8) 713 (48.0)
       Suspected urinary tract infection No 3,635 (76.3) 2,376 (72.5) 1,259 (84.7) <0.001
      Yes 1,127 (23.7) 900 (27.5) 227 (15.3)
       Suspected gastrointestinal infection No 3,744 (78.6) 2,637 (80.5) 1,107 (74.5) <0.001
      Yes 1,018 (21.4) 639 (19.5) 379 (25.5)
       Suspected hepatobiliary infection No 3,610 (75.8) 2,313 (70.6) 1,297 (87.3) <0.001
      Yes 1,152 (24.2) 963 (29.4) 189 (12.7)
       Suspected bone or soft tissue infection No 4,587 (96.3) 3,171 (96.8) 1,416 (95.3) 0.013
      Yes 175 (3.7) 105 (3.2) 70 (4.7)
       Suspected central nervous system infection No 4,744 (99.6) 3,269 (99.8) 1,475 (99.3) 0.013
      Yes 18 (0.4) 7 (0.2) 11 (0.7)
       Suspected catheter related infection No 4,708 (98.9) 3,238 (98.8) 1,470 (98.9) 0.917
      Yes 54 (1.1) 38 (1.2) 16 (1.1)
       Suspected bloodstream infection No 4,572 (96.0) 3,141 (95.9) 1,431 (96.3) 0.545
      Yes 190 (4.0) 135 (4.1) 55 (3.7)
      Past medical history
       Hypertension No 2,680 (56.3) 1,831 (55.9) 849 (57.1) 0.442
      Yes 2,082 (43.7) 1,445 (44.1) 637 (42.9)
       Diabetes mellitus No 3,027 (63.6) 2,067 (63.1) 960 (64.6) 0.332
      Yes 1,735 (36.4) 1,209 (36.9) 526 (35.4)
       Cardiac disease No 4,123 (86.6) 2,858 (87.2) 1,265 (85.1) 0.053
      Yes 639 (13.4) 418 (12.8) 221 (14.9)
       Known cerebrovascular accident No 4,202 (88.2) 2,916 (89.0) 1,286 (86.5) 0.016
      Yes 560 (11.8) 360 (11.0) 200 (13.5)
       Known chronic lung disease No 4,453 (93.5) 3,094 (94.4) 1,359 (91.5) <0.001
      Yes 309 (6.5) 182 (5.6) 127 (8.5)
       Known chronic renal disease No 4,330 (90.9) 3,006 (91.8) 1,324 (89.1) 0.004
      Yes 432 (9.1) 270 (8.2) 162 (10.9)
       Known chronic liver disease No 4,166 (87.5) 2,845 (86.8) 1,321 (88.9) 0.053
      Yes 596 (12.5) 431 (13.2) 165 (11.1)
      Miscellaneous
       Referral from nursing center No 4,337 (91.1) 3,014 (92.0) 1,323 (89.0) 0.001
      Yes 425 (8.9) 262 (8.0) 163 (11.0)
       SOFA score at enroll 7.0 (5.0–9.0) 6.0 (4.0–8.0) 8.0 (6.0–11.0) <0.001
       SOFA score (max over 24 hours) 9.0 (7.0–12.0) 8.0 (6.0–11.0) 12.0 (9.0–15.0) <0.001
       APACHE II score 22.0 (16.0–28.0) 20.0 (15.0–25.0) 28.0 (22.0–36.0) <0.001
       28-Day outcome
        Death 1,188 (24.9) 478 (14.6) 710 (47.8) <0.001
        Survival 3,416 (71.7) 2,663 (81.3) 753 (50.7)
        NA 158 (3.3) 135 (4.1) 23 (1.5)
      Variable Overall Intubation (–) Intubation (+) P-value
      Number 3,571 2,457 1,114 -
      Age (yr) 70 (61–78) 69 (61–78) 70 (62–78) 0.135
      Sex 2,157 (60.4) 1,453 (59.1) 704 (63.2) 0.024
       Male 1,414 (39.6) 1,004 (40.9) 410 (36.8)
      Vital sign
       SBP at enroll 84.0 (75.0–98.0) 84.0 (75.0–94.0) 86.0 (74.0–106.8) <0.001
       DBP at enroll 50.0 (43.0–59.0) 50.0 (43.0–57.0) 52.0 (43.0–63.0) <0.001
       HR at enroll 108.0 (93.0–123.0) 105.0 (91.0–119.0) 114.0 (98.0–130.0) <0.001
       RR at enroll 22.0 (20.0–26.0) 20.0 (18.0–24.0) 24.0 (20.0–30.0) <0.001
       BT at enroll 37.4 (36.6–38.2) 37.5 (36.7–38.3) 37.1 (36.4–38.1) <0.001
       Altered mental status No 2,744 (76.8) 2,044 (83.2) 700 (62.8) <0.001
      Yes 827 (23.2) 413 (16.8) 414 (37.2)
       Vasopressor use No 41 (1.1) 22 (0.9) 19 (1.7) 0.053
      Yes 3,530 (98.9) 2,435 (99.1) 1,095 (98.3)
       Refractory hypotension No 681 (19.1) 384 (15.6) 297 (26.7) <0.001
      Yes 2,890 (80.9) 2,073 (84.4) 817 (73.3)
       qSOFA ≥2 at enroll No 1,750 (49.0) 1,341 (54.6) 409 (36.7) <0.001
      Yes 1,821 (51.0) 1,116 (45.4) 705 (63.3)
      Arterial blood gas analysis
       Initial pH 7.4 (7.3–7.5) 7.4 (7.4–7.5) 7.4 (7.3–7.4) <0.001
       Initial pCO2 27.2 (23.0–32.0) 27.0 (23.0–31.0) 28.0 (22.8–34.9) <0.001
       Initial HCO3– 17.7 (14.2–20.8) 18.1 (15.1–21.1) 15.9 (12.4–19.8) <0.001
       Lactate at enroll 4.6 (3.1–6.8) 4.2 (2.8–5.7) 6.1 (4.3–9.0) <0.001
       Lactate after initial fluid 4.0 (2.8–6.1) 3.5 (2.6–5.1) 5.6 (3.6–8.5) <0.001
      Site of infection
       Suspected lung infection No 2,576 (72.1) 1,999 (81.4) 577 (51.8) <0.001
      Yes 995 (27.9) 458 (18.6) 537 (48.2)
       Suspected urinary tract infection No 2,721 (76.2) 1,779 (72.4) 942 (84.6) <0.001
      Yes 850 (23.8) 678 (27.6) 172 (15.4)
       Suspected gastrointestinal infection No 2,800 (78.4) 1,973 (80.3) 827 (74.2) <0.001
      Yes 771 (21.6) 484 (19.7) 287 (25.8)
       Suspected hepatobiliary infection No 2,729 (76.4) 1,749 (71.2) 980 (88.0) <0.001
      Yes 842 (23.6) 708 (28.8) 134 (12.0)
       Suspected bone or soft tissue infection No 3,434 (96.2) 2,379 (96.8) 1,055 (94.7) 0.003
      Yes 137 (3.8) 78 (3.2) 59 (5.3)
       Suspected central nervous system infection No 3,559 (99.7) 2,452 (99.8) 1,107 (99.4) 0.058
      Yes 12 (0.3) 5 (0.2) 7 (0.6)
       Suspected catheter related infection No 3,536 (99.0) 2,435 (99.1) 1,101 (98.8) 0.562
      Yes 35 (1.0) 22 (0.9) 13 (1.2)
       Suspected bloodstream infection No 3,417 (95.7) 2,348 (95.6) 1,069 (96.0) 0.651
      Yes 154 (4.3) 109 (4.4) 45 (4.0)
      Past medical history
       Hypertension No 2,038 (57.1) 1,396 (56.8) 642 (57.6) 0.676
      Yes 1,533 (42.9) 1,061 (43.2) 472 (42.4)
       Diabetes mellitus No 2,290 (64.1) 1,567 (63.8) 723 (64.9) 0.541
      Yes 1,281 (35.9) 890 (36.2) 391 (35.1)
       Cardiac disease No 3,100 (86.8) 2,156 (87.7) 944 (84.7) 0.016
      Yes 471 (13.2) 301 (12.3) 170 (15.3)
       Known cerebrovascular accident No 3,154 (88.3) 2,193 (89.3) 961 (86.3) 0.012
      Yes 417 (11.7) 264 (10.7) 153 (13.7)
       Known chronic lung disease No 3,336 (93.4) 2,313 (94.1) 1,023 (91.8) 0.012
      Yes 235 (6.6) 144 (5.9) 91 (8.2)
       Known chronic renal disease No 3,246 (90.9) 2,252 (91.7) 994 (89.2) 0.023
      Yes 325 (9.1) 205 (8.3) 120 (10.8)
       Known chronic liver disease No 3,126 (87.5) 2,135 (86.9) 991 (89.0) 0.094
      Yes 445 (12.5) 322 (13.1) 123 (11.0)
      Miscellaneous
       Referral from nursing center 3,252 (91.1) 2,260 (92.0) 992 (89.0) 0.005
      319 (8.9) 197 (8.0) 122 (11.0)
       SOFA score at enroll 6.0 (5.0–9.0) 6.0 (4.0–8.0) 8.0 (6.0–11.0) <0.001
       SOFA score (max over 24 hours) 9.0 (7.0–12.0) 8.0 (6.0–10.0) 12.0 (9.0–15.0) <0.001
       APACHE II score 22.0 (16.0–29.0) 20.0 (15.0–25.0) 28.0 (22.0–36.0) <0.001
       28-Day outcome <0.001
        Death 886 (24.8) 359 (14.6) 527 (47.3)
        Survival 2,566 (71.9) 1,996 (81.2) 570 (51.2)
        NA 119 (3.3) 102 (4.2) 17 (1.5)
      Variable Overall Intubation (–) Intubation (+) P-value
      Number 1,191 819 372
      Age (yr) 70 (61–78) 69 (60–77) 71 (62–79) 0.030
      Sex 730 (61.3) 470 (57.4) 260 (69.9) <0.001
       Male 461 (38.7) 349 (42.6) 112 (30.1)
      Vital sign
       SBP at enroll 85.0 (74.0–96.0) 84.0 (74.0–93.0) 86.5 (72.8–103.0) 0.002
       DBP at enroll 50.0 (43.0–58.0) 50.0 (42.0–56.0) 53.0 (44.0–62.0) <0.001
       HR at enroll 107.0 (93.0–123.0) 104.0 (91.0–119.0) 115.0 (100.0–130.2) <0.001
       RR at enroll 22.0 (19.0–26.0) 20.0 (18.0–24.0) 24.0 (20.0–30.0) <0.001
       BT at enroll 37.4 (36.6–38.2) 37.5 (36.7–38.2) 37.3 (36.5–38.1) 0.014
       Altered mental status No 916 (76.9) 669 (81.7) 247 (66.4) <0.001
      Yes 275 (23.1) 150 (18.3) 125 (33.6)
       Vasopressor use No 15 (1.3) 10 (1.2) 5 (1.3) 1.000
      Yes 1,176 (98.7) 809 (98.8) 367 (98.7)
       Refractory hypotension No 215 (18.1) 115 (14.0) 100 (26.9) <0.001
      Yes 976 (81.9) 704 (86.0) 272 (73.1)
       qSOFA ≥2 at enroll No 595 (50.0) 463 (56.5) 132 (35.5) <0.001
      Yes 596 (50.0) 356 (43.5) 240 (64.5)
      Arterial blood gas analysis
       Initial pH 7.4 (7.4–7.5) 7.4 (7.4–7.5) 7.4(7.3–7.4) <0.001
       Initial pCO2 27.7 (23.3–32.0) 27.3 (23.7–31.4) 28.4 (22.5–35.0) 0.080
       Initial HCO3– 18.0 (14.6–21.0) 18.5 (15.4–21.2) 16.5 (12.5–20.1) <0.001
       Lactate at enroll 4.5 (3.1–6.5) 4.2 (2.8–5.9) 5.6(4.0–8.1) <0.001
       Lactate after initial fluid 3.9 (2.7–6.0) 3.5 (2.6–5.2) 5.0(3.4–7.8) <0.001
      Site of infection
       Suspected lung infection No 857 (72.0) 661 (80.7) 196 (52.7) <0.001
      Yes 334 (28.0) 158 (19.3) 176 (47.3)
       Suspected urinary tract infection No 914 (76.7) 597 (72.9) 317 (85.2) <0.001
      Yes 277 (23.3) 222 (27.1) 55 (14.8)
       Suspected gastrointestinal infection No 944 (79.3) 664 (81.1) 280 (75.3) 0.027
      Yes 247 (20.7) 155 (18.9) 92 (24.7)
       Suspected hepatobiliary infection No 881 (74.0) 564 (68.9) 317 (85.2) <0.001
      Yes 310 (26.0) 255 (31.1) 55 (14.8)
       Suspected bone or soft tissue infection No 1,153 (96.8) 792 (96.7) 361 (97.0) 0.896
      Yes 38 (3.2) 27 (3.3) 11 (3.0)
       Suspected central nervous system infection No 1,185 (99.5) 817 (99.8) 368 (98.9) 0.080
      Yes 6 (0.5) 2 (0.2) 4 (1.1)
       Suspected catheter related infection No 1,172 (98.4) 803 (98.0) 369 (99.2) 0.224
      Yes 19 (1.6) 16 (2.0) 3 (0.8)
       Suspected bloodstream infection No 1,155 (97.0) 793 (96.8) 362 (97.3) 0.786
      Yes 36 (3.0) 26 (3.2) 10 (2.7)
      Past medical history
       Hypertension No 642 (53.9) 435 (53.1) 207 (55.6) 0.454
      Yes 549 (46.1) 384 (46.9) 165 (44.4)
       Diabetes mellitus No 737 (61.9) 500 (61.1) 237 (63.7) 0.417
      Yes 454 (38.1) 319 (38.9) 135 (36.3)
       Cardiac disease No 1,023 (85.9) 702 (85.7) 321 (86.3) 0.861
      Yes 168 (14.1) 117 (14.3) 51 (13.7)
       Known cerebrovascular accident No 1,048 (88.0) 723 (88.3) 325 (87.4) 0.724
      Yes 143 (12.0) 96 (11.7) 47 (12.6)
       Known chronic lung disease No 1,117 (93.8) 781 (95.4) 336 (90.3) 0.001
      Yes 74 (6.2) 38 (4.6) 36 (9.7)
       Known chronic renal disease No 1,084 (91.0) 754 (92.1) 330 (88.7) 0.077
      Yes 107 (9.0) 65 (7.9) 42 (11.3)
       Known chronic liver disease No 1,040 (87.3) 710 (86.7) 330 (88.7) 0.381
      Yes 151 (12.7) 109 (13.3) 42 (11.3)
      Miscellaneous
       Referral from nursing center No 1,085 (91.1) 754 (92.1) 331 (89.0) 0.105
      Yes 106 (8.9) 65 (7.9) 41 (11.0)
       SOFA score at enroll 7.0 (5.0–9.0) 6.0 (5.0–8.0) 8.0 (5.8–11.0) <0.001
       SOFA score (max over 24 hours) 9.0 (7.0–12.0) 9.0 (7.0–11.0) 12.0 (9.0–15.0) <0.001
       APACHE II score 21.0 (16.0–28.0) 19.0 (15.0–24.0) 27.0 (20.0–36.0) <0.001
       28-Day outcome <0.001
        Death 302 (25.4) 119 (14.5) 183 (49.2)
        Survival 850 (71.4) 667 (81.4) 183 (49.2)
        NA 39 (3.3) 33 (4.0) 6 (1.6)
      Metrics Value (95% CI)
      AUROC 0.829 (0.801–0.878)
      AUPRC 0.702 (0.676–0.728)
      Sensitivity 0.740 (0.695–0.784)
      Specificity 0.763 (0.734–0.792)
      Positive predictive value 0.586 (0.542–0.631)
      Negative predictive value 0.866 (0.841–0.891)
      F1 score 0.654 (0.627–0.681)
      Table 1. Demographics and baseline features of the entire set on intubation within 24 hour window

      Values are presented as median (interquartile range) or number (%).

      SBP: systolic blood pressure; DBP: diastolic blood pressure; HR: heart rate; RR: respiratory rate; BT: body temperature; qSOFA: quick Sequential Organ Failure Assessment; APACHE: Acute Physiology and Chronic Health Evaluation; NA: not available.

      Table 2. Demographics and baseline features of the train set on intubation within 24 hour window

      Values are presented as median (interquartile range) or number (%).

      SBP: systolic blood pressure; DBP: diastolic blood pressure; HR: heart rate; RR: respiratory rate; BT: body temperature; qSOFA: quick Sequential Organ Failure Assessment; APACHE: Acute Physiology and Chronic Health Evaluation; NA: not available.

      Table 3. Demographics and baseline features of the test set on intubation within 24 hour window

      Values are presented as median (interquartile range) or number (%).

      SBP: systolic blood pressure; DBP: diastolic blood pressure; HR: heart rate; RR: respiratory rate; BT: body temperature; qSOFA: quick Sequential Organ Failure Assessment; APACHE: Acute Physiology and Chronic Health Evaluation; NA: not available.

      Table 4. Performance metrics regarding intubation within 24 hours after enroll, evaluated on the test set

      AUROC: area under the receiver operating characteristic curve; AUPRC: area under the precision-recall curve.

      Table 1.

      Table 2.

      Table 3.

      Table 4.

      ACC : Acute and Critical Care
      © The Korean Society of Critical Care Medicine.
      TOP