Taeyong Sim, Eun Young Cho, Ji-hyun Kim, Kyung Hyun Lee, Kwang Joon Kim, Sangchul Hahn, Eun Yeong Ha, Eunkyeong Yun, In-Cheol Kim, Sun Hyo Park, Chi-Heum Cho, Gyeong Im Yu, Byung Eun Ahn, Yeeun Jeong, Joo-Yun Won, Hochan Cho, Ki-Byung Lee
Acute Crit Care. 2025;40(2):197-208. Published online May 30, 2025
Background Acute deterioration of patients in general wards often leads to major adverse events (MAEs), including unplanned intensive care unit transfers, cardiac arrest, or death. Traditional early warning scores (EWSs) have shown limited predictive accuracy, with frequent false positives. We conducted a prospective observational external validation study of an artificial intelligence (AI)-based EWS, the VitalCare - Major Adverse Event Score (VC-MAES), at a tertiary medical center in the Republic of Korea.
Methods Adult patients from general wards, including internal medicine (IM) and obstetrics and gynecology (OBGYN)—the latter were rarely investigated in prior AI-based EWS studies—were included. The VC-MAES predictions were compared with National Early Warning Score (NEWS) and Modified Early Warning Score (MEWS) predictions using the area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), and logistic regression for baseline EWS values. False-positives per true positive (FPpTP) were assessed based on the power threshold.
Results Of 6,039 encounters, 217 (3.6%) had MAEs (IM: 9.5%, OBGYN: 0.26%). Six hours prior to MAEs, the VC-MAES achieved an AUROC of 0.918 and an AUPRC of 0.352, including the OBGYN subgroup (AUROC, 0.964; AUPRC, 0.388), outperforming the NEWS (0.797 and 0.124) and MEWS (0.722 and 0.079). The FPpTP was reduced by up to 71%. Baseline VC-MAES was strongly associated with MAEs (P<0.001).
Conclusions The VC-MAES significantly outperformed traditional EWSs in predicting adverse events in general ward patients. The robust performance and lower FPpTP suggest that broader adoption of the VC-MAES may improve clinical efficiency and resource allocation in general wards.
Citations
Citations to this article as recorded by
Clinical Context Is More Important than Data Quantity to the Performance of an Artificial Intelligence-Based Early Warning System Taeyong Sim, Eunyoung Cho, Jihyun Kim, Ho Gwan Kim, Soo-Jeong Kim Journal of Clinical Medicine.2025; 14(13): 4444. CrossRef
Artificial intelligence and machine learning approaches for patient safety in complex surgery: a review Mohamed Mustaf Ahmed, Zhinya Kawa Othman, Uthman Okikiola Adebayo, Omar Kasimieh, Olalekan John Okesanya, Shuaibu Saidu Musa, Francesco Branda, Victor C. Cañezo , Edgar G. Cue, Don Eliseo Lucero Prisno III Patient Safety in Surgery.2025;[Epub] CrossRef
Background Early detection of critical events in hospitalized patients improves clinical outcomes and reduces mortality rates. Traditional early warning score systems, such as the National Early Warning Score 2 (NEWS2), effectively identify at-risk patients. Integrating artificial intelligence (AI) could enhance the predictive accuracy and operational efficiency of such systems. The study describes the development and implementation of an AI-enhanced early warning system based on a modified NEWS2 scale with laboratory parameters (mNEWS2-Lab) and evaluates its ability to improve patient safety in hospital wards.
Methods For this retrospective cohort study of 3,790 adults admitted to hospital wards, data were collected before and after implementing the mNEWS2-Lab protocol with and without AI enhancement. The study used a multivariate prediction model with statistical analyses such as Fisher's chi-square test, relative risk (RR), RR reduction, and various AI models (logistic regression, decision trees, neural networks). The economic cost of the intervention was also analyzed.
Results The mNEWS2-Lab reduced critical events from 6.15% to 2.15% (RR, 0.35; P<0.001), representing a 65% risk reduction. AI integration further reduced events to 1.59% (RR, 0.26; P<0.001) indicating a 10% additional risk reduction and enhancing early warning accuracy by 15%. The intervention was cost-effective, resulting in substantial savings by reducing critical events in hospitalized patients.
Conclusions The mNEWS2-Lab scale, particularly when integrated with AI models, is a powerful and cost-effective tool for the early detection and prevention of critical events in hospitalized patients.
Citations
Citations to this article as recorded by
Bridging the last mile in AI-driven patient safety: the missing link of trainee readiness Rafay Ullah Khan, Nabiah Shakeel, Mohammed Hammad Jaber Amin Annals of Medicine & Surgery.2026; 88(1): 1104. CrossRef
Artificial intelligence and pediatric acute kidney injury: a mini-review and white paper Jieji Hu, Rupesh Raina Frontiers in Nephrology.2025;[Epub] CrossRef
Background Various rapid response systems have been developed to detect clinical deterioration in patients. Few studies have evaluated single-parameter systems in children compared to scoring systems. Therefore, in this study we evaluated a single-parameter system called the acute response system (ARS).
Methods This retrospective study was performed at a tertiary children’s hospital. Patients under 18 years old admitted from January 2012 to August 2023 were enrolled. ARS parameters such as systolic blood pressure, heart rate, respiratory rate, oxygen saturation, and whether the ARS was activated were collected. We divided patients into two groups according to activation status and then compared the occurrence of critical events (cardiopulmonary resuscitation or unexpected intensive care unit admission). We evaluated the ability of ARS to predict critical events and calculated compliance. We also analyzed the correlation between each parameter that activates ARS and critical events.
Results The critical events prediction performance of ARS has a specificity of 98.5%, a sensitivity of 24.0%, a negative predictive value of 99.6%, and a positive predictive value of 8.1%. The compliance rate was 15.6%. Statistically significant increases in the risk of critical events were observed for all abnormal criteria except low heart rate. There was no significant difference in the incidence of critical events.
Conclusions ARS, a single parameter system, had good specificity and negative predictive value for predicting critical events; however, sensitivity and positive predictive value were not good, and medical staff compliance was poor.
Citations
Citations to this article as recorded by
Impact of an independent pediatric rapid response team on pediatric patient safety Jin Hwan Lyu, Soo Keun Lee, Min Joo Kang, Sumin Lee, Mireu Park, Soo Yeon Kim, Kyung Won Kim, Jong Deok Kim Archives of Pediatric Critical Care.2024; 2(2): 91. CrossRef
Yunseob Shin, Kyung-Jae Cho, Yeha Lee, Yu Hyeon Choi, Jae Hwa Jung, Soo Yeon Kim, Yeo Hyang Kim, Young A Kim, Joongbum Cho, Seong Jong Park, Won Kyoung Jhang
Acute Crit Care. 2022;37(4):654-666. Published online October 26, 2022
Background Early recognition of deterioration events is crucial to improve clinical outcomes. For this purpose, we developed a deep-learning-based pediatric early-warning system (pDEWS) and aimed to validate its clinical performance.
Methods This is a retrospective multicenter cohort study including five tertiary-care academic children’s hospitals. All pediatric patients younger than 19 years admitted to the general ward from January 2019 to December 2019 were included. Using patient electronic medical records, we evaluated the clinical performance of the pDEWS for identifying deterioration events defined as in-hospital cardiac arrest (IHCA) and unexpected general ward-to-pediatric intensive care unit transfer (UIT) within 24 hours before event occurrence. We also compared pDEWS performance to those of the modified pediatric early-warning score (PEWS) and prediction models using logistic regression (LR) and random forest (RF).
Results The study population consisted of 28,758 patients with 34 cases of IHCA and 291 cases of UIT. pDEWS showed better performance for predicting deterioration events with a larger area under the receiver operating characteristic curve, fewer false alarms, a lower mean alarm count per day, and a smaller number of cases needed to examine than the modified PEWS, LR, or RF models regardless of site, event occurrence time, age group, or sex.
Conclusions The pDEWS outperformed modified PEWS, LR, and RF models for early and accurate prediction of deterioration events regardless of clinical situation. This study demonstrated the potential of pDEWS as an efficient screening tool for efferent operation of rapid response teams.
Citations
Citations to this article as recorded by
A Realist Evaluation of a Rapid Response System for Mental State Deterioration in Acute Hospital Settings Tendayi Bruce Dziruni, Alison M. Hutchinson, Sandra Keppich‐Arnold, Tracey Bucknall Journal of Advanced Nursing.2026;[Epub] CrossRef
Implementation of the bedside paediatric early warning system, its sustainability in clinical practice and patient outcomes: a quality improvement initiative Ruqiah Ali AlZaher, Syed Jamil, Iris Murabi, Eidah Ahmari BMJ Open Quality.2025; 14(2): e002454. CrossRef
AI-driven transcriptomic biomarker discovery for early identification of pediatric deterioration in Acute Care Qing Wang, Lina Sun, Wei Meng, Chen Chen SLAS Technology.2025; 35: 100357. CrossRef
Predicting cardiac arrest after neonatal cardiac surgery Alexis L. Benscoter, Mark A. Law, Santiago Borasino, A. K. M. Fazlur Rahman, Jeffrey A. Alten, Mihir R. Atreya Intensive Care Medicine – Paediatric and Neonatal.2024;[Epub] CrossRef
Volumetric regional MRI and neuropsychological predictors of motor task variability in cognitively unimpaired, Mild Cognitive Impairment, and probable Alzheimer's disease older adults Michael Malek-Ahmadi, Kevin Duff, Kewei Chen, Yi Su, Jace B. King, Vincent Koppelmans, Sydney Y. Schaefer Experimental Gerontology.2023; 173: 112087. CrossRef
Predicting sepsis using deep learning across international sites: a retrospective development and validation study Michael Moor, Nicolas Bennett, Drago Plečko, Max Horn, Bastian Rieck, Nicolai Meinshausen, Peter Bühlmann, Karsten Borgwardt eClinicalMedicine.2023; 62: 102124. CrossRef
A model study for the classification of high-risk groups for cardiac arrest in general ward patients using simulation techniques Seok Young Song, Won-Kee Choi, Sanggyu Kwak Medicine.2023; 102(37): e35057. CrossRef
An advanced pediatric early warning system: a reliable sentinel, not annoying extra work Young Joo Han Acute and Critical Care.2022; 37(4): 667. CrossRef