Deep Learning in the Medical Domain: Predicting Cardiac Arrest Using Deep Learning

Youngnam Lee; Joon-myoung Kwon; Yeha Lee; Hyunho Park; Hugh Cho; Jinsik Park

doi:10.4266/acc.2018.00290

Articles

Page Path: HOME > Acute Crit Care > Volume 33(3); 2018 > Article

Review Article Deep Learning in the Medical Domain: Predicting Cardiac Arrest Using Deep Learning: Youngnam Lee¹, Joon-myoung Kwon², Yeha Lee¹, Hyunho Park¹, Hugh Cho¹, Jinsik Park²; Acute and Critical Care 2018;33(3):117-120.
DOI: https://doi.org/10.4266/acc.2018.00290
Published online: August 31, 2018

¹VUNO, Seoul, Korea

²Department of Emergency Medicine, Mediplex Sejong Hospital, Incheon, Korea

Corresponding author Joon-myoung Kwon Department of Emergency Medicine, Mediplex Sejong Hospital, 20 Gyeyangmunhwa-ro, Gyeyang-gu, Incheon 21080, Korea Tel: +82-32-240-8000 Fax: +82-32-240-8999 E-mail: kwonjm@sejongh.co.kr

*The first two authors contributed equally to this study.

• Received: August 23, 2018 • Accepted: August 24, 2018

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

13,643 Views
543 Download
18 Web of Science
21 Crossref
27 Scopus

Full Article

Download PDF

Abstract
INTRODUCTION
MACHINE AND DEEP LEARNING
DEEP LEARNING BASED EARLY WARNING SCORE
CONCLUSION
KEY MESSAGES
NOTES
References

Abstract

With the wider adoption of electronic health records, the rapid response team initially believed that mortalities could be significantly reduced but due to low accuracy and false alarms, the healthcare system is currently fraught with many challenges. Rule-based methods (e.g., Modified Early Warning Score) and machine learning (e.g., random forest) were proposed as a solution but not effective. In this article, we introduce the DeepEWS (Deep learning based Early Warning Score), which is based on a novel deep learning algorithm. Relative to the standard of care and current solutions in the marketplace, there is high accuracy, and in the clinical setting even when we consider the number of alarms, the accuracy levels are superior.
Keywords: artificial intelligence; cardiac arrest; deep learning; rapid response team

INTRODUCTION

In 2009, the Health Information Technology for Economic and Clinical Health Act was implemented to roll out electronic health records (EHRs) and it was rapidly adopted. A total of 30 million dollars was invested into this policy that incorporated incentives and penalties for widely adopting EHRs. EHRs include demographic information, present and past diagnoses, laboratory results, vital sign, and clinical notes to efficiently manage patients in the clinical setting and can leverage clinical informatics to predict diseases, patient length of stay, and readmission rates.

Due to the variety of clinical information, providers can quickly access through EHRs; a rapid response team (RRT) can effectively reduce the number of in-hospital cardiac arrest and mortalities. However, with the current standard of care (Modified Early Warning Score [MEWS]), the RRT is suffering from low accuracy and false alarms. In terms of the accuracy, the RRT could only detect 30% of unplanned intensive care unit admissions, and the MEWS that the RRT team employs does not even reach an area under the receiver operating characteristic (AUROC) of 0.8 [1-3]. Previous algorithms have focused on accuracy; however, in the clinical environment false alarms are the most important factor when evaluating an algorithm [4,5]. Whenever alarms sound, in real time RRT reconfirm patients and determine intervention. Therefore, if there are too many false alarms, an unnecessary amount of time and costs are wasted.

To overcome the limits of MEWS, many traditional machine learning algorithms have been incorporated but still suffer from low accuracy and false alarms. In this article, we introduce the Deep learning based Early Warning Score (DeepEWS), that is based on a novel deep learning algorithm [6]. Deep learning has demonstrated state of the art results for computer vision, machine translation and has demonstrated many successful results [7-9]. In the medical domain, deep learning has been limited to being applied to medical images. However, recently there have been new applications to EHR and biosignal research.

DeepEWS was compared to rule-based and the most promising traditional machine learning algorithms in the clinical setting. There were many different scenarios on which the results were compared and particularly, the sensitivity versus the number of alarms effectively demonstrated the superiority in accuracy and ease of clinical implementation for DeepEWS.

MACHINE AND DEEP LEARNING

Both machine learning and deep learning analyze data through self-learning to solve the task or problem at hand (i.e., predict, classify). However, machine learning requires feature engineering, whereas deep learning does not, and this is the essence of why deep learning provides higher accuracy [10]. Feature engineering strives to solve the problem in the model through pattern recognition by sensitizing various scenarios. In this process, domain knowledge and expertise can translate to significantly different results. However, deep learning uses deep neural networks to find these features automatically. As such, even without domain knowledge and specialist expertise, the model can self-learn. The deeper the neural network, the more diverse features are found, and this translates to higher performance and accuracy in results.

DEEP LEARNING BASED EARLY WARNING SCORE

In-hospital cardiac arrests alongside with sepsis is a major factor that determines patient safety. In the United States, every year there are 20,900 cardiac arrest patients, and globally the survival rate for cardiac arrest patients is below 20%. It is well documented in the literature, that 80% of cardiac arrest patients experience deterioration in health, 8 hours before the cardiac arrest event, and there have been many attempts to predict this onset. However, the current standard of care suffers from low accuracy and false alarms that renders the clinical implementation and practical use of these technologies challenging.

DeepEWS solved two major shortcomings that existing algorithms suffer from (low accuracy and false alarms) and provided clinically viable solutions. DeepEWS uses all four vital signs measured for 8 hours that can be easily measured in a hospital setting (systolic blood pressure, body temperature, heart rate, and respiratory rate). The details of the method can be confirmed in [6]. As shown in Figure 1, DeepEWS can measure real-time cardiac arrest risks (within 24 hours of the cardiac arrest event) from vital signs in Electronic Medical Records.

The study period of this paper was from June 2010 to July 2017, with 56,076 patients were admitted to the Mediplex Sejong Hospital, and 415 first cardiac arrest occurred. Four forty-eight patients were excluded because they were admitted or discharged outside the study period.

There are two significant results that the study demonstrated: the accuracy level and applicable in a clinical environment. Figure 2 represents the AUROC for DeepEWS and competing algorithms. DeepEWS shows an AUROC of 0.850 relative to MEWS’s 0.603, a 40.9% improvement in accuracy; (currently, the second version of DeepEWS recorded an AUROC of 0.911, a vast improvement from its previous version). Relative to algorithms used in previous research such as random forest (0.780) and logistic regression (0.613), the accuracy levels are markedly higher as well.

Figure 3 shows the sensitivity relative to the number of alarms and demonstrates how effective DeepEWS is in the clinical environment by maintaining high levels of accuracy even at the low number of alarms. In the real-world setting, an RRT is required to cover many patients so keeping a high level of accuracy with a small number of alarms is quintessential. We can see that at points of the lower number of alarms the accuracy level is significantly higher for DeepEWS relative to other competing algorithms. DeepEWS is more accurate with a lower number of alarms. DeepEWS is 42.7% accurate when the number of alarms is equal to 40 (point B); MEWS is 22.7% accurate when the number of alarms is equal to 110 (point C).

CONCLUSION

This study has two significant implications. First, the first study using deep learning has significantly improved accuracy. Second, it has been shown that DEWS is applicable in a clinical environment by comparing the accuracy according to the number of alarms. A large amount of false alarms is the factor for why MEWS and traditional machine learning algorithms could not be adopted in the clinical environment. DeepEWS predicts cardiac arrest with high accuracy and low false alarms. There are only four vital signs that are used as inputs that can easily be obtained in any clinical environment and augmenting these results with laboratory results, and clinical notes could enhance the accuracy even further.

KEY MESSAGES

▪ We introduce the DeepEWS (Deep learning based Early Warning Score), which is based on a novel deep learning algorithm.

▪ Relative to the standard of care and current solutions in the marketplace, there is high accuracy and low false alarms.

NOTES

Youngnam Lee, Joon-myoung Kwon, Yeha Lee, and Jinsik Park were involved in developing DEWS (Deep learning based Early Warning Score). VUNO provided support in the form of salaries for authors (Youngnam Lee, Yeha Lee, Hyunho Park, and Hugh Cho), but did not provide funding and did not have any additional role. No other conflict of interest relevant to this article was reported.

Figure 1.

DeepEWS (Deep learning based Early Warning Score) operating on data from Electronic Medical Records. MEWS: Modified Early Warning Score; NIBP(S): non-invasive blood pressure (systolic); NIBP(D): non-invasive blood pressure (diastolic); HR: heart rate; RR: respiratory rate; SPO2: blood oxygen saturation; AVPU: alert, verbal, pain, unresponsive.

Figure 2.

The area under the receiver operating characteristic of algorithms for detecting cardiac arrest. DeepEWS: Deep learning based Early Warning Score; MEWS: Modified Early Warning Score.

Figure 3.

The sensitivity according to the number of alarms. The Y-axis indicates sensitivity, and the X-axis means the number of alarms per 1,000 patients in 1 hour (A: 20, B: 40, C: 110, D: 200, E: 330).

References

1. Hillman K, Chen J, Cretikos M, Bellomo R, Brown D, Doig G, et al. Introduction of the medical emergency team (MET) system: a cluster-randomised controlled trial. Lancet 2005;365:2091-7.Article PubMed
2. Trinkle RM, Flabouris A. Documenting rapid response system afferent limb failure and associated patient outcomes. Resuscitation 2011;82:810-4.Article PubMed
3. Churpek MM, Yuen TC, Winslow C, Meltzer DO, Kattan MW, Edelson DP. Multicenter comparison of machine learning methods and conventional regression for predicting clinical deterioration on the wards. Crit Care Med 2016;44:368-74.Article PubMed PMC
4. Gao H, McDonnell A, Harrison DA, Moore T, Adam S, Daly K, et al. Systematic review and evaluation of physiological track and trigger warning systems for identifying at-risk patients on the ward. Intensive Care Med 2007;33:667-79.Article PubMed
5. Bell MB, Konrad D, Granath F, Ekbom A, Martling CR. Prevalence and sensitivity of MET-criteria in a Scandinavian University Hospital. Resuscitation 2006;70:66-73.Article PubMed
6. Kwon JM, Lee Y, Lee Y, Lee S, Park J. An algorithm based on deep learning for predicting in-hospital cardiac arrest. J Am Heart Assoc 2018;7:e008678. Article PubMed PMC
7. Deo RC. Machine learning in medicine. Circulation 2015;132:1920-30.Article PubMed PMC
8. Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 2016;316:2402-10.Article PubMed
9. Jha S, Topol EJ. Adapting to artificial intelligence: radiologists and pathologists as information specialists. JAMA 2016;316:2353-4.Article PubMed
10. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015;521:436-44.Article PubMed PDF

Figure & Data

References

Citations

Citations to this article as recorded by

External Validation of Deep Learning-Based Cardiac Arrest Risk Management System for Predicting In-Hospital Cardiac Arrest in Patients Admitted to General Wards Based on Rapid Response System Operating and Nonoperating Periods: A Single-Center Study
Kyung-Jae Cho, Kwan Hyung Kim, Jaewoo Choi, Dongjoon Yoo, Jeongmin Kim
Critical Care Medicine.2024; 52(3): e110. CrossRef
New Innovations to Address Sudden Cardiac Arrest
Christine P Shen, Sanjeev P Bhavnani, John D Rogers
US Cardiology Review.2024;[Epub] CrossRef
Development of deep learning algorithm for detecting dyskalemia based on electrocardiogram
Jung Nam An, Minje Park, Sunghoon Joo, Mineok Chang, Do Hyoung Kim, Dong Geum Shin, Yeongyeon Na, Jwa-Kyung Kim, Hyung-Seok Lee, Young Rim Song, Yeha Lee, Sung Gyun Kim
Scientific Reports.2024;[Epub] CrossRef
Evaluation of optimal scene time interval for out-of-hospital cardiac arrest using a deep neural network
Seung Jae Shin, Hee Sun Bae, Hyung Jun Moon, Gi Woon Kim, Young Soon Cho, Dong Wook Lee, Dong Kil Jeong, Hyun Joon Kim, Hyun Jung Lee
The American Journal of Emergency Medicine.2023; 63: 29. CrossRef
Short-term load forecasting based on CEEMDAN and Transformer
Peng Ran, Kun Dong, Xu Liu, Jing Wang
Electric Power Systems Research.2023; 214: 108885. CrossRef
Artificial Intelligence in Resuscitation: A Scoping Review
Dmitriy Viderman, Yerkin Abdildin, Kamila Batkuldinova, Rafael Badenes, Federico Bilotta
Journal of Clinical Medicine.2023; 12(6): 2254. CrossRef
Prediction of Out-of-Hospital Cardiac Arrest Survival Outcomes Using a Hybrid Agnostic Explanation TabNet Model
Hung Viet Nguyen, Haewon Byeon
Mathematics.2023; 11(9): 2030. CrossRef
Enhancing breast cancer diagnosis with deep learning and evolutionary algorithms: A comparison of approaches using different thermographic imaging treatments
Alberto Nogales, Fernando Pérez-Lara, Álvaro J. García-Tejedor
Multimedia Tools and Applications.2023; 83(14): 42955. CrossRef
Machine learning pre-hospital real-time cardiac arrest outcome prediction (PReCAP) using time-adaptive cohort model based on the Pan-Asian Resuscitation Outcome Study
Hansol Chang, Ji Woong Kim, Weon Jung, Sejin Heo, Se Uk Lee, Taerim Kim, Sung Yeon Hwang, Sang Do Shin, Won Chul Cha, Marcus Ong
Scientific Reports.2023;[Epub] CrossRef
Attempting cardiac arrest prediction using artificial intelligence on vital signs from Electronic Health Records
Bassel Soudan, Fetna F. Dandachi, Ali Bou Nassif
Smart Health.2022; 25: 100294. CrossRef
BERT Learns From Electroencephalograms About Parkinson’s Disease: Transformer-Based Models for Aid Diagnosis
Alberto Nogales, Alvaro J. Garcia-Tejedor, Ana M. Maitin, Antonio Perez-Morales, Maria Dolores Del Castillo, Juan Pablo Romero
IEEE Access.2022; 10: 101672. CrossRef
Artificial intelligence decision points in an emergency department
Hansol Chang, Won Chul Cha
Clinical and Experimental Emergency Medicine.2022; 9(3): 165. CrossRef
Development of Prediction Models for Acute Myocardial Infarction at Prehospital Stage with Machine Learning Based on a Nationwide Database
Arom Choi, Min Joung Kim, Ji Min Sung, Sunhee Kim, Jayoung Lee, Heejung Hyun, Hyeon Chang Kim, Ji Hoon Kim, Hyuk-Jae Chang
Journal of Cardiovascular Development and Disease.2022; 9(12): 430. CrossRef
Short-Term Load Forecasting Based on Ceemdan and Transformer
Peng Ran, Kun Dong, Xu Liu, Jing Wang
SSRN Electronic Journal .2022;[Epub] CrossRef
A survey of deep learning models in medical therapeutic areas
Alberto Nogales, Álvaro J. García-Tejedor, Diana Monge, Juan Serrano Vara, Cristina Antón
Artificial Intelligence in Medicine.2021; 112: 102020. CrossRef
A predictive framework in healthcare: Case study on cardiac arrest prediction
Samaneh Layeghian Javan, Mohammad Mehdi Sepehri
Artificial Intelligence in Medicine.2021; 117: 102099. CrossRef
Development of Prediction Models for Acute Myocardial Infarction at Prehospital Stage with Machine Learning Based on a Nationwide Database (Preprint)
Arom Choi, Min Joung Kim, Ji Min Sung, Sunhee Kim, Jayong Lee, Heejung Hyun, Ji Hoon Kim, Hyuk-Jae Chang
JMIR Medical Informatics.2021;[Epub] CrossRef
Predicting in-hospital mortality in adult non-traumatic emergency department patients: a retrospective comparison of the Modified Early Warning Score (MEWS) and machine learning approach
Kuan-Han Wu, Fu-Jen Cheng, Hsiang-Ling Tai, Jui-Cheng Wang, Yii-Ting Huang, Chih-Min Su, Yun-Nan Chang
PeerJ.2021; 9: e11988. CrossRef
Cardioinformatics: the nexus of bioinformatics and precision cardiology
Bohdan B Khomtchouk, Diem-Trang Tran, Kasra A Vand, Matthew Might, Or Gozani, Themistocles L Assimes
Briefings in Bioinformatics.2020; 21(6): 2031. CrossRef
Development of a Real-Time Risk Prediction Model for In-Hospital Cardiac Arrest in Critically Ill Patients Using Deep Learning: Retrospective Study
Junetae Kim, Yu Rang Park, Jeong Hoon Lee, Jae-Ho Lee, Young-Hak Kim, Jin Won Huh
JMIR Medical Informatics.2020; 8(3): e16349. CrossRef
Rapid response systems in Korea
Bo Young Lee, Sang-Bum Hong
Acute and Critical Care.2019; 34(2): 108. CrossRef

PubReader
ePub Link

Cite

CITE: export Copy Format; Close

Download Citation

Download a citation file in RIS format that can be imported by all major citation management software, including EndNote, ProCite, RefWorks, and Reference Manager.

Format:

RIS — For EndNote, ProCite, RefWorks, and most other reference management software
BibTeX — For JabRef, BibDesk, and other BibTeX-specific software

Include:

Citation for the content below
Citation and abstract for the content below

Deep Learning in the Medical Domain: Predicting Cardiac Arrest Using Deep Learning

Acute Crit Care. 2018;33(3):117-120. Published online August 31, 2018

DOI: https://doi.org/10.4266/acc.2018.00290

XML Download

Figure

Deep Learning in the Medical Domain: Predicting Cardiac Arrest Using Deep Learning

Figure 1. DeepEWS (Deep learning based Early Warning Score) operating on data from Electronic Medical Records. MEWS: Modified Early Warning Score; NIBP(S): non-invasive blood pressure (systolic); NIBP(D): non-invasive blood pressure (diastolic); HR: heart rate; RR: respiratory rate; SPO2: blood oxygen saturation; AVPU: alert, verbal, pain, unresponsive.

Figure 2. The area under the receiver operating characteristic of algorithms for detecting cardiac arrest. DeepEWS: Deep learning based Early Warning Score; MEWS: Modified Early Warning Score.

Figure 3. The sensitivity according to the number of alarms. The Y-axis indicates sensitivity, and the X-axis means the number of alarms per 1,000 patients in 1 hour (A: 20, B: 40, C: 110, D: 200, E: 330).

Figure 1.

Figure 2.

Figure 3.

Deep Learning in the Medical Domain: Predicting Cardiac Arrest Using Deep Learning