Abstract
-
Background
- Optimal sedation assessment in critically ill children remains challenging due to the subjective nature of behavioral scales and intermittent evaluation schedules. This study aimed to develop a deep learning model based on heart rate variability (HRV) parameters and vital signs to predict effective and safe sedation levels in pediatric patients.
-
Methods
- This retrospective cross-sectional study was conducted in a pediatric intensive care unit at a tertiary children’s hospital. We developed deep learning models incorporating HRV parameters extracted from electrocardiogram waveforms and vital signs to predict Richmond Agitation-Sedation Scale (RASS) scores. Model performance was evaluated using the area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC). The data were split into training, validation, and test sets (6:2:2), and the models were developed using a 1D ResNet architecture.
-
Results
- Analysis of 4,193 feature sets from 324 patients achieved excellent discrimination ability, with AUROC values of 0.867, 0.868, 0.858, 0.851, and 0.811 for whole number RASS thresholds of −5 to −1, respectively. AUPRC values ranged from 0.928 to 0.623, showing superior performance in deeper sedation levels. The HRV metric SDANN2 showed the highest feature importance, followed by systolic blood pressure and heart rate.
-
Conclusions
- A combination of HRV parameters and vital signs can effectively predict sedation levels in pediatric patients, offering the potential for automated and continuous sedation monitoring in pediatric intensive care settings. Future multi-center validation studies are needed to establish broader applicability.
-
Key Words: conscious sedation; deep learning; heart rate; machine learning; patient monitoring; pediatric intensive care unit; vital signs
INTRODUCTION
Appropriate sedation in critically ill children is essential not only to facilitate mechanical ventilation and reduce metabolic demands, but also to manage pain, anxiety, and neurological stress [1]. Achieving optimal sedation levels remains challenging, as both under- and over-sedation can lead to adverse outcomes. Insufficient sedation may result in patient-ventilator asynchrony and increased stress responses, while excessive sedation can lead to prolonged drug accumulation and subsequent withdrawal symptoms [2-6].
Assessment of sedation adequacy has traditionally relied on behavioral assessment scales, including the State Behavioral Scale (SBS), Richmond Agitation-Sedation Scale (RASS), and COMFORT Behavior Scale [7]. However, these observational methods are inherently subjective and present particular challenges in young children, while their intermittent nature precludes real-time monitoring of sedation status [8]. Physiological approaches offer a more objective alternative for sedation assessment. While electroencephalography is the current gold standard [9], clinical application of this method remains limited due to practical constraints. Heart rate variability (HRV), which measures variations in inter-beat intervals on electrocardiogram (ECG) waveforms [10], is a promising alternative due to its continuous availability in intensive care units (ICUs). HRV is reportedly associated with sedation state, suggesting it could be a valuable indicator in ICUs [11]. However, previous studies have focused on adult populations, and there has been little research involving pediatric patients, who exhibit age-dependent variations in heart rate (HR) and other vital signs [12-14]. Direct extrapolation of adult-based research findings to pediatric populations presents significant limitations.
This study was designed under the hypothesis that, despite age-dependent variations in normal vital sign ranges among pediatric patients, sedation assessment can be feasible through HRV analysis when adjusted for age and vital-sign values. The primary objective was to develop an automated model for sedation assessment using HRV and other vital signs. For sedation evaluation, we employed the RASS, an established and widely used assessment tool in pediatric populations.
MATERIALS AND METHODS
Research Ethics
This study was conducted in accordance with the Helsinki Declaration. The study protocol was reviewed and approved by the Institutional Review Board of Seoul National University Hospital (No. E-2411-012-1582). Because it was recognized as a retrospective minimal-risk study, the requirement for written informed consent was waived.
Study Setting and Data Source
This retrospective cross-sectional observational study was conducted in a 24-bed pediatric intensive care unit (PICU) at a tertiary children’s hospital in Seoul, South Korea. The study included patients admitted to the PICU who underwent lead-II ECG monitoring from January 2021 to December 2023. Patients who did not undergo vital sign measurement and lead-II ECG monitoring during their stay in the PICU were excluded from the study. Data used in the study, including age (months), sex, vital signs (systolic blood pressure [SBP], diastolic blood pressure [DBP], HR, respiratory rate [RR], and body temperature [BT]), and RASS scores were obtained from the clinical data warehouse of the hospital information system, while lead-II ECG waveforms were extracted from VitalDB [15], which collects data in the PICU.
Data Preprocessing
Extreme values deemed non-physiological, such as SBP <30 mm Hg, SBP >300 mm Hg, SBP<DBP, HR<10 beats/min, HR >200 beats/min, RR <5 breaths/min, RR >100 breaths/min, BT <32 ºC, and BT >42 ºC, were excluded from the analysis. Data on HRV were extracted from lead-II ECG waveforms using the neurokit2 library (version 0.2.10) in Python (version 3.12.7). The extracted parameters included 23 items, such as mean of the normal-to-normal (NN) intervals, standard deviation of NN (SDNN), and root mean square of successive differences (RMSSD). Detailed descriptions of each item are provided in Supplementary Table 1.
The HRV parameters and vital signs were sorted by subject and extraction (or measurement) time. Where values were missing, imputation was performed using the Python fillna function, with missing values replaced with the most recent available value for the corresponding subject. Any remaining rows containing residual missing values after this imputation process were excluded from further analysis.
From the aforementioned data, only measurements performed within 10 minutes prior to RASS assessment were retained through filtering. This process resulted in the organization of 30 features (age, sex, SBP, DBP, HR, RR, BT, and 23 HRV parameters) corresponding to each RASS score as a single “analytical row.” The 30 features in each row were defined as a “feature set.”
Outcomes
The objective of this study was to develop binary classification models that determine whether RASS scores exceed specific thresholds ranging from −5 to −1 or are less than or equal to these thresholds. The primary outcome was the estimated performance of these models, evaluated using area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC). Secondary outcomes included the importance of the variables in each model, their impact on model performance, and interactions among the variables.
Model Development
Data splitting and normalization
Following dataset loading, each analytical row was processed to include independent variables (a feature set consisting of 30 features) and the dependent variable (RASS scores). The data were then systematically partitioned into training, validation, and test sets at a ratio of 6:2:2, while maintaining the distribution of RASS scores. The training set data were normalized using the MinMaxScaler function from the Scikit-Learn library (version 1.5.2), scaling all features to a range between 0 and 1. The scaling parameters derived from the training set were applied to both the validation and test sets to maintain consistency. The training set was used for model development, while the validation set was used to tune hyperparameters and assess various performance metrics. The final model evaluation was conducted using the independent test set.
Hyperparameter configuration
The model’s hyperparameters were configured systematically. The maximum number of training epochs was set to 5,000, with early termination allowed if no improvement was observed within 1,000 epochs to prevent overfitting. The learning-rate boundaries were between 10−7 and 10−2, with the optimal learning rate set by the learning rate finder function (torch_lr_finder library version 0.2.2). Additional hyperparameters were configured as follows: a dropout rate at 0.4, label smoothing ratio at 0.1, Mixup alpha parameter at 0.2, and stochastic depth probability at 0.1. These parameters were carefully selected to optimize model performance and enhance generalization capabilities.
Model architecture and training process
The model architecture was a 1D ResNet structure based on Residual Blocks. The network initiates with a single input channel, employing 64 filters (kernel size 3) in the initial convolutional layer. Subsequently, it comprises four sequential residual blocks with progressively increasing channel dimensions from 64 to 512 (64 → 128 → 256 → 512). Each ResidualBlock incorporates two convolutional layers with batch normalization, a squeeze-and-excitation block for channel attention, and dropout layers with a rate of 0.4. Additionally, skip connections are implemented for residual learning. The architecture culminates in a global average pooling layer followed by a fully connected layer with a single-dimensional output for binary classification. The training methodology used the custom train_model function and binary cross-entropy loss with label smoothing. The optimization strategy used the AdamW optimizer coupled with a cosine annealing scheduler to adjust the learning rate. To enhance model stability and performance, an exponential moving average model with a decay rate of 0.999 was maintained.
Multiple advanced techniques were incorporated during the training process. Mixup augmentation with α=0.2 was used for improved generalization, while gradient clipping with a maximum norm of 1.0 was applied to prevent gradient explosion. Label smoothing with a factor of 0.1 was used to mitigate overfitting, and a stochastic depth with a drop rate of 0.1 was implemented for regularization. The model’s forward propagation included residual connections and squeeze-and-excitation attention mechanisms, enabling effective feature extraction and representation learning from temporal data.
Performance assessment
The model’s performance was assessed by comprehensively comparing predicted and actual values, using various performance metrics. In this process, both the AUROC and AUPRC were calculated and analyzed as primary performance indicators.
Model interpretation
For model interpretability, the custom analyze_shap function was applied to conduct a SHapley Additive exPlanations (SHAP) analysis to quantify the importance and contribution of each feature to the model predictions. Through this analysis, the features relied predominantly on the model for predictions, and the relative importance of different variables in the decision-making process was determined.
Data Analyses
Data analysis was conducted in Python (version 3.12.7), while development of the deep learning model used PyTorch (version 2.4.1). For data presentation, categorical variables were expressed as number (%), and continuous variables were presented as median (interquartile range). Data visualization was performed using the Matplotlib library (version 3.9.2). Computational acceleration was achieved through graphics processing unit based compute unified device architecture implementation.
RESULTS
Baseline Characteristics
After applying the inclusion and exclusion criteria, 4,193 RASS-matched feature sets from 324 subjects were analyzed. The age of these subjects was 49 months (8–151), and 141 (43.5%) were female. The distribution of vital signs in the entire feature set is shown in Table 1.
Main Outcomes
Learning rates
The optimized learning rates were 0.00278 for the threshold −5, 0.00313 for −4, 0.00156 for −3, 0.00498 for −2, and 0.00175 for −1 (Supplementary Figs. 1-5).
Performance
In this study, the estimated models by threshold showed an AUROC (95% CI) of 0.867 (0.839–0.894) for −5 (i.e., RASS=−5 or RASS >−5), 0.868 (0.840–0.892) for −4, 0.858 (0.834–0.882) for −3, 0.851 (0.822–0.876) for −2, and 0.811 (0.779–0.845) for −1 (Figure 1). The AUPRC values were 0.928 (0.904–0.948), 0.685 (0.829–0.900), 0.805 (0.761–0.849), 0.749 (0.697–0.805), and 0.623 (0.556–0.695), respectively (Figure 2). The performance was highest at a RASS threshold of −5 and showed a decreasing trend toward −1.
Feature importance and impact on model output
For threshold −5, the top 20 of 30 features are shown in Figure 3, and the impact of each feature on the model is displayed in Figure 4. Among these features, the HRV metric standard deviation of average NN intervals calculated over 5-minute segments (SDANN2) showed the highest importance, followed by SBP and HR (Figure 3). Higher values of HRV_SDANN2 (in red) were associated with increased SHAP values, suggesting that patients were more likely to have RASS scores of −4 or higher rather than −5 (Figure 4). The feature importance rankings and their impacts on model output for other thresholds (−4, −3, −2, −1) are presented in Supplementary Figs. 6-13.
DISCUSSION
The high AUROC and AUPRC values demonstrated that a combination of HRV parameters and vital signs effectively distinguishes among sedation levels, indicating its potential as a reliable assessment tool for sedation monitoring in pediatric patients. While behavioral assessment tools such as RASS, SBS, and COMFORT-B have established validity in pediatric sedation evaluation [16,17], they present notable limitations. SBS is validated only for mechanically ventilated patients, COMFORT-B requires time-consuming observations [16,18], and RASS, despite its simplicity, has limited pediatric validation [19]. Furthermore, these tools are inherently subjective and depend on evaluator experience [8], and their intermittent nature may not adequately represent continuous sedation states, particularly in infants.
Our findings extend beyond previous HRV-based studies of sedation prediction by incorporating additional vital signs into the prediction model [8,20,21]. While earlier research since 2013 focused primarily on HRV’s utility in sedation assessment (achieving a sensitivity of 64%, a specificity of 84.8%, and an AUROC of 0.72 in initial studies [20], and improving to 69% accuracy in four-level classification [8]), our comprehensive approach using both HRV parameters and vital signs achieved superior performance. The current model showed excellent discrimination ability, with the AUROC exceeding 0.8 and the AUPRC greater than 0.6 across all sedation levels, with notably higher AUPRC values for deeper sedation levels. This pattern contrasts with previous studies’ superior performance in light-sedation prediction, possibly due to our comprehensive approach to physiological parameters and larger pediatric patient sample size.
Despite the promising results, several limitations of this study warrant consideration. First, as a single-center study, the immediate generalizability to other healthcare institutions remains uncertain. Although our model achieved superior performance compared with previous single-center studies, further validation through multi-center studies and external validation is essential to establish broader applicability. Second, the study did not account for various conditions affecting HRV, particularly cardiovascular and cerebrovascular conditions [22-24]. These conditions can significantly influence the functioning of the autonomic nervous system and, consequently, HRV parameters. Additionally, medications commonly used in intensive care settings, such as beta-blockers, angiotensin-converting enzyme inhibitors, and antidepressants [25], which can modulate HRV, were not considered in the current model. Third, the study population consisted predominantly of pediatric patients, which may limit the model’s interpretability and applicability in adult populations. While our findings demonstrate the feasibility of HRV-based sedation assessment in children, caution should be exercised when extrapolating these results to adult patients due to age-related physiological differences. Nevertheless, this study represents a significant advance in automated sedation assessment, particularly in pediatric populations, with its comprehensive use of both HRV parameters and vital signs. Future research should address these limitations through multi-center studies, incorporating various clinical conditions and medications, and expanding the age range of the study population to enhance the model’s generalizability and clinical utility.
In conclusion, a deep learning model combining HRV parameters and vital signs can effectively predict sedation levels in pediatric patients, showing superior performance compared with previous studies. While its single-center design and unconsidered clinical conditions need to be addressed through future multi-center validation, this approach represents a promising step toward automated and objective sedation assessment in pediatric intensive care settings.
KEY MESSAGES
▪ A deep learning model combining heart rate variability parameters and vital signs effectively estimates sedation levels in pediatric patients, demonstrating superior performance (area under the receiver operating characteristic curve >0.8) compared with previous adult studies.
▪ The model achieves particularly strong performance in assessing deeper sedation levels, with area under the precision-recall curve values progressively increasing as sedation deepens, offering the potential for reliable and continuous monitoring of critically ill children.
▪ While behavioral assessment tools remain the standard, and further validation through multi-center studies is needed for broader clinical implementation, this automated approach provides an objective, continuous alternative for sedation monitoring.
NOTES
-
CONFLICT OF INTEREST
No potential conflict of interest relevant to this article was reported.
-
FUNDING
None.
-
ACKNOWLEDGMENTS
None.
-
AUTHOR CONTRIBUTIONS
Conceptualization: BL. Data curation: YSK, BL. Formal analysis: BL. Methodology: YSK, BL. Project administration: BL. Visualization: BL. Writing – original draft: YSK, BL. Writing – review and editing: WJ, YJ, JDP, BL. All authors read and agreed to the published version of the manuscript.
SUPPLEMENTARY MATERIALS
Supplementary materials can be found via https://doi.org/10.4266/acc.2024.01200.
Supplementary Figure 1.
Learning rate finder plot for binary classification model (RASS=–5 vs. RASS >–5). The plot shows the relationship between learning rate and loss function. The optimal learning rate (red dot, 0.00278) was selected at the point where the loss starts to decrease rapidly but before the point of divergence. The x-axis shows learning rate on a logarithmic scale, and the y-axis shows the corresponding loss value. RASS: Richmond Agitation-Sedation Scale; LR: learning rate.
acc-2024-01200-Supplementary-Fig-1.pdf
Supplementary Figure 2.
Learning rate finder plot for binary classification model (RASS=–4 vs. RASS >–4). The plot shows the relationship between learning rate and loss function. The optimal learning rate (red dot, 0.00313) was selected at the point where the loss starts to decrease rapidly but before the point of divergence. The x-axis shows learning rate on a logarithmic scale, and the y-axis shows the corresponding loss value. RASS: Richmond Agitation-Sedation Scale; LR: learning rate.
acc-2024-01200-Supplementary-Fig-2.pdf
Supplementary Figure 3.
Learning rate finder plot for binary classification model (RASS=–3 vs. RASS >–3). The plot shows the relationship between learning rate and loss function. The optimal learning rate (red dot, 0.00156) was selected at the point where the loss starts to decrease rapidly but before the point of divergence. The x-axis shows learning rate on a logarithmic scale, and the y-axis shows the corresponding loss value. RASS: Richmond Agitation-Sedation Scale; LR: learning rate.
acc-2024-01200-Supplementary-Fig-3.pdf
Supplementary Figure 4.
Learning rate finder plot for binary classification model (RASS=–2 vs. RASS >–2). The plot shows the relationship between learning rate and loss function. The optimal learning rate (red dot, 0.00498) was selected at the point where the loss starts to decrease rapidly but before the point of divergence. The x-axis shows learning rate on a logarithmic scale, and the y-axis shows the corresponding loss value. RASS: Richmond Agitation-Sedation Scale; LR: learning rate.
acc-2024-01200-Supplementary-Fig-4.pdf
Supplementary Figure 5.
Learning rate finder plot for binary classification model (RASS=–1 vs. RASS >–1). The plot shows the relationship between learning rate and loss function. The optimal learning rate (red dot, 0.00175) was selected at the point where the loss starts to decrease rapidly but before the point of divergence. The x-axis shows learning rate on a logarithmic scale, and the y-axis shows the corresponding loss value. RASS: Richmond Agitation-Sedation Scale; LR: learning rate.
acc-2024-01200-Supplementary-Fig-5.pdf
Supplementary Figure 6.
SHAP feature importance for binary classification model (RASS=–4 vs. RASS >–4). The bar plot shows the mean absolute SHAP values for the top 20 features, indicating their relative importance in the model’s predictions. HRV_HTI showed the highest importance, followed by HRV_pNN20 and vital signs (RR, SBP, DBP). The x-axis represents the mean absolute SHAP value, indicating the average impact of each feature on model output magnitude. SHAP: SHapley Additive exPlanations; RASS: Richmond Agitation-Sedation Scale; HRV: heart rate variability; HTI: heart rate turbulence index; pNN20: proportion of successive NN differences >20 ms; RR: respiratory rate; SBP: systolic blood pressure; DBP: diastolic blood pressure; SDANN: standard deviation of average NN intervals; SDNNI: mean of standard deviations of NN intervals; BT: body temperature; MadNN: median absolute deviation of NN intervals; MeanNN: mean of NN intervals; TINN: triangular interpolation of NN interval histogram; MedianNN: median of NN intervals; SDSD: standard deviation of successive differences; RMSSD: root mean square of successive differences; CVSD: coefficient of variation of successive differences; CVNN: coefficient of variation of NN intervals; MinNN: minimum of NN intervals; HR: heart rate.
acc-2024-01200-Supplementary-Fig-6.pdf
Supplementary Figure 7.
SHAP value distribution for features in binary classification model (RASS=–4 vs. RASS >–4). The plot shows the impact of each feature on model output, with colors indicating the feature value (blue=low, red=high). Each point represents a single prediction, and the horizontal position shows the SHAP value (negative values push predictions toward RASS=–4, positive values toward RASS >–4). Features are ordered by their mean absolute SHAP value, with HRV_HTI showing the strongest impact on predictions.
SHAP: SHapley Additive exPlanations; RASS: Richmond Agitation-Sedation Scale; HRV: heart rate variability; HTI: heart rate turbulence index; pNN20: proportion of successive NN differences >20 ms; RR: respiratory rate; SBP: systolic blood pressure; DBP: diastolic blood pressure; SDANN: standard deviation of average NN intervals; SDNNI: mean of standard deviations of NN intervals; BT: body temperature; MadNN: median absolute deviation of NN intervals; MeanNN: mean of NN intervals; TINN: triangular interpolation of NN interval histogram; MedianNN: median of NN intervals; SDSD: standard deviation of successive differences; RMSSD: root mean square of successive differences; CVSD: coefficient of variation of successive differences; CVNN: coefficient of variation of NN intervals; MinNN: minimum of NN intervals; HR: heart rate.
acc-2024-01200-Supplementary-Fig-7.pdf
Supplementary Figure 8.
SHAP feature importance for binary classification model (RASS=–3 vs. RASS >–3). The bar plot shows the mean absolute SHAP values for the top 20 features, indicating their relative importance in the model’s predictions. Age showed the highest importance, followed by SBP and HRV_HTI. The x-axis represents the mean absolute SHAP value, indicating the average impact of each feature on model output magnitude. SHAP: SHapley Additive exPlanations; RASS: Richmond Agitation-Sedation Scale; SBP: systolic blood pressure; HRV: heart rate variability; HTI: heart rate turbulence index; SDNNI: mean of standard deviations of NN intervals; SDANN: standard deviation of average NN intervals; pNN20: proportion of successive NN differences >20 ms; BT: body temperature; RR: respiratory rate; IQRNN: interquartile range of NN intervals; MeanNN: mean of NN intervals; TINN: triangular interpolation of NN interval histogram; MaxNN: maximum of NN intervals; MedianNN: median of NN intervals; RMSSD: root mean square of successive differences; SDNN: standard deviation of NN intervals; CVNN: coefficient of variation of NN intervals; HR: heart rate; CVSD: coefficient of variation of successive differences.
acc-2024-01200-Supplementary-Fig-8.pdf
Supplementary Figure 9.
SHAP value distribution for features in binary classification model (RASS=–3 vs. RASS >–3). The plot shows the impact of each feature on model output, with colors indicating the feature value (blue=low, red =high). Each point represents a single prediction, and the horizontal position shows the SHAP value (negative values push predictions toward RASS=–3, positive values toward RASS >–3). Features are ordered by their mean absolute SHAP value, with Age showing the strongest impact on predictions. SHAP: SHapley Additive exPlanations; RASS: Richmond Agitation-Sedation Scale; SBP: systolic blood pressure; HRV: heart rate variability; HTI: heart rate turbulence index; SDNNI: mean of standard deviations of NN intervals; SDANN: standard deviation of average NN intervals; pNN20: proportion of successive NN differences >20ms; BT: body temperature; RR: respiratory rate; IQRNN: interquartile range of NN intervals; MeanNN: mean of NN intervals; TINN: triangular interpolation of NN interval histogram; MaxNN: maximum of NN intervals; MedianNN: median of NN intervals; RMSSD: root mean square of successive differences; SDNN: standard deviation of NN intervals; CVNN: coefficient of variation of NN intervals; HR: heart rate; CVSD: coefficient of variation of successive differences.
acc-2024-01200-Supplementary-Fig-9.pdf
Supplementary Figure 10.
SHAP feature importance for binary classification model (RASS=–2 vs. RASS >–2). The bar plot shows the mean absolute SHAP values for the top 20 features, indicating their relative importance in the model’s predictions. HRV_HTI and HRV_pNN50 showed the highest importance, followed by HRV_SDNNI2 and vital signs (SBP, DBP). The x-axis represents the mean absolute SHAP value, indicating the average impact of each feature on model output magnitude. SHAP: SHapley Additive exPlanations; RASS: Richmond Agitation-Sedation Scale; HRV: heart rate variability; HTI: heart rate turbulence index; pNN50: proportion of successive NN differences >50 ms; SDNNI: mean of standard deviations of NN intervals; SBP: systolic blood pressure; DBP: diastolic blood pressure; SDANN: standard deviation of average NN intervals; MadNN: median absolute deviation of NN intervals; BT: body temperature; RR: respiratory rate; IQRNN: interquartile range of NN intervals; MeanNN: mean of NN intervals; Prc80NN: 80th percentile of NN intervals; TINN: triangular interpolation of NN interval histogram; MedianNN: median of NN intervals; RMSSD: root mean square of successive differences; CVSD: coefficient of variation of successive differences; SDNN: standard deviation of NN intervals; CVNN: coefficient of variation of NN intervals.
acc-2024-01200-Supplementary-Fig-10.pdf
Supplementary Figure 11.
SHAP value distribution for features in binary classification model (RASS=–2 vs. RASS >–2). The plot shows the impact of each feature on model output, with colors indicating the feature value (blue=low, red=high). Each point represents a single prediction, and the horizontal position shows the SHAP value (negative values push predictions toward RASS=–2, positive values toward RASS >–2). Features are ordered by their mean absolute SHAP value, with HRV_HTI and HRV_pNN50 showing the strongest impact on predictions. SHAP: SHapley Additive exPlanations; RASS: Richmond Agitation-Sedation Scale; HRV: heart rate variability; HTI: heart rate turbulence index; pNN50: proportion of successive NN differences >50 ms; SDNNI: mean of standard deviations of NN intervals; SBP: systolic blood pressure; DBP: diastolic blood pressure; SDANN: standard deviation of average NN intervals; MadNN: median absolute deviation of NN intervals; BT: body temperature; RR: respiratory rate; IQRNN: interquartile range of NN intervals; MeanNN: mean of NN intervals; Prc80NN: 80th percentile of NN intervals; TINN: triangular interpolation of NN interval histogram; MedianNN: median of NN intervals; RMSSD: root mean square of successive differences; CVSD: coefficient of variation of successive differences; SDNN: standard deviation of NN intervals; CVNN: coefficient of variation of NN intervals.
acc-2024-01200-Supplementary-Fig-11.pdf
Supplementary Figure 12.
SHAP feature importance for binary classification model (RASS=–1 vs. RASS >–1). The bar plot shows the mean absolute SHAP values for the top 20 features, indicating their relative importance in the model’s predictions. HRV_HTI showed the highest importance, followed by HRV_pNN20 and vital signs (BT, SBP). The x-axis represents the mean absolute SHAP value, indicating the average impact of each feature on model output magnitude. SHAP: SHapley Additive exPlanations; RASS: Richmond Agitation-Sedation Scale; HRV: heart rate variability; HTI: heart rate turbulence index; pNN20: proportion of successive NN differences >20 ms; BT: body temperature; SBP: systolic blood pressure; SDNNI: mean of standard deviations of NN intervals; RR: respiratory rate; SDANN: standard deviation of average NN intervals; MadNN: median absolute deviation of NN intervals; DBP: diastolic blood pressure; SDSD: standard deviation of successive differences; Prc80NN: 80th percentile of NN intervals; TINN: triangular interpolation of NN interval histogram; SDNN: standard deviation of NN intervals; MedianNN: median of NN intervals; HR: heart rate; RMSSD: root mean square of successive differences; CVNN: coefficient of variation of NN intervals; IQRNN: interquartile range of NN intervals; CVSD: coefficient of variation of successive differences.
acc-2024-01200-Supplementary-Fig-12.pdf
Supplementary Figure 13.
SHAP value distribution for features in binary classification model (RASS=–1 vs. RASS >–1). The plot shows the impact of each feature on model output, with colors indicating the feature value (blue=low, red =high). Each point represents a single prediction, and the horizontal position shows the SHAP value (negative values push predictions toward RASS=–1, positive values toward RASS >–1). Features are ordered by their mean absolute SHAP value, with HRV_HTI showing the strongest impact on predictions.
SHAP: SHapley Additive exPlanations; RASS: Richmond Agitation-Sedation Scale; HRV: heart rate variability; HTI: heart rate turbulence index; pNN20: proportion of successive NN differences >20 ms; BT: body temperature; SBP: systolic blood pressure; SDNNI: mean of standard deviations of NN intervals; RR: respiratory rate; SDANN: standard deviation of average NN intervals; MadNN: median absolute deviation of NN intervals; DBP: diastolic blood pressure; SDSD: standard deviation of successive differences; Prc80NN: 80th percentile of NN intervals; TINN: triangular interpolation of NN interval histogram; SDNN: standard deviation of NN intervals; MedianNN: median of NN intervals; HR: heart rate; RMSSD: root mean square of successive differences; CVNN: coefficient of variation of NN intervals; IQRNN: interquartile range of NN intervals; CVSD: coefficient of variation of successive differences.
acc-2024-01200-Supplementary-Fig-13.pdf
Figure 1.Receiver operating characteristic curves for sedation-level prediction at different Richmond Agitation-Sedation Scale (RASS) thresholds. Each curve represents the performance of a binary classifier distinguishing between scores at or below a threshold versus those above that threshold (e.g., “RASS >−5” distinguishes between RASS=−5 vs. RASS >−5). The model shows consistent discrimination ability across thresholds, with area under the curve (AUC) values and 95% CIs shown in parentheses. The dashed gray line represents random prediction (AUC=0.5).
Figure 2.Precision-recall curves for sedation-level prediction at different Richmond Agitation-Sedation Scale (RASS) thresholds. Each curve represents the performance of a binary classifier distinguishing between scores at or below a threshold versus those above that threshold (e.g., “RASS >−5” distinguishes between RASS=−5 versus RASS >−5). The model demonstrates varying performance levels across different thresholds, with area under the precision-recall curve (AUPRC) values and 95% CIs shown in parentheses. Higher AUPRC values were observed for deeper sedation levels.
Figure 3.SHapley Additive exPlanations (SHAP) feature importance for binary classification model distinguishing between Richmond Agitation-Sedation Scale (RASS)=−5 vs. RASS >−5. The bar plot shows the mean absolute SHAP values for the top 20 features, indicating their relative importance in the model’s predictions. Heart rate variability (heart rate variability) parameters and vital signs are ranked by their average impact on model output magnitude. HRV_SDANN2 showed the highest importance, followed by vital signs (SBP, HR, DBP, RR) and other HRV parameters. SDANN2: standard deviation of average NN intervals over 5-minute segments; NN: normal-to-normal; SBP: systolic blood pressure; HR: heart rate; pNN20: proportion of NN intervals differing by more than 20 ms; IQRNN: interquartile range of NN intervals; DBP: diastolic blood pressure; RR: respiratory rate; HTI: heart rate turbulence index; SDNN: standard deviation of NN intervals; SDNNI1: successive difference of NN intervals first-order; SDNNI2: SDNNI second-order, SDSD: standard deviation of successive differences; TINN: triangular interpolation of NN interval histogram; CVSD: coefficient of variation of successive differences; RMSSD: root mean square of successive differences; CVNN: coefficient of variation of NN intervals; MedianNN: median of NN intervals.
Figure 4.SHapley Additive exPlanations (SHAP) value distribution for features in binary classification model distinguishing between Richmond Agitation-Sedation Scale (RASS)=−5 vs. RASS >−5. The plot shows the impact of each feature on model output, with colors indicating the feature value (blue=low, red=high). Each point represents a single prediction, and the horizontal position shows the SHAP value (negative values push predictions toward RASS=−5, positive values toward RASS >−5). Features are ordered by their mean absolute SHAP value, with HRV_SDANN2 having the strongest impact on predictions. HRV: heart rate variability; SDANN2: standard deviation of average NN intervals over 5-minute segments; NN: normal-to-normal; SBP: systolic blood pressure; HR: heart rate; pNN20: proportion of NN intervals differing by more than 20 ms; IQRNN: interquartile range of NN intervals; DBP: diastolic blood pressure; RR: respiratory rate; HTI: heart rate turbulence index; SDNN: standard deviation of NN intervals; SDNNI1: successive difference of NN intervals first-order; SDNNI2: SDNNI second-order, SDSD: standard deviation of successive differences; TINN: triangular interpolation of NN interval histogram; CVSD: coefficient of variation of successive differences; RMSSD: root mean square of successive differences; CVNN: coefficient of variation of NN intervals; MedianNN: median of NN intervals.
Table 1.Distribution of vital signs across the complete dataset (n=4,193 feature sets)
|
Variable |
Mean±SD |
Median (IQR) |
|
SBP (mm Hg) |
95.7±21.3 |
96 (82–110) |
|
DBP (mm Hg) |
59.1±18 |
58 (47–70) |
|
HR (beats/min) |
111.8±26.6 |
112 (94–131) |
|
RR (breaths/min) |
28.6±12.1 |
25 (20–35) |
|
BT (℃) |
36.7±0.6 |
36.7 (36.3–37.1) |
REFERENCES
- 1. Pomeranz B, Macaulay RJ, Caudill MA, Kutz I, Adam D, Gordon D, et al. Assessment of autonomic function in humans by heart rate spectral analysis. Am J Physiol 1985;248(1 Pt 2):H151-3.ArticlePubMed
- 2. Anand KJ, Willson DF, Berger J, Harrison R, Meert KL, Zimmerman J, et al. Tolerance and withdrawal from prolonged opioid use in critically ill children. Pediatrics 2010;125:e1208-25.ArticlePubMedPDF
- 3. Best KM, Boullata JI, Curley MA. Risk factors associated with iatrogenic opioid and benzodiazepine withdrawal in critically ill pediatric patients: a systematic review and conceptual model. Pediatr Crit Care Med 2015;16:175-83.ArticlePubMedPMC
- 4. de Wit M, Pedram S, Best AM, Epstein SK. Observational study of patient-ventilator asynchrony and relationship to sedation level. J Crit Care 2009;24:74-80.ArticlePubMedPMC
- 5. Devlin JW, Skrobik Y, Gélinas C, Needham DM, Slooter AJ, Pandharipande PP, et al. Clinical practice guidelines for the prevention and management of pain, agitation/sedation, delirium, immobility, and sleep disruption in adult patients in the ICU. Crit Care Med 2018;46:e825-73.ArticlePubMed
- 6. Thille AW, Rodriguez P, Cabello B, Lellouche F, Brochard L. Patient-ventilator asynchrony during assisted mechanical ventilation. Intensive Care Med 2006;32:1515-22.ArticlePubMedPDF
- 7. Balit CR, LaRosa JM, Ong JS, Kudchadkar SR. Sedation protocols in the pediatric intensive care unit: fact or fiction? Transl Pediatr 2021;10:2814-24.ArticlePubMedPMC
- 8. Nagaraj SB, McClain LM, Zhou DW, Biswal S, Rosenthal ES, Purdon PL, et al. Automatic classification of sedation levels in ICU patients using heart rate variability. Crit Care Med 2016;44:e782-9.ArticlePubMedPMC
- 9. Spencer EM, Green JL, Willatts SM. Continuous monitoring of depth of sedation by EEG spectral analysis in patients requiring mechanical ventilation. Br J Anaesth 1994;73:649-54.ArticlePubMed
- 10. Shaffer F, Ginsberg JP. An overview of heart rate variability metrics and norms. Front Public Health 2017;5:258. ArticlePubMedPMC
- 11. Bradley BD, Green G, Ramsay T, Seely AJ. Impact of sedation and organ failure on continuous heart and respiratory rate variability monitoring in critically ill patients: a pilot study. Crit Care Med 2013;41:433-44.ArticlePubMed
- 12. Bae W, Kim K, Lee B. Distribution of pediatric vital signs in the emergency department: a nationwide study. Children (Basel) 2020;7:89. ArticlePubMedPMC
- 13. Goo S, Jang W, Kim YS, Ji S, Park T, Park JD, et al. Streamlining pediatric vital sign assessment: innovations and insights. Sci Rep 2024;14:22542. ArticlePubMedPMCPDF
- 14. Kim L, Yun KS, Park JD, Lee B. Effect of diurnal variation of heart rate and respiratory rate on activation of rapid response system and clinical outcome in hospitalized children. Children (Basel) 2023;10:167. ArticlePubMedPMC
- 15. Lee HC, Jung CW. Vital Recorder: a free research tool for automatic recording of high-resolution time-synchronised physiological data from multiple anaesthesia devices. Sci Rep 2018;8:1527. ArticlePubMedPMC
- 16. Tapia R, López-Herce J, Arias Á, Del Castillo J, Mencía S. Validity and reliability of the Richmond agitation-sedation scale in pediatric intensive care patients: a multicenter study. Front Pediatr 2022;9:795487. ArticlePubMedPMC
- 17. Smith HA, Besunder JB, Betters KA, Johnson PN, Srinivasan V, Stormorken A, et al. 2022 Society of Critical Care Medicine clinical practice guidelines on prevention and management of pain, agitation, neuromuscular blockade, and delirium in critically Ill pediatric patients with consideration of the ICU environment and early mobility. Pediatr Crit Care Med 2022;23:e74-110.ArticlePubMed
- 18. Boerlage AA, Ista E, de Jong M, Tibboel D, van Dijk M. The COMFORT behavior scale: is a shorter observation period feasible? Pediatr Crit Care Med 2012;13:e124-5.ArticlePubMed
- 19. Kerson AG, DeMaria R, Mauer E, Joyce C, Gerber LM, Greenwald BM, et al. Validity of the Richmond Agitation-Sedation Scale (RASS) in critically ill children. J Intensive Care 2016;4:65. ArticlePubMedPMC
- 20. Nagaraj SB, Ramaswamy SM, Biswal S, Boyle EJ, Zhou DW, Mcclain LM, et al. Heart rate variability as a biomarker for sedation depth estimation in ICU patients. Annu Int Conf IEEE Eng Med Biol Soc 2016;2016:6397-400.ArticlePubMed
- 21. Nagaraj SB, Biswal S, Boyle EJ, Zhou DW, McClain LM, Bajwa EK, et al. Patient-specific classification of ICU sedation levels from heart rate variability. Crit Care Med 2017;45:e683-90.ArticlePubMedPMC
- 22. Toyofuku A, Ehrler M, Naef N, Schmid AS, Kretschmar O, Latal B, et al. Heart rate variability and cognitive functions in adolescents with complex congenital heart disease. Pediatr Res 2024;Jul 30; [Epub]. https://doi.org/10.1038/s41390-024-03432-9.Article
- 23. Goldenberg I, Goldkorn R, Shlomo N, Einhorn M, Levitan J, Kuperstein R, et al. Heart rate variability for risk assessment of myocardial ischemia in patients without known coronary artery disease: the HRV-DETECT (Heart Rate Variability for the Detection of Myocardial Ischemia) study. J Am Heart Assoc 2019;8:e014540.ArticlePubMedPMC
- 24. Kong SD, Gordon CJ, Hoyos CM, Wassing R, D'Rozario A, Mowszowski L, et al. Heart rate variability during slow wave sleep is linked to functional connectivity in the central autonomic network. Brain Commun 2023;5:fcad129. ArticlePubMedPMCPDF
- 25. Ernst G. Hidden signals-the history and methods of heart rate variability. Front Public Health 2017;5:265. ArticlePubMedPMC
Citations
Citations to this article as recorded by
KSCCM
, Bongjin Lee1,2
PubReader
ePub Link
Cite
XML Download
