Using alert dwell time to filter universal clinical alerts: A machine learning approach.

Background: Alerts in computerized physician order entry (CPOE) systems can improve patient safety. However, alerts in rule-based systems cannot be customized based on individual patient or user characteristics. This limitation can lead to the presentation of irrelevant alerts and subsequent alert fatigue.

Objective: We used machine learning approaches with alert dwell time to filter out irrelevant alerts for physicians based on contextual factors.

Methods: We utilized five machine learning algorithms and a total of 1,120 features grouped into six categories: alert, demographic, environment, diagnosis, prescription, and laboratory results. The output of the models was the alert dwell time within a specified time window to determine the optimal range by the sensitivity analysis.

Results: We used 813,026 records (19 categories) from the hospital's outpatient clinic data from 2020 to 2021. The sensitivity analysis showed that a time window with a range of 0.3-4.0 s had the best performance, with an area under the receiver operating characteristic (AUROC) curve of 0.73 and an area under the precision-recall curve (AUPRC) of 0.97. The model built with alert and demographic feature groups showed the best performance, with an AUROC of 0.73. The most significant individual feature groups were alert and demographic, with AUROCs of 0.66 and 0.62, respectively.

Conclusion: Our study found that alerts and user and patient demographic features are more crucial than clinical features when constructing universal context-aware alerts. Using alert dwell time in combination with a time window is an effective way to determine the trigger status of an alert. The findings of this study can provide useful insights for researchers working on specific and universal context-aware alerts.